@kensio/yulin-aws-simulation 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,14 @@
1
+ {
2
+ "$schema": "https://anthropic.com/claude-code/plugin.schema.json",
3
+ "name": "yulin-aws-simulation",
4
+ "version": "1.3.0",
5
+ "description": "How to test AWS code well with the @kensio/yulin in-process simulator: using it directly instead of building a harness around it, driving tests, local dev and production from one synthesized CDK template, intercepting SDK clients, controlling simulated time, and handling what the simulator refuses.",
6
+ "author": {
7
+ "name": "Kensio Software",
8
+ "email": "hugh@kensiosoftware.co.uk"
9
+ },
10
+ "homepage": "https://kensio.ai",
11
+ "repository": "https://github.com/KensioSoftware/kensio.ai",
12
+ "license": "Apache-2.0",
13
+ "keywords": ["testing", "aws", "yulin", "cdk", "cloudformation"]
14
+ }
package/README.md ADDED
@@ -0,0 +1,76 @@
1
+ # @kensio/yulin-aws-simulation
2
+
3
+ A Claude Code skill for testing AWS code with [Yulin](https://yulinsim.dev/) (`@kensio/yulin`), an
4
+ AWS simulator that runs in process, in memory, with no network and no AWS account.
5
+
6
+ Yulin's own docs are the authority on its API. This skill is the usage guidance that is not in the
7
+ API: what to reach for, what to avoid, and what to do when the simulator refuses something.
8
+
9
+ ## Install
10
+
11
+ From the marketplace:
12
+
13
+ ```bash
14
+ claude plugin marketplace add KensioSoftware/kensio.ai
15
+ claude plugin install yulin-aws-simulation@kensio
16
+ ```
17
+
18
+ From npm:
19
+
20
+ ```bash
21
+ npm install @kensio/yulin-aws-simulation
22
+ ```
23
+
24
+ ## What it covers
25
+
26
+ **Deploy your real synthesized template.** Deploy the JSON CDK produced, with
27
+ `deployTemplateFile({ templatePath, stackName })`, so a construct change that breaks the system
28
+ breaks the test. Do not hand-roll a wrapper that reads the file and calls `deployTemplate`: the file
29
+ path is how Yulin finds the cloud assembly beside it, so a wrapper loses staged CDK assets.
30
+ `transform` handles what a simulation cannot resolve, such as an ARN carrying a real account or a
31
+ hosted zone ID from a CDK lookup, and `watch` re-applies the file on change for dev servers.
32
+
33
+ **Intercept real SDK clients with `SimSdk`, never hand-roll stubs.**
34
+ `simSdk.intercept(SecretsManagerClient)` makes real clients answer from the simulation, and the code
35
+ under test never learns there is a simulator behind it. On a real project, replacing stubs with
36
+ interception immediately caught a Secrets Manager name ending in a hyphen and six characters,
37
+ ambiguous with the suffix Secrets Manager appends to an ARN and advised against by AWS, which had
38
+ already reached production. It caught a wrongly computed Cognito `SECRET_HASH` the stub had happily
39
+ accepted.
40
+
41
+ **Match service errors by `name`, not `instanceof`.** The SDK exports its exception classes, which
42
+ invites the wrong check. `instanceof` holds only while exactly one copy of the SDK is in play, so it
43
+ passes in production and fails against the simulator. `name` is what the wire carries, and is the
44
+ check that is right in both places.
45
+
46
+ **Expect refusals, and treat them as a feature.** Yulin refuses a property it does not simulate
47
+ rather than ignoring it, because silently accepting something that changes real behaviour is worse.
48
+ The cost is that one unsupported setting can make a whole stack unsimulatable, so enumerate every
49
+ refusal in one pass: strip properties from the synthesized template until it deploys, then raise
50
+ them together.
51
+
52
+ **Raise gaps upstream, and weight false passes far above false refusals.** A simulator that stays
53
+ silent about something costs nothing. One that says 200 where production says 403 converts a
54
+ deploy-time failure into a production one, which is the opposite of what it is for. Yulin once
55
+ authorised a Lambda function URL invocation against `lambda:InvokeFunctionUrl` alone, where
56
+ CloudFront origin access control also needs `lambda:InvokeFunction`. The tests passed, the release
57
+ went out, and the endpoint 403'd in production.
58
+
59
+ **Deploy expensive context once per test file.** Vitest gives each file its own worker, so a stack
60
+ deployed in `beforeAll` is already isolated between files. Isolation inside the file comes from
61
+ randomised names.
62
+
63
+ **Run the handler as a real simulated Lambda.** Bind an in-process handler to a template function
64
+ and its SDK calls are routed into the simulation as the execution role, so a missing permission on
65
+ that role fails the test at the point AWS would have failed it.
66
+
67
+ ## Related skills
68
+
69
+ - [`isolated-testing-style`](https://github.com/KensioSoftware/kensio.ai/tree/main/plugins/isolated-testing-style)
70
+ is the general argument this skill applies to AWS.
71
+ - [`part-factory-test-data`](https://github.com/KensioSoftware/kensio.ai/tree/main/plugins/part-factory-test-data)
72
+ builds the objects those tests are made of.
73
+
74
+ Part of [kensio.ai](https://github.com/KensioSoftware/kensio.ai). Licensed under the Apache License
75
+ 2.0. See the [LICENSE](https://github.com/KensioSoftware/kensio.ai/blob/main/LICENSE) in the
76
+ repository root.
package/package.json ADDED
@@ -0,0 +1,33 @@
1
+ {
2
+ "name": "@kensio/yulin-aws-simulation",
3
+ "version": "1.3.0",
4
+ "description": "How to test AWS code well with the @kensio/yulin in-process simulator: using it directly instead of building a harness around it, driving tests, local dev and production from one synthesized CDK template, intercepting SDK clients, controlling simulated time, and handling what the simulator refuses.",
5
+ "keywords": [
6
+ "aws",
7
+ "cdk",
8
+ "claude",
9
+ "claude-code",
10
+ "claude-code-plugin",
11
+ "cloudformation",
12
+ "kensio",
13
+ "skill",
14
+ "testing",
15
+ "yulin"
16
+ ],
17
+ "homepage": "https://kensio.ai",
18
+ "license": "Apache-2.0",
19
+ "author": "Kensio Software <hugh@kensiosoftware.co.uk>",
20
+ "repository": {
21
+ "type": "git",
22
+ "url": "git+https://github.com/KensioSoftware/kensio.ai.git",
23
+ "directory": "plugins/yulin-aws-simulation"
24
+ },
25
+ "files": [
26
+ ".claude-plugin",
27
+ "skills",
28
+ "README.md"
29
+ ],
30
+ "publishConfig": {
31
+ "access": "public"
32
+ }
33
+ }
@@ -0,0 +1,322 @@
1
+ ---
2
+ name: yulin-aws-simulation
3
+ description: Use the @kensio/yulin in-process AWS simulator well when testing AWS code, using it directly rather than building a harness around it, driving tests, local dev and production from one synthesized CDK template, intercepting SDK clients with SimSdk, controlling simulated time, matching service errors by name, and handling properties Yulin refuses to simulate. Use when writing or reviewing tests that touch AWS, when replacing aws-sdk-client-mock or hand-rolled AWS stubs, when a CDK stack needs testing, when test setup around Yulin is growing helper classes or wrapper functions, and when Yulin refuses a template property or an SDK command.
4
+ ---
5
+
6
+ # Testing with Yulin
7
+
8
+ [Yulin](https://yulinsim.dev/) (`@kensio/yulin`) simulates AWS in process, in memory, with no
9
+ network and no AWS account. Its own docs are the authority on the API. This skill covers how to use
10
+ it well, which is mostly not in the API.
11
+
12
+ Read the package docs for anything API-shaped:
13
+ [the README](https://github.com/KensioSoftware/yulin#readme), `docs/sdk/` for interception, and
14
+ `docs/services/<name>/` for each simulated service.
15
+
16
+ This skill serves the `isolated-testing-style` skill, which is the general argument for simulation
17
+ over stubs.
18
+
19
+ Yulin is deliberately flexible, and plenty of shapes work. What follows is the recommended way to
20
+ get the most out of it rather than a set of rules: each one says what it buys, so a situation that
21
+ does not want that trade can go the other way knowingly.
22
+
23
+ ## Use what Yulin already gives you
24
+
25
+ The most common way to go wrong with Yulin is to build something on top of it. The failure looks
26
+ like a `TestAwsEnvironment` class, a `setupSimulatedAws()` helper returning six things, a factory
27
+ per service, or a `beforeEach` that reassembles the world — a private framework wrapped around a
28
+ tool that is already the framework.
29
+
30
+ It is worth resisting, because Yulin is built to be used directly:
31
+
32
+ - `new SimAws()` and `new SimSdk()` are plain constructors. No side effects, no network, nothing to
33
+ undo, nothing to await. Creating a simulation per test costs approximately nothing.
34
+ - Service accessors take the same Command objects the AWS SDK does, so seeding and asserting need no
35
+ translation layer of their own.
36
+ - `using simSdk = new SimSdk()` is the teardown.
37
+ - `deployTemplateFile` is the environment.
38
+
39
+ So the recommended shape of a test is: construct, deploy if a template is involved, intercept,
40
+ exercise, then assert by reading the simulation back. A wrapper around any of those steps hides the
41
+ one thing a reader of the test needs to see, and it has to be maintained forever after.
42
+
43
+ If a sequence genuinely repeats, make it a small function in the same test file, and keep it
44
+ returning the simulation objects themselves rather than a bespoke shape of its own. The point at
45
+ which it wants a class, an options interface, or a directory, it has stopped being test setup and
46
+ become a second product.
47
+
48
+ ## One synthesized template, for tests, local dev and production
49
+
50
+ Describe the infrastructure once, in CDK, and let the same synthesized output drive all three.
51
+ Production deploys it. The local dev server deploys it. The tests deploy it:
52
+
53
+ ```typescript
54
+ const stack = await simAws.region("eu-west-2").cloudFormation().deployTemplateFile({
55
+ templatePath: "cdk.out/SiteStack.template.json",
56
+ stackName: "site-stack",
57
+ });
58
+
59
+ await stack.waitForDeployComplete();
60
+ ```
61
+
62
+ A test written against a template you wrote by hand only tests the template you wrote by hand.
63
+ Deploying the synthesized output means a construct change that breaks the system breaks the test.
64
+
65
+ The same argument extends past the test suite. A dev environment that creates its buckets and tables
66
+ by hand is a third description of the infrastructure, drifting away from the other two at its own
67
+ pace, and the drift shows up as a bug that reproduces in exactly one of the three places. Pointing
68
+ the dev server at `cdk.out` as well removes the whole category: what runs locally is what CI tested
69
+ and what production deploys, and `watch` turns a `cdk synth` into a stack update in place.
70
+
71
+ The corollary is that infrastructure belongs in the CDK app even when only a test needs it. A bucket
72
+ conjured in test setup is infrastructure that production does not have.
73
+
74
+ **Do not hand-roll a wrapper that reads the file and calls `deployTemplate`.** `deployTemplateFile`
75
+ already reads it, and it locates the cloud assembly beside the file, so the assets manifest and
76
+ staged asset directories resolve. A wrapper that reads the JSON itself loses that, and anything
77
+ needing a CDK asset, such as a `Custom::CDKBucketDeployment` or a `Code.fromAsset` function, fails
78
+ with `No CDK assets manifest is available.`
79
+
80
+ The two options that make a wrapper unnecessary:
81
+
82
+ - **`transform`** is given the parsed template and answers with the one to deploy. It runs on the
83
+ deployment and again on every re-read, which is what a wrapper cannot do. Use it for what a
84
+ simulation genuinely cannot resolve: an ARN carrying a real account, or a hosted zone ID that came
85
+ from `HostedZone.fromLookup`.
86
+ - **`watch`** re-applies the file when it changes, updating the stack in place. This is for dev
87
+ servers: a `cdk synth` becomes a stack update without restarting the process, and resources the
88
+ change left alone keep what they hold.
89
+
90
+ ```typescript
91
+ await simAws.cloudFormation().deployTemplateFile({
92
+ templatePath: "cdk.out/SiteStack.template.json",
93
+ transform: withoutLookedUpHostedZone,
94
+ watch: { onUpdated: () => srv.reload() },
95
+ });
96
+ ```
97
+
98
+ ## Intercept real SDK clients, never hand-roll stubs
99
+
100
+ `SimSdk` replaces the `send` method of an AWS SDK client class or instance, so real clients answer
101
+ from the simulation. The code under test uses the SDK exactly as it does in production and never
102
+ learns there is a simulator behind it.
103
+
104
+ ```typescript
105
+ import { SecretsManagerClient } from "@aws-sdk/client-secrets-manager";
106
+ import { SimSdk } from "@kensio/yulin/sdk";
107
+
108
+ using simSdk = new SimSdk();
109
+ simSdk.intercept(SecretsManagerClient); // Every instance, including ones made later.
110
+ ```
111
+
112
+ A stub asserts that your code called something. The simulator asserts that it called the service
113
+ correctly. On a real project, swapping stubs for interception caught two bugs the same afternoon,
114
+ both already in production:
115
+
116
+ - A Secrets Manager secret whose name ended in a hyphen and six characters, which is exactly the
117
+ suffix Secrets Manager appends to an ARN. AWS advises against names of that shape because they are
118
+ ambiguous with the ARN form. A stub has no naming rules, so it had accepted it happily.
119
+ - A Cognito `SECRET_HASH` computed the wrong way. The stub had accepted that too, because a stub is
120
+ never going to verify a signature.
121
+
122
+ Intercept the class rather than the instance in most cases, since the code under test usually
123
+ constructs its own clients. Intercept an instance when only one client should reach the simulation.
124
+
125
+ Each `SimSdk` owns a `SimAws`, reachable as `simSdk.simAws` for seeding and inspecting state. Pass
126
+ an existing one with `new SimSdk({ simAws })` to share.
127
+
128
+ ### Intercept what the code actually sends through
129
+
130
+ Interception replaces `send` on the thing it is given, so it has to be given the client the code
131
+ under test actually calls. The wrapper clients are where this bites: a `DynamoDBDocumentClient`
132
+ built over a `DynamoDBClient` is what the code sends through, so it is the document client that
133
+ needs intercepting, not the client underneath it.
134
+
135
+ ```typescript
136
+ using simSdk = new SimSdk();
137
+
138
+ const documents = DynamoDBDocumentClient.from(new DynamoDBClient({ region: "eu-west-2" }));
139
+ simSdk.intercept(documents); // Not the DynamoDBClient it was built from.
140
+ ```
141
+
142
+ Every Command through an intercepted client is routed to the simulation by default. An allow list of
143
+ Command classes narrows that, which is worth reaching for only when something else should genuinely
144
+ handle the rest.
145
+
146
+ ### Prefer `using` over a teardown step
147
+
148
+ `SimSdk` and the interception handles it returns are disposable, so `using simSdk = new SimSdk();`
149
+ restores every intercepted client at the end of the scope. `simSdk.restoreAll()` and
150
+ `interception.restore()` do the same thing by hand.
151
+
152
+ The recommendation is `using`, because it is teardown that cannot be forgotten or skipped by an
153
+ early return, and because it leaves nothing for a later test to inherit if the one before it threw.
154
+ Reach for the explicit calls when interception has to stop somewhere other than the end of a scope.
155
+
156
+ ## Freeze the clock and advance it deliberately
157
+
158
+ Each `SimAws` carries its own clock, independent of the host and of every other simulation in the
159
+ process. The recommendation is to start it frozen at a fixed instant and move it only on purpose:
160
+
161
+ ```typescript
162
+ const simAws = new SimAws({
163
+ clock: new SimFixedClock(new Date("2026-07-26T09:00:00.000Z")),
164
+ });
165
+
166
+ // Given a session that has run out while the caller was idle.
167
+ await simAws.clock().advanceBy({ minutes: 20 });
168
+ ```
169
+
170
+ What this buys is that time-dependent behaviour becomes something a test can assert on in
171
+ microseconds rather than something it waits for or gives up on. A good deal of the simulation keys
172
+ off that clock: EventBridge rules and Scheduler schedules fire only when time is advanced past them,
173
+ DynamoDB items pass their TTL, Secrets Manager deletions come due, `AssumeRole` sessions expire, and
174
+ Lambda event source mappings re-poll. Inside a simulated Lambda, `Date.now()` and `new Date()`
175
+ report simulated time, so a handler's own expiry logic is exercised without a stub in sight.
176
+
177
+ That last point is worth drawing out. `isolated-testing-style` allows a stub for a clock, on the
178
+ grounds that a clock has no rules worth modelling. Against Yulin that exception is not needed: the
179
+ clock is part of the simulation, and advancing it exercises the real expiry rules of the services
180
+ around it rather than only the code's own arithmetic.
181
+
182
+ `simAws.clock().resume()` switches to tracking the underlying clock, and `simAws.clock().isFrozen`
183
+ reports which mode it is in. Running mode suits a local dev server; a test that wants it usually
184
+ wants an advance instead.
185
+
186
+ ## Assert by reading the simulation back
187
+
188
+ The simulation holds real state, so the assertion can read it. After exercising the code, ask the
189
+ service what happened rather than asking the SDK what it was told:
190
+
191
+ ```typescript
192
+ // Then the upload is in the bucket, under the key the handler chose.
193
+ const object = await simAws.s3().getObject(new GetObjectCommand({ Bucket: bucket, Key: key }));
194
+ ```
195
+
196
+ Service accessors take the same Command objects the SDK does, so the seeding and assertion code
197
+ reads like the production code between them.
198
+
199
+ This is where simulation pays off over stubs a second time. A call-count assertion holds only for
200
+ today's implementation; a state assertion holds however the handler is rewritten, and it fails if
201
+ the call was made in a way the real service would have rejected.
202
+
203
+ ## Match service errors by name, not instanceof
204
+
205
+ ```typescript
206
+ // Wrong. Passes in production, fails against the simulator.
207
+ if (error instanceof ResourceNotFoundException) { ... }
208
+
209
+ // Right.
210
+ if (error instanceof Error && error.name === "ResourceNotFoundException") { ... }
211
+ ```
212
+
213
+ The SDK exports exception classes, which invites the `instanceof` check. It holds only while exactly
214
+ one copy of the SDK package is in play. Two copies in the module graph, a bundler, or a simulator
215
+ raising its own classes, and it silently stops matching. Yulin's errors carry the service's real
216
+ error names and SDK-shaped `$metadata`, but they are not instances of the SDK classes.
217
+
218
+ This is worth fixing in production code, not worked around in tests. `name` is what the wire
219
+ carries, so the `name` check is the one that is right in both places. A version skew between two
220
+ `@aws-sdk/client-*` packages breaks `instanceof` in production too, just less predictably than the
221
+ simulator does.
222
+
223
+ ## Expect refusals, and treat them as a feature
224
+
225
+ Yulin refuses a property it does not simulate rather than ignoring it. That is the right trade:
226
+ silently accepting something that changes real behaviour turns a deploy-time failure into a
227
+ production one.
228
+
229
+ The cost is that one unsupported setting can make a whole stack unsimulatable, and the refusal
230
+ arrives one property at a time. When that happens, **enumerate every refusal in one pass**. Strip
231
+ properties from the synthesized template until it deploys, keeping a list, then raise them together
232
+ upstream.
233
+
234
+ ```typescript
235
+ // A throwaway transform used to find the floor, not to keep.
236
+ function stripUntilItDeploys(template: CfnTemplateBodyRecord): CfnTemplateBodyRecord {
237
+ // Remove one refused property, re-run, record the next refusal, repeat.
238
+ // Keep the list. Raise it as one issue.
239
+ }
240
+ ```
241
+
242
+ Doing this one release at a time means one round trip per property, and you never learn how far away
243
+ a working simulation actually is. This is the same rule as not discovering service ceilings one
244
+ failed deployment at a time.
245
+
246
+ Not every gap is a refusal. Several services record a property they cannot model and carry on,
247
+ reporting it as an ignored property on the stack and on the resource. Check that report before
248
+ trusting a test that depends on the setting.
249
+
250
+ ## Raise gaps upstream, and weight false passes far above false refusals
251
+
252
+ Fix gaps on [the Yulin repository](https://github.com/KensioSoftware/yulin) rather than working
253
+ around them locally. A local workaround has to be maintained in every project that hits the same
254
+ gap.
255
+
256
+ When reporting, the asymmetry matters more than the volume:
257
+
258
+ - A simulator that **stays silent** about something costs nothing. The test does not cover that
259
+ behaviour, which is where it was already.
260
+ - A simulator that **says 200 where production says 403** converts a deploy-time failure into a
261
+ production one, which is the opposite of what it is for.
262
+
263
+ So a false pass deserves far more attention than a false refusal. A real example: Yulin authorised a
264
+ Lambda function URL invocation against `lambda:InvokeFunctionUrl` alone. CloudFront origin access
265
+ control also needs `lambda:InvokeFunction`. The tests passed, the release went out, and the endpoint
266
+ 403'd in production. A refusal would have cost an afternoon. The false pass cost an incident.
267
+
268
+ Report a false pass with what production does and what the simulation did. Report a false refusal
269
+ with the property and the template that carries it.
270
+
271
+ ## Deploy expensive context once per test file
272
+
273
+ Vitest gives each test file its own worker, so module-level state is already isolated between files.
274
+ Deploy a stack once for the file and let the tests share it. Isolation inside the file comes from
275
+ randomised names, not from rebuilding the environment.
276
+
277
+ ```typescript
278
+ let simAws: SimAws;
279
+
280
+ beforeAll(async () => {
281
+ // Given the real synthesized stack, deployed once for this file.
282
+ simAws = new SimAws();
283
+ const stack = await simAws.cloudFormation().deployTemplateFile({
284
+ templatePath: "cdk.out/SiteStack.template.json",
285
+ });
286
+ await stack.waitForDeployComplete();
287
+ });
288
+
289
+ it("stores an upload", async () => {
290
+ // Given a key no other test in this file is using.
291
+ const key = `uploads/${faker.string.uuid()}.png`;
292
+ // ...
293
+ });
294
+ ```
295
+
296
+ Putting a template deployment in `beforeEach` pays for the whole stack once per test for no
297
+ isolation you did not already have.
298
+
299
+ A template deployment is the only thing usually worth hoisting. Everything else — the `SimSdk`, a
300
+ seeded row, a bucket key — is cheap enough to build inside the test that needs it, which is also
301
+ where it is easiest to read. A `beforeEach` that assembles state for tests that do not all want the
302
+ same state is the beginning of the harness this skill opens by arguing against.
303
+
304
+ ## Run the handler as a real simulated Lambda
305
+
306
+ Yulin can run an in-process handler as a function in the simulation, rather than calling it
307
+ directly. Its SDK calls are routed into the simulation as the execution role, so the IAM policies in
308
+ the template are exercised too.
309
+
310
+ Bind a handler to a template function at deploy time with `bindings`:
311
+
312
+ ```typescript
313
+ await simAws.cloudFormation().deployTemplateFile({
314
+ templatePath: "cdk.out/ApiStack.template.json",
315
+ bindings: [{ logicalId: "UploadFunction", handler: uploadHandler }],
316
+ });
317
+ ```
318
+
319
+ The handler still runs in process, so it can close over test state and be stepped through in a
320
+ debugger. The difference from calling it directly is that a missing `s3:PutObject` on the execution
321
+ role now fails the test, at the point AWS would have failed it. A binding can target a function by
322
+ `logicalId`, `functionName`, `arn`, `cdkPath`, or `imageRepository` for a container image function.