@kensio/yulin 1.20.14 → 1.20.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (154) hide show
  1. package/README.md +13 -0
  2. package/{dist/config/oxlint/cffjs2.oxlintrc.json → cffjs2.oxlintrc.json} +1 -1
  3. package/dist/config/oxlint/cffjs2.oxlint.config.js +4 -3
  4. package/dist/sdk/module/sim-sdk-module-client-interceptor.d.ts +5 -0
  5. package/dist/sdk/module/sim-sdk-module-client-interceptor.js +14 -0
  6. package/dist/sdk/module/sim-sdk-static-client-factory.d.ts +26 -0
  7. package/dist/sdk/module/sim-sdk-static-client-factory.js +59 -0
  8. package/dist/sdk/send-patch.d.ts +8 -0
  9. package/dist/sdk/send-patch.js +11 -0
  10. package/dist/service/aws/caller/sim-aws-caller-resolver.d.ts +28 -6
  11. package/dist/service/aws/caller/sim-aws-caller-resolver.js +18 -11
  12. package/dist/service/aws/caller/sim-aws-caller.d.ts +9 -0
  13. package/dist/service/aws/caller/sim-aws-resolved-caller.d.ts +19 -0
  14. package/dist/service/aws/caller/sim-aws-resolved-caller.js +26 -0
  15. package/dist/service/aws/factory/sim-aws-account-region-service-builder.d.ts +6 -0
  16. package/dist/service/aws/factory/sim-aws-account-region-service-builder.js +3 -0
  17. package/dist/service/aws/factory/sim-aws-account-service-cache.d.ts +6 -0
  18. package/dist/service/aws/factory/sim-aws-account-service-cache.js +5 -0
  19. package/dist/service/aws/factory/sim-aws-service-factory.d.ts +5 -0
  20. package/dist/service/aws/factory/sim-aws-service-factory.js +6 -0
  21. package/dist/service/aws/sim-aws-properties.d.ts +15 -0
  22. package/dist/service/aws/sim-aws.js +2 -1
  23. package/dist/service/cloudformation/resource/resolve/property/sim-cfn-resource-property-resolver.d.ts +11 -4
  24. package/dist/service/cloudformation/resource/resolve/property/sim-cfn-resource-property-resolver.js +21 -8
  25. package/dist/service/cloudformation/resource/sim-cfn-resource.type.d.ts +9 -0
  26. package/dist/service/cloudformation/template/dynamic/make-sim-cfn-dynamic-references.d.ts +2 -0
  27. package/dist/service/cloudformation/template/dynamic/make-sim-cfn-dynamic-references.js +2 -2
  28. package/dist/service/cloudformation/template/dynamic/sim-cfn-dynamic-reference-resolvers.d.ts +10 -0
  29. package/dist/service/cloudformation/template/dynamic/sim-cfn-dynamic-reference-resolvers.js +4 -3
  30. package/dist/service/cloudformation/template/dynamic/sim-cfn-dynamic-reference-scan.d.ts +11 -0
  31. package/dist/service/cloudformation/template/dynamic/sim-cfn-dynamic-reference-scan.js +21 -0
  32. package/dist/service/cloudformation/template/dynamic/sim-cfn-dynamic-references.d.ts +24 -2
  33. package/dist/service/cloudformation/template/dynamic/sim-cfn-dynamic-references.js +50 -11
  34. package/dist/service/cloudformation/template/dynamic/sim-cfn-prefetched-dynamic-references.d.ts +42 -0
  35. package/dist/service/cloudformation/template/dynamic/sim-cfn-prefetched-dynamic-references.js +80 -0
  36. package/dist/service/eventbridge/delivery/sim-event-bridge-delivery-failures.d.ts +16 -0
  37. package/dist/service/eventbridge/delivery/sim-event-bridge-delivery-failures.js +19 -0
  38. package/dist/service/iam/authorize/context/sim-iam-auth-z-caller-context-builder.d.ts +30 -7
  39. package/dist/service/iam/authorize/context/sim-iam-auth-z-caller-context-builder.js +20 -11
  40. package/dist/service/iam/authorize/context/sim-iam-auth-z-context-builder.d.ts +2 -1
  41. package/dist/service/iam/authorize/context/sim-iam-auth-z-context-builder.js +5 -1
  42. package/dist/service/iam/authorize/match/sim-iam-policy-principal-matcher.d.ts +10 -0
  43. package/dist/service/iam/authorize/match/sim-iam-policy-principal-matcher.js +15 -3
  44. package/dist/service/iam/authorize/sim-iam-account-auth-z.d.ts +11 -4
  45. package/dist/service/iam/authorize/sim-iam-account-auth-z.js +5 -3
  46. package/dist/service/iam/authorize/sim-iam-authorizer.d.ts +2 -1
  47. package/dist/service/iam/authorize/sim-iam-authorizer.js +1 -0
  48. package/dist/service/iam/command/policy/create-policy/create-policy-input-resolver.js +4 -1
  49. package/dist/service/iam/command/policy/put-role-policy/put-role-policy.handler.js +5 -1
  50. package/dist/service/iam/command/policy/put-user-policy/put-user-policy.handler.js +5 -1
  51. package/dist/service/iam/policy/parse/sim-iam-document-parser.d.ts +4 -1
  52. package/dist/service/iam/policy/parse/sim-iam-document-parser.js +13 -15
  53. package/dist/service/iam/policy/sim-iam-statement-label.d.ts +28 -0
  54. package/dist/service/iam/policy/sim-iam-statement-label.js +17 -0
  55. package/dist/service/iam/policy/sim-iam-statement-strings.d.ts +11 -0
  56. package/dist/service/iam/policy/sim-iam-statement-strings.js +27 -0
  57. package/dist/service/iam/role/sim-iam-role-with-policy.factory.d.ts +9 -0
  58. package/dist/service/iam/role/sim-iam-role-with-policy.factory.js +4 -2
  59. package/dist/service/iam/sim-iam-account-parts.d.ts +8 -0
  60. package/dist/service/iam/sim-iam-account-parts.js +1 -0
  61. package/dist/service/iam/validate/sim-iam-policy-document-validator.d.ts +9 -2
  62. package/dist/service/iam/validate/sim-iam-policy-document-validator.js +37 -19
  63. package/dist/service/lambda/function/code/sim-lambda-vm-zip-function.factory.d.ts +7 -0
  64. package/dist/service/lambda/function/code/sim-lambda-vm-zip-function.factory.js +1 -0
  65. package/dist/service/lambda/function/code/vm/sdk/sim-lambda-provided-sdk-module.d.ts +19 -0
  66. package/dist/service/lambda/function/code/vm/sdk/sim-lambda-provided-sdk-module.js +78 -0
  67. package/dist/service/lambda/function/code/vm/sdk/sim-lambda-sdk-packages-not-installed.error.d.ts +15 -0
  68. package/dist/service/lambda/function/code/vm/sdk/sim-lambda-sdk-packages-not-installed.error.js +27 -0
  69. package/dist/service/lambda/function/code/vm/sdk/sim-lambda-vm-sdk-module-provider.d.ts +9 -0
  70. package/dist/service/lambda/function/code/vm/sdk/sim-sdk-lambda-vm-module-provider.d.ts +10 -0
  71. package/dist/service/lambda/function/code/vm/sdk/sim-sdk-lambda-vm-module-provider.js +28 -34
  72. package/dist/service/lambda/function/code/vm/sim-lambda-host-modules.d.ts +15 -0
  73. package/dist/service/lambda/function/code/vm/sim-lambda-host-modules.js +31 -0
  74. package/dist/service/lambda/function/code/vm/sim-lambda-vm-module-resolver.d.ts +7 -0
  75. package/dist/service/lambda/function/code/vm/sim-lambda-vm-module-resolver.js +15 -0
  76. package/dist/service/lambda/function/code/vm/sim-lambda-vm-modules.js +5 -4
  77. package/dist/service/s3/command/put-bucket-policy/put-bucket-policy.handler.js +4 -1
  78. package/dist/service/scheduler/delivery/sim-scheduler-delivery-failures.d.ts +16 -0
  79. package/dist/service/scheduler/delivery/sim-scheduler-delivery-failures.js +19 -0
  80. package/dist/service/scheduler/index.d.ts +1 -1
  81. package/dist/service/scheduler/index.js +1 -1
  82. package/dist/service/secretsmanager/cfn/dynamic/sim-cfn-secrets-manager-dynamic-reference-resolver.d.ts +13 -15
  83. package/dist/service/secretsmanager/cfn/dynamic/sim-cfn-secrets-manager-dynamic-reference-resolver.js +22 -43
  84. package/dist/service/secretsmanager/cfn/dynamic/sim-cfn-secrets-manager-reference-stand-in.d.ts +9 -0
  85. package/dist/service/secretsmanager/cfn/dynamic/sim-cfn-secrets-manager-reference-stand-in.js +14 -0
  86. package/dist/service/secretsmanager/cfn/dynamic/sim-cfn-secrets-manager-reference-value.d.ts +12 -0
  87. package/dist/service/secretsmanager/cfn/dynamic/sim-cfn-secrets-manager-reference-value.js +29 -0
  88. package/dist/service/ssm/cfn/dynamic/sim-cfn-ssm-dynamic-reference-resolver.d.ts +22 -6
  89. package/dist/service/ssm/cfn/dynamic/sim-cfn-ssm-dynamic-reference-resolver.js +39 -24
  90. package/dist/service/ssm/cfn/dynamic/sim-cfn-ssm-secure-dynamic-reference-resolver.d.ts +10 -5
  91. package/dist/service/ssm/cfn/dynamic/sim-cfn-ssm-secure-dynamic-reference-resolver.js +10 -8
  92. package/dist/service/ssm/sim-ssm.d.ts +10 -2
  93. package/dist/service/ssm/sim-ssm.js +12 -4
  94. package/dist/service/sts/auth-z/assume-role-auth-z-coordinator.d.ts +9 -2
  95. package/dist/service/sts/auth-z/assume-role-auth-z-coordinator.js +2 -2
  96. package/dist/service/sts/auth-z/assume-role-source-account-auth-z.d.ts +10 -4
  97. package/dist/service/sts/auth-z/assume-role-source-account-auth-z.js +8 -7
  98. package/dist/service/sts/auth-z/assume-role-target-auth-z.d.ts +2 -2
  99. package/dist/service/sts/auth-z/assume-role-trust-policy-authorizer.d.ts +6 -2
  100. package/dist/service/sts/auth-z/assume-role-trust-policy-authorizer.js +14 -11
  101. package/dist/service/sts/command/assume-role/assume-role.handler.d.ts +5 -1
  102. package/dist/service/sts/command/assume-role/assume-role.handler.js +5 -2
  103. package/dist/service/sts/command/get-caller-identity/get-caller-identity.handler.d.ts +5 -1
  104. package/dist/service/sts/command/get-caller-identity/get-caller-identity.handler.js +4 -1
  105. package/dist/service/sts/service-role/sim-service-role.js +5 -1
  106. package/dist/service/sts/sim-sts.d.ts +9 -1
  107. package/dist/service/sts/sim-sts.js +4 -0
  108. package/docs/README.md +54 -0
  109. package/docs/ai-skill/README.md +69 -0
  110. package/docs/cli/README.md +284 -0
  111. package/docs/factories/README.md +140 -0
  112. package/docs/lint/README.md +176 -0
  113. package/docs/non-aws-dependencies/README.md +261 -0
  114. package/docs/sdk/README.md +272 -0
  115. package/docs/serve/README.md +1126 -0
  116. package/docs/services/acm/README.md +823 -0
  117. package/docs/services/apigateway/README.md +1919 -0
  118. package/docs/services/apigatewayv2/README.md +2678 -0
  119. package/docs/services/athena/README.md +1044 -0
  120. package/docs/services/bedrock/README.md +350 -0
  121. package/docs/services/cloudformation/README.md +3440 -0
  122. package/docs/services/cloudfront/README.md +2874 -0
  123. package/docs/services/cloudwatch/README.md +420 -0
  124. package/docs/services/cognito/README.md +4923 -0
  125. package/docs/services/dynamodb/README.md +3784 -0
  126. package/docs/services/ecr/README.md +302 -0
  127. package/docs/services/ecs/README.md +2010 -0
  128. package/docs/services/elbv2/README.md +1897 -0
  129. package/docs/services/eventbridge/README.md +1110 -0
  130. package/docs/services/firehose/README.md +805 -0
  131. package/docs/services/glue/README.md +501 -0
  132. package/docs/services/iam/README.md +1283 -0
  133. package/docs/services/kinesis/README.md +454 -0
  134. package/docs/services/kms/README.md +659 -0
  135. package/docs/services/lambda/README.md +3645 -0
  136. package/docs/services/logs/README.md +868 -0
  137. package/docs/services/organizations/README.md +574 -0
  138. package/docs/services/personalize/README.md +835 -0
  139. package/docs/services/rekognition/README.md +1070 -0
  140. package/docs/services/route53/README.md +1783 -0
  141. package/docs/services/s3/README.md +3024 -0
  142. package/docs/services/scheduler/README.md +806 -0
  143. package/docs/services/secretsmanager/README.md +667 -0
  144. package/docs/services/ses/README.md +1011 -0
  145. package/docs/services/sns/README.md +1904 -0
  146. package/docs/services/sqs/README.md +1185 -0
  147. package/docs/services/ssm/README.md +1072 -0
  148. package/docs/services/stepfunctions/README.md +1332 -0
  149. package/docs/services/sts/README.md +245 -0
  150. package/docs/services/wafv2/README.md +1378 -0
  151. package/docs/terraform/README.md +248 -0
  152. package/docs/time/README.md +284 -0
  153. package/llms.txt +59 -0
  154. package/package.json +9 -5
@@ -0,0 +1,805 @@
1
+ # Simulated Kinesis Data Firehose
2
+
3
+ Yulin includes a simulated Kinesis Data Firehose for tests and local development. A delivery stream
4
+ takes records, buffers them, and writes them into a simulated S3 Bucket under the key format real
5
+ Firehose uses. The records come from `PutRecord` or off a simulated Kinesis stream. A test can put an
6
+ event and assert on the Object it landed in, without an AWS account and without waiting five minutes
7
+ for a buffer to flush.
8
+
9
+ Firehose specific types are imported from the `@kensio/yulin/firehose` subpath.
10
+
11
+ ## Putting a record and finding the Object
12
+
13
+ `simAws.firehose()` gives the service for the default account and region. A delivery stream needs a
14
+ Bucket to write into and a Role to write as, so both exist before it does. Advancing the clock past
15
+ the buffering interval is what delivers the buffer.
16
+
17
+ ```typescript sim-firehose-put-and-deliver
18
+ /**
19
+ * Putting an order event on a delivery stream and finding it in the Bucket.
20
+ */
21
+
22
+ import {
23
+ CreateDeliveryStreamCommand,
24
+ PutRecordCommand,
25
+ } from "@aws-sdk/client-firehose";
26
+ import { CreateRoleCommand, PutRolePolicyCommand } from "@aws-sdk/client-iam";
27
+ import { CreateBucketCommand, ListObjectsV2Command } from "@aws-sdk/client-s3";
28
+
29
+ import { SimAws } from "@kensio/yulin";
30
+
31
+ const simAws = new SimAws();
32
+
33
+ await simAws
34
+ .s3()
35
+ .createBucket(new CreateBucketCommand({ Bucket: "order-archive" }));
36
+
37
+ const { Role } = await simAws.iam().createRole(
38
+ new CreateRoleCommand({
39
+ RoleName: "OrderArchiveRole",
40
+ AssumeRolePolicyDocument: JSON.stringify({
41
+ Version: "2012-10-17",
42
+ Statement: {
43
+ Effect: "Allow",
44
+ Principal: { Service: "firehose.amazonaws.com" },
45
+ Action: "sts:AssumeRole",
46
+ },
47
+ }),
48
+ }),
49
+ );
50
+
51
+ await simAws.iam().putRolePolicy(
52
+ new PutRolePolicyCommand({
53
+ RoleName: "OrderArchiveRole",
54
+ PolicyName: "ArchiveOrders",
55
+ PolicyDocument: JSON.stringify({
56
+ Version: "2012-10-17",
57
+ Statement: {
58
+ Effect: "Allow",
59
+ Action: "s3:PutObject",
60
+ Resource: "arn:aws:s3:::order-archive/*",
61
+ },
62
+ }),
63
+ }),
64
+ );
65
+
66
+ await simAws.firehose().createDeliveryStream(
67
+ new CreateDeliveryStreamCommand({
68
+ DeliveryStreamName: "order-events",
69
+ ExtendedS3DestinationConfiguration: {
70
+ BucketARN: "arn:aws:s3:::order-archive",
71
+ RoleARN: Role.Arn,
72
+ BufferingHints: { IntervalInSeconds: 60 },
73
+ },
74
+ }),
75
+ );
76
+
77
+ const orderEvent = `${JSON.stringify({ id: "order-1" })}\n`;
78
+
79
+ await simAws.firehose().putRecord(
80
+ new PutRecordCommand({
81
+ DeliveryStreamName: "order-events",
82
+ Record: { Data: new TextEncoder().encode(orderEvent) },
83
+ }),
84
+ );
85
+
86
+ await simAws.clock().advanceBy({ seconds: 61 });
87
+
88
+ const { Contents } = await simAws
89
+ .s3()
90
+ .listObjectsV2(new ListObjectsV2Command({ Bucket: "order-archive" }));
91
+
92
+ // 2026/08/22/13/order-events-1-2026-08-22-13-51-01-92076704-cf4a-...
93
+ console.log(Contents?.[0]?.Key);
94
+ ```
95
+
96
+ The put is answered straight away with a record id. The Bucket stays empty until the buffer is
97
+ delivered. That delay is what a Firehose pipeline is built around.
98
+
99
+ ## Buffering
100
+
101
+ A delivery stream holds its records until the buffer passes `SizeInMBs` or `IntervalInSeconds`,
102
+ whichever comes first. Everything in one buffer arrives as one Object, with the records concatenated
103
+ end to end. A producer that wants lines puts the newline on the end of each record. The example
104
+ below does exactly that.
105
+
106
+ `IntervalInSeconds` runs from the first record of a buffer. The default is 300 seconds and the
107
+ default size is 5 MB, as they are on real Firehose.
108
+
109
+ ```typescript sim-firehose-buffering
110
+ /**
111
+ * Three records inside one buffering window, and the single Object they land
112
+ * in.
113
+ */
114
+
115
+ import { text } from "node:stream/consumers";
116
+
117
+ import {
118
+ CreateDeliveryStreamCommand,
119
+ PutRecordBatchCommand,
120
+ } from "@aws-sdk/client-firehose";
121
+ import { CreateRoleCommand, PutRolePolicyCommand } from "@aws-sdk/client-iam";
122
+ import {
123
+ CreateBucketCommand,
124
+ GetObjectCommand,
125
+ ListObjectsV2Command,
126
+ } from "@aws-sdk/client-s3";
127
+ import { assertNonNullable } from "@kensio/smartass";
128
+
129
+ import { SimAws } from "@kensio/yulin";
130
+
131
+ /**
132
+ * A Bucket, a delivery Role allowed the S3 actions given, and a delivery
133
+ * stream writing into the one as the other.
134
+ */
135
+ async function makeOrderArchive(
136
+ simAws: SimAws,
137
+ actions: readonly string[] = ["s3:PutObject"],
138
+ ): Promise<void> {
139
+ await simAws
140
+ .s3()
141
+ .createBucket(new CreateBucketCommand({ Bucket: "order-archive" }));
142
+
143
+ const { Role } = await simAws.iam().createRole(
144
+ new CreateRoleCommand({
145
+ RoleName: "OrderArchiveRole",
146
+ AssumeRolePolicyDocument: JSON.stringify({
147
+ Version: "2012-10-17",
148
+ Statement: {
149
+ Effect: "Allow",
150
+ Principal: { Service: "firehose.amazonaws.com" },
151
+ Action: "sts:AssumeRole",
152
+ },
153
+ }),
154
+ }),
155
+ );
156
+
157
+ await simAws.iam().putRolePolicy(
158
+ new PutRolePolicyCommand({
159
+ RoleName: "OrderArchiveRole",
160
+ PolicyName: "ArchiveOrders",
161
+ PolicyDocument: JSON.stringify({
162
+ Version: "2012-10-17",
163
+ Statement: {
164
+ Effect: "Allow",
165
+ Action: actions,
166
+ Resource: "arn:aws:s3:::order-archive/*",
167
+ },
168
+ }),
169
+ }),
170
+ );
171
+
172
+ await simAws.firehose().createDeliveryStream(
173
+ new CreateDeliveryStreamCommand({
174
+ DeliveryStreamName: "order-events",
175
+ ExtendedS3DestinationConfiguration: {
176
+ BucketARN: "arn:aws:s3:::order-archive",
177
+ RoleARN: Role.Arn,
178
+ BufferingHints: { IntervalInSeconds: 60 },
179
+ },
180
+ }),
181
+ );
182
+ }
183
+
184
+ const simAws = new SimAws();
185
+
186
+ await makeOrderArchive(simAws);
187
+
188
+ await simAws.firehose().putRecordBatch(
189
+ new PutRecordBatchCommand({
190
+ DeliveryStreamName: "order-events",
191
+ Records: ["order-1", "order-2", "order-3"].map((id) => ({
192
+ Data: new TextEncoder().encode(`${JSON.stringify({ id })}\n`),
193
+ })),
194
+ }),
195
+ );
196
+
197
+ await simAws.clock().advanceBy({ minutes: 2 });
198
+
199
+ const { Contents } = await simAws
200
+ .s3()
201
+ .listObjectsV2(new ListObjectsV2Command({ Bucket: "order-archive" }));
202
+
203
+ // 1
204
+ console.log(Contents?.length);
205
+
206
+ const object = await simAws.s3().getObject(
207
+ new GetObjectCommand({
208
+ Bucket: "order-archive",
209
+ Key: Contents?.[0]?.Key,
210
+ }),
211
+ );
212
+
213
+ assertNonNullable(object.Body, "The delivered Object has a body");
214
+
215
+ // {"id":"order-1"}
216
+ // {"id":"order-2"}
217
+ // {"id":"order-3"}
218
+ console.log(await text(object.Body));
219
+ ```
220
+
221
+ A buffer that fills before its interval is delivered on the size instead. That delivery happens on
222
+ the background scheduler, the way real Firehose answers the producer before it writes anything.
223
+ `simAws.backgroundTasksComplete()` is what a test waits on for it.
224
+
225
+ ## The Object key
226
+
227
+ The key is the `Prefix`, then the UTC date path, then the delivery stream name, its version, the
228
+ delivery time and a random string:
229
+
230
+ ```
231
+ <Prefix>YYYY/MM/DD/HH/<delivery-stream-name>-<version>-YYYY-MM-DD-HH-MM-SS-<random>
232
+ ```
233
+
234
+ A delivery stream with no `Prefix` gets the bare date path. The version is `1` and stays there,
235
+ since a delivery stream's configuration is fixed once it is created.
236
+
237
+ Simulated time is what the date path and the timestamp come from. A test that sets the clock to a
238
+ known instant knows the prefix its Objects are under, and can list them.
239
+
240
+ ## Reading from a Kinesis stream
241
+
242
+ A delivery stream can take its records off a simulated Kinesis stream instead. Create it with a
243
+ `DeliveryStreamType` of `KinesisStreamAsSource` and a `KinesisStreamSourceConfiguration` naming the
244
+ stream and the Role to read it as. Records put on the stream from then on are buffered and delivered
245
+ the way put records are.
246
+
247
+ ```typescript sim-firehose-kinesis-source
248
+ /**
249
+ * An order event put on a Kinesis stream, and the Object the delivery stream
250
+ * reading that stream wrote it into.
251
+ */
252
+
253
+ import { CreateDeliveryStreamCommand } from "@aws-sdk/client-firehose";
254
+ import { CreateRoleCommand, PutRolePolicyCommand } from "@aws-sdk/client-iam";
255
+ import {
256
+ CreateStreamCommand,
257
+ DescribeStreamSummaryCommand,
258
+ PutRecordCommand,
259
+ } from "@aws-sdk/client-kinesis";
260
+ import { CreateBucketCommand, ListObjectsV2Command } from "@aws-sdk/client-s3";
261
+
262
+ import { SimAws } from "@kensio/yulin";
263
+
264
+ const simAws = new SimAws();
265
+
266
+ await simAws
267
+ .s3()
268
+ .createBucket(new CreateBucketCommand({ Bucket: "order-archive" }));
269
+
270
+ await simAws
271
+ .kinesis()
272
+ .createStream(
273
+ new CreateStreamCommand({ StreamName: "orders", ShardCount: 2 }),
274
+ );
275
+
276
+ const { StreamDescriptionSummary } = await simAws
277
+ .kinesis()
278
+ .describeStreamSummary(
279
+ new DescribeStreamSummaryCommand({ StreamName: "orders" }),
280
+ );
281
+
282
+ const { Role } = await simAws.iam().createRole(
283
+ new CreateRoleCommand({
284
+ RoleName: "OrderArchiveRole",
285
+ AssumeRolePolicyDocument: JSON.stringify({
286
+ Version: "2012-10-17",
287
+ Statement: {
288
+ Effect: "Allow",
289
+ Principal: { Service: "firehose.amazonaws.com" },
290
+ Action: "sts:AssumeRole",
291
+ },
292
+ }),
293
+ }),
294
+ );
295
+
296
+ await simAws.iam().putRolePolicy(
297
+ new PutRolePolicyCommand({
298
+ RoleName: "OrderArchiveRole",
299
+ PolicyName: "ArchiveOrders",
300
+ PolicyDocument: JSON.stringify({
301
+ Version: "2012-10-17",
302
+ Statement: [
303
+ {
304
+ Effect: "Allow",
305
+ Action: "s3:PutObject",
306
+ Resource: "arn:aws:s3:::order-archive/*",
307
+ },
308
+ {
309
+ Effect: "Allow",
310
+ Action: [
311
+ "kinesis:DescribeStream",
312
+ "kinesis:GetShardIterator",
313
+ "kinesis:GetRecords",
314
+ ],
315
+ Resource: StreamDescriptionSummary.StreamARN,
316
+ },
317
+ ],
318
+ }),
319
+ }),
320
+ );
321
+
322
+ await simAws.firehose().createDeliveryStream(
323
+ new CreateDeliveryStreamCommand({
324
+ DeliveryStreamName: "order-events",
325
+ DeliveryStreamType: "KinesisStreamAsSource",
326
+ KinesisStreamSourceConfiguration: {
327
+ KinesisStreamARN: StreamDescriptionSummary.StreamARN,
328
+ RoleARN: Role.Arn,
329
+ },
330
+ ExtendedS3DestinationConfiguration: {
331
+ BucketARN: "arn:aws:s3:::order-archive",
332
+ RoleARN: Role.Arn,
333
+ BufferingHints: { IntervalInSeconds: 60 },
334
+ },
335
+ }),
336
+ );
337
+
338
+ const orderEvent = `${JSON.stringify({ id: "order-1" })}\n`;
339
+
340
+ await simAws.kinesis().putRecord(
341
+ new PutRecordCommand({
342
+ StreamName: "orders",
343
+ PartitionKey: "order-1",
344
+ Data: new TextEncoder().encode(orderEvent),
345
+ }),
346
+ );
347
+
348
+ await simAws.clock().advanceBy({ seconds: 60 });
349
+
350
+ const { Contents } = await simAws
351
+ .s3()
352
+ .listObjectsV2(new ListObjectsV2Command({ Bucket: "order-archive" }));
353
+
354
+ // 1
355
+ console.log(Contents?.length);
356
+ ```
357
+
358
+ Every shard of the stream is read, so records spread across partition keys all arrive. The Object
359
+ holds the record data as it was put on the stream. Firehose delivers those bytes and nothing else
360
+ about the Kinesis record.
361
+
362
+ Reading starts at the end of the stream when the delivery stream is created. Real Firehose starts
363
+ there too. A record put before that stays on the stream for whatever else is reading it.
364
+
365
+ The read happens on the clock. A record put on the stream is read when simulated time next moves, so
366
+ one `advanceBy` past the buffering interval covers both halves. `simAws.backgroundTasksComplete()`
367
+ does not move simulated time and leaves the record on the stream.
368
+
369
+ `DescribeDeliveryStream` reports the stream, the Role and the instant reading started under
370
+ `Source.KinesisStreamSourceDescription`. `PutRecord` and `PutRecordBatch` on a delivery stream with a
371
+ Kinesis source raise `InvalidArgumentException`, as they do on real Firehose. Put the record onto the
372
+ stream instead.
373
+
374
+ Deleting the delivery stream stops it reading. Records put afterwards stay on the stream.
375
+
376
+ ## Permissions
377
+
378
+ Every Firehose operation authorizes against the `firehose:` action of the same name, on the ARN of
379
+ the delivery stream it names. `ListDeliveryStreams` names none, and authorizes against `*`.
380
+
381
+ The delivery itself is a separate request, made as the delivery stream's `RoleARN`. The caller who
382
+ put the record needs no S3 permission at all, and a Role that cannot write to the Bucket fails the
383
+ delivery. Real
384
+ Firehose answered that `PutRecord` minutes earlier, and what became of the buffer reaches the
385
+ producer through CloudWatch. The simulator keeps the failure for a test to read instead.
386
+
387
+ ```typescript sim-firehose-permissions
388
+ /**
389
+ * A delivery Role that cannot write to the Bucket, and where the failure shows
390
+ * up.
391
+ */
392
+
393
+ import {
394
+ CreateDeliveryStreamCommand,
395
+ PutRecordCommand,
396
+ } from "@aws-sdk/client-firehose";
397
+ import { CreateRoleCommand, PutRolePolicyCommand } from "@aws-sdk/client-iam";
398
+ import { CreateBucketCommand, ListObjectsV2Command } from "@aws-sdk/client-s3";
399
+
400
+ import { SimAws } from "@kensio/yulin";
401
+
402
+ /**
403
+ * A Bucket, a delivery Role allowed the S3 actions given, and a delivery
404
+ * stream writing into the one as the other.
405
+ */
406
+ async function makeOrderArchive(
407
+ simAws: SimAws,
408
+ actions: readonly string[] = ["s3:PutObject"],
409
+ ): Promise<void> {
410
+ await simAws
411
+ .s3()
412
+ .createBucket(new CreateBucketCommand({ Bucket: "order-archive" }));
413
+
414
+ const { Role } = await simAws.iam().createRole(
415
+ new CreateRoleCommand({
416
+ RoleName: "OrderArchiveRole",
417
+ AssumeRolePolicyDocument: JSON.stringify({
418
+ Version: "2012-10-17",
419
+ Statement: {
420
+ Effect: "Allow",
421
+ Principal: { Service: "firehose.amazonaws.com" },
422
+ Action: "sts:AssumeRole",
423
+ },
424
+ }),
425
+ }),
426
+ );
427
+
428
+ await simAws.iam().putRolePolicy(
429
+ new PutRolePolicyCommand({
430
+ RoleName: "OrderArchiveRole",
431
+ PolicyName: "ArchiveOrders",
432
+ PolicyDocument: JSON.stringify({
433
+ Version: "2012-10-17",
434
+ Statement: {
435
+ Effect: "Allow",
436
+ Action: actions,
437
+ Resource: "arn:aws:s3:::order-archive/*",
438
+ },
439
+ }),
440
+ }),
441
+ );
442
+
443
+ await simAws.firehose().createDeliveryStream(
444
+ new CreateDeliveryStreamCommand({
445
+ DeliveryStreamName: "order-events",
446
+ ExtendedS3DestinationConfiguration: {
447
+ BucketARN: "arn:aws:s3:::order-archive",
448
+ RoleARN: Role.Arn,
449
+ BufferingHints: { IntervalInSeconds: 60 },
450
+ },
451
+ }),
452
+ );
453
+ }
454
+
455
+ const simAws = new SimAws();
456
+
457
+ // The Role may read the Bucket, and it may not write to it.
458
+ await makeOrderArchive(simAws, ["s3:GetObject", "s3:ListBucket"]);
459
+
460
+ await simAws.firehose().putRecord(
461
+ new PutRecordCommand({
462
+ DeliveryStreamName: "order-events",
463
+ Record: { Data: new TextEncoder().encode("one\n") },
464
+ }),
465
+ );
466
+
467
+ await simAws.clock().advanceBy({ minutes: 2 });
468
+
469
+ const { Contents } = await simAws
470
+ .s3()
471
+ .listObjectsV2(new ListObjectsV2Command({ Bucket: "order-archive" }));
472
+
473
+ // 0
474
+ console.log(Contents?.length ?? 0);
475
+
476
+ const [failure] = simAws.firehose().getDeliveryFailures();
477
+
478
+ // order-events could not write 1 record to order-archive
479
+ console.log(
480
+ `${failure?.deliveryStreamName} could not write ` +
481
+ `${failure?.recordCount} record to ${failure?.bucketName}`,
482
+ );
483
+
484
+ // true
485
+ console.log(failure?.wasRefused);
486
+ ```
487
+
488
+ The source `RoleARN` is a second Role, and reading is authorized against it in the same way. A Role
489
+ that cannot read the stream stops the delivery stream reading, and the failure goes to
490
+ `getSourceFailures()`. There is one of those per delivery stream at most. The Role is refused every
491
+ time it asks, and going round again would record the same refusal for as long as the simulation ran.
492
+
493
+ A failure carries the delivery stream, the Bucket, the key it tried, how many records were in the
494
+ buffer, the Role it wrote as and the error itself. A source failure carries the delivery stream, the
495
+ stream ARN, the Role it read as and the error. `wasRefused` separates an IAM denial from a
496
+ delivery that broke some other way. A denial is recorded quietly, since removing `s3:PutObject` is
497
+ what a test checking the denial does. Anything else is also warned about on the console.
498
+
499
+ ## Deploying from CloudFormation
500
+
501
+ `AWS::KinesisFirehose::DeliveryStream` deploys a delivery stream. `DeliveryStreamName`,
502
+ `DeliveryStreamType`, `ExtendedS3DestinationConfiguration`, `S3DestinationConfiguration`,
503
+ `KinesisStreamSourceConfiguration` and `Tags` are read. A `Ref` gives the delivery stream name and
504
+ `Fn::GetAtt` on `Arn` gives the ARN, the way real CloudFormation publishes them.
505
+
506
+ A CDK `DeliveryStream` with an `S3Bucket` destination synthesizes that resource, along with the
507
+ delivery Role and its policy. A CDK project reaches a simulated delivery stream by deploying its
508
+ synthesized template, and nothing here has to be written by hand.
509
+
510
+ ```typescript sim-firehose-cloudformation
511
+ /**
512
+ * Deploying a delivery stream from a template and putting a record onto it.
513
+ */
514
+
515
+ import { PutRecordCommand } from "@aws-sdk/client-firehose";
516
+ import { ListObjectsV2Command } from "@aws-sdk/client-s3";
517
+
518
+ import { SimAws } from "@kensio/yulin";
519
+
520
+ const simAws = new SimAws();
521
+
522
+ const stack = await simAws.cloudFormation().deployTemplate({
523
+ stackName: "orders-stack",
524
+ template: {
525
+ Resources: {
526
+ OrderArchive: {
527
+ Type: "AWS::S3::Bucket",
528
+ Properties: { BucketName: "order-archive" },
529
+ },
530
+ DeliveryRole: {
531
+ Type: "AWS::IAM::Role",
532
+ Properties: {
533
+ AssumeRolePolicyDocument: {
534
+ Version: "2012-10-17",
535
+ Statement: [
536
+ {
537
+ Effect: "Allow",
538
+ Principal: { Service: "firehose.amazonaws.com" },
539
+ Action: "sts:AssumeRole",
540
+ },
541
+ ],
542
+ },
543
+ Policies: [
544
+ {
545
+ PolicyName: "ArchiveOrders",
546
+ PolicyDocument: {
547
+ Version: "2012-10-17",
548
+ Statement: [
549
+ {
550
+ Effect: "Allow",
551
+ Action: "s3:PutObject",
552
+ Resource: "arn:aws:s3:::order-archive/*",
553
+ },
554
+ ],
555
+ },
556
+ },
557
+ ],
558
+ },
559
+ },
560
+ OrderEvents: {
561
+ Type: "AWS::KinesisFirehose::DeliveryStream",
562
+ Properties: {
563
+ DeliveryStreamName: "order-events",
564
+ DeliveryStreamType: "DirectPut",
565
+ ExtendedS3DestinationConfiguration: {
566
+ BucketARN: { "Fn::GetAtt": ["OrderArchive", "Arn"] },
567
+ RoleARN: { "Fn::GetAtt": ["DeliveryRole", "Arn"] },
568
+ Prefix: "orders/",
569
+ BufferingHints: { IntervalInSeconds: 60, SizeInMBs: 1 },
570
+ },
571
+ },
572
+ },
573
+ },
574
+ Outputs: {
575
+ DeliveryStreamArn: { Value: { "Fn::GetAtt": ["OrderEvents", "Arn"] } },
576
+ },
577
+ },
578
+ });
579
+
580
+ await stack.waitForDeployComplete();
581
+
582
+ // arn:aws:firehose:us-east-1:<account>:deliverystream/order-events
583
+ console.log(stack.outputs.get("DeliveryStreamArn")?.value);
584
+
585
+ await simAws.firehose().putRecord(
586
+ new PutRecordCommand({
587
+ DeliveryStreamName: "order-events",
588
+ Record: { Data: new TextEncoder().encode('{"id":"order-1"}\n') },
589
+ }),
590
+ );
591
+
592
+ await simAws.clock().advanceBy({ minutes: 2 });
593
+
594
+ const { Contents } = await simAws
595
+ .s3()
596
+ .listObjectsV2(new ListObjectsV2Command({ Bucket: "order-archive" }));
597
+
598
+ // orders/2026/08/22/13/order-events-1-2026-08-22-13-51-01-92076704-cf4a-...
599
+ console.log(Contents?.[0]?.Key);
600
+ ```
601
+
602
+ Both S3 destination properties are read, and a template declaring both is refused, the same as a
603
+ `CreateDeliveryStream` request carrying both.
604
+
605
+ The Bucket and the Role are named by ARN. A stack declaring all three deploys a delivery stream that
606
+ writes into the Bucket beside it, as the Role beside it, and a Role without `s3:PutObject` fails the
607
+ delivery the same way one created through `CreateDeliveryStream` does. Deleting the stack deletes the
608
+ delivery stream.
609
+
610
+ A delivery stream the template does not name is named after the stack and the logical ID, as real
611
+ CloudFormation names one. The name is trimmed to the 64 characters Firehose allows.
612
+
613
+ `DeliveryStreamEncryptionConfigurationInput` and `DirectPutSourceConfiguration` are recorded against
614
+ the resource as unsimulated, and the delivery stream deploys anyway. The destination properties this
615
+ simulation has no behaviour for go the same way, including `ProcessingConfiguration`,
616
+ `DynamicPartitioningConfiguration`, `DataFormatConversionConfiguration`, `CompressionFormat` and the
617
+ `CloudWatchLoggingOptions` CDK writes into every destination. They are all in
618
+ `stack.ignoredProperties`.
619
+
620
+ A `DeliveryStreamType` of `KinesisStreamAsSource` deploys as well. The
621
+ `KinesisStreamSourceConfiguration` names the stream by ARN and the Role to read it as, and a stack
622
+ declaring the stream beside the delivery stream archives what a producer puts on it.
623
+
624
+ A delivery stream this simulation cannot deliver for is skipped and recorded in
625
+ `stack.skippedResources`, and the rest of the stack deploys. That covers a destination other than
626
+ S3, and a source property naming somewhere the records cannot come from, such as
627
+ `MSKSourceConfiguration` or `DatabaseSourceConfiguration`. The source property is what the skip is
628
+ decided on, because a template that leaves `DeliveryStreamType` out gets `DirectPut` by default.
629
+
630
+ ## SDK interception
631
+
632
+ `SimSdk` intercepts a `FirehoseClient`, so application code that constructs its own client reaches
633
+ the simulation without being handed one.
634
+
635
+ ```typescript sim-firehose-sdk-interception
636
+ /**
637
+ * Application code sending to its own FirehoseClient, answered by the
638
+ * simulation.
639
+ */
640
+
641
+ import {
642
+ CreateDeliveryStreamCommand,
643
+ FirehoseClient,
644
+ PutRecordCommand,
645
+ } from "@aws-sdk/client-firehose";
646
+ import { CreateRoleCommand, PutRolePolicyCommand } from "@aws-sdk/client-iam";
647
+ import {
648
+ CreateBucketCommand,
649
+ ListObjectsV2Command,
650
+ S3Client,
651
+ } from "@aws-sdk/client-s3";
652
+
653
+ import type { SimAws } from "@kensio/yulin";
654
+ import { SimSdk } from "@kensio/yulin/sdk";
655
+
656
+ /**
657
+ * A Bucket, a delivery Role allowed the S3 actions given, and a delivery
658
+ * stream writing into the one as the other.
659
+ */
660
+ async function makeOrderArchive(
661
+ simAws: SimAws,
662
+ actions: readonly string[] = ["s3:PutObject"],
663
+ ): Promise<void> {
664
+ await simAws
665
+ .s3()
666
+ .createBucket(new CreateBucketCommand({ Bucket: "order-archive" }));
667
+
668
+ const { Role } = await simAws.iam().createRole(
669
+ new CreateRoleCommand({
670
+ RoleName: "OrderArchiveRole",
671
+ AssumeRolePolicyDocument: JSON.stringify({
672
+ Version: "2012-10-17",
673
+ Statement: {
674
+ Effect: "Allow",
675
+ Principal: { Service: "firehose.amazonaws.com" },
676
+ Action: "sts:AssumeRole",
677
+ },
678
+ }),
679
+ }),
680
+ );
681
+
682
+ await simAws.iam().putRolePolicy(
683
+ new PutRolePolicyCommand({
684
+ RoleName: "OrderArchiveRole",
685
+ PolicyName: "ArchiveOrders",
686
+ PolicyDocument: JSON.stringify({
687
+ Version: "2012-10-17",
688
+ Statement: {
689
+ Effect: "Allow",
690
+ Action: actions,
691
+ Resource: "arn:aws:s3:::order-archive/*",
692
+ },
693
+ }),
694
+ }),
695
+ );
696
+
697
+ await simAws.firehose().createDeliveryStream(
698
+ new CreateDeliveryStreamCommand({
699
+ DeliveryStreamName: "order-events",
700
+ ExtendedS3DestinationConfiguration: {
701
+ BucketARN: "arn:aws:s3:::order-archive",
702
+ RoleARN: Role.Arn,
703
+ BufferingHints: { IntervalInSeconds: 60 },
704
+ },
705
+ }),
706
+ );
707
+ }
708
+
709
+ using simSdk = new SimSdk();
710
+
711
+ simSdk.intercept(FirehoseClient);
712
+ simSdk.intercept(S3Client);
713
+
714
+ await makeOrderArchive(simSdk.simAws);
715
+
716
+ const firehose = new FirehoseClient({ region: "us-east-1" });
717
+ const s3 = new S3Client({ region: "us-east-1" });
718
+
719
+ await firehose.send(
720
+ new PutRecordCommand({
721
+ DeliveryStreamName: "order-events",
722
+ Record: { Data: new TextEncoder().encode("one\n") },
723
+ }),
724
+ );
725
+
726
+ await simSdk.simAws.clock().advanceBy({ minutes: 2 });
727
+
728
+ const { Contents } = await s3.send(
729
+ new ListObjectsV2Command({ Bucket: "order-archive" }),
730
+ );
731
+
732
+ // 1
733
+ console.log(Contents?.length);
734
+ ```
735
+
736
+ ## Supported commands
737
+
738
+ | Command | Notes |
739
+ | ------------------------ | ------------------------------------------------------------------------------------------------------- |
740
+ | `CreateDeliveryStream` | A name already in use raises `ResourceInUseException`. |
741
+ | `DeleteDeliveryStream` | The name is free again at once. Whatever was buffered goes with it. |
742
+ | `ListDeliveryStreams` | Sorted by name, paged with `Limit` and `ExclusiveStartDeliveryStreamName`. |
743
+ | `DescribeDeliveryStream` | Reports the one destination, and a Kinesis source under `Source`. |
744
+ | `PutRecord` | Records up to 1,000 KiB. Refused on a Kinesis-sourced delivery stream. |
745
+ | `PutRecordBatch` | Up to 500 records and 4 MiB, and refused on a Kinesis-sourced one too. `FailedPutCount` is always zero. |
746
+
747
+ Anything else refuses on send with `SimSdkUnsupportedCommandError`, which covers
748
+ `UpdateDestination`, `StartDeliveryStreamEncryption`, `StopDeliveryStreamEncryption` and the tag
749
+ operations.
750
+
751
+ ## Divergences and limitations
752
+
753
+ - **S3 is the only destination.** `RedshiftDestinationConfiguration`,
754
+ `ElasticsearchDestinationConfiguration`, the two OpenSearch ones,
755
+ `SplunkDestinationConfiguration`, `HttpEndpointDestinationConfiguration`,
756
+ `IcebergDestinationConfiguration` and `SnowflakeDestinationConfiguration` are each refused by name
757
+ at `CreateDeliveryStream`. A delivery stream created against one of them would take records and
758
+ drop them, and a test asserting on an empty Bucket would blame the code under test.
759
+ `S3DestinationConfiguration` and `ExtendedS3DestinationConfiguration` are both read, and a
760
+ request carrying both is refused with `InvalidArgumentException`, the way real Firehose refuses a
761
+ request naming more than one destination. Either way the delivery stream describes back through
762
+ `ExtendedS3DestinationDescription`, which is the shape carrying every field read here.
763
+ `S3DestinationDescription` is always absent.
764
+ - **`DirectPut` and `KinesisStreamAsSource` are the two sources.** Every other
765
+ `DeliveryStreamType`, such as `MSKAsSource` and `DatabaseAsSource`, is refused by name. A source
766
+ stream in another account or region is refused as well, since a simulated Firehose reads the
767
+ simulated Kinesis of its own scope.
768
+ - **A source stream is read through `GetRecords`.** Enhanced fan-out is absent from simulated
769
+ Kinesis as well. `RetryOptions` on the destination is read and stored, and nothing here fails a
770
+ read in a way that would use it. A read that fails stops the delivery stream instead, and the
771
+ failure goes to `getSourceFailures()`.
772
+ - **A source stream keeps the shards it was created with.** Simulated Kinesis does not reshard, so
773
+ the shards a delivery stream opens when it is created are the ones it reads for its life.
774
+ - **A delivery stream a template cannot deploy is skipped.** A destination other than S3, a source
775
+ outside the two simulated, and a source property such as `MSKSourceConfiguration` or
776
+ `DatabaseSourceConfiguration`, each leave the Resource in `stack.skippedResources` while the rest
777
+ of the stack deploys. See [deploying from CloudFormation](#deploying-from-cloudformation).
778
+ - **A delivery stream is `ACTIVE` as soon as it exists.** Real Firehose reports `CREATING` for a
779
+ minute or so, and a status a test has to poll through earns its keep only where something is
780
+ being brought up. `DELETING` is absent for the same reason.
781
+ - **Record transformation is absent.** `ProcessingConfiguration` invoking a Lambda over the buffer
782
+ is a second delivery path, and it is ignored. A delivery stream declaring one delivers the records
783
+ as they were put.
784
+ - **Dynamic partitioning is absent.** `DynamicPartitioningConfiguration` derives a partition key
785
+ from record content through jq or a Lambda. A record's bytes are carried here and never read.
786
+ - **Every Object holds the bytes that were put.** GZIP, Snappy, ZIP, Parquet and ORC all change the
787
+ bytes in the Object while leaving what a test is checking alone, so `CompressionFormat` and
788
+ `DataFormatConversionConfiguration` are left out. Every destination reports `UNCOMPRESSED`.
789
+ - **The error output path is unused.** `ErrorOutputPrefix` is read and reported back by
790
+ `DescribeDeliveryStream`. Writing under it takes a record that failed its own transformation or
791
+ its own conversion, and both of those are absent.
792
+ - **Server-side encryption is absent.** `DeliveryStreamEncryptionConfigurationInput` is ignored, and
793
+ `Encrypted` is false on every put.
794
+ - **A failed delivery is recorded once.** Real Firehose retries for `DurationInSeconds` and then
795
+ writes the buffer under the error output prefix. `RetryOptions` is accepted and unused, and a
796
+ buffer that failed goes straight to `getDeliveryFailures()`.
797
+ - **Throughput is unlimited.** Real Firehose takes a fixed number of records and bytes a second per
798
+ delivery stream, and refuses past that with `ServiceUnavailableException`. Nothing here counts.
799
+ `FailedPutCount` on `PutRecordBatch` is always zero for the same reason. Real Firehose fails one
800
+ record of a batch for throughput limits and internal faults, and both are absent here. The
801
+ per-record response shape is still what a consumer of the response reads.
802
+ - **Tags are accepted and never listed.** `TagDeliveryStream`, `ListTagsForDeliveryStream` and
803
+ `UntagDeliveryStream` are absent. The `Tags` a template declares go the same way.
804
+ - **A delivery stream cannot be reconfigured.** `UpdateDestination` is absent. The version in an
805
+ Object key stays at `1` for the life of the delivery stream.