@kensio/yulin 1.21.9 → 1.21.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/serve/payload-2/sim-payload-2-endpoint.d.ts +9 -0
- package/dist/serve/payload-2/sim-payload-2-request-context.js +1 -1
- package/dist/service/apigatewayv2/api/sim-http-api-lambda-proxy.factory.d.ts +6 -0
- package/dist/service/apigatewayv2/api/sim-http-api-lambda-proxy.factory.js +2 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-format.d.ts +8 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-format.js +36 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-request.d.ts +53 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-request.js +1 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-settings.d.ts +26 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-settings.js +44 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-settings.type.d.ts +21 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-settings.type.js +1 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-stream-name.d.ts +13 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-stream-name.js +23 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-variables.d.ts +17 -0
- package/dist/service/apigatewayv2/api/stage/access-log/sim-http-api-access-log-variables.js +86 -0
- package/dist/service/apigatewayv2/api/stage/sim-http-api-stage.d.ts +9 -0
- package/dist/service/apigatewayv2/api/stage/sim-http-api-stage.js +9 -0
- package/dist/service/apigatewayv2/cfn/stage/sim-cfn-http-api-access-log-settings-properties.d.ts +24 -0
- package/dist/service/apigatewayv2/cfn/stage/sim-cfn-http-api-access-log-settings-properties.js +30 -0
- package/dist/service/apigatewayv2/cfn/stage/sim-cfn-http-api-stage-properties.d.ts +1 -0
- package/dist/service/apigatewayv2/cfn/stage/sim-cfn-http-api-stage-properties.js +10 -4
- package/dist/service/apigatewayv2/command/stage/sim-http-api-access-log-settings-input.d.ts +15 -0
- package/dist/service/apigatewayv2/command/stage/sim-http-api-access-log-settings-input.js +33 -0
- package/dist/service/apigatewayv2/command/stage/sim-http-api-stage-commands.js +4 -0
- package/dist/service/apigatewayv2/command/stage/stage.command.d.ts +2 -0
- package/dist/service/apigatewayv2/serve/auth/sim-http-api-route-authorize-input.d.ts +6 -0
- package/dist/service/apigatewayv2/serve/sim-api-gateway-v2-controller.d.ts +13 -15
- package/dist/service/apigatewayv2/serve/sim-api-gateway-v2-controller.js +26 -60
- package/dist/service/apigatewayv2/serve/sim-http-api-access-log-record.d.ts +31 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-access-log-record.js +88 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-access-log.d.ts +43 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-access-log.js +72 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-endpoint.js +1 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-integration-input.d.ts +22 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-integration-input.js +1 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-integration-invocation.d.ts +5 -22
- package/dist/service/apigatewayv2/serve/sim-http-api-integration-invocation.js +17 -20
- package/dist/service/apigatewayv2/serve/sim-http-api-integration-outcome.d.ts +27 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-integration-outcome.js +13 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-may-invoke.d.ts +19 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-may-invoke.js +19 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-serve-pipeline.d.ts +27 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-serve-pipeline.js +75 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-served.d.ts +22 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-served.js +1 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-serving.d.ts +12 -0
- package/dist/service/apigatewayv2/serve/sim-http-api-serving.js +1 -0
- package/dist/service/lambda/cfn/event-source-mapping/sim-cfn-lambda-event-source-mapping-properties.d.ts +1 -1
- package/dist/service/lambda/cfn/event-source-mapping/sim-cfn-lambda-event-source-mapping-properties.js +4 -3
- package/dist/service/lambda/cfn/event-source-mapping/sim-cfn-lambda-event-source-mapping-property-names.d.ts +30 -0
- package/dist/service/lambda/cfn/event-source-mapping/sim-cfn-lambda-event-source-mapping-property-names.js +91 -0
- package/dist/service/lambda/cfn/event-source-mapping/sim-cfn-lambda-event-source-mapping-property-rules.d.ts +7 -12
- package/dist/service/lambda/cfn/event-source-mapping/sim-cfn-lambda-event-source-mapping-property-rules.js +12 -92
- package/dist/service/lambda/command/event-source-mapping/create-event-source-mapping-input.js +1 -0
- package/dist/service/lambda/command/event-source-mapping/create-event-source-mapping-refusals.js +0 -5
- package/dist/service/lambda/event-source/poll/kinesis/sim-lambda-kinesis-shard-poller.js +1 -1
- package/dist/service/lambda/event-source/poll/sim-lambda-dynamodb-stream-polling.js +1 -1
- package/dist/service/lambda/event-source/poll/sim-lambda-stream-bisect.d.ts +53 -0
- package/dist/service/lambda/event-source/poll/sim-lambda-stream-bisect.js +79 -0
- package/dist/service/lambda/event-source/poll/sim-lambda-stream-progress-state.d.ts +2 -0
- package/dist/service/lambda/event-source/poll/sim-lambda-stream-progress-state.js +3 -0
- package/dist/service/lambda/event-source/poll/sim-lambda-stream-progress.d.ts +14 -1
- package/dist/service/lambda/event-source/poll/sim-lambda-stream-progress.js +24 -2
- package/dist/service/lambda/event-source/sim-lambda-event-source-retry-limits.d.ts +8 -7
- package/dist/service/lambda/event-source/sim-lambda-event-source-retry-limits.js +6 -3
- package/dist/service/lambda/event-source/sim-lambda-stream-retry-limits.d.ts +15 -7
- package/dist/service/lambda/event-source/sim-lambda-stream-retry-limits.js +15 -7
- package/dist/service/logs/group/sim-logs-arn.d.ts +15 -0
- package/dist/service/logs/group/sim-logs-arn.js +33 -0
- package/docs/services/apigatewayv2/README.md +162 -6
- package/docs/services/lambda/README.md +54 -7
- package/package.json +1 -1
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* How much of a failing batch the next delivery may carry.
|
|
3
|
+
*
|
|
4
|
+
* A mapping asked to bisect halves a batch its function threw on, and halves
|
|
5
|
+
* again on each further error, until a failing batch holds one record. That is
|
|
6
|
+
* how the record that broke the batch ends up delivered on its own, and how the
|
|
7
|
+
* records beside it get through.
|
|
8
|
+
*
|
|
9
|
+
* A split is progress rather than another attempt at the same work, so it puts
|
|
10
|
+
* the retry count back to the start. Without that the simulator's own cap of
|
|
11
|
+
* five attempts would discard a batch of a hundred long before it was down to
|
|
12
|
+
* one record. A batch halves at every step, so the splitting ends on its own,
|
|
13
|
+
* and the single record it ends on is then counted and discarded as any failing
|
|
14
|
+
* batch is.
|
|
15
|
+
*
|
|
16
|
+
* Two numbers say what the next read may take. The half is how many records a
|
|
17
|
+
* delivery may carry, and the remainder is how many records of the batch being
|
|
18
|
+
* split are still to get through. Reading the lower of the two is what keeps a
|
|
19
|
+
* record written while the splitting is going on out of it: the halves are the
|
|
20
|
+
* batch that failed, and a record that arrived afterwards belongs to the batch
|
|
21
|
+
* after it. The mapping's own batch size comes back once the whole of the split
|
|
22
|
+
* batch is through.
|
|
23
|
+
*/
|
|
24
|
+
export class SimLambdaStreamBisect {
|
|
25
|
+
enabled;
|
|
26
|
+
#half;
|
|
27
|
+
#remaining = 0;
|
|
28
|
+
constructor(enabled) {
|
|
29
|
+
this.enabled = enabled;
|
|
30
|
+
}
|
|
31
|
+
/**
|
|
32
|
+
* How many records the next read may take, within the mapping's batch size.
|
|
33
|
+
*/
|
|
34
|
+
sizeWithin(batchSize) {
|
|
35
|
+
const half = this.#half;
|
|
36
|
+
return half === undefined
|
|
37
|
+
? batchSize
|
|
38
|
+
: Math.min(half, this.#remaining, batchSize);
|
|
39
|
+
}
|
|
40
|
+
/**
|
|
41
|
+
* Split a batch the function threw on, answering with whether it split.
|
|
42
|
+
*
|
|
43
|
+
* A batch the function reported partial failures on is left alone. Lambda
|
|
44
|
+
* bisects around an error the handler raised, and a report already says which
|
|
45
|
+
* record to go back to.
|
|
46
|
+
*/
|
|
47
|
+
split(outcome) {
|
|
48
|
+
const count = outcome.records.length;
|
|
49
|
+
if (!this.enabled || !outcome.functionError || count < 2) {
|
|
50
|
+
return false;
|
|
51
|
+
}
|
|
52
|
+
this.#half = Math.ceil(count / 2);
|
|
53
|
+
this.#remaining = count;
|
|
54
|
+
return true;
|
|
55
|
+
}
|
|
56
|
+
/**
|
|
57
|
+
* Take records of the batch being split that the mapping is finished with,
|
|
58
|
+
* whether the function handled them, they were discarded, or they aged out.
|
|
59
|
+
*
|
|
60
|
+
* The whole of the split batch being through is what puts the mapping back on
|
|
61
|
+
* full batches.
|
|
62
|
+
*/
|
|
63
|
+
finished(count) {
|
|
64
|
+
if (this.#half === undefined || count <= 0) {
|
|
65
|
+
return;
|
|
66
|
+
}
|
|
67
|
+
this.#remaining -= count;
|
|
68
|
+
if (this.#remaining <= 0) {
|
|
69
|
+
this.reset();
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
/**
|
|
73
|
+
* Read whole batches again.
|
|
74
|
+
*/
|
|
75
|
+
reset() {
|
|
76
|
+
this.#half = undefined;
|
|
77
|
+
this.#remaining = 0;
|
|
78
|
+
}
|
|
79
|
+
}
|
|
@@ -2,6 +2,7 @@ import { SimLambdaStreamExpiry } from "./sim-lambda-stream-expiry.js";
|
|
|
2
2
|
import { SimLambdaStreamFailureDestination } from "./sim-lambda-stream-failure-destination.js";
|
|
3
3
|
import { SimLambdaStreamCursor } from "./sim-lambda-stream-cursor.js";
|
|
4
4
|
import { SimLambdaStreamRetry } from "./sim-lambda-stream-retry.js";
|
|
5
|
+
import { SimLambdaStreamBisect } from "./sim-lambda-stream-bisect.js";
|
|
5
6
|
import type { BackgroundScheduler, BackgroundTask } from "../../../../util/background/background.js";
|
|
6
7
|
import type { SimLambdaEventSourceMapping } from "../sim-lambda-event-source-mapping.js";
|
|
7
8
|
import type { SimLambdaEventSourceStreamProgressBatch } from "../stream/sim-lambda-event-source-streams.js";
|
|
@@ -16,6 +17,7 @@ export declare class SimLambdaStreamProgressState {
|
|
|
16
17
|
readonly failures: SimLambdaStreamFailureDestination;
|
|
17
18
|
readonly cursor: SimLambdaStreamCursor;
|
|
18
19
|
readonly retry: SimLambdaStreamRetry;
|
|
20
|
+
readonly bisect: SimLambdaStreamBisect;
|
|
19
21
|
constructor(properties: SimLambdaStreamProgressProperties);
|
|
20
22
|
handled(batch: SimLambdaEventSourceStreamProgressBatch): void;
|
|
21
23
|
}
|
|
@@ -3,6 +3,7 @@ import { SimLambdaStreamBatchAge } from "./sim-lambda-stream-batch-age.js";
|
|
|
3
3
|
import { SimLambdaStreamFailureDestination } from "./sim-lambda-stream-failure-destination.js";
|
|
4
4
|
import { SimLambdaStreamCursor } from "./sim-lambda-stream-cursor.js";
|
|
5
5
|
import { SimLambdaStreamRetry } from "./sim-lambda-stream-retry.js";
|
|
6
|
+
import { SimLambdaStreamBisect } from "./sim-lambda-stream-bisect.js";
|
|
6
7
|
import { PollSchedule } from "../../../../util/background/poll-schedule.js";
|
|
7
8
|
import { assertDefined } from "../../../../util/type-guard/defined.js";
|
|
8
9
|
/** Own the per-shard retry state and its expiry and destination collaborators. */
|
|
@@ -11,6 +12,7 @@ export class SimLambdaStreamProgressState {
|
|
|
11
12
|
failures;
|
|
12
13
|
cursor;
|
|
13
14
|
retry;
|
|
15
|
+
bisect;
|
|
14
16
|
constructor(properties) {
|
|
15
17
|
const { mapping, background } = properties;
|
|
16
18
|
this.failures = new SimLambdaStreamFailureDestination(mapping);
|
|
@@ -22,6 +24,7 @@ export class SimLambdaStreamProgressState {
|
|
|
22
24
|
assertDefined(streamRetryLimits, "stream mapping retry limits");
|
|
23
25
|
this.cursor = new SimLambdaStreamCursor(start, new PollSchedule(properties));
|
|
24
26
|
this.retry = new SimLambdaStreamRetry(streamRetryLimits, background);
|
|
27
|
+
this.bisect = new SimLambdaStreamBisect(streamRetryLimits.bisectBatchOnFunctionError);
|
|
25
28
|
this.expiry = new SimLambdaStreamExpiry({
|
|
26
29
|
age: new SimLambdaStreamBatchAge(streamRetryLimits, background),
|
|
27
30
|
cursor: this.cursor,
|
|
@@ -15,6 +15,11 @@ export declare class SimLambdaStreamProgress {
|
|
|
15
15
|
* Where the next read starts.
|
|
16
16
|
*/
|
|
17
17
|
get position(): SimLambdaEventSourceStreamPosition;
|
|
18
|
+
/**
|
|
19
|
+
* How many records the next read may take, which is the mapping's batch size
|
|
20
|
+
* until a batch is being bisected.
|
|
21
|
+
*/
|
|
22
|
+
batchSizeWithin(batchSize: number): number;
|
|
18
23
|
/**
|
|
19
24
|
* Take a read that came back with nothing.
|
|
20
25
|
*
|
|
@@ -23,9 +28,17 @@ export declare class SimLambdaStreamProgress {
|
|
|
23
28
|
* than working out a new one each time.
|
|
24
29
|
*/
|
|
25
30
|
caughtUp(batch: SimLambdaEventSourceStreamProgressBatch): void;
|
|
31
|
+
/**
|
|
32
|
+
* Drop the records at the front of a batch that are too old to hand over,
|
|
33
|
+
* answering with where the live ones start.
|
|
34
|
+
*
|
|
35
|
+
* Records that aged out are as finished with as records the function took, so
|
|
36
|
+
* a batch being split counts them off its remainder too.
|
|
37
|
+
*/
|
|
26
38
|
before(records: readonly SimLambdaStreamRecordTime[], batch: SimLambdaEventSourceStreamProgressBatch, simFunction: SimLambdaFunction): Promise<number>;
|
|
27
39
|
/**
|
|
28
|
-
* Take what became of a batch: move past it, or wait and try it
|
|
40
|
+
* Take what became of a batch: move past it, split it, or wait and try it
|
|
41
|
+
* again.
|
|
29
42
|
*/
|
|
30
43
|
after(outcome: SimLambdaStreamBatchOutcome, batch: SimLambdaEventSourceStreamProgressBatch, simFunction: SimLambdaFunction): Promise<void>;
|
|
31
44
|
/**
|
|
@@ -17,6 +17,13 @@ export class SimLambdaStreamProgress {
|
|
|
17
17
|
get position() {
|
|
18
18
|
return this.state.cursor.position;
|
|
19
19
|
}
|
|
20
|
+
/**
|
|
21
|
+
* How many records the next read may take, which is the mapping's batch size
|
|
22
|
+
* until a batch is being bisected.
|
|
23
|
+
*/
|
|
24
|
+
batchSizeWithin(batchSize) {
|
|
25
|
+
return this.state.bisect.sizeWithin(batchSize);
|
|
26
|
+
}
|
|
20
27
|
/**
|
|
21
28
|
* Take a read that came back with nothing.
|
|
22
29
|
*
|
|
@@ -27,21 +34,36 @@ export class SimLambdaStreamProgress {
|
|
|
27
34
|
caughtUp(batch) {
|
|
28
35
|
this.state.cursor.advanceTo(batch.next);
|
|
29
36
|
}
|
|
37
|
+
/**
|
|
38
|
+
* Drop the records at the front of a batch that are too old to hand over,
|
|
39
|
+
* answering with where the live ones start.
|
|
40
|
+
*
|
|
41
|
+
* Records that aged out are as finished with as records the function took, so
|
|
42
|
+
* a batch being split counts them off its remainder too.
|
|
43
|
+
*/
|
|
30
44
|
async before(records, batch, simFunction) {
|
|
31
|
-
|
|
45
|
+
const firstLive = await this.state.expiry.before(records, batch, simFunction);
|
|
46
|
+
this.state.bisect.finished(firstLive);
|
|
47
|
+
return firstLive;
|
|
32
48
|
}
|
|
33
49
|
/**
|
|
34
|
-
* Take what became of a batch: move past it, or wait and try it
|
|
50
|
+
* Take what became of a batch: move past it, split it, or wait and try it
|
|
51
|
+
* again.
|
|
35
52
|
*/
|
|
36
53
|
async after(outcome, batch, simFunction) {
|
|
37
54
|
this.state.failures.invoked(outcome);
|
|
38
55
|
if (outcome.isHandled) {
|
|
56
|
+
this.state.bisect.finished(outcome.records.length);
|
|
39
57
|
this.handled(batch);
|
|
40
58
|
return;
|
|
41
59
|
}
|
|
60
|
+
if (this.state.bisect.split(outcome)) {
|
|
61
|
+
this.state.retry.reset();
|
|
62
|
+
}
|
|
42
63
|
const again = this.state.retry.after(outcome, this.position);
|
|
43
64
|
const notification = this.state.failures.deliver(this.state.retry.discarded, batch, simFunction);
|
|
44
65
|
if (again === undefined) {
|
|
66
|
+
this.state.bisect.finished(outcome.records.length);
|
|
45
67
|
this.handled(batch);
|
|
46
68
|
}
|
|
47
69
|
else {
|
|
@@ -1,12 +1,13 @@
|
|
|
1
1
|
import { type SimLambdaEventSourceRetryLimitsInput, SimLambdaStreamRetryLimits } from "./sim-lambda-stream-retry-limits.js";
|
|
2
2
|
/**
|
|
3
|
-
* Whether the event source a mapping names
|
|
3
|
+
* Whether the event source a mapping names decides for itself what becomes of
|
|
4
|
+
* a failed batch.
|
|
4
5
|
*
|
|
5
|
-
* A stream
|
|
6
|
-
* it, so the mapping is the only thing that decides when to
|
|
7
|
-
*
|
|
8
|
-
*
|
|
9
|
-
*
|
|
6
|
+
* A stream leaves it to the mapping: a record stays on the stream whatever the
|
|
7
|
+
* function makes of it, so the mapping is the only thing that decides when to
|
|
8
|
+
* stop trying and whether to split the batch first. A queue does not, because a
|
|
9
|
+
* message the function never handles goes to the queue's own redrive policy,
|
|
10
|
+
* and real Lambda refuses a request that names any of the three for one.
|
|
10
11
|
*/
|
|
11
12
|
export interface SimLambdaEventSourceRetryLimitRules {
|
|
12
13
|
/**
|
|
@@ -23,7 +24,7 @@ export declare class SimLambdaNoRetryLimits implements SimLambdaEventSourceRetry
|
|
|
23
24
|
private readonly insteadDescription;
|
|
24
25
|
constructor(sourceDescription: string, insteadDescription: string);
|
|
25
26
|
/**
|
|
26
|
-
* Refuse
|
|
27
|
+
* Refuse all three, since this source is not the one counting.
|
|
27
28
|
*/
|
|
28
29
|
limitsIn(input: SimLambdaEventSourceRetryLimitsInput): SimLambdaStreamRetryLimits | undefined;
|
|
29
30
|
}
|
|
@@ -11,7 +11,7 @@ export class SimLambdaNoRetryLimits {
|
|
|
11
11
|
this.insteadDescription = insteadDescription;
|
|
12
12
|
}
|
|
13
13
|
/**
|
|
14
|
-
* Refuse
|
|
14
|
+
* Refuse all three, since this source is not the one counting.
|
|
15
15
|
*/
|
|
16
16
|
limitsIn(input) {
|
|
17
17
|
const named = namedLimit(input);
|
|
@@ -40,7 +40,10 @@ function namedLimit(input) {
|
|
|
40
40
|
if (input.maximumRetryAttempts !== undefined) {
|
|
41
41
|
return "MaximumRetryAttempts";
|
|
42
42
|
}
|
|
43
|
-
|
|
43
|
+
if (input.maximumRecordAgeInSeconds !== undefined) {
|
|
44
|
+
return "MaximumRecordAgeInSeconds";
|
|
45
|
+
}
|
|
46
|
+
return input.bisectBatchOnFunctionError === undefined
|
|
44
47
|
? undefined
|
|
45
|
-
: "
|
|
48
|
+
: "BisectBatchOnFunctionError";
|
|
46
49
|
}
|
|
@@ -9,6 +9,7 @@
|
|
|
9
9
|
export interface SimLambdaEventSourceRetryLimitsInput {
|
|
10
10
|
readonly maximumRetryAttempts?: number | undefined;
|
|
11
11
|
readonly maximumRecordAgeInSeconds?: number | undefined;
|
|
12
|
+
readonly bisectBatchOnFunctionError?: boolean | undefined;
|
|
12
13
|
}
|
|
13
14
|
/**
|
|
14
15
|
* The failed-batch limits a mapping reports back, as Lambda names them.
|
|
@@ -16,22 +17,29 @@ export interface SimLambdaEventSourceRetryLimitsInput {
|
|
|
16
17
|
export interface SimLambdaStreamRetryLimitsConfiguration {
|
|
17
18
|
readonly MaximumRetryAttempts: number;
|
|
18
19
|
readonly MaximumRecordAgeInSeconds: number;
|
|
20
|
+
readonly BisectBatchOnFunctionError: boolean;
|
|
19
21
|
}
|
|
20
22
|
/**
|
|
21
|
-
*
|
|
23
|
+
* What a stream mapping does with a batch its function keeps failing.
|
|
22
24
|
*
|
|
23
|
-
* The
|
|
24
|
-
* lifecycle
|
|
25
|
-
*
|
|
26
|
-
*
|
|
27
|
-
*
|
|
25
|
+
* The three settings are one thing rather than three, because they run the same
|
|
26
|
+
* lifecycle. A failing batch is split around the record that broke it where the
|
|
27
|
+
* mapping was asked to bisect, and a record leaves the mapping when it has had
|
|
28
|
+
* its retries or when it is too old to be worth another, whichever comes first.
|
|
29
|
+
* The limits are held as Lambda states them, with `-1` for no limit, so a Get
|
|
30
|
+
* or a List reports back exactly what was asked for.
|
|
28
31
|
*/
|
|
29
32
|
export declare class SimLambdaStreamRetryLimits {
|
|
30
33
|
readonly maximumRetryAttempts: number;
|
|
31
34
|
readonly maximumRecordAgeInSeconds: number;
|
|
35
|
+
/**
|
|
36
|
+
* Whether a batch the function threw on is split in half and delivered again
|
|
37
|
+
* as two batches.
|
|
38
|
+
*/
|
|
39
|
+
readonly bisectBatchOnFunctionError: boolean;
|
|
32
40
|
constructor(input?: SimLambdaEventSourceRetryLimitsInput);
|
|
33
41
|
/**
|
|
34
|
-
* The
|
|
42
|
+
* The three settings as a Get, a List or a create response reports them.
|
|
35
43
|
*/
|
|
36
44
|
configuration(): SimLambdaStreamRetryLimitsConfiguration;
|
|
37
45
|
/**
|
|
@@ -23,17 +23,23 @@ const maximumRecordAgeLimit = 604_800;
|
|
|
23
23
|
const defaultAttemptLimit = 5;
|
|
24
24
|
const millisecondsPerSecond = 1000;
|
|
25
25
|
/**
|
|
26
|
-
*
|
|
26
|
+
* What a stream mapping does with a batch its function keeps failing.
|
|
27
27
|
*
|
|
28
|
-
* The
|
|
29
|
-
* lifecycle
|
|
30
|
-
*
|
|
31
|
-
*
|
|
32
|
-
*
|
|
28
|
+
* The three settings are one thing rather than three, because they run the same
|
|
29
|
+
* lifecycle. A failing batch is split around the record that broke it where the
|
|
30
|
+
* mapping was asked to bisect, and a record leaves the mapping when it has had
|
|
31
|
+
* its retries or when it is too old to be worth another, whichever comes first.
|
|
32
|
+
* The limits are held as Lambda states them, with `-1` for no limit, so a Get
|
|
33
|
+
* or a List reports back exactly what was asked for.
|
|
33
34
|
*/
|
|
34
35
|
export class SimLambdaStreamRetryLimits {
|
|
35
36
|
maximumRetryAttempts;
|
|
36
37
|
maximumRecordAgeInSeconds;
|
|
38
|
+
/**
|
|
39
|
+
* Whether a batch the function threw on is split in half and delivered again
|
|
40
|
+
* as two batches.
|
|
41
|
+
*/
|
|
42
|
+
bisectBatchOnFunctionError;
|
|
37
43
|
constructor(input = {}) {
|
|
38
44
|
this.maximumRetryAttempts = checkedLimit({
|
|
39
45
|
value: input.maximumRetryAttempts,
|
|
@@ -47,14 +53,16 @@ export class SimLambdaStreamRetryLimits {
|
|
|
47
53
|
maximum: maximumRecordAgeLimit,
|
|
48
54
|
unit: "seconds",
|
|
49
55
|
});
|
|
56
|
+
this.bisectBatchOnFunctionError = input.bisectBatchOnFunctionError ?? false;
|
|
50
57
|
}
|
|
51
58
|
/**
|
|
52
|
-
* The
|
|
59
|
+
* The three settings as a Get, a List or a create response reports them.
|
|
53
60
|
*/
|
|
54
61
|
configuration() {
|
|
55
62
|
return {
|
|
56
63
|
MaximumRetryAttempts: this.maximumRetryAttempts,
|
|
57
64
|
MaximumRecordAgeInSeconds: this.maximumRecordAgeInSeconds,
|
|
65
|
+
BisectBatchOnFunctionError: this.bisectBatchOnFunctionError,
|
|
58
66
|
};
|
|
59
67
|
}
|
|
60
68
|
/**
|
|
@@ -33,3 +33,18 @@ export declare function simLogsAnyLogGroupArn(scope: SimAwsAccountRegionScope):
|
|
|
33
33
|
* The ARN of a log stream inside a log group.
|
|
34
34
|
*/
|
|
35
35
|
export declare function simLogsLogStreamArn(scope: SimAwsAccountRegionScope, logGroupName: string, logStreamName: string): string;
|
|
36
|
+
/**
|
|
37
|
+
* The account, region and group name a log group ARN carries, or undefined
|
|
38
|
+
* for a string that is not one.
|
|
39
|
+
*
|
|
40
|
+
* Both forms are read. `DescribeLogGroups` reports `logGroupArn` without a
|
|
41
|
+
* trailing wildcard, while the `arn` field and CDK's `logGroup.logGroupArn`
|
|
42
|
+
* both end in `:*`, and a template naming either means the same group. A log
|
|
43
|
+
* group name cannot contain a colon, so what follows `log-group:` is the name
|
|
44
|
+
* up to an optional trailing `:*`.
|
|
45
|
+
*/
|
|
46
|
+
export declare function simLogsParsedLogGroupArn(arn: string): {
|
|
47
|
+
readonly accountId: string;
|
|
48
|
+
readonly regionName: string;
|
|
49
|
+
readonly logGroupName: string;
|
|
50
|
+
} | undefined;
|
|
@@ -42,3 +42,36 @@ export function simLogsAnyLogGroupArn(scope) {
|
|
|
42
42
|
export function simLogsLogStreamArn(scope, logGroupName, logStreamName) {
|
|
43
43
|
return `${simLogsLogGroupArn(scope, logGroupName)}:log-stream:${logStreamName}`;
|
|
44
44
|
}
|
|
45
|
+
/**
|
|
46
|
+
* The account, region and group name a log group ARN carries, or undefined
|
|
47
|
+
* for a string that is not one.
|
|
48
|
+
*
|
|
49
|
+
* Both forms are read. `DescribeLogGroups` reports `logGroupArn` without a
|
|
50
|
+
* trailing wildcard, while the `arn` field and CDK's `logGroup.logGroupArn`
|
|
51
|
+
* both end in `:*`, and a template naming either means the same group. A log
|
|
52
|
+
* group name cannot contain a colon, so what follows `log-group:` is the name
|
|
53
|
+
* up to an optional trailing `:*`.
|
|
54
|
+
*/
|
|
55
|
+
export function simLogsParsedLogGroupArn(arn) {
|
|
56
|
+
const [prefix, partition, service, region, account, resourceType, ...rest] = arn.split(":");
|
|
57
|
+
if (prefix !== "arn" ||
|
|
58
|
+
service !== "logs" ||
|
|
59
|
+
resourceType !== "log-group" ||
|
|
60
|
+
!isPresent(partition) ||
|
|
61
|
+
!isPresent(region) ||
|
|
62
|
+
!isPresent(account)) {
|
|
63
|
+
return undefined;
|
|
64
|
+
}
|
|
65
|
+
const [logGroupName, wildcard, ...extra] = rest;
|
|
66
|
+
if (logGroupName === undefined ||
|
|
67
|
+
logGroupName.length === 0 ||
|
|
68
|
+
extra.length > 0 ||
|
|
69
|
+
(wildcard !== undefined && wildcard !== "*")) {
|
|
70
|
+
return undefined;
|
|
71
|
+
}
|
|
72
|
+
return { accountId: account, regionName: region, logGroupName };
|
|
73
|
+
}
|
|
74
|
+
/** Whether an ARN component was written at all. */
|
|
75
|
+
function isPresent(component) {
|
|
76
|
+
return component !== undefined && component.length > 0;
|
|
77
|
+
}
|
|
@@ -666,8 +666,146 @@ route configured that way is served unthrottled.
|
|
|
666
666
|
`GetStages` answers with the settings a stage was created with. `AWS::ApiGatewayV2::Stage` deploys
|
|
667
667
|
both properties as well (see [CloudFormation](#cloudformation)). The three members of `RouteSettings`
|
|
668
668
|
that say nothing about throttling, `DetailedMetricsEnabled`, `LoggingLevel` and `DataTraceEnabled`,
|
|
669
|
-
are refused by `CreateStage`.
|
|
670
|
-
`
|
|
669
|
+
are refused by `CreateStage`. They are execution logging, which is a different log from the one
|
|
670
|
+
[`AccessLogSettings` writes](#writing-a-stages-access-log). A template carrying one deploys, and the
|
|
671
|
+
member it named is recorded on `stack.ignoredProperties`.
|
|
672
|
+
|
|
673
|
+
## Writing a stage's access log
|
|
674
|
+
|
|
675
|
+
`AccessLogSettings` sends one line per request to a CloudWatch Logs log group. `DestinationArn` names
|
|
676
|
+
the group and `Format` is the line, with `$context` variables substituted from the request that
|
|
677
|
+
produced it.
|
|
678
|
+
|
|
679
|
+
The line is written whether or not an integration ran. A request the stage throttle refused, one a
|
|
680
|
+
Lambda authorizer denied, and one with no identity source header all reach the log group, and for
|
|
681
|
+
those the access log is the only record of the request.
|
|
682
|
+
|
|
683
|
+
```typescript sim-apigatewayv2-access-log
|
|
684
|
+
/**
|
|
685
|
+
* Recording a simulated HTTP API stage's access log, including a request the
|
|
686
|
+
* stage's throttle refused.
|
|
687
|
+
*/
|
|
688
|
+
|
|
689
|
+
import {
|
|
690
|
+
CreateApiCommand,
|
|
691
|
+
CreateIntegrationCommand,
|
|
692
|
+
CreateRouteCommand,
|
|
693
|
+
CreateStageCommand,
|
|
694
|
+
} from "@aws-sdk/client-apigatewayv2";
|
|
695
|
+
import {
|
|
696
|
+
CreateLogGroupCommand,
|
|
697
|
+
FilterLogEventsCommand,
|
|
698
|
+
} from "@aws-sdk/client-cloudwatch-logs";
|
|
699
|
+
import {
|
|
700
|
+
AddPermissionCommand,
|
|
701
|
+
CreateFunctionCommand,
|
|
702
|
+
} from "@aws-sdk/client-lambda";
|
|
703
|
+
|
|
704
|
+
import { SimAws } from "@kensio/yulin";
|
|
705
|
+
import { makeLambdaZipFileInput } from "@kensio/yulin/lambda";
|
|
706
|
+
import { serveSimAws } from "@kensio/yulin/serve";
|
|
707
|
+
|
|
708
|
+
const simAws = new SimAws();
|
|
709
|
+
const logGroupName = "/aws/vendedlogs/user-api";
|
|
710
|
+
|
|
711
|
+
await simAws.logs().createLogGroup(new CreateLogGroupCommand({ logGroupName }));
|
|
712
|
+
|
|
713
|
+
const { FunctionArn } = await simAws.lambda().createFunction(
|
|
714
|
+
new CreateFunctionCommand({
|
|
715
|
+
FunctionName: "users",
|
|
716
|
+
Role: "arn:aws:iam::111111111111:role/UsersRole",
|
|
717
|
+
Code: {
|
|
718
|
+
ZipFile: makeLambdaZipFileInput(() => ({
|
|
719
|
+
statusCode: 200,
|
|
720
|
+
headers: { "content-type": "text/plain" },
|
|
721
|
+
body: "ok",
|
|
722
|
+
})),
|
|
723
|
+
},
|
|
724
|
+
}),
|
|
725
|
+
);
|
|
726
|
+
|
|
727
|
+
const apiGateway = simAws.apiGatewayV2();
|
|
728
|
+
const { ApiId, ApiEndpoint } = await apiGateway.createApi(
|
|
729
|
+
new CreateApiCommand({ Name: "users", ProtocolType: "HTTP" }),
|
|
730
|
+
);
|
|
731
|
+
|
|
732
|
+
const { IntegrationId } = await apiGateway.createIntegration(
|
|
733
|
+
new CreateIntegrationCommand({
|
|
734
|
+
ApiId,
|
|
735
|
+
IntegrationType: "AWS_PROXY",
|
|
736
|
+
IntegrationUri: FunctionArn,
|
|
737
|
+
PayloadFormatVersion: "2.0",
|
|
738
|
+
}),
|
|
739
|
+
);
|
|
740
|
+
|
|
741
|
+
await apiGateway.createRoute(
|
|
742
|
+
new CreateRouteCommand({
|
|
743
|
+
ApiId,
|
|
744
|
+
RouteKey: "GET /user/profile",
|
|
745
|
+
Target: `integrations/${IntegrationId}`,
|
|
746
|
+
}),
|
|
747
|
+
);
|
|
748
|
+
|
|
749
|
+
await apiGateway.createStage(
|
|
750
|
+
new CreateStageCommand({
|
|
751
|
+
ApiId,
|
|
752
|
+
StageName: "$default",
|
|
753
|
+
AutoDeploy: true,
|
|
754
|
+
DefaultRouteSettings: { ThrottlingRateLimit: 1, ThrottlingBurstLimit: 1 },
|
|
755
|
+
AccessLogSettings: {
|
|
756
|
+
DestinationArn: `arn:aws:logs:us-east-1:888888888888:log-group:${logGroupName}:*`,
|
|
757
|
+
Format:
|
|
758
|
+
"$context.httpMethod $context.path $context.status " +
|
|
759
|
+
"$context.error.message",
|
|
760
|
+
},
|
|
761
|
+
}),
|
|
762
|
+
);
|
|
763
|
+
|
|
764
|
+
await simAws.lambda().addPermission(
|
|
765
|
+
new AddPermissionCommand({
|
|
766
|
+
FunctionName: "users",
|
|
767
|
+
StatementId: "api-gateway-invoke",
|
|
768
|
+
Action: "lambda:InvokeFunction",
|
|
769
|
+
Principal: "apigateway.amazonaws.com",
|
|
770
|
+
SourceArn: `arn:aws:execute-api:us-east-1:888888888888:${ApiId}/*/*`,
|
|
771
|
+
}),
|
|
772
|
+
);
|
|
773
|
+
|
|
774
|
+
const srv = await serveSimAws({ simAws });
|
|
775
|
+
simAws.clock().freeze();
|
|
776
|
+
|
|
777
|
+
const profile = srv.localUrl(`${ApiEndpoint}/user/profile`);
|
|
778
|
+
await fetch(profile);
|
|
779
|
+
await fetch(profile);
|
|
780
|
+
|
|
781
|
+
const { events } = await simAws
|
|
782
|
+
.logs()
|
|
783
|
+
.filterLogEvents(new FilterLogEventsCommand({ logGroupName }));
|
|
784
|
+
|
|
785
|
+
const lines = events ?? [];
|
|
786
|
+
|
|
787
|
+
for (const event of lines) {
|
|
788
|
+
console.log(event.message);
|
|
789
|
+
}
|
|
790
|
+
|
|
791
|
+
await srv.close();
|
|
792
|
+
```
|
|
793
|
+
|
|
794
|
+
The served request and the throttled one are both there. A variable with no value renders as a dash,
|
|
795
|
+
which is what `$context.error.message` does for the request the integration answered:
|
|
796
|
+
|
|
797
|
+
```text
|
|
798
|
+
GET /user/profile 200 -
|
|
799
|
+
GET /user/profile 429 Too Many Requests
|
|
800
|
+
```
|
|
801
|
+
|
|
802
|
+
The lines go to the log group and nowhere else. An API answering a few hundred requests in one test
|
|
803
|
+
file would otherwise bury the suite's own output, so nothing here is forwarded to the console the way
|
|
804
|
+
a [Lambda handler's output](https://yulinsim.dev/services/lambda/#capturing-handler-output) is.
|
|
805
|
+
|
|
806
|
+
A `DestinationArn` naming a log group nothing has created yet is written to anyway, and the group is
|
|
807
|
+
made by the first line. `Format` is copied through apart from its `$context` references, so a JSON
|
|
808
|
+
format string arrives as JSON.
|
|
671
809
|
|
|
672
810
|
## Protecting a route with a Cognito user pool
|
|
673
811
|
|
|
@@ -2287,7 +2425,7 @@ parts behaves differently to the template. The simulated properties are:
|
|
|
2287
2425
|
- `Integration`: `ApiId`, `IntegrationType`, `IntegrationUri`, `PayloadFormatVersion`, `Description`
|
|
2288
2426
|
- `Route`: `ApiId`, `RouteKey`, `Target`, `AuthorizationType`, `AuthorizerId`, `AuthorizationScopes`
|
|
2289
2427
|
- `Stage`: `ApiId`, `StageName`, `AutoDeploy`, `StageVariables`, `Description`,
|
|
2290
|
-
`DefaultRouteSettings`, `RouteSettings`
|
|
2428
|
+
`DefaultRouteSettings`, `RouteSettings`, `AccessLogSettings`
|
|
2291
2429
|
|
|
2292
2430
|
CDK's `HttpIamAuthorizer` deploys too. It emits no `AWS::ApiGatewayV2::Authorizer` and no
|
|
2293
2431
|
`AuthorizerId`, only `AuthorizationType: "AWS_IAM"` on the `Route`, and the deployed route then
|
|
@@ -2436,6 +2574,9 @@ the simulation without being given one. See the
|
|
|
2436
2574
|
under their own path segment, including stage variables
|
|
2437
2575
|
- `DefaultRouteSettings` and `RouteSettings` throttling, with a token bucket per route refilling
|
|
2438
2576
|
against the simulated clock and a 429 for the requests past it
|
|
2577
|
+
- `AccessLogSettings`, writing one line per request to the log group `DestinationArn` names, with the
|
|
2578
|
+
`Format` string's `$context` variables substituted, including for a request the throttle or an
|
|
2579
|
+
authorizer refused before any integration ran
|
|
2439
2580
|
- Serving the generated endpoint through `serveSimAws`, invoking the integrated function with a
|
|
2440
2581
|
payload format 2.0 event and turning its result back into an HTTP response
|
|
2441
2582
|
- The invoke permission of an integration's function and of an authorizer's, each evaluated against
|
|
@@ -2474,9 +2615,24 @@ Current documented limitations:
|
|
|
2474
2615
|
- Deployments are outside the simulation, so `CreateStage` requires `AutoDeploy: true`. A stage
|
|
2475
2616
|
without it serves whichever Deployment it was given, which on real AWS is nothing until one is
|
|
2476
2617
|
created.
|
|
2477
|
-
- `
|
|
2478
|
-
|
|
2479
|
-
|
|
2618
|
+
- Any option `CreateStage` takes and this one lacks is refused. `RouteSettings` and
|
|
2619
|
+
`DefaultRouteSettings` are taken, and only their throttling members are read.
|
|
2620
|
+
`DetailedMetricsEnabled`, `LoggingLevel` and `DataTraceEnabled` are refused by name. Those three
|
|
2621
|
+
are execution logging, which is a different log from the access log and is not simulated.
|
|
2622
|
+
- An access log `DestinationArn` has to name a CloudWatch Logs log group. A Kinesis Data Firehose
|
|
2623
|
+
delivery stream, which a REST API stage may name, is refused.
|
|
2624
|
+
- The access log variables carrying a value are `accountId`, `apiId`, `domainName`, `domainPrefix`,
|
|
2625
|
+
`stage`, `routeKey`, `httpMethod`, `path`, `protocol`, `requestId`, `extendedRequestId`,
|
|
2626
|
+
`requestTime`, `requestTimeEpoch`, `status`, `responseLength`, `responseLatency`,
|
|
2627
|
+
`identity.sourceIp`, `identity.userAgent`, `integrationStatus`, `integrationLatency`,
|
|
2628
|
+
`integrationErrorMessage`, `authorizer.error`, `error.message`, `error.messageString`,
|
|
2629
|
+
`customDomain.basePathMatched`, the `integration.*` aliases, and `authorizer.claims.<property>` and
|
|
2630
|
+
`authorizer.<property>` from the route's authorizer. Anything else AWS documents renders as a dash.
|
|
2631
|
+
- `responseLatency` and `integrationLatency` are simulated milliseconds, so a test holding the clock
|
|
2632
|
+
still logs zero rather than however long the process took.
|
|
2633
|
+
- The access log stream is named for the hour it was written in, dated from the simulation's clock.
|
|
2634
|
+
Real API Gateway appends an identifier this simulation has no counterpart for, so a test should
|
|
2635
|
+
filter the log group rather than name the stream.
|
|
2480
2636
|
- `AWS_PROXY` is the only integration type, and its URI must name a Lambda function ARN, written
|
|
2481
2637
|
either as that ARN or as the
|
|
2482
2638
|
`arn:aws:apigateway:<region>:lambda:path/2015-03-31/functions/<function-arn>/invocations` form.
|