@afora/dynamodb 0.0.0-stage → 0.0.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +164 -2
- package/dist/batch.d.ts +31 -0
- package/dist/encoding.d.ts +32 -0
- package/dist/errors.d.ts +18 -0
- package/dist/exceptions.d.ts +17 -0
- package/dist/index.d.ts +9 -0
- package/dist/index.js +619 -0
- package/dist/provider.d.ts +54 -0
- package/package.json +58 -5
package/README.md
CHANGED
|
@@ -1,3 +1,165 @@
|
|
|
1
|
-
#
|
|
1
|
+
# @afora/dynamodb
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
A repository provider for [`@afora/core`](../../README.md) on one [DynamoDB](https://aws.amazon.com/dynamodb/) table. It implements the provider v2 contract (queries with `where` equality and `in`, `orderBy`, `limit` / `offset`, `withDeleted`, `count`, unique keys) plus reads by cursor, expected versions, atomic increments and records that expire, and passes the `providerContract` kit of `@afora/core/testing` against DynamoDB Local. The contract and the kit are described in [docs/repositories.md](../../docs/repositories.md) and [Testing a provider](../../docs/repositories.md#testing-a-provider).
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
bun add @afora/core typebox @afora/dynamodb @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
`@afora/core` (with the query helpers of `@afora/core/experimental`), `@aws-sdk/client-dynamodb` and `@aws-sdk/lib-dynamodb` (`^3.1164.0`) are peer dependencies.
|
|
10
|
+
|
|
11
|
+
## Example
|
|
12
|
+
|
|
13
|
+
```ts
|
|
14
|
+
import {
|
|
15
|
+
applicationOf,
|
|
16
|
+
blueprintOf,
|
|
17
|
+
entityOf,
|
|
18
|
+
environmentOf,
|
|
19
|
+
providerFrom,
|
|
20
|
+
repositoryOf,
|
|
21
|
+
} from "@afora/core";
|
|
22
|
+
import { dynamoDbProvider } from "@afora/dynamodb";
|
|
23
|
+
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
|
|
24
|
+
|
|
25
|
+
class Entry extends entityOf(
|
|
26
|
+
blueprintOf((ctx) => ({ slug: ctx.string, title: ctx.string })),
|
|
27
|
+
{ name: "Entry", unique: ["slug"] },
|
|
28
|
+
) {}
|
|
29
|
+
|
|
30
|
+
const Entries = repositoryOf({
|
|
31
|
+
entity: Entry,
|
|
32
|
+
actions: ["create", "findBySlug", "findCursor", "update", "delete"],
|
|
33
|
+
});
|
|
34
|
+
|
|
35
|
+
const Env = environmentOf(blueprintOf((ctx) => ({ TABLE_NAME: ctx.string })));
|
|
36
|
+
|
|
37
|
+
export const App = applicationOf({
|
|
38
|
+
name: "cms",
|
|
39
|
+
environments: { Env },
|
|
40
|
+
repositories: { Entries },
|
|
41
|
+
provider: providerFrom(Env, (env) =>
|
|
42
|
+
dynamoDbProvider({ table: env.TABLE_NAME, client: new DynamoDBClient({}) }),
|
|
43
|
+
),
|
|
44
|
+
});
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
`dynamoDbProvider({ table, client, logger?, retry? })`:
|
|
48
|
+
|
|
49
|
+
- `table`: the name of the table (below).
|
|
50
|
+
- `client`: a `DynamoDBClient`, or a `DynamoDBDocumentClient` made from one. A plain client is wrapped with `removeUndefinedValues`; the provider also leaves `undefined` out itself, so a document client of your own works as it is.
|
|
51
|
+
- `logger`: an object with `debug` and `warn` (an application's `ILogger` fits). Retries are told as `debug` `dynamodb.retry`, errors that are not afora's own as `warn` `dynamodb.error` (`{ operation, repository, error, message }`); the error is still thrown. Nothing is logged without it.
|
|
52
|
+
- `retry`: `{ attempts, baseDelayMs }`, 10 and 25 ms by default: how often a request DynamoDB refuses for now is sent again, with an exponential backoff (full jitter, capped at 2 s).
|
|
53
|
+
|
|
54
|
+
## Single-table layout
|
|
55
|
+
|
|
56
|
+
One table, with two string keys, `pk` (partition) and `sk` (sort), holds every repository:
|
|
57
|
+
|
|
58
|
+
| Item | `pk` | `sk` | Attributes |
|
|
59
|
+
| ------ | --------------------------- | ------------------ | -------------------------------------- |
|
|
60
|
+
| Entity | `E#<repository name>` | the id | `data` (the stored value), `m` |
|
|
61
|
+
| Marker | `U#<repository name>#<key>` | the value, encoded | `id` (of the entity holding the value) |
|
|
62
|
+
|
|
63
|
+
- **Keyed by the repository name** (`context.name`, `EntryRepository`), which comes from the `name` option of `entityOf` and so stays the same under minification and across Lambda containers: declare it. Two repositories never share items.
|
|
64
|
+
- **Unique keys other than `id`** are enforced with marker items, written in the same `TransactWriteItems` as the entity with `attribute_not_exists(pk)`. A failed marker condition is a `RepositoryUniqueViolation` on that key, a failed entity condition on insert one on `id`. An `update` moves the markers of the values that changed, and `remove` deletes them.
|
|
65
|
+
- **Encoded values**: the sort key of a marker is the JSON of the value with object keys sorted, so `1` (`1`) and `"1"` (`"1"`) are different values, and `{ a, b }` and `{ b, a }` the same one, as the contract compares them. A marker key holds up to 1024 bytes: a longer unique value throws `DynamoDbLimitExceeded`.
|
|
66
|
+
- **`m`**, on the entity, lists the markers it holds. `update` and `remove` condition on it, so they move the markers of the values the item still holds; when another write came first, they read again and retry.
|
|
67
|
+
- `update` of an unknown id and a failed existence condition are `RepositoryEntityNotFound`.
|
|
68
|
+
|
|
69
|
+
### Reads
|
|
70
|
+
|
|
71
|
+
A read gets its candidates the cheapest way: by `id` (`BatchGetItem`), through the markers of a unique key in `where`, or by querying the partition of the repository (`Query` on `pk`, every page). Then `where`, `withDeleted`, expiry, `orderBy`, `after`, `offset` and `limit` apply in memory with the same rules as `inMemoryProvider` (`selectRows` of `@afora/core/experimental`): numbers numerically, strings by code point, dates as dates, missing values first when ascending. Reads are strongly consistent. `BatchGetItem` keys left unprocessed are asked for again with backoff, up to `retry.attempts` times (then `DynamoDbRetriesExhausted`).
|
|
72
|
+
|
|
73
|
+
One read pages in DynamoDB instead: `findMany` with a `limit`, ordered by `id` ascending (or not ordered, without `after`), with no `offset` and no `where` on `id` or a unique key, which is how the backfill of `app.migrate` and a cursor by `id` read. It queries the partition from the `after` id (`sk > :after`) with `Limit`, applies `withDeleted`, expiry and `where` in memory, and queries on (`ExclusiveStartKey`) until `limit` rows match or the partition ends, so a walk reads each item once instead of the whole partition per page.
|
|
74
|
+
|
|
75
|
+
This suits repositories of thousands of items, not millions: every other read without an id or a unique key reads the whole partition. Declared `indexes` are accepted and ignored for now (`capabilities.indexes` is unset); they will become GSIs.
|
|
76
|
+
|
|
77
|
+
### Capabilities
|
|
78
|
+
|
|
79
|
+
- `keyset`: `findCursor` works (`after` is applied in memory).
|
|
80
|
+
- `versioning`: `update` with an expected version conditions the `Put` on `data.version`; a stale one is `RepositoryConcurrencyConflict`.
|
|
81
|
+
- `atomic`: `increment` is one `UpdateItem` (`SET data.views = if_not_exists(data.views, 0) + :d`), conditioned on the item existing, not soft-deleted and not expired. With `create`, a missing id is inserted (a concurrent insert of the same id gets the deltas added instead), an expired item is replaced, and a soft-deleted one is `RepositoryUniqueViolation` on `id`. An increment cannot change a unique key (`InvalidRepositoryQuery`).
|
|
82
|
+
- `expiry`: reads leave out records whose date is at or before `query.now`, unless `withExpired`. Expired items stay in the table and keep their unique values; the `UpdateItem` of `increment` compares the dates as strings, so keep them in UTC ISO 8601 (`toISOString()`, as the entity writes them).
|
|
83
|
+
- `transactions` is **not** a capability: DynamoDB writes up to 100 items at once, without reads in between, so `commandOf({ transaction: true })` cannot run on it.
|
|
84
|
+
|
|
85
|
+
## Limits
|
|
86
|
+
|
|
87
|
+
- **100 items per write**: an entity and the markers of its unique keys are written in one transaction, so an entity with more than 99 present unique values throws `DynamoDbLimitExceeded` (it is never split, since the parts would not be atomic).
|
|
88
|
+
- **400 KB per item**, DynamoDB's own limit, for the stored value.
|
|
89
|
+
- **1024 bytes per sort key**: an id or an encoded unique value.
|
|
90
|
+
|
|
91
|
+
## Schema evolution
|
|
92
|
+
|
|
93
|
+
DynamoDB has no schema to change, so `migrate` is a no-op, and a change of an entity relies on the upgrade on read and the backfill of `app.migrate` (see [Schema evolution, issue #28](https://github.com/afora-framework/core/issues/28)):
|
|
94
|
+
|
|
95
|
+
- The schema version key (`$v`, `context.schema.key`) and any other key of the stored value are kept as they are in `data`, so rows round-trip it.
|
|
96
|
+
- An entity renamed with `previousNames: ["Post"]` reads its old rows from the partition of the old repository name (`E#PostRepository`), since every item is keyed by `context.name`; the backfill writes them under the new name.
|
|
97
|
+
|
|
98
|
+
## The table
|
|
99
|
+
|
|
100
|
+
`PAY_PER_REQUEST`, string `pk` and `sk`, nothing else. With the AWS CDK:
|
|
101
|
+
|
|
102
|
+
<!-- docs:code skip: the AWS CDK is not a dependency of this repo; this is the body of a Stack constructor, and handler is your function -->
|
|
103
|
+
```ts
|
|
104
|
+
import { RemovalPolicy } from "aws-cdk-lib";
|
|
105
|
+
import { AttributeType, Billing, TableV2 } from "aws-cdk-lib/aws-dynamodb";
|
|
106
|
+
|
|
107
|
+
const table = new TableV2(this, "Table", {
|
|
108
|
+
partitionKey: { name: "pk", type: AttributeType.STRING },
|
|
109
|
+
sortKey: { name: "sk", type: AttributeType.STRING },
|
|
110
|
+
billing: Billing.onDemand(),
|
|
111
|
+
pointInTimeRecoverySpecification: { pointInTimeRecoveryEnabled: true },
|
|
112
|
+
removalPolicy: RemovalPolicy.RETAIN,
|
|
113
|
+
});
|
|
114
|
+
table.grantReadWriteData(handler); // GetItem, BatchGetItem, Query, PutItem, UpdateItem, DeleteItem, TransactWriteItems
|
|
115
|
+
handler.addEnvironment("TABLE_NAME", table.tableName);
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
With CloudFormation:
|
|
119
|
+
|
|
120
|
+
```yaml
|
|
121
|
+
Resources:
|
|
122
|
+
Table:
|
|
123
|
+
Type: AWS::DynamoDB::Table
|
|
124
|
+
DeletionPolicy: Retain
|
|
125
|
+
UpdateReplacePolicy: Retain
|
|
126
|
+
Properties:
|
|
127
|
+
BillingMode: PAY_PER_REQUEST
|
|
128
|
+
AttributeDefinitions:
|
|
129
|
+
- { AttributeName: pk, AttributeType: S }
|
|
130
|
+
- { AttributeName: sk, AttributeType: S }
|
|
131
|
+
KeySchema:
|
|
132
|
+
- { AttributeName: pk, KeyType: HASH }
|
|
133
|
+
- { AttributeName: sk, KeyType: RANGE }
|
|
134
|
+
PointInTimeRecoverySpecification:
|
|
135
|
+
PointInTimeRecoveryEnabled: true
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
The provider needs `dynamodb:GetItem`, `BatchGetItem`, `Query`, `PutItem`, `UpdateItem`, `DeleteItem` and `TransactWriteItems` on the table.
|
|
139
|
+
|
|
140
|
+
## Codegen and extraction
|
|
141
|
+
|
|
142
|
+
The package exports the catalog entry for `@afora/codegen`:
|
|
143
|
+
|
|
144
|
+
```ts
|
|
145
|
+
export const providers = [
|
|
146
|
+
{ name: "dynamodb", module: "@afora/dynamodb", export: "dynamoDbProvider" },
|
|
147
|
+
];
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
`client` is not JSON, so `dynamoDbProvider` is an opaque provider: give it to codegen as an import (`{ "import": "./provider.ts#provider" }`), and to `extractSpec` as an import mapping.
|
|
151
|
+
|
|
152
|
+
## Tests
|
|
153
|
+
|
|
154
|
+
`bun run test:dynamodb` runs `providerContract` against DynamoDB Local, with a fresh table per case, plus repository-level tests and unit tests of the encoding, the error mapping and the retries. Without `DYNAMODB_ENDPOINT`, the tests that need DynamoDB are skipped with a message:
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
docker run --rm -p 8000:8000 amazon/dynamodb-local
|
|
158
|
+
DYNAMODB_ENDPOINT=http://localhost:8000 bun run test:dynamodb
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
## Later
|
|
162
|
+
|
|
163
|
+
- The declared `indexes` as GSIs (`capabilities.indexes`), so reads by them stop scanning the partition.
|
|
164
|
+
- The expiry of records as DynamoDB TTL (a number attribute in seconds), so expired items are purged.
|
|
165
|
+
- Paging the partition query by `limit` for orders other than `id` (it needs a GSI on the order key).
|
package/dist/batch.d.ts
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
/** A key of the table. */
|
|
2
|
+
export type ItemKey = {
|
|
3
|
+
readonly pk: string;
|
|
4
|
+
readonly sk: string;
|
|
5
|
+
};
|
|
6
|
+
/** What `BatchGetItem` answers, as the document client gives it. */
|
|
7
|
+
export interface BatchGetPage {
|
|
8
|
+
readonly Responses?: Readonly<Record<string, Record<string, unknown>[]>>;
|
|
9
|
+
readonly UnprocessedKeys?: Readonly<Record<string, {
|
|
10
|
+
readonly Keys?: Record<string, unknown>[];
|
|
11
|
+
}>>;
|
|
12
|
+
}
|
|
13
|
+
export interface BatchGetOptions {
|
|
14
|
+
readonly table: string;
|
|
15
|
+
/** Sends one `BatchGetItem` of at most 100 keys of `table`. */
|
|
16
|
+
readonly send: (keys: readonly ItemKey[]) => Promise<BatchGetPage>;
|
|
17
|
+
/** The attempts of each chunk, the first included. */
|
|
18
|
+
readonly attempts: number;
|
|
19
|
+
readonly baseDelayMs: number;
|
|
20
|
+
readonly sleep?: (ms: number) => Promise<void>;
|
|
21
|
+
/** Called before each retry, with the keys left. */
|
|
22
|
+
readonly onRetry?: (attempt: number, left: number) => void;
|
|
23
|
+
/** Called when the attempts run out; must throw. */
|
|
24
|
+
readonly onExhausted: (attempts: number) => never;
|
|
25
|
+
}
|
|
26
|
+
/**
|
|
27
|
+
* Every item of `keys` that exists, in chunks of 100, asking again for the
|
|
28
|
+
* keys DynamoDB leaves unprocessed (throttling, the 16 MB limit) with an
|
|
29
|
+
* exponential backoff, up to `attempts` times per chunk.
|
|
30
|
+
*/
|
|
31
|
+
export declare function batchGetAll(keys: readonly ItemKey[], options: BatchGetOptions): Promise<Record<string, unknown>[]>;
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The keys of the single-table layout and the encoding of unique values.
|
|
3
|
+
*
|
|
4
|
+
* - entity: `pk = E#<context.name>`, `sk = <id>`
|
|
5
|
+
* - marker: `pk = U#<context.name>#<key>`, `sk = markerOf(value)`
|
|
6
|
+
*/
|
|
7
|
+
/** The partition of the entities of a repository. */
|
|
8
|
+
export declare const entityPk: (name: string) => string;
|
|
9
|
+
/** The partition of the markers of one unique key of a repository. */
|
|
10
|
+
export declare const uniquePk: (name: string, key: string) => string;
|
|
11
|
+
/** The largest sort key DynamoDB takes, in bytes of UTF-8. */
|
|
12
|
+
export declare const MAX_SORT_KEY_BYTES = 1024;
|
|
13
|
+
export declare const isPresent: (value: unknown) => boolean;
|
|
14
|
+
/**
|
|
15
|
+
* The sort key of the marker of a unique value: its JSON with object keys
|
|
16
|
+
* sorted, so `1` (`1`) and `"1"` (`"1"`) are apart, and `{ a, b }` and
|
|
17
|
+
* `{ b, a }` are the same value.
|
|
18
|
+
*/
|
|
19
|
+
export declare function markerOf(value: unknown): string;
|
|
20
|
+
/**
|
|
21
|
+
* The markers an entity holds, as one string stored on its item (`m`): the
|
|
22
|
+
* encoded value of each present unique key but `id`. A write conditions on
|
|
23
|
+
* it, so markers are moved from the values the item still holds.
|
|
24
|
+
*/
|
|
25
|
+
export declare function markersOf(data: Readonly<Record<string, unknown>>, keys: readonly string[]): string;
|
|
26
|
+
/**
|
|
27
|
+
* A copy of `value` DynamoDB can store: `undefined` left out of objects (and
|
|
28
|
+
* `null` in arrays, as JSON), dates as ISO 8601 strings.
|
|
29
|
+
*/
|
|
30
|
+
export declare function storable(value: unknown): unknown;
|
|
31
|
+
/** The size in bytes of `text` as UTF-8. */
|
|
32
|
+
export declare const byteLength: (text: string) => number;
|
package/dist/errors.d.ts
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* What a failed write of DynamoDB means for the provider, read from the error
|
|
3
|
+
* by name (not `instanceof`, which two copies of the SDK would break):
|
|
4
|
+
*
|
|
5
|
+
* - `condition`: the condition of item `index` failed (a single `Put`,
|
|
6
|
+
* `Update` or `Delete` is item 0);
|
|
7
|
+
* - `conflict`: another transaction held an item, worth retrying;
|
|
8
|
+
* - `undefined`: anything else, rethrown as it is.
|
|
9
|
+
*/
|
|
10
|
+
export type WriteFailure = {
|
|
11
|
+
readonly kind: "condition";
|
|
12
|
+
readonly index: number;
|
|
13
|
+
} | {
|
|
14
|
+
readonly kind: "conflict";
|
|
15
|
+
};
|
|
16
|
+
export declare function writeFailureOf(error: unknown): WriteFailure | undefined;
|
|
17
|
+
/** The delay before attempt `attempt` (1 for the first retry): exponential, capped, with full jitter. */
|
|
18
|
+
export declare function backoffOf(attempt: number, baseMs: number, random?: () => number): number;
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* A write or a value goes past a limit of DynamoDB: a transaction of more
|
|
3
|
+
* than 100 items, or a unique value whose marker key passes 1024 bytes.
|
|
4
|
+
* `source` is the repository name.
|
|
5
|
+
*/
|
|
6
|
+
export declare const DynamoDbLimitExceeded: import("@afora/core").DeclaredExceptionClass<{
|
|
7
|
+
readonly reason: string;
|
|
8
|
+
}>;
|
|
9
|
+
/**
|
|
10
|
+
* DynamoDB kept refusing a request after every attempt: keys left
|
|
11
|
+
* unprocessed by `BatchGetItem` (throttling), or a transaction cancelled by
|
|
12
|
+
* conflicting ones. `source` is the repository name.
|
|
13
|
+
*/
|
|
14
|
+
export declare const DynamoDbRetriesExhausted: import("@afora/core").DeclaredExceptionClass<{
|
|
15
|
+
readonly operation: string;
|
|
16
|
+
readonly attempts: number;
|
|
17
|
+
}>;
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
export { DynamoDbLimitExceeded, DynamoDbRetriesExhausted } from "./exceptions.js";
|
|
2
|
+
export type { DynamoDbProvider, DynamoDbProviderOptions } from "./provider.js";
|
|
3
|
+
export { dynamoDbProvider } from "./provider.js";
|
|
4
|
+
/** The provider catalog entries for `afora-codegen --providers`. */
|
|
5
|
+
export declare const providers: readonly [{
|
|
6
|
+
readonly name: "dynamodb";
|
|
7
|
+
readonly module: "@afora/dynamodb";
|
|
8
|
+
readonly export: "dynamoDbProvider";
|
|
9
|
+
}];
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,619 @@
|
|
|
1
|
+
// src/exceptions.ts
|
|
2
|
+
import { ExceptionLayers, exceptionOf } from "@afora/core";
|
|
3
|
+
var DynamoDbLimitExceeded = exceptionOf({
|
|
4
|
+
layer: ExceptionLayers.Repository,
|
|
5
|
+
name: "DynamoDbLimitExceeded",
|
|
6
|
+
message: ({ reason }) => reason
|
|
7
|
+
});
|
|
8
|
+
var DynamoDbRetriesExhausted = exceptionOf({
|
|
9
|
+
layer: ExceptionLayers.Repository,
|
|
10
|
+
name: "DynamoDbRetriesExhausted",
|
|
11
|
+
message: ({
|
|
12
|
+
operation,
|
|
13
|
+
attempts
|
|
14
|
+
}) => `DynamoDB ${operation} did not complete after ${attempts} attempts.`
|
|
15
|
+
});
|
|
16
|
+
// src/provider.ts
|
|
17
|
+
import {
|
|
18
|
+
Exception,
|
|
19
|
+
InvalidRepositoryDeclaration,
|
|
20
|
+
InvalidRepositoryQuery,
|
|
21
|
+
RepositoryConcurrencyConflict,
|
|
22
|
+
RepositoryEntityNotFound,
|
|
23
|
+
RepositoryUniqueViolation
|
|
24
|
+
} from "@afora/core";
|
|
25
|
+
import { matchesQuery, selectRows } from "@afora/core/experimental";
|
|
26
|
+
import {
|
|
27
|
+
BatchGetCommand,
|
|
28
|
+
DeleteCommand,
|
|
29
|
+
DynamoDBDocumentClient,
|
|
30
|
+
GetCommand,
|
|
31
|
+
PutCommand,
|
|
32
|
+
QueryCommand,
|
|
33
|
+
TransactWriteCommand,
|
|
34
|
+
UpdateCommand
|
|
35
|
+
} from "@aws-sdk/lib-dynamodb";
|
|
36
|
+
|
|
37
|
+
// src/errors.ts
|
|
38
|
+
function writeFailureOf(error) {
|
|
39
|
+
if (typeof error !== "object" || error === null)
|
|
40
|
+
return;
|
|
41
|
+
const { name, CancellationReasons } = error;
|
|
42
|
+
if (name === "ConditionalCheckFailedException")
|
|
43
|
+
return { kind: "condition", index: 0 };
|
|
44
|
+
if (name === "TransactionConflictException")
|
|
45
|
+
return { kind: "conflict" };
|
|
46
|
+
if (name !== "TransactionCanceledException")
|
|
47
|
+
return;
|
|
48
|
+
const reasons = CancellationReasons ?? [];
|
|
49
|
+
const index = reasons.findIndex((reason) => reason.Code === "ConditionalCheckFailed");
|
|
50
|
+
if (index >= 0)
|
|
51
|
+
return { kind: "condition", index };
|
|
52
|
+
if (reasons.some((reason) => reason.Code === "TransactionConflict"))
|
|
53
|
+
return { kind: "conflict" };
|
|
54
|
+
return;
|
|
55
|
+
}
|
|
56
|
+
function backoffOf(attempt, baseMs, random = Math.random) {
|
|
57
|
+
const ceiling = Math.min(baseMs * 2 ** (attempt - 1), 2000);
|
|
58
|
+
return Math.round(ceiling / 2 + random() * ceiling / 2);
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
// src/batch.ts
|
|
62
|
+
var BATCH_GET_LIMIT = 100;
|
|
63
|
+
var wait = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
64
|
+
async function batchGetAll(keys, options) {
|
|
65
|
+
const { table, send, attempts, baseDelayMs, sleep = wait } = options;
|
|
66
|
+
const found = [];
|
|
67
|
+
for (let start = 0;start < keys.length; start += BATCH_GET_LIMIT) {
|
|
68
|
+
let pending = keys.slice(start, start + BATCH_GET_LIMIT);
|
|
69
|
+
for (let attempt = 1;pending.length > 0; attempt++) {
|
|
70
|
+
if (attempt > attempts)
|
|
71
|
+
options.onExhausted(attempts);
|
|
72
|
+
if (attempt > 1) {
|
|
73
|
+
options.onRetry?.(attempt, pending.length);
|
|
74
|
+
await sleep(backoffOf(attempt - 1, baseDelayMs));
|
|
75
|
+
}
|
|
76
|
+
const page = await send(pending);
|
|
77
|
+
found.push(...page.Responses?.[table] ?? []);
|
|
78
|
+
pending = page.UnprocessedKeys?.[table]?.Keys ?? [];
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
return found;
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
// src/encoding.ts
|
|
85
|
+
var entityPk = (name) => `E#${name}`;
|
|
86
|
+
var uniquePk = (name, key) => `U#${name}#${key}`;
|
|
87
|
+
var MAX_SORT_KEY_BYTES = 1024;
|
|
88
|
+
var isPresent = (value) => value !== undefined && value !== null;
|
|
89
|
+
function canonical(value) {
|
|
90
|
+
if (value instanceof Date)
|
|
91
|
+
return value.toISOString();
|
|
92
|
+
if (Array.isArray(value))
|
|
93
|
+
return value.map((item) => item === undefined ? null : canonical(item));
|
|
94
|
+
if (typeof value === "object" && value !== null) {
|
|
95
|
+
const sorted = {};
|
|
96
|
+
for (const key of Object.keys(value).sort()) {
|
|
97
|
+
const item = value[key];
|
|
98
|
+
if (item !== undefined)
|
|
99
|
+
sorted[key] = canonical(item);
|
|
100
|
+
}
|
|
101
|
+
return sorted;
|
|
102
|
+
}
|
|
103
|
+
return value;
|
|
104
|
+
}
|
|
105
|
+
function markerOf(value) {
|
|
106
|
+
return JSON.stringify(canonical(value)) ?? "null";
|
|
107
|
+
}
|
|
108
|
+
function markersOf(data, keys) {
|
|
109
|
+
const markers = {};
|
|
110
|
+
for (const key of keys) {
|
|
111
|
+
if (isPresent(data[key]))
|
|
112
|
+
markers[key] = markerOf(data[key]);
|
|
113
|
+
}
|
|
114
|
+
return JSON.stringify(markers);
|
|
115
|
+
}
|
|
116
|
+
function storable(value) {
|
|
117
|
+
if (value instanceof Date)
|
|
118
|
+
return value.toISOString();
|
|
119
|
+
if (Array.isArray(value))
|
|
120
|
+
return value.map((item) => item === undefined ? null : storable(item));
|
|
121
|
+
if (typeof value === "object" && value !== null) {
|
|
122
|
+
const copy = {};
|
|
123
|
+
for (const [key, item] of Object.entries(value)) {
|
|
124
|
+
if (item !== undefined)
|
|
125
|
+
copy[key] = storable(item);
|
|
126
|
+
}
|
|
127
|
+
return copy;
|
|
128
|
+
}
|
|
129
|
+
return value;
|
|
130
|
+
}
|
|
131
|
+
var byteLength = (text) => new TextEncoder().encode(text).length;
|
|
132
|
+
|
|
133
|
+
// src/provider.ts
|
|
134
|
+
var TRANSACTION_LIMIT = 100;
|
|
135
|
+
var SOURCE = "dynamodb";
|
|
136
|
+
function invalid(reason) {
|
|
137
|
+
throw new InvalidRepositoryDeclaration({
|
|
138
|
+
source: SOURCE,
|
|
139
|
+
details: { reason }
|
|
140
|
+
});
|
|
141
|
+
}
|
|
142
|
+
function isMembership(condition) {
|
|
143
|
+
return typeof condition === "object" && condition !== null && !Array.isArray(condition) && Object.keys(condition).length === 1 && Array.isArray(condition.in);
|
|
144
|
+
}
|
|
145
|
+
var expiryKeyOf = (context) => context.expiry?.key;
|
|
146
|
+
var markedKeys = (context) => context.unique.filter((key) => key !== "id");
|
|
147
|
+
var NEW_ITEM = { ConditionExpression: "attribute_not_exists(pk)" };
|
|
148
|
+
function unchanged(m, equal = []) {
|
|
149
|
+
const names = { "#m": "m" };
|
|
150
|
+
const values = {};
|
|
151
|
+
const parts = [
|
|
152
|
+
"attribute_exists(pk)",
|
|
153
|
+
m === undefined ? "attribute_not_exists(#m)" : "#m = :m"
|
|
154
|
+
];
|
|
155
|
+
if (m !== undefined)
|
|
156
|
+
values[":m"] = m;
|
|
157
|
+
equal.forEach(([key, value], index) => {
|
|
158
|
+
names["#data"] = "data";
|
|
159
|
+
names[`#e${index}`] = key;
|
|
160
|
+
values[`:e${index}`] = value;
|
|
161
|
+
parts.push(`#data.#e${index} = :e${index}`);
|
|
162
|
+
});
|
|
163
|
+
return {
|
|
164
|
+
ConditionExpression: parts.join(" AND "),
|
|
165
|
+
ExpressionAttributeNames: names,
|
|
166
|
+
...Object.keys(values).length > 0 ? { ExpressionAttributeValues: values } : {}
|
|
167
|
+
};
|
|
168
|
+
}
|
|
169
|
+
function documentClientOf(client) {
|
|
170
|
+
if (client instanceof DynamoDBDocumentClient)
|
|
171
|
+
return client;
|
|
172
|
+
return DynamoDBDocumentClient.from(client, {
|
|
173
|
+
marshallOptions: { removeUndefinedValues: true }
|
|
174
|
+
});
|
|
175
|
+
}
|
|
176
|
+
function dynamoDbProvider(options) {
|
|
177
|
+
if (!options || typeof options !== "object" || typeof options.table !== "string" || options.table.length === 0 || typeof options.client !== "object" || options.client === null || typeof options.client.send !== "function")
|
|
178
|
+
invalid("Expected { table, client } with a table name and a DynamoDBClient.");
|
|
179
|
+
const { table: TableName, logger } = options;
|
|
180
|
+
const attempts = Math.max(1, options.retry?.attempts ?? 10);
|
|
181
|
+
const baseDelayMs = Math.max(0, options.retry?.baseDelayMs ?? 25);
|
|
182
|
+
const db = documentClientOf(options.client);
|
|
183
|
+
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
184
|
+
const exhausted = (context, operation) => {
|
|
185
|
+
throw new DynamoDbRetriesExhausted({
|
|
186
|
+
source: context.name,
|
|
187
|
+
details: { operation, attempts }
|
|
188
|
+
});
|
|
189
|
+
};
|
|
190
|
+
const notFound = (context, id) => new RepositoryEntityNotFound({ source: context.name, details: { id } });
|
|
191
|
+
const uniqueViolation = (context, key) => new RepositoryUniqueViolation({ source: context.name, details: { key } });
|
|
192
|
+
const sortKey = (context, sk) => {
|
|
193
|
+
if (byteLength(sk) > MAX_SORT_KEY_BYTES) {
|
|
194
|
+
throw new DynamoDbLimitExceeded({
|
|
195
|
+
source: context.name,
|
|
196
|
+
details: {
|
|
197
|
+
reason: `A key of "${context.name}" takes more than ${MAX_SORT_KEY_BYTES} bytes, the most a DynamoDB sort key holds.`
|
|
198
|
+
}
|
|
199
|
+
});
|
|
200
|
+
}
|
|
201
|
+
return sk;
|
|
202
|
+
};
|
|
203
|
+
const idOf = (context, data) => {
|
|
204
|
+
if (typeof data.id !== "string" || data.id.length === 0)
|
|
205
|
+
invalid(`An entity of "${context.name}" has no string id.`);
|
|
206
|
+
return sortKey(context, data.id);
|
|
207
|
+
};
|
|
208
|
+
const entityKey = (context, id) => ({
|
|
209
|
+
pk: entityPk(context.name),
|
|
210
|
+
sk: id
|
|
211
|
+
});
|
|
212
|
+
const markerKey = (context, key, value) => ({
|
|
213
|
+
pk: uniquePk(context.name, key),
|
|
214
|
+
sk: sortKey(context, markerOf(value))
|
|
215
|
+
});
|
|
216
|
+
async function getItem(context, id) {
|
|
217
|
+
const { Item } = await db.send(new GetCommand({
|
|
218
|
+
TableName,
|
|
219
|
+
Key: entityKey(context, id),
|
|
220
|
+
ConsistentRead: true
|
|
221
|
+
}));
|
|
222
|
+
return Item;
|
|
223
|
+
}
|
|
224
|
+
const batchGet = (context, keys) => batchGetAll(keys, {
|
|
225
|
+
table: TableName,
|
|
226
|
+
attempts,
|
|
227
|
+
baseDelayMs,
|
|
228
|
+
send: (Keys) => db.send(new BatchGetCommand({
|
|
229
|
+
RequestItems: {
|
|
230
|
+
[TableName]: { Keys: [...Keys], ConsistentRead: true }
|
|
231
|
+
}
|
|
232
|
+
})),
|
|
233
|
+
onRetry: (attempt, left) => logger?.debug("dynamodb.retry", {
|
|
234
|
+
operation: "BatchGetItem",
|
|
235
|
+
repository: context.name,
|
|
236
|
+
attempt,
|
|
237
|
+
left
|
|
238
|
+
}),
|
|
239
|
+
onExhausted: () => exhausted(context, "BatchGetItem")
|
|
240
|
+
});
|
|
241
|
+
async function getMany(context, ids) {
|
|
242
|
+
const unique = [
|
|
243
|
+
...new Set(ids.filter((id) => typeof id === "string" && id.length > 0 && byteLength(id) <= MAX_SORT_KEY_BYTES))
|
|
244
|
+
];
|
|
245
|
+
const items = await batchGet(context, unique.map((id) => entityKey(context, id)));
|
|
246
|
+
return items.sort((a, b) => a.sk < b.sk ? -1 : a.sk > b.sk ? 1 : 0).map((item) => item.data);
|
|
247
|
+
}
|
|
248
|
+
async function markedIds(context, key, values) {
|
|
249
|
+
const keys = new Map;
|
|
250
|
+
for (const value of values) {
|
|
251
|
+
const sk = markerOf(value);
|
|
252
|
+
if (byteLength(sk) <= MAX_SORT_KEY_BYTES)
|
|
253
|
+
keys.set(sk, { pk: uniquePk(context.name, key), sk });
|
|
254
|
+
}
|
|
255
|
+
const markers = await batchGet(context, [...keys.values()]);
|
|
256
|
+
return markers.map((marker) => marker.id);
|
|
257
|
+
}
|
|
258
|
+
async function partition(context) {
|
|
259
|
+
const rows = [];
|
|
260
|
+
let ExclusiveStartKey;
|
|
261
|
+
do {
|
|
262
|
+
const page = await db.send(new QueryCommand({
|
|
263
|
+
TableName,
|
|
264
|
+
KeyConditionExpression: "pk = :pk",
|
|
265
|
+
ExpressionAttributeValues: { ":pk": entityPk(context.name) },
|
|
266
|
+
ConsistentRead: true,
|
|
267
|
+
ExclusiveStartKey
|
|
268
|
+
}));
|
|
269
|
+
for (const item of page.Items ?? [])
|
|
270
|
+
rows.push(item.data);
|
|
271
|
+
ExclusiveStartKey = page.LastEvaluatedKey;
|
|
272
|
+
} while (ExclusiveStartKey);
|
|
273
|
+
return rows;
|
|
274
|
+
}
|
|
275
|
+
function inIdOrder(query) {
|
|
276
|
+
const orders = query.orderBy ?? [];
|
|
277
|
+
if (orders.length === 0)
|
|
278
|
+
return query.after === undefined;
|
|
279
|
+
const [order] = orders;
|
|
280
|
+
return orders.length === 1 && order?.key === "id" && (order.direction ?? "asc") === "asc";
|
|
281
|
+
}
|
|
282
|
+
async function pageInIdOrder(context, query) {
|
|
283
|
+
const { limit, after } = query;
|
|
284
|
+
if (limit === undefined || (query.offset ?? 0) !== 0)
|
|
285
|
+
return;
|
|
286
|
+
if (!inIdOrder(query))
|
|
287
|
+
return;
|
|
288
|
+
const where = query.where ?? {};
|
|
289
|
+
if (where.id !== undefined)
|
|
290
|
+
return;
|
|
291
|
+
if (markedKeys(context).some((key) => where[key] !== undefined))
|
|
292
|
+
return;
|
|
293
|
+
const from = after?.[0];
|
|
294
|
+
if (after !== undefined && (typeof from !== "string" || byteLength(from) > MAX_SORT_KEY_BYTES))
|
|
295
|
+
return;
|
|
296
|
+
const rows = [];
|
|
297
|
+
if (limit <= 0)
|
|
298
|
+
return rows;
|
|
299
|
+
const expiry = expiryKeyOf(context);
|
|
300
|
+
let ExclusiveStartKey;
|
|
301
|
+
do {
|
|
302
|
+
const page = await db.send(new QueryCommand({
|
|
303
|
+
TableName,
|
|
304
|
+
KeyConditionExpression: from === undefined ? "pk = :pk" : "pk = :pk AND sk > :after",
|
|
305
|
+
ExpressionAttributeValues: {
|
|
306
|
+
":pk": entityPk(context.name),
|
|
307
|
+
...from === undefined ? {} : { ":after": from }
|
|
308
|
+
},
|
|
309
|
+
ConsistentRead: true,
|
|
310
|
+
Limit: limit - rows.length,
|
|
311
|
+
ExclusiveStartKey
|
|
312
|
+
}));
|
|
313
|
+
for (const item of page.Items ?? []) {
|
|
314
|
+
const { data } = item;
|
|
315
|
+
if (matchesQuery(data, query, expiry))
|
|
316
|
+
rows.push(data);
|
|
317
|
+
}
|
|
318
|
+
ExclusiveStartKey = page.LastEvaluatedKey;
|
|
319
|
+
} while (ExclusiveStartKey && rows.length < limit);
|
|
320
|
+
return rows;
|
|
321
|
+
}
|
|
322
|
+
async function candidates(context, query) {
|
|
323
|
+
const where = query.where ?? {};
|
|
324
|
+
const valuesOf = (condition) => isMembership(condition) ? condition.in : [condition];
|
|
325
|
+
if (where.id !== undefined)
|
|
326
|
+
return getMany(context, valuesOf(where.id));
|
|
327
|
+
for (const key of markedKeys(context)) {
|
|
328
|
+
if (where[key] === undefined)
|
|
329
|
+
continue;
|
|
330
|
+
const values = valuesOf(where[key]);
|
|
331
|
+
if (!values.every(isPresent))
|
|
332
|
+
continue;
|
|
333
|
+
return getMany(context, await markedIds(context, key, values));
|
|
334
|
+
}
|
|
335
|
+
return partition(context);
|
|
336
|
+
}
|
|
337
|
+
async function sendOne(item) {
|
|
338
|
+
if (item.Put)
|
|
339
|
+
await db.send(new PutCommand(item.Put));
|
|
340
|
+
else if (item.Delete)
|
|
341
|
+
await db.send(new DeleteCommand(item.Delete));
|
|
342
|
+
else if (item.Update)
|
|
343
|
+
await db.send(new UpdateCommand(item.Update));
|
|
344
|
+
else
|
|
345
|
+
invalid("Unexpected write.");
|
|
346
|
+
}
|
|
347
|
+
async function write(context, items) {
|
|
348
|
+
if (items.length > TRANSACTION_LIMIT) {
|
|
349
|
+
throw new DynamoDbLimitExceeded({
|
|
350
|
+
source: context.name,
|
|
351
|
+
details: {
|
|
352
|
+
reason: `A write of "${context.name}" needs ${items.length} items (the entity and the markers of its unique keys), more than the ${TRANSACTION_LIMIT} of a DynamoDB transaction.`
|
|
353
|
+
}
|
|
354
|
+
});
|
|
355
|
+
}
|
|
356
|
+
for (let attempt = 1;attempt <= attempts; attempt++) {
|
|
357
|
+
try {
|
|
358
|
+
if (items.length === 1)
|
|
359
|
+
await sendOne(items[0]);
|
|
360
|
+
else
|
|
361
|
+
await db.send(new TransactWriteCommand({ TransactItems: [...items] }));
|
|
362
|
+
return;
|
|
363
|
+
} catch (error) {
|
|
364
|
+
const failure = writeFailureOf(error);
|
|
365
|
+
if (failure === undefined)
|
|
366
|
+
throw error;
|
|
367
|
+
if (failure.kind === "condition")
|
|
368
|
+
return failure.index;
|
|
369
|
+
logger?.debug("dynamodb.retry", {
|
|
370
|
+
operation: "TransactWriteItems",
|
|
371
|
+
repository: context.name,
|
|
372
|
+
attempt
|
|
373
|
+
});
|
|
374
|
+
await sleep(backoffOf(attempt, baseDelayMs));
|
|
375
|
+
}
|
|
376
|
+
}
|
|
377
|
+
return exhausted(context, "TransactWriteItems");
|
|
378
|
+
}
|
|
379
|
+
const putEntity = (context, data, condition) => ({
|
|
380
|
+
Put: {
|
|
381
|
+
TableName,
|
|
382
|
+
Item: {
|
|
383
|
+
...entityKey(context, idOf(context, data)),
|
|
384
|
+
data,
|
|
385
|
+
m: markersOf(data, markedKeys(context))
|
|
386
|
+
},
|
|
387
|
+
...condition
|
|
388
|
+
}
|
|
389
|
+
});
|
|
390
|
+
const putMarker = (context, key, value, id) => ({
|
|
391
|
+
Put: {
|
|
392
|
+
TableName,
|
|
393
|
+
Item: { ...markerKey(context, key, value), id },
|
|
394
|
+
ConditionExpression: "attribute_not_exists(pk)"
|
|
395
|
+
}
|
|
396
|
+
});
|
|
397
|
+
const deleteMarker = (context, key, value) => ({
|
|
398
|
+
Delete: { TableName, Key: markerKey(context, key, value) }
|
|
399
|
+
});
|
|
400
|
+
async function create(context, data) {
|
|
401
|
+
const id = idOf(context, data);
|
|
402
|
+
const keys = markedKeys(context).filter((key) => isPresent(data[key]));
|
|
403
|
+
const failed = await write(context, [
|
|
404
|
+
putEntity(context, data, NEW_ITEM),
|
|
405
|
+
...keys.map((key) => putMarker(context, key, data[key], id))
|
|
406
|
+
]);
|
|
407
|
+
if (failed === undefined)
|
|
408
|
+
return true;
|
|
409
|
+
if (failed === 0)
|
|
410
|
+
return false;
|
|
411
|
+
throw uniqueViolation(context, keys[failed - 1]);
|
|
412
|
+
}
|
|
413
|
+
async function replace(context, previous, data, equal) {
|
|
414
|
+
const id = idOf(context, data);
|
|
415
|
+
const before = previous.data;
|
|
416
|
+
const changed = markedKeys(context).filter((key) => isPresent(before[key]) !== isPresent(data[key]) || isPresent(data[key]) && markerOf(before[key]) !== markerOf(data[key]));
|
|
417
|
+
const removed = changed.filter((key) => isPresent(before[key]));
|
|
418
|
+
const added = changed.filter((key) => isPresent(data[key]));
|
|
419
|
+
const failed = await write(context, [
|
|
420
|
+
putEntity(context, data, unchanged(previous.m, equal)),
|
|
421
|
+
...removed.map((key) => deleteMarker(context, key, before[key])),
|
|
422
|
+
...added.map((key) => putMarker(context, key, data[key], id))
|
|
423
|
+
]);
|
|
424
|
+
if (failed === undefined)
|
|
425
|
+
return true;
|
|
426
|
+
if (failed === 0)
|
|
427
|
+
return false;
|
|
428
|
+
throw uniqueViolation(context, added[failed - 1 - removed.length] ?? "id");
|
|
429
|
+
}
|
|
430
|
+
const isExpired = (context, data, now) => {
|
|
431
|
+
const expiry = expiryKeyOf(context);
|
|
432
|
+
return expiry !== undefined && !matchesQuery(data, { withDeleted: true, now }, expiry);
|
|
433
|
+
};
|
|
434
|
+
const logged = (operation, run) => async (context, ...args) => {
|
|
435
|
+
try {
|
|
436
|
+
return await run(context, ...args);
|
|
437
|
+
} catch (error) {
|
|
438
|
+
if (!(error instanceof Exception)) {
|
|
439
|
+
logger?.warn("dynamodb.error", {
|
|
440
|
+
operation,
|
|
441
|
+
repository: context.name,
|
|
442
|
+
error: error instanceof Error ? error.name : String(error),
|
|
443
|
+
message: error instanceof Error ? error.message : undefined
|
|
444
|
+
});
|
|
445
|
+
}
|
|
446
|
+
throw error;
|
|
447
|
+
}
|
|
448
|
+
};
|
|
449
|
+
return {
|
|
450
|
+
capabilities: Object.freeze({
|
|
451
|
+
keyset: true,
|
|
452
|
+
versioning: true,
|
|
453
|
+
atomic: true,
|
|
454
|
+
expiry: true
|
|
455
|
+
}),
|
|
456
|
+
findOne: logged("findOne", async (context, query = {}) => {
|
|
457
|
+
const [first] = selectRows(await candidates(context, query), { ...query, limit: Math.min(query.limit ?? 1, 1) }, expiryKeyOf(context));
|
|
458
|
+
return first;
|
|
459
|
+
}),
|
|
460
|
+
findMany: logged("findMany", async (context, query = {}) => await pageInIdOrder(context, query) ?? selectRows(await candidates(context, query), query, expiryKeyOf(context))),
|
|
461
|
+
count: logged("count", async (context, query = {}) => {
|
|
462
|
+
const expiry = expiryKeyOf(context);
|
|
463
|
+
return (await candidates(context, query)).filter((data) => matchesQuery(data, query, expiry)).length;
|
|
464
|
+
}),
|
|
465
|
+
insert: logged("insert", async (context, data) => {
|
|
466
|
+
const value = storable(data);
|
|
467
|
+
if (!await create(context, value))
|
|
468
|
+
throw uniqueViolation(context, "id");
|
|
469
|
+
}),
|
|
470
|
+
update: logged("update", async (context, data, options = {}) => {
|
|
471
|
+
const value = storable(data);
|
|
472
|
+
const id = idOf(context, value);
|
|
473
|
+
const expected = options.expectedVersion;
|
|
474
|
+
for (let attempt = 1;attempt <= attempts; attempt++) {
|
|
475
|
+
const previous = await getItem(context, id);
|
|
476
|
+
if (!previous)
|
|
477
|
+
throw notFound(context, id);
|
|
478
|
+
const actual = previous.data.version;
|
|
479
|
+
if (expected !== undefined && actual !== expected) {
|
|
480
|
+
throw new RepositoryConcurrencyConflict({
|
|
481
|
+
source: context.name,
|
|
482
|
+
details: {
|
|
483
|
+
id,
|
|
484
|
+
expected,
|
|
485
|
+
...typeof actual === "number" ? { actual } : {}
|
|
486
|
+
}
|
|
487
|
+
});
|
|
488
|
+
}
|
|
489
|
+
const equal = expected === undefined ? undefined : [["version", expected]];
|
|
490
|
+
if (await replace(context, previous, value, equal))
|
|
491
|
+
return;
|
|
492
|
+
}
|
|
493
|
+
exhausted(context, "update");
|
|
494
|
+
}),
|
|
495
|
+
remove: logged("remove", async (context, id) => {
|
|
496
|
+
for (let attempt = 1;attempt <= attempts; attempt++) {
|
|
497
|
+
const previous = typeof id === "string" && id.length > 0 ? await getItem(context, id) : undefined;
|
|
498
|
+
if (!previous)
|
|
499
|
+
throw notFound(context, id);
|
|
500
|
+
const keys = markedKeys(context).filter((key) => isPresent(previous.data[key]));
|
|
501
|
+
const failed = await write(context, [
|
|
502
|
+
{
|
|
503
|
+
Delete: {
|
|
504
|
+
TableName,
|
|
505
|
+
Key: entityKey(context, id),
|
|
506
|
+
...unchanged(previous.m)
|
|
507
|
+
}
|
|
508
|
+
},
|
|
509
|
+
...keys.map((key) => deleteMarker(context, key, previous.data[key]))
|
|
510
|
+
]);
|
|
511
|
+
if (failed === undefined)
|
|
512
|
+
return;
|
|
513
|
+
}
|
|
514
|
+
exhausted(context, "remove");
|
|
515
|
+
}),
|
|
516
|
+
increment: logged("increment", async (context, id, deltas, options = {}) => {
|
|
517
|
+
const marked = markedKeys(context);
|
|
518
|
+
const set = storable(options.set ?? {});
|
|
519
|
+
for (const key of [...Object.keys(deltas), ...Object.keys(set)]) {
|
|
520
|
+
if (key === "id" || marked.includes(key)) {
|
|
521
|
+
throw new InvalidRepositoryQuery({
|
|
522
|
+
source: context.name,
|
|
523
|
+
details: {
|
|
524
|
+
reason: `increment cannot change the unique key "${key}" on DynamoDB.`
|
|
525
|
+
}
|
|
526
|
+
});
|
|
527
|
+
}
|
|
528
|
+
}
|
|
529
|
+
const names = {
|
|
530
|
+
"#data": "data",
|
|
531
|
+
"#deletedAt": "deletedAt"
|
|
532
|
+
};
|
|
533
|
+
const values = { ":null": "NULL" };
|
|
534
|
+
const assignments = [];
|
|
535
|
+
Object.entries(deltas).forEach(([key, delta], index) => {
|
|
536
|
+
names[`#d${index}`] = key;
|
|
537
|
+
values[`:d${index}`] = delta;
|
|
538
|
+
values[":zero"] = 0;
|
|
539
|
+
assignments.push(`#data.#d${index} = if_not_exists(#data.#d${index}, :zero) + :d${index}`);
|
|
540
|
+
});
|
|
541
|
+
Object.entries(set).forEach(([key, value], index) => {
|
|
542
|
+
if (Object.hasOwn(deltas, key))
|
|
543
|
+
return;
|
|
544
|
+
names[`#s${index}`] = key;
|
|
545
|
+
values[`:s${index}`] = value;
|
|
546
|
+
assignments.push(`#data.#s${index} = :s${index}`);
|
|
547
|
+
});
|
|
548
|
+
const conditions = [
|
|
549
|
+
"attribute_exists(pk)",
|
|
550
|
+
"(attribute_not_exists(#data.#deletedAt) OR attribute_type(#data.#deletedAt, :null))"
|
|
551
|
+
];
|
|
552
|
+
const expiry = expiryKeyOf(context);
|
|
553
|
+
if (expiry !== undefined) {
|
|
554
|
+
names["#expiry"] = expiry;
|
|
555
|
+
values[":now"] = options.now ?? new Date().toISOString();
|
|
556
|
+
conditions.push("(attribute_not_exists(#data.#expiry) OR attribute_type(#data.#expiry, :null) OR #data.#expiry > :now)");
|
|
557
|
+
}
|
|
558
|
+
for (let attempt = 1;attempt <= attempts; attempt++) {
|
|
559
|
+
if (assignments.length > 0) {
|
|
560
|
+
try {
|
|
561
|
+
const { Attributes } = await db.send(new UpdateCommand({
|
|
562
|
+
TableName,
|
|
563
|
+
Key: entityKey(context, id),
|
|
564
|
+
UpdateExpression: `SET ${assignments.join(", ")}`,
|
|
565
|
+
ConditionExpression: conditions.join(" AND "),
|
|
566
|
+
ExpressionAttributeNames: names,
|
|
567
|
+
ExpressionAttributeValues: values,
|
|
568
|
+
ReturnValues: "ALL_NEW"
|
|
569
|
+
}));
|
|
570
|
+
return Attributes.data;
|
|
571
|
+
} catch (error) {
|
|
572
|
+
const failure = writeFailureOf(error);
|
|
573
|
+
if (failure === undefined)
|
|
574
|
+
throw error;
|
|
575
|
+
if (failure.kind === "conflict") {
|
|
576
|
+
await sleep(backoffOf(attempt, baseDelayMs));
|
|
577
|
+
continue;
|
|
578
|
+
}
|
|
579
|
+
}
|
|
580
|
+
}
|
|
581
|
+
const stored = await getItem(context, id);
|
|
582
|
+
const alive = stored !== undefined && !isPresent(stored.data.deletedAt) && !isExpired(context, stored.data, options.now);
|
|
583
|
+
if (alive && assignments.length === 0)
|
|
584
|
+
return stored.data;
|
|
585
|
+
if (!options.create) {
|
|
586
|
+
if (alive)
|
|
587
|
+
continue;
|
|
588
|
+
throw notFound(context, id);
|
|
589
|
+
}
|
|
590
|
+
const created = storable({ ...options.create, id });
|
|
591
|
+
if (!stored) {
|
|
592
|
+
if (await create(context, created))
|
|
593
|
+
return created;
|
|
594
|
+
continue;
|
|
595
|
+
}
|
|
596
|
+
if (isPresent(stored.data.deletedAt))
|
|
597
|
+
throw uniqueViolation(context, "id");
|
|
598
|
+
if (alive)
|
|
599
|
+
continue;
|
|
600
|
+
const at = expiry === undefined ? undefined : stored.data[expiry];
|
|
601
|
+
if (await replace(context, stored, created, expiry === undefined || at === undefined ? undefined : [[expiry, at]]))
|
|
602
|
+
return created;
|
|
603
|
+
}
|
|
604
|
+
return exhausted(context, "increment");
|
|
605
|
+
}),
|
|
606
|
+
async migrate() {}
|
|
607
|
+
};
|
|
608
|
+
}
|
|
609
|
+
|
|
610
|
+
// src/index.ts
|
|
611
|
+
var providers = [
|
|
612
|
+
{ name: "dynamodb", module: "@afora/dynamodb", export: "dynamoDbProvider" }
|
|
613
|
+
];
|
|
614
|
+
export {
|
|
615
|
+
DynamoDbLimitExceeded,
|
|
616
|
+
DynamoDbRetriesExhausted,
|
|
617
|
+
dynamoDbProvider,
|
|
618
|
+
providers
|
|
619
|
+
};
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
import { type ILogger } from "@afora/core";
|
|
2
|
+
import type { IRepositoryProvider } from "@afora/core/adapters";
|
|
3
|
+
import type { DynamoDBClient } from "@aws-sdk/client-dynamodb";
|
|
4
|
+
import { DynamoDBDocumentClient } from "@aws-sdk/lib-dynamodb";
|
|
5
|
+
export interface DynamoDbProviderOptions {
|
|
6
|
+
/** The name of the table: string keys `pk` (partition) and `sk` (sort). */
|
|
7
|
+
readonly table: string;
|
|
8
|
+
/** A `DynamoDBClient`, or a `DynamoDBDocumentClient` made from one. */
|
|
9
|
+
readonly client: DynamoDBClient | DynamoDBDocumentClient;
|
|
10
|
+
/**
|
|
11
|
+
* Where retries (`debug`, `dynamodb.retry`) and errors that are not
|
|
12
|
+
* afora's own (`warn`, `dynamodb.error`) are told; an application's
|
|
13
|
+
* `ILogger` fits. Nothing is logged without it.
|
|
14
|
+
*/
|
|
15
|
+
readonly logger?: Pick<ILogger, "debug" | "warn">;
|
|
16
|
+
/**
|
|
17
|
+
* How often a request DynamoDB refuses for now is sent again: keys left
|
|
18
|
+
* unprocessed by `BatchGetItem`, transactions cancelled by a conflicting
|
|
19
|
+
* one, writes that lost a race. Defaults to 10 attempts, from 25 ms.
|
|
20
|
+
*/
|
|
21
|
+
readonly retry?: {
|
|
22
|
+
readonly attempts?: number;
|
|
23
|
+
readonly baseDelayMs?: number;
|
|
24
|
+
};
|
|
25
|
+
}
|
|
26
|
+
/**
|
|
27
|
+
* The provider of `dynamoDbProvider`: it reads by cursor, honors expected
|
|
28
|
+
* versions, increments atomically and leaves expired records out. It has no
|
|
29
|
+
* transactions: DynamoDB only writes up to 100 items at once, without reads
|
|
30
|
+
* in between.
|
|
31
|
+
*/
|
|
32
|
+
export type DynamoDbProvider = IRepositoryProvider & {
|
|
33
|
+
readonly capabilities: {
|
|
34
|
+
readonly keyset: true;
|
|
35
|
+
readonly versioning: true;
|
|
36
|
+
readonly atomic: true;
|
|
37
|
+
readonly expiry: true;
|
|
38
|
+
};
|
|
39
|
+
/** A no-op: DynamoDB has no schema to change. See the README. */
|
|
40
|
+
migrate(): Promise<void>;
|
|
41
|
+
};
|
|
42
|
+
/**
|
|
43
|
+
* A repository provider on one DynamoDB table shared by every entity:
|
|
44
|
+
*
|
|
45
|
+
* - entity: `pk = E#<context.name>`, `sk = <id>`, `data`, `m`
|
|
46
|
+
* - marker: `pk = U#<context.name>#<key>`, `sk = <encoded value>`, `id`
|
|
47
|
+
*
|
|
48
|
+
* Unique keys other than `id` are enforced with marker items written in the
|
|
49
|
+
* same transaction as the entity. Reads get their candidates by `id`, through
|
|
50
|
+
* the marker of a unique key, or by querying the partition of the repository,
|
|
51
|
+
* then filter, order and page them as `inMemoryProvider` does. See the
|
|
52
|
+
* package README for the layout and its limits.
|
|
53
|
+
*/
|
|
54
|
+
export declare function dynamoDbProvider(options: DynamoDbProviderOptions): DynamoDbProvider;
|
package/package.json
CHANGED
|
@@ -1,6 +1,59 @@
|
|
|
1
1
|
{
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
2
|
+
"name": "@afora/dynamodb",
|
|
3
|
+
"version": "0.0.11",
|
|
4
|
+
"description": "A repository provider for @afora/core on one DynamoDB table.",
|
|
5
|
+
"keywords": [
|
|
6
|
+
"afora",
|
|
7
|
+
"dynamodb",
|
|
8
|
+
"aws",
|
|
9
|
+
"repository",
|
|
10
|
+
"provider",
|
|
11
|
+
"single-table"
|
|
12
|
+
],
|
|
13
|
+
"author": "Alan Reis <alanreisanjo@gmail.com> (https://github.com/Hoyasumii)",
|
|
14
|
+
"license": "MIT",
|
|
15
|
+
"homepage": "https://github.com/afora-framework/core/tree/main/packages/dynamodb#readme",
|
|
16
|
+
"repository": {
|
|
17
|
+
"type": "git",
|
|
18
|
+
"url": "git+https://github.com/afora-framework/core.git",
|
|
19
|
+
"directory": "packages/dynamodb"
|
|
20
|
+
},
|
|
21
|
+
"type": "module",
|
|
22
|
+
"main": "./dist/index.js",
|
|
23
|
+
"module": "./dist/index.js",
|
|
24
|
+
"types": "./dist/index.d.ts",
|
|
25
|
+
"exports": {
|
|
26
|
+
".": {
|
|
27
|
+
"types": "./dist/index.d.ts",
|
|
28
|
+
"import": "./dist/index.js",
|
|
29
|
+
"default": "./dist/index.js"
|
|
30
|
+
},
|
|
31
|
+
"./package.json": "./package.json"
|
|
32
|
+
},
|
|
33
|
+
"files": [
|
|
34
|
+
"dist"
|
|
35
|
+
],
|
|
36
|
+
"engines": {
|
|
37
|
+
"bun": ">=1.4.4",
|
|
38
|
+
"node": ">=22"
|
|
39
|
+
},
|
|
40
|
+
"publishConfig": {
|
|
41
|
+
"access": "public",
|
|
42
|
+
"provenance": true
|
|
43
|
+
},
|
|
44
|
+
"scripts": {
|
|
45
|
+
"typecheck": "bun check",
|
|
46
|
+
"build": "rm -rf dist && bun build ./src/index.ts --outdir ./dist --target node --format esm --packages external --tsconfig-override tsconfig.build.json && tsc -p tsconfig.build.json && bun ../../scripts/add-dts-extensions.ts dist",
|
|
47
|
+
"test": "bun test"
|
|
48
|
+
},
|
|
49
|
+
"devDependencies": {
|
|
50
|
+
"@aws-sdk/client-dynamodb": "^3.1164.0",
|
|
51
|
+
"@aws-sdk/lib-dynamodb": "^3.1164.0",
|
|
52
|
+
"@types/bun": "^1.4.4"
|
|
53
|
+
},
|
|
54
|
+
"peerDependencies": {
|
|
55
|
+
"@afora/core": ">=0.0.9",
|
|
56
|
+
"@aws-sdk/client-dynamodb": "^3.1164.0",
|
|
57
|
+
"@aws-sdk/lib-dynamodb": "^3.1164.0"
|
|
58
|
+
}
|
|
59
|
+
}
|