@aws-blocks/bb-distributed-table 0.1.3 → 0.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DESIGN.md +23 -0
- package/README.md +40 -0
- package/dist/errors.d.ts +25 -0
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js +71 -0
- package/dist/index.aws.d.ts +10 -2
- package/dist/index.aws.d.ts.map +1 -1
- package/dist/index.aws.js +22 -8
- package/dist/index.cdk.d.ts +1 -1
- package/dist/index.cdk.d.ts.map +1 -1
- package/dist/index.mock.d.ts +10 -2
- package/dist/index.mock.d.ts.map +1 -1
- package/dist/index.mock.js +21 -7
- package/dist/index.test.js +172 -0
- package/dist/parity.test.js +49 -0
- package/dist/types.d.ts +51 -0
- package/dist/types.d.ts.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/package.json +14 -4
- package/src/errors.ts +85 -0
- package/src/index.aws.ts +27 -8
- package/src/index.cdk.ts +1 -1
- package/src/index.mock.ts +26 -7
- package/src/index.test.ts +205 -0
- package/src/parity.test.ts +76 -0
- package/src/types.ts +54 -0
- package/src/version.ts +1 -1
package/DESIGN.md
CHANGED
|
@@ -100,6 +100,29 @@ A generic `ValidationException` is exactly the kind of catch-all bucket worth av
|
|
|
100
100
|
|
|
101
101
|
**Mock/AWS parity:** the mock checks serialized byte length client-side and throws `ItemTooLarge` directly. On AWS, DynamoDB raises a generic `ValidationException` for an oversized item; the runtime narrows on the size-specific message (`size has exceeded`) and re-maps only that case to `ItemTooLarge`. Other `ValidationException` causes (malformed expressions, type mismatches) propagate unchanged. Both layers therefore surface the same `error.name`, and the shared message lives in `errors.ts` (`DistributedTableMessages.itemTooLarge`) so the two stay byte-for-byte aligned.
|
|
102
102
|
|
|
103
|
+
### D-DT-9: `readValidation` — `off | coerce | strict`, defaulting to `coerce`
|
|
104
|
+
|
|
105
|
+
> The **default choice** (`'coerce'` over `'off'`) is recorded as a cross-cutting architectural decision in [`docs/DECISIONS.md` D-015](../../docs/DECISIONS.md#d-015-distributedtable-reads-default-to-readvalidation-coerce-not-off). This section covers the per-BB mechanics.
|
|
106
|
+
|
|
107
|
+
**Decision:** Writes always validate. Reads reconcile a stored item with the schema per the `readValidation` mode, which defaults to **`'coerce'`**. `get`/`getBatch`/`query`/`scan` pass each stored value through `schema['~standard'].validate()`:
|
|
108
|
+
- **`'coerce'`** (default) — return the schema's output (defaults filled / types narrowed for transform-bearing schemas); on validation failure return the **raw** value with a `warn` log — never throws.
|
|
109
|
+
- **`'strict'`** — throw `ValidationFailed` on any item that doesn't satisfy the schema.
|
|
110
|
+
- **`'off'`** — return the raw stored value, no validation.
|
|
111
|
+
|
|
112
|
+
**Rationale:** The asymmetry "validate on write, return raw on read" breaks the documented read-modify-write update pattern after a schema change, in two ways. A newly added field is absent from the read, so `get()` returns a value that silently violates the declared type `T` and any schema `.default()` is neither applied nor persisted on write-back; and if the new field is required with no default, the `put()` half of the cycle throws `ValidationFailed`, stranding the row. Coercing on read fixes the round-trip for the coercible case (added fields with `.default()`, widened/narrowed types) by handing the caller a value the current schema accepts. It cannot invent a required-no-default value — those rows fall through to the raw + warn path and still need an explicit migration.
|
|
113
|
+
|
|
114
|
+
**Why `coerce` is the default.** The bug is that the *default* read violates `T`; a fix only users who discover a flag can enable isn't a fix. Market research across comparable typed data layers backs a coercing default: Rails ActiveRecord type-casts on load, Mongoose applies defaults/casts when hydrating, ElectroDB runs getters and returns schema-shaped items — none re-throw on stored data, and Postgres itself synthesizes an added column's `DEFAULT` at read time (`ALTER TABLE ADD COLUMN … DEFAULT`), which is coerce-on-read in all but name.
|
|
115
|
+
|
|
116
|
+
**Why not `strict` by default.** Throwing on read makes legacy/corrupt rows unreadable — you couldn't fetch them to migrate — turns one bad row into a whole-`scan`/`getBatch` outage, and violates the project rule that reads return data or `null` and throw only for violated preconditions. Every surveyed library that ships a strict read (DynamoDB-Toolbox `format()`, the `zod-firebase` converter) also ships an escape hatch. So `strict` is offered as an opt-in for tables that want mismatch treated as corruption, not as the default.
|
|
117
|
+
|
|
118
|
+
**Why keep `off`.** The raw escape hatch (ElectroDB's `data:'raw'`, Mongoose's `.lean()`) is needed for hot paths, trusted write-validated data, and reading rows that can't yet be coerced during a migration.
|
|
119
|
+
|
|
120
|
+
**Best-effort coercion caveat.** Standard Schema only guarantees `validate()` *checks*; it does not require transformation. Zod fills defaults/coerces, but a check-only Valibot/ArkType schema returns its input unchanged — so `'coerce'` degrades to pass-through for those validators. Documented as best-effort; coercion never fabricates a required value.
|
|
121
|
+
|
|
122
|
+
**`coerce` preserves unknown keys.** Most validators discard unrecognized keys when they produce their output (Zod `.strip()`s by default), so a stored row carrying attributes beyond the current schema — fields from an older schema version, or columns another writer owns — would lose them on read, and a read-modify-write (`get()` → `put()`) would persist that loss, silently deleting data. To prevent this, `'coerce'` deep-merges the coerced value **over the raw stored item** (`mergeCoercedOverRaw` in `errors.ts`, built on `defu`'s `createDefu`): schema output wins per key (defaults filled, types narrowed), while keys the schema doesn't declare — including nested ones — survive. Arrays are treated as opaque leaves: the coerced array replaces the raw array wholesale (defu concatenates arrays by default, which would *duplicate* elements here since the coerced array is derived from the raw one, so that one behavior is overridden). The merge runs only when both the stored item and the coerced value are plain objects; defu's `__proto__`/`constructor` guard is retained. `'coerce'` is therefore additive-and-lossless for the common cases (added fields, nested unknowns, type coercion). The one residual: a schema whose *transform* deliberately renames or removes a key — the merge resurrects the old key, so transform-heavy schemas should use `'strict'` or `'off'`. `defu` is used rather than a hand-rolled merge because treating arrays as leaves requires custom config in every merge library regardless, and defu is a maintained, prototype-safe, zero-dependency ESM package.
|
|
123
|
+
|
|
124
|
+
**Mock/AWS parity:** both layers call the same `applyReadValidation()` helper in `errors.ts`, so all three modes (coerced output, raw-fallback + warn, strict throw) behave identically. `null` (a missing item) passes through untouched in every mode, preserving not-found semantics.
|
|
125
|
+
|
|
103
126
|
## Infrastructure (CDK)
|
|
104
127
|
|
|
105
128
|
Creates a single DynamoDB table:
|
package/README.md
CHANGED
|
@@ -40,6 +40,7 @@ const table = new DistributedTable(scope, id, options)
|
|
|
40
40
|
| `key` | `TableKeyConfig<T>` | Yes | Primary key configuration: `{ partitionKey, sortKey? }`. Field names must exist in the schema. |
|
|
41
41
|
| `indexes` | `Record<string, TableKeyConfig<T>>` | No | Global secondary index definitions. |
|
|
42
42
|
| `ttl` | `keyof T & string` | No | Enable DynamoDB TTL on the specified attribute. The field should contain a Unix epoch timestamp in seconds. |
|
|
43
|
+
| `readValidation` | `'off' \| 'coerce' \| 'strict'` | No | How reads (`get`/`getBatch`/`query`/`scan`) reconcile a stored item with `schema`. `'coerce'` (**default**) returns the coerced value and, on failure, the raw value + a warning (never throws); `'strict'` throws `ValidationFailed` on a non-conforming item; `'off'` returns the raw value with no validation. See [Reads and schema evolution](#reads-and-schema-evolution). |
|
|
43
44
|
| `table` | `ExternalTableRef` | No | Wrap an existing DynamoDB table instead of creating one. |
|
|
44
45
|
| `logger` | `ChildLogger` | No | Optional logger for internal operations. When omitted, a default Logger at error level is created. |
|
|
45
46
|
|
|
@@ -129,6 +130,45 @@ All condition failures throw with `error.name === DistributedTableErrors.Conditi
|
|
|
129
130
|
|
|
130
131
|
> **No partial update:** There is no `update()` or `patch()` method. To change a field, do a read-modify-write — `get()` the item, mutate it, then `put()` the full item back. For safe concurrent updates, pass `{ ifFieldEquals: { version: <previous> } }` to `put()` so the write fails (via `ConditionalCheckFailed`) if another writer changed the item in the meantime (optimistic locking).
|
|
131
132
|
|
|
133
|
+
### Reads and schema evolution
|
|
134
|
+
|
|
135
|
+
Writes always validate against `schema`. Reads reconcile a stored item with the schema according to the **`readValidation`** option, which matters after a schema change: a row written under an older schema may no longer match the declared type `T` — a newly added field is **absent** from the read (so the value silently violates `T`, and a `.default()` is neither applied nor persisted on write-back), and a **required, no-default** field makes the read-modify-write cycle above **fail on the write** as `put()` rejects the legacy shape.
|
|
136
|
+
|
|
137
|
+
`readValidation` has three modes:
|
|
138
|
+
|
|
139
|
+
| Mode | On read | On a non-conforming item |
|
|
140
|
+
|---|---|---|
|
|
141
|
+
| **`'coerce'`** (default) | returns the schema's coerced output (defaults filled, types narrowed) | returns the **raw** value + logs a warning — **never throws** |
|
|
142
|
+
| **`'strict'`** | validates against the schema | **throws** `ValidationFailed` |
|
|
143
|
+
| **`'off'`** | returns the raw stored value, no validation | returns it as-is |
|
|
144
|
+
|
|
145
|
+
The default `'coerce'` closes the schema-evolution gap so a legacy row round-trips cleanly:
|
|
146
|
+
|
|
147
|
+
```typescript
|
|
148
|
+
const orderSchema = z.object({
|
|
149
|
+
orderId: z.string(),
|
|
150
|
+
total: z.number(),
|
|
151
|
+
currency: z.string().default('USD'), // added in a later release
|
|
152
|
+
});
|
|
153
|
+
|
|
154
|
+
const orders = new DistributedTable(scope, 'orders', {
|
|
155
|
+
schema: orderSchema,
|
|
156
|
+
key: { partitionKey: 'orderId' },
|
|
157
|
+
// readValidation: 'coerce' is the default
|
|
158
|
+
});
|
|
159
|
+
|
|
160
|
+
const order = await orders.get({ orderId: 'o1' }); // legacy row → { …, currency: 'USD' }
|
|
161
|
+
await orders.put({ ...order, total: 20 }); // round-trips without ValidationFailed
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
**`'coerce'` never throws:** a value that genuinely can't be coerced (e.g. a required field with no default) is returned **as-is** with a warning, so unrecoverable rows stay readable for migration and `get()` never throws for a bad row.
|
|
165
|
+
|
|
166
|
+
> **Best-effort coercion (validator-dependent).** Coercion depends on the schema *transforming* its input. Zod fills defaults and casts; a check-only Standard Schema validator (some Valibot/ArkType schemas) validates without transforming, so `'coerce'` returns the value unchanged for those — it never invents data.
|
|
167
|
+
|
|
168
|
+
> **`'coerce'` preserves stored keys not in the schema.** Many schemas discard unrecognized keys when they validate (Zod object schemas `.strip()` by default), which would drop attributes a stored row carries beyond the current schema — and a read-modify-write would then persist the loss. To avoid that, `'coerce'` deep-merges the coerced output **over the raw stored item**: schema output wins per key (defaults filled, types narrowed), while attributes the schema doesn't declare — from an older schema version, or columns another writer owns — are **kept**. Arrays are replaced wholesale (the coerced array wins), and nested unknown keys are preserved too. So a routine read-modify-write never silently deletes a field you didn't touch. (Use **`'strict'`** if you instead want a schema mismatch to be rejected, or **`'off'`** to skip the schema pass entirely.)
|
|
169
|
+
|
|
170
|
+
Choose **`'strict'`** for tables where a schema mismatch should be treated as corruption and rejected (note: one bad row then fails the whole `query`/`scan`/`getBatch`). Choose **`'off'`** for hot paths, data you trust was written through this schema, or to read (and preserve) rows you can't yet coerce during a migration.
|
|
171
|
+
|
|
132
172
|
### Error Handling
|
|
133
173
|
|
|
134
174
|
Errors thrown by DistributedTable carry an `error.name` you can match with `isBlocksError`:
|
package/dist/errors.d.ts
CHANGED
|
@@ -1,3 +1,6 @@
|
|
|
1
|
+
import type { StandardSchemaV1 } from '@standard-schema/spec';
|
|
2
|
+
import type { ChildLogger } from '@aws-blocks/bb-logger';
|
|
3
|
+
import type { ReadValidationMode } from './types.js';
|
|
1
4
|
/**
|
|
2
5
|
* Typed error constants for DistributedTable. Use with `isBlocksError()` in catch blocks.
|
|
3
6
|
*
|
|
@@ -108,4 +111,26 @@ export declare const DistributedTableMessages: {
|
|
|
108
111
|
* its stack and requestId remain available for debugging.
|
|
109
112
|
*/
|
|
110
113
|
export declare function remapItemTooLarge(err: unknown): unknown;
|
|
114
|
+
/**
|
|
115
|
+
* @internal Reconcile a stored item with the schema on read, per the
|
|
116
|
+
* `readValidation` mode. Shared by the mock and AWS runtime so all three modes
|
|
117
|
+
* behave identically in both. `null` (a missing item) always passes through
|
|
118
|
+
* untouched, preserving not-found semantics.
|
|
119
|
+
*
|
|
120
|
+
* - `'off'` — return the raw item, no validation.
|
|
121
|
+
* - `'coerce'` — apply the schema (fill defaults / narrow types for
|
|
122
|
+
* transform-bearing schemas) **without dropping data**: the coerced output is
|
|
123
|
+
* deep-merged over the raw item, so schema output wins per key while attributes
|
|
124
|
+
* the schema doesn't declare (from an older schema version or another writer)
|
|
125
|
+
* are preserved. Without this merge, returning the bare validator output would
|
|
126
|
+
* strip unknown keys (Zod `.strip()` default), and a read-modify-write would
|
|
127
|
+
* then persist the stripped item — silently deleting stored data. On validation
|
|
128
|
+
* failure, return the **raw** item and `warn` — never throws — so drifted/legacy
|
|
129
|
+
* rows stay readable and the "reads return data or `null`" contract holds.
|
|
130
|
+
* - `'strict'` — throw `ValidationFailed` on any item that doesn't satisfy the
|
|
131
|
+
* schema.
|
|
132
|
+
*
|
|
133
|
+
* Schemas may validate synchronously or asynchronously; this awaits either.
|
|
134
|
+
*/
|
|
135
|
+
export declare function applyReadValidation<T>(mode: ReadValidationMode, schema: StandardSchemaV1<T>, item: T | null, log: Pick<ChildLogger, 'warn'>, context?: Record<string, unknown>): Promise<T | null>;
|
|
111
136
|
//# sourceMappingURL=errors.d.ts.map
|
package/dist/errors.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"errors.d.ts","sourceRoot":"","sources":["../src/errors.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"errors.d.ts","sourceRoot":"","sources":["../src/errors.ts"],"names":[],"mappings":"AAIA,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,uBAAuB,CAAC;AAC9D,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,uBAAuB,CAAC;AACzD,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,YAAY,CAAC;AAwBrD;;;;;;;;;;;;;;;;;GAiBG;AACH,eAAO,MAAM,sBAAsB;;;IAGlC;;;;;;;;;;;OAWG;;IAEH;;;;;;;;;;;;OAYG;;IAEH;;;;;;;;;OASG;;CAEM,CAAC;AAEX;;;;GAIG;AACH,wBAAgB,WAAW,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,KAAK,CAIhE;AAED;;;;;;;;;;;;;;;GAeG;AACH,wBAAgB,yBAAyB,CAAC,CAAC,SAAS,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,EAC1E,SAAS,EAAE,CAAC,GAAG,SAAS,GACtB,CAAC,GAAG,SAAS,CAQf;AAED;;;;GAIG;AACH,eAAO,MAAM,wBAAwB;oCACb,MAAM,GAAG,SAAS;sCAChB,MAAM;mDAEO,MAAM;wDAED,MAAM,EAAE;;mCAI7B,MAAM;0CAEC,MAAM,aAAa,MAAM,YAAY,MAAM;CAG/D,CAAC;AAEX;;;;;;;;;;;GAWG;AACH,wBAAgB,iBAAiB,CAAC,GAAG,EAAE,OAAO,GAAG,OAAO,CAOvD;AAED;;;;;;;;;;;;;;;;;;;;GAoBG;AACH,wBAAsB,mBAAmB,CAAC,CAAC,EAC1C,IAAI,EAAE,kBAAkB,EACxB,MAAM,EAAE,gBAAgB,CAAC,CAAC,CAAC,EAC3B,IAAI,EAAE,CAAC,GAAG,IAAI,EACd,GAAG,EAAE,IAAI,CAAC,WAAW,EAAE,MAAM,CAAC,EAC9B,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAC/B,OAAO,CAAC,CAAC,GAAG,IAAI,CAAC,CAsBnB"}
|
package/dist/errors.js
CHANGED
|
@@ -1,5 +1,27 @@
|
|
|
1
1
|
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
|
2
2
|
// SPDX-License-Identifier: Apache-2.0
|
|
3
|
+
import { createDefu } from 'defu';
|
|
4
|
+
/**
|
|
5
|
+
* @internal Right-biased deep merge for the `'coerce'` read path: overlay the
|
|
6
|
+
* schema-coerced item onto the raw stored item so schema output (filled defaults,
|
|
7
|
+
* narrowed types) wins, while keys the schema stripped (e.g. attributes from an
|
|
8
|
+
* older schema version or another writer) are preserved rather than silently
|
|
9
|
+
* dropped on a read-modify-write.
|
|
10
|
+
*
|
|
11
|
+
* Arrays are treated as **opaque leaves** — the coerced array replaces the raw
|
|
12
|
+
* array wholesale. defu concatenates arrays by default, which is wrong here: the
|
|
13
|
+
* coerced array is derived from the same raw array, so concatenation would
|
|
14
|
+
* duplicate every element. This customizer overrides that one behavior; plain
|
|
15
|
+
* objects still deep-merge (so nested unknown keys survive), and defu's built-in
|
|
16
|
+
* `__proto__`/`constructor` guard is retained.
|
|
17
|
+
*/
|
|
18
|
+
const mergeCoercedOverRaw = createDefu((obj, key, value) => {
|
|
19
|
+
if (Array.isArray(obj[key]) || Array.isArray(value)) {
|
|
20
|
+
obj[key] = obj[key] ?? value;
|
|
21
|
+
return true;
|
|
22
|
+
}
|
|
23
|
+
return false;
|
|
24
|
+
});
|
|
3
25
|
/**
|
|
4
26
|
* Typed error constants for DistributedTable. Use with `isBlocksError()` in catch blocks.
|
|
5
27
|
*
|
|
@@ -133,3 +155,52 @@ export function remapItemTooLarge(err) {
|
|
|
133
155
|
}
|
|
134
156
|
return err;
|
|
135
157
|
}
|
|
158
|
+
/**
|
|
159
|
+
* @internal Reconcile a stored item with the schema on read, per the
|
|
160
|
+
* `readValidation` mode. Shared by the mock and AWS runtime so all three modes
|
|
161
|
+
* behave identically in both. `null` (a missing item) always passes through
|
|
162
|
+
* untouched, preserving not-found semantics.
|
|
163
|
+
*
|
|
164
|
+
* - `'off'` — return the raw item, no validation.
|
|
165
|
+
* - `'coerce'` — apply the schema (fill defaults / narrow types for
|
|
166
|
+
* transform-bearing schemas) **without dropping data**: the coerced output is
|
|
167
|
+
* deep-merged over the raw item, so schema output wins per key while attributes
|
|
168
|
+
* the schema doesn't declare (from an older schema version or another writer)
|
|
169
|
+
* are preserved. Without this merge, returning the bare validator output would
|
|
170
|
+
* strip unknown keys (Zod `.strip()` default), and a read-modify-write would
|
|
171
|
+
* then persist the stripped item — silently deleting stored data. On validation
|
|
172
|
+
* failure, return the **raw** item and `warn` — never throws — so drifted/legacy
|
|
173
|
+
* rows stay readable and the "reads return data or `null`" contract holds.
|
|
174
|
+
* - `'strict'` — throw `ValidationFailed` on any item that doesn't satisfy the
|
|
175
|
+
* schema.
|
|
176
|
+
*
|
|
177
|
+
* Schemas may validate synchronously or asynchronously; this awaits either.
|
|
178
|
+
*/
|
|
179
|
+
export async function applyReadValidation(mode, schema, item, log, context) {
|
|
180
|
+
if (item == null || mode === 'off')
|
|
181
|
+
return item;
|
|
182
|
+
const result = schema['~standard'].validate(item);
|
|
183
|
+
const resolved = result instanceof Promise ? await result : result;
|
|
184
|
+
if (resolved.issues) {
|
|
185
|
+
if (mode === 'strict') {
|
|
186
|
+
throw blocksError(DistributedTableErrors.ValidationFailed, resolved.issues[0]?.message ?? 'stored item failed schema validation on read');
|
|
187
|
+
}
|
|
188
|
+
log.warn(`readValidation: stored item failed schema validation, returning the raw value. ${resolved.issues[0]?.message ?? ''}`.trim(), context);
|
|
189
|
+
return item;
|
|
190
|
+
}
|
|
191
|
+
// Merge coerced output over the raw item: schema wins per key, but keys the
|
|
192
|
+
// schema stripped are preserved (see mergeCoercedOverRaw). Only merge when both
|
|
193
|
+
// sides are plain objects — a schema whose output is a primitive/array (rare
|
|
194
|
+
// for a table item) is returned as-is.
|
|
195
|
+
if (isPlainObject(item) && isPlainObject(resolved.value)) {
|
|
196
|
+
return mergeCoercedOverRaw(resolved.value, item);
|
|
197
|
+
}
|
|
198
|
+
return resolved.value;
|
|
199
|
+
}
|
|
200
|
+
/** True for a plain `{}` object (not null, array, Date, or class instance). */
|
|
201
|
+
function isPlainObject(v) {
|
|
202
|
+
if (v === null || typeof v !== 'object' || Array.isArray(v))
|
|
203
|
+
return false;
|
|
204
|
+
const proto = Object.getPrototypeOf(v);
|
|
205
|
+
return proto === Object.prototype || proto === null;
|
|
206
|
+
}
|
package/dist/index.aws.d.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { Scope } from '@aws-blocks/core';
|
|
2
2
|
import type { ScopeParent } from '@aws-blocks/core';
|
|
3
3
|
export { DistributedTableErrors } from './errors.js';
|
|
4
|
-
export type { TableKeyConfig, DistributedTableOptions, ExternalTableRef, TableKey, PartitionKeyCondition, SortKeyCondition, KeyCondition, QueryOptions, ScanOptions, PutOptions, DeleteOptions, } from './types.js';
|
|
4
|
+
export type { TableKeyConfig, DistributedTableOptions, ReadValidationMode, ExternalTableRef, TableKey, PartitionKeyCondition, SortKeyCondition, KeyCondition, QueryOptions, ScanOptions, PutOptions, DeleteOptions, } from './types.js';
|
|
5
5
|
import type { TableKeyConfig, DistributedTableOptions, ExternalTableRef, ScanOptions, PutOptions, DeleteOptions, TableKey } from './types.js';
|
|
6
6
|
import type { QueryOptions } from './types.js';
|
|
7
7
|
import type { ChildLogger } from '@aws-blocks/bb-logger';
|
|
@@ -11,8 +11,9 @@ export declare class DistributedTable<T, K extends TableKeyConfig<T> = TableKeyC
|
|
|
11
11
|
private schema;
|
|
12
12
|
private keyConfig;
|
|
13
13
|
private indexes;
|
|
14
|
+
private readValidation;
|
|
14
15
|
private docClient;
|
|
15
|
-
/** @internal Logger for internal operations. Defaults to
|
|
16
|
+
/** @internal Logger for internal operations. Defaults to warn-level when not provided. */
|
|
16
17
|
protected log: ChildLogger;
|
|
17
18
|
constructor(scope: ScopeParent, id: string, options: DistributedTableOptions<T, K, Indexes>);
|
|
18
19
|
get(key: TableKey<T, K>): Promise<T | null>;
|
|
@@ -34,6 +35,13 @@ export declare class DistributedTable<T, K extends TableKeyConfig<T> = TableKeyC
|
|
|
34
35
|
deleteBatch(keys: TableKey<T, K>[]): Promise<void>;
|
|
35
36
|
static fromExisting(tableName: string): ExternalTableRef;
|
|
36
37
|
private validateItem;
|
|
38
|
+
/**
|
|
39
|
+
* Reconcile a stored value with the schema per this table's `readValidation`
|
|
40
|
+
* mode (`off` → raw, `coerce` → coerced output / raw+warn on failure, `strict`
|
|
41
|
+
* → throw on mismatch). `null` (a missing item) passes straight through. See
|
|
42
|
+
* {@link applyReadValidation}.
|
|
43
|
+
*/
|
|
44
|
+
private reconcileRead;
|
|
37
45
|
private buildKey;
|
|
38
46
|
private backoff;
|
|
39
47
|
/**
|
package/dist/index.aws.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.aws.d.ts","sourceRoot":"","sources":["../src/index.aws.ts"],"names":[],"mappings":"AAcA,OAAO,EAAE,KAAK,EAA6C,MAAM,kBAAkB,CAAC;AACpF,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,kBAAkB,CAAC;AAIpD,OAAO,EAAE,sBAAsB,EAAE,MAAM,aAAa,CAAC;AACrD,YAAY,EACX,cAAc,EACd,uBAAuB,EACvB,gBAAgB,EAChB,QAAQ,EACR,qBAAqB,EACrB,gBAAgB,EAChB,YAAY,EACZ,YAAY,EACZ,WAAW,EACX,UAAU,EACV,aAAa,GACb,MAAM,YAAY,CAAC;AAEpB,OAAO,KAAK,EACX,cAAc,EACd,uBAAuB,EACvB,gBAAgB,EAChB,WAAW,EACX,UAAU,EACV,aAAa,EAGb,QAAQ,
|
|
1
|
+
{"version":3,"file":"index.aws.d.ts","sourceRoot":"","sources":["../src/index.aws.ts"],"names":[],"mappings":"AAcA,OAAO,EAAE,KAAK,EAA6C,MAAM,kBAAkB,CAAC;AACpF,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,kBAAkB,CAAC;AAIpD,OAAO,EAAE,sBAAsB,EAAE,MAAM,aAAa,CAAC;AACrD,YAAY,EACX,cAAc,EACd,uBAAuB,EACvB,kBAAkB,EAClB,gBAAgB,EAChB,QAAQ,EACR,qBAAqB,EACrB,gBAAgB,EAChB,YAAY,EACZ,YAAY,EACZ,WAAW,EACX,UAAU,EACV,aAAa,GACb,MAAM,YAAY,CAAC;AAEpB,OAAO,KAAK,EACX,cAAc,EACd,uBAAuB,EACvB,gBAAgB,EAChB,WAAW,EACX,UAAU,EACV,aAAa,EAGb,QAAQ,EAER,MAAM,YAAY,CAAC;AAEpB,OAAO,KAAK,EAAgB,YAAY,EAAE,MAAM,YAAY,CAAC;AAE7D,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,uBAAuB,CAAC;AAwBzD,qBAAa,gBAAgB,CAC5B,CAAC,EACD,CAAC,SAAS,cAAc,CAAC,CAAC,CAAC,GAAG,cAAc,CAAC,CAAC,CAAC,EAC/C,OAAO,SAAS,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC,CAAC,CAAC,GAAG,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC,CAAC,CAAC,CACpF,SAAQ,KAAK;IAWqC,OAAO,EAAE,uBAAuB,CAAC,CAAC,EAAE,CAAC,EAAE,OAAO,CAAC;IAVlG,QAAQ,CAAC,MAAM,sBAAW;IAC1B,OAAO,CAAC,MAAM,CAAsB;IACpC,OAAO,CAAC,SAAS,CAAI;IACrB,OAAO,CAAC,OAAO,CAAU;IACzB,OAAO,CAAC,cAAc,CAAqB;IAC3C,OAAO,CAAC,SAAS,CAAyB;IAE1C,0FAA0F;IAC1F,SAAS,CAAC,GAAG,EAAE,WAAW,CAAC;gBAEf,KAAK,EAAE,WAAW,EAAE,EAAE,EAAE,MAAM,EAAS,OAAO,EAAE,uBAAuB,CAAC,CAAC,EAAE,CAAC,EAAE,OAAO,CAAC;IAkB5F,GAAG,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC,EAAE,CAAC,CAAC,GAAG,OAAO,CAAC,CAAC,GAAG,IAAI,CAAC;IAQ3C,GAAG,CAAC,IAAI,EAAE,CAAC,EAAE,OAAO,CAAC,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC;IAkBpD,MAAM,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,OAAO,CAAC,EAAE,aAAa,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC;IAa5E;;;;;;;;OAQG;IACI,KAAK,CACX,OAAO,EAAE,YAAY,CAAC,CAAC,EAAE,CAAC,EAAE,OAAO,CAAC,GAClC,aAAa,CAAC,CAAC,CAAC;IAsCZ,IAAI,CAAC,OAAO,CAAC,EAAE,WAAW,GAAG,aAAa,CAAC,CAAC,CAAC;IAoB9C,QAAQ,CAAC,IAAI,EAAE,QAAQ,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,GAAG,OAAO,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,EAAE,CAAC;IAwBvD,QAAQ,CAAC,KAAK,EAAE,CAAC,EAAE,GAAG,OAAO,CAAC,IAAI,CAAC;IAsBnC,WAAW,CAAC,IAAI,EAAE,QAAQ,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,GAAG,OAAO,CAAC,IAAI,CAAC;IAiBxD,MAAM,CAAC,YAAY,CAAC,SAAS,EAAE,MAAM,GAAG,gBAAgB;YAM1C,YAAY;IAQ1B;;;;;OAKG;IACH,OAAO,CAAC,aAAa;IAIrB,OAAO,CAAC,QAAQ;IAMhB,OAAO,CAAC,OAAO;IAWf;;;;;;;;;;;;;;;OAeG;YACW,gBAAgB;IAkB9B,OAAO,CAAC,yBAAyB;IAkBjC,OAAO,CAAC,iBAAiB;CA4CzB"}
|
package/dist/index.aws.js
CHANGED
|
@@ -5,7 +5,7 @@ import { DynamoDBDocumentClient, GetCommand, PutCommand, DeleteCommand, QueryCom
|
|
|
5
5
|
import { Scope, registerSdkIdentifiers, getSdkIdentifiers } from '@aws-blocks/core';
|
|
6
6
|
import { BB_NAME, BB_VERSION } from './version.js';
|
|
7
7
|
export { DistributedTableErrors } from './errors.js';
|
|
8
|
-
import { DistributedTableErrors, DistributedTableMessages, blocksError, normalizeSortKeyCondition, remapItemTooLarge } from './errors.js';
|
|
8
|
+
import { DistributedTableErrors, DistributedTableMessages, blocksError, normalizeSortKeyCondition, remapItemTooLarge, applyReadValidation } from './errors.js';
|
|
9
9
|
import { Logger } from '@aws-blocks/bb-logger';
|
|
10
10
|
// ── Helpers ─────────────────────────────────────────────────────────────────
|
|
11
11
|
// DynamoDB batch API limits and retry tuning.
|
|
@@ -32,17 +32,22 @@ export class DistributedTable extends Scope {
|
|
|
32
32
|
schema;
|
|
33
33
|
keyConfig;
|
|
34
34
|
indexes;
|
|
35
|
+
readValidation;
|
|
35
36
|
docClient;
|
|
36
|
-
/** @internal Logger for internal operations. Defaults to
|
|
37
|
+
/** @internal Logger for internal operations. Defaults to warn-level when not provided. */
|
|
37
38
|
log;
|
|
38
39
|
constructor(scope, id, options) {
|
|
39
40
|
super(id, { parent: scope, bbName: BB_NAME, bbVersion: BB_VERSION });
|
|
40
41
|
this.options = options;
|
|
41
|
-
|
|
42
|
+
// Default level is 'warn' (not 'error') so the readValidation='coerce'
|
|
43
|
+
// raw-fallback warning actually surfaces — it's the only log the block
|
|
44
|
+
// emits, so this doesn't add noise. Callers can pass their own logger.
|
|
45
|
+
this.log = options?.logger ?? new Logger(this, 'logger', { level: 'warn' });
|
|
42
46
|
const tableName = options.table?.tableName ?? this.fullId.substring(0, 255);
|
|
43
47
|
this.schema = options.schema;
|
|
44
48
|
this.keyConfig = options.key;
|
|
45
49
|
this.indexes = (options.indexes ?? {});
|
|
50
|
+
this.readValidation = options.readValidation ?? 'coerce';
|
|
46
51
|
const client = new DynamoDBClient({
|
|
47
52
|
customUserAgent: this.buildUserAgentChain(),
|
|
48
53
|
});
|
|
@@ -54,7 +59,7 @@ export class DistributedTable extends Scope {
|
|
|
54
59
|
TableName: getSdkIdentifiers(this).tableName,
|
|
55
60
|
Key: this.buildKey(key),
|
|
56
61
|
}));
|
|
57
|
-
return result.Item ?? null;
|
|
62
|
+
return this.reconcileRead(result.Item ?? null);
|
|
58
63
|
}
|
|
59
64
|
async put(item, options) {
|
|
60
65
|
await this.validateItem(item);
|
|
@@ -119,7 +124,7 @@ export class DistributedTable extends Scope {
|
|
|
119
124
|
const command = this.buildQueryCommand(options.index, pkField, pkValue, skField, skCondition, lastEvaluatedKey, options);
|
|
120
125
|
const result = await this.docClient.send(command);
|
|
121
126
|
for (const item of result.Items ?? []) {
|
|
122
|
-
yield item;
|
|
127
|
+
yield (await this.reconcileRead(item));
|
|
123
128
|
if (options.limit && ++count >= options.limit)
|
|
124
129
|
return;
|
|
125
130
|
}
|
|
@@ -136,7 +141,7 @@ export class DistributedTable extends Scope {
|
|
|
136
141
|
Limit: options?.limit,
|
|
137
142
|
}));
|
|
138
143
|
for (const item of result.Items ?? []) {
|
|
139
|
-
yield item;
|
|
144
|
+
yield (await this.reconcileRead(item));
|
|
140
145
|
if (options?.limit && ++count >= options.limit)
|
|
141
146
|
return;
|
|
142
147
|
}
|
|
@@ -157,7 +162,7 @@ export class DistributedTable extends Scope {
|
|
|
157
162
|
return resp.UnprocessedKeys?.[tableName]?.Keys;
|
|
158
163
|
});
|
|
159
164
|
}
|
|
160
|
-
return keys.map(key => results.get(JSON.stringify(this.buildKey(key))) ?? null);
|
|
165
|
+
return Promise.all(keys.map(key => this.reconcileRead(results.get(JSON.stringify(this.buildKey(key))) ?? null)));
|
|
161
166
|
}
|
|
162
167
|
async putBatch(items) {
|
|
163
168
|
for (const item of items)
|
|
@@ -199,6 +204,15 @@ export class DistributedTable extends Scope {
|
|
|
199
204
|
throw blocksError(DistributedTableErrors.ValidationFailed, resolved.issues[0].message);
|
|
200
205
|
}
|
|
201
206
|
}
|
|
207
|
+
/**
|
|
208
|
+
* Reconcile a stored value with the schema per this table's `readValidation`
|
|
209
|
+
* mode (`off` → raw, `coerce` → coerced output / raw+warn on failure, `strict`
|
|
210
|
+
* → throw on mismatch). `null` (a missing item) passes straight through. See
|
|
211
|
+
* {@link applyReadValidation}.
|
|
212
|
+
*/
|
|
213
|
+
reconcileRead(item) {
|
|
214
|
+
return applyReadValidation(this.readValidation, this.schema, item, this.log, { table: this.fullId });
|
|
215
|
+
}
|
|
202
216
|
buildKey(key) {
|
|
203
217
|
const result = { [this.keyConfig.partitionKey]: key[this.keyConfig.partitionKey] };
|
|
204
218
|
if (this.keyConfig.sortKey)
|
|
@@ -211,7 +225,7 @@ export class DistributedTable extends Scope {
|
|
|
211
225
|
// and lets concurrent callers re-collide; equal jitter preserves a minimum
|
|
212
226
|
// spacing while still de-synchronising retries under shared throttling.
|
|
213
227
|
// See: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Programming.Errors.html#Programming.Errors.BatchOperations
|
|
214
|
-
const capped = Math.min(BASE_BACKOFF_MS *
|
|
228
|
+
const capped = Math.min(BASE_BACKOFF_MS * 2 ** attempt, MAX_BACKOFF_MS);
|
|
215
229
|
const ms = capped / 2 + Math.random() * (capped / 2);
|
|
216
230
|
return new Promise(resolve => setTimeout(resolve, ms));
|
|
217
231
|
}
|
package/dist/index.cdk.d.ts
CHANGED
|
@@ -2,7 +2,7 @@ import { Scope } from '@aws-blocks/core/cdk';
|
|
|
2
2
|
import type { ScopeParent } from '@aws-blocks/core';
|
|
3
3
|
import type { ExternalTableRef } from './types.js';
|
|
4
4
|
export { DistributedTableErrors } from './errors.js';
|
|
5
|
-
export type { DistributedTableOptions, TableKeyConfig, TableKey, PutOptions, DeleteOptions, QueryOptions, ScanOptions, ExternalTableRef } from './types.js';
|
|
5
|
+
export type { DistributedTableOptions, ReadValidationMode, TableKeyConfig, TableKey, PutOptions, DeleteOptions, QueryOptions, ScanOptions, ExternalTableRef } from './types.js';
|
|
6
6
|
export declare class DistributedTable<T = any> extends Scope {
|
|
7
7
|
options: any;
|
|
8
8
|
private table;
|
package/dist/index.cdk.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.cdk.d.ts","sourceRoot":"","sources":["../src/index.cdk.ts"],"names":[],"mappings":"AAUA,OAAO,EAAE,KAAK,EAAoC,MAAM,sBAAsB,CAAC;AAC/E,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,kBAAkB,CAAC;AACpD,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAInD,OAAO,EAAE,sBAAsB,EAAE,MAAM,aAAa,CAAC;AACrD,YAAY,EAAE,uBAAuB,EAAE,cAAc,EAAE,QAAQ,EAAE,UAAU,EAAE,aAAa,EAAE,YAAY,EAAE,WAAW,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;
|
|
1
|
+
{"version":3,"file":"index.cdk.d.ts","sourceRoot":"","sources":["../src/index.cdk.ts"],"names":[],"mappings":"AAUA,OAAO,EAAE,KAAK,EAAoC,MAAM,sBAAsB,CAAC;AAC/E,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,kBAAkB,CAAC;AACpD,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAInD,OAAO,EAAE,sBAAsB,EAAE,MAAM,aAAa,CAAC;AACrD,YAAY,EAAE,uBAAuB,EAAE,kBAAkB,EAAE,cAAc,EAAE,QAAQ,EAAE,UAAU,EAAE,aAAa,EAAE,YAAY,EAAE,WAAW,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAEhL,qBAAa,gBAAgB,CAAC,CAAC,GAAG,GAAG,CAAE,SAAQ,KAAK;IAcA,OAAO,EAAE,GAAG;IAb/D,OAAO,CAAC,KAAK,CAAS;IAEtB;;;;;;OAMG;IACH,MAAM,CAAC,YAAY,CAAC,SAAS,EAAE,MAAM,GAAG,gBAAgB;gBAI5C,KAAK,EAAE,WAAW,EAAE,EAAE,EAAE,MAAM,EAAS,OAAO,EAAE,GAAG;IAmG/D,GAAG,CAAC,GAAG,KAAK,EAAE,OAAO,EAAE,GAAG,KAAK;IAC/B,GAAG,CAAC,GAAG,KAAK,EAAE,OAAO,EAAE,GAAG,KAAK;IAC/B,MAAM,CAAC,GAAG,KAAK,EAAE,OAAO,EAAE,GAAG,KAAK;IAClC,KAAK,CAAC,GAAG,KAAK,EAAE,OAAO,EAAE,GAAG,KAAK;IACjC,IAAI,CAAC,GAAG,KAAK,EAAE,OAAO,EAAE,GAAG,KAAK;IAChC,QAAQ,CAAC,GAAG,KAAK,EAAE,OAAO,EAAE,GAAG,KAAK;IACpC,QAAQ,CAAC,GAAG,KAAK,EAAE,OAAO,EAAE,GAAG,KAAK;IACpC,WAAW,CAAC,GAAG,KAAK,EAAE,OAAO,EAAE,GAAG,KAAK;CACvC"}
|
package/dist/index.mock.d.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { Scope } from '@aws-blocks/core';
|
|
2
2
|
import type { ScopeParent } from '@aws-blocks/core';
|
|
3
3
|
export { DistributedTableErrors } from './errors.js';
|
|
4
|
-
export type { TableKeyConfig, DistributedTableOptions, ExternalTableRef, TableKey, PartitionKeyCondition, SortKeyCondition, KeyCondition, QueryOptions, ScanOptions, PutOptions, DeleteOptions, } from './types.js';
|
|
4
|
+
export type { TableKeyConfig, DistributedTableOptions, ReadValidationMode, ExternalTableRef, TableKey, PartitionKeyCondition, SortKeyCondition, KeyCondition, QueryOptions, ScanOptions, PutOptions, DeleteOptions, } from './types.js';
|
|
5
5
|
import type { TableKeyConfig, DistributedTableOptions, ExternalTableRef, ScanOptions, PutOptions, DeleteOptions, TableKey } from './types.js';
|
|
6
6
|
/**
|
|
7
7
|
* Structured data storage backed by DynamoDB with secondary indexes and
|
|
@@ -32,10 +32,18 @@ export declare class DistributedTable<T, K extends TableKeyConfig<T> = TableKeyC
|
|
|
32
32
|
private schema;
|
|
33
33
|
private keyConfig;
|
|
34
34
|
private indexes;
|
|
35
|
-
|
|
35
|
+
private readValidation;
|
|
36
|
+
/** @internal Logger for internal operations. Defaults to warn-level when not provided. */
|
|
36
37
|
protected log: ChildLogger;
|
|
37
38
|
constructor(scope: ScopeParent, id: string, options: DistributedTableOptions<T, K, Indexes>);
|
|
38
39
|
get(key: TableKey<T, K>): Promise<T | null>;
|
|
40
|
+
/**
|
|
41
|
+
* Reconcile a stored value with the schema per this table's `readValidation`
|
|
42
|
+
* mode (`off` → raw, `coerce` → coerced output / raw+warn on failure, `strict`
|
|
43
|
+
* → throw on mismatch). `null` (a missing item) passes straight through. See
|
|
44
|
+
* {@link applyReadValidation}.
|
|
45
|
+
*/
|
|
46
|
+
private reconcileRead;
|
|
39
47
|
put(item: T, options?: PutOptions<T>): Promise<void>;
|
|
40
48
|
delete(key: TableKey<T, K>, options?: DeleteOptions<T>): Promise<void>;
|
|
41
49
|
/**
|
package/dist/index.mock.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.mock.d.ts","sourceRoot":"","sources":["../src/index.mock.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,KAAK,EAA0B,MAAM,kBAAkB,CAAC;AAEjE,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,kBAAkB,CAAC;AAMpD,OAAO,EAAE,sBAAsB,EAAE,MAAM,aAAa,CAAC;AACrD,YAAY,EACX,cAAc,EACd,uBAAuB,EACvB,gBAAgB,EAChB,QAAQ,EACR,qBAAqB,EACrB,gBAAgB,EAChB,YAAY,EACZ,YAAY,EACZ,WAAW,EACX,UAAU,EACV,aAAa,GACb,MAAM,YAAY,CAAC;AAEpB,OAAO,KAAK,EACX,cAAc,EACd,uBAAuB,EACvB,gBAAgB,EAEhB,WAAW,EACX,UAAU,EACV,aAAa,EACb,QAAQ,
|
|
1
|
+
{"version":3,"file":"index.mock.d.ts","sourceRoot":"","sources":["../src/index.mock.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,KAAK,EAA0B,MAAM,kBAAkB,CAAC;AAEjE,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,kBAAkB,CAAC;AAMpD,OAAO,EAAE,sBAAsB,EAAE,MAAM,aAAa,CAAC;AACrD,YAAY,EACX,cAAc,EACd,uBAAuB,EACvB,kBAAkB,EAClB,gBAAgB,EAChB,QAAQ,EACR,qBAAqB,EACrB,gBAAgB,EAChB,YAAY,EACZ,YAAY,EACZ,WAAW,EACX,UAAU,EACV,aAAa,GACb,MAAM,YAAY,CAAC;AAEpB,OAAO,KAAK,EACX,cAAc,EACd,uBAAuB,EACvB,gBAAgB,EAEhB,WAAW,EACX,UAAU,EACV,aAAa,EACb,QAAQ,EAER,MAAM,YAAY,CAAC;AA4DpB;;;;;;;;;;;;;;;;;;;;;GAqBG;AACH,qBAAa,gBAAgB,CAC5B,CAAC,EACD,CAAC,SAAS,cAAc,CAAC,CAAC,CAAC,GAAG,cAAc,CAAC,CAAC,CAAC,EAC/C,OAAO,SAAS,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC,CAAC,CAAC,GAAG,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC,CAAC,CAAC,CACpF,SAAQ,KAAK;IAWqC,OAAO,EAAE,uBAAuB,CAAC,CAAC,EAAE,CAAC,EAAE,OAAO,CAAC;IAVlG,OAAO,CAAC,QAAQ,CAAS;IACzB,OAAO,CAAC,IAAI,CAAiB;IAC7B,OAAO,CAAC,MAAM,CAAsB;IACpC,OAAO,CAAC,SAAS,CAAI;IACrB,OAAO,CAAC,OAAO,CAAU;IACzB,OAAO,CAAC,cAAc,CAAqB;IAE3C,0FAA0F;IAC1F,SAAS,CAAC,GAAG,EAAE,WAAW,CAAC;gBAEf,KAAK,EAAE,WAAW,EAAE,EAAE,EAAE,MAAM,EAAS,OAAO,EAAE,uBAAuB,CAAC,CAAC,EAAE,CAAC,EAAE,OAAO,CAAC;IAe5F,GAAG,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC,EAAE,CAAC,CAAC,GAAG,OAAO,CAAC,CAAC,GAAG,IAAI,CAAC;IAIjD;;;;;OAKG;IACH,OAAO,CAAC,aAAa;IAIf,GAAG,CAAC,IAAI,EAAE,CAAC,EAAE,OAAO,CAAC,EAAE,UAAU,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC;IAqBpD,MAAM,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,OAAO,CAAC,EAAE,aAAa,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC;IAc5E;;;;;;;;;;;;;;;;;;;;;;;;OAwBG;IACI,KAAK,CACX,OAAO,EAAE,YAAY,CAAC,CAAC,EAAE,CAAC,EAAE,OAAO,CAAC,GAClC,aAAa,CAAC,CAAC,CAAC;IAoDZ,IAAI,CAAC,OAAO,CAAC,EAAE,WAAW,GAAG,aAAa,CAAC,CAAC,CAAC;IAQpD;;;;;;;OAOG;IACG,QAAQ,CAAC,IAAI,EAAE,QAAQ,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,GAAG,OAAO,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,EAAE,CAAC;IAM7D;;;;;;OAMG;IACG,QAAQ,CAAC,KAAK,EAAE,CAAC,EAAE,GAAG,OAAO,CAAC,IAAI,CAAC;IAczC;;;;;;OAMG;IACG,WAAW,CAAC,IAAI,EAAE,QAAQ,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,GAAG,OAAO,CAAC,IAAI,CAAC;IAKxD,MAAM,CAAC,YAAY,CAAC,SAAS,EAAE,MAAM,GAAG,gBAAgB;IAMxD,OAAO,CAAC,gBAAgB;IAiBxB,OAAO,CAAC,YAAY;IAMpB,OAAO,CAAC,YAAY;IAOpB,OAAO,CAAC,WAAW;CAGnB;AAOD,OAAO,KAAK,EAAgE,YAAY,EAAE,MAAM,YAAY,CAAC;AAE7G,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,uBAAuB,CAAC"}
|
package/dist/index.mock.js
CHANGED
|
@@ -6,7 +6,7 @@ import { existsSync, readFileSync, writeFileSync } from 'node:fs';
|
|
|
6
6
|
import { join } from 'node:path';
|
|
7
7
|
import { BB_NAME, BB_VERSION } from './version.js';
|
|
8
8
|
export { DistributedTableErrors } from './errors.js';
|
|
9
|
-
import { DistributedTableErrors, DistributedTableMessages, blocksError, normalizeSortKeyCondition } from './errors.js';
|
|
9
|
+
import { DistributedTableErrors, DistributedTableMessages, blocksError, normalizeSortKeyCondition, applyReadValidation } from './errors.js';
|
|
10
10
|
// ── Helpers ─────────────────────────────────────────────────────────────────
|
|
11
11
|
const MAX_ITEM_BYTES = 400 * 1024;
|
|
12
12
|
async function validateSchema(schema, value) {
|
|
@@ -94,21 +94,35 @@ export class DistributedTable extends Scope {
|
|
|
94
94
|
schema;
|
|
95
95
|
keyConfig;
|
|
96
96
|
indexes;
|
|
97
|
-
|
|
97
|
+
readValidation;
|
|
98
|
+
/** @internal Logger for internal operations. Defaults to warn-level when not provided. */
|
|
98
99
|
log;
|
|
99
100
|
constructor(scope, id, options) {
|
|
100
101
|
super(id, { parent: scope, bbName: BB_NAME, bbVersion: BB_VERSION });
|
|
101
102
|
this.options = options;
|
|
102
|
-
|
|
103
|
+
// Default level is 'warn' (not 'error') so the readValidation='coerce'
|
|
104
|
+
// raw-fallback warning actually surfaces — it's the only log the block
|
|
105
|
+
// emits, so this doesn't add noise. Callers can pass their own logger.
|
|
106
|
+
this.log = options?.logger ?? new Logger(this, 'logger', { level: 'warn' });
|
|
103
107
|
this.filePath = join(getMockDataDir(this), 'data.json');
|
|
104
108
|
this.data = this.loadFromDisk();
|
|
105
109
|
this.schema = options.schema;
|
|
106
110
|
this.keyConfig = options.key;
|
|
107
111
|
this.indexes = (options.indexes ?? {});
|
|
112
|
+
this.readValidation = options.readValidation ?? 'coerce';
|
|
108
113
|
registerSdkIdentifiers(this.fullId, { tableName: `mock-${this.fullId}`.substring(0, 255) });
|
|
109
114
|
}
|
|
110
115
|
async get(key) {
|
|
111
|
-
return this.data.get(this.serializeKey(key)) ?? null;
|
|
116
|
+
return this.reconcileRead(this.data.get(this.serializeKey(key)) ?? null);
|
|
117
|
+
}
|
|
118
|
+
/**
|
|
119
|
+
* Reconcile a stored value with the schema per this table's `readValidation`
|
|
120
|
+
* mode (`off` → raw, `coerce` → coerced output / raw+warn on failure, `strict`
|
|
121
|
+
* → throw on mismatch). `null` (a missing item) passes straight through. See
|
|
122
|
+
* {@link applyReadValidation}.
|
|
123
|
+
*/
|
|
124
|
+
reconcileRead(item) {
|
|
125
|
+
return applyReadValidation(this.readValidation, this.schema, item, this.log, { table: this.fullId });
|
|
112
126
|
}
|
|
113
127
|
async put(item, options) {
|
|
114
128
|
await validateSchema(this.schema, item);
|
|
@@ -203,7 +217,7 @@ export class DistributedTable extends Scope {
|
|
|
203
217
|
}
|
|
204
218
|
let count = 0;
|
|
205
219
|
for (const item of items) {
|
|
206
|
-
yield item;
|
|
220
|
+
yield (await this.reconcileRead(item));
|
|
207
221
|
if (options.limit && ++count >= options.limit)
|
|
208
222
|
return;
|
|
209
223
|
}
|
|
@@ -211,7 +225,7 @@ export class DistributedTable extends Scope {
|
|
|
211
225
|
async *scan(options) {
|
|
212
226
|
let count = 0;
|
|
213
227
|
for (const item of this.data.values()) {
|
|
214
|
-
yield item;
|
|
228
|
+
yield (await this.reconcileRead(item));
|
|
215
229
|
if (options?.limit && ++count >= options.limit)
|
|
216
230
|
return;
|
|
217
231
|
}
|
|
@@ -225,7 +239,7 @@ export class DistributedTable extends Scope {
|
|
|
225
239
|
* sustained throttling. The local mock never throttles, so it does not throw this.
|
|
226
240
|
*/
|
|
227
241
|
async getBatch(keys) {
|
|
228
|
-
return keys.map(key => this.data.get(this.serializeKey(key)) ?? null);
|
|
242
|
+
return Promise.all(keys.map(key => this.reconcileRead(this.data.get(this.serializeKey(key)) ?? null)));
|
|
229
243
|
}
|
|
230
244
|
/**
|
|
231
245
|
* Write multiple items in batches. Each item is schema-validated first.
|