graphddb 0.4.2 → 0.5.0

package/docs/testing.md

@@ -0,0 +1,265 @@
+# GraphDDB Testing — In-Memory Test Adapter (`graphddb/testing`)
+
+GraphDDB ships an in-process test adapter that runs the full engine — planner,
+key resolver, projection, hydrator, relation traversal, conditional/transactional
+writes, and CDC — against an in-memory store. It requires **no DynamoDB and no
+Docker**, so model logic, query plans, relation traversal, and change-data-capture
+can be unit-tested in milliseconds.
+
+This is the **recommended path for unit tests**. Integration tests that must
+verify exact DynamoDB marshalling and engine-level behavior still run against
+DynamoDB Local.
+
+```
+unit test         → graphddb/testing (in-memory)   no Docker, fast
+integration test  → DynamoDB Local / dynalite
+production verify → real DynamoDB
+```
+
+## Purpose and scope
+
+The adapter is **not** a DynamoDB emulator. It is a verifier of GraphDDB's
+*contract*: the access, projection, and change-capture operations the engine
+emits. Concretely, it exercises:
+
+- **Model mapping** — how each model is stored physically (`PK` / `SK` / GSI keys
+  and attributes).
+- **Query plans** — `GetItem`, partition `Query` (PK eq + optional `SK` equality
+  or `begins_with`), GSI `Query`, and `BatchGetItem`.
+- **Relation traversal** — `hasMany` / `belongsTo`, including nested and deep
+  selections.
+- **Projection** — comma-separated projection expressions over dotted paths.
+- **Conditional and transactional writes** — `ConditionExpression` evaluation and
+  all-or-nothing `TransactWrite`.
+- **CDC** — `ChangeEvent`s (INSERT / MODIFY / REMOVE with old/new images) produced
+  by writes, when enabled.
+
+The adapter implements exactly the DynamoDB subset GraphDDB uses — nothing more.
+
+## How it works
+
+GraphDDB funnels all read and write I/O through a single injectable seam, the
+`Executor` interface (`src/executor/types.ts`). The default implementation,
+`DynamoExecutor`, talks to real DynamoDB. Reads receive a structured
+`DynamoDBOperation`; writes receive the structured inputs the operation layer
+already builds (`PutInput` / `UpdateInput` / `DeleteInput`, plus batch and
+transaction variants) — never an AWS SDK command, and never a key-condition
+string. Because the seam sits at the *structured operation boundary*, a fake
+needs no expression-string parser.
+
+`createTestDB()` installs a `MemoryExecutor` (`src/memory/memory-executor.ts`)
+over a `MemoryStore` (`src/memory/memory-store.ts`) on the `ClientManager` seam
+via `ClientManager.setExecutor()`. Every `Model.putItem / query / list / updateItem /
+deleteItem / batchGet` call, the planner, hydrator, and relation traversal then run
+unchanged against the in-memory store. `db.testing.close()` (or
+`ClientManager.resetExecutor()`) restores the default `DynamoExecutor`.
+
+The store keeps items in their **physical** form — exactly the bytes GraphDDB
+writes (`PK` / `SK` / `GSI*PK` / `GSI*SK` plus attributes), with no marshalling
+and no hydration. Tables are `Map<PK, Map<SK, Item>>`; sort keys compare
+lexicographically (byte order). GSIs are not stored separately: since GraphDDB
+projects ALL onto its GSIs, a GSI query filters the base partition rows on their
+`{index}PK` / `{index}SK` attributes — equivalent to a maintained ALL-projection
+index that is trivially consistent on every write.
+
+Only the `FilterExpression`, `UpdateExpression`, and `ConditionExpression`
+compiled strings are interpreted (the one boundary where the engine hands the
+executor a compiled string), by dedicated in-memory evaluators in
+`src/memory/filter-eval.ts`, `apply-update.ts`, and `apply-condition.ts`. No
+`ADD` / `DELETE` update clauses are interpreted because GraphDDB never emits
+them.
+
+## Using it
+
+```ts
+import { createTestDB } from 'graphddb/testing';
+import { User, Group, GroupMembership } from './models';
+
+it('loads group permissions through relation traversal', async () => {
+  const db = createTestDB({ models: [User, Group, GroupMembership] });
+  try {
+    await Group.putItem({ groupId: 'eng', name: 'Engineering' });
+    await GroupMembership.putItem({ groupId: 'eng', userId: 'alice', role: 'admin', seats: 1 });
+
+    const group = await Group.query(
+      { groupId: 'eng' },
+      { name: true, permissions: { select: { permissionId: true, action: true } } },
+    );
+    expect(group?.name).toBe('Engineering');
+  } finally {
+    db.testing.close();
+  }
+});
+```
+
+`createTestDB(options?)` returns a `TestDB`:
+
+```ts
+interface TestDB {
+  testing: MemoryInspector; // inspection / fixtures / isolation
+  store: MemoryStore;       // backing store (advanced use)
+}
+
+interface CreateTestDBOptions {
+  // Models whose physical table name is addressable in the inspector by logical
+  // name, and (with cdc: true) are stream-enabled. Optional: the inspector also
+  // reflects any table the store has written to.
+  models?: (ModelStatic | Function)[];
+  // Enable the CDC emulator (record mode) over the configured models.
+  cdc?: boolean;
+}
+```
+
+`createTestDB()` is the recommended entry point. For callers that wire the seam
+manually, `graphddb/testing` also exports `MemoryExecutor` and `MemoryStore`
+(install with `ClientManager.setExecutor(new MemoryExecutor(store))`).
+
+Tests must call `db.testing.close()` when done (typically in `afterEach`) to
+detach CDC and restore the default executor. `db.testing.reset()` clears the
+store between tests.
+
+## `MemoryInspector` API
+
+`db.testing` is a `MemoryInspector` (`src/testing/index.ts`). It is a direct,
+**hydration-free** view of the store for assertions, fixtures, and isolation.
+`dump` / `dumpTable` / `scan` / `getRaw` return **physical items** — the keys and
+attributes exactly as stored — so model mapping can be asserted without going
+through the engine. The system-under-test still uses `Model.query` / `Model.list`.
+
+Table arguments accept either a model's logical table name (resolved to its
+physical name for configured models) or a physical name directly.
+
+```ts
+interface MemoryInspector {
+  // Every physical item, keyed by physical table name.
+  dump(): Record<string, Item[]>;
+  // Every physical item in one table.
+  dumpTable(table: string): Item[];
+  // Raw physical rows, optionally filtered by PK / SK-prefix / GSI partition.
+  scan(table: string, filter?: ScanFilter): Item[];
+  // Item count for a table.
+  count(table: string): number;
+  // One physical item by base-table key, or null.
+  getRaw(table: string, key: { PK: string; SK: string }): Item | null;
+  // Insert physical items directly (fixtures).
+  seed(table: string, items: Item[]): void;
+  // Clear every table (and reset captured changes).
+  reset(): void;
+  // Structured-clone the whole store (save).
+  snapshot(): MemorySnapshot;
+  // Restore a snapshot (load).
+  restore(snapshot: MemorySnapshot): void;
+  // ChangeEvents captured so far (requires cdc: true; throws otherwise).
+  changes(): ChangeEvent[];
+  // Tear down: restore the default executor and detach CDC.
+  close(): void;
+}
+
+interface ScanFilter {
+  pk?: string;        // match the base-table PK exactly (or {index}PK with `index`)
+  skPrefix?: string;  // match an SK prefix (or {index}SK prefix with `index`)
+  index?: string;     // inspect a GSI partition instead of the base table, e.g. 'GSI1'
+}
+
+type Item = Record<string, unknown>;
+type MemorySnapshot = Record<string, Item[]>;
+```
+
+### Asserting physical layout
+
+```ts
+await GroupMembership.putItem({ groupId: 'eng', userId: 'alice', role: 'admin', seats: 3 });
+
+expect(db.testing.dumpTable('AppTable')).toContainEqual(
+  expect.objectContaining({
+    PK: 'GROUP#eng',
+    SK: 'USER#alice',
+    GSI1PK: 'USER#alice',
+    GSI1SK: 'MEMBERSHIP',
+    role: 'admin',
+    seats: 3,
+  }),
+);
+
+// Scan a base partition, an SK prefix, or a GSI partition.
+db.testing.scan('AppTable', { pk: 'GROUP#eng' });
+db.testing.scan('AppTable', { pk: 'GROUP#eng', skPrefix: 'USER#a' });
+db.testing.scan('AppTable', { index: 'GSI1', pk: 'USER#alice' });
+```
+
+### Fixtures and isolation
+
+```ts
+// Seed physical items directly (bypasses the model API); they are queryable
+// through the normal model API afterward.
+db.testing.seed('AppTable', [
+  { PK: 'GROUP#eng', SK: 'META', groupId: 'eng', name: 'Engineering' },
+]);
+
+// snapshot/restore is a deep (structured) clone — isolate state across mutations.
+const base = db.testing.snapshot();
+// …mutate…
+db.testing.restore(base); // no pollution carried to the next test
+```
+
+## `explain()` parity
+
+The query planner produces an `ExecutionPlan` independently of which executor is
+installed, so `Model.explain()` returns the **same plan** with or without the
+memory adapter. Plans can therefore be asserted under the in-memory config:
+
+```ts
+const plan = User.explain(
+  { userId: 'u1' },
+  { select: { memberships: { select: { group: { select: { name: true } } } } } },
+);
+plan.operations.map((op) => op.type); // e.g. ['GetItem', 'Query', 'BatchGetItem', ...]
+```
+
+## CDC in-memory
+
+With `{ cdc: true }`, `createTestDB()` attaches a CDC emulator (record mode) over
+the configured models. Writes flowing through the memory executor produce
+`ChangeEvent`s readable via `db.testing.changes()`. The emulator derives
+INSERT / MODIFY / REMOVE and the old/new images from each write's pre-image
+(the executor returns `oldItem` when the caller requests `ReturnValues: ALL_OLD`).
+
+```ts
+const db = createTestDB({ models: [Group], cdc: true });
+try {
+  await Group.putItem({ groupId: 'eng', name: 'Engineering' });
+
+  const events = db.testing.changes();
+  expect(events[0]).toMatchObject({
+    eventName: 'INSERT',
+    keys: { pk: 'GROUP#eng', sk: 'META' },
+  });
+  expect(events[0].newImage).toMatchObject({ name: 'Engineering' });
+} finally {
+  db.testing.close();
+}
+```
+
+`ChangeEvent` mirrors DynamoDB Streams `NEW_AND_OLD_IMAGES` (`eventName`, `table`,
+`keys`, `oldImage`, `newImage`, `approximateCreationTime`, `sequenceNumber`,
+`shardId`), so a consumer written against the emulator runs unchanged against
+real Streams. Overwriting puts yield `MODIFY` with an `oldImage`; deletes yield
+`REMOVE` with an `oldImage` and no `newImage`. Calling `changes()` without
+`{ cdc: true }` throws.
+
+## Fidelity non-goals
+
+The adapter reproduces GraphDDB's contract, not DynamoDB's full behavior.
+Anything that depends on exact DynamoDB semantics belongs in integration tests
+(DynamoDB Local). The adapter does **not** reproduce:
+
+- RCU/WCU, throttling, or partition splitting.
+- TTL real-time expiry.
+- Real DynamoDB Streams delivery (timing, shard mechanics, retry).
+- IAM, the AWS SDK surface, or partition-size limits.
+- Exact marshalling of floats, big numbers, and binary (the DocumentClient may
+  differ at the type boundary).
+
+Sort keys compare as **byte-order lexicographic** strings; numeric key
+zero-padding is the model's responsibility, matching real DynamoDB. `ADD` /
+`DELETE` update clauses are not interpreted (GraphDDB never emits them).

package/package.json

@@ -1,12 +1,13 @@
 {
   "name": "graphddb",
-  "version": "0.4.2",
+  "version": "0.5.0",
   "description": "Graph data modeling on DynamoDB with adjacency list pattern",
   "type": "module",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
   "files": [
-    "dist"
+    "dist",
+    "docs"
   ],
   "exports": {
     ".": {
@@ -36,6 +37,7 @@
     "test:py": "python3 -m pytest python/tests -m \"not integration\"",
     "test:py:integration": "python3 -m pytest python/tests -m integration",
     "test:conformance": "tsx conformance/run.ts",
+    "validate:cfn": "node scripts/validate-cfn.mjs",
     "gen:cli": "cli-contracts generate",
     "gen:cli:check": "cli-contracts generate --dry-run",
     "clean": "rm -rf dist",
@@ -69,7 +71,8 @@
     "tsup": "^8.0.0",
     "tsx": "^4.22.4",
     "typescript": "^5.5.0",
-    "vitest": "^3.0.0"
+    "vitest": "^3.0.0",
+    "yaml": "^2.9.0"
   },
   "engines": {
     "node": ">=22"