graphddb 0.5.3 → 0.7.0

package/docs/design-patterns.md

@@ -39,14 +39,14 @@ graph, the stream drain, and the rebuild planner consume every pattern uniformly
 | # | Pattern | graphddb feature | Status |
 |---|---|---|---|
 | 1 | [Same Partition / Containment](#1-same-partition--containment) | segmented SK + `hasMany`, `pattern: 'samePartition'` | ✅ Phase 1 |
-| 2 | [Embedded Refs](#2-embedded-refs) | `embeddedSnapshot` collection projecting id fields only | ✅ Phase 1 (sync append) / stream for trim |
+| 2 | [Embedded Refs](#2-embedded-refs) | `@list` id-list + `@refs` relation (read = one deduped BatchGet) | ✅ Phase 1 (sync append) / stream for trim |
 | 3 | [Embedded Snapshot](#3-embedded-snapshot) | `pattern: 'embeddedSnapshot'` (snapshot / collection) | ✅ Phase 1 |
 | 4 | [Edge / Adjacency](#4-edge--adjacency) | `edgeWrites` `putEdge` / `deleteEdge` / `updateKeyChangeEdge` | ✅ Phase 1 |
 | 5 | [Materialized View](#5-materialized-view) | `@model({ kind: 'materializedView' })` + `@maintainedFrom` | ✅ Phase 2 (stream) |
 | 6 | [Sparse View](#6-sparse-view) | `@model({ kind: 'sparseView' })` + `@maintainedFrom` + `whenMember` | ✅ Phase 3 (stream-only) |
 | 7 | [Aggregate Counter](#7-aggregate-counter) | `@aggregate(..., { pattern: 'counter', value: count() })` | ✅ Phase 1 (`count()`) / stream (`max()`) |
 | 8 | [Versioned / Latest Pointer](#8-versioned--latest-pointer) | `@hasOne(... versionedLatest)` + `@hasMany(... versionedHistory)` | ✅ Phase 3 |
-| 9 | [Reverse Lookup](#9-reverse-lookup) | GSI (both directions) or `dualEdge` two-row sync | ✅ Phase 1 (GSI) / Phase 3 (`dualEdge`) |
+| 9 | [Reverse Lookup](#9-reverse-lookup) | GSI (both directions) or `dualEdge` two-row sync | ✅ Phase 1 (GSI) / `dualEdge` (#133, command-IF driven #195) |
 | 10 | [External Projection](#10-external-projection) | typed-consumer-IF contract (`@cdcProjected` + `fromChange` / `subscribe`) | ✅ |
 
 ---
@@ -117,31 +117,76 @@ without a query; the child bodies are fetched on demand (e.g. a follow-up
 `BatchGet`). On the table this is an attribute holding a list of id strings/maps on
 the parent item.
 
-**(b) How to declare it.** There is no dedicated `refs` preset; embedded refs are an
-**`embeddedSnapshot` collection that projects only the id field(s)** — the snapshot
-machinery, narrowed to keys. (The RFC marks this pattern ⚠️ precisely because it is
-not its own primitive but a thin projection of the snapshot collection.)
+**(b) How to declare it — write (maintain the id-list) + read (resolve the bodies).**
+Embedded refs have two sides, and each has a first-class declaration.
+
+The **stored id-list** is just a `@list` attribute on the parent; you append to it
+however you maintain the parent (directly, or as a narrowed `embeddedSnapshot`
+collection that projects only the id field). The **read side** is a dedicated
+**`refs` relation** (`@refs`) that resolves that inline id-list into the referenced
+child bodies in ONE deduped `BatchGet`:
 
 ```ts
-@hasMany(() => PostModel, { threadId: 'threadId' }, {
-  pattern: 'embeddedSnapshot',
-  read:  { maxItems: 20, order: 'DESC' },
-  write: { maintainedOn: ['Post.created'], updateMode: 'stream' },
-  projection: { postId: 'postId' },   // ids only — the "refs" shape
-})
-postRefs!: PostModel[];
+@model({ table: TABLE, prefix: 'Post' })
+class PostModel extends DDBModel {
+  static readonly keys = key<{ threadId: string; postId: string }>((c) => ({
+    pk: k`THREAD#${c.threadId}`,
+    sk: k`POST#${c.postId}`,
+  }));
+  @string threadId!: string;
+  @string postId!: string;
+  // The stored inline id-list (ids only — the "refs" shape).
+  @list tagRefs!: { tagId: string }[];
+
+  // READ side: resolve `tagRefs[].tagId` → the `Tag` bodies, one deduped BatchGet.
+  @refs(() => TagModel, { from: 'tagRefs', key: 'tagId' })
+  tags!: TagModel[];
+}
 ```
 
-**(c) Read & write behavior.** The ref list is read **inline** off the parent row
-(it *is* the maintained access path). On `Post.created` the maintainer appends the
-projected `{ postId }` ref; resolving the bodies is a separate `BatchGet` against
-the children's keys.
+`from` names the parent list attribute; `key` is the ref field read off each element
+(add `to` when the target's key field name differs from `key`). The decorated
+property is declared as `TagModel[]`, so it selects and returns like a bounded
+collection.
+
+**(c) Read & write behavior.** The id-list is maintained inline on the parent row
+(it *is* the materialized access path). Resolving the child bodies is a single
+declarative query — the `refs` relation reads the parent, pulls each `tagRefs`
+element's `tagId`, **dedupes**, and hydrates the `Tag` bodies in ONE `BatchGetItem`
+(never a per-ref `GetItem`) — the exact physical cost (parent read + 1 BatchGet) of a
+hand-written two-call `Get` + `BatchGet`, but declarative and typed:
+
+```ts
+// ONE query — parent read + one deduped BatchGet for the referenced tag bodies.
+const post = await Post.query(
+  { threadId, postId },
+  { postId: true, tagRefs: true, tags: { select: { tagId: true, name: true } } },
+);
+// post.tags → { items: { tagId: string; name: string }[]; cursor: null }
+```
 
-**(d) Constraints & phase status.** ✅ The synchronous append is Phase 1; bounding
-the list (`maxItems` trim) or removing a ref on `Post.removed` requires the
-read-modify-write **stream** path (declare `updateMode: 'stream'`), since a
-synchronous collection is append-only. Same payload-only projection rule as
-[Embedded Snapshot](#3-embedded-snapshot).
+`select` projects the child body fields (the result is fully typed to the
+projection); a repeated ref costs nothing extra (deduped into one BatchGet key); a
+ref whose body is absent is skipped. The connection carries a `null` cursor — a refs
+list is finite, not paged. This is the same nested-BatchGet auto-resolution
+`@belongsTo`/`@hasOne` use, extended to fan out over a **parent-row list** rather
+than a scalar key.
+
+**(d) Constraints & phase status.** ✅ The read-side `refs` relation is first-class
+in the **TS in-process runtime** (`Model.query`). Maintaining the id-list
+synchronously (append) is Phase 1; bounding it (`maxItems` trim) or removing a ref
+requires the read-modify-write **stream** path. The child bodies are resolved via
+BatchGet, so a `refs` read's `select` cannot push a server-side `FilterExpression`
+on the children (BatchGet has none); filter on the returned `items` if needed.
+
+> **Runtime scope.** A `refs` read fans a **parent-row list attribute** out into a
+> BatchGet. The static operations spec (`operations.json`) and the generated Python
+> runtime bind relation keys from a single scalar `{result.<field>}` and have no
+> parent-list key-fan-out primitive, so a `refs` relation is resolvable **only**
+> through the TS in-process runtime. Compiling a select that traverses a `refs`
+> relation to `operations.json` / generated Python **fails loudly** (rather than
+> emitting a physically wrong scalar-key BatchGet); resolve it in the generated-code
+> consumer with a separate BatchGet if you need it there.
 
 ---
 
@@ -185,6 +230,53 @@ single mirrored row in place; a `hasMany` collection appends a projected item
 (`list_append`). On the stream path, the collection additionally trims to `maxItems`
 and splices on `removed`.
 
+**Reading the snapshot (type-safe, no join).** The maintained copy lives on the owner
+row, so it is read with a single point read — but it must NOT be confused with a
+**traversal** to the relation's real child rows. GraphDDB keeps the two apart in both
+the types and the runtime:
+
+- **Collection (`@hasMany` embeddedSnapshot).** Select the field with
+  `{ inline: true }` to read the *maintained copy stored on the owner row* — projected
+  off that one row, returned in the field's declared shape, no child Query:
+
+  ```ts
+  // INLINE READ — the maintained latest-N copy on the ThreadSummary owner row.
+  const summary = await ThreadSummary.query(
+    { threadId },
+    { latestPosts: { inline: true } },
+  );
+  summary?.latestPosts;   // → the projected [{ postId, textPreview }, …] off the row
+
+  // TRAVERSAL — a DIFFERENT read: a Query against the real child Post partition,
+  // returning a `{ items, cursor }` connection of the actual child rows.
+  const traversed = await ThreadSummary.query(
+    { threadId },
+    { latestPosts: { select: { postId: true, body: true } } },
+  );
+  traversed?.latestPosts.items;   // → the real child Posts (full bodies), not the copy
+  ```
+
+  `{ inline: true }` (the maintained copy) and `{ select: … }` (a child traversal) are
+  structurally distinct shapes, so the compiler tells them apart; at runtime an
+  `{ inline: true }` on any field that is **not** an `embeddedSnapshot` relation is a
+  loud error, so the inline read can never be silently mistaken for a traversal (or a
+  bogus attribute). The owner row's key fields are encoded in the PK/SK (known from the
+  query key) and are not re-stored as top-level scalars, so an inline read selects the
+  maintained attribute itself.
+
+- **Single-row mirror (`@belongsTo`/`@hasOne` embeddedSnapshot).** The mirror projects
+  its captured fields onto the owner row's **own scalar columns** (the `projection` map
+  keys — e.g. `displayName`, `bioPreview` on the `ProfileCard`). Those are already
+  first-class typed fields, so the mirror is read inline with an ordinary scalar select
+  — no cast, no traversal:
+
+  ```ts
+  const card = await ProfileCard.query(
+    { userId },
+    { displayName: true, bioPreview: true },   // the mirrored snapshot, read inline
+  );
+  ```
+
 **(d) Constraints & phase status.** ✅ Phase 1 for the synchronous path. A
 synchronous collection is **append-only** — `maxItems` trim and `removed`-driven
 splice are the **stream** path ([§7.2](./spec.md#72-stream-maintenance--outbox--cdc-drain-updatemode-stream-130)).
@@ -444,8 +536,18 @@ physical rows** (a forward edge and an inverse edge on its own partition).
 
 **(b) How to declare it.** *GSI form* — declare a `gsi` on the edge that swaps the key
 so the reverse `@hasMany` resolves against it (no extra write; DynamoDB maintains the
-GSI). *Dual-edge form* — `edgeWrites` with `w.dualEdge(forward, inverse)` to keep a
-**two-row** bidirectional edge in sync without an inverse GSI.
+GSI). *Dual-edge form* — declare `w.dualEdge(forward, inverse)` on the forward edge to
+keep a **two-row** bidirectional edge in sync without an inverse GSI. The dual edge can
+be recorded in either of two write vocabularies carrying the SAME `w.dualEdge`:
+
+- **`entityWrites` (recommended when driving through the command / mutation IF, #195)** —
+  place `w.dualEdge(...)` in a lifecycle's `edges` array. The dual edge is then derived by
+  `executeCommandMethod` / `DDBModel.mutate`, so a `mutation` over the edge maintains BOTH
+  rows in the SAME atomic transaction with no low-level handwork. Use this form if the edge
+  is created / removed through a command.
+- **`edgeWrites` (the low-level edge-only primitive)** — the historical adjacency-only
+  `writes` member. Same two-row synchronization, but driven through the low-level edge-write
+  path rather than the command IF.
 
 ```ts
 // GSI form: one row, reverse direction served by a GSI.
@@ -461,9 +563,28 @@ class GroupMembershipModel extends DDBModel {
   @string userId!: string;
 }
 
-// Dual-edge form: two physical rows kept in sync (no inverse GSI).
+// Dual-edge form (command-IF driven, recommended): two physical rows kept in sync
+// (no inverse GSI). `w.dualEdge` sits in a lifecycle's `edges`, so a `mutation` over
+// the edge derives both rows in one atomic transaction (#195).
 @model({ table: 'App', prefix: 'FOLLOW' })
 class FollowEdgeModel extends DDBModel {
+  // … key: FOLLOWER#{a} / FOLLOWING#{b}
+  static readonly writes = entityWrites<FollowEdgeModel>((w) => ({
+    create: w.lifecycle({
+      edges: [
+        w.dualEdge(
+          { target: () => UserModel, relationProperty: 'following' },
+          { adjacency: () => FollowedByEdgeModel, target: () => UserModel, relationProperty: 'followers' },
+        ),
+      ],
+    }),
+  }));
+}
+
+// Dual-edge form (low-level `edgeWrites` primitive): same two-row synchronization,
+// driven through the edge-write path rather than the command IF.
+@model({ table: 'App', prefix: 'FOLLOW' })
+class FollowEdgeModelLowLevel extends DDBModel {
   // … key: FOLLOWER#{a} / FOLLOWING#{b}
   static readonly writes = edgeWrites((w) => [
     w.dualEdge(
@@ -482,8 +603,10 @@ together, so the two directions stay consistent without a GSI.
 
 **(d) Constraints & phase status.** ✅ Phase 1 for the GSI form (the existing
 bidirectional edge: base row + inverse GSI). The two-row `dualEdge` synchronization
-is Phase 3 (#133). A dual edge is *exactly* two rows — a round-trip guard validates
-the inverse half (and rejects pointing both directions at the same adjacency model).
+landed in #133 and is reachable through the command / mutation IF via the recommended
+`entityWrites` form in #195. A dual edge is *exactly* two rows — a round-trip guard
+validates the inverse half (and rejects pointing both directions at the same adjacency
+model).
 
 ---
 

package/docs/prepared-statements.md

@@ -0,0 +1,165 @@
+# GraphDDB Prepared Statements
+
+`DDBModel.prepare($ => ({...}))` → `.execute(params)` is the read/write-unified
+**prepared statement**: a declarative route body is compiled **once**, and each
+`execute(params)` binds the per-call values into the precompiled plan and runs
+it through the **same execution cores** `DDBModel.mutate` / `Model.query` /
+`Model.list` and the public CQRS contracts use — effects are identical by
+construction (design #203, runtime #205, compile-time transform #206).
+
+## The three execution layers (orthogonal, pick per use)
+
+| Use | API |
+|---|---|
+| ad-hoc one-shot | `DDBModel.mutate({...})` / `Model.query(...)` / `Model.list(...)` — recompiles per call |
+| repeated hot path (prepared statement) | `DDBModel.prepare($ => ({...}))` → `.execute(params)` |
+| public CQRS contract | `publicCommandModel({...})` / `publicQueryModel({...})` |
+
+`prepare` is the missing middle: **precompiled without the contract ceremony**.
+
+```ts
+// write
+const createPost = DDBModel.prepare(($) => ({
+  post: { create: () => Post, key: { threadId: $.threadId, postId: $.postId }, input: { body: $.body } },
+}));
+await createPost.execute({ threadId, postId, body });
+
+// read — symmetric; only the key values (+ limit/cursor/consistentRead) are dynamic
+const userById = DDBModel.prepare(($) => ({
+  user: { query: () => User, key: { userId: $.userId }, select: { userId: true, name: true } },
+}));
+await userById.execute({ userId });
+```
+
+## The key constraint: no runtime capture
+
+A prepared body must be **pure-declarative**: it may reference only
+
+- `$.<name>` — per-call values, bound at `execute`;
+- **module-static** references — imports, module-scope `const` / `class` /
+  `enum`, the `() => Model` model refs, shared `const` select templates;
+- static literals (string / number / boolean / bigint / `null`).
+
+Capturing a per-call runtime value into the body — an enclosing function's
+parameter or `let`, `p.foo`, `this`, mutable module state, any helper **call**
+(the indirect-capture channel; even `Date.now()` is a fresh per-call value) —
+breaks preparability and is rejected **loudly**:
+
+- **at build time** by the static lint (`graphddb transform prepared`, below) —
+  the primary, phase-2 enforcement, including indirect capture via same-file
+  helper constants (transitively verified);
+- **at runtime** by the phase-1 guard (`assertNoRuntimeCapture` and the
+  recording `$` proxy) as a backstop for what a per-file syntactic pass cannot
+  see (cross-module mutable state, mutated template objects).
+
+## Zero overhead vs. fallback: the compile-time transform
+
+**With the transform applied, prepared statements are zero-overhead per
+operation regardless of where you write them. Without it, they still behave
+identically — they just pay a small per-call memoization cost.** That is the
+whole contract:
+
+| Build | Per-op cost of an inline `prepare(fn).execute()` | Result |
+|---|---|---|
+| **transform applied** (`graphddb transform prepared --write`) | **zero** planning / compiling / structure hashing — the call site is normalized to a module-scope prepared slot; after the first call it is a nullish check + `.execute` | identical |
+| **no transform** (fallback) | phase-1 **structural memoization**: the body is re-evaluated and structure-hashed per call, then the compiled handle is reused from a bounded LRU (no recompile) | identical |
+| hand-hoisted module-level `const stmt = DDBModel.prepare(...)` | zero (already optimal; the transform leaves it untouched) | identical |
+
+The transform exists to remove the placement footgun: without it, an inline
+`prepare` is *correct but warmer* than a module-level one (silent-slow). With
+it, both spellings compile to the same optimal form.
+
+### CLI
+
+```bash
+# build lint (CI gate): verify no-runtime-capture, report hoistable inline
+# call sites as notes; exit 1 on violations
+graphddb transform prepared --src src/
+
+# apply: rewrite verified inline call sites in place (idempotent)
+graphddb transform prepared --src src/ --write
+```
+
+The rewrite normalizes each inline call site to a module-scope **lazy slot**:
+
+```ts
+// before (inline, per-call structural memoization)
+export async function getUser(userId: string) {
+  return DDBModel.prepare(($) => ({
+    u: { query: () => User, key: { userId: $.userId }, select: USER_SELECT },
+  })).execute({ userId });
+}
+
+// after `graphddb transform prepared --write`
+let __gddbPrepared1;
+export async function getUser(userId: string) {
+  return (__gddbPrepared1 ??= DDBModel.prepare(($) => ({
+    u: { query: () => User, key: { userId: $.userId }, select: USER_SELECT },
+  }))).execute({ userId });
+}
+```
+
+The body stays lexically in place (every reference resolves exactly as
+before — no TDZ / module-init-order hazards with models declared later in the
+file), the plan compiles once per module at first use, and every later call is
+a single nullish check straight into `.execute`. Files containing any
+violation are **never rewritten** — the build fails loudly instead.
+
+A file is scanned when it contains a `DDBModel.prepare` call site; detection
+follows import aliases (`import { DDBModel as M }`), namespace imports
+(`g.DDBModel`), module-const aliases, and same-file `extends DDBModel`
+subclasses. Call sites already evaluated once per module load (module-scope
+`const`, static class fields) are left as-is.
+
+### Programmatic API
+
+```ts
+import { transformPreparedSource, lintPreparedSource } from 'graphddb/transform';
+
+const result = transformPreparedSource(fileName, sourceText); // { text, changed, diagnostics, ... }
+const diagnostics = lintPreparedSource(fileName, sourceText); // check-only (the build gate)
+```
+
+Build-time only (requires the optional `typescript` dependency, like the JSDoc
+codegen pass); wire it into a bundler plugin or a pre-build step if you prefer
+that over the CLI.
+
+### What the static lint proves — and its limits
+
+The lint accepts only the declarative grammar the runtime itself accepts
+(object/array literals, static literals, `$.<name>`, module-static reference
+chains, `() => Model` thunks), applied **transitively** through same-file
+module-const templates. It therefore rejects, per-file and soundly: direct
+captures (enclosing-scope bindings, `this`), indirect captures via **any** call
+or `new`, mutable module state (`let` / `var`), unresolved globals, per-call
+control flow (conditionals, computed keys, substitution templates), and bodies
+it cannot statically see.
+
+Per-file syntactic analysis cannot prove two things: an **imported** binding is
+accepted as module-static (that is the design's allowance for `() => Model`
+refs — a pathological mutable re-export is invisible), and mutation of a
+same-file `const` **object** between calls. Both are backstopped by the
+phase-1 runtime guard (only `$`-refs and scalar static literals are bindable
+value leaves) and by the fallback's correctness: an undetected unhoistable
+site is never wrong, only warmer.
+
+Detection also does **not check the import's module specifier**: a binding
+imported as `DDBModel` from **any** package (`import { DDBModel } from
+'<other-package>'`) matches, so another library's same-named `DDBModel.prepare`
+call site is transformed / linted too (a false transform or a spurious lint
+error is possible). Take care in files that mix graphddb with a same-named
+foreign API.
+
+## Semantics (both layers, guaranteed identical)
+
+- **write** — the params-independent op template compiles once
+  (`compileWriteFragment`); `execute` runs the identical core the envelope and
+  `publicCommandModel` use: derived maintainers and atomic
+  `TransactWriteItems` semantics match exactly. Multi-alias bodies compose one
+  atomic transaction (`mode: 'parallel'` opts out per call).
+- **read** — the params-independent products (resolved model, normalized
+  select, validated relation structure) compute once; `execute` binds key
+  values + per-call pagination/consistency and runs `executeQuery` /
+  `executeList` exactly like `Model.query` / `Model.list`.
+- A body is all-read or all-write; an option the target core would ignore is a
+  loud compile-time reject (never a silent drop).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "graphddb",
-  "version": "0.5.3",
+  "version": "0.7.0",
   "description": "Graph data modeling on DynamoDB with adjacency list pattern",
   "type": "module",
   "main": "dist/index.js",
@@ -29,6 +29,10 @@
     "./cdc": {
       "types": "./dist/cdc/index.d.ts",
       "import": "./dist/cdc/index.js"
+    },
+    "./transform": {
+      "types": "./dist/transform/index.d.ts",
+      "import": "./dist/transform/index.js"
     }
   },
   "bin": {
@@ -56,7 +60,13 @@
     "size": "size-limit",
     "size:why": "size-limit --why",
     "docker:up": "docker compose -f docker-compose.test.yml up -d",
-    "docker:down": "docker compose -f docker-compose.test.yml down"
+    "docker:down": "docker compose -f docker-compose.test.yml down",
+    "bench:docker:up": "docker compose -p graphddb-bench -f docker-compose.bench.yml up -d",
+    "bench:docker:down": "docker compose -p graphddb-bench -f docker-compose.bench.yml down",
+    "embedoc:build": "embedoc build",
+    "embedoc:watch": "embedoc watch",
+    "embedoc:generate": "embedoc generate --all",
+    "embedoc:check": "node scripts/embedoc-drift-check.mjs"
   },
   "size-limit": [
     {
@@ -108,6 +118,8 @@
     "dynamodb-onetable": "^2.7.7",
     "dynamodb-toolbox": "^2.9.0",
     "electrodb": "^3.9.1",
+    "embedoc": "0.14.2",
+    "prettier": "^3.9.4",
     "size-limit": "^11.0.0",
     "tsup": "^8.0.0",
     "tsx": "^4.22.4",