graphddb 0.5.0 → 0.5.2

package/docs/cdc-projection.md

@@ -0,0 +1,167 @@
+# GraphDDB — CDC Projection (typed-consumer-IF)
+
+CDC projection lets you turn a model's change stream into **typed records** you can
+push to an **external sink** — BigQuery, OpenSearch, S3, Neptune — for analytics or
+search, decoupled from the table. Delivery is asynchronous over CDC (DynamoDB Streams)
+and must tolerate at-least-once semantics (a redelivered event must not double-write).
+
+graphddb's job here is narrow and deliberate: **parse a change event into a typed
+record**. It does *not* subscribe to the stream, write to the sink, or de-duplicate —
+those belong to your infrastructure. This keeps the library out of your delivery
+semantics (upsert, idempotency, dedup) while still giving you fully typed access to
+what changed.
+
+## The boundary
+
+graphddb owns the **parse → typed record** contract, and nothing downstream of it.
+
+| graphddb (the contract) | you (the runtime) |
+| --- | --- |
+| Enable CDC parse on a source model (`@cdcProjected`) | Subscribe to the CDC stream (emulator / DynamoDB Streams) |
+| Parse `oldImage` / `newImage` into typed instances | Write to the sink (upsert / delete) |
+| Route an event to its owning model | Idempotency store (dedup / at-least-once) |
+| Produce the `(event) => [old, new]` typed mapper | Actual delivery and retry decisions |
+
+What graphddb hands you is a pure function (and a batch handler built on it) — never a
+subscription, a sink write, or a dedup store.
+
+## API
+
+### 1. `@cdcProjected()` — mark a source model
+
+Enables CDC parse on the model. `fromChange` and `subscribe` typing only apply to a
+`@cdcProjected` model (calling `fromChange` on an un-annotated model is an error). The
+decorator has no runtime effect — it only marks the model as CDC-parseable.
+
+```ts
+import { model, cdcProjected, DDBModel, string, number, datetime, key, k } from 'graphddb';
+
+@model({ table: 'Orders', prefix: 'ORDER' })
+@cdcProjected()
+class OrderModel extends DDBModel {
+  static readonly keys = key<{ orderId: string }>((c) => ({
+    pk: k`ORDER#${c.orderId}`, sk: k`META`,
+  }));
+
+  @string orderId!: string;
+  @string status!: string;
+  @number amountTotal!: number;
+  @datetime updatedAt!: Date;
+}
+```
+
+### 2. `Model.fromChange(event)` — procedural, single event
+
+Takes a `ChangeEvent` and returns the `[oldRecord, newRecord]` tuple, parsing
+`oldImage` / `newImage` into typed model instances (the same hydration used for reads —
+ISO strings become `Date`, embedded objects are reconstructed). It returns **all
+fields** (there is no field projection — see [Design notes](#design-notes)).
+
+```ts
+const [oldRecord, newRecord] = OrderModel.fromChange(event);
+//     ^ OrderModel | null    ^ OrderModel | null
+
+// One side is null depending on the event kind:
+//   INSERT → [null,      newRecord]
+//   MODIFY → [oldRecord, newRecord]
+//   REMOVE → [oldRecord, null     ]
+//   (event for a different model) → [null, null]   ← self-routing
+
+if (oldRecord || newRecord) {
+  if (!newRecord) {
+    // REMOVE — delete; derive the sink key from oldRecord
+  } else {
+    // INSERT | MODIFY — upsert; diff against oldRecord if you need change detection
+  }
+}
+```
+
+A valid event for this model always has at least one non-null side, so `[null, null]`
+is an unambiguous "this event is not for this model" signal. You don't need a separate
+guard — `fromChange` does routing **and** parse in one call.
+
+### 3. `DDBModel.subscribe(handlers)` — declarative, batch
+
+The third sibling of `query` / `mutate` (GraphQL's query / mutation / **subscription**).
+It takes a handler map keyed by model name. Unlike `query` / `mutate`, which hit
+DynamoDB, **`subscribe` does not subscribe**: it *returns* a `ChangeHandler`
+(`(batch) => Promise<BatchResult>`) that you mount on your own stream source.
+
+```ts
+// graphddb side: build a batch handler that parses + routes + dispatches (no sink, no dedup)
+const handler = DDBModel.subscribe({
+  OrderModel: (old, neu) => {        // old, neu: OrderModel | null (typed via the generated registry)
+    if (!neu) return opensearch.delete({ id: old!.orderId });
+    return opensearch.index({ id: neu.orderId, body: { status: neu.status } });
+  },
+  UserModel: (old, neu) => { /* old, neu: UserModel | null */ },
+});
+
+// your side: the real subscription lives here (delivery is yours)
+emulator.subscribe(handler);              // local (CDC emulator)
+// export const lambdaHandler = handler;   // production (DynamoDB Streams → Lambda)
+```
+
+The returned handler:
+
+- loops the batch and routes each event to its owning model;
+- typed-parses via that model's `fromChange`, then calls the matching handler;
+- if a handler throws, records that record's `sequenceNumber` into `batchItemFailures`
+  (the Streams / Lambda partial-batch-failure protocol, so only the failed records are
+  redelivered);
+- **never writes a sink and never dedups** — the `opensearch.*` calls above are your code.
+
+Recording `batchItemFailures` is the delivery *protocol* shape (which records to
+redeliver), not delivery *semantics* (upsert / dedup), so it stays inside the boundary.
+
+### 4. `CdcModelRegistry` — generated type registry
+
+The `subscribe` handlers are keyed by model name (a string), so type inference needs a
+type-level "model name → type" map. graphddb's code generation emits it for you as a
+module augmentation:
+
+```ts
+// generated by `graphddb generate` (you do not write this)
+declare module 'graphddb' {
+  interface CdcModelRegistry {
+    OrderModel: OrderModel;
+    UserModel: UserModel;
+  }
+}
+```
+
+With it in your build, `subscribe`'s keys are constrained to your registered models
+(typos and unknown model names are compile errors) and each handler's `old` / `neu` is
+inferred from the key.
+
+## The GraphQL trio
+
+| GraphQL | DDBModel | Executed by | Returns |
+| --- | --- | --- | --- |
+| query | `DDBModel.query(envelope)` | graphddb (hits DynamoDB) | result object |
+| mutation | `DDBModel.mutate(envelope)` | graphddb (hits DynamoDB) | result object |
+| subscription | `DDBModel.subscribe(handlers)` | **you** (mount it) | `ChangeHandler` |
+
+Only subscription is "executed by you, returns a handler rather than a result" —
+because CDC delivery lives outside the boundary.
+
+## Design notes
+
+- **No field projection (`fromChange` returns all fields).** Narrowing the fields does
+  not reduce network cost (the image is already on the stream), so there is no `select`.
+  Shape the record however you like in plain TS inside your handler.
+- **`[null, null]` for routing.** A valid event always has a non-null side, so both
+  being null is an unambiguous "not for this model" signal — `fromChange` handles
+  routing and parse without an extra guard.
+- **`subscribe` returns a handler.** So that graphddb owns neither the subscription nor
+  delivery. The return type being a `ChangeHandler` (not a result) is the visible
+  expression of the boundary.
+- **Naming.** `subscribe` completes the GraphQL query / mutation / subscription trio
+  (distinct from `emulator.subscribe(handler)`, which is the actual subscription).
+
+## See also
+
+- [`spec.md` §7 — Relations & maintained access paths](./spec.md#7-relations)
+- [`design-patterns.md` §10 — External Projection](./design-patterns.md#10-external-projection)
+- [`cdc-emulator.md`](./cdc-emulator.md) — the CDC emulator that drives stream handlers locally
+- [`class-hydration.md`](./class-hydration.md) — how raw items become typed instances

package/docs/design-patterns.md

@@ -47,7 +47,7 @@ graph, the stream drain, and the rebuild planner consume every pattern uniformly
 | 7 | [Aggregate Counter](#7-aggregate-counter) | `@aggregate(..., { pattern: 'counter', value: count() })` | ✅ Phase 1 (`count()`) / stream (`max()`) |
 | 8 | [Versioned / Latest Pointer](#8-versioned--latest-pointer) | `@hasOne(... versionedLatest)` + `@hasMany(... versionedHistory)` | ✅ Phase 3 |
 | 9 | [Reverse Lookup](#9-reverse-lookup) | GSI (both directions) or `dualEdge` two-row sync | ✅ Phase 1 (GSI) / Phase 3 (`dualEdge`) |
-| 10 | [External Projection](#10-external-projection) | typed-consumer-IF contract (redesign — see #153) | 🚧 redesign |
+| 10 | [External Projection](#10-external-projection) | typed-consumer-IF contract (`@cdcProjected` + `fromChange` / `subscribe`) | ✅ |
 
 ---
 
@@ -494,23 +494,28 @@ OpenSearch, S3, Neptune — for analytics or search, decoupled from the table. T
 projection is asynchronous and must tolerate at-least-once delivery (a redelivered
 event must not double-write).
 
-**(b) How it is being redesigned (🚧 #153).** The previous `defineProjection` builder +
+**(b) How to declare it in graphddb.** The previous `defineProjection` builder +
 `ProjectionSinkDrain` / `InMemoryProjectionSink` runtime baked the sink-delivery
 semantics (upsert, idempotency, dedup) into the library — a boundary overreach: the
 actual sink write happens inside the consumer's CDC infrastructure, outside graphddb.
-That runtime was **removed in 0.4.0** (issue #152). The redesign (issue #153) reframes
-external projection as a **typed-consumer-IF contract**: from a declared source +
-projection fields + idempotency-key extraction, graphddb emits a typed
-`(changeImage) => TypedProjectionRecord` mapper for the consumer to call inside its own
-sink delivery — graphddb owns the *parse → typed record* contract, not the delivery.
-External projection is neither a Model nor a relation, so it does not ride a decorator.
+That runtime was **removed in 0.4.0** (issue #152). External projection is now a
+**typed-consumer-IF contract** (issue #153): mark the source model with
+`@cdcProjected()`, then parse a CDC change event into typed instances — `Model.fromChange(event)`
+returns `[oldRecord, newRecord]`, and `DDBModel.subscribe(handlers)` (the GraphQL-style
+sibling of `query` / `mutate`) returns a `ChangeHandler` the consumer mounts on its own
+stream. graphddb owns the *parse → typed record* contract; subscription, sink delivery,
+and idempotency stay with the consumer. See [`cdc-projection.md`](./cdc-projection.md)
+for the full design.
 
 **(c) Read & write behavior.** Out of scope for this library (the data is read from the
-sink; delivery + idempotency live in the consumer's CDC code). See #153 for the
-typed-consumer-IF contract design.
-
-**(d) Constraints & phase status.** 🚧 Redesign (#153). The sink runtime is gone in
-0.4.0; the typed-consumer-IF replacement lands separately.
+sink; delivery + idempotency live in the consumer's CDC code). graphddb only parses the
+change event into a typed record; see [`cdc-projection.md`](./cdc-projection.md) for the
+typed-consumer-IF contract.
+
+**(d) Constraints & phase status.** ✅ Shipped (issue #153). The sink runtime was removed
+in 0.4.0; the typed-consumer-IF replacement (`@cdcProjected` + `fromChange` + `subscribe`
++ the generated `CdcModelRegistry`) is implemented and tested (unit + emulator-driven
+integration).
 
 ---
 

package/docs/spec.md

@@ -546,9 +546,10 @@ Shared vocabulary (confirmed across the epic):
   view model** — `materializedView` / `sparseView` — are declared with `@model({ kind })`
   + `@maintainedFrom` (§7.1 below). The **versioned** presets ride the `@hasOne` /
   `@hasMany` `pattern` discriminated union (`versionedLatest` / `versionedHistory`).
-  External projection is being redesigned as a typed-consumer-IF contract (#153) and
-  has no decorator. Unknown values on a relation are accepted by the open-ended union
-  but not lowered.
+  External projection is a typed-consumer-IF contract (#153); it is not a relation
+  `pattern` — the source model is marked with `@cdcProjected()` instead (see
+  [`cdc-projection.md`](./cdc-projection.md)). Unknown values on a relation are
+  accepted by the open-ended union but not lowered.
 - **`write.maintainedOn`** — cross-entity triggers `"<Entity>.<event>"`. The
   Entity segment is the **logical** entity name (`PostModel → 'Post'`; or the
   `@model({ prefix })` value sans `#`), and the event is one of
@@ -634,7 +635,11 @@ The latest/history key binding is derived from each target model's own `key` fie
 composed snapshot×2 IR. (A self-co-located form is a deferred follow-up.)
 
 External projection's old `defineProjection` / `ProjectionSinkDrain` runtime was
-**removed in 0.4.0** and is being redesigned as a typed-consumer-IF contract (#153).
+**removed in 0.4.0** and is now a typed-consumer-IF contract (#153):
+the source model is marked `@cdcProjected()`, `Model.fromChange(event)` parses a CDC
+change event into `[oldRecord, newRecord]`, and `DDBModel.subscribe(handlers)` returns a
+`ChangeHandler` the consumer mounts on its own stream (subscription / sink delivery /
+idempotency stay with the consumer). See [`cdc-projection.md`](./cdc-projection.md).
 
 ### 7.2 Stream maintenance — outbox → CDC drain (`updateMode: 'stream'`, #130)
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "graphddb",
-  "version": "0.5.0",
+  "version": "0.5.2",
   "description": "Graph data modeling on DynamoDB with adjacency list pattern",
   "type": "module",
   "main": "dist/index.js",
@@ -80,5 +80,8 @@
   "license": "MIT",
   "dependencies": {
     "commander": "^15.0.0"
+  },
+  "optionalDependencies": {
+    "typescript": "^5.5.0"
   }
 }