graphddb 0.4.2 → 0.5.0

package/docs/class-hydration.md

@@ -0,0 +1,409 @@
+# Opt-in Class Hydration for Read Results (`options.hydrate`)
+
+> **Status.** The design below and **Phase 1** — top-level `query`
+> `options.hydrate` — are **IMPLEMENTED**. **Phase 2** (`list` per-item
+> hydrate) and **Phase 3** (per-relation hydrate) are **FUTURE**, described here
+> as the planned, well-defined extensions; they are not yet shipped.
+
+## Premise
+
+GraphDDB's Read surface (`query` / `list` / relation traversal) returns a
+**Typed Plain Object**. `hydrate()` (`src/hydrator/hydrator.ts`) builds a plain
+`Record` carrying only the projected fields — it constructs no class instance,
+and the internal `PK` / `SK` / GSI key attributes are never copied in. This is
+the **correct default** and does not change:
+
+- it is the CQRS Read Model shape — a projection, not a domain entity;
+- it is JSON / OpenAPI-native (a `query` result serializes verbatim);
+- it is what the multi-language runtime (`#48`, the Python bridge) reconstructs
+  from a declarative, serializable plan — a class instance is a *host-language*
+  notion that does not cross the bridge.
+
+On top of that default, some callers — e.g. a persistence layer that wants to
+read into a **semantic object** (a domain object with behavior) rather than a
+bag of fields — need to **hydrate the read result onto a host-language object**.
+GraphDDB provides that as an **opt-in extension**: a `hydrate` factory passed
+in the call's `options` bag.
+
+```ts
+// default: Typed Plain Object (UNCHANGED)
+const user = await User.query({ userId: 'alice' }, { userId: true, name: true });
+//    ^? { userId: string; name: string } | null
+
+// opt-in: hydrate the plain object onto a host-side domain object
+const user = await User.query(
+  { userId: 'alice' },
+  { userId: true, name: true },
+  { hydrate: (raw) => new UserSemanticObject(raw) },
+);
+//    ^? UserSemanticObject | null
+```
+
+The decisive precedent is **`updatable` (#54)**: a host-only, non-serialized,
+*return-type-changing* read option, implemented as a third-argument flag with
+overloaded return types (`QueryResult<…> & Updatable`). `hydrate` is the same
+*class* of option — a host-runtime read-result transform — and follows the same
+design rules. Its only addition over `updatable` is that the new return type is
+**caller-supplied** (`R`) rather than a fixed brand, so it propagates through a
+generic instead of a constant.
+
+```
+declarative "what to fetch"   = select + key + filter      → serializable (operations.json, #48)
+host-runtime "how to read it"  = consistentRead / maxDepth / updatable / hydrate / middleware (#50)
+                                  → NEVER serialized; never in the SSoT / operations.json
+```
+
+---
+
+## API shape
+
+### Position — the third-argument `options` bag (`hydrate`)
+
+`query(key, select, options?)` already takes a third options bag with
+`consistentRead` / `maxDepth` / `updatable`. `hydrate` is **added to that bag**,
+*not* as a new method:
+
+```ts
+query(
+  key,
+  select,
+  {
+    consistentRead?: boolean;
+    maxDepth?: number;
+    updatable?: boolean;
+    hydrate?: (raw: QueryResult<T, ProjectionOf<T, Sel>>) => R;   // ← new
+  },
+)
+```
+
+Rationale (mirrors the issue's "API shape policy"):
+
+- The options bag already exists and is the established place for *host-runtime*
+  read settings. Adding `hydrate` there avoids the combinatorial explosion a
+  dedicated `queryAs(Ctor, …)` method would create (every existing/future option
+  × every dedicated method).
+- `hydrate` is a **factory** `(raw) => R`, not a constructor. A factory subsumes
+  the `queryAs(Ctor, …)` ergonomics — `hydrate: (raw) => new Ctor(raw)` is the
+  constructor case — *and* the arbitrary-mapping case (`hydrate: (raw) =>
+  toDomain(raw)`, or even a non-instance shape). So no extra method is needed.
+
+### The factory input is the fully-resolved plain object
+
+The factory receives the **plain object the default path already produces** —
+the `QueryResult<T, ProjectionOf<T, Sel>>` for that exact `select` (projection +
+resolved relations). Concretely, the factory runs **after** `hydrate()` (the
+plain-object builder) and after relation resolution, on the value that would
+otherwise be returned. This is what makes `hydrate`'s input type *follow the
+projection*: `select` → `ProjectionOf<T, Sel>` → `QueryResult<T, …>` → factory
+input (the same chain the default return type already uses). A `select` that
+omits a field also omits it from the factory's input type, so the factory cannot
+read an unprojected attribute by accident.
+
+### `null` semantics (single-item reads)
+
+For `query` (and single-value relations) the read can resolve to `null` (no
+item). `hydrate` is applied **only to a present item** — a `null` read stays
+`null` and the factory is **not** invoked. So the return type is
+`R | null`, never `R` for `(raw: …null) ⇒ …`. This matches the `updatable`
+overload's `… | null`.
+
+---
+
+## Return-type propagation
+
+`hydrate` changes the return type from the plain `QueryResult<…>` to the
+factory's return `R`. This is a conditional/inference on the options bag,
+identical in shape to the `updatable` overload split that already exists in
+`ModelStatic` (`src/model/types.ts`).
+
+```ts
+// when options carry a hydrate factory → return R (| null), inferred from the factory
+query<Sel extends SelectSpec<T>, R>(
+  key: QueryKey<C>,
+  select: Sel & StrictSelectSpec<T, Sel>,
+  options: {
+    consistentRead?: boolean;
+    maxDepth?: number;
+    updatable?: boolean;
+    hydrate: (raw: QueryResult<T, ProjectionOf<T, Sel>>) => R;   // present ⇒ this overload
+  },
+): Promise<R | null>;
+
+// existing overloads (no hydrate) — UNCHANGED, still return the plain object
+query<Sel extends SelectSpec<T>>(
+  key: QueryKey<C>,
+  select: Sel & StrictSelectSpec<T, Sel>,
+  options?: { consistentRead?: boolean; maxDepth?: number; updatable?: false },
+): Promise<QueryResult<T, ProjectionOf<T, Sel>> | null>;
+```
+
+Key properties:
+
+- `R` is inferred **from the factory**, so `hydrate: (raw) => new UserSO(raw)`
+  yields `Promise<UserSO | null>` with no annotation. This is the
+  `queryAs(Ctor, …)`-style ergonomics, expressed by inference.
+- The factory's *input* is `QueryResult<T, ProjectionOf<T, Sel>>` — the exact
+  projection type — so the factory body is fully typed against the projected
+  fields, and `select`-driven projection inference flows straight into it
+  (the issue's "type inference: the select-derived projection type flows to hydrate's input type").
+- **The no-`hydrate` overloads are byte-for-byte the existing ones.** The
+  default return type is unchanged; `hydrate` is purely additive.
+
+### Interaction with `updatable`
+
+`updatable` brands the *plain object* with `& Updatable` (a hidden key for
+`updateItem()`). `hydrate` replaces the return value with `R`. If both are passed,
+the **factory receives the branded+updatable plain object** (so the factory may
+read the hidden key if it wants), but the **returned value is `R`** — the brand
+does not survive onto an arbitrary `R` (the factory decides what `R` is, and a
+fresh class instance is not `Updatable`). The recommended guidance: pick one —
+either keep the plain (optionally updatable) object, or hydrate to a domain
+object that manages its own persistence. The phased plan ships `hydrate` first
+without combining the two return modes; combining is a documented type fork
+(see *Design forks*).
+
+---
+
+## Application point
+
+`hydrate` is an **after-fetch transform**, applied at the very end of the read,
+on the fully-built plain result:
+
+```
+plan(select) → execute → hydrate()  [plain object]
+            → resolveRelations()    [attach relations]
+            → (updatable: attach hidden key)
+            → options.hydrate(plainResult)   ← LAST, host-only
+            → return
+```
+
+In `executeQuery` (`src/operations/query.ts`) this is the single point where
+`hydratedItem` is about to be returned (after relations are merged) — the
+factory wraps that value.
+
+### Order vs the #50 middleware after-fetch hook
+
+`#50` (middleware / hooks) is the **same class** of host-side runtime config: a
+cross-cutting insertion point that is *not* serialized into the SSoT. Both act
+after-fetch, so their ordering must be defined. Recommended contract:
+
+```
+fetch → plain object built → #50 middleware after-fetch hook(s) → options.hydrate → return
+```
+
+Rationale: middleware is **cross-cutting** (logging, metrics, soft-delete
+filtering, tenant scoping) and should observe/transform the **canonical plain
+Read Model** — the same shape regardless of whether a particular call opts into
+`hydrate`. `hydrate` is the **per-call, innermost** transform that decides the
+*caller's* return shape. So middleware runs on the plain object first, and
+`hydrate` is the last step, closest to the caller. (`#50` is designed
+separately; this spec only fixes the ordering invariant so the two compose
+predictably. If `#50` later wants a hook that runs *after* hydration, it would
+be a distinct, explicitly-named hook — not a reordering of this one.)
+
+The division of responsibility: middleware = cross-cutting, shape-preserving,
+applies to all reads; `hydrate` = per-call, shape-changing, opt-in.
+
+---
+
+## Relation scope
+
+The issue raises whether `hydrate` applies only at the top level or per
+relation. Recommendation, with the fork stated explicitly:
+
+### Phase 1–2 (recommended default): TOP-LEVEL ONLY
+
+`hydrate` applies to the **whole read result** at the top level. Nested
+relations inside the result remain **plain objects** (their plain
+`QueryResult` shape). The factory therefore receives a `QueryResult<T, …>`
+whose relation fields are still plain `{ items, cursor }` / `single | null`,
+and the factory decides what to do with them (often: wrap the top-level entity
+and leave relations as data, or map relations itself inside the factory body).
+
+Why top-level-first:
+
+- It is **unambiguous** about the *application unit*: exactly one factory call
+  per top-level read (one for `query`; per top-level item for `list`).
+- It keeps the type story simple — one `R`, inferred from one factory.
+- It does not entangle hydration with N+1 batching (below).
+
+### Per-relation hydrate (deferred extension, fork)
+
+A future extension could let a relation's select-position builder carry its own
+hydrate factory (`User.relation({...}).hydrate((raw) => new OrderSO(raw))`), so
+each relation level produces its own `R`. This is **strictly more expressive**
+but adds real complexity:
+
+- **Application unit.** A relation result is `{ items, cursor }` (hasMany) or
+  `single | null` (belongsTo / hasOne). Per-relation hydrate would apply the
+  factory **per item** of `items` (cursor preserved verbatim:
+  `{ items: items.map(hydrate), cursor }`), and to the single value (or `null`
+  → not invoked) for belongsTo / hasOne — exactly mirroring the top-level
+  null-skip rule.
+- **Timing vs N+1 BatchGet.** belongsTo / hasOne resolve via batched
+  `BatchGetItem` (`fetchBelongsToHasOneBatch` in `src/relation/traversal.ts`);
+  hasMany via `Query`. A per-relation factory must run **after** the batch
+  resolves and after that relation's own nested relations are merged — i.e. at
+  the same after-fetch point, one level down. Batching (the *fetch* strategy)
+  is unaffected: hydration is applied to the *resolved* items post-batch, so it
+  never reintroduces an N+1. The order is: batch-get → hydrate plain → resolve
+  nested → per-relation factory.
+- **Type propagation** must thread each relation's `R` into the parent
+  `QueryResult` at that relation's position (a deeper conditional type over the
+  select tree). This is feasible but is a meaningfully larger type change than
+  the top-level overload.
+
+**Recommendation:** ship top-level-only (Phases 1–2). Treat per-relation
+hydrate as a separate, later issue, only if a concrete need appears — the
+top-level factory can already map relations in its body, so per-relation
+hydrate is an ergonomics improvement, not a capability gap.
+
+---
+
+## Python-bridge non-interference (CRITICAL — hard constraint)
+
+`hydrate` is **host-runtime-only**. It MUST NOT appear in the SSoT, the
+serialized query/`OperationSpec`, or `operations.json`. This is the same rule
+`#50` (middleware) and `updatable` / `consistentRead` already follow.
+
+- **The declarative layer is "what to fetch."** `operations.json` is generated
+  from *contract definitions* (`publicQueryModel`, etc.) — key, select, filter,
+  projection, result paths, cardinality, execution plan. It is the serializable,
+  language-neutral plan the Python runtime (`#48`) replays. It already contains
+  **none** of the host-runtime call options: a `grep` of
+  `python/tests/fixtures/generated/operations.json` finds **zero** occurrences
+  of `hydrate`, `updatable`, or `consistentRead` — these are passed at call
+  time on the host, never baked into the spec.
+- **`hydrate` is "what host object to load the result onto."** That is a
+  *host-language* concern — a TS class / domain object has no representation in
+  the serialized plan and no meaning to the Python runtime, which reconstructs
+  its own (Python) plain result from the same plan. A factory is a TS closure;
+  it is **not serializable** and is structurally incapable of crossing the
+  bridge.
+- **Mechanically guaranteed by where it lives.** `hydrate` is only ever a
+  field of the *call-time* `options` argument (`executeQuery`'s third param /
+  `QueryOptions`), exactly like `updatable`. The contract-recording path
+  (`recordContractOp` in `DDBModel`) records only `{ keys, select }` from the
+  contract method body — it never reads the options bag — so a `hydrate` passed
+  at execution time can never reach the recorder or the generator. There is no
+  code path from `options.hydrate` to the SSoT.
+
+**Net:** `#48` is not impeded. "What to fetch" stays declarative and
+serializable; "what object to load it onto" stays host-side and is never
+serialized. The design states this explicitly as an invariant: *no read-call
+option that changes only the host return shape is ever serialized.*
+
+---
+
+## Error handling
+
+The `hydrate` factory is caller code and may `throw` (e.g. a domain-object
+constructor validating invariants). Contract:
+
+- A throw from `hydrate` **propagates to the caller** as the rejection of the
+  read `Promise`, unchanged (not wrapped, not swallowed). The read itself has
+  already succeeded at this point; the failure is purely in the caller-supplied
+  transform, so surfacing it verbatim is the least-surprising behavior and lets
+  the caller distinguish it via its own error types.
+- `hydrate` is **not** invoked for a `null` read (no item) — so a factory need
+  not be null-tolerant.
+- For `list` (Phase 2), if hydrating one item throws, the whole `list` call
+  rejects (it is a single `Promise`); the design does **not** silently drop the
+  offending item or partially hydrate, because a partially-applied transform
+  would be a worse surprise than a clean rejection. (A caller wanting
+  per-item resilience can hydrate `result.items` itself with its own
+  try/catch — the plain path is always available by simply omitting `hydrate`.)
+- The factory must be **synchronous** in Phase 1 (`(raw) => R`, not
+  `=> Promise<R>`). A read returning `Promise<R | null>` with a synchronous
+  factory keeps the type clean and avoids an await-in-await. Async factories are
+  a possible later extension but are intentionally out of scope (a caller can
+  `await User.query(...)` then `await asyncMap(result)` today).
+
+---
+
+## Phased implementation plan
+
+Mirroring the issue's "phased implementation plan (query top-level first → list/relation)":
+
+1. **Phase 1 — `query` top-level after-fetch hydrate.** Add `hydrate?:
+   (raw) => R` to `QueryOptions` (`src/operations/query.ts`) and apply it as the
+   final step in `executeQuery` (only on a present item; `null` stays `null`).
+   Add the `hydrate` overload to `ModelStatic.query` (`src/model/types.ts`) so
+   `R` propagates and the default overloads stay unchanged. **Tests:** type
+   tests proving `R` propagation (`Promise<R | null>`) and that the no-`hydrate`
+   default return type is byte-identical; a unit test proving the default
+   (no `hydrate`) returns the unchanged plain object; a generation test proving
+   `hydrate` never appears in `operations.json`. *(IMPLEMENTED — see the
+   *Phase 1 implementation* section below.)*
+
+2. **Phase 2 (FUTURE) — `list` top-level after-fetch hydrate.** Apply the same factory
+   **per item** of `executeListInternal`'s hydrated `items`
+   (`{ items: items.map(hydrate), cursor }`; cursor untouched). Add the
+   `hydrate` overload to `ModelStatic.list`. Same test matrix, plus the
+   per-item error-rejection contract.
+
+3. **Phase 3 (FUTURE) — relation scope (only if needed).** If per-relation hydrate is
+   wanted, add a `.hydrate(...)` to the relation select builder and thread each
+   relation's `R` through `QueryResult` and `resolveRelations` /
+   `fetchBelongsToHasOneBatch` (apply post-batch, post-nested-resolution, per
+   the *Relation scope* section). Until then, top-level `hydrate` + in-factory
+   relation mapping covers the use case.
+
+Each phase keeps the plain-object default unchanged, adds only host-runtime
+behavior, and adds nothing to the SSoT / `operations.json`.
+
+---
+
+## Phase 1 implementation (TS, shipped)
+
+Phase 1 is small, non-invasive, and follows the `updatable` precedent exactly
+(Phases 2–3 remain the plan above):
+
+- `QueryOptions` (`src/operations/query.ts`) gains
+  `hydrate?: (raw: Record<string, unknown> | null) => unknown` — applied as the
+  final step in `executeQuery`, *only* on a present item.
+- `ModelStatic.query` (`src/model/types.ts`) gains a leading overload that
+  infers `R` from `hydrate` and returns `Promise<R | null>`; the existing
+  no-`hydrate` overloads are unchanged.
+
+Verified invariants (covered by the test suite):
+
+- **Default unchanged:** a `query` with no `hydrate` returns the same plain
+  object as before (unit + type).
+- **`R` propagates:** `hydrate: (raw) => new UserSO(raw)` types as
+  `Promise<UserSO | null>` (type test); the factory's input is the projected
+  `QueryResult` (type test).
+- **`null` skips the factory:** a missing item returns `null`, factory not
+  called (unit).
+- **Not serialized:** `operations.json` contains no `hydrate` (generation
+  test / `grep`).
+
+---
+
+## Design forks surfaced
+
+1. **Relation scope: top-level-only (recommended) vs per-relation.** Resolved in
+   favor of top-level-only for Phases 1–2, with per-relation deferred to a later
+   issue (the top-level factory can map relations in its body, so this is
+   ergonomics, not capability). Documented in *Relation scope*.
+2. **Combining `updatable` + `hydrate`.** Phase 1 ships them as independent
+   options; the factory receives the (possibly updatable) plain object but the
+   returned `R` does not inherit the `Updatable` brand. A combined "updatable
+   domain object" return is a possible later type fork; recommendation is to use
+   one or the other per call. Documented in *Interaction with `updatable`*.
+3. **Sync vs async factory.** Phase 1 is synchronous (`(raw) => R`); async is a
+   deferred extension. Documented in *Error handling*.
+
+## Bottom line
+
+`options.hydrate` is the read-side analog of `updatable` (#54): a host-runtime,
+non-serialized, return-type-changing read option, added to the existing options
+bag rather than as a new method. A factory `(raw: QueryResult<T, …>) => R`
+propagates `R` to the call's return type by inference (subsuming `queryAs`),
+applied as the **last** after-fetch step (after the #50 middleware hook, on a
+present item only). It is **top-level scope** to start, with per-relation
+hydrate a deferred, well-defined extension. Critically, it is **host-only and
+never enters the SSoT / `operations.json`** — so the declarative "what to fetch"
+stays serializable for the Python bridge (#48), and only the host-side "what
+object to load it onto" is added. The plain Typed-Plain-Object default is
+unchanged.