@sanity/workflow-engine 0.13.0 → 0.15.0

package/DATAMODEL.md

@@ -0,0 +1,183 @@
+# The engine's persisted data model
+
+The engine owns three standalone document types in the Content Lake:
+
+| Doc type                     | Constructed by                               | Mutated after create                                                                          |
+| ---------------------------- | -------------------------------------------- | --------------------------------------------------------------------------------------------- |
+| `sanity.workflow.definition` | `planDefinitionDeploy` (`src/api/deploy.ts`) | never — deploys are create-only                                                               |
+| `sanity.workflow.instance`   | `buildInstanceBase` (`src/instance.ts`)      | every persist writes the canonical state via `instanceStateFields` (`src/engine/mutation.ts`) |
+| `temp.system.guard`          | `compileGuard` (`src/core/guards.ts`)        | full-body refresh on stage re-entry; predicate lift on retract                                |
+
+These field trees are **our contract with every past and future engine
+version reading the same dataset**. This file is where changes to them are
+declared; the model-surface gates keep undeclared drift red:
+
+- **The shape ledger** — `src/__tests__/model-shapes/model-<N>.json`, one
+  file per `DATA_MODEL_VERSION`, **append-only**: only the current version's
+  file is ever regenerated (`vitest -u` on the gate test), so past model
+  shapes stay checked in verbatim — the historical corpus future upgrade
+  machinery tests against. It pins the field trees of every doc
+  type built through the real constructors, plus the stored grammar's
+  vocabulary (every enum's values, every variant discriminator), so a value
+  rename is as red as a key rename.
+- **Coverage tests** (`src/__tests__/model-surface.test.ts`) walk the stored
+  definition schema and fail when any declared key, discriminator, or
+  instantiable enum value stops appearing in the canonical document — the
+  grammar cannot grow past the fixture silently.
+- **The lifecycle ledger** (`model-shapes/model-<N>.lifecycle.json` in
+  `workflow-engine-test`) drives real instances through the verbs —
+  including the recovery paths (expired-lease sweep, loop-back re-entry
+  adoption, dead-end propagation, abort) — and pins the interior trees a
+  fresh construction can't reach: `StageEntry` / `ActivityEntry` /
+  `SubworkflowEntry` rows, pending effects (claim included), effect
+  history, the idempotency ledger, and ALL `HistoryEntry` variants (the
+  accounting is compile-checked against the union, so a new variant fails
+  the build until the scenarios produce it).
+
+Residual review responsibility: each ledger entry pins one shape per
+variant (the scenario's first), so an _optional_ key that only appears on
+some entries of an already-covered variant — or a conditionally-written
+field no fixture or scenario reaches — still needs a reviewer to demand a
+ledger-visible sample. Fixture width is part of reviewing a model change.
+
+This is **not** definition-content versioning: `version` / `pinnedVersion` /
+`contentHash` track what an author deployed; the model stamp tracks the shape
+of the documents the engine itself writes.
+
+## The stamp pair
+
+The two doc types whose format the engine owns outright — definition and
+instance — carry two numbers (`MODEL_STAMP` in `src/core/model-version.ts`).
+Provenance and compatibility are deliberately separate, because consumers
+sharing a dataset upgrade months apart (Functions lag Studios lag CLIs) and
+"newer" must not mean "incompatible".
+
+**The guard doc carries NO stamp.** `temp.system.guard` is a foreign
+contract: the persisted format mirrors the Content Lake's forthcoming guard
+primitive, and the engine must not grow fields on a shape it doesn't own —
+the lake dictates that format long-term. The guard doc's versioning event is
+the `_type` cutover itself (an engine that predates `system.guard` simply
+never matches the new type).
+
+- **`modelVersion` — provenance ("conforms-to").** The value of
+  `DATA_MODEL_VERSION` at write time: "this document conforms to model N
+  now". Bumps on **every** declared shape change, additive included.
+  Instances are re-stamped on every full persist (and rollback restore).
+  Partial patches deliberately leave the pair alone — they don't normalize
+  the shape, so restamping there would lie.
+- **`minReaderModel` — the reader floor.** The value of
+  `DATA_MODEL_MIN_READER` at write time: the oldest engine model that can
+  safely _interpret_ this document. An **additive** change leaves the floor
+  untouched — older engines keep reading and writing newer docs, so
+  mixed-version fleets interoperate. Only a **breaking** change raises it,
+  and raising it is a declared decision to fence out older readers (fleet
+  must upgrade readers before such writers deploy).
+- **Missing stamp = model 0** (floor 0) — the document was last written
+  before the stamp existed. A `modelVersion` with no `minReaderModel` is
+  malformed foreign data and gates conservatively: the stamp is the floor.
+- **Backward compatibility is unconditional.** An engine at model N reads
+  every stamp `0..N`. There is no supported-version floor for _old_ docs;
+  introducing one is a deliberate future mechanism, never a side effect of
+  a shape change.
+- **Fail hard on `minReaderModel` ahead — and only that.** Every engine read
+  of a stamped doc type (definition, instance) passes through
+  `assertReadableModel` — point reads, list queries, the drained-children
+  projection, snapshot hydration's raw reads. A floor ahead of the engine's
+  `DATA_MODEL_VERSION` throws
+  `ModelVersionAheadError` ("upgrade `@sanity/workflow-engine`") instead of
+  silently misreading a shape this engine predates. Deliberately wholesale:
+  one such document in a read's scope fails the whole read (a listing, a
+  verdict computation) rather than degrading silently. Two structural
+  exceptions: existence-only probes (no shape is interpreted), and lake-side
+  GROQ filters, which evaluate persisted fields server-side before any
+  client gate can run — their safety rests entirely on rule 3's
+  additive-only constraint.
+- **Old writers restamp down — the round-trip rule.** An older engine
+  persisting a newer-but-compatible instance stamps its own `modelVersion`
+  (honest conforms-to; the stamp oscillates in mixed fleets, harmlessly).
+  This is safe because full persists are set-only patches and mutation
+  copies are spreads, so fields the older engine doesn't know **survive the
+  round-trip**. That is a load-bearing invariant: an additive change whose
+  fields would NOT survive an old writer's round-trip is not additive — it
+  is breaking, and must raise the floor.
+- **The floor is raise-only.** A rewrite writes
+  `max(stored minReaderModel, own DATA_MODEL_MIN_READER)` — it never lowers
+  or drops the stored floor, because the preserved unknown fields of a newer
+  model may still require it. (The failure mode this fences off is
+  documented across the industry: a rewriter that drops the compatibility
+  gate it doesn't understand silently reopens the door the floor closed.)
+- **Raising the floor is an availability decision, sequenced readers-first.**
+  Before any writer that raises `DATA_MODEL_MIN_READER` deploys, every
+  reader in the fleet — customer-deployed Functions foremost — must already
+  run an engine at or above the new floor (the BACKWARD deployment order:
+  upgrade consumers before producers). And like every engine-side check the
+  floor is advisory — it gates our own clients; the Content Lake stays the
+  sole enforcement point.
+- **The gate ships from the engine.** Consumer surfaces that read these docs
+  without routing through the engine (the CLI's direct reads, MCP query
+  projections, the reactive studio/sdk adapters) adopt `assertReadableModel`
+  with the parse-don't-cast follow-up; until then they render newer-model
+  docs unchecked.
+
+## Rules for changing the model
+
+1. **Any change to a persisted field tree is a model change** and must be
+   declared: update the model-surface snapshot, add an entry to the model
+   log below, and decide — in the same change — whether `DATA_MODEL_VERSION`
+   moves **and whether the reader floor moves with it**. Nothing lands
+   "incidentally".
+2. **Additive optional fields are allowed** without machinery: bump
+   `DATA_MODEL_VERSION`, leave `DATA_MODEL_MIN_READER` untouched, and
+   declare the tolerant read (what an absent value means on docs written
+   before the field existed) plus why the field survives an old writer's
+   round-trip.
+3. **Renames, removals, and semantic changes are not allowed yet.** They
+   require versioned upgrade machinery (per-model normalization steps applied
+   on read) that does not exist; until it does, the model may only grow
+   additively. When such a change eventually lands, it raises
+   `DATA_MODEL_MIN_READER` — a declared fence against older readers, shipped
+   only after the fleet story is written.
+4. **`DATA_MODEL_VERSION` moves when compatibility reasoning changes** — when
+   a reader would need to know which shape it is looking at. A purely
+   internal refactor that provably writes an identical tree does not bump.
+5. **Classify by misinterpretation, not parseability.** The snapshot gate
+   catches structural drift only. A change can be structurally additive yet
+   change the MEANING of existing fields — old readers parse it fine and
+   silently misinterpret. That judgment (would an old reader misread, not
+   just fail to parse?) is exactly what decides whether
+   `DATA_MODEL_MIN_READER` moves, and no snapshot can make it — each model
+   log entry states it explicitly.
+
+## Model log
+
+### Model 0 — pre-governance
+
+Everything written before the stamp existed. Documents carry no
+`modelVersion`. The shape history of model 0 was absorbed by tolerant reads
+that predate this file — recorded here so they are declared compatibility
+rules, not folklore:
+
+- **Instances without `subworkflows[]`** (persisted before the subworkflow
+  registry existed) — read as `[]`; the next full persist materializes it.
+- **Instances without `processedRequests[]`** (persisted before the
+  idempotency ledger existed) — read as `[]`; the next full persist
+  materializes it.
+- **Definitions without `contentHash`** (deployed before content-addressing)
+  — always read as changed, so the first redeploy after upgrade mints a
+  fresh version.
+
+### Model 1 — stamping introduced (reader floor: 0)
+
+The `modelVersion` + `minReaderModel` pair is stamped on the definition and
+instance doc types; the guard doc deliberately carries none (foreign lake
+contract — see "The stamp pair"). No other shape change. The floor is 0: any
+engine ever released reads model-1 documents fine, and model-0 documents
+keep reading exactly as before.
+
+## Pending governed changes
+
+- **`temp.system.guard` → `system.guard`** — the guard doc type's `temp.`
+  prefix drops when the lake's own guard primitive ships (see
+  `GUARD_DOC_TYPE` in `src/types/authorization.ts`). A breaking type change
+  for already-persisted guard docs; it must land as a declared model change
+  with a migration story, through this process.

package/README.md

@@ -13,6 +13,36 @@ effects for runtimes to drain.
 npm install @sanity/workflow-engine
 ```
 
+## Fields vs Sanity schema fields
+
+A workflow **field** is the engine's state primitive: a typed value a workflow
+instance carries — an assignee, a decision, a due date, a reference to the
+document under review. Fields are declared in `fields: [...]` at workflow,
+stage, or activity scope and live on the workflow **instance** document. They
+are not Sanity schema fields: they never touch your content documents' shape.
+
+The authoring DSL deliberately borrows Sanity's schema idiom, so
+`@sanity/workflow-engine/define` exports its own `defineField`. A Studio file
+that also imports `defineField` from `sanity` should alias one of them:
+
+```ts
+import {defineField as defineWorkflowField} from '@sanity/workflow-engine/define'
+```
+
+The `fields` key itself appears at four nesting levels, meaning something
+different at each — read it by where it sits:
+
+1. **Declared field entries** — on a definition's workflow / stage / activity
+   nodes: the typed state that scope carries.
+2. **Sub-field shapes** — inside an `object`- or `array`-kind field entry:
+   lightweight `{type, name}` shapes for the value's structure, mirroring
+   Sanity's `object.fields` / `array.of`.
+3. **Object value expressions** — a `{type: 'object', fields: {...}}` value
+   expression in an op payload: one expression per key of the computed
+   object value.
+4. **Resolved runtime values** — `fields` on a workflow instance document: the
+   declared entries with their current values, written by ops and edits.
+
 ## License
 
 [MIT](./LICENSE)