@marianmeres/workflow 1.0.1 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENTS.md CHANGED
@@ -4,7 +4,9 @@
4
4
 
5
5
  - **Stack**: Deno/Node.js, PostgreSQL, TypeScript
6
6
  - **Test**: `deno task test` (requires `TEST_PG_*` env; see [tests/_pg.ts](./tests/_pg.ts))
7
+ - **Check**: `deno task check` — `deno fmt --check && deno lint && deno check` over `src/`, `tests/` and the example's server-side files
7
8
  - **Build (npm)**: `deno task npm:build`
9
+ - **Example app**: `deno task example` (needs `EXAMPLE_PG_*`); `deno task example:build` after touching `example/src/`, because `example/dist/bundle.js` is committed. See [example/README.md](./example/README.md).
8
10
  - **Entry point**: [src/mod.ts](./src/mod.ts)
9
11
 
10
12
  ## Package Overview
@@ -54,78 +56,130 @@ src/
54
56
  ├── correlator.ts # WorkflowInboxCorrelator (attaches match tick to injected Cron)
55
57
  ├── log.ts # createClog("workflow")
56
58
  ├── persistence/
57
- │ ├── instances.ts # CRUD + lock + claim due wake-ups + find by correlation
58
- │ ├── inbox.ts # append, claim-unprocessed, mark-processed
59
+ │ ├── instances.ts # CRUD + lock + seq bump + due/stale scans + find by correlation
60
+ │ ├── inbox.ts # append, claim-unprocessed, lock-row, mark-processed
59
61
  │ ├── history.ts # append + read
60
62
  │ └── tx.ts # withTransaction helper
61
63
  └── migrations/
62
64
  ├── 1_0_0.ts # CREATE/DROP three tables
65
+ ├── 1_1_0.ts # rename project_id -> tenant_id (in place)
66
+ ├── 1_2_0.ts # add seq (fence) + stale-pending partial index
63
67
  └── index.ts # createMigrate(pool) — @marianmeres/migrate setup
64
68
  tests/
65
69
  ├── _pg.ts # env-driven pool + schema reset
66
70
  ├── fixtures/
67
- │ └── stock-replenishment.ts # reference workflow used by tests only
68
- └── stock_replenishment.test.ts # happy/timeout/fail paths + validator
71
+ │ ├── stock-replenishment.ts # reference workflow used by tests only
72
+ │ └── weekly-digest.ts # recurring-trigger reference workflow
73
+ ├── admin.test.ts # cancel/retry: guards, fencing of in-flight jobs, resume
74
+ ├── correlator.test.ts # deferral, dedupe, matcher throw/reject, delivery-in-advance
75
+ ├── durability.test.ts # fence, preconditions, expiry re-dispatch, stale-pending re-poke
76
+ ├── migrations.test.ts # schema rename/fence up/down + legacy payload drain
77
+ ├── stock_replenishment.test.ts # happy/timeout/fail paths + validator
78
+ └── weekly_digest.test.ts # recurring trigger via cron
79
+ example/ # the interactive app — consumer code, imports by package name
80
+ ├── workflow.ts # userland: one definition + handlers + matcher
81
+ ├── server.ts # runtime wiring + HTTP API (raw SQL for the listing views)
82
+ ├── src/main.ts # the page (@marianmeres/vanilla); bundled to dist/bundle.js
83
+ ├── index.html # shell, styles, <template>s
84
+ └── theme.css, reboot.css # generated / vendored; excluded from fmt
85
+ scripts/
86
+ ├── build-npm.ts
87
+ └── gen-example-{version,theme}.ts # generated example assets; run by the example tasks
69
88
  ```
70
89
 
71
90
  ## Conceptual Model — Three Layers
72
91
 
73
- | Layer | What it is | Lives in |
74
- |---|---|---|
75
- | **Definition** | Static graph: nodes, transitions, handler ids (strings). Pure data, JSON-safe. | Code, registered in `WorkflowRegistry` |
76
- | **Instance** | Live state of one running occurrence. Cursor, context, execution_state, wake_at, correlation_token. | `__workflow_instances` row |
77
- | **Driver** | Event-driven engine that advances instances. Owns no state. | `workflow.advance` steve job |
92
+ | Layer | What it is | Lives in |
93
+ | -------------- | --------------------------------------------------------------------------------------------------- | -------------------------------------- |
94
+ | **Definition** | Static graph: nodes, transitions, handler ids (strings). Pure data, JSON-safe. | Code, registered in `WorkflowRegistry` |
95
+ | **Instance** | Live state of one running occurrence. Cursor, context, execution_state, wake_at, correlation_token. | `__workflow_instances` row |
96
+ | **Driver** | Event-driven engine that advances instances. Owns no state. | `workflow.advance` steve job |
78
97
 
79
98
  ## Two Orthogonal States
80
99
 
81
100
  Never conflate. Always two columns on `__workflow_instances`:
82
101
 
83
- | Column | Meaning | Source of truth for |
84
- |---|---|---|
85
- | `cursor` | Which node am I at? | The FSM's `state` (`getSnapshot().state`) |
102
+ | Column | Meaning | Source of truth for |
103
+ | ----------------- | --------------------------- | ------------------------------------------------------------------------ |
104
+ | `cursor` | Which node am I at? | The FSM's `state` (`getSnapshot().state`) |
86
105
  | `execution_state` | What's my lifecycle status? | `pending` / `running` / `waiting` / `completed` / `failed` / `cancelled` |
87
106
 
88
107
  An instance can be at node `await_reply` (cursor) and simultaneously `waiting` (execution_state).
89
108
 
90
109
  ## Node Taxonomy (`NodeMeta`)
91
110
 
92
- Each FSM state's `meta` field carries a discriminated `NodeMeta`. The driver dispatches on `meta.kind`:
111
+ Each FSM state's `meta` field carries a discriminated `NodeMeta` — required and typed, because a definition's `fsm` is a `WorkflowFSMConfig`, not a raw `FSMConfig` (fsm's own `meta` is `unknown`). The driver dispatches on `meta.kind`:
93
112
 
94
- | Kind | Driver behavior | Required fields |
95
- |---|---|---|
96
- | `pure` | Auto-fire `ENTER` event; transition by guards. Loop inline until terminal/effectful/suspending. Max 64 hops. | none |
97
- | `effectful` | Enqueue `workflow.effect.<handler>` steve job, flip to `running`. On completion, an `advance` is enqueued with `{ outcome, outcome_data }`. | `handler: string` |
98
- | `suspending` | Set `execution_state='waiting'`, `wake_at` (if `timeoutSec`), keep `correlation_token`. Wakes via scheduler (`TIMEOUT`) or correlator (`MATCHED`). | optional `matcher`, optional `timeoutSec` |
99
- | `terminal` | Set `execution_state='completed'`. | none |
113
+ | Kind | Driver behavior | Required fields |
114
+ | ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |
115
+ | `pure` | Auto-fire `ENTER` event; transition by guards. Loop inline until terminal/effectful/suspending. Max 64 hops. | none |
116
+ | `effectful` | Enqueue `workflow.effect.<handler>` steve job, flip to `running`. On completion, an `advance` is enqueued with `{ outcome, outcome_data, correlation_token? }`. | `handler: string` |
117
+ | `suspending` | Set `execution_state='waiting'`, `wake_at` (if `timeoutSec`), keep the `correlation_token` (or the one the last handler returned). Wakes via scheduler (`TIMEOUT`) or correlator (`MATCHED`). | optional `matcher`, optional `timeoutSec` |
118
+ | `terminal` | Set `execution_state='completed'`. | none |
119
+
120
+ ## The Fence and the Poke Model
121
+
122
+ Two invariants hold the durability story together. Break either one and the failure is silent.
123
+
124
+ **1. The advance is the only writer of `__workflow_instances`.** Ticks (scheduler, correlator) are read-only: they select what looks actionable and enqueue a `workflow.advance` for it. The advance re-checks the precondition under `SELECT ... FOR UPDATE` and does the write. Consequence: a crash anywhere in a tick costs nothing — whatever made the row look actionable is still true next tick, so the next tick pokes again. A stale or duplicate poke is a debug log, never a history row. The exceptions are all deliberate settle points outside the driver loop: the job-failure hooks (`failAdvanceJob` / `failEffectJob`) and the `cancel` / `retry` admin calls — each fenced or `seq`-bumping in the same way.
125
+
126
+ **2. Every settle-point write bumps `seq`; every job carries the `seq` it was issued against.** `expected_seq` on `AdvanceJobPayload`, `seq` on `EffectJobPayload`. On a mismatch the job is dropped:
127
+
128
+ | Where | Check | On mismatch |
129
+ | ---------------------------------- | ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
130
+ | `runAdvance` | `row.seq !== payload.expected_seq` | debug log, no-op |
131
+ | `runEffect` | `row.seq !== payload.seq` | `row.seq > payload.seq` → skip (`{ skipped: "stale" }`); `row.seq < payload.seq` → **throw**, so steve retries: the dispatching tx has not committed yet |
132
+ | `failAdvanceJob` / `failEffectJob` | same | no-op — a dead job must not fail an instance that has moved on |
133
+
134
+ This is what makes at-least-once job delivery safe. Duplicate deliveries are _expected_, not exceptional.
135
+
136
+ **Per-`kind` preconditions** are the second half of the fence, and only apply when the payload states its `kind` (a legacy payload's kind is a guess):
137
+
138
+ | `kind` | Requires |
139
+ | --------- | ------------------------------------------------------------------- |
140
+ | `start` | `execution_state = 'pending'` |
141
+ | `effect` | `execution_state = 'running'` |
142
+ | `signal` | `execution_state = 'waiting'` |
143
+ | `timeout` | `execution_state = 'waiting'` **and** `wake_at` set and in the past |
144
+
145
+ `timeout` re-checks due-ness because the tick is read-only — the timer is still live until this advance clears it, and applying `TIMEOUT` to a not-yet-due row would cut the wait short.
146
+
147
+ **Signal delivery commits with the transition.** A `kind: "signal"` advance carries `inbox_id`, not the payload: the driver locks the inbox row, reads the payload off it, transitions, and calls `markProcessed` — all in one transaction. A signal is therefore never consumed without being delivered. The correlator's own tick never marks a row processed except when it is _rejecting_ it (no live owner, or the matcher said no).
148
+
149
+ **Deferral.** The correlator leaves a row unprocessed (and re-examines it next tick) when the instance is live but not yet `waiting`, when the current node has no `MATCHED`/`*` edge, when the matcher throws, or when that instance was already poked this tick. Rationale: an early signal is early, not wrong; consuming it would lose it, and poking a node with no `MATCHED` edge would make the fsm reject and fail the instance. Known cost: a deterministic matcher throw retries (and logs) every tick, and a deferred backlog larger than `tickBatchSize` shadows newer rows (`ORDER BY received_at`).
100
150
 
101
151
  ## Public API (from `src/mod.ts`)
102
152
 
103
- | Export | Type | Purpose |
104
- |---|---|---|
105
- | `Workflow` | class | Main driver: attaches handlers to an injected `steve.Jobs`. Exposes `create/find/appendInbox/enqueueAdvance/enqueueEffect`. **No own start/stop** — consumer runs the Jobs. |
106
- | `WorkflowScheduler` | class | Attaches a wake tick to an injected `cron.Cron`. Exposes `register/unregister/tickOnce`. **No own start/stop** — consumer runs the Cron. |
107
- | `WorkflowInboxCorrelator` | class | Attaches a match tick to an injected `cron.Cron`. Exposes `register/unregister/tickOnce`. **No own start/stop** — consumer runs the Cron. |
108
- | `WorkflowRegistry` | class | Immutable registry of definitions + handlers + matchers; validates on construction |
109
- | `validateDefinition` | function | Standalone validator usable outside `WorkflowRegistry` |
110
- | `defKey(id, version)` | function | `"id@version"` registry key helper |
111
- | `createMigrate(pool)` | function | Builds `@marianmeres/migrate` instance pre-loaded with schema versions |
112
- | `getHistory(exec, id, limit?)` | function | Reads `__workflow_history` rows for an instance (observability) |
113
- | `effectJobType(name)` | function | Steve job-type string for an effect handler: `workflow.effect.<name>` |
114
- | `JOB_TYPE_ADVANCE` | const | `"workflow.advance"` |
115
- | `JOB_TYPE_EFFECT_PREFIX` | const | `"workflow.effect."` |
116
- | `PURE_ENTER_EVENT` | const | `"ENTER"` — synthetic event fired by driver into pure nodes |
117
- | `DEFAULT_PROJECT_ID` | const | `"_default"` |
118
- | `EXECUTION_STATE` | const | `{ PENDING, RUNNING, WAITING, COMPLETED, FAILED, CANCELLED }` |
119
- | `HISTORY_EVENT` | const | Audit event-type strings (`created`, `transition`, `effect_dispatched`, ...) |
120
- | `NodeMeta` | type | Discriminated union: `pure` \| `effectful` \| `suspending` \| `terminal` |
121
- | `WorkflowDefinition` | type | `{ id, version, fsm: FSMConfig }` |
122
- | `Handler` | type | `(args: HandlerArgs) => Promise<HandlerResult> \| HandlerResult` |
123
- | `Matcher` | type | `(args: MatcherArgs) => boolean \| Promise<boolean>` |
124
- | `WorkflowInstanceRow` | type | Schema-aligned row type |
125
- | `InboxRow` | type | Schema-aligned row type |
126
- | `HistoryRow` | type | Schema-aligned row type |
127
- | `WorkflowContext` | type | `Record<string, unknown>` — accumulated payload |
128
- | `WorkflowSnapshot` | type | `FSMSnapshot<string, WorkflowContext>` |
153
+ | Export | Type | Purpose |
154
+ | ------------------------------ | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
155
+ | `Workflow` | class | Main driver: attaches handlers to an injected `steve.Jobs`. Exposes `create/find/cancel/retry/appendInbox/enqueueAdvance/enqueueEffect`. **No own start/stop** — consumer runs the Jobs. |
156
+ | `WorkflowScheduler` | class | Attaches a wake tick to an injected `cron.Cron`. Exposes `register/unregister/tickOnce`. **No own start/stop** — consumer runs the Cron. |
157
+ | `WorkflowInboxCorrelator` | class | Attaches a match tick to an injected `cron.Cron`. Exposes `register/unregister/tickOnce`. **No own start/stop** — consumer runs the Cron. |
158
+ | `WorkflowRegistry` | class | Immutable registry of definitions + handlers + matchers; validates on construction |
159
+ | `validateDefinition` | function | Standalone validator usable outside `WorkflowRegistry` |
160
+ | `defKey(id, version)` | function | `"id@version"` registry key helper |
161
+ | `createMigrate(pool)` | function | Builds `@marianmeres/migrate` instance pre-loaded with schema versions |
162
+ | `getHistory(exec, id, limit?)` | function | Reads `__workflow_history` rows for an instance (observability) |
163
+ | `effectJobType(name)` | function | Steve job-type string for an effect handler: `workflow.effect.<name>` |
164
+ | `JOB_TYPE_ADVANCE` | const | `"workflow.advance"` |
165
+ | `JOB_TYPE_EFFECT_PREFIX` | const | `"workflow.effect."` |
166
+ | `PURE_ENTER_EVENT` | const | `"ENTER"` — synthetic event fired by driver into pure nodes |
167
+ | `DEFAULT_TENANT_ID` | const | `"_default"` |
168
+ | `EXECUTION_STATE` | const | `{ PENDING, RUNNING, WAITING, COMPLETED, FAILED, CANCELLED }` |
169
+ | `HISTORY_EVENT` | const | Audit event-type strings (`created`, `transition`, `effect_dispatched`, ...) |
170
+ | `NodeMeta` | type | Discriminated union: `pure` \| `effectful` \| `suspending` \| `terminal` |
171
+ | `AdvanceKind` | type | `"start" \| "effect" \| "timeout" \| "signal"` — what produced an advance; drives its preconditions |
172
+ | `SchedulerTickResult` | type | `{ woken, repoked }` — what one scheduler tick poked |
173
+ | `WorkflowDefinition` | type | `{ id, version, fsm: WorkflowFSMConfig }` |
174
+ | `WorkflowFSMConfig` | type | `FSMConfig` with `states: Record<TState, WorkflowStateConfig>` — narrowing only, still a valid `FSMConfig` |
175
+ | `WorkflowStateConfig` | type | fsm state config with `meta: NodeMeta` required instead of `meta?: unknown` |
176
+ | `Handler` | type | `(args: HandlerArgs) => Promise<HandlerResult> \| HandlerResult` |
177
+ | `Matcher` | type | `(args: MatcherArgs) => boolean \| Promise<boolean>` |
178
+ | `WorkflowInstanceRow` | type | Schema-aligned row type |
179
+ | `InboxRow` | type | Schema-aligned row type |
180
+ | `HistoryRow` | type | Schema-aligned row type |
181
+ | `WorkflowContext` | type | `Record<string, unknown>` — accumulated payload |
182
+ | `WorkflowSnapshot` | type | `FSMSnapshot<string, WorkflowContext>` |
129
183
 
130
184
  See [API.md](./API.md) for full signatures, options, and examples.
131
185
 
@@ -133,32 +187,47 @@ See [API.md](./API.md) for full signatures, options, and examples.
133
187
 
134
188
  The driver passes outcome labels as fsm events. Three labels have special meaning:
135
189
 
136
- | Label | Source | Used by |
137
- |---|---|---|
138
- | `ENTER` | Driver (auto) | Pure nodes — wire guarded transitions on this event |
139
- | `MATCHED` | `WorkflowInboxCorrelator` | Suspending nodes whose matcher returns true |
140
- | `TIMEOUT` | `WorkflowScheduler` | Suspending nodes whose `wake_at` fires |
190
+ | Label | Source | Used by |
191
+ | --------- | ------------------------- | --------------------------------------------------- |
192
+ | `ENTER` | Driver (auto) | Pure nodes — wire guarded transitions on this event |
193
+ | `MATCHED` | `WorkflowInboxCorrelator` | Suspending nodes whose matcher returns true |
194
+ | `TIMEOUT` | `WorkflowScheduler` | Suspending nodes whose `wake_at` fires |
141
195
 
142
196
  Effectful node outcomes are whatever the handler returns (e.g. `LOW`, `OK`, `CONFIRMED`).
143
197
 
144
198
  ## DB Schema (managed by `createMigrate`)
145
199
 
146
- | Table | Purpose | Indexes |
147
- |---|---|---|
148
- | `__workflow_instances` | Live instance rows | `(project_id, execution_state, wake_at)` partial, `(project_id, execution_state, correlation_token)` partial |
149
- | `__workflow_inbox` | Append-only intake of external signals | `(project_id, correlation_token)` partial WHERE unprocessed |
150
- | `__workflow_history` | Append-only audit log | `(instance_id, at)` |
151
- | `__workflow_migrations` | Active migration version | — |
200
+ | Table | Purpose | Indexes |
201
+ | ----------------------- | ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
202
+ | `__workflow_instances` | Live instance rows (incl. the `seq` fence) | `(tenant_id, execution_state, wake_at)` partial, `(tenant_id, execution_state, correlation_token)` partial, `(tenant_id, updated_at)` partial WHERE `execution_state = 'pending'` |
203
+ | `__workflow_inbox` | Append-only intake of external signals | `(tenant_id, correlation_token)` partial WHERE unprocessed |
204
+ | `__workflow_history` | Append-only audit log | `(instance_id, at)` |
205
+ | `__workflow_migrations` | Active migration version | — |
152
206
 
153
207
  Steve and Cron own their own tables (`__job*`, `__cron*`); managed by their respective libraries.
154
208
 
209
+ **Schema versions** (registered in `src/migrations/index.ts`, applied via `createMigrate(pool).up("latest")`):
210
+
211
+ | Version | Change |
212
+ | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
213
+ | `1.0.0` | Creates the three framework tables + indexes. |
214
+ | `1.1.0` | Renames `project_id` -> `tenant_id` on all three tables (in place, data preserved, idempotent, reversible). |
215
+ | `1.2.0` | Adds `seq integer NOT NULL DEFAULT 0` (the fence) to `__workflow_instances` + the partial index for the stale-`pending` scan. Additive and non-volatile — on PG >= 11 a metadata-only change, existing rows start at `0`. |
216
+
217
+ Migration files are an append-only ledger: `1_0_0.ts` still creates `project_id` and is deliberately left untouched — `1_1_0.ts` performs the rename, so fresh installs and upgrades converge on the same schema.
218
+
219
+ `driver.ts` carries two transitional shims for jobs enqueued by an older version and still sitting in the queue at upgrade time. Both are removable together, once no pre-2.1 job can be queued anywhere:
220
+
221
+ - `payloadTenantId()` falls back to a legacy `project_id` key (pre-1.1.0 payloads).
222
+ - `advanceKind()` infers `kind` from the payload shape when it is absent (pre-fence payloads). Such a payload also has no `expected_seq`, i.e. it is **unfenced**: it runs without the fence check, and without the per-`kind` preconditions — guessing the kind must not swallow the job.
223
+
155
224
  ## Runtime Ownership
156
225
 
157
226
  The framework **does not start or stop steve.Jobs or cron.Cron.** The consumer constructs both, passes them in, and runs their lifecycle:
158
227
 
159
228
  ```ts
160
- const jobs = new Jobs({ db: pool }); // consumer-owned
161
- const cron = new Cron({ db: pool }); // consumer-owned
229
+ const jobs = new Jobs({ db: pool, autoCleanup: true }); // consumer-owned
230
+ const cron = new Cron({ db: pool }); // consumer-owned
162
231
 
163
232
  const wf = new Workflow({ db: pool, jobs, ... }); // setHandler on jobs
164
233
  const scheduler = new WorkflowScheduler({ cron, workflow: wf }); // register tick on cron
@@ -174,17 +243,28 @@ await cron.stop(); // consumer
174
243
  await jobs.stop(); // consumer
175
244
  ```
176
245
 
177
- The consumer's other steve job types and cron jobs coexist on the same `Jobs` / `Cron` instances — handlers are keyed by `type` (steve) / `name` (cron), and the workflow's are namespaced (`workflow.advance`, `workflow.effect.<name>`, `workflow.scheduler.<projectId>`, `workflow.correlator.<projectId>`) so collisions are unlikely. Running parallel `Jobs` instances on the same `__job` table is **unsafe**: an instance polling a type it doesn't know falls back to a noop handler that silently marks the job completed (see [steve/jobs.ts:499](file:///Users/mm/projects/@marianmeres/steve/src/steve/jobs.ts)). Always share.
246
+ `autoCleanup` is load-bearing, not a tuning knob: steve's reaper is what turns a crashed worker into an `expired` job, which is what the framework's re-dispatch reacts to. Default reap threshold is 5 minutes of `running`; the check interval defaults to 60s.
247
+
248
+ The consumer's other steve job types and cron jobs coexist on the same `Jobs` / `Cron` instances — handlers are keyed by `type` (steve) / `name` (cron), and the workflow's are namespaced (`workflow.advance`, `workflow.effect.<name>`, `workflow.scheduler.<tenantId>`, `workflow.correlator.<tenantId>`) so collisions are unlikely.
249
+
250
+ **Both runtimes noop what they cannot handle**, which makes a second unregistered runtime a silent data-loss bug rather than a loud one:
251
+
252
+ - A `Jobs` instance polling a `type` it doesn't know falls back to a noop handler that marks the job completed ([steve/jobs.ts](file:///Users/mm/projects/@marianmeres/steve/src/steve/jobs.ts)). What that costs depends on which job was eaten: a `timeout`/`signal` poke is re-poked next tick and a `start` advance is recovered by the stale-`pending` scan, but a noop'd **effect job or effect-completion advance strands the instance in `running` with nothing left to move it** — the reaper never sees it, because the job is `completed`.
253
+ - A `Cron` instance claims due rows **globally, regardless of tenant**, and runs a noop with a `warn` for a `name` it has no handler for ([cron/cron.ts](file:///Users/mm/projects/@marianmeres/cron/src/cron/cron.ts)) — then advances `next_run_at` as if the tick had run. A process that starts a `Cron` without calling `scheduler.register()` / `correlator.register()` therefore _steals_ ticks from the process that did.
254
+
255
+ Rule: one `Jobs` and one `Cron` per process, and every process that _starts_ them on these tables constructs the `Workflow` and registers the ticks. A producer-only process (`create`, `appendInbox`) still needs a `Jobs` to construct the `Workflow` — it just never calls `jobs.start()` / `cron.start()`.
178
256
 
179
257
  ## Critical Conventions
180
258
 
181
- 1. **Definitions are pure data.** Handlers and matchers are referenced by string name. Never store function pointers in a definition. Litmus test: `JSON.stringify(def)` must lose no meaning.
259
+ 1. **Definitions are pure data.** Handlers and matchers are referenced by string name. Never store function pointers in a definition. Litmus test: `JSON.stringify(def)` must lose no meaning. Threading a handler's output into the context needs no `action` hook either — the handler returns `HandlerResult.context`, a shallow patch merged before the outcome transition.
182
260
  2. **Version-pin instances.** Every instance carries `(definition_id, definition_version)`. Never edit an in-use definition in place — bump the version. Old versions stay registered until the last instance using them completes.
183
- 3. **Handlers must be idempotent.** Steve may retry handlers if a worker crashes mid-execution. Side effects must be safe to repeat.
261
+ 3. **Handlers must be idempotent.** The `seq` fence protects the instance row, not the outside world: a worker that finished the side effect and died before steve recorded the attempt gets retried, and the retry is _not_ stale (the instance never left that `seq`). Side effects must be safe to repeat.
184
262
  4. **No fsm `onEnter`/`onExit` hooks for side effects.** Driver re-constructs FSMs via `FSM.fromSnapshot()`, which skips `onEnter`. Side effects belong in effectful handlers — they're the only way to get durable, retryable, observable execution.
185
263
  5. **Cursor + execution_state are separate columns.** Don't fold one into the other.
186
264
  6. **Outcome labels are fsm events.** The driver calls `fsm.transition(outcome, data)`. If fsm rejects (unknown event in current state), the instance is marked `failed` and history records `transition_rejected`.
187
265
  7. **Correlation token is the index, matcher is the gate.** Token alone is not sufficient — the matcher does the semantic check.
266
+ 8. **Only the advance writes instance rows.** Anything else that wants an instance moved enqueues an advance and lets it re-check under the lock. A new settle-point write must pass `{ bumpSeq: true }` to `updateInstance` and a new job payload must carry the fence, or duplicate delivery stops being safe.
267
+ 9. **Fenced-out work is not an event.** Duplicate/stale jobs and pokes get a `clog.debug`, never a history row — with re-poking ticks they are routine, and history is an event log, not a trace.
188
268
 
189
269
  ## Before Making Changes
190
270
 
@@ -192,49 +272,82 @@ The consumer's other steve job types and cron jobs coexist on the same `Jobs` /
192
272
  - [ ] Run `deno task test` (set `TEST_PG_*` env first; see `.env` example)
193
273
  - [ ] If adding a new history event type, add it to `HISTORY_EVENT` in [src/types.ts](./src/types.ts) and emit it consistently
194
274
  - [ ] If changing schema, write a new migration in [src/migrations/](./src/migrations/) — never edit `1_0_0.ts` in place
275
+ - [ ] Changed the public API? Check [example/](./example) still compiles (`deno task check`) and still tells the truth — it is the one consumer in the repo
276
+ - [ ] Touched `example/src/`? Run `deno task example:build` — `example/dist/bundle.js` is committed
195
277
 
196
278
  ## Dependency Versions
197
279
 
198
- | Package | Min | Why |
199
- |---|---|---|
200
- | `@marianmeres/fsm` | `^3.1.0` | Requires `state.meta` and `FSM.fromSnapshot` (added in 3.1.0) |
201
- | `@marianmeres/steve` | `^2.0.2` | Job queue (advance + effect dispatch) |
202
- | `@marianmeres/cron` | `^2.0.1` | Scheduler + correlator ticks |
203
- | `@marianmeres/migrate` | `^1.2.2` | Schema versioning |
204
- | `@marianmeres/clog` | `^3.21.0` | Logging |
205
- | `pg` | `^8.20.0` | PostgreSQL client |
280
+ | Package | Min | Why |
281
+ | ---------------------- | --------- | ------------------------------------------------------------- |
282
+ | `@marianmeres/fsm` | `^3.1.0` | Requires `state.meta` and `FSM.fromSnapshot` (added in 3.1.0) |
283
+ | `@marianmeres/steve` | `^3.0.0` | Job queue (advance + effect dispatch) |
284
+ | `@marianmeres/cron` | `^3.2.0` | Scheduler + correlator ticks |
285
+ | `@marianmeres/migrate` | `^1.3.1` | Schema versioning |
286
+ | `@marianmeres/clog` | `^3.21.0` | Logging |
287
+ | `pg` | `^8.23.0` | PostgreSQL client |
206
288
 
207
289
  ## Failure Modes & Recovery
208
290
 
209
- | Failure | Behavior |
210
- |---|---|
211
- | Handler throws | Steve retries per `effectMaxAttempts`. On terminal failure, `Workflow`'s `onDone` listener marks the instance `failed` + appends `effect_failed` history. |
212
- | fsm rejects outcome | Driver appends `transition_rejected` history, marks instance `failed`. |
213
- | Pure node with no matching guard | Driver appends `transition_rejected` history, marks instance `failed`. |
214
- | Pure-state cycle (>64 hops) | Driver appends `failed` history with "exceeded MAX_PURE_HOPS" reason. |
215
- | Definition missing on advance | Driver marks instance `failed`. (Pin-and-freeze: keep old versions registered until drained.) |
216
- | Crashed worker mid-effect | Steve auto-cleanup reaps stuck `running` jobs. Steve retries per `max_attempts`. |
217
- | Lost advance after effect (process dies between handler success and `jobs.create('workflow.advance')`) | Steve retries the effect job; handler runs again; advance enqueued. Handlers must be idempotent. |
291
+ | Failure | Behavior |
292
+ | ----------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
293
+ | Handler throws | Steve retries the job `effectMaxAttempts` times. On terminal failure (`status: failed`), `Workflow`'s `onDone` listener marks the instance `failed` + appends `effect_failed` history. |
294
+ | Advance job throws | Steve retries the job `advanceMaxAttempts` times (default 10 — ~17min of exponential backoff, so a brief DB outage costs nothing). When they are exhausted, `onDone` marks the instance `failed` with the last attempt's `error_message`. |
295
+ | fsm rejects outcome | Driver appends `transition_rejected` history, marks instance `failed`. |
296
+ | Pure node with no matching guard | Driver appends `transition_rejected` history, marks instance `failed`. |
297
+ | Pure-state cycle (>64 hops) | Driver appends `failed` history with "exceeded MAX_PURE_HOPS" reason. |
298
+ | Definition missing on advance | Driver marks instance `failed`. (Pin-and-freeze: keep old versions registered until drained.) |
299
+ | Crashed worker mid-job (advance or effect) | Steve's reaper marks the job `expired` — **terminal, never retried by steve**. `Workflow`'s `onDone` re-queues the identical payload instead, `redispatchLimit` dispatches in total (default 3), then fails the instance. Safe because the fence no-ops a re-dispatch whose original did commit. **Requires the reaper to run at all**: `new Jobs({ db, autoCleanup: true })`, or a periodic `jobs.cleanup()`. Without it the job sits in `running` forever and the instance never moves. |
300
+ | Lost advance after effect (the side effect happened, `jobs.create('workflow.advance')` did not) | A throwing enqueue → steve retries the effect job; a dead process → the job expires and is re-dispatched. Either way the fence lets the retry through — the instance never left that `seq` — so the handler runs again and re-enqueues the advance. Handlers must be idempotent. |
301
+ | Crash between `create()`'s commit and steve's job insert | The instance sits in `pending` with no job. The scheduler's second scan re-pokes it after `stalePendingSec` (default 300s; `0` disables). |
302
+ | Duplicate advance / effect job | Fenced out by `seq`: debug-logged no-op, no history row. Duplicates are routine (every tick re-pokes what still looks due), so they are not events. |
303
+ | Second `Jobs` / `Cron` without the workflow handlers | Silent loss — see [Runtime Ownership](#runtime-ownership). |
304
+
305
+ ## The Example App
306
+
307
+ [example/](./example) is consumer code, not a test: it imports `@marianmeres/workflow` by
308
+ package name (mapped to `./src/mod.ts` in `deno.json`) and wires the runtime exactly the
309
+ way the README does. It is also the fastest way to see the two orthogonal states, the
310
+ timer and the poke model behave, so keep it working.
311
+
312
+ Three things in it deviate from production defaults, all commented where they are set:
313
+
314
+ | Deviation | Why |
315
+ | --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
316
+ | `scheduler.tickOnce()` / `correlator.tickOnce()` on a 500 ms interval | a cron expression cannot go below a minute. The real cron ticks stay registered; the UI can switch the fast loop off. |
317
+ | `effectMaxAttempts: 1` | so a throwing handler becomes a `failed` instance immediately, with a Retry button. The default 3 would heal it invisibly. |
318
+ | `stalePendingSec: 30` (default 300) | so the stranded-`pending` re-poke is witnessable. |
319
+
320
+ It also does two things the package deliberately does not offer, and both are marked as
321
+ such: **listing instances and inbox rows with raw SQL** (the package exposes `find(id)` /
322
+ `getHistory(id)` — anything that lists is an admin surface, i.e. your query), and nothing
323
+ else. If you add a listing helper to `src/`, the example is where it should be used first.
324
+
325
+ Touching `example/src/` means re-running `deno task example:build`; the bundle is
326
+ committed so the app runs with no build step.
218
327
 
219
328
  ## Testing
220
329
 
221
- Integration tests in [tests/stock_replenishment.test.ts](./tests/stock_replenishment.test.ts) cover happy, timeout, and failure paths against a real PostgreSQL. Skip gracefully when `TEST_PG_DATABASE` / `TEST_PG_USER` are unset.
330
+ All tests run against a real PostgreSQL and skip gracefully when `TEST_PG_DATABASE` / `TEST_PG_USER` are unset — the driver is SQL-bound, so mocking it would test nothing. [stock_replenishment.test.ts](./tests/stock_replenishment.test.ts) covers happy/timeout/failure paths, [durability.test.ts](./tests/durability.test.ts) the fence and the recovery paths, [correlator.test.ts](./tests/correlator.test.ts) the signal semantics.
222
331
 
223
- Per-tick helpers `WorkflowScheduler.tickOnce()` and `WorkflowInboxCorrelator.tickOnce()` are exposed for tests so they don't have to wait for cron expressions.
332
+ Per-tick helpers `WorkflowScheduler.tickOnce()` and `WorkflowInboxCorrelator.tickOnce()` are exposed for tests so they don't have to wait for cron expressions. Durability behavior is generally testable without a running `Jobs`: enqueue nothing, call `runAdvance` / `runEffect` directly with a hand-built payload, and assert on the row + history.
224
333
 
225
334
  ## Glossary
226
335
 
227
- | Term | Meaning |
228
- |---|---|
229
- | Definition | Static, versioned, declarative shape of a workflow. Pure data |
230
- | Instance | Single running occurrence of a definition; one DB row |
231
- | Driver | The engine (`workflow.advance` steve handler) that advances instances |
232
- | Cursor | The instance's current node id |
233
- | Execution state | The instance's lifecycle status |
234
- | Outcome | The label a handler returns (or `MATCHED`/`TIMEOUT`/`ENTER`); fed to fsm as the event name |
235
- | Effect port | A typed interface to the outside world (email, AI, HTTP) — userland concern |
236
- | Inbox | Append-only `__workflow_inbox` intake of external signals |
237
- | Correlation token | Unique token that maps an external event back to a waiting instance |
238
- | Scheduler | Cron-driven loop that wakes time-suspended instances |
239
- | Correlator | Cron-driven loop that matches inbox signals to waiting instances |
240
- | Pin-and-freeze | Versioning strategy: every instance runs on its birth-version forever |
336
+ | Term | Meaning |
337
+ | ----------------- | -------------------------------------------------------------------------------------------------------------------- |
338
+ | Definition | Static, versioned, declarative shape of a workflow. Pure data |
339
+ | Instance | Single running occurrence of a definition; one DB row |
340
+ | Driver | The engine (`workflow.advance` steve handler) that advances instances |
341
+ | Cursor | The instance's current node id |
342
+ | Execution state | The instance's lifecycle status |
343
+ | Outcome | The label a handler returns (or `MATCHED`/`TIMEOUT`/`ENTER`); fed to fsm as the event name |
344
+ | Effect port | A typed interface to the outside world (email, AI, HTTP) — userland concern |
345
+ | Inbox | Append-only `__workflow_inbox` intake of external signals |
346
+ | Correlation token | Unique token that maps an external event back to a waiting instance |
347
+ | Scheduler | Cron-driven loop that wakes time-suspended instances |
348
+ | Correlator | Cron-driven loop that matches inbox signals to waiting instances |
349
+ | Fence | The `seq` column. Bumped by every settle-point write; carried by every job, so a stale one is recognized and dropped |
350
+ | Poke | A `workflow.advance` job enqueued by something that will not write the row itself. The advance re-checks and decides |
351
+ | Settle point | The end of an advance: terminal / effectful / suspending. Where the instance row is written and `seq` bumped |
352
+ | Defer | Correlator verdict: leave the inbox row unprocessed and re-examine it next tick |
353
+ | Pin-and-freeze | Versioning strategy: every instance runs on its birth-version forever |