@marianmeres/workflow 1.0.0 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +207 -94
- package/API.md +277 -157
- package/README.md +182 -83
- package/dist/correlator.d.ts +31 -14
- package/dist/correlator.js +130 -73
- package/dist/definition.d.ts +5 -1
- package/dist/definition.js +54 -1
- package/dist/driver.d.ts +28 -23
- package/dist/driver.js +223 -59
- package/dist/migrations/1_1_0.d.ts +11 -0
- package/dist/migrations/1_1_0.js +61 -0
- package/dist/migrations/1_2_0.d.ts +16 -0
- package/dist/migrations/1_2_0.js +44 -0
- package/dist/migrations/index.js +5 -1
- package/dist/mod.d.ts +8 -5
- package/dist/mod.js +7 -4
- package/dist/persistence/history.d.ts +1 -1
- package/dist/persistence/history.js +3 -3
- package/dist/persistence/inbox.d.ts +9 -3
- package/dist/persistence/inbox.js +16 -7
- package/dist/persistence/instances.d.ts +46 -13
- package/dist/persistence/instances.js +78 -33
- package/dist/scheduler.d.ts +33 -9
- package/dist/scheduler.js +45 -21
- package/dist/types.d.ts +126 -13
- package/dist/types.js +28 -2
- package/dist/workflow.d.ts +76 -4
- package/dist/workflow.js +237 -35
- package/docs/_archive/analysis/00-overview-and-roadmap.md +233 -0
- package/docs/_archive/analysis/01-durability.md +404 -0
- package/docs/_archive/analysis/02-correlation.md +228 -0
- package/docs/_archive/analysis/03-definition-and-api.md +248 -0
- package/docs/_archive/analysis/04-docs-tests-ops.md +191 -0
- package/docs/_archive/analysis/PROGRESS.md +172 -0
- package/docs/_archive/analysis/README.md +38 -0
- package/package.json +8 -7
package/AGENTS.md
CHANGED
|
@@ -4,7 +4,9 @@
|
|
|
4
4
|
|
|
5
5
|
- **Stack**: Deno/Node.js, PostgreSQL, TypeScript
|
|
6
6
|
- **Test**: `deno task test` (requires `TEST_PG_*` env; see [tests/_pg.ts](./tests/_pg.ts))
|
|
7
|
+
- **Check**: `deno task check` — `deno fmt --check && deno lint && deno check` over `src/`, `tests/` and the example's server-side files
|
|
7
8
|
- **Build (npm)**: `deno task npm:build`
|
|
9
|
+
- **Example app**: `deno task example` (needs `EXAMPLE_PG_*`); `deno task example:build` after touching `example/src/`, because `example/dist/bundle.js` is committed. See [example/README.md](./example/README.md).
|
|
8
10
|
- **Entry point**: [src/mod.ts](./src/mod.ts)
|
|
9
11
|
|
|
10
12
|
## Package Overview
|
|
@@ -54,78 +56,130 @@ src/
|
|
|
54
56
|
├── correlator.ts # WorkflowInboxCorrelator (attaches match tick to injected Cron)
|
|
55
57
|
├── log.ts # createClog("workflow")
|
|
56
58
|
├── persistence/
|
|
57
|
-
│ ├── instances.ts # CRUD + lock +
|
|
58
|
-
│ ├── inbox.ts # append, claim-unprocessed, mark-processed
|
|
59
|
+
│ ├── instances.ts # CRUD + lock + seq bump + due/stale scans + find by correlation
|
|
60
|
+
│ ├── inbox.ts # append, claim-unprocessed, lock-row, mark-processed
|
|
59
61
|
│ ├── history.ts # append + read
|
|
60
62
|
│ └── tx.ts # withTransaction helper
|
|
61
63
|
└── migrations/
|
|
62
64
|
├── 1_0_0.ts # CREATE/DROP three tables
|
|
65
|
+
├── 1_1_0.ts # rename project_id -> tenant_id (in place)
|
|
66
|
+
├── 1_2_0.ts # add seq (fence) + stale-pending partial index
|
|
63
67
|
└── index.ts # createMigrate(pool) — @marianmeres/migrate setup
|
|
64
68
|
tests/
|
|
65
69
|
├── _pg.ts # env-driven pool + schema reset
|
|
66
70
|
├── fixtures/
|
|
67
|
-
│
|
|
68
|
-
└──
|
|
71
|
+
│ ├── stock-replenishment.ts # reference workflow used by tests only
|
|
72
|
+
│ └── weekly-digest.ts # recurring-trigger reference workflow
|
|
73
|
+
├── admin.test.ts # cancel/retry: guards, fencing of in-flight jobs, resume
|
|
74
|
+
├── correlator.test.ts # deferral, dedupe, matcher throw/reject, delivery-in-advance
|
|
75
|
+
├── durability.test.ts # fence, preconditions, expiry re-dispatch, stale-pending re-poke
|
|
76
|
+
├── migrations.test.ts # schema rename/fence up/down + legacy payload drain
|
|
77
|
+
├── stock_replenishment.test.ts # happy/timeout/fail paths + validator
|
|
78
|
+
└── weekly_digest.test.ts # recurring trigger via cron
|
|
79
|
+
example/ # the interactive app — consumer code, imports by package name
|
|
80
|
+
├── workflow.ts # userland: one definition + handlers + matcher
|
|
81
|
+
├── server.ts # runtime wiring + HTTP API (raw SQL for the listing views)
|
|
82
|
+
├── src/main.ts # the page (@marianmeres/vanilla); bundled to dist/bundle.js
|
|
83
|
+
├── index.html # shell, styles, <template>s
|
|
84
|
+
└── theme.css, reboot.css # generated / vendored; excluded from fmt
|
|
85
|
+
scripts/
|
|
86
|
+
├── build-npm.ts
|
|
87
|
+
└── gen-example-{version,theme}.ts # generated example assets; run by the example tasks
|
|
69
88
|
```
|
|
70
89
|
|
|
71
90
|
## Conceptual Model — Three Layers
|
|
72
91
|
|
|
73
|
-
| Layer
|
|
74
|
-
|
|
75
|
-
| **Definition** | Static graph: nodes, transitions, handler ids (strings). Pure data, JSON-safe.
|
|
76
|
-
| **Instance**
|
|
77
|
-
| **Driver**
|
|
92
|
+
| Layer | What it is | Lives in |
|
|
93
|
+
| -------------- | --------------------------------------------------------------------------------------------------- | -------------------------------------- |
|
|
94
|
+
| **Definition** | Static graph: nodes, transitions, handler ids (strings). Pure data, JSON-safe. | Code, registered in `WorkflowRegistry` |
|
|
95
|
+
| **Instance** | Live state of one running occurrence. Cursor, context, execution_state, wake_at, correlation_token. | `__workflow_instances` row |
|
|
96
|
+
| **Driver** | Event-driven engine that advances instances. Owns no state. | `workflow.advance` steve job |
|
|
78
97
|
|
|
79
98
|
## Two Orthogonal States
|
|
80
99
|
|
|
81
100
|
Never conflate. Always two columns on `__workflow_instances`:
|
|
82
101
|
|
|
83
|
-
| Column
|
|
84
|
-
|
|
85
|
-
| `cursor`
|
|
102
|
+
| Column | Meaning | Source of truth for |
|
|
103
|
+
| ----------------- | --------------------------- | ------------------------------------------------------------------------ |
|
|
104
|
+
| `cursor` | Which node am I at? | The FSM's `state` (`getSnapshot().state`) |
|
|
86
105
|
| `execution_state` | What's my lifecycle status? | `pending` / `running` / `waiting` / `completed` / `failed` / `cancelled` |
|
|
87
106
|
|
|
88
107
|
An instance can be at node `await_reply` (cursor) and simultaneously `waiting` (execution_state).
|
|
89
108
|
|
|
90
109
|
## Node Taxonomy (`NodeMeta`)
|
|
91
110
|
|
|
92
|
-
Each FSM state's `meta` field carries a discriminated `NodeMeta
|
|
111
|
+
Each FSM state's `meta` field carries a discriminated `NodeMeta` — required and typed, because a definition's `fsm` is a `WorkflowFSMConfig`, not a raw `FSMConfig` (fsm's own `meta` is `unknown`). The driver dispatches on `meta.kind`:
|
|
93
112
|
|
|
94
|
-
| Kind
|
|
95
|
-
|
|
96
|
-
| `pure`
|
|
97
|
-
| `effectful`
|
|
98
|
-
| `suspending` | Set `execution_state='waiting'`, `wake_at` (if `timeoutSec`), keep `correlation_token
|
|
99
|
-
| `terminal`
|
|
113
|
+
| Kind | Driver behavior | Required fields |
|
|
114
|
+
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |
|
|
115
|
+
| `pure` | Auto-fire `ENTER` event; transition by guards. Loop inline until terminal/effectful/suspending. Max 64 hops. | none |
|
|
116
|
+
| `effectful` | Enqueue `workflow.effect.<handler>` steve job, flip to `running`. On completion, an `advance` is enqueued with `{ outcome, outcome_data, correlation_token? }`. | `handler: string` |
|
|
117
|
+
| `suspending` | Set `execution_state='waiting'`, `wake_at` (if `timeoutSec`), keep the `correlation_token` (or the one the last handler returned). Wakes via scheduler (`TIMEOUT`) or correlator (`MATCHED`). | optional `matcher`, optional `timeoutSec` |
|
|
118
|
+
| `terminal` | Set `execution_state='completed'`. | none |
|
|
119
|
+
|
|
120
|
+
## The Fence and the Poke Model
|
|
121
|
+
|
|
122
|
+
Two invariants hold the durability story together. Break either one and the failure is silent.
|
|
123
|
+
|
|
124
|
+
**1. The advance is the only writer of `__workflow_instances`.** Ticks (scheduler, correlator) are read-only: they select what looks actionable and enqueue a `workflow.advance` for it. The advance re-checks the precondition under `SELECT ... FOR UPDATE` and does the write. Consequence: a crash anywhere in a tick costs nothing — whatever made the row look actionable is still true next tick, so the next tick pokes again. A stale or duplicate poke is a debug log, never a history row. The exceptions are all deliberate settle points outside the driver loop: the job-failure hooks (`failAdvanceJob` / `failEffectJob`) and the `cancel` / `retry` admin calls — each fenced or `seq`-bumping in the same way.
|
|
125
|
+
|
|
126
|
+
**2. Every settle-point write bumps `seq`; every job carries the `seq` it was issued against.** `expected_seq` on `AdvanceJobPayload`, `seq` on `EffectJobPayload`. On a mismatch the job is dropped:
|
|
127
|
+
|
|
128
|
+
| Where | Check | On mismatch |
|
|
129
|
+
| ---------------------------------- | ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
130
|
+
| `runAdvance` | `row.seq !== payload.expected_seq` | debug log, no-op |
|
|
131
|
+
| `runEffect` | `row.seq !== payload.seq` | `row.seq > payload.seq` → skip (`{ skipped: "stale" }`); `row.seq < payload.seq` → **throw**, so steve retries: the dispatching tx has not committed yet |
|
|
132
|
+
| `failAdvanceJob` / `failEffectJob` | same | no-op — a dead job must not fail an instance that has moved on |
|
|
133
|
+
|
|
134
|
+
This is what makes at-least-once job delivery safe. Duplicate deliveries are _expected_, not exceptional.
|
|
135
|
+
|
|
136
|
+
**Per-`kind` preconditions** are the second half of the fence, and only apply when the payload states its `kind` (a legacy payload's kind is a guess):
|
|
137
|
+
|
|
138
|
+
| `kind` | Requires |
|
|
139
|
+
| --------- | ------------------------------------------------------------------- |
|
|
140
|
+
| `start` | `execution_state = 'pending'` |
|
|
141
|
+
| `effect` | `execution_state = 'running'` |
|
|
142
|
+
| `signal` | `execution_state = 'waiting'` |
|
|
143
|
+
| `timeout` | `execution_state = 'waiting'` **and** `wake_at` set and in the past |
|
|
144
|
+
|
|
145
|
+
`timeout` re-checks due-ness because the tick is read-only — the timer is still live until this advance clears it, and applying `TIMEOUT` to a not-yet-due row would cut the wait short.
|
|
146
|
+
|
|
147
|
+
**Signal delivery commits with the transition.** A `kind: "signal"` advance carries `inbox_id`, not the payload: the driver locks the inbox row, reads the payload off it, transitions, and calls `markProcessed` — all in one transaction. A signal is therefore never consumed without being delivered. The correlator's own tick never marks a row processed except when it is _rejecting_ it (no live owner, or the matcher said no).
|
|
148
|
+
|
|
149
|
+
**Deferral.** The correlator leaves a row unprocessed (and re-examines it next tick) when the instance is live but not yet `waiting`, when the current node has no `MATCHED`/`*` edge, when the matcher throws, or when that instance was already poked this tick. Rationale: an early signal is early, not wrong; consuming it would lose it, and poking a node with no `MATCHED` edge would make the fsm reject and fail the instance. Known cost: a deterministic matcher throw retries (and logs) every tick, and a deferred backlog larger than `tickBatchSize` shadows newer rows (`ORDER BY received_at`).
|
|
100
150
|
|
|
101
151
|
## Public API (from `src/mod.ts`)
|
|
102
152
|
|
|
103
|
-
| Export
|
|
104
|
-
|
|
105
|
-
| `Workflow`
|
|
106
|
-
| `WorkflowScheduler`
|
|
107
|
-
| `WorkflowInboxCorrelator`
|
|
108
|
-
| `WorkflowRegistry`
|
|
109
|
-
| `validateDefinition`
|
|
110
|
-
| `defKey(id, version)`
|
|
111
|
-
| `createMigrate(pool)`
|
|
112
|
-
| `getHistory(exec, id, limit?)` | function | Reads `__workflow_history` rows for an instance (observability)
|
|
113
|
-
| `effectJobType(name)`
|
|
114
|
-
| `JOB_TYPE_ADVANCE`
|
|
115
|
-
| `JOB_TYPE_EFFECT_PREFIX`
|
|
116
|
-
| `PURE_ENTER_EVENT`
|
|
117
|
-
| `
|
|
118
|
-
| `EXECUTION_STATE`
|
|
119
|
-
| `HISTORY_EVENT`
|
|
120
|
-
| `NodeMeta`
|
|
121
|
-
| `
|
|
122
|
-
| `
|
|
123
|
-
| `
|
|
124
|
-
| `
|
|
125
|
-
| `
|
|
126
|
-
| `
|
|
127
|
-
| `
|
|
128
|
-
| `
|
|
153
|
+
| Export | Type | Purpose |
|
|
154
|
+
| ------------------------------ | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
155
|
+
| `Workflow` | class | Main driver: attaches handlers to an injected `steve.Jobs`. Exposes `create/find/cancel/retry/appendInbox/enqueueAdvance/enqueueEffect`. **No own start/stop** — consumer runs the Jobs. |
|
|
156
|
+
| `WorkflowScheduler` | class | Attaches a wake tick to an injected `cron.Cron`. Exposes `register/unregister/tickOnce`. **No own start/stop** — consumer runs the Cron. |
|
|
157
|
+
| `WorkflowInboxCorrelator` | class | Attaches a match tick to an injected `cron.Cron`. Exposes `register/unregister/tickOnce`. **No own start/stop** — consumer runs the Cron. |
|
|
158
|
+
| `WorkflowRegistry` | class | Immutable registry of definitions + handlers + matchers; validates on construction |
|
|
159
|
+
| `validateDefinition` | function | Standalone validator usable outside `WorkflowRegistry` |
|
|
160
|
+
| `defKey(id, version)` | function | `"id@version"` registry key helper |
|
|
161
|
+
| `createMigrate(pool)` | function | Builds `@marianmeres/migrate` instance pre-loaded with schema versions |
|
|
162
|
+
| `getHistory(exec, id, limit?)` | function | Reads `__workflow_history` rows for an instance (observability) |
|
|
163
|
+
| `effectJobType(name)` | function | Steve job-type string for an effect handler: `workflow.effect.<name>` |
|
|
164
|
+
| `JOB_TYPE_ADVANCE` | const | `"workflow.advance"` |
|
|
165
|
+
| `JOB_TYPE_EFFECT_PREFIX` | const | `"workflow.effect."` |
|
|
166
|
+
| `PURE_ENTER_EVENT` | const | `"ENTER"` — synthetic event fired by driver into pure nodes |
|
|
167
|
+
| `DEFAULT_TENANT_ID` | const | `"_default"` |
|
|
168
|
+
| `EXECUTION_STATE` | const | `{ PENDING, RUNNING, WAITING, COMPLETED, FAILED, CANCELLED }` |
|
|
169
|
+
| `HISTORY_EVENT` | const | Audit event-type strings (`created`, `transition`, `effect_dispatched`, ...) |
|
|
170
|
+
| `NodeMeta` | type | Discriminated union: `pure` \| `effectful` \| `suspending` \| `terminal` |
|
|
171
|
+
| `AdvanceKind` | type | `"start" \| "effect" \| "timeout" \| "signal"` — what produced an advance; drives its preconditions |
|
|
172
|
+
| `SchedulerTickResult` | type | `{ woken, repoked }` — what one scheduler tick poked |
|
|
173
|
+
| `WorkflowDefinition` | type | `{ id, version, fsm: WorkflowFSMConfig }` |
|
|
174
|
+
| `WorkflowFSMConfig` | type | `FSMConfig` with `states: Record<TState, WorkflowStateConfig>` — narrowing only, still a valid `FSMConfig` |
|
|
175
|
+
| `WorkflowStateConfig` | type | fsm state config with `meta: NodeMeta` required instead of `meta?: unknown` |
|
|
176
|
+
| `Handler` | type | `(args: HandlerArgs) => Promise<HandlerResult> \| HandlerResult` |
|
|
177
|
+
| `Matcher` | type | `(args: MatcherArgs) => boolean \| Promise<boolean>` |
|
|
178
|
+
| `WorkflowInstanceRow` | type | Schema-aligned row type |
|
|
179
|
+
| `InboxRow` | type | Schema-aligned row type |
|
|
180
|
+
| `HistoryRow` | type | Schema-aligned row type |
|
|
181
|
+
| `WorkflowContext` | type | `Record<string, unknown>` — accumulated payload |
|
|
182
|
+
| `WorkflowSnapshot` | type | `FSMSnapshot<string, WorkflowContext>` |
|
|
129
183
|
|
|
130
184
|
See [API.md](./API.md) for full signatures, options, and examples.
|
|
131
185
|
|
|
@@ -133,32 +187,47 @@ See [API.md](./API.md) for full signatures, options, and examples.
|
|
|
133
187
|
|
|
134
188
|
The driver passes outcome labels as fsm events. Three labels have special meaning:
|
|
135
189
|
|
|
136
|
-
| Label
|
|
137
|
-
|
|
138
|
-
| `ENTER`
|
|
139
|
-
| `MATCHED` | `WorkflowInboxCorrelator` | Suspending nodes whose matcher returns true
|
|
140
|
-
| `TIMEOUT` | `WorkflowScheduler`
|
|
190
|
+
| Label | Source | Used by |
|
|
191
|
+
| --------- | ------------------------- | --------------------------------------------------- |
|
|
192
|
+
| `ENTER` | Driver (auto) | Pure nodes — wire guarded transitions on this event |
|
|
193
|
+
| `MATCHED` | `WorkflowInboxCorrelator` | Suspending nodes whose matcher returns true |
|
|
194
|
+
| `TIMEOUT` | `WorkflowScheduler` | Suspending nodes whose `wake_at` fires |
|
|
141
195
|
|
|
142
196
|
Effectful node outcomes are whatever the handler returns (e.g. `LOW`, `OK`, `CONFIRMED`).
|
|
143
197
|
|
|
144
198
|
## DB Schema (managed by `createMigrate`)
|
|
145
199
|
|
|
146
|
-
| Table
|
|
147
|
-
|
|
148
|
-
| `__workflow_instances`
|
|
149
|
-
| `__workflow_inbox`
|
|
150
|
-
| `__workflow_history`
|
|
151
|
-
| `__workflow_migrations` | Active migration version
|
|
200
|
+
| Table | Purpose | Indexes |
|
|
201
|
+
| ----------------------- | ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
202
|
+
| `__workflow_instances` | Live instance rows (incl. the `seq` fence) | `(tenant_id, execution_state, wake_at)` partial, `(tenant_id, execution_state, correlation_token)` partial, `(tenant_id, updated_at)` partial WHERE `execution_state = 'pending'` |
|
|
203
|
+
| `__workflow_inbox` | Append-only intake of external signals | `(tenant_id, correlation_token)` partial WHERE unprocessed |
|
|
204
|
+
| `__workflow_history` | Append-only audit log | `(instance_id, at)` |
|
|
205
|
+
| `__workflow_migrations` | Active migration version | — |
|
|
152
206
|
|
|
153
207
|
Steve and Cron own their own tables (`__job*`, `__cron*`); managed by their respective libraries.
|
|
154
208
|
|
|
209
|
+
**Schema versions** (registered in `src/migrations/index.ts`, applied via `createMigrate(pool).up("latest")`):
|
|
210
|
+
|
|
211
|
+
| Version | Change |
|
|
212
|
+
| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
213
|
+
| `1.0.0` | Creates the three framework tables + indexes. |
|
|
214
|
+
| `1.1.0` | Renames `project_id` -> `tenant_id` on all three tables (in place, data preserved, idempotent, reversible). |
|
|
215
|
+
| `1.2.0` | Adds `seq integer NOT NULL DEFAULT 0` (the fence) to `__workflow_instances` + the partial index for the stale-`pending` scan. Additive and non-volatile — on PG >= 11 a metadata-only change, existing rows start at `0`. |
|
|
216
|
+
|
|
217
|
+
Migration files are an append-only ledger: `1_0_0.ts` still creates `project_id` and is deliberately left untouched — `1_1_0.ts` performs the rename, so fresh installs and upgrades converge on the same schema.
|
|
218
|
+
|
|
219
|
+
`driver.ts` carries two transitional shims for jobs enqueued by an older version and still sitting in the queue at upgrade time. Both are removable together, once no pre-2.1 job can be queued anywhere:
|
|
220
|
+
|
|
221
|
+
- `payloadTenantId()` falls back to a legacy `project_id` key (pre-1.1.0 payloads).
|
|
222
|
+
- `advanceKind()` infers `kind` from the payload shape when it is absent (pre-fence payloads). Such a payload also has no `expected_seq`, i.e. it is **unfenced**: it runs without the fence check, and without the per-`kind` preconditions — guessing the kind must not swallow the job.
|
|
223
|
+
|
|
155
224
|
## Runtime Ownership
|
|
156
225
|
|
|
157
226
|
The framework **does not start or stop steve.Jobs or cron.Cron.** The consumer constructs both, passes them in, and runs their lifecycle:
|
|
158
227
|
|
|
159
228
|
```ts
|
|
160
|
-
const jobs = new Jobs({ db: pool });
|
|
161
|
-
const cron = new Cron({ db: pool });
|
|
229
|
+
const jobs = new Jobs({ db: pool, autoCleanup: true }); // consumer-owned
|
|
230
|
+
const cron = new Cron({ db: pool }); // consumer-owned
|
|
162
231
|
|
|
163
232
|
const wf = new Workflow({ db: pool, jobs, ... }); // setHandler on jobs
|
|
164
233
|
const scheduler = new WorkflowScheduler({ cron, workflow: wf }); // register tick on cron
|
|
@@ -174,17 +243,28 @@ await cron.stop(); // consumer
|
|
|
174
243
|
await jobs.stop(); // consumer
|
|
175
244
|
```
|
|
176
245
|
|
|
177
|
-
|
|
246
|
+
`autoCleanup` is load-bearing, not a tuning knob: steve's reaper is what turns a crashed worker into an `expired` job, which is what the framework's re-dispatch reacts to. Default reap threshold is 5 minutes of `running`; the check interval defaults to 60s.
|
|
247
|
+
|
|
248
|
+
The consumer's other steve job types and cron jobs coexist on the same `Jobs` / `Cron` instances — handlers are keyed by `type` (steve) / `name` (cron), and the workflow's are namespaced (`workflow.advance`, `workflow.effect.<name>`, `workflow.scheduler.<tenantId>`, `workflow.correlator.<tenantId>`) so collisions are unlikely.
|
|
249
|
+
|
|
250
|
+
**Both runtimes noop what they cannot handle**, which makes a second unregistered runtime a silent data-loss bug rather than a loud one:
|
|
251
|
+
|
|
252
|
+
- A `Jobs` instance polling a `type` it doesn't know falls back to a noop handler that marks the job completed ([steve/jobs.ts](file:///Users/mm/projects/@marianmeres/steve/src/steve/jobs.ts)). What that costs depends on which job was eaten: a `timeout`/`signal` poke is re-poked next tick and a `start` advance is recovered by the stale-`pending` scan, but a noop'd **effect job or effect-completion advance strands the instance in `running` with nothing left to move it** — the reaper never sees it, because the job is `completed`.
|
|
253
|
+
- A `Cron` instance claims due rows **globally, regardless of tenant**, and runs a noop with a `warn` for a `name` it has no handler for ([cron/cron.ts](file:///Users/mm/projects/@marianmeres/cron/src/cron/cron.ts)) — then advances `next_run_at` as if the tick had run. A process that starts a `Cron` without calling `scheduler.register()` / `correlator.register()` therefore _steals_ ticks from the process that did.
|
|
254
|
+
|
|
255
|
+
Rule: one `Jobs` and one `Cron` per process, and every process that _starts_ them on these tables constructs the `Workflow` and registers the ticks. A producer-only process (`create`, `appendInbox`) still needs a `Jobs` to construct the `Workflow` — it just never calls `jobs.start()` / `cron.start()`.
|
|
178
256
|
|
|
179
257
|
## Critical Conventions
|
|
180
258
|
|
|
181
|
-
1. **Definitions are pure data.** Handlers and matchers are referenced by string name. Never store function pointers in a definition. Litmus test: `JSON.stringify(def)` must lose no meaning.
|
|
259
|
+
1. **Definitions are pure data.** Handlers and matchers are referenced by string name. Never store function pointers in a definition. Litmus test: `JSON.stringify(def)` must lose no meaning. Threading a handler's output into the context needs no `action` hook either — the handler returns `HandlerResult.context`, a shallow patch merged before the outcome transition.
|
|
182
260
|
2. **Version-pin instances.** Every instance carries `(definition_id, definition_version)`. Never edit an in-use definition in place — bump the version. Old versions stay registered until the last instance using them completes.
|
|
183
|
-
3. **Handlers must be idempotent.**
|
|
261
|
+
3. **Handlers must be idempotent.** The `seq` fence protects the instance row, not the outside world: a worker that finished the side effect and died before steve recorded the attempt gets retried, and the retry is _not_ stale (the instance never left that `seq`). Side effects must be safe to repeat.
|
|
184
262
|
4. **No fsm `onEnter`/`onExit` hooks for side effects.** Driver re-constructs FSMs via `FSM.fromSnapshot()`, which skips `onEnter`. Side effects belong in effectful handlers — they're the only way to get durable, retryable, observable execution.
|
|
185
263
|
5. **Cursor + execution_state are separate columns.** Don't fold one into the other.
|
|
186
264
|
6. **Outcome labels are fsm events.** The driver calls `fsm.transition(outcome, data)`. If fsm rejects (unknown event in current state), the instance is marked `failed` and history records `transition_rejected`.
|
|
187
265
|
7. **Correlation token is the index, matcher is the gate.** Token alone is not sufficient — the matcher does the semantic check.
|
|
266
|
+
8. **Only the advance writes instance rows.** Anything else that wants an instance moved enqueues an advance and lets it re-check under the lock. A new settle-point write must pass `{ bumpSeq: true }` to `updateInstance` and a new job payload must carry the fence, or duplicate delivery stops being safe.
|
|
267
|
+
9. **Fenced-out work is not an event.** Duplicate/stale jobs and pokes get a `clog.debug`, never a history row — with re-poking ticks they are routine, and history is an event log, not a trace.
|
|
188
268
|
|
|
189
269
|
## Before Making Changes
|
|
190
270
|
|
|
@@ -192,49 +272,82 @@ The consumer's other steve job types and cron jobs coexist on the same `Jobs` /
|
|
|
192
272
|
- [ ] Run `deno task test` (set `TEST_PG_*` env first; see `.env` example)
|
|
193
273
|
- [ ] If adding a new history event type, add it to `HISTORY_EVENT` in [src/types.ts](./src/types.ts) and emit it consistently
|
|
194
274
|
- [ ] If changing schema, write a new migration in [src/migrations/](./src/migrations/) — never edit `1_0_0.ts` in place
|
|
275
|
+
- [ ] Changed the public API? Check [example/](./example) still compiles (`deno task check`) and still tells the truth — it is the one consumer in the repo
|
|
276
|
+
- [ ] Touched `example/src/`? Run `deno task example:build` — `example/dist/bundle.js` is committed
|
|
195
277
|
|
|
196
278
|
## Dependency Versions
|
|
197
279
|
|
|
198
|
-
| Package
|
|
199
|
-
|
|
200
|
-
| `@marianmeres/fsm`
|
|
201
|
-
| `@marianmeres/steve`
|
|
202
|
-
| `@marianmeres/cron`
|
|
203
|
-
| `@marianmeres/migrate` | `^1.
|
|
204
|
-
| `@marianmeres/clog`
|
|
205
|
-
| `pg`
|
|
280
|
+
| Package | Min | Why |
|
|
281
|
+
| ---------------------- | --------- | ------------------------------------------------------------- |
|
|
282
|
+
| `@marianmeres/fsm` | `^3.1.0` | Requires `state.meta` and `FSM.fromSnapshot` (added in 3.1.0) |
|
|
283
|
+
| `@marianmeres/steve` | `^3.0.0` | Job queue (advance + effect dispatch) |
|
|
284
|
+
| `@marianmeres/cron` | `^3.2.0` | Scheduler + correlator ticks |
|
|
285
|
+
| `@marianmeres/migrate` | `^1.3.1` | Schema versioning |
|
|
286
|
+
| `@marianmeres/clog` | `^3.21.0` | Logging |
|
|
287
|
+
| `pg` | `^8.23.0` | PostgreSQL client |
|
|
206
288
|
|
|
207
289
|
## Failure Modes & Recovery
|
|
208
290
|
|
|
209
|
-
| Failure
|
|
210
|
-
|
|
211
|
-
| Handler throws
|
|
212
|
-
|
|
|
213
|
-
|
|
|
214
|
-
| Pure
|
|
215
|
-
|
|
|
216
|
-
|
|
|
217
|
-
|
|
|
291
|
+
| Failure | Behavior |
|
|
292
|
+
| ----------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
293
|
+
| Handler throws | Steve retries the job `effectMaxAttempts` times. On terminal failure (`status: failed`), `Workflow`'s `onDone` listener marks the instance `failed` + appends `effect_failed` history. |
|
|
294
|
+
| Advance job throws | Steve retries the job `advanceMaxAttempts` times (default 10 — ~17min of exponential backoff, so a brief DB outage costs nothing). When they are exhausted, `onDone` marks the instance `failed` with the last attempt's `error_message`. |
|
|
295
|
+
| fsm rejects outcome | Driver appends `transition_rejected` history, marks instance `failed`. |
|
|
296
|
+
| Pure node with no matching guard | Driver appends `transition_rejected` history, marks instance `failed`. |
|
|
297
|
+
| Pure-state cycle (>64 hops) | Driver appends `failed` history with "exceeded MAX_PURE_HOPS" reason. |
|
|
298
|
+
| Definition missing on advance | Driver marks instance `failed`. (Pin-and-freeze: keep old versions registered until drained.) |
|
|
299
|
+
| Crashed worker mid-job (advance or effect) | Steve's reaper marks the job `expired` — **terminal, never retried by steve**. `Workflow`'s `onDone` re-queues the identical payload instead, `redispatchLimit` dispatches in total (default 3), then fails the instance. Safe because the fence no-ops a re-dispatch whose original did commit. **Requires the reaper to run at all**: `new Jobs({ db, autoCleanup: true })`, or a periodic `jobs.cleanup()`. Without it the job sits in `running` forever and the instance never moves. |
|
|
300
|
+
| Lost advance after effect (the side effect happened, `jobs.create('workflow.advance')` did not) | A throwing enqueue → steve retries the effect job; a dead process → the job expires and is re-dispatched. Either way the fence lets the retry through — the instance never left that `seq` — so the handler runs again and re-enqueues the advance. Handlers must be idempotent. |
|
|
301
|
+
| Crash between `create()`'s commit and steve's job insert | The instance sits in `pending` with no job. The scheduler's second scan re-pokes it after `stalePendingSec` (default 300s; `0` disables). |
|
|
302
|
+
| Duplicate advance / effect job | Fenced out by `seq`: debug-logged no-op, no history row. Duplicates are routine (every tick re-pokes what still looks due), so they are not events. |
|
|
303
|
+
| Second `Jobs` / `Cron` without the workflow handlers | Silent loss — see [Runtime Ownership](#runtime-ownership). |
|
|
304
|
+
|
|
305
|
+
## The Example App
|
|
306
|
+
|
|
307
|
+
[example/](./example) is consumer code, not a test: it imports `@marianmeres/workflow` by
|
|
308
|
+
package name (mapped to `./src/mod.ts` in `deno.json`) and wires the runtime exactly the
|
|
309
|
+
way the README does. It is also the fastest way to see the two orthogonal states, the
|
|
310
|
+
timer and the poke model behave, so keep it working.
|
|
311
|
+
|
|
312
|
+
Three things in it deviate from production defaults, all commented where they are set:
|
|
313
|
+
|
|
314
|
+
| Deviation | Why |
|
|
315
|
+
| --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|
|
316
|
+
| `scheduler.tickOnce()` / `correlator.tickOnce()` on a 500 ms interval | a cron expression cannot go below a minute. The real cron ticks stay registered; the UI can switch the fast loop off. |
|
|
317
|
+
| `effectMaxAttempts: 1` | so a throwing handler becomes a `failed` instance immediately, with a Retry button. The default 3 would heal it invisibly. |
|
|
318
|
+
| `stalePendingSec: 30` (default 300) | so the stranded-`pending` re-poke is witnessable. |
|
|
319
|
+
|
|
320
|
+
It also does two things the package deliberately does not offer, and both are marked as
|
|
321
|
+
such: **listing instances and inbox rows with raw SQL** (the package exposes `find(id)` /
|
|
322
|
+
`getHistory(id)` — anything that lists is an admin surface, i.e. your query), and nothing
|
|
323
|
+
else. If you add a listing helper to `src/`, the example is where it should be used first.
|
|
324
|
+
|
|
325
|
+
Touching `example/src/` means re-running `deno task example:build`; the bundle is
|
|
326
|
+
committed so the app runs with no build step.
|
|
218
327
|
|
|
219
328
|
## Testing
|
|
220
329
|
|
|
221
|
-
|
|
330
|
+
All tests run against a real PostgreSQL and skip gracefully when `TEST_PG_DATABASE` / `TEST_PG_USER` are unset — the driver is SQL-bound, so mocking it would test nothing. [stock_replenishment.test.ts](./tests/stock_replenishment.test.ts) covers happy/timeout/failure paths, [durability.test.ts](./tests/durability.test.ts) the fence and the recovery paths, [correlator.test.ts](./tests/correlator.test.ts) the signal semantics.
|
|
222
331
|
|
|
223
|
-
Per-tick helpers `WorkflowScheduler.tickOnce()` and `WorkflowInboxCorrelator.tickOnce()` are exposed for tests so they don't have to wait for cron expressions.
|
|
332
|
+
Per-tick helpers `WorkflowScheduler.tickOnce()` and `WorkflowInboxCorrelator.tickOnce()` are exposed for tests so they don't have to wait for cron expressions. Durability behavior is generally testable without a running `Jobs`: enqueue nothing, call `runAdvance` / `runEffect` directly with a hand-built payload, and assert on the row + history.
|
|
224
333
|
|
|
225
334
|
## Glossary
|
|
226
335
|
|
|
227
|
-
| Term
|
|
228
|
-
|
|
229
|
-
| Definition
|
|
230
|
-
| Instance
|
|
231
|
-
| Driver
|
|
232
|
-
| Cursor
|
|
233
|
-
| Execution state
|
|
234
|
-
| Outcome
|
|
235
|
-
| Effect port
|
|
236
|
-
| Inbox
|
|
237
|
-
| Correlation token | Unique token that maps an external event back to a waiting instance
|
|
238
|
-
| Scheduler
|
|
239
|
-
| Correlator
|
|
240
|
-
|
|
|
336
|
+
| Term | Meaning |
|
|
337
|
+
| ----------------- | -------------------------------------------------------------------------------------------------------------------- |
|
|
338
|
+
| Definition | Static, versioned, declarative shape of a workflow. Pure data |
|
|
339
|
+
| Instance | Single running occurrence of a definition; one DB row |
|
|
340
|
+
| Driver | The engine (`workflow.advance` steve handler) that advances instances |
|
|
341
|
+
| Cursor | The instance's current node id |
|
|
342
|
+
| Execution state | The instance's lifecycle status |
|
|
343
|
+
| Outcome | The label a handler returns (or `MATCHED`/`TIMEOUT`/`ENTER`); fed to fsm as the event name |
|
|
344
|
+
| Effect port | A typed interface to the outside world (email, AI, HTTP) — userland concern |
|
|
345
|
+
| Inbox | Append-only `__workflow_inbox` intake of external signals |
|
|
346
|
+
| Correlation token | Unique token that maps an external event back to a waiting instance |
|
|
347
|
+
| Scheduler | Cron-driven loop that wakes time-suspended instances |
|
|
348
|
+
| Correlator | Cron-driven loop that matches inbox signals to waiting instances |
|
|
349
|
+
| Fence | The `seq` column. Bumped by every settle-point write; carried by every job, so a stale one is recognized and dropped |
|
|
350
|
+
| Poke | A `workflow.advance` job enqueued by something that will not write the row itself. The advance re-checks and decides |
|
|
351
|
+
| Settle point | The end of an advance: terminal / effectful / suspending. Where the instance row is written and `seq` bumped |
|
|
352
|
+
| Defer | Correlator verdict: leave the inbox row unprocessed and re-examine it next tick |
|
|
353
|
+
| Pin-and-freeze | Versioning strategy: every instance runs on its birth-version forever |
|