@marianmeres/workflow 2.1.0 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +146 -89
- package/API.md +27 -13
- package/README.md +91 -13
- package/dist/correlator.d.ts +1 -1
- package/dist/correlator.js +25 -21
- package/dist/definition.d.ts +1 -1
- package/dist/driver.d.ts +17 -10
- package/dist/driver.js +60 -64
- package/dist/migrations/1_0_0.d.ts +5 -4
- package/dist/migrations/1_0_0.js +6 -20
- package/dist/migrations/1_1_0.d.ts +3 -2
- package/dist/migrations/1_1_0.js +5 -23
- package/dist/migrations/1_2_0.d.ts +3 -2
- package/dist/migrations/1_2_0.js +4 -18
- package/dist/migrations/index.d.ts +15 -5
- package/dist/migrations/index.js +16 -38
- package/dist/mod.d.ts +19 -13
- package/dist/mod.js +14 -8
- package/dist/persistence/history.d.ts +10 -10
- package/dist/persistence/history.js +23 -22
- package/dist/persistence/inbox.d.ts +12 -13
- package/dist/persistence/inbox.js +40 -31
- package/dist/persistence/instances.d.ts +25 -20
- package/dist/persistence/instances.js +99 -103
- package/dist/registry.d.ts +1 -1
- package/dist/scheduler.d.ts +5 -4
- package/dist/scheduler.js +1 -1
- package/dist/workflow.d.ts +33 -11
- package/dist/workflow.js +44 -33
- package/package.json +89 -39
- package/dist/persistence/tx.d.ts +0 -6
- package/dist/persistence/tx.js +0 -25
package/README.md
CHANGED
|
@@ -1,26 +1,23 @@
|
|
|
1
1
|
# @marianmeres/workflow
|
|
2
2
|
|
|
3
3
|
[](https://www.npmjs.com/package/@marianmeres/workflow)
|
|
4
|
-
[](https://jsr.io/@marianmeres/workflow)
|
|
5
4
|
[](LICENSE)
|
|
6
5
|
|
|
7
6
|
A durable workflow / orchestration framework for long-lived state machines. Built on PostgreSQL.
|
|
8
7
|
|
|
9
|
-
Instances persist across process restarts, sleep for days waiting for a signal or a timer, and resume from disk via [`@marianmeres/fsm`](https://
|
|
8
|
+
Instances persist across process restarts, sleep for days waiting for a signal or a timer, and resume from disk via [`@marianmeres/fsm`](https://www.npmjs.com/package/@marianmeres/fsm)'s `fromSnapshot`. Effects run as [`@marianmeres/steve`](https://www.npmjs.com/package/@marianmeres/steve) jobs (transactional outbox). The scheduler and inbox correlator are driven by [`@marianmeres/cron`](https://www.npmjs.com/package/@marianmeres/cron). Every statement runs through [`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit), which also runs the schema migrations ([`@marianmeres/migrate`](https://www.npmjs.com/package/@marianmeres/migrate)).
|
|
10
9
|
|
|
11
10
|
**Ships no domain code.** Userland brings the definitions, handlers, and matchers. The framework provides the dispatch core and the durability story.
|
|
12
11
|
|
|
13
12
|
## Install
|
|
14
13
|
|
|
15
14
|
```sh
|
|
16
|
-
|
|
15
|
+
npm install @marianmeres/workflow pg
|
|
17
16
|
```
|
|
18
17
|
|
|
19
|
-
|
|
20
|
-
npm install @marianmeres/workflow
|
|
21
|
-
```
|
|
18
|
+
Node 24.20+ or 26.6+. The package does not depend on `pg`: install the one you build your pool with. `db` takes that `pg.Pool`, a connected `pg.Client`, or a sqlkit `Driver` — see [The database](#the-database-a-pool-a-client-or-a-driver).
|
|
22
19
|
|
|
23
|
-
|
|
20
|
+
Upgrading from 2.x? See [Upgrading to 3.0](#upgrading-to-30).
|
|
24
21
|
|
|
25
22
|
## Mental Model
|
|
26
23
|
|
|
@@ -97,7 +94,8 @@ const stockReplenishment: WorkflowDefinition = {
|
|
|
97
94
|
},
|
|
98
95
|
};
|
|
99
96
|
|
|
100
|
-
// 2. Connect + migrate
|
|
97
|
+
// 2. Connect + migrate. Everything below takes this one pool: sharing it is
|
|
98
|
+
// what puts a job insert in the same transaction as the instance write.
|
|
101
99
|
const pool = new pg.Pool({/* ... */});
|
|
102
100
|
await createMigrate(pool).up("latest");
|
|
103
101
|
|
|
@@ -200,7 +198,7 @@ A single steve job type — **`workflow.advance`** — encapsulates one step of
|
|
|
200
198
|
|
|
201
199
|
The **fence** is the `seq` column. Every settle-point write bumps it, and every advance/effect job carries the value it was issued against. A job whose value no longer matches belongs to a step the instance has already left — a duplicate, or a zombie from a worker that hung — and is dropped with a debug log rather than applied to a row that moved on.
|
|
202
200
|
|
|
203
|
-
The **scheduler** runs as a registered cron job (default `* * * * *`). Each tick is read-only: it selects `waiting` rows whose `wake_at` has passed and pokes one `TIMEOUT` advance per row (the advance re-checks due-ness under the lock and clears `wake_at`). A second scan pokes instances stuck in `pending` for longer than `stalePendingSec` —
|
|
201
|
+
The **scheduler** runs as a registered cron job (default `* * * * *`). Each tick is read-only: it selects `waiting` rows whose `wake_at` has passed and pokes one `TIMEOUT` advance per row (the advance re-checks due-ness under the lock and clears `wake_at`). A second scan pokes instances stuck in `pending` for longer than `stalePendingSec` — rows whose `start` advance was lost (see [Persistence Guarantees](#persistence-guarantees--idempotency)).
|
|
204
202
|
|
|
205
203
|
The **correlator** runs as a registered cron job (default `* * * * *`). Each tick claims a batch of unprocessed `__workflow_inbox` rows, finds the live instance owning the `correlation_token`, runs the user's `matcher` for the current state, and pokes a `MATCHED` advance carrying the _inbox row id_. The advance re-reads that row under a lock and marks it processed in the same transaction as the transition, so a signal is never consumed without being delivered.
|
|
206
204
|
|
|
@@ -210,11 +208,12 @@ Each instance row stores `definition_version`. The framework looks up the defini
|
|
|
210
208
|
|
|
211
209
|
## Persistence Guarantees & Idempotency
|
|
212
210
|
|
|
213
|
-
- **The instance write is one PG transaction.** Cursor, execution_state, wake_at, `seq` and history are written together, and a signal delivery marks its inbox row processed in that same transaction.
|
|
214
|
-
- **
|
|
211
|
+
- **The instance write is one PG transaction.** Cursor, execution_state, wake_at, `seq` and history are written together, and a signal delivery marks its inbox row processed in that same transaction.
|
|
212
|
+
- **The follow-up job is in that transaction too, when the queue shares the database handle.** Give `Jobs` the same pool, client or driver as the `Workflow` and a job enqueued inside a transaction is part of it: `create()` writes the instance and its first advance together, and an advance writes the row and the effect job it dispatches together, or neither. That is the transactional outbox. A `Jobs` built on a connection of its own still works, as it did before 3.0: its inserts stand alone, so a job can be claimed before the transaction that enqueued it has committed, or exist for one that rolled back. The fence covers both.
|
|
213
|
+
- **Job delivery is at-least-once either way, and fenced.** Steve retries a failed attempt, an expired job is re-queued, every tick re-pokes what still looks due. A duplicate advance or effect job is recognized by `seq` and dropped. The fence is what makes the extra deliveries harmless, not their absence. `seq` arrives with schema 1.2.0 — run `createMigrate(pool).up("latest")` when upgrading.
|
|
215
214
|
- **Handlers must be idempotent.** The fence protects the _instance row_, not the outside world. A handler can still run twice — a worker that completed the side effect and then died before steve recorded the attempt gets retried. Use upserts, idempotency keys, or check the existing state.
|
|
216
215
|
- **Run `Jobs` with `autoCleanup: true`** (or call `jobs.cleanup()` periodically). Steve's reaper is the only thing that marks a job whose worker died as `expired`; without it that job stays `running` forever and its instance never moves again. With it, the framework re-queues the identical payload up to `redispatchLimit` times before giving up and failing the instance — the fence makes the re-queue a no-op if the dead worker's transaction did commit.
|
|
217
|
-
- **A `pending` instance is re-poked** after `stalePendingSec` (default 300s)
|
|
216
|
+
- **A `pending` instance is re-poked** after `stalePendingSec` (default 300s). That covers a `start` advance that was lost: `retry()` enqueues after its commit, so a crash in between leaves the row with no job, and a job claimed by a `Jobs` that has no handler for it is consumed without effect.
|
|
218
217
|
|
|
219
218
|
## Signals: early, late, deferred
|
|
220
219
|
|
|
@@ -265,6 +264,62 @@ Pass `tenantId` to `Workflow`, `WorkflowScheduler`, and `WorkflowInboxCorrelator
|
|
|
265
264
|
|
|
266
265
|
The supported shape today is one runtime per database, serving whichever tenants it has definitions for. Tenant scoping is for keeping data apart, not for partitioning workers.
|
|
267
266
|
|
|
267
|
+
## The database: a pool, a client or a driver
|
|
268
|
+
|
|
269
|
+
`db` — on `Workflow`, `createMigrate()` and `getHistory()` — takes one of three things:
|
|
270
|
+
|
|
271
|
+
- a node-postgres `pg.Pool`, the recommendation;
|
|
272
|
+
- a connected `pg.Client`: one session, so every statement and every transaction takes its turn on it;
|
|
273
|
+
- a [sqlkit](https://www.npmjs.com/package/@marianmeres/sqlkit) `Driver` of the `pg` dialect.
|
|
274
|
+
|
|
275
|
+
Anything else is a `TypeError` at construction. The package never closes what it is given.
|
|
276
|
+
|
|
277
|
+
```typescript
|
|
278
|
+
import { createPgDriver } from "@marianmeres/sqlkit/pg";
|
|
279
|
+
|
|
280
|
+
const db = createPgDriver(new pg.Pool({ connectionString: process.env.DATABASE_URL }));
|
|
281
|
+
await createMigrate(db).up("latest");
|
|
282
|
+
|
|
283
|
+
const jobs = new Jobs({ db, autoCleanup: true });
|
|
284
|
+
const cron = new Cron({ db });
|
|
285
|
+
const wf = new Workflow({ db, jobs, definitions, handlers });
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
A pool or a client is turned into sqlkit's driver for it (`toPgDriver`), and sqlkit keeps one driver per pool and per client. So handing the same pool to `Jobs`, `Cron` and `Workflow` gives all three the same driver, exactly as building it yourself does (which needs `@marianmeres/sqlkit` among your own dependencies). What that means:
|
|
289
|
+
|
|
290
|
+
- **It carries transactions in the calling context.** A statement issued inside a `db.transaction()` callback is part of that transaction, whichever library issues it. That is how a steve job enqueued by an advance commits with the instance row.
|
|
291
|
+
- **Your own transactions count too.** `wf.create()`, `wf.appendInbox()`, `wf.cancel()` and `wf.retry()` called inside your `db.transaction()` are part of it: the instance exists only if you commit. Create an order and the workflow that fulfils it in one transaction.
|
|
292
|
+
- **A matcher runs inside the correlator tick's transaction.** What it reads through the shared driver, it reads there. Each matcher call is wrapped in a savepoint, so a statement of its own that fails defers that one signal and leaves the rest of the tick intact.
|
|
293
|
+
- **Make steve's first call of a process outside any transaction.** Steve checks its schema once per `Jobs` instance, on first use; inside a transaction that then rolls back, on a database with no steve tables yet, the schema is rolled back with it and is not checked again. `jobs.start()` is such a call. A producer-only process never starts its queue, so its first steve call would be the one inside `wf.create()`'s transaction: give it one before that, such as `await jobs.healthPreview()` at boot.
|
|
294
|
+
- **`dbRetry` on the `Jobs` cannot help inside a transaction**: after a failed statement Postgres has aborted it. The advance fails as a whole and steve retries the job.
|
|
295
|
+
- **One copy of sqlkit.** The one-driver-per-pool rule holds among callers of the same copy of the package. If your dependency tree ends up with two (a version of steve or cron on another sqlkit minor), build the driver once and pass that, as above.
|
|
296
|
+
|
|
297
|
+
A `Jobs` on a different pool is a different driver: none of the above applies to it, and its inserts are statements of their own. See [Persistence Guarantees](#persistence-guarantees--idempotency).
|
|
298
|
+
|
|
299
|
+
## Upgrading to 3.0
|
|
300
|
+
|
|
301
|
+
3.0 runs every statement through [`@marianmeres/sqlkit`](https://www.npmjs.com/package/@marianmeres/sqlkit) and moves to `@marianmeres/steve` 4 and `@marianmeres/cron` 4, which do the same. The database schema is unchanged: there is no migration to run. Code that hands one `pg.Pool` to `Jobs`, `Cron` and `Workflow` and calls the documented methods compiles as it is.
|
|
302
|
+
|
|
303
|
+
### Changed
|
|
304
|
+
|
|
305
|
+
- **Requires `@marianmeres/steve` 4 and `@marianmeres/cron` 4**, and Node 24.20+ or 26.6+ (sqlkit's floor). Both are source-compatible with their 3.x; their own upgrade notes cover what changes underneath.
|
|
306
|
+
- **A job enqueued inside a transaction is part of it**, when the `Jobs` was given the same pool, client or driver as the `Workflow` — which the documented wiring always did. In 2.x steve inserted on a connection of its own, so the job stood apart from the instance write. Nothing to change; two windows the fence used to cover no longer open (a job claimed before its transaction committed, a job left behind by one that rolled back). See [The database](#the-database-a-pool-a-client-or-a-driver).
|
|
307
|
+
- **`Workflow.db` is a sqlkit `Driver`**, not the `pg.Pool` you passed. Code that called `wf.db.connect()` or `wf.db.end()` keeps its own reference to the pool instead; `wf.db.query(text, params)` still works.
|
|
308
|
+
- **`enqueueAdvance(payload)` and `enqueueEffect(handler, payload)` lost their first argument**, a database handle that was never used. They are internal, but public: a stub of `JobEnqueuer` in your tests drops the parameter.
|
|
309
|
+
- **Database errors are sqlkit's.** A failed statement rejects with a `SqlError` (`ConnectionError`, `DeadlockError`, `UniqueViolationError`, …). `code` and `message` are node-postgres's; the original error is `cause`.
|
|
310
|
+
- **An invalid `db` throws at construction**, a `TypeError` from `new Workflow()` or `createMigrate()`, not at the first query.
|
|
311
|
+
- **Statements run on a checked-out connection** (`pool.connect()`), never through `pool.query()`. A wrapper around `pool.query` that logs or measures does not see them.
|
|
312
|
+
- **The pool's `error` event has a listener.** sqlkit's driver listens to its pool (or client), so an idle connection the server dropped is a process warning; with no listener of your own it used to end the process.
|
|
313
|
+
- **`createMigrate()` returns a `@marianmeres/migrate` 2 instance** over sqlkit's migrate wiring: a run holds an advisory lock, so replicas booting together wait for each other, and each version is a transaction of its own, committed with its version mark. `up("latest")` and `down(version)` are called as before. An existing `__workflow_migrations` table is read and continued as it is.
|
|
314
|
+
- **The package no longer depends on `pg`.** It never imported it at runtime. Install the `pg` you build your pool with, as you already do.
|
|
315
|
+
- **No new versions on JSR.** 2.2.0 is the last one there: sqlkit, steve 4 and cron 4 are published to npm only.
|
|
316
|
+
|
|
317
|
+
### New
|
|
318
|
+
|
|
319
|
+
- **`db` accepts a connected `pg.Client` or a sqlkit `Driver`** beside a pool, on `Workflow`, `createMigrate()` and `getHistory()`.
|
|
320
|
+
- **Workflow calls join your transaction** when you share the driver: an instance, or an inbox signal, written together with your own rows.
|
|
321
|
+
- **A matcher's failed statement no longer costs more than its own signal**: each matcher call runs in a savepoint of the tick's transaction.
|
|
322
|
+
|
|
268
323
|
## Breaking changes in 2.0
|
|
269
324
|
|
|
270
325
|
- **`project_id` renamed to `tenant_id` throughout**, aligning with the
|
|
@@ -304,8 +359,9 @@ run out, hand it a signal its matcher declines, break the effect that books it,
|
|
|
304
359
|
retry it — and watch the graph, the history and the inbox as each of those lands.
|
|
305
360
|
|
|
306
361
|
```sh
|
|
362
|
+
pnpm install
|
|
307
363
|
createdb example_workflow # or point EXAMPLE_PG_* at a database you already have
|
|
308
|
-
|
|
364
|
+
pnpm run example # → http://127.0.0.1:21340
|
|
309
365
|
```
|
|
310
366
|
|
|
311
367
|
The highlighted node is the cursor; the badge on it is the execution state. The countdown
|
|
@@ -321,6 +377,28 @@ See [`example/`](./example) — the workflow itself (definition, handlers, match
|
|
|
321
377
|
|
|
322
378
|
See [API.md](API.md) for the full reference. See [AGENTS.md](AGENTS.md) for the agent-oriented guide.
|
|
323
379
|
|
|
380
|
+
## Development
|
|
381
|
+
|
|
382
|
+
Node LTS (`.node-version`), pnpm, Biome, `node:test`. The source is erasable-only
|
|
383
|
+
TypeScript that Node runs directly; `dist/` is built only for publishing to npm. The
|
|
384
|
+
tests need a PostgreSQL database: copy
|
|
385
|
+
`.env.example` to `.env` and set the `TEST_PG_*` credentials (without them every
|
|
386
|
+
database test is skipped).
|
|
387
|
+
|
|
388
|
+
| Command | Does |
|
|
389
|
+
| ----------------------------------- | -------------------------------------------------------------------- |
|
|
390
|
+
| `pnpm install` | installs the deps and the dev tooling |
|
|
391
|
+
| `pnpm test` / `pnpm run test:watch` | `node --test` over `tests/**/*.test.ts`, one file at a time |
|
|
392
|
+
| `pnpm run typecheck` | `tsc --noEmit` over `src`, `tests`, `scripts` and the example server |
|
|
393
|
+
| `pnpm run lint` / `pnpm run format` | `biome check .` read-only / with `--write` |
|
|
394
|
+
| `pnpm run ci` | typecheck + lint + test, what CI runs |
|
|
395
|
+
| `pnpm run build` | `tsc -p tsconfig.build.json` → `dist/` |
|
|
396
|
+
| `pnpm run example` | the interactive example (see above) |
|
|
397
|
+
| `pnpm run example:build` | type-checks and bundles `example/src` → `example/dist/bundle.js` |
|
|
398
|
+
| `pnpm run example:dev` | the same, rebuilding on change |
|
|
399
|
+
| `pnpm run example:theme [name]` | regenerates `example/theme.css` from a design-tokens theme |
|
|
400
|
+
| `pnpm run rp` / `pnpm run rpm` | patch / minor release, then publish to npm |
|
|
401
|
+
|
|
324
402
|
## License
|
|
325
403
|
|
|
326
404
|
[MIT](LICENSE)
|
package/dist/correlator.d.ts
CHANGED
package/dist/correlator.js
CHANGED
|
@@ -3,7 +3,6 @@ import { clog } from "./log.js";
|
|
|
3
3
|
import { appendHistory } from "./persistence/history.js";
|
|
4
4
|
import { claimUnprocessed, markProcessed } from "./persistence/inbox.js";
|
|
5
5
|
import { findByCorrelation, findTerminalByCorrelation } from "./persistence/instances.js";
|
|
6
|
-
import { withTransaction } from "./persistence/tx.js";
|
|
7
6
|
import { EXECUTION_STATE, HISTORY_EVENT, } from "./types.js";
|
|
8
7
|
const DEFAULT_TICK_EXPRESSION = "* * * * *";
|
|
9
8
|
/**
|
|
@@ -79,26 +78,26 @@ export class WorkflowInboxCorrelator {
|
|
|
79
78
|
}
|
|
80
79
|
async #tick() {
|
|
81
80
|
const pokes = [];
|
|
82
|
-
const resolved = await
|
|
83
|
-
const rows = await claimUnprocessed(
|
|
81
|
+
const resolved = await this.#workflow.db.transaction(async (tx) => {
|
|
82
|
+
const rows = await claimUnprocessed(tx, this.tenantId, this.#tickBatchSize);
|
|
84
83
|
// One poke per instance per tick: a second signal for the same
|
|
85
84
|
// instance would be fenced out by the first anyway, and deferring it
|
|
86
85
|
// keeps it for the instance's next wait point.
|
|
87
86
|
const poked = new Set();
|
|
88
87
|
let n = 0;
|
|
89
88
|
for (const row of rows) {
|
|
90
|
-
if (await this.#processOne(
|
|
89
|
+
if (await this.#processOne(tx, row, poked, pokes))
|
|
91
90
|
n++;
|
|
92
91
|
}
|
|
93
92
|
return n;
|
|
94
93
|
});
|
|
95
|
-
// Deliberately after the commit
|
|
96
|
-
// lock, which this tick holds until then,
|
|
97
|
-
// connection of its own
|
|
98
|
-
//
|
|
99
|
-
// unprocessed, so the next tick pokes again.
|
|
94
|
+
// Deliberately after the commit, so outside the tick's transaction: the
|
|
95
|
+
// advance blocks on the inbox row's lock, which this tick holds until then,
|
|
96
|
+
// and a `Jobs` on a connection of its own would need a second connection
|
|
97
|
+
// while the claim holds the first. Nothing is lost if the process dies
|
|
98
|
+
// here; the row is still unprocessed, so the next tick pokes again.
|
|
100
99
|
for (const payload of pokes) {
|
|
101
|
-
await this.#workflow.enqueueAdvance(
|
|
100
|
+
await this.#workflow.enqueueAdvance(payload);
|
|
102
101
|
}
|
|
103
102
|
if (resolved > 0)
|
|
104
103
|
clog.debug?.(`correlator: resolved ${resolved} signals`);
|
|
@@ -113,16 +112,16 @@ export class WorkflowInboxCorrelator {
|
|
|
113
112
|
* True if the row was resolved, false if it was deferred. A delivery is
|
|
114
113
|
* resolved by appending to `pokes` — the tick enqueues those after it commits.
|
|
115
114
|
*/
|
|
116
|
-
async #processOne(
|
|
117
|
-
const instance = await findByCorrelation(
|
|
115
|
+
async #processOne(tx, row, poked, pokes) {
|
|
116
|
+
const instance = await findByCorrelation(tx, this.tenantId, row.correlation_token);
|
|
118
117
|
// A token no live instance owns is an upstream bug, not something to wait
|
|
119
118
|
// for. (A completed or cancelled instance has its token cleared, so it
|
|
120
119
|
// lands here too — only failed ones are still findable.)
|
|
121
120
|
if (!instance) {
|
|
122
121
|
clog.warn?.(`correlator: no live instance for token "${row.correlation_token}" (inbox row ${row.id})`);
|
|
123
|
-
const terminal = await findTerminalByCorrelation(
|
|
122
|
+
const terminal = await findTerminalByCorrelation(tx, this.tenantId, row.correlation_token);
|
|
124
123
|
if (terminal) {
|
|
125
|
-
await appendHistory(
|
|
124
|
+
await appendHistory(tx, {
|
|
126
125
|
tenant_id: this.tenantId,
|
|
127
126
|
instance_id: terminal.id,
|
|
128
127
|
event_type: HISTORY_EVENT.SIGNAL_REJECTED,
|
|
@@ -134,7 +133,7 @@ export class WorkflowInboxCorrelator {
|
|
|
134
133
|
},
|
|
135
134
|
});
|
|
136
135
|
}
|
|
137
|
-
await markProcessed(
|
|
136
|
+
await markProcessed(tx, row.id);
|
|
138
137
|
return true;
|
|
139
138
|
}
|
|
140
139
|
if (instance.execution_state !== EXECUTION_STATE.WAITING) {
|
|
@@ -144,8 +143,7 @@ export class WorkflowInboxCorrelator {
|
|
|
144
143
|
return this.#defer(row, `instance ${instance.id} already poked this tick`);
|
|
145
144
|
}
|
|
146
145
|
const def = this.#workflow.registry.getDefinition(instance.definition_id, instance.definition_version);
|
|
147
|
-
const stateCfg = def?.fsm
|
|
148
|
-
.states[instance.cursor];
|
|
146
|
+
const stateCfg = def?.fsm.states[instance.cursor];
|
|
149
147
|
// A node with no MATCHED (or wildcard) edge is not a delivery target:
|
|
150
148
|
// poking it would make the fsm reject the transition and fail the
|
|
151
149
|
// instance. Typically a timer-only delay node on the way to the real wait.
|
|
@@ -159,12 +157,18 @@ export class WorkflowInboxCorrelator {
|
|
|
159
157
|
let matched;
|
|
160
158
|
try {
|
|
161
159
|
const matcherFn = this.#workflow.registry.requireMatcher(meta.matcher);
|
|
162
|
-
|
|
160
|
+
// The matcher is userland code running inside this tick's
|
|
161
|
+
// transaction: a statement it issues through a driver it shares
|
|
162
|
+
// with the workflow is part of that transaction, and on Postgres
|
|
163
|
+
// one failed statement aborts all of it. The savepoint confines
|
|
164
|
+
// that to the matcher — its failure is undone alone, and the rows
|
|
165
|
+
// this tick already resolved stay resolved.
|
|
166
|
+
matched = await tx.savepoint(async () => matcherFn({
|
|
163
167
|
instanceId: instance.id,
|
|
164
168
|
tenantId: this.tenantId,
|
|
165
169
|
context: instance.context,
|
|
166
170
|
signal: row,
|
|
167
|
-
});
|
|
171
|
+
}));
|
|
168
172
|
}
|
|
169
173
|
catch (e) {
|
|
170
174
|
// Deferring keeps a transient failure (a lookup, an HTTP call) from
|
|
@@ -174,14 +178,14 @@ export class WorkflowInboxCorrelator {
|
|
|
174
178
|
return this.#defer(row, "matcher threw");
|
|
175
179
|
}
|
|
176
180
|
if (!matched) {
|
|
177
|
-
await appendHistory(
|
|
181
|
+
await appendHistory(tx, {
|
|
178
182
|
tenant_id: this.tenantId,
|
|
179
183
|
instance_id: instance.id,
|
|
180
184
|
event_type: HISTORY_EVENT.SIGNAL_REJECTED,
|
|
181
185
|
from_node: instance.cursor,
|
|
182
186
|
data: { source: row.source, inbox_id: row.id },
|
|
183
187
|
});
|
|
184
|
-
await markProcessed(
|
|
188
|
+
await markProcessed(tx, row.id);
|
|
185
189
|
return true;
|
|
186
190
|
}
|
|
187
191
|
}
|
package/dist/definition.d.ts
CHANGED
package/dist/driver.d.ts
CHANGED
|
@@ -1,18 +1,25 @@
|
|
|
1
|
-
import type
|
|
2
|
-
import type { WorkflowRegistry } from "./registry.
|
|
3
|
-
import { type AdvanceJobPayload, type EffectJobPayload, JOB_TYPE_ADVANCE, PURE_ENTER_EVENT, SIGNAL_MATCHED_EVENT } from "./types.
|
|
1
|
+
import type { Driver } from "@marianmeres/sqlkit";
|
|
2
|
+
import type { WorkflowRegistry } from "./registry.ts";
|
|
3
|
+
import { type AdvanceJobPayload, type EffectJobPayload, JOB_TYPE_ADVANCE, PURE_ENTER_EVENT, SIGNAL_MATCHED_EVENT } from "./types.ts";
|
|
4
4
|
/**
|
|
5
5
|
* Function the driver uses to enqueue follow-up steve jobs. Supplied by the
|
|
6
6
|
* `Workflow` class — the driver itself doesn't reach back into steve.Jobs.
|
|
7
|
+
*
|
|
8
|
+
* Neither method takes a database handle: where the insert lands is decided by
|
|
9
|
+
* where the call is made. Called inside a `db.transaction()` callback, on a
|
|
10
|
+
* queue that shares the workflow's driver, the job is part of that transaction;
|
|
11
|
+
* anywhere else, and always on a queue with a database connection of its own,
|
|
12
|
+
* it is a statement of its own.
|
|
7
13
|
*/
|
|
8
14
|
export interface JobEnqueuer {
|
|
9
|
-
enqueueAdvance(
|
|
10
|
-
enqueueEffect(
|
|
15
|
+
enqueueAdvance(payload: AdvanceJobPayload): Promise<void>;
|
|
16
|
+
enqueueEffect(handler: string, payload: EffectJobPayload): Promise<void>;
|
|
11
17
|
}
|
|
12
18
|
/**
|
|
13
19
|
* The `workflow.advance` job body.
|
|
14
20
|
*
|
|
15
|
-
* Wrapped in a single PG transaction
|
|
21
|
+
* Wrapped in a single PG transaction (`db.transaction()`; a call made from
|
|
22
|
+
* inside a caller's own transaction on the same driver joins that one):
|
|
16
23
|
* 1. Lock the instance row.
|
|
17
24
|
* 2. If in a terminal execution_state, no-op.
|
|
18
25
|
* 2b. If the payload's fence (`expected_seq`) does not match the locked row, or
|
|
@@ -28,7 +35,7 @@ export interface JobEnqueuer {
|
|
|
28
35
|
* - effectful → enqueue an effect job + mark running, return.
|
|
29
36
|
* - suspending → mark waiting, set wake_at/correlation_token, return.
|
|
30
37
|
*/
|
|
31
|
-
export declare function runAdvance(
|
|
38
|
+
export declare function runAdvance(db: Driver, registry: WorkflowRegistry, enqueuer: JobEnqueuer, payload: AdvanceJobPayload): Promise<void>;
|
|
32
39
|
/**
|
|
33
40
|
* Called by the Workflow layer when steve gives up on an advance job — every
|
|
34
41
|
* attempt threw, or the job expired once too often to keep re-dispatching it.
|
|
@@ -38,7 +45,7 @@ export declare function runAdvance(pool: pg.Pool, registry: WorkflowRegistry, en
|
|
|
38
45
|
* Fenced like every other write: an advance that lost the race to a later one
|
|
39
46
|
* must not fail the instance that later one moved on.
|
|
40
47
|
*/
|
|
41
|
-
export declare function failAdvanceJob(
|
|
48
|
+
export declare function failAdvanceJob(db: Driver, payload: AdvanceJobPayload, reason: string, jobUid?: string): Promise<void>;
|
|
42
49
|
/**
|
|
43
50
|
* The `workflow.effect.<handlerName>` job body.
|
|
44
51
|
*
|
|
@@ -58,7 +65,7 @@ export declare function failAdvanceJob(pool: pg.Pool, payload: AdvanceJobPayload
|
|
|
58
65
|
* mid-execution, and even successful handlers can run twice if completion-write
|
|
59
66
|
* fails after the advance has been enqueued.
|
|
60
67
|
*/
|
|
61
|
-
export declare function runEffect(
|
|
68
|
+
export declare function runEffect(db: Driver, registry: WorkflowRegistry, enqueuer: JobEnqueuer, payload: EffectJobPayload, signal?: AbortSignal): Promise<{
|
|
62
69
|
outcome: string;
|
|
63
70
|
data?: Record<string, unknown>;
|
|
64
71
|
} | {
|
|
@@ -70,7 +77,7 @@ export declare function runEffect(pool: pg.Pool, registry: WorkflowRegistry, enq
|
|
|
70
77
|
* unless the job is fenced out, i.e. it belongs to a step the instance has
|
|
71
78
|
* already left, in which case a dead job must not kill a healthy instance.
|
|
72
79
|
*/
|
|
73
|
-
export declare function failEffectJob(
|
|
80
|
+
export declare function failEffectJob(db: Driver, payload: EffectJobPayload, reason: string): Promise<void>;
|
|
74
81
|
/** Composite steve job-type string for an effect handler. */
|
|
75
82
|
export declare function effectJobType(handlerName: string): string;
|
|
76
83
|
export { JOB_TYPE_ADVANCE, PURE_ENTER_EVENT, SIGNAL_MATCHED_EVENT };
|