fyflow-orchestration 0.0.0-stage → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/AGENTS.md +3757 -0
  2. package/LICENSE +21 -0
  3. package/README.md +319 -2
  4. package/dist/browser/chunks/chunk-YVOP3DIN.js +4456 -0
  5. package/dist/browser/index.js +435 -0
  6. package/dist/browser/jsonata/index.js +75 -0
  7. package/dist/browser/scheduler/index.js +666 -0
  8. package/dist/node/chunks/chunk-YVOP3DIN.js +4456 -0
  9. package/dist/node/index.js +435 -0
  10. package/dist/node/jsonata/index.js +75 -0
  11. package/dist/node/scheduler/index.js +666 -0
  12. package/dist/types/index.d.ts +17 -0
  13. package/dist/types/jsonata/index.d.ts +2 -0
  14. package/dist/types/scheduler/index.d.ts +1 -0
  15. package/dist/types/src/definition/assist.d.ts +221 -0
  16. package/dist/types/src/definition/regions.d.ts +359 -0
  17. package/dist/types/src/definition/registry.d.ts +185 -0
  18. package/dist/types/src/definition/schema.d.ts +91 -0
  19. package/dist/types/src/definition/types.d.ts +748 -0
  20. package/dist/types/src/definition/validate.d.ts +299 -0
  21. package/dist/types/src/definition/wiring.d.ts +186 -0
  22. package/dist/types/src/definition/witness.d.ts +39 -0
  23. package/dist/types/src/driver/driver.d.ts +1364 -0
  24. package/dist/types/src/driver/events.d.ts +87 -0
  25. package/dist/types/src/driver/ids.d.ts +49 -0
  26. package/dist/types/src/driver/listener.d.ts +41 -0
  27. package/dist/types/src/driver/log.d.ts +59 -0
  28. package/dist/types/src/driver/policy.d.ts +65 -0
  29. package/dist/types/src/driver/routing.d.ts +85 -0
  30. package/dist/types/src/fold/policies.d.ts +169 -0
  31. package/dist/types/src/fold/run-state.d.ts +307 -0
  32. package/dist/types/src/jsonata/index.d.ts +40 -0
  33. package/dist/types/src/scheduler/bootstrap.d.ts +97 -0
  34. package/dist/types/src/scheduler/envelope-worker.d.ts +199 -0
  35. package/dist/types/src/scheduler/index.d.ts +13 -0
  36. package/dist/types/src/scheduler/runtime.d.ts +321 -0
  37. package/package.json +137 -4
package/AGENTS.md ADDED
@@ -0,0 +1,3757 @@
1
+ # fyflow-orchestration — Reference
2
+
3
+ Flow orchestration over [`fyflow-scheduler`](https://github.com/fyflow/fyflow-scheduler): sequence,
4
+ decision routing, fan-out and join.
5
+
6
+ > **Every ```ts snippet in this file is extracted and typechecked against the real `index.ts` by
7
+ > `npm run gate`, and most of them are executed.** `scripts/check-docs.mjs` pulls them out, rewrites
8
+ > the `fyflow-orchestration` import to this repository's own, and compiles them together. **A stale
9
+ > snippet is a failing gate rather than a lie** — which is why they are worth reading.
10
+ >
11
+ > `run` snippets execute and must assert something; `check` snippets are typechecked only, because
12
+ > they need a live scheduler or exist to show a type. **The prose between them is not checked**, so
13
+ > where a claim matters it is pushed into a snippet's assertion rather than left in a sentence.
14
+
15
+ ---
16
+
17
+ ## 1. What this library is, and what it is not
18
+
19
+ `fyflow-scheduler` runs tasks. **It has no dependency graph** — no `dependsOn`, no `parents`, no
20
+ ordering — and its own reference lists them under *"Things that do NOT exist"*.
21
+
22
+ **This library decides which task runs next, and why.** It owns four things the scheduler does not
23
+ have: **sequence**, **decision routing**, **fan-out** and **join**.
24
+
25
+ **And it tells you whether the flow you wrote FITS the workers you named, before you run it.** Where
26
+ a worker declares what it expects and what it returns — it need not, and `unchecked` is the verdict
27
+ when it does not — `checkWiring` gives every arc of a flow one of five verdicts from those
28
+ declarations, `tryTransform` runs a candidate expression on a producer's own
29
+ declared example and holds the result to the consumer's schema, and `signatureOf` turns the same
30
+ schemas into the TypeScript a worker's author writes against. **Section 15 is a session that does all
31
+ three on a real miswire and then runs the flow it repaired** — it is executed by `npm run gate` like
32
+ everything else here.
33
+
34
+ | | belongs to |
35
+ |---|---|
36
+ | worker pools, threads, retry, resource limits, task execution | `fyflow-scheduler` |
37
+ | what runs after what, and on what condition | **here** |
38
+
39
+ **This package depends on the scheduler, and says so where your installer reads it.** Since
40
+ [ADR 0048](doc/decisions/0048-the-scheduler-is-a-declared-dependency-reached-by-one-bare-specifier.md)
41
+ `fyflow-scheduler` is a `peerDependencies` entry at **exactly** the version section 12 pins — npm 7
42
+ and later install it for you — and the JSR package maps it to `@fyflow/scheduler` at the same
43
+ version. The driver is written against that version's event shapes, and the range is exact because
44
+ there is no upstream changelog to read a wider one from. **You still pass your own scheduler in**:
45
+ `SchedulerLike` is a two-method structural interface, a real `FyflowScheduler` satisfies it as it is,
46
+ and the one thing the library builds itself is a `FyflowTask` when you omit `createTask`. *(This
47
+ paragraph read "does not depend on the scheduler … install whichever version you like" from W9 to
48
+ W57, while section 12 said to pin it — two sentences a consumer reads first, contradicting each
49
+ other twenty-one sections apart.)*
50
+
51
+ ```ts check
52
+ import type { SchedulerLike } from 'fyflow-orchestration';
53
+
54
+ // The entire surface this library needs from a scheduler. The listener takes `object` — what an
55
+ // `EventTarget` hands a listener — and the driver reads `detail` off it (section 12, Q71).
56
+ const shape: SchedulerLike = {
57
+ addTask(_task: unknown) { return undefined; },
58
+ addEventListener(_type: string, _listener: (e: object) => void) {},
59
+ };
60
+ void shape;
61
+ ```
62
+
63
+ ---
64
+
65
+ ## 2. Getting started — a flow and its runtime from two documents
66
+
67
+ **Two documents, and one file of your own.** A REGISTRY says what exists — a worker module, and a
68
+ node type that runs on it. A FLOW says what happens — nodes naming those types, and the edges between
69
+ them. `createRuntime` builds every pool, group and scheduler the registry names and hands back
70
+ **`{ driver, done, header }`** per run — the driver already running, and `done` a promise that resolves with
71
+ the run's terminal state ([ADR 0058](doc/decisions/0058-runtime-start-returns-a-completion-handle-the-runtime-keeps.md)). **The only code you write is the worker**:
72
+ `examples/getting-started/worker.ts`, a class with a `run` method and nothing else — no base class,
73
+ no import. Everything below is executed by the gate on every `npm run gate`, and the same snippet is
74
+ in `README.md`, held byte-equal to this one by `check-docs`'s `X8`
75
+ ([W69](doc/work-items/w69-a-getting-started-example-a-consumer-can-copy.md)).
76
+
77
+ ```ts run
78
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
79
+ import { type FlowDefinition, type RegistryDocument } from 'fyflow-orchestration';
80
+
81
+ // THE REGISTRY — what exists. One worker module, and one node type that runs on it. This is data:
82
+ // in your program it is a JSON file you import. The worker url is relative, and `base` below says
83
+ // to what; the file itself is `examples/getting-started/worker.ts` — a class with a `run` method.
84
+ const registry: RegistryDocument = {
85
+ workers: {
86
+ greeter: {
87
+ url: '../../examples/getting-started/worker.ts', // in your program: './worker.ts'
88
+ settingsSchema: { type: 'object', required: ['say'], properties: { say: { type: 'string' } } },
89
+ },
90
+ },
91
+ nodeTypes: { Greeter: { worker: 'greeter', inline: true } },
92
+ };
93
+
94
+ // THE FLOW — what happens. Two nodes of that type, each with its own `settings`, then an end. A flow
95
+ // names registry KEYS and nothing else — no file, no url, no class — so it is portable across
96
+ // runtimes that agree on the keys. The end node's `outcome` is a LABEL of yours that the driver does
97
+ // not interpret; the run's own outcome is the fold's — `state.outcome` below.
98
+ const flow: FlowDefinition = {
99
+ flowId: 'hello', version: 1, entry: 'greet',
100
+ nodes: [
101
+ { id: 'greet', kind: 'task', type: 'Greeter', settings: { say: 'world' }, edges: [{ to: 'shout' }] },
102
+ { id: 'shout', kind: 'task', type: 'Greeter', settings: { say: '!' }, edges: [{ to: 'done' }] },
103
+ { id: 'done', kind: 'end', outcome: 'completed' },
104
+ ],
105
+ };
106
+
107
+ // ONE call builds the pool, the scheduler and everything between, and refuses every defect in the
108
+ // REGISTRY at construction, by name. It never sees the flow: a defect THERE is refused where the
109
+ // flow arrives, at `start`, which is what builds the `Driver` — so a `settings` value the worker's
110
+ // schema rejects names its node from the call three lines below, not from this one. `base` is what
111
+ // relative worker urls resolve against — `import.meta.url` when the registry sits beside this file.
112
+ const runtime = createRuntime(registry, {}, { base: import.meta.url });
113
+
114
+ // `start` hands back `{ driver, done }` — the driver already running, and `done` a promise that
115
+ // resolves with the run's terminal `RunState` (ADR 0058). **It never rejects**: `failed`, `stuck`
116
+ // and `stopped` are terminal OUTCOMES, so a run that goes wrong arrives as `state.outcome` and not
117
+ // as a `catch`. This snippet wants only the end, so it names only `done`. *A run with a person in
118
+ // it does not end by itself — section 7 is where that is taught, and section 8 is what `done` waits
119
+ // for.*
120
+ const { done } = runtime.start(flow, { text: 'hello' });
121
+ const state = await done;
122
+ await runtime.shutdown();
123
+
124
+ // What the run did, read off its state: `greet` turned `hello` into `hello world`, and `shout` was
125
+ // handed THAT as its input — the previous node's result is the next node's `input`.
126
+ const shout = Object.values(state.nodes).find((n) => n.nodeId === 'shout');
127
+ console.log(state.outcome); // 'completed'
128
+ console.log(shout?.result); // { text: 'hello world !' }
129
+
130
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
131
+ if (JSON.stringify(shout?.result) !== '{"text":"hello world !"}') throw new Error(`shout returned ${JSON.stringify(shout?.result)}`);
132
+ ```
133
+
134
+ **What to read next.** Everything this snippet leaves out is `examples/flows/triage.flow.json` with
135
+ `examples/run-flow.ts` — a custom resource group, a node-type CLASS with its own derivation, a node
136
+ on a NAMED pool instance, a decision edge, a fan-out and its join, and a declared transform — which is
137
+ [P18](doc/plans/p18-a-flow-and-its-runtime-from-documents-alone.md)'s exit artifact and runs on real
138
+ workers in both L6 runtimes. Section 3 is the definition grammar; section 12 is the runtime in full,
139
+ including the driver with **no scheduler at all**; section 13 is node types; **section 15 is what
140
+ their schemas buy you before a run**.
141
+
142
+ ---
143
+
144
+ ## 3. The flow definition
145
+
146
+ **Six** node kinds. A definition is JSON: no functions, so it can be stored, versioned and sent over
147
+ a wire.
148
+
149
+ > **`gate` is the sixth, and since [W93](doc/work-items/w93-what-a-gate-does-in-a-run.md) a `Driver` RUNS
150
+ > one** — `spec/flow-definition.md` v0.23-v0.24,
151
+ > [ADR 0066](doc/decisions/0066-a-gate-is-a-decision-with-memory-that-names-a-registered-type-and-wires-every-route-exactly-once.md)
152
+ > and [ADR 0067](doc/decisions/0067-a-gate-runs-in-the-driver-its-memory-is-the-drivers-and-its-decision-is-recorded.md),
153
+ > [W92](doc/work-items/w92-what-a-gate-is.md). A gate is a decision WITH MEMORY: it routes the branch that
154
+ > reaches it by what the run's other branches have already shown it — `distinct`, the one built-in type,
155
+ > sends the FIRST branch per key down its default route `first` and every later one down `duplicate` — and it
156
+ > hands on exactly what reached it. The driver evaluates it where a branch reaches it — the gate's memory is
157
+ > ONE per gate per RUN, the driver's, rebuilt by replay on a resume — records each decision as a `gated` fold
158
+ > input (`RunState.gated`), and stops the run with an `UndeclaredRoute` if a type returns a route it does not
159
+ > declare (`FD-R5`). *(Until W93 `new Driver` refused a flow holding one, at construction.)* Every route its
160
+ > type declares must be wired by exactly one edge — a drop is an edge to an `end` — and an edge with no
161
+ > `route` carries the type's DEFAULT route, never every route.
162
+
163
+ > **`map` is the fifth, and since [W33](doc/work-items/w33-a-map-at-run-time-arity.md) a `Driver`
164
+ > RUNS one.** It is a valid definition since `spec/flow-definition.md` v0.8 —
165
+ > [ADR 0036](doc/decisions/0036-a-map-is-a-second-node-kind-and-a-branch-path-inside-one-is-a-property-of-the-dispatch.md),
166
+ > answering [Q12](doc/questions/q12.md): a `fanout` declares N branches known at authoring time, a
167
+ > `map` declares **one** and a run opens N instances of it, N coming from the value the map node
168
+ > receives. `over` is a **key name** on that value and must resolve to an array, or the run stops
169
+ > with an `UnresolvedMapArity` naming the node and the key; an empty array is a lawful map of zero
170
+ > items and makes the run `stuck`, because the join can never fire.
171
+ >
172
+ > **What each instance receives, and what the join hands on.** Each instance is dispatched with
173
+ > **its own item** — `over`'s array at that instance's index, as its own copy, so no two instances
174
+ > share an object — and the join carries **every item's result keyed by that item's branch path**. A
175
+ > map of three items at the top level dispatches three tasks and its successor receives
176
+ > `{ "0": …, "1": …, "2": … }`; **a map nested in a fan-out branch keys its collection with the whole
177
+ > path**, so a map inside branch 1 gives `{ "1.0": …, "1.1": … }`. *(This said "keyed by its item
178
+ > index", which is the depth-1 case stated as if it were general — W34's conformity reader.)*
179
+ > [ADR 0037](doc/decisions/0037-what-a-branch-receives-is-a-property-of-the-fan-out-and-what-a-join-carries-is-a-property-of-the-region.md),
180
+ > and see *What a join carries downstream* in section 5 for what happens when an item's branch does
181
+ > not arrive.
182
+ >
183
+ > *(This paragraph read **"every instance receives the whole value the map node received rather than
184
+ > its own item, and a satisfied join still carries `null`"** until W34, which is where
185
+ > [Q11](doc/questions/q11.md) and [Q36](doc/questions/q36.md) were deliberately held so the rule
186
+ > would be written from cases rather than from a mechanism.)*
187
+ >
188
+ > **This paragraph is the minimum that is not false and is not the feature write-up.** It read
189
+ > *"NOTHING DISPATCHES ONE YET"* until W33, which is [Q157](doc/questions/q157.md)'s second
190
+ > occurrence: no [P10](doc/plans/p10-is-fan-out-one-feature-or-two.md) item owns this file, W35 is
191
+ > the natural owner, and W33 took it as a declared late exception to keep the reference true.
192
+
193
+ ```ts run
194
+ import { validateDefinition, type FlowDefinition } from 'fyflow-orchestration';
195
+
196
+ const shapes: FlowDefinition = {
197
+ flowId: 'shapes',
198
+ version: 1,
199
+ entry: 'score',
200
+ nodes: [
201
+ // A task runs a worker. `optional` decides what a failure MEANS — section 6.
202
+ // `retryPolicy` and `workerGroups` say what the node wants of its OWN task — below.
203
+ { id: 'score', kind: 'task', type: 'ScoreWorker', optional: false,
204
+ retryPolicy: { maxRetries: 1, backoffMs: 250 }, // exactly the scheduler's shape
205
+ workerGroups: ['gpu'], // must be a group the registry declares
206
+ edges: [
207
+ { on: ['failed'], to: 'quarantine' }, // eligible only when the task failed
208
+ { when: { predicate: 'isLowConfidence' }, to: 'review' }, // guarded by a registered predicate
209
+ { to: 'publish' }, // the default: no `on`, no `when`
210
+ ] },
211
+ { id: 'review', kind: 'task', type: 'ReviewWorker', edges: [{ to: 'publish' }] },
212
+ { id: 'publish', kind: 'task', type: 'PublishWorker', edges: [{ to: 'done' }] },
213
+ { id: 'quarantine', kind: 'end', outcome: 'quarantined' },
214
+ { id: 'done', kind: 'end', outcome: 'completed' },
215
+ ],
216
+ };
217
+
218
+ // Not merely well-typed — VALID, and checked here rather than asserted. Note the
219
+ // `on: ['failed']` edge is declared FIRST: "the default comes last" is scoped per
220
+ // terminal state, not to the array. See section 4.
221
+ const bad = validateDefinition(shapes, {
222
+ predicates: ['isLowConfidence'],
223
+ workers: { flow: { url: 'worker.js' } }, nodeTypes: { ScoreWorker: { worker: 'flow' }, ReviewWorker: { worker: 'flow' }, PublishWorker: { worker: 'flow' } },
224
+ groups: ['gpu'], // the third set — pass Object.keys(scheduler.groups)
225
+ });
226
+ if (bad.length) throw new Error(`section 3's own example is invalid: ${bad.map((f) => f.rule).join()}`);
227
+
228
+ // And `groups` is what makes `workerGroups` checkable at all: leave it out and the SAME
229
+ // definition is refused, because a registry that lists no groups declares none. The scheduler
230
+ // would have run the task with the unknown group silently ignored — no throw, no
231
+ // `resource.blocked` — so construction is the only place a typo in a group id is caught.
232
+ const noGroups = validateDefinition(shapes, {
233
+ predicates: ['isLowConfidence'], workers: { flow: { url: 'worker.js' } }, nodeTypes: { ScoreWorker: { worker: 'flow' }, ReviewWorker: { worker: 'flow' }, PublishWorker: { worker: 'flow' } },
234
+ });
235
+ if (!noGroups.some((f) => f.rule === 'FD-V31' && f.at === 'score')) {
236
+ throw new Error(`expected FD-V31 at score, got ${noGroups.map((f) => `${f.rule}@${f.at}`).join() || 'nothing'}`);
237
+ }
238
+ ```
239
+
240
+ | kind | | refused if |
241
+ |---|---|---|
242
+ | `task` | runs a worker — of its node **`type`**, REQUIRED since v0.19, on the type's default pool or a named **`pool`** instance of it, with per-node **`settings`** and optionally a **`transform`** applied to its input before the worker sees it (section 13). Has `edges`, and optionally **`needs`**, **`retryPolicy`** and **`workerGroups`** — what it says about its own task, below | it has no `edges`, or **no edge of it is eligible for `done`** (`FD-V7`); it has no `type` (`FD-V23`) — `workerType` is retired and is an undeclared key like any other (`FD-V17`); its `retryPolicy` is not exactly `{ maxRetries, backoffMs }` — an integer >= 0 and a finite number >= 0, no other key (`FD-V30`); its `workerGroups` is not a non-empty array of distinct ids the registry's `groups` declares (`FD-V31`); its `type` is not a registered node type whose worker the registry declares (`FD-V32`); its `settings` is not a plain object the type's worker `settingsSchema` accepts (`FD-V33`); its `transform` names a language the registry does not (`FD-V35`), an expression that language's compiler refuses (`FD-V36`), or is not `{ lang, expr }` (`FD-V37`); its `pool` is not a registered instance of its type (`FD-V38`) |
243
+ | `fanout` | `branches: string[]` — the node each branch starts at. Dispatches nothing itself | fewer than two branches, a repeated entry, any `edges` at all, or a branch entry that is not a `task` (`FD-V9`) |
244
+ | `map` | `over: '<key>'` and `branch: '<node id>'` — **one** branch, opened once per item at run time. Dispatches nothing itself; the RUN opens N instances of its one branch. Optionally **`needs`**, which is what `over` is then a key on | any `edges` at all, a `branch` that is not a `task`, or an `over` outside `^[A-Za-z0-9_-]{1,64}$` (`FD-V25`); a missing `over` or `branch` (`FD-V23`) |
245
+ | `join` | `of: '<fanout id>'`. Fires when its policy says so | it does not have **exactly one** edge, or that edge carries `when` or `on` (`FD-V7`) |
246
+ | `end` | terminal. `outcome` is **your label**, never interpreted | — |
247
+ | `gate` | `type: '<gate type>'`, optional `settings`, and `edges` of `{ to, route? }` — routes the branch that reaches it and hands on what reached it. `settings.key` is a dotted key path over that value (`series.id`; no index). **Not run by any driver yet** | its `type` is not a registered gate type (`FD-V39`); its `settings` are not a plain object its type's schema accepts, or its `key` is not a key path (`FD-V40`); a route is unwired, wired twice, or not its type's, or a gate edge carries `when`, `on` or `needs` (`FD-V41`) — and `route` on any other node's edge is `FD-V41` too |
248
+
249
+ **A `task` node must have an edge that can be taken when it succeeds.** A node whose only edge is
250
+ `{ when: { predicate: 'p' }, to: 'x' }` is refused at construction — it is the mistake the format most invites, and
251
+ `FD-V7` is what catches it.
252
+
253
+ **`end.outcome` is a string you choose.** The library never reads it. What the *run* is called is a
254
+ separate, closed vocabulary — section 8.
255
+
256
+ ### `retryPolicy` and `workerGroups` — what a node says about its own task
257
+
258
+ `optional` was the one scheduler task field a node could set; since
259
+ `spec/flow-definition.md` v0.16 ([W58](doc/work-items/w58-what-a-node-says-about-its-own-task.md))
260
+ there are two more, and both are **passed through to the scheduler task exactly as `optional` is**.
261
+ They are declared keys rather than an opaque bag, so a typo is refused at construction instead of
262
+ reaching the scheduler, which would drop it in silence.
263
+
264
+ | key | | |
265
+ |---|---|---|
266
+ | `retryPolicy` | `{ maxRetries, backoffMs }` — **exactly** the scheduler's shape, an integer >= 0 and a finite number >= 0, no other key (`FD-V30`) | the scheduler retries a failed attempt **inside the same task id**, and the outcome record's `attempts` says how many retries were used. `{}` and a string count are refused because the scheduler would accept them and mean something else |
267
+ | `workerGroups` | a non-empty array of distinct group ids, every one in the registry's **`groups`** set (`FD-V31`) | the task acquires these **in addition to** its pool's `groups`. Pass `groups: Object.keys(scheduler.groups)` to the `Driver`, as you pass `nodeTypes` — **omitting it means the registry declares NO groups**, and any node naming one is refused, because a group id the scheduler does not know is silently ignored at run time |
268
+
269
+ **A driver may carry a `defaultRetryPolicy`**, applied to every task node that declares none — held
270
+ to `FD-V30` at construction like a node's own — and
271
+ **the node wins, whole** — a node with its own policy is dispatched with exactly that object, and the
272
+ default is consulted only when the node declares none. There is no `defaultWorkerGroups`: a pool's own `groups` already applies to
273
+ every task it runs, and the scheduler unions the two. A node that declares neither key is dispatched
274
+ with the exact object it was before, so nothing you have already written changes.
275
+
276
+ **Flow-level retry — re-dispatching a NODE under a new attempt, possibly routing elsewhere — is
277
+ still not built** (section 11). This is the scheduler's retry of one task, and it is the whole of
278
+ what `attempts` counts.
279
+
280
+ ### `needs` — reaching an output the previous node did not produce
281
+
282
+ By default **a node receives the previous node's result**, deep-copied. That is still true of every
283
+ node that says nothing, and it is what the whole of this section above describes.
284
+
285
+ **`needs` is how a node asks for something else** —
286
+ [ADR 0040](doc/decisions/0040-what-a-node-can-reach-is-the-runs-own-outputs-and-a-node-id-names-them-relative-to-a-branch-path.md),
287
+ answering [Q11](doc/questions/q11.md). It is an optional object on a `task` or a `map`, mapping the
288
+ keys of the payload you want to **declared references**. *(Since v0.12 an **edge** may carry one too,
289
+ and it says what that edge's PREDICATE sees rather than what a worker receives — section 4.)* A node that declares one receives **exactly**
290
+ that object and nothing else — there is no wrapper and no reserved key.
291
+
292
+ | a reference is | and you get |
293
+ |---|---|
294
+ | `"."` | what the node would have received anyway — the previous node's result, and for a `map` instance **its own item** |
295
+ | `"someNode"` | that node's result, **on your own branch** |
296
+ | `["someNode"]` | that node's results as an **array**, in ascending branch-path order. Legal only when you are OUTSIDE a region that node is in — which is what makes it more than one |
297
+ | **a `?` on either of those** — `"someNode?"`, `["someNode?"]` | the same thing, except that **nothing to give leaves the key out** (or gives `[]` for the array form) instead of stopping the run. This is how you say *if it ran* |
298
+
299
+ **Which of the two aimed forms is legal is decided by the graph and refused at construction**
300
+ (`FD-V28`), so you are told which to write rather than discovering the shape at run time. The message
301
+ names the form to use instead.
302
+
303
+ **And the bare form is a CLAIM that the node ran.** If a run can reach you without it having run — a
304
+ decision routes around it, or it sits in a region whose branch may be lost — that is `FD-V29` at
305
+ construction, and the fix is the `?`. **It is not dominance**: a `fanout`'s branches are all entered,
306
+ so a branch entry is certain from after the join even though it is not on EVERY path there. See
307
+ [ADR 0042](doc/decisions/0042-a-reference-says-whether-it-is-conditional-and-the-bare-form-means-the-target-certainly-ran.md).
308
+
309
+ ```ts run
310
+ import { Driver, validateDefinition, type FlowDefinition } from 'fyflow-orchestration';
311
+
312
+ // *Score each item against the run's config* — a map instance seeing its own item AND
313
+ // the context around it. Before ADR 0040 `config` was reachable from a `score` worker
314
+ // by no route at all.
315
+ const withContext: FlowDefinition = {
316
+ flowId: 'score-against-config', version: 1, entry: 'ingest',
317
+ nodes: [
318
+ { id: 'ingest', kind: 'task', type: 'IngestWorker', edges: [{ to: 'split' }] },
319
+ { id: 'split', kind: 'map', over: 'items', branch: 'score' },
320
+ { id: 'score', kind: 'task', type: 'ScoreWorker',
321
+ needs: { item: '.', run: 'ingest' }, // its own item, and ingest's whole result
322
+ edges: [{ to: 'collect' }] },
323
+ { id: 'collect', kind: 'join', of: 'split', edges: [{ to: 'publish' }] },
324
+ { id: 'publish', kind: 'task', type: 'PublishWorker', edges: [{ to: 'done' }] },
325
+ { id: 'done', kind: 'end', outcome: 'completed' },
326
+ ],
327
+ };
328
+
329
+ // *Score, then fan out over the scores* — a map whose `over` names a key composed from a
330
+ // whole region's output. No branch numbers appear anywhere, so inserting a branch cannot
331
+ // silently re-point it.
332
+ const overRegion: FlowDefinition = {
333
+ flowId: 'fan-out-over-the-scores', version: 1, entry: 'ingest',
334
+ nodes: [
335
+ { id: 'ingest', kind: 'task', type: 'IngestWorker', edges: [{ to: 'split' }] },
336
+ { id: 'split', kind: 'map', over: 'items', branch: 'score' },
337
+ { id: 'score', kind: 'task', type: 'ScoreWorker', edges: [{ to: 'collect' }] },
338
+ { id: 'collect', kind: 'join', of: 'split', edges: [{ to: 'rank' }] },
339
+ { id: 'rank', kind: 'map', over: 'scores', branch: 'publish',
340
+ needs: { scores: ['score'] } }, // every score, in item order
341
+ { id: 'publish', kind: 'task', type: 'PublishWorker', edges: [{ to: 'gather' }] },
342
+ { id: 'gather', kind: 'join', of: 'rank', edges: [{ to: 'done' }] },
343
+ { id: 'done', kind: 'end', outcome: 'completed' },
344
+ ],
345
+ };
346
+
347
+ const registry = {
348
+ predicates: [],
349
+ workers: { flow: { url: 'worker.js' } }, nodeTypes: { IngestWorker: { worker: 'flow' }, ScoreWorker: { worker: 'flow' }, PublishWorker: { worker: 'flow' } },
350
+ };
351
+ for (const def of [withContext, overRegion]) {
352
+ const bad = validateDefinition(def, registry);
353
+ if (bad.length) throw new Error(`${def.flowId} is invalid: ${bad.map((f) => f.rule).join()}`);
354
+ }
355
+
356
+ // And the wrong FORM is refused at construction, naming the one to write instead.
357
+ const wrongForm: FlowDefinition = {
358
+ ...overRegion,
359
+ nodes: overRegion.nodes.map((n) =>
360
+ n.id === 'rank' ? { ...n, needs: { scores: 'score' } } as typeof n : n),
361
+ };
362
+ const findings = validateDefinition(wrongForm, registry);
363
+ if (!findings.some((f) => f.rule === 'FD-V28' && f.detail.includes('write ["score"]'))) {
364
+ throw new Error('FD-V28 did not name the form to write instead');
365
+ }
366
+
367
+ // *Route to review if an earlier step flagged it, and publish either way* — a reference
368
+ // to a value only SOME paths produce. Written bare it is FD-V29 at construction; the `?`
369
+ // is how you say *if it ran*, and then `verdict` is simply missing on the other road.
370
+ const ifItRan = (ref: string): FlowDefinition => ({
371
+ flowId: 'publish-either-way', version: 1, entry: 'decide',
372
+ nodes: [
373
+ { id: 'decide', kind: 'task', type: 'ScoreWorker', edges: [
374
+ { when: { predicate: 'isLowConfidence' }, to: 'review' },
375
+ { to: 'publish' },
376
+ ] },
377
+ { id: 'review', kind: 'task', type: 'ScoreWorker', edges: [{ to: 'publish' }] },
378
+ { id: 'publish', kind: 'task', type: 'PublishWorker',
379
+ needs: { own: '.', verdict: ref },
380
+ edges: [{ to: 'done' }] },
381
+ { id: 'done', kind: 'end', outcome: 'completed' },
382
+ ],
383
+ });
384
+ const guarded = { predicates: ['isLowConfidence'], workers: registry.workers, nodeTypes: registry.nodeTypes };
385
+ const bare = validateDefinition(ifItRan('review'), guarded);
386
+ if (!bare.some((f) => f.rule === 'FD-V29' && f.detail.includes('"review?"'))) {
387
+ throw new Error('FD-V29 did not refuse the bare form, or did not name the fix');
388
+ }
389
+ if (validateDefinition(ifItRan('review?'), guarded).length) {
390
+ throw new Error('the conditional form should be accepted');
391
+ }
392
+ void Driver;
393
+ ```
394
+
395
+ **What it costs.** An **unconditional** reference that names a node with nothing to give you stops the
396
+ run with an `UnresolvedReference` (`FD-R3`, section 9). Since v0.13 most of that is caught at
397
+ construction — `FD-V27` checks the node is *upstream of you* and `FD-V29` that it must have RUN — so
398
+ what is left at run time is **whether the join a policy fired had actually waited for it**. *(This
399
+ read *"the TWO things the graph cannot know"* and named the arity case first — two `map` regions of
400
+ different arity, where a reader inside one could see neither output of the other. **`spec/`
401
+ v0.15 removes that road**: [ADR 0044](doc/decisions/0044-a-reference-is-visible-when-two-paths-agree-on-the-regions-that-contain-both-of-them.md)
402
+ makes a reader see all of a sibling region's outputs, so the case is a defect that was fixed rather
403
+ than a limit of what a validator can see, and one road is left. `doc/workflow.md` section 1 rule 5's
404
+ third direction, and this sentence is the copy a consumer reads first — it said *the resolver follows
405
+ in W50* for exactly as long as that was true.)*
406
+ And a reference names a **node**, never a path inside its value: `needs: { run: 'ingest' }` gives you
407
+ the whole of `ingest`'s result and your worker reads `.config` off it.
408
+
409
+ **And inside a `map`, what it costs is what you point it at.** Each instance gets its OWN COPY of
410
+ each key, so a reference to a large result is copied once per item. Measured at a map arity of 1000
411
+ over a ~126 KiB payload, **one case per process**: no `needs` **5.5 ms**, `{item: '.'}` **7.7 ms**,
412
+ one more key naming a genuinely small node **8.7 ms**, and the same key naming the large one
413
+ **613.6 ms** — **71x**, and the `?` suffix costs nothing (**617.4 ms**). The mechanism makes one copy
414
+ per instance and the cost is linear in the size of each, so the product is what you pay. That is also
415
+ why a reference names a whole node and never a path inside it — a path language would hide this
416
+ choice. *(These five figures are compared to `spec/flow-definition.md` section 3's own table by
417
+ `check-docs`' `X6`, both ways — three documents carried them and nothing compared any two, which is
418
+ `doc/workflow.md` section 5 rule 8 obligation (e)'s third shape.)*
419
+
420
+ **Nothing warns about it, and that is a decision rather than a gap.** There is no threshold to warn
421
+ at; the static predicate that needs no threshold fires on **every** intended use of the feature,
422
+ including the flow it exists for, so it restates the definition rather than diagnosing anything; and
423
+ a report channel exists (`onUnmintedTask`'s shape) and is declined because *this reference is resolved
424
+ once per item* is a fact about your document rather than about your run. **Measuring is not the
425
+ obstacle** — a size measurement is about 0.03% of what it would warn about at arity 1000.
426
+ `spec/flow-definition.md` section 3 carries the whole verdict and the measurements.
427
+
428
+ ### Validation happens at construction, not at routing time
429
+
430
+ `new Driver(...)` validates the whole definition before anything is queued. A typo cannot become a
431
+ routing failure discovered months later on the one workload that takes that branch.
432
+
433
+ ```ts run
434
+ import { validateDefinition, type FlowDefinition } from 'fyflow-orchestration';
435
+
436
+ const broken = {
437
+ flowId: 'broken', version: 1, entry: 'a',
438
+ nodes: [{ id: 'a', kind: 'task', type: 'NoSuchType', edges: [{ to: 'ghost' }] }],
439
+ } as unknown as FlowDefinition;
440
+
441
+ const findings = validateDefinition(broken, { predicates: [], workers: { flow: { url: 'worker.js' } }, nodeTypes: { RealType: { worker: 'flow' } } });
442
+ console.log(findings.map((f) => `${f.rule} at ${f.at}`).sort());
443
+ // FD-V2 (edge to a missing node) and FD-V32 (a node type the registry does not hold)
444
+ const rules = findings.map((f) => f.rule).sort();
445
+ if (!rules.includes('FD-V2') || !rules.includes('FD-V32')) {
446
+ throw new Error(`expected FD-V2 and FD-V32, got ${rules.join(',')}`);
447
+ }
448
+ ```
449
+
450
+ There are **42 refusals** over the id range `FD-V1` … `FD-V46` *(41 over `FD-V1` … `FD-V45` until W109 shipped the first BUILT-IN node type and added `FD-V46`, which refuses a node of a type that only transforms and declares no `transform`; 37 over `FD-V1` … `FD-V41` until W108 made a predicate DATA and added the four a `when` needs; 34 over `FD-V1` … `FD-V38` until W92 added the gate's three; 35 over `FD-V1` … `FD-V37` until W65 retired `workerType` and with it `FD-V5` and `FD-V34`, adding `FD-V38`; 32 over `FD-V1` … `FD-V34` until W61 added the three transform rules; 29 over `FD-V1` … `FD-V31` until W59 added the three node-type rules — and this sentence stood at 29 twelve lines below the table W59 rewrote, until its conformity reader read the two together)*, each with a failing example in
451
+ `spec/examples/invalid/`. Four ids in that range are **deliberately absent** and have no rule behind
452
+ them: `FD-V14`, which never existed; **`FD-V15`, retired when nested fan-out became
453
+ expressible**; and **`FD-V5` and `FD-V34`, retired with `workerType`** at v0.19 (section 13). The ids are never renumbered, so it is safe to assert on them. The full
454
+ list is `spec/flow-definition.md`.
455
+
456
+ **`FD-V14` is deliberately absent, and its id is held open so the gap is not read as a lost rule.**
457
+ It would have said *every path from a branch entry reaches the join* — which would make a decision
458
+ that routes a branch away from its join invalid input, and that case is the one this library exists
459
+ to get right (`ADR 0003`). Do not add it.
460
+
461
+ ---
462
+
463
+ ## 4. Decisions and predicates
464
+
465
+ **A `when` is an object, and it is either a registered NAME or a DECLARED comparison.** Since v0.27
466
+ ([ADR 0074](doc/decisions/0074-a-predicate-is-a-registered-name-or-a-declared-comparison-over-the-context.md))
467
+ a flow document can say what a branch depends on without any code at all, which is what lets a UI or
468
+ an agent emit a complete flow. A name still works and is what a predicate that COMPUTES uses.
469
+
470
+ ```ts run
471
+ import { validateDefinition, type FlowDefinition } from 'fyflow-orchestration';
472
+
473
+ // A `when` is EXACTLY ONE of three shapes, told apart by which keys it carries.
474
+ const flow: FlowDefinition = {
475
+ flowId: 'declared', version: 1, entry: 'score',
476
+ nodes: [
477
+ { id: 'score', kind: 'task', type: 'ScoreWorker', edges: [
478
+ // 1. a COMPARISON — a key path over the context, an operator, a literal.
479
+ { when: { path: 'outcome.result.confidence', op: 'lt', value: 0.5 }, to: 'review' },
480
+ // 2. a flat GROUP — `all` or `any`, whose members are comparisons.
481
+ { when: { any: [
482
+ { path: 'outcome.result.score', op: 'gte', value: 90 },
483
+ // `default` stands in where the path does not resolve. It is what `?? 0` is, and it
484
+ // is what makes `lte` the negation of `gt` — without it BOTH are false when absent.
485
+ { path: 'outcome.result.retries', op: 'gt', value: 2, default: 0 },
486
+ ] }, to: 'review' },
487
+ // 3. a registered NAME — for anything the subset cannot say.
488
+ { when: { predicate: 'needsAHuman' }, to: 'review' },
489
+ { to: 'publish' },
490
+ ] },
491
+ { id: 'review', kind: 'task', type: 'ReviewWorker', edges: [{ to: 'done' }] },
492
+ { id: 'publish', kind: 'task', type: 'PublishWorker', edges: [{ to: 'done' }] },
493
+ { id: 'done', kind: 'end', outcome: 'completed' },
494
+ ],
495
+ };
496
+ const registry = {
497
+ predicates: ['needsAHuman'],
498
+ workers: { flow: { url: 'worker.js' } },
499
+ nodeTypes: { ScoreWorker: { worker: 'flow' }, ReviewWorker: { worker: 'flow' }, PublishWorker: { worker: 'flow' } },
500
+ };
501
+ if (validateDefinition(flow, registry).length !== 0) throw new Error('the three shapes are all valid');
502
+
503
+ // **Every refusal is at CONSTRUCTION, with the key it is about.** Four rules:
504
+ const refuse = (when: unknown) => validateDefinition(
505
+ { ...flow, nodes: flow.nodes.map((n) => n.id === 'score' ? { ...n, edges: [{ when, to: 'review' }, { to: 'publish' }] } : n) } as FlowDefinition,
506
+ registry).map((f) => f.rule);
507
+
508
+ // FD-V42 — the bare STRING that was the form until v0.27, and the message names the migration.
509
+ if (refuse('needsAHuman')[0] !== 'FD-V42') throw new Error('a bare string is FD-V42');
510
+ // FD-V43 — a path whose first segment is neither `outcome` nor `needs` would resolve to
511
+ // nothing in SILENCE, so it is refused rather than answered.
512
+ if (refuse({ path: 'result.confidence', op: 'lt', value: 0.5 })[0] !== 'FD-V43') throw new Error('a rootless path is FD-V43');
513
+ // FD-V44 — the seven operators are a CLOSED set and one outside it is named.
514
+ if (refuse({ path: 'outcome.result.name', op: 'matches', value: '^a' })[0] !== 'FD-V44') throw new Error('an unknown operator is FD-V44');
515
+ // FD-V45 — a group is FLAT, and a member naming a registered predicate could be IMPURE.
516
+ if (refuse({ all: [{ any: [{ path: 'outcome.result.a', op: 'eq', value: 1 }] }] })[0] !== 'FD-V45') throw new Error('nesting is FD-V45');
517
+ ```
518
+
519
+ | | |
520
+ |---|---|
521
+ | `path` | a key path over the context, **rooted at `outcome` or `needs`** — `outcome.result.x`, `outcome.taskId`, `needs.earlier.flag`. No array index, no wildcard, no expression |
522
+ | `op` | one of `eq`, `ne`, `lt`, `lte`, `gt`, `gte`, `startsWith`, and **nothing else** |
523
+ | `value` | a scalar: `null`, a boolean, a finite number or a string |
524
+ | `default` | optional — what is compared when the path does not resolve **or resolves to `null`**, which is exactly `??`. Without it an absent path is **no match**, and a resolved `null` is compared as the scalar it is |
525
+ | `length` | optional `true` — compare the LENGTH of an array or a string, after `default` |
526
+
527
+ **Three things to know before you write one.**
528
+ **A comparison between kinds is no match, never a coercion** — `{ op: 'lt', value: 50 }` is false
529
+ against the string `"9"` where JavaScript's `<` would say true.
530
+ **An absent path is no match for every operator, including `ne`** — so `ne` does not mean *the key
531
+ is missing or different*; add a `default` if that is what you want, and a `default` covers a key
532
+ holding `null` as well as a missing one.
533
+ **Write a negation as the complementary operator** — `gte` for *not* `lt` — and know its scope:
534
+ that holds over NUMBERS, and over an absent path where you declared a `default`. Over a value an
535
+ ordering operator cannot compare, a string or a boolean, **both it and its complement are false**.
536
+ `startsWith` has no complement here at all.
537
+ **A declared predicate CANNOT be impure.** There is no clock, no random source and no way to name
538
+ anything but the context, so re-evaluating it over the same log gives the same answer. A registered
539
+ name has no such guarantee — which is the trade, and why a regular expression, a date window or a
540
+ lookup stays a name.
541
+
542
+ ### A registered NAME, which is what the subset cannot say
543
+
544
+ **An edge names a predicate; it does not carry one.** The definition stays data, and an unknown name
545
+ fails at construction rather than mid-run.
546
+
547
+ ```ts run
548
+ import { eligibleEdges, selectEdge, type FlowNode, type Predicate } from 'fyflow-orchestration';
549
+
550
+ const node: FlowNode = {
551
+ id: 'score', kind: 'task', type: 'ScoreWorker',
552
+ edges: [{ when: { predicate: 'isLowConfidence' }, to: 'review' }, { to: 'publish' }],
553
+ };
554
+
555
+ const isLowConfidence: Predicate = (ctx) => {
556
+ const r = ctx.outcome.result as { confidence?: number } | null;
557
+ return typeof r?.confidence === 'number' && r.confidence < 0.5;
558
+ };
559
+
560
+ const outcome = {
561
+ kind: 'outcome' as const, taskId: 'r1:score:-:0', terminalState: 'done' as const,
562
+ result: { confidence: 0.2 }, error: null, attempts: 0,
563
+ schedulerTimestamp: 1, observedAt: 1, startTime: 1, endTime: 2, executionTime: 1,
564
+ };
565
+
566
+ // Eligibility is decided FIRST, by terminal state; `when` is evaluated only over
567
+ // the edges that are already eligible.
568
+ console.log(eligibleEdges(node, 'done').map((e) => e.to)); // [ 'review', 'publish' ]
569
+
570
+ // **A context BUILDER, not a context** - each edge declares its own `needs`, so each
571
+ // predicate sees a different `ctx.needs`. `selectEdge` calls this only for an edge
572
+ // that carries a `when`, so a default edge costs nothing.
573
+ const taken = selectEdge(node, 'r1', 'done', () => ({ outcome, needs: {} }),
574
+ new Map([['isLowConfidence', isLowConfidence]]));
575
+ console.log(taken.taken?.to); // 'review'
576
+ if (taken.taken?.to !== 'review') throw new Error('the guarded edge should win at 0.2');
577
+ ```
578
+
579
+ **Edges are tried in declaration order** and the first whose `when` passes is taken. An edge with no
580
+ `when` is the default and matches immediately, so **it must come last among the edges eligible for
581
+ that terminal state** — not last in the array. `FD-V7` is scoped per terminal state, which is why
582
+ section 3's example can declare its `on: ['failed']` edge first: for `done` the eligible edges are
583
+ `review` then `publish`, and the default is last of those.
584
+
585
+ A predicate receives the node's **effective outcome** — the same shape whether the task settled or a
586
+ human resolved it. It should be pure: it is called during routing, and the library does not enforce
587
+ that.
588
+
589
+ ### An edge can let its predicate see an EARLIER node
590
+
591
+ By default a predicate sees **only its own node's outcome**. *Route to review if an earlier step
592
+ flagged low confidence* was not expressible until v0.12 —
593
+ [ADR 0041](doc/decisions/0041-a-decision-sees-its-nodes-outcome-and-whatever-its-own-edge-declares.md),
594
+ answering [Q8](doc/questions/q8.md).
595
+
596
+ **An edge may carry `needs`, and its predicate sees the result under `ctx.needs`.** Same grammar as
597
+ a node's (section 3) — a node id, or a one-element array holding one — **minus `"."`**, which an edge
598
+ has no use for: *what a predicate would have received anyway* is `ctx.outcome`.
599
+
600
+ ```ts run
601
+ import { validateDefinition, type FlowDefinition, type Predicate } from 'fyflow-orchestration';
602
+
603
+ const registry = {
604
+ predicates: ['flaggedEarlier'],
605
+ workers: { flow: { url: 'worker.js' } }, nodeTypes: { IngestWorker: { worker: 'flow' }, ScoreWorker: { worker: 'flow' }, ReviewWorker: { worker: 'flow' }, PublishWorker: { worker: 'flow' } },
606
+ };
607
+
608
+ const flow: FlowDefinition = {
609
+ flowId: 'route-on-an-earlier-step', version: 1, entry: 'ingest',
610
+ nodes: [
611
+ { id: 'ingest', kind: 'task', type: 'IngestWorker', edges: [{ to: 'score' }] },
612
+ { id: 'score', kind: 'task', type: 'ScoreWorker', edges: [
613
+ // this edge's predicate can see `ingest`; the default edge below sees `{}`
614
+ { when: { predicate: 'flaggedEarlier' }, to: 'review', needs: { earlier: 'ingest' } },
615
+ { to: 'publish' },
616
+ ] },
617
+ { id: 'review', kind: 'task', type: 'ReviewWorker', edges: [{ to: 'publish' }] },
618
+ { id: 'publish', kind: 'task', type: 'PublishWorker', edges: [{ to: 'done' }] },
619
+ { id: 'done', kind: 'end', outcome: 'completed' },
620
+ ],
621
+ };
622
+
623
+ const flaggedEarlier: Predicate = (ctx) =>
624
+ (ctx.needs as { earlier?: { flagged?: boolean } }).earlier?.flagged === true;
625
+
626
+ const bad = validateDefinition(flow, registry);
627
+ if (bad.length) throw new Error(`invalid: ${bad.map((f) => f.rule).join()}`);
628
+
629
+ // And the predicate is CALLED, so the one thing this section exists to show is
630
+ // executed rather than only typechecked.
631
+ const asDriverWouldBuildIt = {
632
+ outcome: { result: { score: 0.9 } } as never,
633
+ needs: { earlier: { flagged: true } },
634
+ };
635
+ if (!flaggedEarlier(asDriverWouldBuildIt)) throw new Error('ctx.needs.earlier was not what the edge declared');
636
+ if (flaggedEarlier({ ...asDriverWouldBuildIt, needs: {} })) {
637
+ throw new Error('an edge that declares nothing must give {} and the predicate must say no');
638
+ }
639
+
640
+ // `"."` is refused ON AN EDGE, and the message names what you already have.
641
+ const withSelf = {
642
+ ...flow,
643
+ nodes: flow.nodes.map((n) =>
644
+ n.id === 'score'
645
+ ? { ...n, edges: [{ when: { predicate: 'flaggedEarlier' }, to: 'review', needs: { x: '.' } }, { to: 'publish' }] }
646
+ : n),
647
+ } as FlowDefinition;
648
+ const findings = validateDefinition(withSelf, registry);
649
+ if (!findings.some((f) => f.rule === 'FD-V26' && f.detail.includes('ctx.outcome'))) {
650
+ throw new Error('FD-V26 did not refuse `"."` on an edge');
651
+ }
652
+ ```
653
+
654
+ **It is on the EDGE and not on the node, for two reasons worth knowing.** A `needs` on the node
655
+ changes what its **worker** receives, so you could not give a predicate reach without editing a
656
+ worker; and *what the node received* is not something the driver can reconstruct when the decision is
657
+ made — the **entry** node's payload is never written to the log at all, and reconstructing any other
658
+ node's would mean re-deriving the whole payload contract a second time. **A reference resolves from
659
+ what the run produced**, which is available either way.
660
+
661
+ **`ctx.needs` is `{}` when the edge declares nothing**, never `undefined`; it is **frozen**, so a
662
+ predicate that writes to it throws rather than editing what the next predicate reads; and a reference
663
+ the run did not produce stops the run with an `UnresolvedReference` (`FD-R3`, section 9) exactly as a
664
+ node's does. **An edge with no `when` may carry a `needs` and nothing will ever read it** — accepted
665
+ and inert, which `spec/flow-definition.md` section 3 records rather than refuses.
666
+
667
+ ---
668
+
669
+ ## 5. Fan-out, join, and which policy to pass
670
+
671
+ ```ts run
672
+ import { validateDefinition, type FlowDefinition } from 'fyflow-orchestration';
673
+
674
+ const fanout: FlowDefinition = {
675
+ flowId: 'score-both', version: 1, entry: 'ingest',
676
+ nodes: [
677
+ { id: 'ingest', kind: 'task', type: 'IngestWorker', edges: [{ to: 'split' }] },
678
+ { id: 'split', kind: 'fanout', branches: ['scoreA', 'scoreB'] },
679
+ { id: 'scoreA', kind: 'task', type: 'ScoreWorker', edges: [{ to: 'collect' }] },
680
+ { id: 'scoreB', kind: 'task', type: 'ScoreWorker', edges: [{ to: 'collect' }] },
681
+ { id: 'collect', kind: 'join', of: 'split', edges: [{ to: 'publish' }] },
682
+ { id: 'publish', kind: 'task', type: 'PublishWorker', edges: [{ to: 'done' }] },
683
+ { id: 'done', kind: 'end', outcome: 'completed' },
684
+ ],
685
+ };
686
+
687
+ const bad = validateDefinition(fanout, {
688
+ predicates: [],
689
+ workers: { flow: { url: 'worker.js' } }, nodeTypes: { IngestWorker: { worker: 'flow' }, ScoreWorker: { worker: 'flow' }, PublishWorker: { worker: 'flow' } },
690
+ });
691
+ if (bad.length) throw new Error(`section 5's own example is invalid: ${bad.map((f) => f.rule).join()}`);
692
+ ```
693
+
694
+ ### Pass `liveToken`
695
+
696
+ Three policies are exported. **Use `liveToken`.**
697
+
698
+ ```ts run
699
+ import { POLICIES, liveToken, arrivalCount, liveTokenRequiringArrival } from 'fyflow-orchestration';
700
+
701
+ const ids = Object.keys(POLICIES).sort();
702
+ console.log(ids);
703
+ if (ids.join() !== 'arrival-count,live-token,live-token-requiring-arrival') {
704
+ throw new Error(`this reference lists three policies and the library exports ${ids.join(', ')}`);
705
+ }
706
+ console.log(liveToken.id, arrivalCount.id, liveTokenRequiringArrival.id);
707
+ if (POLICIES['live-token'] !== liveToken) throw new Error('POLICIES must expose the same objects');
708
+ ```
709
+
710
+ **That `throw` used to be a trailing comment**, and it is the one line of this
711
+ section a reader could rely on being current: `check-docs` executes the snippet
712
+ and asserts nothing about a comment, so a fourth policy would have left it
713
+ silently wrong with the whole gate green. `JP-R9` and `npm run check:policies`
714
+ are the other half — they compare `POLICIES` against `spec/join-policy.md`
715
+ section 3, which is where the names are actually fixed.
716
+
717
+ `arrivalCount` waits for **N** branches, where N is fixed when the fan-out opens. That is wrong
718
+ whenever *"how many will arrive"* differs from *"how many opened"* — and **every non-success outcome
719
+ makes it differ**. A branch routed away by a decision never arrives. A branch whose `optional` task
720
+ fails never arrives. The counter never completes, and the run waits forever with nothing queued and
721
+ nothing running. **Nothing throws** — but run state does report it: such a run folds to **`stuck`**,
722
+ the sixth outcome, and `driver.joinStatusOf()` names the join that can no longer fire
723
+ ([ADR 0015](doc/decisions/0015-a-run-whose-join-can-no-longer-fire-is-stuck.md)). A hang you have to
724
+ notice yourself is still a hang.
725
+
726
+ `liveToken` counts *live branches* instead: opening one increments, and arriving, exiting or being
727
+ lost each decrement. The join fires when the count reaches zero **having been positive**, whichever
728
+ road each branch took.
729
+
730
+ `liveTokenRequiringArrival` is `liveToken` **with one clause added**: it fires only if at least one
731
+ branch actually arrived. On a flow where every branch is routed away, `liveToken` fires a join **no
732
+ branch reached** and carries **`{}`** onward — an empty collection, which is what a join with no
733
+ arrivals has to say; this one instead reports the join as unfireable — through
734
+ `onUnsatisfiableJoin`, with the policy's own state, and the run folds to **`stuck`** with
735
+ `regions[…][…].joined === false` whether or not you registered that callback — **two subscripts**,
736
+ because `RunState.regions` is keyed by a region's opener and then by that opener's own `branchPath`
737
+ ([ADR 0062](doc/decisions/0062-a-region-instance-is-its-opener-and-the-openers-own-branch-path.md)).
738
+ A region whose opener is at top level has one entry, at `'-'`.
739
+
740
+ ```ts run
741
+ import {
742
+ liveToken, liveTokenRequiringArrival,
743
+ type JoinPolicy, type Region, type RegionEvent,
744
+ } from 'fyflow-orchestration';
745
+
746
+ const region: Region = { fanoutId: 'split', joinId: 'collect', branches: ['a', 'b'], members: [] };
747
+ // `regionPath` is the OPENER'S own branchPath, so the pair (regionId, regionPath)
748
+ // names ONE region instance — `'-'` here because `split` opens at top level. A map
749
+ // enters one opener once per item, so its node id alone stopped naming one region.
750
+ const everyBranchExits: RegionEvent[] = [
751
+ { type: 'branch.opened', regionId: 'split', regionPath: '-', branchPath: '0' },
752
+ { type: 'branch.opened', regionId: 'split', regionPath: '-', branchPath: '1' },
753
+ { type: 'branch.exited', regionId: 'split', regionPath: '-', branchPath: '0', at: 'stop' },
754
+ { type: 'branch.exited', regionId: 'split', regionPath: '-', branchPath: '1', at: 'stop' },
755
+ ];
756
+
757
+ function verdict<S>(policy: JoinPolicy<S>, events: RegionEvent[]) {
758
+ let state = policy.open(region);
759
+ for (const event of events) state = policy.step(state, event);
760
+ return policy.status(state);
761
+ }
762
+
763
+ console.log(verdict(liveToken, everyBranchExits), verdict(liveTokenRequiringArrival, everyBranchExits));
764
+ if (verdict(liveToken, everyBranchExits) !== 'satisfied') {
765
+ throw new Error('liveToken fires with zero arrivals');
766
+ }
767
+ if (verdict(liveTokenRequiringArrival, everyBranchExits) !== 'unsatisfiable') {
768
+ throw new Error('liveTokenRequiringArrival declines a join no branch reached');
769
+ }
770
+ ```
771
+
772
+ **Which to pass is a judgement about your flow, and neither answer is safe by default.** If every
773
+ branch of a fan-out ending somewhere else is a legitimate way for your flow to finish, `liveToken`
774
+ is right and the run completes. If it means the work the join was going to combine never happened,
775
+ `liveTokenRequiringArrival` reports that join dead and does not run its successor on nothing. The
776
+ recommendation is still `liveToken`, because a run that stops is the more expensive mistake to make
777
+ by accident.
778
+
779
+ **"Stops the run" is exact at depth 1 and not at depth 2**, which is worth knowing before you nest a
780
+ fan-out. A dead join stops **its own** region; if that region sits inside a branch of another one,
781
+ the branch **ends** and the outer join gets to decide for itself — so the outer join may fire and its
782
+ successor may run, doing real work in a run that will still fold to `stuck` (`joined: false` on the
783
+ dead region is what `stuck` reads, and it reads every region). Section 11's nested-fan-out row, and
784
+ [ADR 0030](doc/decisions/0030-a-join-is-transitive-and-a-dead-join-ends-the-branch-it-sits-in.md).
785
+
786
+ **One refinement is deliberately still unbuilt.** `arrivalCount`'s **N is fixed when the fan-out
787
+ opens** and is *not* decremented when a branch is routed away. Decrementing it is
788
+ [`JP-R7`](spec/join-policy.md), and
789
+ [ADR 0025](doc/decisions/0025-the-zero-arrival-join-is-reportable-and-arrival-counting-is-not-repaired.md)
790
+ refused it: measured over the join matrix, every row where it differs from `liveToken` is a row where
791
+ it **waits forever** and `liveToken` proceeds, and it can never say why. It is `liveToken` weakened.
792
+
793
+ **`arrivalCount` is still exported deliberately** — it is the falsifiable form of the simpler
794
+ mechanism, and the comparison that chose between them is only meaningful while both can be run. It
795
+ is not a supported choice. See
796
+ [ADR 0003](doc/decisions/0003-a-join-is-satisfied-when-its-region-has-no-live-branches.md) and
797
+ [ADR 0025](doc/decisions/0025-the-zero-arrival-join-is-reportable-and-arrival-counting-is-not-repaired.md).
798
+
799
+ ### What a join carries downstream
800
+
801
+ **The arrived branches' results, keyed by branch path.** A satisfied join dispatches its successor
802
+ with an object whose keys are the `branchPath`s of the branches that **reached** it, and whose values
803
+ are what each of those branches carried in — so a fan-out's results are not thrown away, and the
804
+ value is the same whatever order the branches finished in.
805
+
806
+ ```ts run
807
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
808
+ import { type FlowDefinition, type FoldInput, type RegistryDocument } from 'fyflow-orchestration';
809
+
810
+ // Two workers of your own: one returns a score after a stated delay, the other tags whatever it was
811
+ // handed so the collection is legible in what it returns.
812
+ const registry: RegistryDocument = {
813
+ workers: {
814
+ scorer: { url: '../../examples/getting-started/scorer.ts' },
815
+ tag: { url: '../../examples/getting-started/tag.ts' },
816
+ },
817
+ nodeTypes: {
818
+ Scorer: { worker: 'scorer', inline: true },
819
+ Tag: { worker: 'tag', inline: true },
820
+ },
821
+ };
822
+ const flow: FlowDefinition = {
823
+ flowId: 'collect', version: 1, entry: 'split',
824
+ nodes: [
825
+ { id: 'split', kind: 'fanout', branches: ['scoreA', 'scoreB'] },
826
+ // `scoreA` is branch path "0" and is made SLOW on purpose, so "1" settles first.
827
+ { id: 'scoreA', kind: 'task', type: 'Scorer', settings: { score: 0.9, ms: 40 }, edges: [{ to: 'collect' }] },
828
+ { id: 'scoreB', kind: 'task', type: 'Scorer', settings: { score: 0.4 }, edges: [{ to: 'collect' }] },
829
+ { id: 'collect', kind: 'join', of: 'split', edges: [{ to: 'publish' }] },
830
+ { id: 'publish', kind: 'task', type: 'Tag', settings: { label: 'publish' }, edges: [{ to: 'done' }] },
831
+ { id: 'done', kind: 'end', outcome: 'completed' },
832
+ ],
833
+ };
834
+
835
+ const runtime = createRuntime(registry, {}, { base: import.meta.url });
836
+ // **`onFoldInput` is still here and what it is for is the ORDER, not the END.** `await done` is the
837
+ // end (ADR 0058). `RunState.nodes` keys everything by task id and carries no settlement order — *which
838
+ // branch settled first* is a fact only the input SEQUENCE carries, and it is what the
839
+ // claim below is measured against. **This is the residue**: `done` takes the termination out of a
840
+ // snippet and leaves whatever else the callback was doing.
841
+ const inputs: FoldInput[] = [];
842
+ const state = await runtime.start(flow, {}, { onFoldInput: (input) => { inputs.push(input); } }).done;
843
+ await runtime.shutdown();
844
+
845
+ // **The order is READ, not assumed.** The delay above ASKS for branch "1" to settle first; this is
846
+ // what the run actually did, and the claim below is about the collection being independent of it.
847
+ const settledOrder = inputs.flatMap((i) =>
848
+ (i.kind === 'outcome' && /:score[AB]:/.test(i.record.taskId) ? [i.record.taskId.split(':')[1]] : []));
849
+ console.log(settledOrder.join(' then ')); // 'scoreB then scoreA'
850
+
851
+ // `publish` is the join's successor, so what it received IS the collection — and `tag` returns what
852
+ // it was handed with its own label added, so the collection is readable in the result.
853
+ const published = Object.values(state.nodes).find((n) => n.nodeId === 'publish')?.result;
854
+ console.log(JSON.stringify(published)); // {"0":{"score":0.9},"1":{"score":0.4},"publish":true}
855
+
856
+ const want = '{"0":{"score":0.9},"1":{"score":0.4},"publish":true}';
857
+ if (JSON.stringify(published) !== want) throw new Error(`the join carried ${JSON.stringify(published)}`);
858
+ // **B settled FIRST and is still under key "1".** The collection is keyed by BRANCH PATH, not by
859
+ // arrival order, which is the whole point — and the two assertions are a pair: the first says which
860
+ // order the run took, the second says the record did not follow it.
861
+ if (settledOrder.join() !== 'scoreB,scoreA') throw new Error(`settled ${settledOrder.join()} — the arrangement did not hold, so the claim below is untested`);
862
+ ```
863
+
864
+ **Three things to know before you write code against it.**
865
+
866
+ 1. **A branch that did not arrive has no key.** Routed away to an `end`, lost with no eligible edge,
867
+ or still suspended on a person — all three are absent rather than `null`. So `Object.keys(payload)`
868
+ is *which branches arrived*, and it is **not** the width of the fan-out. Code that assumes N
869
+ entries is wrong on every non-success outcome.
870
+ 2. **`{}` is a real answer, not an error.** Under `liveToken` a join fires once its region has no live
871
+ branches, and every branch reaching an `end` satisfies that with nothing arrived. If that should be
872
+ a failure instead, pass `liveTokenRequiringArrival` — see above.
873
+ 3. **The keys are strings, in ascending branch-path order**, compared segment by segment as numbers —
874
+ so `"0"`, `"1"`, `"2"`, `"10"` at the top level, and `"1.0"`, `"1.1"`, `"1.10"` inside branch 1.
875
+ A nested region's inner join carries its own object, which becomes the value of the outer branch it
876
+ sits in. **Every key of one collection has the same number of segments**, because they are the
877
+ branch paths of one region. *(An earlier version of this bullet illustrated the order as `"1"`,
878
+ `"1.0"`, `"2"`, `"10"` — a mix of depths that no single collection can hold, and one that
879
+ `Object.keys` would not even return in that order, because JavaScript re-orders integer-like keys
880
+ ahead of string ones. W34's conformity reader.)*
881
+
882
+ *(This section read **"`null`, currently … this is [Q36](doc/questions/q36.md) and it is open"**
883
+ until [ADR 0037](doc/decisions/0037-what-a-branch-receives-is-a-property-of-the-fan-out-and-what-a-join-carries-is-a-property-of-the-region.md).
884
+ The argument for the `null` was real and is unchanged — which branch finishes last is genuinely
885
+ non-deterministic — and what answered it is that carrying **all** the arrivals keyed by branch is not
886
+ picking one. `spec/join-policy.md` section 4, `JP-R12`.)*
887
+
888
+ ### The same on the RUNTIME — a decision, a fan-out and a join on real workers
889
+
890
+ *Everything in this section runs on a REAL `FyflowScheduler`. The three snippets above this line
891
+ touch no scheduler at all — they validate a document, list the policies, and call a policy
892
+ directly — and the one that did build a fake, *What a join carries downstream*, now builds a
893
+ runtime like everything else ([W71](doc/work-items/w71-the-reference-stops-teaching-a-fake-scheduler.md)).
894
+ *(This sentence has been wrong twice in two days and in opposite directions. It read *everything
895
+ above this line runs on a fake scheduler* while section 2 above it built a real one — W70's
896
+ conformity reader — and then, scoped to this section, it read the same thing while ZERO snippets
897
+ here did, because W71 converted the only one that had. Both times an item rewrote the code inside
898
+ the fences and left every sentence around them untouched.)* This is the same three node kinds on
899
+ `createRuntime`, two runs on one runtime, every value read off the run — [W70](doc/work-items/w70-a-flow-that-decides-and-a-custom-group-on-the-runtime.md).
900
+ It is also `README.md`'s second snippet, held byte-equal to this one by `check-docs`'s `X8`.*
901
+
902
+ ```ts run
903
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
904
+ import { type FlowDefinition, type Predicate, type RegistryDocument, type RunState } from 'fyflow-orchestration';
905
+
906
+ // THE REGISTRY — one worker, one type, and the NAME of a predicate the flow may use. A predicate is
907
+ // code (ADR 0002), supplied beside the document under that name; a flow using a name nobody supplied
908
+ // is refused at construction, not at the decision. The worker tags its input with the node's label.
909
+ const registry: RegistryDocument = {
910
+ workers: { tag: { url: '../../examples/getting-started/tag.ts' } }, // in your program: './tag.ts'
911
+ nodeTypes: { Tag: { worker: 'tag', inline: true } },
912
+ predicates: ['isLowConfidence'],
913
+ };
914
+ // A predicate reads the EFFECTIVE outcome, and `result` is `null` for an optional task that
915
+ // failed (`OR-R9`) — so it is guarded, as section 4's is. Unreachable here, because `ingest` is
916
+ // not optional; guarded anyway, because this is the form a consumer copies.
917
+ const isLowConfidence: Predicate = (ctx) => {
918
+ const r = ctx.outcome.result as { confidence?: number } | null;
919
+ return typeof r?.confidence === 'number' && r.confidence < 0.5;
920
+ };
921
+
922
+ // THE FLOW — a DECISION, a FAN-OUT, a JOIN. `ingest`'s first edge is taken when its predicate says so,
923
+ // otherwise the last, unconditional one. `split` opens two branches that run concurrently; `collect`
924
+ // waits for both and hands `publish` every branch's result keyed by branch path (JP-R12). `review` is
925
+ // the road the run takes when confidence is low — it never reaches the join, and the run completes.
926
+ const flow: FlowDefinition = {
927
+ flowId: 'decide', version: 1, entry: 'ingest',
928
+ nodes: [
929
+ { id: 'ingest', kind: 'task', type: 'Tag', settings: { label: 'ingest' },
930
+ edges: [{ when: { predicate: 'isLowConfidence' }, to: 'review' }, { to: 'split' }] },
931
+ { id: 'review', kind: 'task', type: 'Tag', settings: { label: 'review' }, edges: [{ to: 'done' }] },
932
+ { id: 'split', kind: 'fanout', branches: ['enrich', 'classify'] },
933
+ { id: 'enrich', kind: 'task', type: 'Tag', settings: { label: 'enrich' }, edges: [{ to: 'collect' }] },
934
+ { id: 'classify', kind: 'task', type: 'Tag', settings: { label: 'classify' }, edges: [{ to: 'collect' }] },
935
+ { id: 'collect', kind: 'join', of: 'split', edges: [{ to: 'publish' }] },
936
+ { id: 'publish', kind: 'task', type: 'Tag', settings: { label: 'publish' }, edges: [{ to: 'done' }] },
937
+ { id: 'done', kind: 'end', outcome: 'completed' },
938
+ ],
939
+ };
940
+
941
+ // ONE runtime, TWO runs. Pools belong to the runtime and are shared across runs; each run is its own
942
+ // `Driver` with its own fold, and the join policy defaults to `live-token` (ADR 0003).
943
+ const runtime = createRuntime(registry, { predicates: { isLowConfidence } }, { base: import.meta.url });
944
+ // Each run has its OWN `done`, so two runs are two promises — `start` is called twice and the whole
945
+ // of the wait is `await`. Nothing here folds anything: the runtime keeps the fold (ADR 0058).
946
+ const run = (payload: unknown) => runtime.start(flow, payload).done;
947
+ const high = await run({ confidence: 0.9 });
948
+ const low = await run({ confidence: 0.2 });
949
+ await runtime.shutdown();
950
+
951
+ // WHICH nodes ran, read off each run's own state — the decision took a different edge each time.
952
+ const ran = (s: RunState) => Object.values(s.nodes).map((n) => n.nodeId).sort();
953
+ const resultOf = (s: RunState, id: string) => Object.values(s.nodes).find((n) => n.nodeId === id)?.result;
954
+ console.log(ran(high)); // [ 'classify', 'enrich', 'ingest', 'publish' ]
955
+ console.log(ran(low)); // [ 'ingest', 'review' ]
956
+ console.log(resultOf(high, 'publish')); // { '0': { confidence: 0.9, ingest: true, enrich: true }, '1': { …, classify: true }, publish: true }
957
+
958
+ if (high.outcome !== 'completed' || low.outcome !== 'completed') throw new Error(`outcomes ${high.outcome} / ${low.outcome}`);
959
+ if (ran(high).join() !== 'classify,enrich,ingest,publish') throw new Error(`high ran ${ran(high).join()}`);
960
+ if (ran(low).join() !== 'ingest,review') throw new Error(`low ran ${ran(low).join()}`);
961
+ // `publish` received BOTH branches' results, keyed by branch path (JP-R12), and each branch's value
962
+ // carries the whole road it came by. What this string pins is the branch-to-KEY MAPPING: the ASCENDING
963
+ // order that rule also fixes is not observable at depth 1, because "0" and "1" are integer-like keys
964
+ // and the language orders those ascending whatever the driver did. A nested region would show it.
965
+ const want = '{"0":{"confidence":0.9,"ingest":true,"enrich":true},"1":{"confidence":0.9,"ingest":true,"classify":true},"publish":true}';
966
+ if (JSON.stringify(resultOf(high, 'publish')) !== want) throw new Error(`publish received ${JSON.stringify(resultOf(high, 'publish'))}`);
967
+ ```
968
+
969
+ ---
970
+
971
+ ## 6. When something fails
972
+
973
+ **A non-success outcome routes only where an edge opts in.** This is the rule most people guess
974
+ wrong.
975
+
976
+ ```ts run
977
+ import { eligibleEdges, type FlowNode } from 'fyflow-orchestration';
978
+
979
+ const node: FlowNode = {
980
+ id: 'extract', kind: 'task', type: 'ExtractWorker', optional: true,
981
+ edges: [{ on: ['failed'], to: 'quarantine' }, { to: 'publish' }],
982
+ };
983
+
984
+ console.log(eligibleEdges(node, 'done').map((e) => e.to)); // [ 'publish' ]
985
+ console.log(eligibleEdges(node, 'failed').map((e) => e.to)); // [ 'quarantine' ]
986
+ if (eligibleEdges(node, 'failed').map((e) => e.to).join() !== 'quarantine') {
987
+ throw new Error('an edge with no `on` must not be eligible for a failure');
988
+ }
989
+ ```
990
+
991
+ An edge with no `on` defaults to `['done']`. **If nothing opts into a failure, the path ends there** —
992
+ that is a legitimate way to write a flow, not an error, and it is declared by omission.
993
+
994
+ ### `optional` decides what a failure *means*
995
+
996
+ | the task | the scheduler ends it | the branch |
997
+ |---|---|---|
998
+ | `optional: true` | `failed` | routes if an edge opts in; otherwise **lost** |
999
+ | `optional: false` (default) | **`user_action`** | **suspends** — section 7 |
1000
+
1001
+ **`optional: false` does not mean "must not fail".** It means *"a human decides"*. If you want the
1002
+ flow to handle failure without a person, set `optional: true` and give it an `on: ['failed']` edge.
1003
+
1004
+ **An `on: ['failed']` edge on a non-optional node is not dead code**, which is the natural reading of
1005
+ the table above and is wrong. `resolve(taskId, 'failed', …)` routes exactly as an `optional` node's
1006
+ failure would, and `RS-R3` notes it is **the only road by which a non-`optional` node reaches an
1007
+ `on: ['failed']` edge**. That is the half of the human-in-the-loop story section 7 completes.
1008
+
1009
+ ---
1010
+
1011
+ ## 7. The human in the loop
1012
+
1013
+ A non-optional task that runs out of retries ends in `user_action`. The branch **suspends**: the join
1014
+ keeps waiting, and the run is not stuck.
1015
+
1016
+ > **A `user_action` task fires TWO events, and the first one is called `task.failed`.** The scheduler
1017
+ > sets `state = 'user_action'` and then dispatches `task.failed` *and* `task.user_action` carrying
1018
+ > the same object. **Never key on the event name alone**: that first `task.failed` is for a task
1019
+ > that did not fail, and treating it as one both double-counts the settlement and routes a
1020
+ > human-intervention case down an error edge. The driver drops it and records on the second — it
1021
+ > matters if you subscribe to the scheduler yourself.
1022
+ > [ADR 0007](doc/decisions/0007-the-driver-normalizes-events-into-an-immutable-outcome-record.md)
1023
+ > rule 2; the scheduler's own reference does not document it for the ordinary path.
1024
+
1025
+ ```ts run
1026
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
1027
+ import { fold, type FlowDefinition, type FoldInput, type RegistryDocument } from 'fyflow-orchestration';
1028
+
1029
+ // A worker that FAILS — `examples/getting-started/flaky.ts`, a class with a `run` that throws. The
1030
+ // node is not `optional`, so when its retries are spent the scheduler parks it for a person.
1031
+ const registry: RegistryDocument = {
1032
+ workers: { flaky: { url: '../../examples/getting-started/flaky.ts' } },
1033
+ nodeTypes: { Flaky: { worker: 'flaky', inline: true } },
1034
+ };
1035
+ const flow: FlowDefinition = {
1036
+ flowId: 'review', version: 1, entry: 'score',
1037
+ nodes: [
1038
+ { id: 'score', kind: 'task', type: 'Flaky', settings: {}, edges: [
1039
+ { on: ['failed'], to: 'quarantine' },
1040
+ { to: 'done' },
1041
+ ] },
1042
+ { id: 'quarantine', kind: 'end', outcome: 'quarantined' },
1043
+ { id: 'done', kind: 'end', outcome: 'completed' },
1044
+ ],
1045
+ };
1046
+
1047
+ const runtime = createRuntime(registry, {}, { base: import.meta.url });
1048
+ const inputs: FoldInput[] = [];
1049
+ let parked: string | null = null;
1050
+
1051
+ // **The run does not end; it STOPS AND WAITS.** `awaiting_action` is the run's outcome and
1052
+ // `suspended` is the node's — and those are the words the FOLD uses. The scheduler's own event is
1053
+ // called `task.user_action`, which is not what you poll on, and is the mistake this snippet used to
1054
+ // invite by delivering that event to a hand-built stub.
1055
+ const { driver, done } = runtime.start(flow, {}, { onFoldInput: (input) => { inputs.push(input); } });
1056
+
1057
+ // **`done` IS STILL PENDING HERE, and that is the trap this section exists to teach.** Its
1058
+ // settlement rule is terminality and nothing else (ADR 0058 rule 3), and `isTerminal` is
1059
+ // `outcome !== 'running' && outcome !== 'awaiting_action'` — so a run parked on a person is not
1060
+ // terminal and `done` waits for the resolution. **A promise cannot tell you a run has PARKED**,
1061
+ // because parking is not the end; the poll below is for the state `done` is waiting THROUGH, and it
1062
+ // is the one thing here a completion handle does not give you.
1063
+ // **THE DEADLINE, ARMED FIRST — one line, because `done` IS a promise.** There is no library
1064
+ // default and no `StartOptions` entry for one (ADR 0058 rule 8): a default would be wrong for
1065
+ // exactly this run, where the honest wait is *as long as it takes a person*. What a hand-rolled
1066
+ // `new Promise` could not give whoever held it is a way to RACE it. **It is armed before the wait
1067
+ // and not after**, because what `done` is
1068
+ // waiting for is the decision below — a race started once the person has already decided is a race
1069
+ // whose other side has already lost, and it would demonstrate the shape while being unable to fire.
1070
+ let timer: ReturnType<typeof setTimeout> | undefined;
1071
+ const nobodyCame = new Promise<never>((_, reject) => {
1072
+ timer = setTimeout(() => reject(new Error('nobody resolved the suspended task in time')), 10_000);
1073
+ });
1074
+
1075
+ // THE PERSON, deciding while `done` is pending. Started, not awaited: it polls the fold for the
1076
+ // park — the state `done` waits THROUGH — and resolves the task, which is what settles `done`.
1077
+ const person = new Promise<void>((resolve, reject) => {
1078
+ const poll = () => {
1079
+ const s = fold(inputs);
1080
+ if (s.outcome !== 'awaiting_action') { setTimeout(poll, 5); return; }
1081
+ parked = Object.keys(s.nodes).find((id) => s.nodes[id].state === 'suspended') ?? null;
1082
+ if (parked === null) { reject(new Error('awaiting_action with no suspended node')); return; }
1083
+
1084
+ // ONE pair of events, ONE record. The scheduler dispatches `task.failed` AND
1085
+ // `task.user_action` carrying the same object; the driver drops the first, whose detail
1086
+ // already reads `user_action`, and records on the second. Read off a REAL pair, not a
1087
+ // hand-delivered one.
1088
+ const records = driver.snapshot().filter((r) => r.taskId === parked);
1089
+ if (records.length !== 1) { reject(new Error(`the event pair produced ${records.length} records`)); return; }
1090
+
1091
+ // The only way anything outside this process changes a run — and it takes THREE arguments.
1092
+ // `actor` is opaque: never interpreted, never authenticated (Q29).
1093
+ const accepted = driver.resolve(parked, 'failed', { actor: 'a-person' });
1094
+ const again = driver.resolve(parked, 'failed', { actor: 'a-person' });
1095
+ if (!accepted || again) { reject(new Error('a second resolution must be refused, not applied twice')); return; }
1096
+ resolve();
1097
+ };
1098
+ poll();
1099
+ });
1100
+
1101
+ // **`done` AND the person, against the clock.** Neither the poll nor the deadline folds anything or
1102
+ // decides what "over" means: the first is the state the run PARKS in and the second is your patience.
1103
+ //
1104
+ // **`person` is inside the race and not awaited after it**, which is two defects in one line and
1105
+ // both were real. Awaited afterwards it is UNBOUNDED — a run that reaches a terminal outcome without
1106
+ // ever parking leaves the poll looping for ever, which is the very shape this section teaches the
1107
+ // counter-measure for; and its three `reject`s reached a promise nothing had attached to yet, so
1108
+ // Node killed the process with `ERR_UNHANDLED_REJECTION` and the snippet failed as a CRASH rather
1109
+ // than as a throw. *Both found by W78's conformity reader, one line past the deadline written to
1110
+ // prevent the first.*
1111
+ const state = await Promise.race([Promise.all([done, person]).then(([s]) => s), nobodyCame]);
1112
+ clearTimeout(timer);
1113
+ await runtime.shutdown();
1114
+
1115
+ // `resolvedAs: 'failed'` took the `on: ['failed']` edge — the only road there for a non-optional
1116
+ // node — so the run ends at `quarantine`. The END node's `outcome` is a label of yours; the RUN's
1117
+ // outcome is the fold's, and it is `completed` because a path reached an end.
1118
+ const ended = Object.values(state.nodes).find((n) => n.nodeId === 'score');
1119
+ console.log(state.outcome, ended?.state, state.pathEnds.map((p) => p.at).join());
1120
+ // 'completed' 'failed' 'quarantine'
1121
+ // The node's own state is `failed`, not `resolved`: `resolvedAs` is what a person DECIDED,
1122
+ // and the node records the terminal state that decision gave it.
1123
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
1124
+ if (parked !== 'r1:score:-:0') throw new Error(`parked at ${parked}`);
1125
+ if (ended?.state !== 'failed') throw new Error(`the node settled ${ended?.state}`);
1126
+ // **WHICH END the run reached is what `resolvedAs` decided**, and until W71's adequacy reader
1127
+ // read this snippet nothing observed it: both roads fold to `completed`, so resolving 'done'
1128
+ // instead of 'failed' changed the road and no assertion moved. `pathEnds` carries the end each
1129
+ // path reached, which is the whole of the paragraph below.
1130
+ if (state.pathEnds.map((p) => p.at).join() !== 'quarantine') {
1131
+ throw new Error(`the run ended at ${state.pathEnds.map((p) => p.at).join()}, not at quarantine`);
1132
+ }
1133
+ ```
1134
+
1135
+ `resolve()` returns `false` if the task is not suspended — a rejected command, not a run failure, so
1136
+ you may retry. It does **one** thing: supply an outcome for a dispatch that already happened. It
1137
+ cannot re-run the node and it cannot send it somewhere else. It also returns `false` once the run has
1138
+ been **stopped** by a refusal (section 9), and there retrying can never succeed: check
1139
+ `driver.hasFailed()` to tell the two `false`s apart.
1140
+
1141
+ A resolved node is **indistinguishable** from one that settled that way: same node state, same
1142
+ `result`, same routing.
1143
+
1144
+ > **A suspended run never finishes on its own, and that is correct.** If you are waiting for a run to
1145
+ > reach a terminal outcome, `awaiting_action` is not one — see section 8.
1146
+
1147
+ ---
1148
+
1149
+ ## 8. Run state
1150
+
1151
+ **Run state is a pure fold of an ordered log.** The driver produces the sequence; `fold` turns it
1152
+ into state and touches nothing else.
1153
+
1154
+ ```ts run
1155
+ import { fold, isTerminal, type FoldInput } from 'fyflow-orchestration';
1156
+
1157
+ const settled = (taskId: string, state: 'done' | 'failed' | 'user_action'): FoldInput => ({
1158
+ kind: 'outcome',
1159
+ record: { kind: 'outcome', taskId, terminalState: state, result: null, error: null,
1160
+ attempts: 0, schedulerTimestamp: 1, observedAt: 1, startTime: 1, endTime: 2, executionTime: 1 },
1161
+ });
1162
+
1163
+ const dispatched = (taskId: string): FoldInput => ({ kind: 'dispatched', taskId });
1164
+ const ended = (reason: 'end' | 'no-eligible-edge'): FoldInput =>
1165
+ ({ kind: 'pathEnded', reason, at: 'x' });
1166
+ const openJoin: FoldInput =
1167
+ { kind: 'joinSettled', regionId: 'split', regionPath: '-', joinId: 'collect', fired: false };
1168
+
1169
+ // Each row adds ONE input to the row below it and must outrank it. Read downwards:
1170
+ // this is the precedence table, executed.
1171
+ const rows: [string, FoldInput[]][] = [
1172
+ ['completed', [dispatched('r1:a:-:0'), settled('r1:a:-:0', 'done'), ended('end')]],
1173
+ ['failed', [dispatched('r1:a:-:0'), settled('r1:a:-:0', 'done'), ended('end'),
1174
+ ended('no-eligible-edge')]],
1175
+ ['stuck', [dispatched('r1:a:-:0'), settled('r1:a:-:0', 'done'), ended('end'),
1176
+ ended('no-eligible-edge'), openJoin]],
1177
+ ['running', [dispatched('r1:a:-:0'), settled('r1:a:-:0', 'done'), ended('end'),
1178
+ ended('no-eligible-edge'), openJoin, dispatched('r1:b:-:0')]],
1179
+ ['awaiting_action', [dispatched('r1:a:-:0'), settled('r1:a:-:0', 'done'), ended('end'),
1180
+ ended('no-eligible-edge'), openJoin, dispatched('r1:b:-:0'),
1181
+ dispatched('r1:c:-:0'), settled('r1:c:-:0', 'user_action')]],
1182
+ ['stopped', [dispatched('r1:a:-:0'), settled('r1:a:-:0', 'done'), ended('end'),
1183
+ ended('no-eligible-edge'), openJoin, dispatched('r1:b:-:0'),
1184
+ dispatched('r1:c:-:0'), settled('r1:c:-:0', 'user_action'),
1185
+ { kind: 'stopped', reason: 'DuplicateTaskId: x' }]],
1186
+ ];
1187
+ for (const [expected, inputs] of rows) {
1188
+ const got = fold(inputs).outcome;
1189
+ console.log(expected, '->', got);
1190
+ if (got !== expected) throw new Error(`precedence: expected ${expected}, got ${got}`);
1191
+ }
1192
+
1193
+ // `isTerminal` is false for exactly `running` and `awaiting_action`.
1194
+ const terminal = Object.fromEntries(rows.map(([o, i]) => [o, isTerminal(fold(i))]));
1195
+ console.log(terminal); // completed/failed/stuck/stopped true, running/awaiting_action false
1196
+ if (!terminal.stuck) throw new Error('`stuck` is terminal — that is the point of it existing');
1197
+ if (terminal.running || terminal.awaiting_action) throw new Error('a live run is not terminal');
1198
+ if (!terminal.completed || !terminal.failed || !terminal.stopped) {
1199
+ throw new Error('completed, failed and stopped are all terminal');
1200
+ }
1201
+
1202
+ // `completed` means every live path reached an `end` — NOT "no node settled failed".
1203
+ // A failed node that routed on through an `on: ["failed"]` edge completes the run.
1204
+ const routedOn = fold([dispatched('r1:a:-:0'), settled('r1:a:-:0', 'failed'), ended('end')]);
1205
+ console.log(routedOn.outcome); // 'completed'
1206
+ if (routedOn.outcome !== 'completed') throw new Error('a failed node that routed on still completes');
1207
+ ```
1208
+
1209
+ In your own program you fold what the driver gives you:
1210
+
1211
+ ```ts check
1212
+ import { fold, isTerminal, type Driver } from 'fyflow-orchestration';
1213
+
1214
+ declare const driver: Driver;
1215
+ const state = fold(driver.foldInputs());
1216
+ if (isTerminal(state)) console.log('finished as', state.outcome);
1217
+ ```
1218
+
1219
+ ### The six outcomes, highest first
1220
+
1221
+ | | means |
1222
+ |---|---|
1223
+ | `stopped` | a refusal terminated the run — section 9 |
1224
+ | `awaiting_action` | a node is suspended and nobody has resolved it |
1225
+ | `running` | a dispatched node has not settled |
1226
+ | **`stuck`** | nothing is live and **nothing ever will be**: a join has not fired and cannot |
1227
+ | `failed` | a path ended with no eligible edge |
1228
+ | `completed` | every live path reached an `end` node |
1229
+
1230
+ **Read the outcome at a quiescent point** — after the driver's turn, not part-way through it. The
1231
+ fold is defined over any prefix, and a prefix taken mid-turn is not a resting state: a settlement is
1232
+ recorded before the routing it causes, so there are instants where nothing is running and the run is
1233
+ not over. If you fold `driver.foldInputs()` after the fact you are always at a quiescent point; if
1234
+ you fold incrementally from `onFoldInput`, read on a `queueMicrotask` — and that microtask is
1235
+ **load-bearing rather than decoration**, which is why `done` asks its own predicate at one
1236
+ ([ADR 0058](doc/decisions/0058-runtime-start-returns-a-completion-handle-the-runtime-keeps.md)
1237
+ rule 6). *(That phrase lived in §2's copied block until [W78](doc/work-items/w78-the-run-a-consumer-holds-in-one-line.md)
1238
+ reduced it to `await done`; ADR 0058's Context cites it, so it moved here rather than going out with
1239
+ the block — W78's conformity reader, finding 3.)*
1240
+
1241
+ **`isTerminal(state)` is how you wait for a run without a clock.** It is false for `running` and
1242
+ `awaiting_action` and true for everything else — including `stuck`, which is the point: before
1243
+ `stuck` existed, a permanently hung run folded to `completed` and a waiter reported success.
1244
+
1245
+ **And `done` is that rule applied for you.** `runtime.start` returns `{ driver, done, header }` —
1246
+ `header` is the run's CHECKPOINT header since
1247
+ [ADR 0064](doc/decisions/0064-a-checkpoint-is-the-whole-event-sequence-and-a-header-and-it-is-a-copy.md),
1248
+ and `driver.checkpoint()` hands out the header with every event so far (`spec/run-state.md`
1249
+ section 8); `runtime.resume(flow, checkpoint)` resumes a run from one, returning the same handle
1250
+ ([ADR 0065](doc/decisions/0065-a-resume-replays-the-stored-stimuli-refuses-what-it-cannot-regenerate-and-re-dispatches-what-was-in-flight.md), `spec/run-state.md` section 9 — section 18 teaches it, as an executed snippet) — and `done`
1251
+ resolves with the first `RunState` this predicate calls terminal
1252
+ ([ADR 0058](doc/decisions/0058-runtime-start-returns-a-completion-handle-the-runtime-keeps.md)). It
1253
+ is the same two functions — the runtime keeps one `RunState`, steps it per input and asks
1254
+ `isTerminal` at a quiescent point — so there is one definition of *over* with one more caller, and
1255
+ nothing above about reading at rest stops being true. **Three things follow, and two of them catch
1256
+ people:**
1257
+
1258
+ - **`done` never rejects.** `failed`, `stuck` and `stopped` are terminal OUTCOMES, not errors, so a
1259
+ run that goes wrong arrives as `state.outcome`. A driver refusal resolves it with `stopped` and
1260
+ `stoppedBy` naming the refusal — section 9.
1261
+ - **`awaiting_action` is NOT terminal, so `done` waits through it.** A run parked on a person stays
1262
+ pending until somebody calls `resolve()`, which is correct and is the trap: if you are waiting for
1263
+ a run and a human is in it, you are waiting for the human. Section 7 is the worked case.
1264
+ - **There is no deadline and there is no option for one**, because once `done` is a promise
1265
+ `Promise.race([done, timeout])` is your one line — and a library default would be wrong for the
1266
+ bullet above.
1267
+
1268
+ ---
1269
+
1270
+ ## 9. Refusals
1271
+
1272
+ Two kinds, and **the distinction is the stack, not the refusal class** (`OR-R10`). The same class can
1273
+ do either: whether you are on the stack when it is raised is what decides. A refusal raised while
1274
+ *your* call is running is thrown; one raised inside a scheduler event listener is reported, because
1275
+ `EventTarget.dispatchEvent` **discards a listener throw** and there is nobody to catch it.
1276
+
1277
+ **Thrown**, because you are on the stack and can act:
1278
+
1279
+ ```ts run
1280
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
1281
+ import { type FlowDefinition, type RegistryDocument } from 'fyflow-orchestration';
1282
+
1283
+ // **THROWN — raised while YOUR call is running.** A registry defect is refused by `createRuntime`,
1284
+ // on your stack, so a `try` catches it and names the entry.
1285
+ let thrown = '';
1286
+ try {
1287
+ createRuntime({
1288
+ workers: { scorer: { url: '../../examples/getting-started/scorer.ts' } },
1289
+ nodeTypes: { Scorer: { worker: 'typo' } }, // no such worker key
1290
+ } as RegistryDocument, {}, { base: import.meta.url });
1291
+ } catch (e) {
1292
+ thrown = (e as Error).constructor.name;
1293
+ }
1294
+ console.log(thrown); // 'WorkerMissing'
1295
+ if (thrown !== 'WorkerMissing') throw new Error(`construction refused with ${thrown}`);
1296
+
1297
+ // **REPORTED — raised inside a scheduler event listener.** Nobody is on the stack to catch it:
1298
+ // `EventTarget.dispatchEvent` DISCARDS a listener throw, so the driver reports it instead and the
1299
+ // run stops. This is the half a `try` around your own call can never see.
1300
+ const registry: RegistryDocument = {
1301
+ workers: { scorer: { url: '../../examples/getting-started/scorer.ts' } },
1302
+ nodeTypes: { Scorer: { worker: 'scorer', inline: true } },
1303
+ };
1304
+ const flow: FlowDefinition = {
1305
+ flowId: 'bad-arity', version: 1, entry: 'first',
1306
+ nodes: [
1307
+ { id: 'first', kind: 'task', type: 'Scorer', settings: { score: 0.9 }, edges: [{ to: 'each' }] },
1308
+ // `over` names a key of what `first` returned — and `score` is a NUMBER, not a list.
1309
+ { id: 'each', kind: 'map', over: 'score', branch: 'one' },
1310
+ { id: 'one', kind: 'task', type: 'Scorer', settings: { score: 1 }, edges: [{ to: 'join' }] },
1311
+ { id: 'join', kind: 'join', of: 'each', edges: [{ to: 'done' }] },
1312
+ { id: 'done', kind: 'end', outcome: 'completed' },
1313
+ ],
1314
+ };
1315
+
1316
+ const runtime = createRuntime(registry, {}, { base: import.meta.url });
1317
+ const reported = await new Promise<{ kind: string; message: string }>((resolve) => {
1318
+ // A DEADLINE resolving to a SENTINEL, so that *nothing was reported* is this snippet's own
1319
+ // failure rather than a hang. W71's adequacy reader made the driver swallow the refusal and
1320
+ // the snippet died as `Detected unsettled top-level await` — a crash is not a detection.
1321
+ const deadline = setTimeout(() => resolve({ kind: 'NOTHING WAS REPORTED', message: '' }), 10_000);
1322
+ runtime.start(flow, {}, { onRunFailure: (f) => { clearTimeout(deadline); resolve({ kind: f.kind, message: f.message }); } });
1323
+ });
1324
+ await runtime.shutdown();
1325
+
1326
+ console.log(reported.kind); // 'UnresolvedMapArity'
1327
+ if (reported.kind !== 'UnresolvedMapArity') throw new Error(`the run reported ${reported.kind}`);
1328
+ // Same library, same kind of mistake, two different roads out — and which road it takes is decided
1329
+ // by whether you were on the stack, not by the class (`OR-R10`).
1330
+ // *(The OTHER way this leg can go wrong — the refusal ESCAPING the listener instead of being
1331
+ // reported — takes the whole process down through `EventTarget`'s own rethrow, so it cannot be
1332
+ // this snippet's assertion whatever the snippet does. The deadline above closes the half that
1333
+ // can be closed: a refusal SWALLOWED in silence.)*
1334
+ // **`MissingJoinPolicy` is NOT reachable on this road**, and it is in the error-class enumeration below: a fan-out
1335
+ // with no policy is refused when YOU build a `Driver`, and `createRuntime` always supplies one
1336
+ // (`live-token` by default), so the runtime cannot raise it. A refusal a road cannot reach is
1337
+ // still a refusal — it belongs to the road in section 12, which is where a caller passes its own
1338
+ // scheduler and its own policy.
1339
+ ```
1340
+
1341
+ `InvalidDefinition` carries the violated rule ids on `.rules`, so you can act on them rather than
1342
+ parse a string. `InvalidRunId` is thrown by the **constructor** for a `runId` outside `[A-Za-z0-9_-]{1,64}`
1343
+ (`ID-R2`) — `:` is excluded because it is the task-id delimiter, and excluding it is what makes the
1344
+ task-id parse total.
1345
+
1346
+ **A defect in the REGISTRY's SHAPE is thrown at `new Driver` too, as a NAMED class extending
1347
+ `RegistryError`** — `spec/flow-definition.md` v0.19, section 13 here — each carrying `at`, the entry's
1348
+ key, and a message naming the entry. `new Driver` runs the shape half only, six classes:
1349
+ `WorkerMissing` (a type naming no worker entry, an entry with no `url`), `PoolTypeMissing`,
1350
+ `NameCollision` (a pool named as a type), `SchemaRefused` (a keyword outside the subset), and
1351
+ `UndeclaredKey` and `DerivationRefused` for a type's or a pool's field of the wrong shape. The other
1352
+ five — `ConfigRefused`, `TypeDefaultRefused`, `GroupTypeMissing`, `GroupConfigRefused`,
1353
+ `NameMissing` — need a derivation run or a group built, and are thrown by
1354
+ `resolveRegistry(document, code)` when a registry DOCUMENT is resolved against the code that
1355
+ supplies its predicates, languages, group types and node-type classes, by `createRuntime` (which
1356
+ calls that resolver first — section 12), and by the bootstrap helper before it builds a pool. The
1357
+ twelfth, **`GateTypeRefused`** (v0.23), is thrown by `checkRegistry` when a gate type it holds is not
1358
+ a `GateType` — the built-in `distinct` on every call, and any the registry adds in `gateTypes` — and a gate
1359
+ type named as a built-in is `NameCollision` — at `new Driver`, which since W93 takes yours as `gateTypes`, and
1360
+ at `createRuntime`, which takes them as `code.gateTypes` and hands them to every run (section 3). None is `InvalidDefinition`, on purpose: a registry defect is fixed in the registry and a flow defect in the
1361
+ flow, and the class says which document to open.
1362
+
1363
+ **And four the RUNTIME adds, all `RuntimeError` subclasses from `fyflow-orchestration/scheduler`**
1364
+ (W66): `GroupUnusable` — a group factory returned something the scheduler cannot use, naming the
1365
+ member and the group's key; `GroupNotBuildable` — a group type with no `create`; `LanguageNotBuilt` —
1366
+ `start` was given a flow that transforms on a type no pool was built for; and, since W89, `RunIdInUse`
1367
+ — a `start` or `resume` naming a run id that runtime has already built a driver for, a finished run's included (`RN-R25`, ADR 0091). A bare
1368
+ `RuntimeError` is the fifth road: a relative worker url with no `base`, and a predicate or adapter supplied that the
1369
+ document names nowhere. **They are not exported from the package ROOT**, so the enumeration below —
1370
+ which walks the root's own exports — does not see them, and `BootstrapError` has never been there
1371
+ either.
1372
+
1373
+ **And more that throw, because you called them.** Some are the same classes the next paragraph calls
1374
+ reported, which is why the rule is about the stack:
1375
+
1376
+ | | |
1377
+ |---|---|
1378
+ | `driver.start()` a **second time** | throws `MultipleRuns` — a driver serves one run and its id was fixed at construction (`ID-R3`) |
1379
+ | `driver.resolve()` with a `result` that is not structured-cloneable | throws the platform's own `DataCloneError`, **not** `UncloneableResult`, and the run is not stopped |
1380
+ | `driver.start()` with a payload that is not structured-cloneable | throws the platform's own `DataCloneError` with the driver UNSTARTED — a run whose entry payload cannot be copied into its checkpoint header cannot be checkpointed, so it is refused before it exists (`RN-R15`, [ADR 0064](doc/decisions/0064-a-checkpoint-is-the-whole-event-sequence-and-a-header-and-it-is-a-copy.md)). **New in W88, and BREAKING**: such a payload used to start a run on an inline pool |
1381
+ | `driver.checkpoint()` before `start()` | throws `NotStarted` — there is no run, and no entry payload to put in a header (`RN-R14`) |
1382
+ | `driver.resume(checkpoint)` or `runtime.resume(flow, checkpoint)` with an entry payload that is not structured-cloneable | throws the platform's `DataCloneError` before anything is replayed — the driver unstarted, and on a runtime no run id spent (`RN-R15`) |
1383
+ | `driver.checkpoint()` on a driver whose resume was refused | throws `NotStarted` — a refused resume leaves the driver SPENT and deaf to its scheduler, with no run to checkpoint (`RN-R23`) — **except** a resume refused `stopped` because its replay regenerated a stop in the lost tail: that driver keeps the STOPPED run, and its checkpoint is one a store may take. *(This row said `NotStarted` without the exception until W90 read it against `RN-R23`.)* |
1384
+ | `driver.resume(checkpoint)` or `runtime.resume(flow, checkpoint)` on a checkpoint it will not resume | throws `ResumeRefused`, whose `reason` is `format`, `run`, `flow`, `stopped` or `diverged` and whose `index` names the position for the last two — nothing is handed to a scheduler ([ADR 0065](doc/decisions/0065-a-resume-replays-the-stored-stimuli-refuses-what-it-cannot-regenerate-and-re-dispatches-what-was-in-flight.md)). `resume()` on a driver that already started or resumed throws `MultipleRuns` (`ID-R3`). **On a runtime, a refusal DURING the replay spends the run id there** — a driver was built for it, so that runtime answers the id with `RunIdInUse` from then on (ADR 0091 keeps the id after it releases the driver); one refused by its header spends nothing ([Q371](doc/questions/q371.md), section 18) |
1385
+
1386
+ `MultipleRuns` is thrown by a second `start()`: a driver's sequence and join state carry no run id,
1387
+ so a second run would silently merge into the first. **One driver, one run** — and the run id is a
1388
+ constructor option, so there is no second id to pass.
1389
+
1390
+ **A second driver on the same scheduler is the supported answer**, which is what `MultipleRuns`'
1391
+ message tells you to build — **under one condition.** Give each driver a **different** run id: two
1392
+ sharing one id mint the same task ids, so `dispatched.has()` is true in both, neither can see the
1393
+ collision, and one settlement dispatches the successor twice. Nothing enforces this and nothing can:
1394
+ a driver is handed two scheduler methods and neither asks who else is subscribed.
1395
+
1396
+ **The scheduler may carry work this library did not mint** — a nightly cleanup, another library, a
1397
+ worker's own `context.spawnTask()` children. A task id that does not parse was minted by no driver,
1398
+ so it is **not** any run's: the driver reports it through `onUnmintedTask` and drops it, and no run
1399
+ stops (`ID-R5`). *This used to be the second condition, and it stopped **every** driver on the
1400
+ scheduler; ADR 0027 is where it changed.*
1401
+
1402
+ ```ts check
1403
+ import { Driver, type UnmintedTask } from 'fyflow-orchestration';
1404
+
1405
+ declare const opts: ConstructorParameters<typeof Driver>[1];
1406
+ const watched = {
1407
+ ...opts,
1408
+ // Optional: without it the run still routes correctly, because a settlement that
1409
+ // parses as nothing was never yours. Register it if you do not control everything
1410
+ // on the scheduler — it is the only thing that would tell you the ids stopped
1411
+ // being the ones you dispatched.
1412
+ onUnmintedTask: (t: UnmintedTask) =>
1413
+ console.warn(`run ${t.runId}: ${t.event} for ${String(t.taskId)} — ${t.reason}`),
1414
+ };
1415
+ void watched;
1416
+ ```
1417
+
1418
+ **Do not let this callback throw.** It runs inside the driver's scheduler listener, so a throw from it
1419
+ is caught at that boundary and **stops your run** — under your own error class, for a task that was
1420
+ never yours, which is the exact damage the rule exists to remove. Log and return.
1421
+ **A settlement reaches only the driver that dispatched that task id.** One
1422
+ for another run is ignored, and one carrying **this** driver's run id for a task it never
1423
+ dispatched stops the run with `UndispatchedTask` (`ID-R4`). The second half is a refusal rather than
1424
+ a filter for `ID-R1`'s reason — within one run the driver mints every id it dispatches, so a
1425
+ settlement it did not mint means its bookkeeping is wrong.
1426
+
1427
+ **Reported, not thrown**, when there is nobody on the stack: `DuplicateTaskId`, `UncloneableResult`,
1428
+ `NoMatchingEdge` and `UndispatchedTask` raised *inside a scheduler event listener*. Give the driver
1429
+ `onRunFailure` or you will not hear about them — the run stops and folds to `stopped`, silently.
1430
+
1431
+ **`InvalidTaskId` is the ninth class and it is the one that does NOT stop a run.** It is thrown by
1432
+ `buildTaskId` and `parseTaskId`, both exported, and it is how you check an id yourself — *did some
1433
+ driver mint this?* is a `try`/`catch` around `parseTaskId`. Inside the driver's listener it is caught
1434
+ rather than reported: the settlement goes to `onUnmintedTask` and the run carries on (`ID-R5`), so
1435
+ **it never reaches `onRunFailure` and is never a `RunFailure.kind`.**
1436
+
1437
+ ```ts run
1438
+ import { InvalidTaskId, parseTaskId } from 'fyflow-orchestration';
1439
+
1440
+ let mine = false;
1441
+ let foreignFailedAsInvalidTaskId = false;
1442
+ try { parseTaskId('run-1:ingest:-:0'); mine = true; } catch { /* not ours */ }
1443
+ try { parseTaskId('nightly-cleanup'); } catch (e) { foreignFailedAsInvalidTaskId = e instanceof InvalidTaskId; }
1444
+ console.log(mine, foreignFailedAsInvalidTaskId);
1445
+ if (!mine) throw new Error('a four-segment id in the charset must parse');
1446
+ if (!foreignFailedAsInvalidTaskId) throw new Error('a foreign task id must fail, and as InvalidTaskId');
1447
+ ```
1448
+
1449
+ *Written without a rethrow on purpose.* The obvious form of this — a `mintedByADriver(id)` helper
1450
+ whose `catch` rethrows anything that is not an `InvalidTaskId` — is the shape the driver itself uses,
1451
+ and **W18 shipped it and then measured that its assertions could all be deleted with the gate
1452
+ green**: `check-docs`' `X4a` asks whether a `run` snippet contains a `throw`, and the helper's own
1453
+ rethrow answered yes on the assertions' behalf. The general weakness is
1454
+ [Q104](doc/questions/q104.md); here the snippet simply does not give `X4a` a `throw` that is not
1455
+ an assertion.
1456
+
1457
+ ```ts check
1458
+ import { Driver, type RunFailure } from 'fyflow-orchestration';
1459
+
1460
+ declare const opts: ConstructorParameters<typeof Driver>[1];
1461
+ const withReporting = {
1462
+ ...opts,
1463
+ onRunFailure: (f: RunFailure) => console.error(`run ${f.runId} stopped: ${f.kind} — ${f.message}`),
1464
+ };
1465
+ void withReporting;
1466
+ ```
1467
+
1468
+ `f.kind` is the **class name**, so these are the strings you switch on. Imported here rather than
1469
+ quoted in prose, so renaming one in `src/` fails this gate instead of silently falsifying the
1470
+ paragraphs above.
1471
+
1472
+ **Two of the classes belong to `map`, and they fail for different people.**
1473
+
1474
+ `UnresolvedBranchPath` is `ID-R6` — `branchPathOf` was asked for the path of a node inside a **`map`**
1475
+ region without the item index that region's branch is indexed by, and refused rather than guessing,
1476
+ because guessing mints one task id for every dispatch and the run then dies at `ID-R1` naming a
1477
+ duplicate. **A `Driver` supplies that index and never sees this**; it is what a caller composing
1478
+ `branchPathOf` or `branchIndexOf` themselves gets, and it names the map rather than the symptom.
1479
+
1480
+ `UnresolvedMapArity` is **W33's, and it is a data error rather than a definition error** — which is
1481
+ why it is not an `InvalidDefinition`. A `map` node's `over` is a key NAME (`FD-V25` checks the
1482
+ charset and can check nothing else, because no validator can see a value that does not exist yet),
1483
+ and at run time the value it names must be an **array**. An absent key, an object, a number, a `null`
1484
+ payload or a string payload each stop the run with this class naming the node and the key. **The
1485
+ definition is valid and the previous node did not produce what the flow says it does**, which is a
1486
+ different person's bug — a flow author reading `InvalidDefinition` would go and read their flow,
1487
+ where there is nothing to find. **An empty array is not this**: zero items is a lawful arity.
1488
+
1489
+ `UnresolvedReference` is **W41's**, and it is [ADR 0040](doc/decisions/0040-what-a-node-can-reach-is-the-runs-own-outputs-and-a-node-id-names-them-relative-to-a-branch-path.md)'s
1490
+ `FD-R3`. **A node OR AN EDGE** declared a `needs` naming another node — the edge site is
1491
+ [ADR 0041](doc/decisions/0041-a-decision-sees-its-nodes-outcome-and-whatever-its-own-edge-declares.md)'s
1492
+ and means **a predicate can now stop a run**, with the message saying `edge needs[…]` so you are sent
1493
+ to the edge — and **no output of that node is on this
1494
+ node's branch path** — a decision routed around it, or it settled `user_action` and nobody has
1495
+ resolved it. `FD-V27` has already checked at instantiation that the declaring node is *reachable
1496
+ from* the named one, which is everything the graph can say; whether it actually ran is a fact about
1497
+ this run. **It is a different class from `UnresolvedMapArity` deliberately**: that one says *the
1498
+ previous node did not return an array*, which is a **worker's** bug, and this says *the node you
1499
+ named did not run on your path*, which is the **flow's** shape.
1500
+
1501
+ `BoundaryRefused` is **W73's, and it is the one class here that STOPS NOTHING** — [P19](doc/plans/p19-what-a-schema-buys-an-author.md) decision 5's `FD-R4`. The two values no static check can reach are held to the workers' declared
1502
+ schemas ONCE each: the run's payload at `start()`, against the `inputSchema` of every task it reaches with no
1503
+ worker in between (the entry, or every branch task of an entry fan-out, or a map's branch once per item),
1504
+ and a person's `done` resolution at `resolve()`, against the parked node's worker `outputSchema`. Refused, it is
1505
+ **thrown at you as a REJECTED COMMAND**: nothing appended, nothing reported, the run unaffected — at `start`
1506
+ the driver is still UNSTARTED and a corrected payload starts it, at `resolve` the task is still suspended and
1507
+ a corrected value (the worker's declared example is one) resolves it. It carries `boundary`, `nodeId`,
1508
+ `taskId` and the schema's `findings`, and its message names the worker and every finding, so you read
1509
+ the VALUE and not the flow. A worker declaring no schema on that side is unchecked; `validateBoundaries: false`
1510
+ on `DriverOptions` or `RuntimeOptions` turns both off. **`FD-R4` is never per settlement**: it holds the run's PAYLOAD once and a person's RESOLUTION
1511
+ once, as rejected commands. What a WORKER returns is checked at authoring time by
1512
+ `npm run check:workers` and by the wiring check (section 13) — **and, since
1513
+ [ADR 0060](doc/decisions/0060-a-run-time-result-check-is-a-reading-and-never-a-refusal.md), by the
1514
+ RUN as well if you ask it to**: `OR-R13`, section 16, opt-in and a READING rather than a refusal.
1515
+ *(This read *"not by the run"*, which W80 falsified and which stood four sections above the section
1516
+ that teaches the opposite — W82's conformity reader, finding 2.)*
1517
+
1518
+ *(This paragraph read *"`UnresolvedBranchPath` is the tenth class, and it belongs to a feature nothing
1519
+ dispatches yet … no `Driver` opens one"* until W33, which is the item that made it run.
1520
+ [Q157](doc/questions/q157.md) is the gap behind both edits: no
1521
+ [P10](doc/plans/p10-is-fan-out-one-feature-or-two.md) item owns this file, and W35 is the natural
1522
+ owner of describing the feature rather than merely not lying about it.)*
1523
+
1524
+ **And the list is checked against the PACKAGE, not against itself** — [Q101](doc/questions/q101.md).
1525
+ Until W18 this snippet imported eight classes, mapped them to `.name`, and compared the result to a
1526
+ literal it also contained. That is a closed list agreeing with a copy of itself: it was executed by
1527
+ `npm run gate`, it **omitted `InvalidTaskId`** — exported from the package root the whole time — and
1528
+ it passed. *Rot is what the gate prevents; a list that never looks outside itself is a third thing the
1529
+ gate cannot see.* The second half below is what makes adding a refusal class in `src/` and not
1530
+ mentioning it here a **failure** rather than an omission nobody can observe:
1531
+
1532
+ ```ts run
1533
+ import * as lib from 'fyflow-orchestration';
1534
+ import {
1535
+ BoundaryRefused,
1536
+ ConfigRefused, DerivationRefused, DuplicateTaskId, GateTypeRefused, GroupConfigRefused, GroupTypeMissing, InvalidDefinition,
1537
+ InvalidRunId, InvalidTaskId, MissingJoinPolicy, MultipleRuns, NameCollision, NameMissing, NoMatchingEdge, NotStarted,
1538
+ PoolTypeMissing, RegistryError, ResumeRefused, SchemaRefused, TypeDefaultRefused, UncloneableResult, UndeclaredKey,
1539
+ UndeclaredRoute, UndispatchedTask, UnresolvedBranchPath, UnresolvedMapArity, UnresolvedReference, WorkerMissing,
1540
+ } from 'fyflow-orchestration';
1541
+
1542
+ const documented = [
1543
+ DuplicateTaskId, UncloneableResult, NoMatchingEdge, UndispatchedTask, // reported inside a listener
1544
+ MultipleRuns, InvalidRunId, InvalidDefinition, MissingJoinPolicy, // thrown at you
1545
+ NotStarted, // thrown at you from checkpoint() before start() — there is no run yet (RN-R14)
1546
+ ResumeRefused, // thrown at you from resume() — a checkpoint it will not resume, `reason` says why (RN-R23)
1547
+ RegistryError, WorkerMissing, PoolTypeMissing, ConfigRefused, TypeDefaultRefused, GroupTypeMissing, // thrown at you — a REGISTRY defect,
1548
+ GroupConfigRefused, NameCollision, SchemaRefused, UndeclaredKey, NameMissing, DerivationRefused, // the base class and its twelve (section 13)
1549
+ GateTypeRefused, // a registered GATE type that is not a GateType — checkRegistry (v0.23)
1550
+ UnresolvedMapArity, UnresolvedReference, // reported in a listener; reported AND thrown from start()/resolve()
1551
+ UndeclaredRoute, // the same two roads — a GATE's type returned a route it does not declare (FD-R5, W93)
1552
+ BoundaryRefused, // thrown at you from start()/resolve() with the run UNTOUCHED — a rejected command (FD-R4)
1553
+ InvalidTaskId, UnresolvedBranchPath, // neither — see above
1554
+ ].map((c) => c.name);
1555
+
1556
+ // Every Error subclass the package root exports, discovered rather than restated.
1557
+ const exported = Object.entries(lib)
1558
+ .filter(([, v]) => typeof v === 'function' && Error.isPrototypeOf(v as object))
1559
+ .map(([name]) => name);
1560
+
1561
+ const missing = exported.filter((n) => !documented.includes(n));
1562
+ const stale = documented.filter((n) => !exported.includes(n));
1563
+ console.log(documented.length, exported.length);
1564
+ if (missing.length) throw new Error(`section 9 does not mention: ${missing.join(' ')}`);
1565
+ if (stale.length) throw new Error(`section 9 names classes the package no longer exports: ${stale.join(' ')}`);
1566
+ ```
1567
+
1568
+ After a refusal the run is **stopped**: nothing routes for it again, and its state folds to
1569
+ `stopped` whatever its nodes say.
1570
+
1571
+ ---
1572
+
1573
+ ## 10. Common mistakes
1574
+
1575
+ - **Expecting a failure to route by default.** It does not. Add `on: ['failed']`, or accept that the
1576
+ path ends — section 6.
1577
+ - **Using `optional: false` to mean "important".** It means *"a human decides"*, and it produces a
1578
+ suspended run rather than a failed one.
1579
+ - **Passing `arrivalCount`.** It hangs on an ordinary `optional` failure. Section 5.
1580
+ - **Waiting on `scheduler.completed`.** It fires **every time the scheduler drains**, which with a
1581
+ flow is between every step. It is not "the run finished". `await done` — the second half of what
1582
+ `runtime.start` returns (section 8).
1583
+ - **Expecting `await done` to reject when a run goes wrong.** It resolves, always: `failed`, `stuck`
1584
+ and `stopped` are outcomes you read off the state, and `stoppedBy` names a refusal. A `.catch` on
1585
+ `done` is a `.catch` that never fires.
1586
+ - **Waiting on `done` for a run with a person in it.** `awaiting_action` is not terminal, so `done`
1587
+ is still pending — by design. Poll the fold for the park, resolve the task, then await; section 7
1588
+ is that snippet, and `Promise.race([done, timeout])` is the deadline nothing ships for you.
1589
+ - **Reading run state mid-turn.** Section 8.
1590
+ - **Retaining a scheduler event's `detail`.** It is the live task object and it mutates — including
1591
+ its `timestamp`, so a retained detail reports the *most recent* event's time. `{...e.detail}` is
1592
+ **not** compliance: it still aliases `result` and `error`. The driver copies at the listener
1593
+ boundary, so the four values that carry the run — records from `snapshot()`, inputs from
1594
+ `foldInputs()` and `onFoldInput`, and events from `onRegionEvent` — are **frozen copies**, safe to
1595
+ hold. *Until [W76](doc/work-items/w76-what-start-accepts.md) the fourth of those was reachable
1596
+ only by building a `Driver` by hand, so this bullet named four values and the snippet below could
1597
+ assert three; `createRuntime` forwards `onRegionEvent` now, and all four are on one road.*
1598
+ **The other accessors return copies that are not frozen** (`dispatchedIds()`,
1599
+ `regionsForPolicies()`, `effectiveOutcomeOf()`, `terminalStateOf()`): writing to one changes
1600
+ nothing in the driver, but it is not a document to edit and hand back.
1601
+ - **Returning a CLASS INSTANCE from a worker.** It does not throw and it does not arrive intact.
1602
+ `structuredClone` copies the fields and drops the prototype, so the next node receives a plain
1603
+ object: `instanceof` is false, the methods are gone, and **nothing is reported** — `OR-R11` in
1604
+ `spec/outcome-record.md` section 4. **Return data, read data.** The loud half is a result that
1605
+ cannot be cloned at all (a function, a `Promise`, a `WeakMap`), which on an **inline** pool stops
1606
+ the run with `UncloneableResult`, and on a **thread** pool is refused first by the scheduler's
1607
+ own clone, so the value never reaches the driver, `UncloneableResult` never fires and
1608
+ `onRunFailure` is not called — the task ends as any failure does: a non-optional task out of
1609
+ retries ends `user_action` and parks the run `awaiting_action`, and an `optional` one ends
1610
+ `failed` and routes as a failure (`OR-R6`; the node type's `inline` decides which pool); the class
1611
+ instance is the quiet half, and it is the one people hit.
1612
+ - **Assuming `joinStatusOf().describe` is yours.** It is whatever **your** policy's `describe()`
1613
+ returned. The two shipped policies return a fresh object; a policy that returns its own state hands
1614
+ you that state, and writing to it changes what the join decides. Return a copy from `describe()`
1615
+ — [Q82](doc/questions/q82.md).
1616
+
1617
+ The first half of that is executed rather than asserted:
1618
+
1619
+ ```ts run
1620
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
1621
+ import { type FlowDefinition, type FoldInput, type RegistryDocument } from 'fyflow-orchestration';
1622
+
1623
+ const registry: RegistryDocument = {
1624
+ workers: { scorer: { url: '../../examples/getting-started/scorer.ts' } },
1625
+ nodeTypes: { Scorer: { worker: 'scorer', inline: true } },
1626
+ };
1627
+ const flow: FlowDefinition = {
1628
+ flowId: 'frozen', version: 1, entry: 'a',
1629
+ nodes: [
1630
+ { id: 'a', kind: 'task', type: 'Scorer', settings: { score: 0.1 }, edges: [{ to: 'split' }] },
1631
+ { id: 'split', kind: 'fanout', branches: ['b', 'c'] },
1632
+ { id: 'b', kind: 'task', type: 'Scorer', settings: { score: 0.2 }, edges: [{ to: 'j' }] },
1633
+ { id: 'c', kind: 'task', type: 'Scorer', settings: { score: 0.3 }, edges: [{ to: 'j' }] },
1634
+ { id: 'j', kind: 'join', of: 'split', edges: [{ to: 'e' }] },
1635
+ { id: 'e', kind: 'end', outcome: 'completed' },
1636
+ ],
1637
+ };
1638
+
1639
+ const runtime = createRuntime(registry, {}, { base: import.meta.url });
1640
+ const inputs: FoldInput[] = [];
1641
+ const frozen: Record<string, boolean> = {};
1642
+ // **The handles this road HAS, named rather than counted** — a count is not a membership, and
1643
+ // re-sourcing one of these onto another left a count of three unmoved (W71's adequacy reader).
1644
+ const HANDLES = ['onFoldInput', 'onRegionEvent', 'snapshot', 'foldInputs'];
1645
+
1646
+ // **`onRegionEvent` is FORWARDED, so this road can watch a fan-out open and close** — W76. It was
1647
+ // not, until W76: `StartOptions` is a `Pick` of the driver's options, and of the driver's twenty
1648
+ // (eighteen when W76 wrote this; `onNonconformingResult` is ADR 0060's, W80's; `gateTypes` ADR 0067's, W93's)
1649
+ // this was the one no `createRuntime` consumer could reach at all, which is
1650
+ // [Q297](doc/questions/q297.md). **The previous version of this snippet asserted the opposite and
1651
+ // passed a CAST**, because that was the only way to try; there is no cast now, and the two counts
1652
+ // below are what changed. *The reason it went unnoticed for so long is that the snippet which
1653
+ // asserted it built its own `Driver`, and a reference that constructs its own driver can
1654
+ // demonstrate options its readers have no way to pass.*
1655
+ //
1656
+ // **What a consumer gets is the driver's own event, already frozen** (`OR-R7`,
1657
+ // [Q73](doc/questions/q73.md)) — nothing is copied or re-frozen at the runtime seam, which is why
1658
+ // the freeze is asserted here on the road a consumer actually uses rather than inferred from the
1659
+ // driver's own test.
1660
+ let regionEvents = 0;
1661
+ const regionKinds: string[] = [];
1662
+ const { driver, done } = runtime.start(flow, {}, {
1663
+ onFoldInput: (i: FoldInput) => { frozen.onFoldInput = Object.isFrozen(i); inputs.push(i); },
1664
+ onRegionEvent: (e) => {
1665
+ regionEvents += 1;
1666
+ regionKinds.push(e.type);
1667
+ frozen.onRegionEvent = (frozen.onRegionEvent ?? true) && Object.isFrozen(e);
1668
+ },
1669
+ });
1670
+
1671
+ // **The end, and its deadline, in two lines** — `await done` is the wait (ADR 0058) and
1672
+ // `Promise.race` is the deadline (rule 8). `onFoldInput` above is still here because THIS snippet
1673
+ // is about what the four handles hand out, and one of the four IS that callback.
1674
+ let timer: ReturnType<typeof setTimeout> | undefined;
1675
+ const tooLong = new Promise<never>((_, reject) => {
1676
+ timer = setTimeout(() => reject(new Error('the run never reached a terminal state')), 10_000);
1677
+ });
1678
+ await Promise.race([done, tooLong]);
1679
+ clearTimeout(timer);
1680
+ await runtime.shutdown();
1681
+
1682
+ // The two PULL handles, read after the run — and each must have handed back something, because
1683
+ // `Object.isFrozen([])` is true for free and an empty answer would pass while asserting nothing.
1684
+ const snapshot = driver.snapshot();
1685
+ const foldInputs = driver.foldInputs();
1686
+ if (snapshot.length === 0 || foldInputs.length === 0) throw new Error('a pull handle returned nothing, so freezing it says nothing');
1687
+ frozen.snapshot = Object.isFrozen(snapshot) && snapshot.every(Object.isFrozen);
1688
+ frozen.foldInputs = Object.isFrozen(foldInputs) && foldInputs.every(Object.isFrozen);
1689
+
1690
+ console.log(frozen, regionEvents, [...new Set(regionKinds)].sort());
1691
+ // all four handles true, then 4, then [ 'branch.arrived', 'branch.opened' ]
1692
+
1693
+ // **Where each claim below is observed a SECOND time**, because every guard in an executed snippet
1694
+ // is bound by its own PRESENCE and by nothing else ([Q310](doc/questions/q310.md)) — W76's adequacy
1695
+ // reader made all six guards and all four `isFrozen` readings vacuous at once with `check:docs` at
1696
+ // exit 0. The four freeze claims are re-asserted per element in `test/driver/outbound.test.ts` and
1697
+ // `test/driver/fold-inputs.test.ts`; the region-event count, kinds and freeze ACROSS THE RUNTIME
1698
+ // SEAM — which is the part those files cannot see, because they build a `Driver` by hand — are in
1699
+ // `test/scheduler/runtime.test.ts`, where the forwarded stream is held equal to the driver's own
1700
+ // region stream element for element. *Nothing here is the only observer of anything.*
1701
+ for (const [name, isFrozen] of Object.entries(frozen)) {
1702
+ if (!isFrozen) throw new Error(`${name} handed out something you can write to`);
1703
+ }
1704
+ // **The guard that fires if the answer changes BACK, and it is FIRST on purpose.** Q297 was a
1705
+ // member of `StartOptions` going missing, which nothing noticed for two plans; this is what would
1706
+ // notice now, from a consumer's side of the seam rather than from the type. *It sat BELOW the
1707
+ // membership check in the first version and was unreachable: `frozen.onRegionEvent` is written in
1708
+ // the same callback as `regionEvents`, so a regression removes the key too and the membership
1709
+ // throws first — a regression caught, by a message about `HANDLES` that says nothing about Q297.
1710
+ // W76's conformity reader; the tree's own "right verdict by the wrong route".*
1711
+ if (regionEvents === 0) {
1712
+ throw new Error('onRegionEvent fired 0 times: the runtime has stopped forwarding it, and a consumer on this road cannot watch a fan-out again — Q297 has regressed');
1713
+ }
1714
+ if (Object.keys(frozen).sort().join() !== HANDLES.slice().sort().join()) {
1715
+ throw new Error(`expected exactly ${HANDLES.join()}, saw ${Object.keys(frozen).sort().join()}`);
1716
+ }
1717
+ // The KINDS are the membership and the count stands beside them: a two-branch fan-out opens twice
1718
+ // and arrives twice, and a snippet asserting only `> 0` would pass on one branch of one kind.
1719
+ if ([...new Set(regionKinds)].sort().join() !== 'branch.arrived,branch.opened') {
1720
+ throw new Error(`expected a fan-out's opens and arrivals, saw ${[...new Set(regionKinds)].sort().join() || '(nothing)'}`);
1721
+ }
1722
+ if (regionEvents !== 4) throw new Error(`expected 4 region events on a two-branch fan-out — two opened, two arrived — and saw ${regionEvents}`);
1723
+ ```
1724
+ - **Subscribing to the scheduler yourself and counting `task.failed`.** A `user_action` task fires
1725
+ it *and* `task.user_action`. Key on the terminal state — section 7.
1726
+ - **Sharing one `Driver` between runs.** `start()` is called at most once and a second throws
1727
+ `MultipleRuns` — section 9. *Sharing one **scheduler** between `Driver`s is fine*, and used to be
1728
+ the other half of this bullet: a settlement now reaches only the driver that dispatched it.
1729
+
1730
+ ---
1731
+
1732
+ **`OR-R11`, executed rather than asserted** — a worker returns a class with a method, and what
1733
+ the driver logs is a plain object with the field and no method:
1734
+
1735
+ ```ts run
1736
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
1737
+ import { type FlowDefinition, type RegistryDocument } from 'fyflow-orchestration';
1738
+
1739
+ // `classy` returns `new Score(0.9)` — a class with a method, and it CHECKS that it does before
1740
+ // handing it over, so the thing being lost below is known to have existed. `inspect` is the NEXT
1741
+ // NODE, and reports what arrived. `uncloneable` returns a function, which is the loud half.
1742
+ const registry: RegistryDocument = {
1743
+ workers: {
1744
+ classy: { url: '../../examples/getting-started/classy.ts' },
1745
+ inspect: { url: '../../examples/getting-started/inspect.ts' },
1746
+ uncloneable: { url: '../../examples/getting-started/uncloneable.ts' },
1747
+ },
1748
+ nodeTypes: {
1749
+ Classy: { worker: 'classy', inline: true },
1750
+ Inspect: { worker: 'inspect', inline: true },
1751
+ Uncloneable: { worker: 'uncloneable', inline: true },
1752
+ },
1753
+ };
1754
+ const lossy: FlowDefinition = {
1755
+ flowId: 'or-r11', version: 1, entry: 'a',
1756
+ nodes: [
1757
+ { id: 'a', kind: 'task', type: 'Classy', settings: { value: 0.9 }, edges: [{ to: 'next' }] },
1758
+ { id: 'next', kind: 'task', type: 'Inspect', settings: {}, edges: [{ to: 'done' }] },
1759
+ { id: 'done', kind: 'end', outcome: 'completed' },
1760
+ ],
1761
+ };
1762
+ const loud: FlowDefinition = {
1763
+ flowId: 'uncloneable', version: 1, entry: 'a',
1764
+ nodes: [
1765
+ { id: 'a', kind: 'task', type: 'Uncloneable', settings: {}, edges: [{ to: 'done' }] },
1766
+ { id: 'done', kind: 'end', outcome: 'completed' },
1767
+ ],
1768
+ };
1769
+
1770
+ const runtime = createRuntime(registry, {}, { base: import.meta.url });
1771
+ const failures: string[] = [];
1772
+ // `onRunFailure` is what the LOUD half below is read from; `done` is the end. They are two
1773
+ // different facts about the same run, which is why both are here.
1774
+ const run = (flow: FlowDefinition) =>
1775
+ runtime.start(flow, {}, { onRunFailure: (f) => { failures.push(f.kind); } }).done;
1776
+
1777
+ // **THE QUIET HALF.** The fields cross; the behaviour does not; nothing is reported.
1778
+ const quiet = await run(lossy);
1779
+ const arrived = Object.values(quiet.nodes).find((n) => n.nodeId === 'next')?.result as
1780
+ { sawMethod?: string; prototypeIs?: string; keys?: string[] };
1781
+ console.log(arrived.sawMethod, arrived.prototypeIs, JSON.stringify(arrived.keys));
1782
+ // 'undefined' 'Object.prototype' ["value"]
1783
+ if (arrived.keys?.join() !== 'value') throw new Error(`the next node received ${JSON.stringify(arrived.keys)}`);
1784
+ if (arrived.sawMethod !== 'undefined') throw new Error('the BEHAVIOUR reached the next node, which OR-R11 says it cannot');
1785
+ if (arrived.prototypeIs !== 'Object.prototype') throw new Error('the PROTOTYPE survived the boundary');
1786
+ if (quiet.outcome !== 'completed') throw new Error(`the lossy run ${quiet.outcome}`);
1787
+ if (failures.length !== 0) throw new Error(`the quiet half reported ${failures.join()} — OR-R11 is silent by design`);
1788
+
1789
+ // **THE LOUD HALF, and it is what makes the line above mean anything.** A result that cannot be
1790
+ // cloned at ALL stops the run and IS reported — so `failures` has a case that fills it, and
1791
+ // `failures.length === 0` above is a measurement rather than a sentence about an empty array.
1792
+ //
1793
+ // **And `done` RESOLVES here rather than rejecting** (ADR 0058 rules 3 and 4): a refusal is a
1794
+ // terminal OUTCOME, not an error, so there is no `.catch` — the run arrives as `stopped` with
1795
+ // `stoppedBy` naming the refusal. One settlement rule for every way a run can end.
1796
+ const stoppedRun = await run(loud);
1797
+ await runtime.shutdown();
1798
+ console.log(failures.join(), stoppedRun.outcome, stoppedRun.stoppedBy?.split(':')[0]);
1799
+ // 'UncloneableResult' 'stopped' 'UncloneableResult'
1800
+ if (failures.join() !== 'UncloneableResult') throw new Error(`the loud half reported ${failures.join() || 'nothing'}`);
1801
+ if (stoppedRun.outcome !== 'stopped') throw new Error(`a refused run resolved \`done\` with ${stoppedRun.outcome}, not stopped`);
1802
+ if (!stoppedRun.stoppedBy?.startsWith('UncloneableResult')) throw new Error(`stoppedBy: ${stoppedRun.stoppedBy}`);
1803
+ ```
1804
+
1805
+ ## 11. What does NOT exist — do not write code using it
1806
+
1807
+ | not available | instead |
1808
+ |---|---|
1809
+ | flow-level **retry** — re-dispatching a NODE under a new attempt, possibly routing elsewhere | **task-level retry IS built** since [W58](doc/work-items/w58-what-a-node-says-about-its-own-task.md): declare `retryPolicy` on the node (section 3) and the scheduler retries inside the same task id, reported on the record's `attempts`. Re-dispatching the node is [Q16](doc/questions/q16.md)'s other half and the task id's fourth segment, and nothing mints one |
1810
+ | **cancellation** | nothing. The scheduler cannot cancel a queued task either |
1811
+ | **durability** — resuming a run after a restart | nothing. The fold is designed to make it additive; it is not built |
1812
+ | **nested fan-out** — a branch that fans out again | **built, and the join over it is settled.** `FD-V15` is retired; a task inside a nested region is dispatched at its whole dotted `branchPath` (`r1:scoreC:1.0:0`); and every policy in section 5 is **transitive** — each region answers exactly as it would if it were not nested ([ADR 0030](doc/decisions/0030-a-join-is-transitive-and-a-dead-join-ends-the-branch-it-sits-in.md)). One thing to know: if an inner join is reported through `onUnsatisfiableJoin`, the outer branch containing it **ends** rather than waiting, so the outer join may still fire and its successor may run — and once that work settles the run folds to `stuck`, because a region that never fired is what `stuck` reads. **Fold too early and you will see `running`.** The four matrix rows and the join seam are tested at **depth 2**; the dead-join cascade is asserted at depth 3, and nothing else about depth 3 is a tested claim |
1813
+ | ~~**map-style fan-out**~~ — **BUILT** since [W33](doc/work-items/w33-a-map-at-run-time-arity.md); see section 3 |
1814
+ | a **join timeout** | nothing. A stuck join is detected by the `stuck` outcome, with no clock |
1815
+ | **per-join policy choice** | one policy per driver |
1816
+ | a **declared event family** for `fyflow-react-components` | `onRegionEvent`, `onFoldInput` and `joinStatusOf()` — see below |
1817
+
1818
+ **On that last row.** Flow structure is invisible to the *scheduler*, so a UI wired only to a
1819
+ scheduler cannot show which node a run is at, which branch it took, or what a join is waiting on.
1820
+ That is [Q21](doc/questions/q21.md), and it is about the scheduler's stream, not about this
1821
+ library — **this library does expose all three.** `onFoldInput` is the push half of `foldInputs()`,
1822
+ so you can fold incrementally and see a run finish without polling; `onRegionEvent` reports branches
1823
+ opening, arriving, exiting, suspending and resuming; `joinStatusOf()` answers what a join is waiting
1824
+ on. What is missing is an agreed envelope shape between the two repositories, not the data.
1825
+
1826
+ ---
1827
+
1828
+ ## 12. Wiring it to a real scheduler
1829
+
1830
+ **The short way is a registry DOCUMENT and one call** — `createRuntime` in
1831
+ `fyflow-orchestration/scheduler`, [W66](doc/work-items/w66-the-runtime-entry.md). It resolves every
1832
+ name the document uses against the code you supply, refuses every defect at construction by a named
1833
+ error, builds one pool per type instance and one object per group, holds a `FyflowScheduler` over
1834
+ them, and hands you **`{ driver, done, header }` per run** — the driver already running, `done` a
1835
+ promise that resolves with the run's terminal `RunState`, and `header` the run's checkpoint header
1836
+ ([ADR 0058](doc/decisions/0058-runtime-start-returns-a-completion-handle-the-runtime-keeps.md);
1837
+ section 8 is what it waits for). **What stays yours is exactly what is code**: predicates,
1838
+ language adapters, custom group types and custom node-type classes, in one `code` argument.
1839
+
1840
+ ```ts run
1841
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
1842
+ import { type FlowDefinition, type RegistryDocument } from 'fyflow-orchestration';
1843
+
1844
+ // The registry, as data. In your program this is a JSON file you import; here it is a literal so the
1845
+ // snippet is self-contained. A worker url may be RELATIVE, and then `base` is what it resolves
1846
+ // against — pass `import.meta.url`, because inside a bundle the library cannot guess it.
1847
+ const registry: RegistryDocument = {
1848
+ workers: {
1849
+ ingest: { url: '../../examples/workers/ingest.ts' },
1850
+ publish: { url: '../../examples/workers/publish.ts' },
1851
+ },
1852
+ groups: { gate: { type: 'ConcurrentLimitGroup', config: { limit: 1 } } },
1853
+ nodeTypes: {
1854
+ Ingest: { worker: 'ingest', inline: true },
1855
+ Publish: { worker: 'publish', inline: true, groups: ['gate'] },
1856
+ },
1857
+ };
1858
+
1859
+ // The flow, as data. It names registry KEYS and nothing else — no file, no URL, no class.
1860
+ const flow: FlowDefinition = {
1861
+ flowId: 'wired', version: 1, entry: 'ingest',
1862
+ nodes: [
1863
+ { id: 'ingest', kind: 'task', type: 'Ingest', edges: [{ to: 'publish' }] },
1864
+ { id: 'publish', kind: 'task', type: 'Publish', edges: [{ to: 'done' }] },
1865
+ { id: 'done', kind: 'end', outcome: 'completed' },
1866
+ ],
1867
+ };
1868
+
1869
+ const runtime = createRuntime(registry, {}, { base: import.meta.url });
1870
+
1871
+ // A run is over when its OWN fold says so — never `scheduler.completed`, which fires every time the
1872
+ // scheduler drains (section 8). `await done` IS that, in one line: the runtime keeps the fold and
1873
+ // resolves with the terminal state (ADR 0058). This is the one snippet here that wants BOTH halves
1874
+ // of what `start` returns, and the reason is the line under it.
1875
+ const { driver, done } = runtime.start(flow, { confidence: 0.9 });
1876
+ // **`driver.runId` is readable** (W66): the runtime minted this run's id, and every task id of the
1877
+ // run begins with it. Pass your own as `start`'s `runId` when you need to choose it. It is readable
1878
+ // the instant `start` returns and does not wait for `done`, which is what holding the PAIR buys
1879
+ // over holding the promise alone.
1880
+ if (!/^r\d+$/.test(driver.runId)) throw new Error(`expected a minted run id, got ${driver.runId}`);
1881
+ const state = await done;
1882
+ const pools = Object.keys(runtime.pools).sort();
1883
+ await runtime.shutdown();
1884
+
1885
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
1886
+ // One pool per type instance, keyed by the name the document uses — that is what a node's `type` names
1887
+ // — **and one more for every BUILT-IN node type**, which is `Transform` and nothing else today
1888
+ // (section 13). The built-ins come LAST, so adding one never renumbers yours; a `WorkerManager` that
1889
+ // never receives a task spawns no thread, measured at W109.
1890
+ if (pools.join() !== 'Ingest,Publish,Transform') throw new Error(`pools: ${pools.join()}`);
1891
+ // And the group the document declared was built and is the scheduler's, keyed as declared.
1892
+ if (Object.keys(runtime.groups).join() !== 'gate') throw new Error(`groups: ${Object.keys(runtime.groups).join()}`);
1893
+ ```
1894
+
1895
+ **A custom group type is a FACTORY, and the runtime holds its output to what the scheduler CALLS.**
1896
+ The document names a group by KEY with a `type`; the code supplies `{ configSchema?, create(config,
1897
+ key) }` under that type name; the three built-ins are pre-registered in the same shape. What comes
1898
+ back is checked for ADR 0055's six members and refused by name otherwise. Read the counts off the
1899
+ object the runtime built — [W70](doc/work-items/w70-a-flow-that-decides-and-a-custom-group-on-the-runtime.md):
1900
+
1901
+ ```ts run
1902
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
1903
+ import { type FlowDefinition, type RegistryDocument } from 'fyflow-orchestration';
1904
+ import type { ResourceGroup } from 'fyflow-scheduler';
1905
+
1906
+ // A CUSTOM GROUP is the scheduler's own `ResourceGroup`, and the runtime holds a factory's output to
1907
+ // the six members the scheduler CALLS or a viewer READS — `canRun`, `onStart`, `onFinish`,
1908
+ // `getMetrics`, `id`, `type` — refusing a missing one at construction, by name (ADR 0055: the
1909
+ // interface marks two of those optional and the scheduler calls them unguarded, which is S10
1910
+ // upstream). This one counts what it sees. The constructor is written long-hand on purpose: a
1911
+ // parameter property is TypeScript Node 24 refuses to LOAD (RT-R9).
1912
+ class CountingGroup extends EventTarget implements ResourceGroup {
1913
+ readonly type = 'Counting';
1914
+ readonly id: string;
1915
+ private readonly limit: number;
1916
+ private active = 0;
1917
+ started = 0;
1918
+ finished = 0;
1919
+ constructor(id: string, limit: number) { super(); this.id = id; this.limit = limit; }
1920
+ canRun(): boolean { return this.active < this.limit; }
1921
+ onStart(): void { this.active += 1; this.started += 1; }
1922
+ onFinish(): void { this.active -= 1; this.finished += 1; }
1923
+ getMetrics(): { limit: number; running: number; available: number; utilization: number } {
1924
+ return { limit: this.limit, running: this.active, available: this.limit - this.active, utilization: this.active / this.limit };
1925
+ }
1926
+ }
1927
+
1928
+ // THE REGISTRY declares the group by KEY, with its TYPE by name and its config; the CODE supplies the
1929
+ // factory for that type name — `{ configSchema?, create(config, key) }`. The three built-ins,
1930
+ // `ConcurrentLimitGroup`, `RateLimitGroup` and `KeyedRateLimitGroup`, are pre-registered in the same
1931
+ // shape, so `"type": "ConcurrentLimitGroup"` needs no code at all. A node type names the groups its
1932
+ // pool's tasks acquire.
1933
+ const registry: RegistryDocument = {
1934
+ workers: { greeter: { url: '../../examples/getting-started/worker.ts' } },
1935
+ groups: { budget: { type: 'Counting', config: { limit: 1 } } },
1936
+ nodeTypes: { Metered: { worker: 'greeter', inline: true, groups: ['budget'] } },
1937
+ };
1938
+ const flow: FlowDefinition = {
1939
+ flowId: 'metered', version: 1, entry: 'greet',
1940
+ nodes: [
1941
+ { id: 'greet', kind: 'task', type: 'Metered', settings: { say: 'world' }, edges: [{ to: 'shout' }] },
1942
+ { id: 'shout', kind: 'task', type: 'Metered', settings: { say: '!' }, edges: [{ to: 'done' }] },
1943
+ { id: 'done', kind: 'end', outcome: 'completed' },
1944
+ ],
1945
+ };
1946
+ const runtime = createRuntime(registry, {
1947
+ groupTypes: {
1948
+ Counting: {
1949
+ configSchema: { type: 'object', required: ['limit'], properties: { limit: { type: 'integer', minimum: 1 } } },
1950
+ create: (config, key) => new CountingGroup(key, (config as { limit: number }).limit),
1951
+ },
1952
+ },
1953
+ }, { base: import.meta.url });
1954
+
1955
+ const state = await runtime.start(flow, { text: 'hello' }).done;
1956
+ await runtime.shutdown();
1957
+
1958
+ // The group the runtime built is the scheduler's, keyed as the document declared it — and it was
1959
+ // acquired and released once per ADMISSION of a task of the type that names it. Twice here, because
1960
+ // these two tasks are admitted once each; a retry, or a threaded pool that finds no free worker,
1961
+ // re-admits, so a budget is spent per admission and NOT per task (section 13).
1962
+ const budget = runtime.groups['budget'] as CountingGroup;
1963
+ // THREE observers, because `runtime.groups['budget']` is the KEY the document filed it under and
1964
+ // says nothing about the object: the counters the group kept, the metrics IT reports — `running`
1965
+ // back at zero is `onFinish` having run, which is the member ADR 0055 exists for — and its own
1966
+ // `id` and `type`, which a factory ignoring the key it is handed would get wrong in silence.
1967
+ const metrics = budget.getMetrics();
1968
+ console.log(state.outcome, budget.started, budget.finished, metrics.running, budget.id); // 'completed' 2 2 0 'budget'
1969
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
1970
+ if (budget.started !== 2 || budget.finished !== 2) throw new Error(`the group saw ${budget.started} starts and ${budget.finished} finishes`);
1971
+ if (metrics.running !== 0 || metrics.available !== 1) throw new Error(`the group reports ${metrics.running} running, ${metrics.available} available`);
1972
+ if (budget.id !== 'budget' || budget.type !== 'Counting') throw new Error(`the group calls itself ${budget.id} / ${budget.type}`);
1973
+ ```
1974
+
1975
+ **Every refusal is at construction and named.** A worker a type names and the document does not
1976
+ declare, a pool whose type does not exist, a config that fails its schema, a schema keyword outside
1977
+ the built-in subset, a name declared twice — each is a `RegistryError` subclass naming the entry
1978
+ (section 9, section 13). Three more are the runtime's own: a **relative worker url with no `base`**,
1979
+ a group type with no factory, and a group factory returning an object the scheduler cannot use —
1980
+ `canRun`, `onStart`, `onFinish`, `getMetrics`, `id` and `type`, each refused by name. *(That list is
1981
+ wider than the scheduler's own interface marks required, and deliberately: measured at 0.5.2, a group
1982
+ with no `onStart` throws on its first task and one with no `onFinish` leaves every task naming it
1983
+ running for ever, while one with no `getMetrics` runs fine —
1984
+ [ADR 0055](doc/decisions/0055-a-group-factorys-output-is-held-to-what-the-scheduler-calls.md).)*
1985
+
1986
+ **If a flow transforms, tell the runtime about it.** A language adapter's worker half is written
1987
+ into a pool's config at construction, and only for the types whose nodes transform (section 13), so
1988
+ pass every flow the runtime will run: `createRuntime(registry, code, { base, flows: [flow] })`. A
1989
+ flow handed to `start` that transforms on a type no pool was built for is refused there, naming the
1990
+ type and the language, rather than failing every task of that node one run in.
1991
+
1992
+ **Thousands of runs at once: start them all on ONE runtime.** Every run `start` (or `resume`) opens
1993
+ shares the runtime's pools and its one scheduler, and the runtime ROUTES each settlement to the driver
1994
+ holding its run id — [ADR 0088](doc/decisions/0088-a-runtime-routes-each-settlement-to-the-driver-holding-its-run-id.md).
1995
+ A driver is handed its own run's settlements and nothing of any other run's, so N concurrent runs cost
1996
+ one delivery per settlement and the topology is **linear in N**; before it, every settlement reached
1997
+ every driver and one runtime cost N² (`examples/retention.ts topology N` measures either topology, one runtime or
1998
+ `own` per run, and ADR 0088 carries both figures). Two things are
1999
+ not routed, and both are yours to size:
2000
+
2001
+ - **A `Driver` you build by hand** on `runtime.scheduler` or on your own `FyflowScheduler` (the long
2002
+ way, below) subscribes to the scheduler itself and hears every settlement on it, dropping the ones
2003
+ that are not its run's (`ID-R4`). That is correct, and N of them on one scheduler is N² deliveries.
2004
+ For a handful of runs it does not matter; for thousands, go through `start`.
2005
+ - **A listener of your own** on `runtime.scheduler` — a dashboard, a logger — hears every event of
2006
+ every run. Key it on `detail.id` and do O(1) work per event, or N such listeners are N² again.
2007
+
2008
+ A task id that does not parse — work this library did not mint — still reaches EVERY live run's
2009
+ driver, which each report it through `onUnmintedTask` (section 9): nobody owns it, so everybody says so.
2010
+
2011
+ **A runtime keeps a run until its `done` settles, and then only its run id** —
2012
+ [ADR 0091](doc/decisions/0091-a-runtime-releases-a-run-when-its-done-settles-and-keeps-only-its-run-id.md).
2013
+ At that moment the runtime drops the run's driver from its routing, so one long-lived runtime's memory
2014
+ is bounded by its LIVE runs plus one id string per run it ever opened. A run that is `running` or
2015
+ parked `awaiting_action` is kept — `resolve()` works as before. **The runtime has no way to reach a
2016
+ past run, so a postmortem is what YOU keep**: the `RunHandle` (`driver.foldInputs()`, `snapshot()`,
2017
+ `checkpoint()`, the `RunState` `done` resolved with), or what your `onFoldInput` stored. Drop the
2018
+ handle and the run is gone. After `done`, a released run is handed nothing — not its own run id, not
2019
+ an unminted one — so a `stopped` run's tasks still in flight settle into no driver; listen on
2020
+ `runtime.scheduler` yourself if you want their results. **The id stays in use**: `start` or `resume`
2021
+ naming it on the same runtime throws `RunIdInUse`, because those late settlements carry it. And the
2022
+ SCHEDULER keeps every completed task in `scheduler.tasks` unless `maxCompletedTasks` caps it (its own
2023
+ `AGENTS.md`); `createRuntime` builds it uncapped, so pass `createScheduler` to cap it.
2024
+
2025
+ **The long way, and it is the same objects.** You can build the scheduler yourself and hand the
2026
+ `Driver` the registry sets — which is what the runtime does, and what ladder L2-L5 here do with a
2027
+ double. You supply the scheduler, and that is all you have to supply. Since
2028
+ [ADR 0048](doc/decisions/0048-the-scheduler-is-a-declared-dependency-reached-by-one-bare-specifier.md)
2029
+ this library **depends on `fyflow-scheduler`** and says so where your installer reads it — a
2030
+ `peerDependencies` entry, at the pin stated at the end of this section — and `createTask` is
2031
+ **optional**: the driver's default is `(config) => new FyflowTask(config)`, the object a
2032
+ `FyflowScheduler` consumes.
2033
+
2034
+ ```ts run
2035
+ import { Driver, fold, isTerminal, liveToken, type FlowDefinition, type FoldInput, type RunState } from 'fyflow-orchestration';
2036
+ import { FyflowScheduler, FyflowTask, WorkerManager } from 'fyflow-scheduler';
2037
+
2038
+ // Your worker modules — here two this repository's own L6 rung runs, so this snippet is a real run
2039
+ // on a real scheduler rather than a sketch. **One module per worker**: a module's default export is
2040
+ // the class a `WorkerManager` instantiates, which is the scheduler's own contract.
2041
+ const ingestUrl = new URL('../../examples/workers/ingest.ts', import.meta.url).href;
2042
+ const publishUrl = new URL('../../examples/workers/publish.ts', import.meta.url).href;
2043
+
2044
+ const definition: FlowDefinition = {
2045
+ flowId: 'wired', version: 1, entry: 'ingest',
2046
+ nodes: [
2047
+ { id: 'ingest', kind: 'task', type: 'IngestWorker', edges: [{ to: 'publish' }] },
2048
+ { id: 'publish', kind: 'task', type: 'PublishWorker', edges: [{ to: 'done' }] },
2049
+ { id: 'done', kind: 'end', outcome: 'completed' },
2050
+ ],
2051
+ };
2052
+
2053
+ // The registry: one worker entry, and one node type per pool — the pools are built BY HAND here,
2054
+ // keyed by the type's name, which is what `addTask` routes on. The bootstrap helper builds pools
2055
+ // FROM entries like these (section 13) — envelope-fronted, the worker loaded by URL with its config
2056
+ // under `config.config` — where these two load the worker directly with `{ role }`; the helper's
2057
+ // call is the one a consumer copies, and this snippet keeps the raw scheduler in view.
2058
+ const registry = {
2059
+ workers: { ingest: { url: ingestUrl }, publish: { url: publishUrl } },
2060
+ nodeTypes: { IngestWorker: { worker: 'ingest' }, PublishWorker: { worker: 'publish' } },
2061
+ };
2062
+ const scheduler = new FyflowScheduler({
2063
+ IngestWorker: new WorkerManager(ingestUrl, { maxThreads: 1, inline: true }),
2064
+ PublishWorker: new WorkerManager(publishUrl, { maxThreads: 1, inline: true }),
2065
+ });
2066
+
2067
+ // A run is over when its OWN fold says so — never `scheduler.completed`, which fires every
2068
+ // time the scheduler drains (section 8). `onFoldInput` is the push half of `foldInputs()`, and
2069
+ // the outcome is read on a `queueMicrotask`, AFTER the driver's turn: a prefix cut mid-turn is
2070
+ // not a resting state, and section 8 says so.
2071
+ const inputs: FoldInput[] = [];
2072
+ const state = await new Promise<RunState>((resolve, reject) => {
2073
+ const driver = new Driver(definition, {
2074
+ runId: 'run-1',
2075
+ scheduler, // the real class, as it is — no cast (Q71)
2076
+ predicates: new Map(),
2077
+ ...registry, // the types the flow names, and their workers (section 13)
2078
+ groups: Object.keys(scheduler.groups), // the resource groups a node may name (section 3)
2079
+ joinPolicy: liveToken,
2080
+ onFoldInput: (input) => {
2081
+ inputs.push(input);
2082
+ queueMicrotask(() => { const s = fold(inputs); if (isTerminal(s)) resolve(s); });
2083
+ },
2084
+ onRunFailure: (f) => reject(new Error(`${f.kind}: ${f.message}`)),
2085
+ });
2086
+ driver.start({ confidence: 0.9 });
2087
+ });
2088
+ const tasks = [...scheduler.tasks.values()]; // read BEFORE shutdown, which clears the map
2089
+ await scheduler.shutdown();
2090
+
2091
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
2092
+ // `nodes` is keyed by TASK id (`run-1:publish:-:0`), so look the node up by its id.
2093
+ const publish = Object.values(state.nodes).find((n) => n.nodeId === 'publish');
2094
+ if (publish?.state !== 'done') throw new Error(`publish never settled: ${publish?.state}`);
2095
+ // And the tasks the scheduler ran were the class's own — the driver built them, you did not.
2096
+ // The scheduler would have run a plain object too (it duck-types its task, with `attempts` left
2097
+ // undefined), which is why this is asserted rather than assumed.
2098
+ if (tasks.length !== 2 || !tasks.every((t) => t instanceof FyflowTask)) throw new Error('the default createTask did not build FyflowTasks');
2099
+ ```
2100
+
2101
+ `nodeTypes` and `workers` are the sets the validator checks a definition against, and **a type's
2102
+ pool is keyed by the type's name** — so the object you hand the driver is the object you build the
2103
+ pools from, as above, rather than a second list; every task the driver dispatches carries the
2104
+ envelope `{ settings, input }` (section 13), which is what this repository's worker reads `input`
2105
+ from. Pass `createTask` only to run against a
2106
+ double — ladder L2-L5 here hand the driver `(c) => c` and read the plain object back — or to build
2107
+ something other than a plain `FyflowTask`.
2108
+
2109
+ **The `scheduler` option takes a `FyflowScheduler` as it is.** Until W57 it did not, under the one
2110
+ configuration most consumers have: `FyflowScheduler extends EventTarget`, so with the DOM lib in
2111
+ scope — TypeScript's default when `lib` is unset — its `addEventListener` was not assignable to
2112
+ `SchedulerLike`'s, and this section told you to write `scheduler as unknown as SchedulerLike`. That
2113
+ was [Q71](doc/questions/q71.md), a defect in our type and not in yours, and it is closed:
2114
+ `SchedulerLike`'s listener parameter is `object`, which is what makes the real class satisfy it,
2115
+ and every snippet in this document is typechecked under the DOM lib by `npm run check:docs`.
2116
+
2117
+ **A worker file can be TypeScript.** Node 24 runs it by stripping types, and Deno natively, so **one
2118
+ `.ts` worker serves Node and Deno**. *(The scheduler's own reference **used to say** a Node worker
2119
+ must be JavaScript, which was stale at Node 24 — [S9](doc/scheduler-upstream.md), raised 2026-09-07
2120
+ and **fixed in 0.5.2**. Its §3 now version-qualifies: a `.ts` worker works on **Node >= 22.18**, and
2121
+ below that it does not, so the old advice is right for that range and wrong for ours. This paragraph
2122
+ called the reference stale for nine days and then for one more, in the document above the sentence
2123
+ `D7` was built around.)* **Not the browser**: nothing there executes TypeScript, so
2124
+ a browser build needs the worker emitted as a bundled `.js` asset. `RT-R10` — ladder L6 runs in Node
2125
+ and Deno and not in the browser, and the browser rung is [Q52](doc/questions/q52.md).
2126
+
2127
+ **Pin the scheduler at 0.5.2.** This library's listener boundary is written against **0.3.0
2128
+ semantics, which neither 0.5.1 nor 0.5.2 changes**: `task.progress`'s detail changed shape at 0.3.0 (the id is
2129
+ `id`, not `taskId`), the second `task.user_action` dispatch site arrived there, and `task.timestamp`
2130
+ did not exist before it. **`core/FyflowScheduler.ts` is byte-identical between 0.3.0, 0.5.1 and 0.5.2**, so
2131
+ every one of those still holds; what 0.5.1 added is a `worker.setup.*` / `worker.teardown.*` event
2132
+ family this library subscribes to not at all. Install it from a registry — npm and JSR both carry it —
2133
+ and **never from a working tree**, which is
2134
+ [ADR 0029](doc/decisions/0029-the-scheduler-is-a-registry-dependency-and-nothing-reaches-the-working-tree.md)
2135
+ superseding [ADR 0013](doc/decisions/0013-the-scheduler-is-a-development-dependency-reached-from-the-working-tree.md)
2136
+ rules 3 and 5. **The manifests say the same thing**, since
2137
+ [ADR 0048](doc/decisions/0048-the-scheduler-is-a-declared-dependency-reached-by-one-bare-specifier.md):
2138
+ `package.json` names it in `peerDependencies` at exactly that version and `deno.json` maps it to
2139
+ `jsr:@fyflow/scheduler` at exactly that version, and `npm run check:scheduler`'s `D8` holds both to
2140
+ the pin the same way `D7` holds this sentence. *(This read **"pin at 0.3.0 … npm and JSR both stop at 0.2.1, so 0.3.0 is reached from
2141
+ a working tree"** — a live instruction to a consumer, wrong in the version AND in the mechanism, the
2142
+ second having been superseded on 2026-08-31. W36's conformity reader.)*
2143
+
2144
+ The worked flows on a real scheduler are in [`examples/`](examples/README.md), which counts them —
2145
+ including a routed-away branch, a lost branch, a human resolving a suspension, a refusal that stops
2146
+ a run, and a join's successor reducing the region's record through a declared transform. *(This
2147
+ sentence read **"Thirteen worked flows"** from the day it was written until W63, over a directory
2148
+ that had held twenty-two for two plans — a count in prose that no check reads, so it carries none
2149
+ now.)*
2150
+
2151
+ ### The driver with no scheduler at all — the shape ladder L2 uses
2152
+
2153
+ *This was section 2, "Minimal working program", until
2154
+ [W69](doc/work-items/w69-a-getting-started-example-a-consumer-can-copy.md) — and since P18 it is the
2155
+ HARDER road, so it moved here byte for byte. It is still the one demonstration of a `Driver` with
2156
+ nothing underneath it: a fake scheduler in twelve lines, which is what ladder L2 runs against and
2157
+ what a unit test of your own routing wants.*
2158
+
2159
+ A flow is **data**. You supply the predicates and the task factory; the driver does the rest.
2160
+
2161
+ ```ts run
2162
+ import { Driver, fold, liveToken, type FlowDefinition, type Predicate } from 'fyflow-orchestration';
2163
+
2164
+ const definition: FlowDefinition = {
2165
+ flowId: 'greet',
2166
+ version: 1,
2167
+ entry: 'ingest',
2168
+ nodes: [
2169
+ { id: 'ingest', kind: 'task', type: 'IngestWorker', edges: [{ to: 'publish' }] },
2170
+ { id: 'publish', kind: 'task', type: 'PublishWorker', edges: [{ to: 'done' }] },
2171
+ { id: 'done', kind: 'end', outcome: 'completed' },
2172
+ ],
2173
+ };
2174
+
2175
+ // A fake scheduler, so this snippet runs with no workers. In your program this is
2176
+ // `new FyflowScheduler({ ... })` — see section 12.
2177
+ //
2178
+ // **Key the listeners by event type.** The driver subscribes to three of them and
2179
+ // normalizes on the task's terminal STATE rather than the event name (`OR-R3`), so a
2180
+ // double that fans one settlement to every listener writes THREE records for it and
2181
+ // the run stops on a `DuplicateTaskId`. A real `EventTarget` does not do this; a
2182
+ // hand-written double is where it happens.
2183
+ const listeners = new Map<string, ((e: { detail: unknown }) => void)[]>();
2184
+ const added: { id: string }[] = [];
2185
+ const failures: string[] = [];
2186
+ const scheduler = {
2187
+ addTask(task: unknown) { added.push(task as { id: string }); },
2188
+ addEventListener(type: string, fn: (e: { detail: unknown }) => void) {
2189
+ listeners.set(type, [...(listeners.get(type) ?? []), fn]);
2190
+ },
2191
+ };
2192
+ const settle = (detail: unknown, type = 'task.completed') => {
2193
+ for (const fn of listeners.get(type) ?? []) fn({ detail });
2194
+ };
2195
+
2196
+ const driver = new Driver(definition, {
2197
+ runId: 'run-1',
2198
+ scheduler,
2199
+ createTask: (config) => config, // your `new FyflowTask(config)`
2200
+ predicates: new Map<string, Predicate>(),
2201
+ // Since v0.19 a task names a node TYPE and a type names a WORKER entry — section 13. The
2202
+ // validator and the driver require the entries to be DECLARED and load nothing; the URL is what
2203
+ // the bootstrap helper hands a real pool, so a fake scheduler's registry can carry any string.
2204
+ workers: { flow: { url: 'worker.js' } },
2205
+ nodeTypes: { IngestWorker: { worker: 'flow' }, PublishWorker: { worker: 'flow' } },
2206
+ joinPolicy: liveToken,
2207
+ // Section 9: a refusal raised inside a listener is REPORTED, never thrown.
2208
+ // Without this the run stops silently.
2209
+ onRunFailure: (f) => { failures.push(f.kind); },
2210
+ });
2211
+
2212
+ driver.start({ name: 'world' });
2213
+ console.log(added.map((t) => t.id)); // [ 'run-1:ingest:-:0' ]
2214
+
2215
+ // The scheduler reports a settlement; the driver routes.
2216
+ settle({ id: 'run-1:ingest:-:0', state: 'done', result: { greeting: 'hello world' } });
2217
+ settle({ id: 'run-1:publish:-:0', state: 'done', result: null });
2218
+ console.log(added.map((t) => t.id)); // [ 'run-1:ingest:-:0', 'run-1:publish:-:0' ]
2219
+
2220
+ const state = fold(driver.foldInputs());
2221
+ console.log(state.outcome); // 'completed'
2222
+
2223
+ if (added.map((t) => t.id).join() !== 'run-1:ingest:-:0,run-1:publish:-:0') {
2224
+ throw new Error(`wrong dispatches: ${added.map((t) => t.id).join()}`);
2225
+ }
2226
+ if (failures.length) throw new Error(`the run reported ${failures.join(', ')}`);
2227
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
2228
+ ```
2229
+
2230
+ **Three things bite everyone.** The **run id is a constructor option**, not a `start()` argument,
2231
+ and it must match `[A-Za-z0-9_-]{1,64}`. A definition with a fan-out and no `joinPolicy` is
2232
+ **refused at construction**. A driver serves **exactly one run**, so `start()` is called at most
2233
+ once — **and a second driver on the same scheduler is fine**, because a settlement only affects the
2234
+ driver that dispatched it. Built by hand, each driver still HEARS every settlement and drops the
2235
+ others; through `runtime.start` it is only handed its own (ADR 0088, above). See section 9.
2236
+
2237
+ ---
2238
+
2239
+ ## 13. Node types — a registry DOCUMENT entry, a derivation to a pool, and the envelope
2240
+
2241
+ **Every `task` names a node type**, and may carry per-node **`settings`** and a **`pool`** —
2242
+ `spec/flow-definition.md` v0.19, [ADR 0049](doc/decisions/0049-a-node-type-is-a-registry-entry-and-its-worker-receives-an-envelope.md),
2243
+ [ADR 0054](doc/decisions/0054-every-task-node-names-a-type-and-a-type-is-a-derivation-to-pool-options.md),
2244
+ [W59](doc/work-items/w59-what-a-node-type-is.md), [W65](doc/work-items/w65-the-registry-document-and-a-node-type-is-a-derivation.md).
2245
+ **A node type is a REGISTRY entry and the definition stays data**: the registry is a DOCUMENT —
2246
+ `workers` (a module URL and the four JSON schemas that worker owns), `nodeTypes` (each naming a
2247
+ worker, with its pool's options, its groups, a config, a retry default and `settings` DEFAULTS),
2248
+ `pools` (named instances of a type), `groups` (built through group-type factories), plus the
2249
+ `predicates` and `languages` it names and your code supplies. **A type is a DERIVATION from an
2250
+ instance's config to pool options**; the document form is the identity derivation — what it
2251
+ declares is what its default pool is built from — and a class registered in code is any other, both
2252
+ passing the same checks. **Every type has a default pool, keyed by its name**; the flow names the
2253
+ type and nothing else — never a file, never a URL. *(Until v0.19 a task could name a `workerType`
2254
+ instead — a pool key, its worker handed the raw payload. That road is retired: `workerType` is an
2255
+ undeclared key, and a worker written for it reads under `input` what it read as its whole payload.)*
2256
+
2257
+ | | | |
2258
+ |---|---|---|
2259
+ | `type` | **required** | the name of an entry in `nodeTypes` whose `worker` the registry's `workers` declares (`FD-V32`, both clauses) — **or a BUILT-IN type, which no document declares and which names no worker** (`Transform`, below). A task with no `type` is `FD-V23` |
2260
+ | `pool` | optional | the name of an instance in the registry's `pools` whose `type` is this node's (`FD-V38`); absent, the type's DEFAULT instance. The type's own name is that default, stated explicitly |
2261
+ | `settings` | optional | a plain JSON object which — with the type's `settings` defaults merged UNDER it — the worker's **`settingsSchema`** accepts (`FD-V33`); **absent means the defaults alone**, `{}` when the type declares none, and the schema sees that. A schema is JSON Schema in the built-in subset (ADR 0053): a keyword outside it is refused at construction, never ignored |
2262
+ | `transform` | optional | `{ lang, expr }`: an expression, in a language the registry's **`languages`** set names, applied to the node's input BEFORE its worker sees it — below. `lang` unregistered is `FD-V35`, `expr` the language refuses is `FD-V36`, not `{ lang, expr }` is `FD-V37` |
2263
+
2264
+ ### The one node type the library ships: a node that only TRANSFORMS
2265
+
2266
+ **`Transform` is a built-in node type, it names NO worker, and its task is its own `transform`** —
2267
+ `spec/flow-definition.md` v0.28,
2268
+ [ADR 0075](doc/decisions/0075-a-node-that-only-transforms-is-a-built-in-type-with-no-inner-worker.md),
2269
+ [W109](doc/work-items/w109-a-node-that-only-transforms.md). Write `{ "kind": "task", "type":
2270
+ "Transform", "transform": { "lang": …, "expr": … } }` and the node reshapes its input with no worker
2271
+ module, no worker entry and no node type of yours. **It is merged at every READ and stored in no
2272
+ registry**, so a flow may name it against a registry that declares no `nodeTypes` at all — which is what
2273
+ lets a document a tool EMITS carry a reshaping step without carrying code for it. Name your own type
2274
+ `Transform` and the registry refuses it by name, on every road: a built-in is never replaced. **It names no LANGUAGE**: `lang` is the
2275
+ node's, exactly as it is on any `transform`, so the adapter stays yours and no engine ships with the
2276
+ library.
2277
+
2278
+ **Use it last, and the reason is a cost.** A computation belongs at the cheapest tier that can hold it:
2279
+
2280
+ | tier | where it goes | what it costs |
2281
+ |---|---|---|
2282
+ | **selection** — *I need a value from another node* | **`needs`** on the consuming node (section 3) | **no task at all** |
2283
+ | **computation feeding a worker** — *reshape what THIS worker reads* | **`transform`** on that worker's own node | **no extra task** |
2284
+ | **computation with no consuming node** — *reshape for a join, a decision, an `end`* | **a `Transform` node** | **a real task**: measured at **+1 dispatch, +4 clones and +2 log records** per node, and inside a `map`, N of them |
2285
+
2286
+ Measured on a real `FyflowScheduler` at W109: a `map` whose branch is a `Transform` node feeding a
2287
+ worker runs **1.93x** the same `map` with the transform on the worker's own node, inline at N = 128,
2288
+ and a flat **~1.47x** threaded. *A reshaping task node is what
2289
+ [ADR 0040](doc/decisions/0040-what-a-node-can-reach-is-the-runs-own-outputs-and-a-node-id-names-them-relative-to-a-branch-path.md)
2290
+ and [ADR 0051](doc/decisions/0051-a-transform-is-a-declared-expression-in-a-registered-language-applied-in-the-types-worker-wrapper.md)
2291
+ exist to remove, so this is an escape hatch rather than a default — the ADR says so in those words.*
2292
+
2293
+ **Three things to know before you use one.** A node of this type with **no `transform` is refused at
2294
+ construction** by `FD-V46`, because the alternative is a task that returns what it was given. It
2295
+ **takes `settings`**, bound as `$settings` in the expression exactly as on any transform, so one type is
2296
+ reusable across flows. And **every arc LEAVING it is `unchecked`**: it declares no `outputSchema`
2297
+ because it has no worker, so section 15's authoring feedback cannot judge what comes next. What you get
2298
+ instead is a SAMPLE — `sampledOutput(registry, flow, nodeId)` returns the shape the node emitted on the
2299
+ producer's own example, the sample it came from, and the departing verdicts, which it returns rather
2300
+ than hides because a sample is evidence about one input and `fits` is a claim about every input
2301
+ ([Q446](doc/questions/q446.md)).
2302
+
2303
+ **What EVERY task's worker receives is the ENVELOPE `{ settings, input }`** — `FD-C5`. `settings`
2304
+ is the node's effective settings — the merged object — as its own copy; **`input` is the value this
2305
+ node receives** — the previous node's result, the object `needs` describes, its own item as a map
2306
+ instance, the collection after a join. Write a worker to read `payload.settings` and
2307
+ `payload.input`, and nothing else changes about it.
2308
+
2309
+ ```ts run
2310
+ import { validateDefinition, type FlowDefinition, type NodeTypeDeclaration, type Registry, type WorkerEntry } from 'fyflow-orchestration';
2311
+
2312
+ // The WORKER entries: a module URL, and the schemas the worker owns. `settingsSchema` is what
2313
+ // FD-V33 applies to every node of EVERY type running that entry — over the node's settings with
2314
+ // the type's defaults merged under, `{}` when neither says anything — so a schema that REQUIRES a
2315
+ // key refuses absence by name, and a type whose nodes carry no settings is written over an entry
2316
+ // that declares none (two entries here, one module each — since W67 a worker is a module). `configSchema` is held against every pool's
2317
+ // derived config at construction; `inputSchema` and `outputSchema` are declared and carried, enforced on ONE
2318
+ // value per run each (the entry payload at start(), a person's resolution at resolve() — FD-R4, W73)
2319
+ // and never per settlement — since v0.20 (ADR 0056) any schema may carry `examples`, held to it at
2320
+ // construction, `npm run check:workers` runs a worker on its input example (P19, W72), and since v0.21
2321
+ // `checkWiring(registry, flow)` gives every arc of a flow a verdict from the two (W73).
2322
+ const ingestUrl = new URL('../../examples/workers/ingest.ts', import.meta.url).href;
2323
+ const typedUrl = new URL('../../examples/workers/typed.ts', import.meta.url).href;
2324
+ const ingest: WorkerEntry = { url: ingestUrl };
2325
+ const scorer: WorkerEntry = {
2326
+ url: typedUrl,
2327
+ settingsSchema: { type: 'object', required: ['factor'], properties: { factor: { type: 'number', minimum: 0 } }, additionalProperties: false },
2328
+ };
2329
+ // Two node types, in the DOCUMENT form — the identity derivation: what each declares is what its
2330
+ // default pool is built from. `config` is every worker instance's constructor argument, `pool` the
2331
+ // WorkerManager options minus what the type states beside it, `groups` the POOL's groups (every
2332
+ // task of the type holds them), `retryPolicy` for every node of the type that declares none.
2333
+ const IngestWorker: NodeTypeDeclaration = { worker: 'ingest', inline: true };
2334
+ const Scorer: NodeTypeDeclaration = {
2335
+ worker: 'scorer',
2336
+ inline: true,
2337
+ pool: { maxThreads: 2 },
2338
+ groups: ['scorer'],
2339
+ retryPolicy: { maxRetries: 1, backoffMs: 0 },
2340
+ };
2341
+
2342
+ // Two nodes of ONE type with DIFFERENT settings — the whole point.
2343
+ const typed: FlowDefinition = {
2344
+ flowId: 'typed', version: 1, entry: 'ingest',
2345
+ nodes: [
2346
+ { id: 'ingest', kind: 'task', type: 'IngestWorker', edges: [{ to: 'low' }] },
2347
+ { id: 'low', kind: 'task', type: 'Scorer', settings: { factor: 10 }, edges: [{ to: 'high' }] },
2348
+ { id: 'high', kind: 'task', type: 'Scorer', settings: { factor: 100 }, edges: [{ to: 'done' }] },
2349
+ { id: 'done', kind: 'end', outcome: 'completed' },
2350
+ ],
2351
+ };
2352
+
2353
+ // The registry the driver reads — the same sections the bootstrap helper builds pools from (below),
2354
+ // so a type the flow names, the pool the scheduler holds and the schema the driver applies are one
2355
+ // object and not three copies. `groups` is the set of group KEYS a node or a type may name.
2356
+ const registry: Registry = { predicates: [], workers: { ingest, scorer }, nodeTypes: { IngestWorker, Scorer }, groups: ['scorer'] };
2357
+ const bad = validateDefinition(typed, registry);
2358
+ if (bad.length) throw new Error(`the typed flow is invalid: ${bad.map((f) => `${f.rule}@${f.at}`).join()}`);
2359
+
2360
+ // The refusals, each at construction and never on the one workload that hits it.
2361
+ const rules = (def: FlowDefinition, reg = registry) => validateDefinition(def, reg).map((f) => f.rule);
2362
+ const low = (node: object): FlowDefinition => ({ ...typed, nodes: [typed.nodes[0]!, node as never, typed.nodes[3]!] });
2363
+ // FD-V32 — a type the registry does not hold; `nodeTypes` OMITTED means NONE registered; and a
2364
+ // registered type whose WORKER the registry's `workers` does not declare.
2365
+ if (!rules(low({ id: 'low', kind: 'task', type: 'Nope', edges: [{ to: 'done' }] })).includes('FD-V32')) throw new Error('FD-V32');
2366
+ if (!rules(typed, { ...registry, nodeTypes: undefined }).includes('FD-V32')) throw new Error('FD-V32 with nodeTypes omitted');
2367
+ if (!rules(typed, { ...registry, workers: {} }).includes('FD-V32')) throw new Error('FD-V32: the type names a worker the registry lacks');
2368
+ // FD-V33 — settings the worker's schema refuses; and absent settings are the type's defaults — none
2369
+ // here, so `{}` — which this schema refuses because it requires `factor`.
2370
+ if (!rules(low({ id: 'low', kind: 'task', type: 'Scorer', settings: { factor: 'ten' }, edges: [{ to: 'done' }] })).includes('FD-V33')) throw new Error('FD-V33');
2371
+ if (!rules(low({ id: 'low', kind: 'task', type: 'Scorer', edges: [{ to: 'done' }] })).includes('FD-V33')) throw new Error('FD-V33 on absent settings');
2372
+ // FD-V38 — a `pool` that is not a registered instance of the node's type; the type's own name is
2373
+ // its default instance and is accepted.
2374
+ if (!rules(low({ id: 'low', kind: 'task', type: 'Scorer', settings: { factor: 1 }, pool: 'nope', edges: [{ to: 'done' }] })).includes('FD-V38')) throw new Error('FD-V38');
2375
+ if (rules(low({ id: 'low', kind: 'task', type: 'Scorer', settings: { factor: 1 }, pool: 'Scorer', edges: [{ to: 'done' }] })).length) throw new Error('the type`s own name is its default instance');
2376
+ // FD-V23 — there is no untyped road: a task without `type` is a task missing a required key.
2377
+ if (!rules(low({ id: 'low', kind: 'task', edges: [{ to: 'done' }] })).includes('FD-V23')) throw new Error('FD-V23: a task with no type');
2378
+ ```
2379
+
2380
+ **Wiring it: `createRuntime`, or the bootstrap helper underneath it.** `fyflow-orchestration/scheduler`
2381
+ is a sub-path — the one module of this package that constructs scheduler objects, kept out of the
2382
+ root so the validator, the driver and the fold load no `WorkerManager` (`RT-R13`). **`createRuntime`
2383
+ is the call a consumer copies** (section 12): it does everything below from a registry DOCUMENT,
2384
+ including creating the group objects. `bootstrap` is what it uses for the pools, and it is exported
2385
+ for a consumer who builds their groups and their scheduler themselves. `bootstrap(registry, groups, options)` builds
2386
+ **one pool per type, keyed by the type's name, and one per named instance in `pools`**, each
2387
+ through the type's derivation and each running the package's envelope worker around your worker,
2388
+ and refuses **both ways** a type naming a group you gave no object for and an object no type names
2389
+ (the scheduler ignores an unknown group id in silence, at the pool level too — measured). Hand the
2390
+ scheduler its result, and the driver the SAME registry sections:
2391
+
2392
+ ```ts check
2393
+ import { type FlowDefinition, type Languages, type NodeTypeDeclaration, type WorkerEntry } from 'fyflow-orchestration';
2394
+ import { bootstrap } from 'fyflow-orchestration/scheduler';
2395
+ import { ConcurrentLimitGroup, FyflowScheduler } from 'fyflow-scheduler';
2396
+ import { Driver, liveToken } from 'fyflow-orchestration';
2397
+
2398
+ declare const ingest: WorkerEntry;
2399
+ declare const scorer: WorkerEntry;
2400
+ declare const IngestWorker: NodeTypeDeclaration;
2401
+ declare const Scorer: NodeTypeDeclaration;
2402
+ declare const typed: FlowDefinition;
2403
+ declare const languages: Languages;
2404
+
2405
+ // The types' groups are given as OBJECTS here and only here; the helper returns them keyed as given.
2406
+ // The languages and the flows are what tells the helper WHICH typed pools need which adapter's
2407
+ // worker module — written into a pool's config only where a node of that type transforms (W61).
2408
+ const registry = { workers: { ingest, scorer }, nodeTypes: { IngestWorker, Scorer } };
2409
+ const built = bootstrap(registry, { scorer: new ConcurrentLimitGroup(4, 'scorer') }, { languages, flows: [typed] });
2410
+ const scheduler = new FyflowScheduler(built.workerPools, built.resourceGroups);
2411
+ const driver = new Driver(typed, {
2412
+ runId: 'run-1', scheduler, predicates: new Map(), joinPolicy: liveToken,
2413
+ ...registry, // the SAME object the helper built every pool from
2414
+ groups: Object.keys(scheduler.groups), // and its groups, read back off the scheduler
2415
+ languages, // and the same languages
2416
+ });
2417
+ driver.start({ confidence: 0.9 });
2418
+ ```
2419
+
2420
+ **What a type declares for its nodes, and how it merges.** A node's own `retryPolicy` wins, whole;
2421
+ then the type's; then the driver's `defaultRetryPolicy`. A type's `groups` are the pool's and the
2422
+ scheduler unions them with a node's own `workerGroups`, so nothing merges — and a task acquires its
2423
+ pool's group **once per ADMISSION**: a retry is admitted again, and so is a task that was admitted
2424
+ to a threaded pool and found no free worker (both measured at ladder L6), so a group's budget is
2425
+ spent per admission and not per task. Both are held at
2426
+ construction: a type's policy to `FD-V30` and a type's groups to the registry's `groups` under
2427
+ `FD-V31`, naming the type.
2428
+
2429
+ **A transform — what a node may DO to its input before its worker sees it** — `spec/flow-definition.md`
2430
+ v0.18, [ADR 0051](doc/decisions/0051-a-transform-is-a-declared-expression-in-a-registered-language-applied-in-the-types-worker-wrapper.md),
2431
+ [W61](doc/work-items/w61-what-a-transform-is.md). When the value a typed node receives is not the
2432
+ shape its worker reads — a join's collection where the worker wants a total, an item where it wants
2433
+ one field — the node declares `transform: { lang, expr }` instead of a reshaping task. **The language
2434
+ is a registry entry**, `Registry.languages`, the fifth set: `{ compile, worker }`, where `compile`
2435
+ runs on the main thread at construction and THROWS on an expression it refuses (so a SYNTAX error in
2436
+ the expression is `FD-V36` at `new Driver`, never a failure on the one workload that reaches the
2437
+ node — a misspelt KEY is not: it compiles and evaluates to `undefined`, which is P18's and P19's
2438
+ to catch),
2439
+ and `worker` is the URL of a module the envelope worker loads in `setup()` and evaluates with in
2440
+ `run()` — two halves, because a function cannot cross to a thread and a URL can. The expression sees
2441
+ the node's input as `$` and its settings as `$settings`; the worker receives `{ settings, input }`
2442
+ with `input` REPLACED by the result. **Evaluation happens in the type's worker wrapper, never in the
2443
+ driver**, so a transform is invisible to anything that reads what the driver dispatched. **A transform
2444
+ that cannot be evaluated FAILS THE TASK** — `TransformFailed`, naming the language, the expression and
2445
+ the language's own error code — and takes the two roads any worker throw takes: an `on: ["failed"]`
2446
+ edge on an `optional` node, `user_action` on a non-optional one. A result of `undefined` is not a
2447
+ failure. And a language module that failed to LOAD is not a worker error either: the wrapper
2448
+ remembers the failure and fails, at `run()`, exactly the tasks that name that language. `$now`,
2449
+ `$millis` and `$random` are unbound in every adapter: a transform is a function of what the run
2450
+ produced. The language this package ships is JSONata, as the sub-path `fyflow-orchestration/jsonata`
2451
+ — section 14; the snippet below uses a toy one so it loads no engine, and it is the same interface.
2452
+
2453
+ ```ts run
2454
+ import { validateDefinition, type FlowDefinition, type LanguageAdapter, type NodeTypeDeclaration, type Registry, type WorkerEntry } from 'fyflow-orchestration';
2455
+ import { EnvelopeWorker, TransformFailed } from 'fyflow-orchestration/scheduler';
2456
+
2457
+ // A toy language: an expression is a dotted key path into the input, or `$settings.<path>`.
2458
+ // The REGISTRY half compiles synchronously and throws on anything else — what FD-V36 reports.
2459
+ // The WORKER half is a module URL (here a data: URL) whose default export compiles the same text
2460
+ // and evaluates it; `evaluate` may return a Promise, as JSONata's does.
2461
+ const GRAMMAR = /^(\$settings\.)?[A-Za-z0-9_]+(\.[A-Za-z0-9_]+)*$/;
2462
+ const workerSrc = `export default function compile(expr) {
2463
+ const segs = expr.split('.'); const fromSettings = segs[0] === '$settings'; const walk = fromSettings ? segs.slice(1) : segs;
2464
+ return { evaluate(input, settings) {
2465
+ let v = fromSettings ? settings : input;
2466
+ for (const k of walk) { if (v == null) return undefined; if (typeof v !== 'object') { const e = new Error('cannot read "' + k + '" of ' + typeof v); e.code = 'P0002'; throw e; } v = v[k]; }
2467
+ return v;
2468
+ } };
2469
+ }`;
2470
+ const path: LanguageAdapter = {
2471
+ compile: (expr) => { if (!GRAMMAR.test(expr)) throw new SyntaxError(`not a dotted key path: ${expr}`); },
2472
+ worker: `data:text/javascript,${encodeURIComponent(workerSrc)}`,
2473
+ };
2474
+
2475
+ // A worker that echoes its input, as a data: URL, and two types over it.
2476
+ const echo: WorkerEntry = { url: 'data:text/javascript,export default class W { run(p) { return p.input; } }' };
2477
+ const Scorer: NodeTypeDeclaration = { worker: 'echo' };
2478
+ const flow: FlowDefinition = {
2479
+ flowId: 'transformed', version: 1, entry: 'ingest',
2480
+ nodes: [
2481
+ { id: 'ingest', kind: 'task', type: 'IngestWorker', edges: [{ to: 'score' }] },
2482
+ // The worker wants the confidence alone; the transform picks it out of the previous result.
2483
+ { id: 'score', kind: 'task', type: 'Scorer', settings: { factor: 10 }, transform: { lang: 'path', expr: 'confidence' }, edges: [{ to: 'done' }] },
2484
+ { id: 'done', kind: 'end', outcome: 'completed' },
2485
+ ],
2486
+ };
2487
+ const registry: Registry = { predicates: [], workers: { echo }, nodeTypes: { IngestWorker: { worker: 'echo' }, Scorer }, languages: { path } };
2488
+ const bad = validateDefinition(flow, registry);
2489
+ if (bad.length) throw new Error(`the transformed flow is invalid: ${bad.map((f) => `${f.rule}@${f.at}`).join()}`);
2490
+
2491
+ // The three refusals, each at construction.
2492
+ const rules = (node: object, reg = registry) => validateDefinition({ ...flow, nodes: [flow.nodes[0]!, node as never, flow.nodes[2]!] }, reg).map((f) => f.rule);
2493
+ const score = (transform: unknown, rest: object = { type: 'Scorer' }) => ({ id: 'score', kind: 'task', transform, edges: [{ to: 'done' }], ...rest });
2494
+ if (!rules(score({ lang: 'jsonata', expr: 'confidence' })).includes('FD-V35')) throw new Error('FD-V35: a language the registry does not name');
2495
+ if (!rules(score({ lang: 'path', expr: 'confidence' }), { ...registry, languages: undefined }).includes('FD-V35')) throw new Error('FD-V35 with languages omitted — absent means EMPTY');
2496
+ if (!rules(score({ lang: 'path', expr: '$sum(*.score' })).includes('FD-V36')) throw new Error('FD-V36: an expression the language`s compile refuses');
2497
+ if (!rules(score({ lang: 'path' })).includes('FD-V37')) throw new Error('FD-V37: not { lang, expr }');
2498
+ // And there is no untyped road for a transform to sit on: a node with no `type` is FD-V23, with the transform's own clauses applied beside it.
2499
+ if (!rules(score({ lang: 'path', expr: 'confidence' }, {})).includes('FD-V23')) throw new Error('FD-V23: a transform on a node with no type is a node with no type');
2500
+
2501
+ // And what the wrapper does with it — the class the bootstrap helper puts around your worker,
2502
+ // driven here directly. The helper writes `languages` into a typed pool's config for you when you
2503
+ // pass it the flows and the languages: `bootstrap(registry, groups, { languages: { path }, flows: [flow] })`.
2504
+ const w = new EnvelopeWorker({ worker: echo.url, languages: { path: path.worker } });
2505
+ await w.setup();
2506
+ const seen = await w.run({ settings: { factor: 10 }, input: { confidence: 0.9, text: 'a bb ccc' }, transform: { lang: 'path', expr: 'confidence' } });
2507
+ if (seen !== 0.9) throw new Error(`the worker received ${JSON.stringify(seen)} — not the transformed input`);
2508
+ const bound = await w.run({ settings: { factor: 10 }, input: { confidence: 0.9 }, transform: { lang: 'path', expr: '$settings.factor' } });
2509
+ if (bound !== 10) throw new Error('the node`s settings are bound as $settings');
2510
+ const plain = await w.run({ settings: { factor: 10 }, input: { confidence: 0.9 } });
2511
+ if (JSON.stringify(plain) !== JSON.stringify({ confidence: 0.9 })) throw new Error('no transform, the value received, untouched');
2512
+ let failed: unknown;
2513
+ try { await w.run({ settings: {}, input: { a: 5 }, transform: { lang: 'path', expr: 'a.b' } }); } catch (e) { failed = e; }
2514
+ if (!(failed instanceof TransformFailed) || failed.code !== 'P0002') throw new Error('a loud evaluation failure is a TransformFailed carrying the language`s own code — the TASK fails');
2515
+ await w.teardown();
2516
+ ```
2517
+
2518
+ **What this is not.** What a type says about what its worker EXPECTS and RETURNS — input and
2519
+ output schemas, and a sample-based check across a transformed edge — was
2520
+ [P16](doc/plans/p16-what-a-node-type-is-and-what-the-library-owes-the-scheduler.md)'s plan after
2521
+ this one (P18) and is now [P19](doc/plans/p19-what-a-schema-buys-an-author.md)'s, answered in
2522
+ **section 15**: since v0.19 a worker DECLARES `inputSchema` and `outputSchema`, no RUN enforces
2523
+ them per settlement (that is OUT of P19 as a default) — since v0.21 the two values no static check
2524
+ can reach are held to them ONCE each, `FD-R4` (section 9's `BoundaryRefused`), and `checkWiring(registry, flow)`
2525
+ gives every arc of a flow one of five verdicts from them before any run (W73), with `pathsOf`,
2526
+ `tryTransform` and `signatureOf` beside it as the authoring assists (W74) — and since v0.20 any schema may carry
2527
+ `examples` — an annotation held to its schema at construction ([ADR 0056](doc/decisions/0056-an-example-is-an-annotation-validated-against-the-schema-it-sits-on.md),
2528
+ [W72](doc/work-items/w72-the-contract-an-example-beside-every-schema-and-a-worker-run-on-it.md)) —
2529
+ so that `npm run check:workers` can RUN a worker on its input example and validate what it returned
2530
+ against its output schema, which is an authoring-time check and never a driver behaviour; the
2531
+ adapter's optional `check` hook is the door left for the rest. A transform on a
2532
+ `map`'s `over`, on an edge or on a decision is refused, with the reasons in ADR 0051. The validator runs at construction and
2533
+ nowhere else: the driver holds your definition by reference, so a document you mutate after
2534
+ `new Driver` is dispatched as mutated, for every key ([Q259](doc/questions/q259.md)). Every one of
2535
+ the twenty-five worked flows in [`examples/`](examples/README.md) runs typed on a real scheduler
2536
+ since v0.19 — three of them with a declared `transform` (section 14), one at three arities, one
2537
+ whose transforms fail, and P18's own sample reducing a join's record for the node after it.
2538
+ *(Twenty-four until [W67](doc/work-items/w67-a-flow-from-documents-on-real-workers.md) added
2539
+ `triage.flow.json`, which is that plan's exit artifact: a flow and the runtime it needs from two
2540
+ JSON documents and a shim.)*
2541
+
2542
+ ## 14. The JSONata adapter — `fyflow-orchestration/jsonata`
2543
+
2544
+ The language this package ships for a `transform` is **JSONata**, at the one exact version
2545
+ `package.json` names, as the sub-path
2546
+ `fyflow-orchestration/jsonata` — [ADR 0052](doc/decisions/0052-a-second-dependency-is-declared-where-its-one-sub-path-is-reached.md),
2547
+ [W62](doc/work-items/w62-the-jsonata-adapter.md). **The engine is a dependency of this package and
2548
+ is loaded by that sub-path alone**: the root and `fyflow-orchestration/scheduler` import none of it,
2549
+ so a consumer who never transforms pays the install and never the load, and a browser bundle carries
2550
+ the engine only where the sub-path is imported (237 KiB). Import it, put it in the registry's
2551
+ `languages` under the name a flow's `transform.lang` uses, and hand the same object to the
2552
+ bootstrap helper:
2553
+
2554
+ ```ts check
2555
+ import { Driver, liveToken, type FlowDefinition, type RegistryDocument } from 'fyflow-orchestration';
2556
+ import { bootstrap } from 'fyflow-orchestration/scheduler';
2557
+ import { jsonata } from 'fyflow-orchestration/jsonata';
2558
+ import { ConcurrentLimitGroup, FyflowScheduler } from 'fyflow-scheduler';
2559
+
2560
+ declare const registry: Pick<RegistryDocument, 'workers' | 'nodeTypes'>;
2561
+ declare const flow: FlowDefinition;
2562
+
2563
+ // The adapter is `{ compile, worker }`: `compile` runs on the main thread at `new Driver` and
2564
+ // `worker` is the URL of the SAME module, which the envelope worker imports inside the pool.
2565
+ const languages = { jsonata };
2566
+ const built = bootstrap(registry, { scorer: new ConcurrentLimitGroup(4, 'scorer') }, { languages, flows: [flow] });
2567
+ const scheduler = new FyflowScheduler(built.workerPools, built.resourceGroups);
2568
+ const driver = new Driver(flow, {
2569
+ runId: 'run-1', scheduler, predicates: new Map(), joinPolicy: liveToken,
2570
+ ...registry, groups: Object.keys(scheduler.groups), languages,
2571
+ });
2572
+ driver.start({ items: [1, 2, 3] });
2573
+ ```
2574
+
2575
+ **What an expression sees, and what it may not.** `$` is the node's input — the previous node's
2576
+ result, the object `needs` described, a map instance's own item, or a join's collection keyed by
2577
+ branch path — and the node's effective `settings` are bound as `$settings`, so one expression is
2578
+ parameterised per node. **Five names are unbound, by name, on every compiled expression**: `$now`,
2579
+ `$millis`, `$random` — the three every adapter must refuse, because a transform is a function of what
2580
+ the run produced and run state is a fold of the log — **`$shuffle`**, JSONata's own fourth, which
2581
+ draws from `Math.random()` (found by walking the engine's function table: of the 61 of its 63
2582
+ built-ins the walk can observe, nothing else differs between two evaluations of one input) — **and
2583
+ `$eval`**, for a different reason: it re-compiles its
2584
+ argument at RUN time, so a syntax error inside it would be a task failure carrying the engine's
2585
+ `D3120` that `FD-V36` never saw, and a transform is declared text refused at construction
2586
+ ([Q265](doc/questions/q265.md)). Each throws naming itself and the task fails with
2587
+ `TransformFailed [UNBOUND]`; a computed key is read with `$lookup`, which recompiles nothing. **And the worker receives plain JSON values**: JSONata marks the
2588
+ arrays it builds with an own `sequence` property that survives `structuredClone` across a thread and
2589
+ that a deep-equal sees; the adapter strips it at every depth.
2590
+
2591
+ **Two things about JSONata itself that decide how you write a reduction.** A path over a join's
2592
+ record changes TYPE with the region's arity — `*.score` is an array at two matching branches, the
2593
+ scalar at one, absent at zero ([Q263](doc/questions/q263.md)) — so a reduction is written over the
2594
+ array form, `$sum([*.score])`, which is `0` for a join nothing reached. And a misspelt key is silent:
2595
+ `input.scroe` evaluates to `undefined` with no error, and the worker receives `undefined`, where
2596
+ `$sum("a")` is loud (`T0412`). Only a syntax error is refused at construction.
2597
+
2598
+ ```ts run
2599
+ import { validateDefinition, type FlowDefinition, type NodeTypeDeclaration, type Registry, type WorkerEntry } from 'fyflow-orchestration';
2600
+ import { EnvelopeWorker, TransformFailed } from 'fyflow-orchestration/scheduler';
2601
+ import { jsonata } from 'fyflow-orchestration/jsonata';
2602
+
2603
+ const echo: WorkerEntry = { url: 'data:text/javascript,export default class W { run(p) { return p.input; } }' };
2604
+ const Scorer: NodeTypeDeclaration = { worker: 'echo' };
2605
+ const flow: FlowDefinition = {
2606
+ flowId: 'reduced', version: 1, entry: 'ingest',
2607
+ nodes: [
2608
+ { id: 'ingest', kind: 'task', type: 'IngestWorker', edges: [{ to: 'split' }] },
2609
+ { id: 'split', kind: 'fanout', branches: ['scoreA', 'scoreB'] },
2610
+ { id: 'scoreA', kind: 'task', type: 'IngestWorker', edges: [{ to: 'collect' }] },
2611
+ { id: 'scoreB', kind: 'task', type: 'IngestWorker', edges: [{ to: 'collect' }] },
2612
+ { id: 'collect', kind: 'join', of: 'split', edges: [{ to: 'publish' }] },
2613
+ // The join's successor wants the total, not the record: the transform reduces it, no reshaping node.
2614
+ { id: 'publish', kind: 'task', type: 'Scorer', settings: { factor: 2 }, transform: { lang: 'jsonata', expr: '$sum([*.score]) * $settings.factor' }, edges: [{ to: 'done' }] },
2615
+ { id: 'done', kind: 'end', outcome: 'completed' },
2616
+ ],
2617
+ };
2618
+ const registry: Registry = { predicates: [], workers: { echo }, nodeTypes: { IngestWorker: { worker: 'echo' }, Scorer }, languages: { jsonata } };
2619
+ const bad = validateDefinition(flow, registry);
2620
+ if (bad.length) throw new Error(`the reduced flow is invalid: ${bad.map((f) => `${f.rule}@${f.at}`).join()}`);
2621
+
2622
+ // FD-V36 carries JSONata's own refusal — its code and the position of the missing token — at `new Driver`, never on a workload.
2623
+ const publish = (expr: string, reg = registry) => validateDefinition({ ...flow, nodes: flow.nodes.map((n) => (n.id === 'publish' ? { ...n, transform: { lang: 'jsonata', expr } } : n)) }, reg);
2624
+ const syntax = publish('$sum(*.score');
2625
+ if (syntax.map((f) => f.rule).join() !== 'FD-V36' || !syntax[0]!.detail.includes('S0203') || !syntax[0]!.detail.includes('position 12')) throw new Error(`FD-V36 with the engine's code and position: ${JSON.stringify(syntax)}`);
2626
+ // FD-V35 — the language is a registry entry: a registry without it refuses the flow, and an absent set means none.
2627
+ if (!publish('$sum([*.score])', { ...registry, languages: {} }).some((f) => f.rule === 'FD-V35')) throw new Error('FD-V35: jsonata not registered');
2628
+
2629
+ // The envelope worker — the class the bootstrap helper puts around your worker — with the adapter's worker half by URL.
2630
+ const w = new EnvelopeWorker({ worker: echo.url, languages: { jsonata: jsonata.worker } });
2631
+ await w.setup();
2632
+ const record = { '0': { score: 40 }, '1': { score: 20 } }; // JP-R12's collection, two branches
2633
+ const total = await w.run({ settings: { factor: 2 }, input: record, transform: { lang: 'jsonata', expr: '$sum([*.score]) * $settings.factor' } });
2634
+ if (total !== 120) throw new Error(`the worker received ${JSON.stringify(total)} — expected the reduction, 120`);
2635
+ const one = await w.run({ settings: {}, input: { '0': { score: 7 } }, transform: { lang: 'jsonata', expr: '$sum([*.score])' } });
2636
+ if (one !== 7) throw new Error('arity one, the same expression: 7');
2637
+ const none = await w.run({ settings: {}, input: {}, transform: { lang: 'jsonata', expr: '$sum([*.score])' } });
2638
+ if (none !== 0) throw new Error('a join nothing reached: 0, because the reduction is over the ARRAY form');
2639
+ let unbound: unknown;
2640
+ try { await w.run({ settings: {}, input: record, transform: { lang: 'jsonata', expr: '$now()' } }); } catch (e) { unbound = e; }
2641
+ if (!(unbound instanceof TransformFailed) || unbound.code !== 'UNBOUND') throw new Error('$now() is unbound: the TASK fails, naming it');
2642
+ await w.teardown();
2643
+ ```
2644
+
2645
+ **How it is declared, and what that costs a consumer.** `package.json` names `jsonata` in
2646
+ `dependencies` at one EXACT version and `deno.json` maps it to `npm:jsonata@` that same version
2647
+ (there is no JSR package for it) — one pin, written twice, and `check:source` `S15` holds the two
2648
+ together and refuses a second field naming the engine;
2649
+ `S14` refuses any published file but `src/jsonata/**` importing the engine, and any form of the
2650
+ specifier there but the bare `jsonata`. An npm consumer installs one 834 KiB package with no
2651
+ dependencies of its own, whether or not they ever import the sub-path; a Deno consumer's
2652
+ `deno.json` resolves the mapping; a bundler includes the engine only in a bundle that imports the
2653
+ sub-path. **A bundled worker is your asset**: inside a bundle `import.meta.url` is the bundle's own
2654
+ URL ([Q261](doc/questions/q261.md)), so `jsonata.worker` — like the envelope worker itself — has to be
2655
+ served beside your bundle as a module of its own for a pool's thread to load it.
2656
+
2657
+ ---
2658
+
2659
+ ## 15. What a schema buys an AUTHOR — an authoring session
2660
+
2661
+ **The schemas in section 13 were never for the run.** A worker's `inputSchema` and `outputSchema`
2662
+ are held to the run's PAYLOAD once and to a person's RESOLUTION once each (`FD-R4`, section 9) and
2663
+ never per settlement — and neither of those is *two values per run*, because a fan-out or map entry
2664
+ validates one value per branch task it reaches and a run nobody parks validates no resolution at
2665
+ all. So what are they for? **They are for whoever is wiring the flow**, a person or an agent, and this section is
2666
+ the session that shows it: a flow written, a MISWIRE caught before anything runs, a transform
2667
+ written by a loop rather than guessed, and a worker typed against a signature generated from the
2668
+ declarations. Three functions do it, all exported from the package root, and none of them refuses
2669
+ anything or is reachable from a run. Two are pure over data; `tryTransform` is not, and the
2670
+ difference matters to a caller — it is `checkWiring`'s one dynamic `import()` of the language
2671
+ adapter's worker half, so it loads a module by URL and returns a promise:
2672
+
2673
+ | | answers |
2674
+ |---|---|
2675
+ | `pathsOf(schema)` | *what is there to reach for* — every path a value may carry, and which are guaranteed |
2676
+ | `tryTransform(registry, flow, node, transform)` | *does THIS expression fit* — one candidate, one verdict, on a real sample |
2677
+ | `signatureOf(name, worker)` | *what do I write my worker against* — TypeScript, as text |
2678
+
2679
+ The registry below is
2680
+ [`examples/getting-started/session-registry.ts`](examples/getting-started/session-registry.ts) — two
2681
+ workers whose schemas OPPOSE on purpose. `reader` returns `{ lines: string[], source }` and closes
2682
+ the object; `summarise` REQUIRES a `text` string and closes its own. Everything here runs on every
2683
+ `npm run gate`, and the same snippet is in `README.md`, held byte-equal by `check-docs`'s `X8`.
2684
+
2685
+ ```ts run
2686
+ import { checkWiring, describePaths, pathsOf, signatureOf, tryTransform, type FlowDefinition } from 'fyflow-orchestration';
2687
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
2688
+ import { jsonata } from 'fyflow-orchestration/jsonata';
2689
+ import { SESSION_BASE, SESSION_REGISTRY, SESSION_WORKERS } from '../../examples/getting-started/session-registry.ts'; // in your program: './registry.ts', a module or a JSON file of your own
2690
+
2691
+ // The three things the check needs from a registry: the workers' schemas, the types that name them,
2692
+ // and the languages a transform may be written in. `createRuntime` takes the whole document below.
2693
+ const registry = { workers: SESSION_WORKERS, nodeTypes: SESSION_REGISTRY.nodeTypes, languages: { jsonata } };
2694
+
2695
+ // THE FLOW, as an author first writes it: read a batch, summarise it. It is a VALID document —
2696
+ // `validateDefinition` has nothing to say about it, because nothing here is about the grammar.
2697
+ const flow: FlowDefinition = {
2698
+ flowId: 'session', version: 1, entry: 'read',
2699
+ nodes: [
2700
+ { id: 'read', kind: 'task', type: 'Reader', edges: [{ to: 'sum' }] },
2701
+ { id: 'sum', kind: 'task', type: 'Summarise', edges: [{ to: 'done' }] },
2702
+ { id: 'done', kind: 'end', outcome: 'completed' },
2703
+ ],
2704
+ };
2705
+
2706
+ // STEP 1 — the wiring check, before a worker has run once. Every arc gets one of five verdicts.
2707
+ const first = await checkWiring(registry, flow);
2708
+ console.log(first.counts); // { fits: 0, no: 1, ... }
2709
+ const arc = first.verdicts.find((v) => v.consumer === 'sum')!;
2710
+ console.log(`${arc.kind}: ${arc.why}`); // no: $: consumer requires "text", producer does not guarantee it
2711
+ if (arc.kind !== 'no') throw new Error(`the miswire is the point of this section: got ${arc.kind}`);
2712
+
2713
+ // STEP 2 — so what IS there? The producer's own output schema, enumerated. A leading `?` marks a
2714
+ // path a value may not carry at all; the rest are guaranteed by `required`.
2715
+ console.log(describePaths(pathsOf(SESSION_WORKERS.reader.outputSchema)));
2716
+ // $ object
2717
+ // $.lines array
2718
+ // ? $.lines[] string
2719
+ // $.source string
2720
+
2721
+ // STEP 3 — a first candidate. It is run on the producer's declared EXAMPLE, and the result is held
2722
+ // to the consumer's input schema, so the answer is about a value rather than about two declarations.
2723
+ const attempt = await tryTransform(registry, flow, 'sum', { lang: 'jsonata', expr: '{ "text": lines }' });
2724
+ console.log(`${attempt[0]!.kind}: ${attempt[0]!.why}`);
2725
+ // transform-fails: the transform's result on the example fails the consumer's inputSchema — $.text expected string, got array
2726
+ if (attempt[0]!.kind !== 'transform-fails') throw new Error('an array is not a string, and the loop says which');
2727
+
2728
+ // STEP 4 — ONE expression edit, and the same loop. `transform-passes` is never `fits`: it is a fact
2729
+ // about the sample the verdict names, which is why `sample.source` is on it (a declared example
2730
+ // here; where a schema declares none, a generated WITNESS, and the verdict says so).
2731
+ const good = { lang: 'jsonata', expr: '{ "text": $join(lines, " ") }' };
2732
+ const passed = await tryTransform(registry, flow, 'sum', good);
2733
+ console.log(`${passed[0]!.kind} on the ${passed[0]!.sample!.source}: ${JSON.stringify(passed[0]!.sample!.result)}`);
2734
+ if (passed[0]!.kind !== 'transform-passes') throw new Error(`expected the sample to pass, got ${passed[0]!.kind}`);
2735
+
2736
+ // STEP 5 — the signature the worker is written against, generated from the same three schemas.
2737
+ // `examples/getting-started/session-types.ts` is this text, checked in, and `summarise.ts` imports
2738
+ // it. A type holds the SHAPE: `words` is `number` here and its schema says `minimum: 0`.
2739
+ const signature = signatureOf('Summarise', SESSION_WORKERS.summarise);
2740
+ console.log(signature.split('\n').filter((l) => l.startsWith('export')).join('\n'));
2741
+ if (!signature.includes('"text": string;')) throw new Error('the required key is required in the type too');
2742
+
2743
+ // STEP 6 — and only now the runtime. Pools are built ONCE, so a flow whose transform was decided
2744
+ // during the session has to reach `createRuntime` as `flows`, or `start` throws `LanguageNotBuilt`
2745
+ // naming the type. Check, then build.
2746
+ const wired: FlowDefinition = { ...flow, nodes: flow.nodes.map((n) => (n.id === 'sum' ? { ...n, transform: good } : n)) };
2747
+ const runtime = createRuntime(SESSION_REGISTRY, { languages: { jsonata } }, { base: SESSION_BASE, flows: [wired] });
2748
+ const state = await runtime.start(wired, { source: 'inbox' }).done;
2749
+ await runtime.shutdown();
2750
+ console.log(state.outcome, Object.values(state.nodes).find((n) => n.nodeId === 'sum')?.result); // completed { words: 3 }
2751
+
2752
+ // The expected values are the DOCUMENT's, not literals beside the run: the consumer's own declared
2753
+ // output example, and the producer's declared line count. A registry whose examples stop describing
2754
+ // its workers is red here and in `test/definition/assist.test.ts`, which runs the same chain.
2755
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
2756
+ const got = Object.values(state.nodes).find((n) => n.nodeId === 'sum')?.result;
2757
+ if (JSON.stringify(got) !== JSON.stringify(SESSION_WORKERS.summarise.outputSchema.examples[0])) {
2758
+ throw new Error(`the run returned ${JSON.stringify(got)}, not the consumer's declared output example`);
2759
+ }
2760
+ if ((got as { words: number }).words !== SESSION_WORKERS.reader.outputSchema.examples[0].lines.length) {
2761
+ throw new Error(`the producer declares ${SESSION_WORKERS.reader.outputSchema.examples[0].lines.length} lines, joined into one text`);
2762
+ }
2763
+ const after = await checkWiring(registry, wired);
2764
+ if (after.counts.no !== 0) throw new Error('and the flow that ran has no unwired arc left');
2765
+ ```
2766
+
2767
+ **What a verdict is and is not.** `fits` is a statement about two DECLARATIONS and is checked
2768
+ statically over the twelve constraining keywords of the subset. `transform-passes` is a statement
2769
+ about ONE VALUE — the producer's first declared example, or a generated witness where it declares
2770
+ none — and the verdict carries which it ran on and, over a region, the ARITY the sample had, because
2771
+ a transform's result changes type with the arity ([Q263](doc/questions/q263.md)). **A witness is not
2772
+ a proof**, and the reference says *the sample passed* rather than *fits* for exactly that reason.
2773
+ Two more the check knows and a document cannot say: an OPEN producer is read as never emitting a key
2774
+ it does not name **unless it REQUIRES that key**, which is a stated leniency and its one scope
2775
+ ([Q305](doc/questions/q305.md); the scope is W100's, and `required` is what refutes the leniency's own
2776
+ premise); and a join successor's
2777
+ record is the check's own knowledge, because the subset cannot spell *any key, each of shape S*
2778
+ ([Q306](doc/questions/q306.md)).
2779
+
2780
+ **One document per verdict** lives in [`spec/examples/wiring/`](spec/examples/wiring/), each named
2781
+ by the verdict it earns — `fits`, `no`, `transform-passes`, `transform-fails` and `unchecked`, the
2782
+ fifth being what the check says when a side declares no schema at all. `spec/flow-definition.md`
2783
+ section 7 records their verdicts and the suite compares the directory and the table both ways.
2784
+
2785
+ **And the generated signature is a RECIPE rather than enforcement.** `signatureOf` emits text from
2786
+ the subset with no dependency; `json-schema-to-typescript` is the richer consumer tool and reads the
2787
+ same schemas. **The two AGREE wherever a schema says something** — an open object with a property
2788
+ gets `[k: string]: unknown` from both — and they differ wherever it says LESS: an empty or
2789
+ annotation-only schema is `unknown` here and an OBJECT type there, and a closed object with no
2790
+ properties is `Record<string, never>` here and `{}` there, which in TypeScript means *any non-nullish
2791
+ value*. Both shapes are in the corpus. *A schema that names no `type` admits every type, and this
2792
+ emits the type that says so* — the same reading `pathsOf` gives it as `*`. *(This paragraph said the
2793
+ opposite — that the two differ on the OPEN object — in the reference, in the module header and in the
2794
+ item's own fact: two runs of one harness, one of which CONFIGURED the generator to close what the
2795
+ schema leaves open, and the later run's line count was kept beside the earlier run's conclusion.
2796
+ W74's conformity reader, from the author's own scratchpad output.)* Neither holds a
2797
+ `minimum`, a `maxLength` or a `minItems`: **a type catches the shape, and `npm run check:workers` —
2798
+ which RUNS each worker on its declared example and validates what came back — is what catches the
2799
+ rest.** Even that sees one value: a bound the code does not keep past the example is invisible to it
2800
+ and visible at every consumer the wiring check reads ([Q302](doc/questions/q302.md)).
2801
+ ---
2802
+
2803
+ ## 16. What a schema buys a RUN — holding a result to the schema its worker declares
2804
+
2805
+ **Section 15's last sentence is where this one starts.** A bound a worker's code does not keep past
2806
+ its declared example is invisible to every authoring-time tool here, and there is a second kind of
2807
+ gap beside it that no such tool could ever reach: **what the run CARRIES is not what the worker
2808
+ returned.** `OR-R2` clones every settled result into the log, and `structuredClone` copies a value's
2809
+ own **enumerable** string-keyed data and nothing else (`OR-R11`, section 9) — so a worker can return
2810
+ a value that satisfies its own `outputSchema` and the record can fail it.
2811
+
2812
+ **So a result may be held to that schema at run time, per settlement, and the answer is a READING.**
2813
+ Supply `onNonconformingResult` and every `done` settlement whose worker declares an `outputSchema`
2814
+ is checked; omit it and nothing is. **There is no boolean beside it — the callback's presence IS the
2815
+ switch**, because two members would be two sources that can disagree, and both disagreements are
2816
+ defects: a switch with no channel is a cost paid for a report nobody receives, and a channel with the
2817
+ switch off is a typed option silently dropped
2818
+ ([ADR 0060](doc/decisions/0060-a-run-time-result-check-is-a-reading-and-never-a-refusal.md),
2819
+ `OR-R13`).
2820
+
2821
+ **A finding changes NOTHING about the run.** No record is written differently, no terminal state is
2822
+ authored, no dispatch moves, and the value reaches its successor unaltered — so the run completes
2823
+ exactly as it would with the check off. *You cannot audit a behaviour with an instrument that alters
2824
+ it*, and that is the whole reason this is not a refusal: what it buys you is **knowledge rather than
2825
+ protection**, and a value that fails its schema still flows downstream. The one way it can change a
2826
+ run is **your own callback throwing**, which is caught at the listener, reported through
2827
+ `onRunFailure` and stops the run — the treatment every callback here gets (section 9).
2828
+
2829
+ Three more things worth knowing before you switch it on:
2830
+
2831
+ | | |
2832
+ |---|---|
2833
+ | **`done` settlements only** | a failed or parked task has `result: null` (`OR-R2`), and an output schema no worker reached would refuse every one of them |
2834
+ | **settlements only, never a resolution** | a person's `done` value is held to the same schema at `resolve()`, once, as a REJECTED command (`FD-R4`, section 9). One value refused twice by two mechanisms with different answers is worse than either |
2835
+ | **a worker with no `outputSchema` is UNCHECKED** | *absent means unchecked*, exactly as `FD-R4` reads it |
2836
+
2837
+ *What it costs is a RATIO to the `structuredClone` the driver already pays on the same settlement,
2838
+ and on this repository's own corpus it stays UNDER it at every shape measured. The share FALLS as the
2839
+ value grows and RISES as the schema does — the clone is linear in the value and the check is linear in
2840
+ the schema, and `enum` is where a schema's size is unbounded, so one large `enum` puts the check OVER
2841
+ the clone. `spec/outcome-record.md` section 4 carries the figures with the method that took them. It
2842
+ is a ratio rather than a millisecond, deliberately, and the reason is visible in the one number
2843
+ `examples/l6.ts` PRINTS: over every settlement that rung makes, the same two values read about twice
2844
+ as dear in Node as in Deno. **A single figure would be a property of a machine**
2845
+ ([Q179](doc/questions/q179.md), [Q335](doc/questions/q335.md)). (This paragraph said "about a third",
2846
+ which is the Node reading — W82's conformity reader, finding 12.)*
2847
+
2848
+ The worker below is
2849
+ [`examples/getting-started/drift.ts`](examples/getting-started/drift.ts) and it has **both** shapes
2850
+ in one return value, neither of which is a mistake anybody would call out in review. Everything here
2851
+ runs on every `npm run gate`. *(This sentence also said **and the same snippet is in `README.md`,
2852
+ held byte-equal by `check-docs`'s `X8`** until
2853
+ [W107](doc/work-items/w107-the-readme-is-for-a-person-who-will-use-the-library.md), which gave the
2854
+ three root documents one audience each
2855
+ ([ADR 0073](doc/decisions/0073-the-readme-is-for-a-person-the-reference-is-for-an-agent.md)) and took
2856
+ this section's pairing out with the README section it mirrored. `X8` pairs three sections now — 2, 5
2857
+ and 15 — and each of those three still carries this claim and still earns it. Found by W107's
2858
+ conformity reader: the item edited every other document the change reaches and not this one, which is
2859
+ the document ADR 0073 designates as the agent's reference, about the mechanism the item changed.)*
2860
+
2861
+ ```ts run
2862
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
2863
+ import { checkWiring, type FlowDefinition, type NonconformingResult, type RegistryDocument } from 'fyflow-orchestration';
2864
+
2865
+ // THE REGISTRY — one worker with both of its VALUE schemas declared, and one with none. The schemas
2866
+ // are the same objects the wiring check reads; what changes below is that a RUN reads them too.
2867
+ const registry: RegistryDocument = {
2868
+ workers: {
2869
+ drift: {
2870
+ url: '../../examples/getting-started/drift.ts', // in your program: './drift.ts'
2871
+ inputSchema: { type: 'object', properties: { text: { type: 'string' } }, examples: [{ text: 'alpha bb' }] },
2872
+ outputSchema: {
2873
+ type: 'object', required: ['words', 'load', 'label'], additionalProperties: false,
2874
+ properties: {
2875
+ words: { type: 'integer', minimum: 0 },
2876
+ load: { type: 'number', minimum: 0, maximum: 1 },
2877
+ label: { type: 'string' },
2878
+ },
2879
+ examples: [{ words: 2, load: 0.2, label: 'w2' }],
2880
+ },
2881
+ },
2882
+ inspect: { url: '../../examples/getting-started/inspect.ts' }, // no outputSchema: UNCHECKED
2883
+ },
2884
+ nodeTypes: { Drift: { worker: 'drift', inline: true }, Inspect: { worker: 'inspect', inline: true } },
2885
+ };
2886
+
2887
+ const flow: FlowDefinition = {
2888
+ flowId: 'drift', version: 1, entry: 'measure',
2889
+ nodes: [
2890
+ { id: 'measure', kind: 'task', type: 'Drift', edges: [{ to: 'after' }] },
2891
+ { id: 'after', kind: 'task', type: 'Inspect', edges: [{ to: 'done' }] },
2892
+ { id: 'done', kind: 'end', outcome: 'completed' },
2893
+ ],
2894
+ };
2895
+
2896
+ // EVERY authoring-time tool says this flow is fine, and both are right: one reads two SCHEMAS and
2897
+ // the other runs a worker on ONE declared example. Neither holds a record. The wiring check takes
2898
+ // the document's two relevant PARTS rather than the document (section 15).
2899
+ const wiring = await checkWiring({ workers: registry.workers, nodeTypes: registry.nodeTypes }, flow);
2900
+ console.log(wiring.counts.no); // 0
2901
+ if (wiring.counts.no !== 0) throw new Error('the static check has nothing to say about either defect');
2902
+
2903
+ // SUPPLY THE CALLBACK AND THE CHECK IS ON. There is no boolean beside it: its presence IS the
2904
+ // switch (ADR 0060). A finding is a READING — nothing is appended differently, no terminal state is
2905
+ // authored, no dispatch moves, and the value reaches the next node exactly as it would with the
2906
+ // check off.
2907
+ const found: NonconformingResult[] = [];
2908
+ const runtime = createRuntime(registry, {}, { base: import.meta.url, flows: [flow] });
2909
+ const state = await runtime.start(flow, { text: 'one two three four five six seven eight nine ten eleven' }, {
2910
+ onNonconformingResult: (f) => { found.push(f); },
2911
+ }).done;
2912
+ await runtime.shutdown();
2913
+
2914
+ console.log(state.outcome); // completed — a finding changes nothing
2915
+ // ONE report per settlement, carrying EVERY way that result failed, in document order.
2916
+ console.log(found.map((f) => `${f.nodeId} (${f.worker}): ${f.findings.map((x) => `${x.path} ${x.says}`).join('; ')}`));
2917
+ // [ 'measure (drift): $.label required; $.load above maximum 1' ]
2918
+
2919
+ if (state.outcome !== 'completed') throw new Error(`a finding is a reading: expected completed, got ${state.outcome}`);
2920
+ if (found.length !== 1) throw new Error(`one report per settlement, not one per finding: got ${found.length}`);
2921
+ if (found[0]?.worker !== 'drift' || found[0]?.type !== 'Drift') throw new Error(`a finding names the node, its type and the worker whose schema refused it: ${JSON.stringify(found[0])}`);
2922
+
2923
+ // `found` rather than `found[0]!`: a guard that raises a TypeError on an empty report is a CRASH and
2924
+ // not a detection, and every `includes` below would inherit it.
2925
+ const says = found.flatMap((f) => f.findings).map((x) => `${x.path} ${x.says}`).join('; ');
2926
+ // The BOUND is one the worker's own code does not keep past its declared domain: at the two words
2927
+ // its declared example carries, `load` is 0.2 and nothing is reported. `check-workers` runs that
2928
+ // example and is green; one example confirms a schema at ONE value.
2929
+ if (!says.includes('$.load above maximum 1')) throw new Error(`expected the bound to be reported, got ${says}`);
2930
+ // The MISSING KEY is not a defect in the worker at all. It returned `label`; `structuredClone`
2931
+ // copies own ENUMERABLE keys and nothing else (OR-R11), so the RECORD the run carries does not have
2932
+ // it — and what this check reads is the record. No authoring-time tool holds one.
2933
+ if (!says.includes('$.label required')) throw new Error(`expected the clone's loss to be reported, got ${says}`);
2934
+
2935
+ // AND THE VALUE FLOWED ON ANYWAY. `after` declares no `outputSchema`, so it is UNCHECKED; what it
2936
+ // reports is the key set it RECEIVED, which is the record's and not the worker's.
2937
+ const after = Object.values(state.nodes).find((n) => n.nodeId === 'after')?.result as { keys: string[] };
2938
+ console.log(after.keys); // [ 'load', 'words' ]
2939
+ if (after.keys.join() !== 'load,words') throw new Error(`the successor received ${after.keys.join()}`);
2940
+
2941
+ // OMIT THE CALLBACK AND NOTHING IS CHECKED — and the run is the same run. This is the property to
2942
+ // rely on: you can turn the reading on in one environment and off in another without changing what
2943
+ // the flow DOES, because an instrument that altered what it measures would be useless for auditing.
2944
+ const quiet = createRuntime(registry, {}, { base: import.meta.url, flows: [flow] });
2945
+ const same = await quiet.start(flow, { text: 'one two three four five six seven eight nine ten eleven' }).done;
2946
+ await quiet.shutdown();
2947
+ const shape = (s: typeof state) => JSON.stringify(Object.values(s.nodes).map((n) => [n.nodeId, n.result]));
2948
+ if (same.outcome !== state.outcome || shape(same) !== shape(state)) {
2949
+ throw new Error('a finding is a reading: the same run with the check off must reach the same nodes with the same results');
2950
+ }
2951
+ ```
2952
+
2953
+ **Where the two halves of this are proved, and it is not here.** Every guard in the snippet above
2954
+ can be made a tautology with `npm run check:docs` at exit 0 — `X4` counts a `throw` textually and
2955
+ refuses `if (false)` by name, and neither reaches `found.length !== found.length`
2956
+ ([Q310](doc/questions/q310.md), measured on this section by W82's adequacy reader). **What holds the
2957
+ two claims is `examples/l6.ts`**, which runs the same two shapes as two separate workers on a real
2958
+ `FyflowScheduler` in both runtimes — `nonconforming — eleven words` and its control at two words,
2959
+ where the bound is kept and the clone's loss is reported anyway — and holds the check-ON run against
2960
+ the identical run with the check OFF, record for record. *A worker here made to keep its bound is
2961
+ green at this rung and red at that one.*
2962
+ ## 17. Fanning out PER ITEM — a region inside a `map` region
2963
+
2964
+ **The shape this section teaches was refused at `Driver` construction until `0.4.0`.** *Score each item, then fan
2965
+ out per item* — a `map` over a list, and inside each of its branches a SECOND `map` over a list that
2966
+ branch computed — was refused at `Driver` construction, because a region's identity was its opener's
2967
+ node id and a `map` makes one opener reachable once per item. The workaround was the one this library
2968
+ exists to remove: a reshaping node and a second flow.
2969
+
2970
+ **A region INSTANCE is now the pair `(opener node id, the opener's own branch path)`**
2971
+ ([ADR 0062](doc/decisions/0062-a-region-instance-is-its-opener-and-the-openers-own-branch-path.md)).
2972
+ Three things follow, and they are the whole of what you need:
2973
+
2974
+ | | |
2975
+ |---|---|
2976
+ | **`RunState.regions` is TWO-LEVEL** | `regions[openerId][openerPath]`. The opener's id is unchanged in meaning and in value; the second key is `-` for a region in no region, and inside a `map` region it is that map's item index |
2977
+ | **every `RegionEvent` carries `regionPath` beside `regionId`** | so an event says WHICH instance it is about. The same for `UnsatisfiableJoin` |
2978
+ | **a join's collection is per INSTANCE** | `JP-R12` keys it by branch path, and a branch inside an instance carries that instance's path as its own prefix — so two instances of one opener hold DISJOINT key sets and cannot collide |
2979
+
2980
+ **Both arities come from data, at both levels, and neither is in the document.** That is what
2981
+ separates a `map` from a `fanout`: `FD-V9` requires a fan-out's branch entries to be written down, so
2982
+ a fan-out's width is an authoring-time fact. Here the outer width is *how many orders the entry
2983
+ returned* and each inner width is *how many lines THAT order has* — different per instance.
2984
+
2985
+ **Writing it needs exactly one node between the two maps.** A `map`'s `over` names a key of what the
2986
+ previous node returned, and a `map`'s branch is a TASK (`FD-V25` refuses a map whose branch is a
2987
+ map) — so the outer branch entry is the node whose output carries the item's own list. No `needs`, no
2988
+ `transform`, no reshaping node and no second flow.
2989
+
2990
+ ```ts run
2991
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
2992
+ import type { FlowDefinition, RegionEvent, RegistryDocument } from 'fyflow-orchestration';
2993
+
2994
+ const registry: RegistryDocument = {
2995
+ workers: {
2996
+ orders: { url: '../../examples/getting-started/orders.ts' }, // in your program: './orders.ts'
2997
+ lines: { url: '../../examples/getting-started/lines.ts' },
2998
+ tag: { url: '../../examples/getting-started/tag.ts' },
2999
+ },
3000
+ nodeTypes: {
3001
+ Orders: { worker: 'orders', inline: true },
3002
+ Lines: { worker: 'lines', inline: true },
3003
+ Tag: { worker: 'tag', inline: true },
3004
+ },
3005
+ };
3006
+
3007
+ // TWO MAPS, one inside the other's branch. `expand` is the single node between them: a map's `over`
3008
+ // names a key of what the PREVIOUS node returned, so the outer branch entry is what produces the
3009
+ // item's own list. `FD-V25` refuses a map whose branch is a map, which is why that node is required
3010
+ // and why it is the only one.
3011
+ const flow: FlowDefinition = {
3012
+ flowId: 'per-item', version: 1, entry: 'intake',
3013
+ nodes: [
3014
+ { id: 'intake', kind: 'task', type: 'Orders', edges: [{ to: 'perOrder' }] },
3015
+ { id: 'perOrder', kind: 'map', over: 'orders', branch: 'expand' },
3016
+ { id: 'expand', kind: 'task', type: 'Lines', edges: [{ to: 'perLine' }] },
3017
+ { id: 'perLine', kind: 'map', over: 'lines', branch: 'price' },
3018
+ { id: 'price', kind: 'task', type: 'Tag', settings: { label: 'priced' }, edges: [{ to: 'lineJoin' }] },
3019
+ { id: 'lineJoin', kind: 'join', of: 'perLine', edges: [{ to: 'orderJoin' }] },
3020
+ { id: 'orderJoin', kind: 'join', of: 'perOrder', edges: [{ to: 'report' }] },
3021
+ { id: 'report', kind: 'task', type: 'Tag', settings: { label: 'reported' }, edges: [{ to: 'done' }] },
3022
+ { id: 'done', kind: 'end', outcome: 'completed' },
3023
+ ],
3024
+ };
3025
+
3026
+ // A region event now says WHICH INSTANCE it is about. Before 0.4.0 it carried `regionId` alone,
3027
+ // which is the same string for every instance of one opener.
3028
+ const arrivals: string[] = [];
3029
+ const runtime = createRuntime(registry, {}, { base: import.meta.url, flows: [flow] });
3030
+ const state = await runtime.start(flow, { region: 'ab c' }, {
3031
+ onRegionEvent: (e: RegionEvent) => {
3032
+ if (e.type === 'branch.arrived') arrivals.push(`${e.regionId}@${e.regionPath} <- ${e.branchPath}`);
3033
+ },
3034
+ }).done;
3035
+ await runtime.shutdown();
3036
+
3037
+ console.log(state.outcome); // completed
3038
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
3039
+
3040
+ // ONE OPENER, TWO INSTANCES — keyed by the opener's own branch path, which inside `perOrder` is the
3041
+ // item index. `'ab c'` is two orders, so `perOrder` opened two branches and `perLine` was entered
3042
+ // twice; `ab` has two lines and `c` has one, and neither number is in the flow above.
3043
+ console.log(Object.keys(state.regions['perLine'] ?? {}).sort()); // [ '0', '1' ]
3044
+ console.log(Object.keys(state.regions['perOrder'] ?? {}).sort()); // [ '-' ]
3045
+ const instances = Object.keys(state.regions['perLine'] ?? {}).sort();
3046
+ if (instances.join() !== '0,1') throw new Error(`one opener, two instances: got ${instances.join()}`);
3047
+ if (Object.keys(state.regions['perOrder'] ?? {}).join() !== '-') throw new Error('perOrder is in no region, so its instance key is `-`');
3048
+
3049
+ // EACH INSTANCE HAS ITS OWN WIDTH AND ITS OWN ANSWER, and their branch keys are disjoint because a
3050
+ // branch inside an instance begins with that instance's path.
3051
+ const widths = instances.map((p) => Object.keys(state.regions['perLine']![p]!.settled).sort().join(' '));
3052
+ console.log(widths); // [ '0.0 0.1', '1.0' ]
3053
+ if (widths.join(' | ') !== '0.0 0.1 | 1.0') throw new Error(`each instance has its own width: got ${widths.join(' | ')}`);
3054
+ if (!instances.every((p) => state.regions['perLine']![p]!.joined === true)) throw new Error('both inner joins fired');
3055
+
3056
+ // AND THE EVENTS NAMED THE INSTANCE, not just the region.
3057
+ console.log(arrivals.sort());
3058
+ // [ 'perLine@0 <- 0.0', 'perLine@0 <- 0.1', 'perLine@1 <- 1.0', 'perOrder@- <- 0', 'perOrder@- <- 1' ]
3059
+ if (!arrivals.includes('perLine@0 <- 0.0') || !arrivals.includes('perLine@1 <- 1.0')) {
3060
+ throw new Error(`a region event names the instance: ${arrivals.sort().join(', ')}`);
3061
+ }
3062
+
3063
+ // WHAT THE OUTER JOIN CARRIES IS TWO LEVELS DEEP. `JP-R12`: one entry per branch that arrived, keyed
3064
+ // by branch path — and each of the outer entries IS that instance's own collection.
3065
+ const report = Object.values(state.nodes).find((n) => n.nodeId === 'report')?.result as Record<string, Record<string, { sku: string }>>;
3066
+ console.log(Object.keys(report).filter((k) => k !== 'reported').sort()); // [ '0', '1' ]
3067
+ console.log(report['0']!['0.1']!.sku); // ab-1
3068
+ if (report['0']!['0.1']!.sku !== 'ab-1' || report['1']!['1.0']!.sku !== 'c-0') {
3069
+ throw new Error(`the outer collection's values are the inner collections: ${JSON.stringify(report)}`);
3070
+ }
3071
+ ```
3072
+
3073
+ **What is still refused, and by what.** `FD-V19` keeps one node out of two branch entries of one
3074
+ fan-out; `FD-V25` keeps a map's branch from being a map; `ID-R6` refuses a branch path across a map
3075
+ region with no item index; `ID-R1` refuses a duplicate dispatch. **Those four are what make the
3076
+ derived key total**, and they are why this cost no new refusal and no new error class.
3077
+
3078
+ **An empty inner list still stops the run.** A `map` over zero items opens no branch, so its join can
3079
+ never fire and the run is `stuck` (section 8) — one empty list anywhere makes the whole run stuck,
3080
+ and that ruling is unchanged by nesting.
3081
+
3082
+ **`0.4.0` is BREAKING in two ways and neither is `regionId`.** `RunState.regions` gaining its second
3083
+ level is a break for anyone reading it, and `UnsatisfiableJoin.regionPath` is a REQUIRED member on an
3084
+ interface a consumer may construct — so a test double reporting a dead join no longer compiles.
3085
+ `regionId` keeps its meaning AND its value, which is why no flow, no policy and no region name moves.
3086
+
3087
+ ---
3088
+
3089
+ ## 18. Saving a run and RESUMING it — append the events, resume from them
3090
+
3091
+ **A run can outlive the process that ran it, and what it needs is the events you were already
3092
+ handed.** The library defines the value and never where it goes
3093
+ ([P23](doc/plans/p23-what-a-run-needs-to-outlive-its-process.md) decision 1): no file, no database,
3094
+ no serializer. A CHECKPOINT is `{ header, events }`
3095
+ ([ADR 0064](doc/decisions/0064-a-checkpoint-is-the-whole-event-sequence-and-a-header-and-it-is-a-copy.md)) —
3096
+ the header is five facts the events cannot regenerate (`format`, `runId`, `flowId`, `version` and a
3097
+ copy of the entry payload), and `events` is every fold input so far, so **every event after a
3098
+ checkpoint is an append to it**. You never have to call `driver.checkpoint()` to keep one:
3099
+
3100
+ | | where you get it | what you do with it |
3101
+ |---|---|---|
3102
+ | the header | `runtime.start(…)` returns it as `header`, beside `driver` and `done` | write it once |
3103
+ | every event | `onFoldInput(event, position)` | write it AT `position` — the driver's index, never a count you start again in each process |
3104
+ | a resumed run | `runtime.resume(flow, { header, events }, options)` returns the same `{ driver, done, header }` | keep writing: it hands you positions from the STORED LENGTH on |
3105
+
3106
+ **Key the store by POSITION, and the reason is measured.** The first events are delivered WHILE
3107
+ `start` runs, so the header arrives after them — a store that must write its header first cannot —
3108
+ and a resumed run's first position is the length you stored, not 0, so an arrival counter per process
3109
+ overwrites the run it is resuming. ([W90](doc/work-items/w90-a-run-killed-and-resumed-on-real-workers.md)
3110
+ runs exactly that wrong store at L6 and it is refused `diverged` at position 0.) **The contract is
3111
+ `structuredClone`, not JSON**: a result is whatever survived `OR-R2`'s clone, which includes `Map`,
3112
+ `Set`, `Date`, `BigInt` and `-0` — pick a serializer that keeps them, or revive them.
3113
+
3114
+ ```ts run
3115
+ import { createRuntime, RunIdInUse } from 'fyflow-orchestration/scheduler';
3116
+ import { fold, ResumeRefused } from 'fyflow-orchestration';
3117
+ import type { CheckpointHeader, FlowDefinition, FoldInput, RegistryDocument } from 'fyflow-orchestration';
3118
+
3119
+ // Section 17's flow and workers — ONE map inside another — so there is something in flight to lose.
3120
+ const registry: RegistryDocument = {
3121
+ workers: {
3122
+ orders: { url: '../../examples/getting-started/orders.ts' }, // in your program: './orders.ts'
3123
+ lines: { url: '../../examples/getting-started/lines.ts' },
3124
+ tag: { url: '../../examples/getting-started/tag.ts' },
3125
+ },
3126
+ nodeTypes: {
3127
+ Orders: { worker: 'orders', inline: true },
3128
+ Lines: { worker: 'lines', inline: true },
3129
+ Tag: { worker: 'tag', inline: true },
3130
+ },
3131
+ };
3132
+ const flow: FlowDefinition = {
3133
+ flowId: 'per-item', version: 1, entry: 'intake',
3134
+ nodes: [
3135
+ { id: 'intake', kind: 'task', type: 'Orders', edges: [{ to: 'perOrder' }] },
3136
+ { id: 'perOrder', kind: 'map', over: 'orders', branch: 'expand' },
3137
+ { id: 'expand', kind: 'task', type: 'Lines', edges: [{ to: 'perLine' }] },
3138
+ { id: 'perLine', kind: 'map', over: 'lines', branch: 'price' },
3139
+ { id: 'price', kind: 'task', type: 'Tag', settings: { label: 'priced' }, edges: [{ to: 'lineJoin' }] },
3140
+ { id: 'lineJoin', kind: 'join', of: 'perLine', edges: [{ to: 'orderJoin' }] },
3141
+ { id: 'orderJoin', kind: 'join', of: 'perOrder', edges: [{ to: 'report' }] },
3142
+ { id: 'report', kind: 'task', type: 'Tag', settings: { label: 'reported' }, edges: [{ to: 'done' }] },
3143
+ { id: 'done', kind: 'end', outcome: 'completed' },
3144
+ ],
3145
+ };
3146
+
3147
+ // YOUR STORE. The library never writes one: here it is an object in memory, in your program a
3148
+ // file, a table or a queue. It holds the HEADER once and every event AT ITS POSITION — the second
3149
+ // argument `onFoldInput` is handed — so a later process can go on appending to it.
3150
+ const store: { header?: CheckpointHeader; events: FoldInput[] } = { events: [] };
3151
+
3152
+ // ---- PROCESS ONE, which dies mid-run.
3153
+ let alive = true;
3154
+ let die!: () => void;
3155
+ const died = new Promise<void>((resolve) => { die = resolve; });
3156
+ const first = createRuntime(registry, {}, { base: import.meta.url, flows: [flow] });
3157
+ const run = first.start(flow, { region: 'ab c' }, {
3158
+ onFoldInput: (event, position) => {
3159
+ if (!alive) return; // a dead process writes nothing
3160
+ store.events[position] = structuredClone(event); // your durable append, BY POSITION
3161
+ // THE CRASH, simulated: the process dies the instant the first `expand` settles — inside the
3162
+ // driver's turn, BEFORE it dispatches what that settlement causes.
3163
+ if (event.kind === 'outcome' && event.record.taskId.includes(':expand:')) { alive = false; die(); }
3164
+ },
3165
+ });
3166
+ // The header arrives on the handle, AFTER the first events: they were delivered while `start` ran.
3167
+ // A store that had to write its header first could not; one keyed by position does not care.
3168
+ store.header = run.header;
3169
+ await died;
3170
+ // `shutdown()` is cleanup, NOT the kill: it lets running work finish, and here that finishes the
3171
+ // whole run. What makes this a crash is the store — nothing after `alive = false` reached it.
3172
+ await first.shutdown();
3173
+
3174
+ const storedSettled = new Set(store.events.flatMap((e) => (e.kind === 'outcome' ? [e.record.taskId] : [])));
3175
+ const storedDispatched = new Set(store.events.flatMap((e) => (e.kind === 'dispatched' ? [e.taskId] : [])));
3176
+ console.log(`stored ${store.events.length} events, the last one the settlement the process died in`);
3177
+
3178
+ // ---- PROCESS TWO: a FRESH runtime, which is all a restarted process has. Resume from the store and
3179
+ // keep appending BY POSITION: a resumed run hands you positions from the stored length on, so the
3180
+ // same store holds the whole run again.
3181
+ const header = store.header!; // written once, by process one
3182
+ const second = createRuntime(registry, {}, { base: import.meta.url, flows: [flow] });
3183
+ const resumed = second.resume(flow, { header, events: store.events.slice() }, {
3184
+ onFoldInput: (event, position) => { store.events[position] = structuredClone(event); },
3185
+ });
3186
+ const state = await resumed.done;
3187
+
3188
+ console.log(state.outcome); // completed
3189
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
3190
+ // The same answer a run nobody killed gives (section 17's):
3191
+ const report = Object.values(state.nodes).find((n) => n.nodeId === 'report')?.result as Record<string, Record<string, { sku: string }>>;
3192
+ if (report['0']!['0.1']!.sku !== 'ab-1' || report['1']!['1.0']!.sku !== 'c-0') {
3193
+ throw new Error(`the resumed run reaches the same report: ${JSON.stringify(report)}`);
3194
+ }
3195
+
3196
+ // WHAT RAN AGAIN — named by the `resumed` event, and recorded by the fold in `state.resumes`.
3197
+ const resumes = store.events.flatMap((e) => (e.kind === 'resumed' ? [e.taskIds] : []));
3198
+ if (resumes.length !== 1) throw new Error(`one resume is recorded as one \`resumed\` event: ${resumes.length}`);
3199
+ const again = resumes[0]!;
3200
+ console.log(again); // e.g. [ 'r1:expand:1:0', 'r1:price:0.0:0', 'r1:price:0.1:0' ]
3201
+ if (JSON.stringify(state.resumes) !== JSON.stringify(resumes)) throw new Error('RunState.resumes records it');
3202
+ // Nothing that SETTLED in the store runs again...
3203
+ if (again.some((id) => storedSettled.has(id))) throw new Error('a stored settlement is never re-run');
3204
+ // ...but work the store NEVER SAW does: the prices the dying settlement dispatched after the last
3205
+ // thing that was written. The re-run set is what was in flight at the END of the replay.
3206
+ const neverStored = again.filter((id) => !storedDispatched.has(id));
3207
+ console.log(neverStored); // the `price` tasks of the order whose `expand` the process died in
3208
+ if (neverStored.length === 0 || !neverStored.every((id) => id.includes(':price:'))) {
3209
+ throw new Error(`the lost tail's dispatches run again: ${JSON.stringify(neverStored)}`);
3210
+ }
3211
+
3212
+ // THE STORE HOLDS THE WHOLE RUN AGAIN: no gap at any position, and its fold IS the run.
3213
+ if (Object.keys(store.events).length !== store.events.length) throw new Error('a store keyed by position has no holes');
3214
+ if (JSON.stringify(fold(store.events)) !== JSON.stringify(state)) throw new Error('the store folds to the run');
3215
+
3216
+ // A run id belongs to the runtime that holds it: resuming the same run on `second` again is refused.
3217
+ try {
3218
+ second.resume(flow, { header, events: store.events.slice() });
3219
+ throw new Error('expected RunIdInUse');
3220
+ } catch (err) {
3221
+ if (!(err instanceof RunIdInUse)) throw err;
3222
+ }
3223
+ // And a checkpoint is resumed against the definition it was stored under, or not at all.
3224
+ const third = createRuntime(registry, {}, { base: import.meta.url, flows: [flow] });
3225
+ try {
3226
+ third.resume({ ...flow, version: 2 }, { header, events: store.events.slice() });
3227
+ throw new Error('expected ResumeRefused');
3228
+ } catch (err) {
3229
+ if (!(err instanceof ResumeRefused) || err.reason !== 'flow') throw err;
3230
+ }
3231
+ await second.shutdown();
3232
+ await third.shutdown();
3233
+ ```
3234
+
3235
+ **What a resume PROMISES, and what it does not** — [ADR 0065](doc/decisions/0065-a-resume-replays-the-stored-stimuli-refuses-what-it-cannot-regenerate-and-re-dispatches-what-was-in-flight.md), `spec/run-state.md` section 9:
3236
+
3237
+ - **It is AT-LEAST-ONCE, and your workers must tolerate running twice.** The resume REPLAYS the
3238
+ stored settlements and resolutions through the routing that produced them, with every dispatch
3239
+ held, and then hands every task still in flight to the new scheduler at the SAME task id. That set
3240
+ is what was in flight at the END of the replay — including a task that finished in the dead process
3241
+ whose settlement you had not stored yet, and a task the dying settlement dispatched after your last
3242
+ write, which the snippet shows the store never saw. The run records it: a `resumed` event naming
3243
+ those ids, and `state.resumes`.
3244
+ - **A re-run task gets a FRESH retry budget.** The log records a task's `attempts` only when it
3245
+ SETTLES, so the attempts a task spent before the crash are recorded nowhere — measured: a task with
3246
+ a budget of two attempts ran THREE times across one kill ([Q365](doc/questions/q365.md)).
3247
+ - **A task PARKED for a person is not in flight.** It stays parked, the resumed run is
3248
+ `awaiting_action`, and `resolve()` on the RESUMED driver completes it (section 7).
3249
+ - **The check sees only what you STORED.** The stored events must be a prefix of what the replay
3250
+ regenerates, and the first position where they are not is refused `diverged` — so a predicate or a
3251
+ flow you changed is caught where the stored run's road crosses it, and NOT where that road lies
3252
+ past your last event: there the resumed run takes the new road silently
3253
+ ([Q372](doc/questions/q372.md)). Change a flow, change its `version`.
3254
+ - **The store is TRUSTED for what the resume feeds.** A stored settlement's result and a stored
3255
+ person's resolution are replayed as stored — nothing can check they are what the process saw
3256
+ ([Q374](doc/questions/q374.md)). A store you do not trust is a store you should not resume from.
3257
+ - **`ResumeRefused` names its `reason`**: `format` (not a checkpoint this version writes), `run` (the
3258
+ header's run is not the driver's), `flow` (another flow id or `version` — there is no migration),
3259
+ `stopped` and `diverged` — the last two with the `index` of the position. **A STOPPED run is not
3260
+ resumable**: the refusal that stopped it would stop it again. A completed or `stuck` run resumes to
3261
+ itself and dispatches nothing.
3262
+ - **A refused resume may SPEND its run id.** Refused by its header, nothing exists and the id is still
3263
+ free; refused during its replay, the runtime built a driver for it — so that runtime refuses the id
3264
+ with `RunIdInUse` from then on, as it refuses any id it ever built a driver for, a finished run's
3265
+ included, long after it released the driver (section 12). Resume on a fresh runtime
3266
+ ([Q371](doc/questions/q371.md)).
3267
+ - **`shutdown()` is not a kill.** It DRAINS: measured in node and deno, a run one settlement in
3268
+ completes inside it, while a retry waiting on its backoff never runs. It waits while tasks are RUNNING
3269
+ and for at most five seconds, so a run with longer work in flight, or only queued or blocked tasks, is
3270
+ left where the wait ended. A crash is what the snippet's
3271
+ `alive` flag stands for — the store stops, whatever the process was doing.
3272
+
3273
+ **`0.5.0` is BREAKING for a consumer of the fold, in two ways.** `FoldInput` has a new kind,
3274
+ `resumed`, so a `switch` that is exhaustive over `kind` no longer compiles, and `RunState` has a new
3275
+ cell, `resumes`, so a `RunState` you construct needs it. A run nobody resumes carries no `resumed`
3276
+ event and `resumes: []`.
3277
+
3278
+ ---
3279
+
3280
+ ## 19. What a node may REMEMBER across branches — a gate
3281
+
3282
+ **A branch cannot see its siblings, and that is usually what you want.** Two instances of one `map` are
3283
+ invisible to each other (section 3's reference rule), a join waits for all of them, and a worker's own
3284
+ memory is lost when its pool tears down or runs it on another thread. So a flow that must route each branch
3285
+ by what the run's OTHER branches have already seen — *keep the first file of each TV series and drop the
3286
+ rest, while every file gets a thumbnail* — had nowhere to keep that memory.
3287
+ ([P24](doc/plans/p24-what-a-node-may-remember-across-a-runs-branches.md) is that ingest.)
3288
+
3289
+ **A `gate` is a decision WITH MEMORY, and the memory is the DRIVER's**
3290
+ ([ADR 0066](doc/decisions/0066-a-gate-is-a-decision-with-memory-that-names-a-registered-type-and-wires-every-route-exactly-once.md),
3291
+ [ADR 0067](doc/decisions/0067-a-gate-runs-in-the-driver-its-memory-is-the-drivers-and-its-decision-is-recorded.md)):
3292
+
3293
+ | | |
3294
+ |---|---|
3295
+ | **it names a gate TYPE** | `distinct` is built in: routes `first` (its DEFAULT), `duplicate` and `nokey`, keyed by `settings.key` — a DOTTED path of object keys over the value that reached the gate, no index, no wildcard, no expression. A key a flow needs COMPUTED is computed by the task before the gate |
3296
+ | **every route is wired, exactly once** | an edge with no `route` carries the type's DEFAULT route, never every route; a DROP is an edge to an `end`, and each is refused at load if it is missing (`FD-V41`) |
3297
+ | **it computes nothing** | the branch leaves on the chosen route with EXACTLY the value that reached the gate |
3298
+ | **its memory is ONE per gate per RUN** | shared by every branch that reaches it, whatever map item or region instance it comes from — and it starts empty with the run and ends with it |
3299
+ | ***first* means first to ARRIVE in this run** | which varies between runs — so a test asserts that exactly ONE branch per key went `first`, never WHICH |
3300
+ | **every decision is recorded** | a `gated` fold input — the gate, the branch path, the route and the key — and `RunState.gated`, in the order the branches arrived |
3301
+ | **a resume REBUILDS the memory** | the replay asks the gate again for every stored settlement that reached it — the recorded decisions are REGENERATED and compared, never fed — so a run killed at any point resumes routing exactly as the run that was killed would have (section 18) |
3302
+
3303
+ ```ts run
3304
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
3305
+ import type { FlowDefinition, GateType, RegistryDocument } from 'fyflow-orchestration';
3306
+
3307
+ const registry: RegistryDocument = {
3308
+ workers: {
3309
+ download: { url: '../../examples/workers/download.ts' }, // in your program: './download.ts'
3310
+ identify: { url: '../../examples/workers/identify.ts' }, // parses `Show.S01E02.mkv` into series `show`
3311
+ thumbnail: { url: '../../examples/workers/thumbnail.ts' },
3312
+ series: { url: '../../examples/workers/series.ts' }, // the per-series work; returns the file that opened it
3313
+ publish: { url: '../../examples/workers/publish.ts' },
3314
+ },
3315
+ nodeTypes: {
3316
+ Download: { worker: 'download', inline: true, pool: { maxThreads: 8 } },
3317
+ Identify: { worker: 'identify', inline: true, pool: { maxThreads: 8 } },
3318
+ Thumbnail: { worker: 'thumbnail', inline: true, pool: { maxThreads: 8 } },
3319
+ SeriesSetup: { worker: 'series', inline: true },
3320
+ Publish: { worker: 'publish', inline: true },
3321
+ },
3322
+ };
3323
+
3324
+ // A map over files. Each branch fetches its file and fans out BESIDE: a thumbnail for EVERY file, and `identify`,
3325
+ // which assigns the SERIES, then the gate. No task could know whether another file of its series came first — the
3326
+ // gate can, because its memory is the run's.
3327
+ const flow: FlowDefinition = {
3328
+ flowId: 'tv-ingest', version: 1, entry: 'episodes',
3329
+ nodes: [
3330
+ { id: 'episodes', kind: 'map', over: 'files', branch: 'download' },
3331
+ { id: 'download', kind: 'task', type: 'Download', edges: [{ to: 'beside' }] },
3332
+ { id: 'beside', kind: 'fanout', branches: ['thumbnail', 'identify'] },
3333
+ { id: 'thumbnail', kind: 'task', type: 'Thumbnail', edges: [{ to: 'perFile' }] },
3334
+ { id: 'identify', kind: 'task', type: 'Identify', edges: [{ to: 'dedupe' }] },
3335
+ // ALL THREE of `distinct`'s routes wired, once each. The edge with no `route` carries its DEFAULT, `first`.
3336
+ { id: 'dedupe', kind: 'gate', type: 'distinct', settings: { key: 'series.id' }, edges: [
3337
+ { to: 'setupSeries' },
3338
+ { route: 'duplicate', to: 'dropDuplicate' }, // a DROP is an edge to an `end`
3339
+ { route: 'nokey', to: 'dropUnidentified' }, // `series.id` resolved to nothing: a name with no series in it
3340
+ ] },
3341
+ { id: 'setupSeries', kind: 'task', type: 'SeriesSetup', edges: [{ to: 'seriesReady' }] },
3342
+ { id: 'seriesReady', kind: 'end', outcome: 'series-ready' },
3343
+ { id: 'dropDuplicate', kind: 'end', outcome: 'duplicate' },
3344
+ { id: 'dropUnidentified', kind: 'end', outcome: 'unidentified' },
3345
+ { id: 'perFile', kind: 'join', of: 'beside', edges: [{ to: 'all' }] },
3346
+ { id: 'all', kind: 'join', of: 'episodes', edges: [{ to: 'publish' }] },
3347
+ { id: 'publish', kind: 'task', type: 'Publish', edges: [{ to: 'done' }] },
3348
+ { id: 'done', kind: 'end', outcome: 'completed' },
3349
+ ],
3350
+ };
3351
+
3352
+ const files = ['Dark.S01E01.mkv', 'Lost.S01E01.mkv', 'Dark.S01E02.mkv', 'trailer.mkv', 'Lost.S01E02.mkv']
3353
+ .map((name) => ({ name, sizeMB: 1 }));
3354
+ const runtime = createRuntime(registry, {}, { base: import.meta.url, flows: [flow] });
3355
+ const state = await runtime.start(flow, { files }).done;
3356
+ await runtime.shutdown();
3357
+
3358
+ console.log(state.outcome); // completed
3359
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
3360
+
3361
+ // EVERY DECISION, in the order the branches reached the gate. WHICH file of a series went `first` is whichever
3362
+ // ARRIVED first in this run — so assert the count per key, never the file.
3363
+ console.log(state.gated.map((d) => `${d.branchPath} ${d.route} ${d.key ?? '-'}`));
3364
+ // e.g. [ '0.1 first dark', '1.1 first lost', '2.1 duplicate dark', '3.1 nokey -', '4.1 duplicate lost' ]
3365
+ const opened = state.gated.filter((d) => d.route === 'first').map((d) => String(d.key)).sort();
3366
+ if (opened.join() !== 'dark,lost') throw new Error(`exactly ONE first per series: ${opened.join()}`);
3367
+ if (state.gated.filter((d) => d.route === 'duplicate').length !== 2) throw new Error('the other two identified files are duplicates');
3368
+ if (state.gated.filter((d) => d.route === 'nokey').length !== 1) throw new Error('the trailer carries no series');
3369
+
3370
+ // The per-series task ran for EXACTLY the branches the gate routed `first` — and received exactly what reached the
3371
+ // gate: `identify`'s whole result, the file name included, which neither the key nor the settings carries.
3372
+ const setups = Object.values(state.nodes).filter((n) => n.nodeId === 'setupSeries');
3373
+ const firsts = state.gated.filter((d) => d.route === 'first');
3374
+ if (setups.map((n) => n.branchPath).sort().join() !== firsts.map((d) => d.branchPath).sort().join()) {
3375
+ throw new Error('setupSeries ran for exactly the branches routed first');
3376
+ }
3377
+ for (const d of firsts) {
3378
+ const r = setups.find((n) => n.branchPath === d.branchPath)!.result as { series: string; openedBy: string };
3379
+ if (r.series !== d.key || r.openedBy !== files[Number(d.branchPath.split('.')[0])]!.name) {
3380
+ throw new Error(`what left the gate is what reached it: ${JSON.stringify(r)}`);
3381
+ }
3382
+ }
3383
+
3384
+ // A DROP IS AN `end`, so it is a path end like any other — and the thumbnail ran for EVERY file: each file's
3385
+ // `perFile` record holds its thumbnail, and nothing from the gate road, which ended at an `end` in every branch.
3386
+ console.log(state.pathEnds.map((p) => p.at).sort());
3387
+ // [ 'done', 'dropDuplicate', 'dropDuplicate', 'dropUnidentified', 'seriesReady', 'seriesReady' ]
3388
+ const published = Object.values(state.nodes).find((n) => n.nodeId === 'publish')!.result as
3389
+ { joinPayload: Record<string, Record<string, { thumbnail?: string }>> };
3390
+ const thumbs = Object.entries(published.joinPayload).map(([i, perFile]) => perFile[`${i}.0`]?.thumbnail);
3391
+ if (thumbs.length !== files.length || thumbs.some((t) => t === undefined)) throw new Error(`a thumbnail per file: ${thumbs.join()}`);
3392
+
3393
+ // A GATE TYPE OF YOUR OWN is CODE, registered by name the way `distinct` is written. A route its `fn` returns that
3394
+ // it did not declare STOPS the run by name — `UndeclaredRoute` (`FD-R5`): no validator can see it, because only
3395
+ // running `fn` reveals it.
3396
+ const sloppy: GateType = { routes: ['keep', 'drop'], defaultRoute: 'keep', fn: () => 'maybe' };
3397
+ const filter: FlowDefinition = {
3398
+ flowId: 'sloppy-filter', version: 1, entry: 'download',
3399
+ nodes: [
3400
+ { id: 'download', kind: 'task', type: 'Download', edges: [{ to: 'filter' }] },
3401
+ { id: 'filter', kind: 'gate', type: 'sloppy', edges: [{ to: 'thumbnail' }, { route: 'drop', to: 'dropped' }] },
3402
+ { id: 'thumbnail', kind: 'task', type: 'Thumbnail', edges: [{ to: 'done' }] },
3403
+ { id: 'dropped', kind: 'end', outcome: 'dropped' },
3404
+ { id: 'done', kind: 'end', outcome: 'completed' },
3405
+ ],
3406
+ };
3407
+ const second = createRuntime(registry, { gateTypes: { sloppy } }, { base: import.meta.url, flows: [filter] });
3408
+ const kinds: string[] = [];
3409
+ const stopped = await second.start(filter, { name: 'Dark.S01E01.mkv', sizeMB: 1 }, { onRunFailure: (f) => kinds.push(f.kind) }).done;
3410
+ await second.shutdown();
3411
+ console.log(stopped.outcome, kinds); // stopped [ 'UndeclaredRoute' ]
3412
+ if (stopped.outcome !== 'stopped' || kinds.join() !== 'UndeclaredRoute' || stopped.gated.length !== 0) {
3413
+ throw new Error(`an undeclared route stops the run by name, recording nothing: ${stopped.outcome} ${kinds.join()}`);
3414
+ }
3415
+ ```
3416
+
3417
+ **What the snippet cannot show, and you need:**
3418
+
3419
+ - **Test the COUNT, never the winner.** On an inline pool the first file to reach a gate is almost always the
3420
+ first file in the list, so a test that names it passes for the wrong reason and fails the day a pool is threaded
3421
+ ([Q379](doc/questions/q379.md)). If a test must show that *first to arrive* is not *first in the list*, ARRANGE
3422
+ the arrival order — `examples/l6.ts` does it with the files' sizes, on a pool as wide as the map — and assert
3423
+ what the arrangement did beside the claim.
3424
+ - **The memory is per RUN, not per region instance.** A gate inside a nested `map` remembers across every outer
3425
+ item; there is no per-season scope. It forgets at the run's end — deduplicating across RUNS is your store's job.
3426
+ - **A gate's type must be the PURE function of its context** a decision predicate is asked to be
3427
+ ([Q9](doc/questions/q9.md)): a resume replays every stored decision through `fn` to rebuild the memory, and a
3428
+ type that answered differently the second time is refused `diverged`. An ENTRY gate's type is also asked once more
3429
+ at `start()`, against an empty throwaway memory, to hold the payload to the target it chooses (`FD-R4`,
3430
+ [Q387](doc/questions/q387.md)).
3431
+ - **Keep your gate's memory in the gate.** A decision predicate that remembers in a closure looks like a gate and is
3432
+ not one: its memory lives in your process, a resume cannot rebuild it, and a run killed before its first decision
3433
+ resumes WRONG in silence ([Q386](doc/questions/q386.md)).
3434
+ - **`checkWiring` reads a route's GUARANTEE, where its type declares one.** A gate type may name the routes on
3435
+ which its key path resolved — `distinct` names `first` and `duplicate` — and what leaves the gate on such a
3436
+ route is what arrived with `required` added along the gate's key path, at every step. So a worker after `first`
3437
+ that REQUIRES the key `fits`. *It read `no` there until v0.25, which is [Q391](doc/questions/q391.md) and is
3438
+ [ADR 0068](doc/decisions/0068-a-route-guarantees-what-its-type-declares-and-the-library-derives-it-from-the-key-path.md).*
3439
+ - **A route that guarantees something else guarantees nothing a check can use.** A type declaring no
3440
+ `keyResolvedRoutes`, a gate naming no `key`, and a route the type does not name each pass the value through
3441
+ exactly as before — and the guarantee is PRESENCE along the path, never the leaf's TYPE, which is the type's own
3442
+ judgement (`distinct` wants a string or a finite number, and no static tool derives that).
3443
+
3444
+ **`0.6.0` is BREAKING in four ways, all of them for code that enumerates or matches.** `FlowNode` has a SIXTH kind,
3445
+ `GateNode` ([Q381](doc/questions/q381.md)), so a `switch` exhaustive over `node.kind` no longer compiles;
3446
+ `FoldInput` has a new kind, `gated`, so one exhaustive over `input.kind` no longer compiles either; `RunState` has a
3447
+ new cell, `gated`, so a `RunState` you construct needs it; and a `route` key on the edge of a node that is not a gate
3448
+ was refused as `FD-V17` and is refused as `FD-V41` — the same document refused under another rule id, so code
3449
+ matching a finding's `rule` moves. A flow with no gate records no `gated` input and has `gated: []`.
3450
+
3451
+ ## 20. What a ROUTE buys an AUTHOR — wiring a gate before it runs
3452
+
3453
+ **Section 15 showed what a schema buys whoever is wiring a flow. A gate adds something a schema alone
3454
+ cannot say.** A gate routes by a KEY PATH into the value reaching it — `owner.id`, `series.id` — and two
3455
+ things follow that an author has to get right before anything runs:
3456
+
3457
+ 1. **The path has to be one the producer can actually carry.** Misspell it and the key resolves for no
3458
+ branch, so every branch takes whichever route the gate's TYPE gives an unresolved key: the run
3459
+ completes, having done none of the work the gate guards. Nothing about the document is wrong —
3460
+ `owner.ident` is a perfectly good string — so the grammar accepts it, and step 5 below RUNS that
3461
+ case rather than describing it. *(Which route an unresolved key takes is the type's judgement, not
3462
+ the library's: `FD-V41` requires only that every route a type declares is wired exactly once, and a
3463
+ type need not declare a no-key route at all. `distinct` and the `quota` below both do.)*
3464
+ 2. **On the routes where the path DID resolve, the value carries more than its producer promised.** A
3465
+ producer may declare `owner` optional; a branch leaving by a key-resolved route has an `owner`, or it
3466
+ would not have left that way. A consumer requiring it is correctly wired, and only the route says so.
3467
+
3468
+ **`checkWiring` answers both, and it answers them from the gate TYPE's own declaration**
3469
+ ([ADR 0068](doc/decisions/0068-a-route-guarantees-what-its-type-declares-and-the-library-derives-it-from-the-key-path.md)):
3470
+ a type lists the routes on which its key path resolved — `keyResolvedRoutes` — and the library derives
3471
+ the guarantee from the path. `distinct` declares `first` and `duplicate`; the type below declares two of
3472
+ its three routes, so the third is the control sitting in the same report.
3473
+
3474
+ | the author writes | the check says |
3475
+ |---|---|
3476
+ | a key path the producer FORBIDS at any step | **`no`** on every arc leaving by a key-resolved route, naming the gate, the key, the STEP it stopped on, the word the schema itself uses, and the route |
3477
+ | a key path the producer describes | those arcs carry `required` along the path, so a consumer requiring it **`fits`** |
3478
+ | the same arc, with the type declaring no guarantee | **`no`** again — *consumer requires "owner", producer does not guarantee it* |
3479
+ | an arc leaving by a route the type does NOT call key-resolved | its own verdict, unchanged — the refusal is the ROUTE's and not the gate's |
3480
+
3481
+ **The type never walks the key path.** The library resolves it and hands the value over as `ctx.key`,
3482
+ which is why the library can derive what a route guarantees while the type decides what to do with it —
3483
+ including deciding that a path which resolved to the empty string is not a key, which is one of the two
3484
+ roads to `nokey` the run below takes.
3485
+
3486
+ **The registry, the gate type and the flow are
3487
+ [`examples/getting-started/quota-registry.ts`](examples/getting-started/quota-registry.ts)**, imported
3488
+ rather than written out here, for the reason that file's header gives: the session needs a second
3489
+ observer and a snippet cannot be one, so `test/definition/gate-session.test.ts` asserts the same
3490
+ verdicts from the same declarations.
3491
+
3492
+ ```ts run
3493
+ import { checkWiring, distinct } from 'fyflow-orchestration';
3494
+ import type { GateType } from 'fyflow-orchestration';
3495
+ import { createRuntime } from 'fyflow-orchestration/scheduler';
3496
+ // Everything the session claims lives in ONE module, so a TEST asserts the same verdicts from the same
3497
+ // declarations — `test/definition/gate-session.test.ts`. A snippet cannot be its own second observer.
3498
+ import { QUOTA, QUOTA_BASE, QUOTA_FILES, QUOTA_REGISTRY, QUOTA_WIRING, quotaFlow } from '../../examples/getting-started/quota-registry.ts';
3499
+
3500
+ // `report` has exactly ONE arc — the one leaving by `within`, the route the quota guards.
3501
+ const arc = async (key: string) => {
3502
+ const verdicts = (await checkWiring(QUOTA_WIRING, quotaFlow(key))).verdicts;
3503
+ const mine = verdicts.filter((v) => v.consumer === 'report');
3504
+ if (mine.length !== 1) throw new Error(`one arc into report, not ${mine.length}`);
3505
+ return mine[0]!;
3506
+ };
3507
+
3508
+ // ---- 1. the author misspells the key. Nothing is wrong with the DOCUMENT. ----
3509
+ const typo = await arc('ownr.id');
3510
+ console.log(typo.kind, '—', typo.why);
3511
+ // no — gate "quota" is keyed on "ownr.id", which the producer's schema FORBIDS at $: the schema forbids
3512
+ // every key it does not name (additionalProperties: false) and it names file, owner — so route "within"
3513
+ // can never be taken, and a run may route every branch by another route, doing none of the work this gate guards
3514
+ if (typo.kind !== 'no') throw new Error(`a forbidden key path is a wiring failure, got ${typo.kind}`);
3515
+ const NAMED = ['is keyed on "ownr.id"', 'FORBIDS at $:', 'it names file, owner', 'route "within"'];
3516
+ if (NAMED.length !== 4) throw new Error('the reason names FOUR things and each is asserted');
3517
+ for (const part of NAMED) {
3518
+ if (part === '' || !typo.why.includes(part)) throw new Error(`the reason names the gate, the key, the step, the schema's own words and the route: ${part}`);
3519
+ }
3520
+
3521
+ // ---- 2. one step in, the reason moves with it ----
3522
+ const deeper = await arc('owner.ident');
3523
+ if (!deeper.why.includes('FORBIDS at $.owner:') || !deeper.why.includes('it names id, name')) {
3524
+ throw new Error(`the STEP it stopped on is where to look: ${deeper.why}`);
3525
+ }
3526
+
3527
+ // ---- 3. the refusal belongs to the ROUTE. `skip` has two arcs and two verdicts at once. ----
3528
+ const both = (await checkWiring(QUOTA_WIRING, quotaFlow('ownr.id'))).verdicts.filter((v) => v.consumer === 'skip');
3529
+ console.log(both.map((v) => v.kind)); // [ 'no', 'fits' ]
3530
+ // A verdict carries no `route` field (Q410), so the ROUTE is read from the reason the refusal itself
3531
+ // gives: the `no` arc names `over`, and swapping which routes are key-resolved is caught here.
3532
+ const refused = both.filter((v) => v.kind === 'no');
3533
+ if (refused.length !== 1 || !refused[0]!.why.includes('route "over"')) {
3534
+ throw new Error('exactly the over arc is refused and its reason says so: ' + both.map((v) => v.kind + ' ' + v.why).join(' | '));
3535
+ }
3536
+ if (both.filter((v) => v.kind === 'fits').length !== 1) throw new Error('and the nokey arc, which the type does not call key-resolved, keeps its own verdict');
3537
+
3538
+ // ---- 4. spelled right, it fits — and the GUARANTEE is what makes it fit ----
3539
+ const right = await arc('owner.id');
3540
+ if (right.kind !== 'fits') throw new Error(`the path the producer describes: ${right.why}`);
3541
+ const { keyResolvedRoutes: _none, ...noGuarantee } = QUOTA;
3542
+ const blind = (await checkWiring({ ...QUOTA_WIRING, gateTypes: { quota: noGuarantee as GateType } }, quotaFlow('owner.id'))).verdicts;
3543
+ const unguaranteed = blind.find((v) => v.consumer === 'report')!;
3544
+ console.log(unguaranteed.kind, '—', unguaranteed.why); // no — $: consumer requires "owner", producer does not guarantee it
3545
+ if (unguaranteed.kind !== 'no' || !unguaranteed.why.includes('does not guarantee it')) {
3546
+ throw new Error('with no declared guarantee the same arc is `no`, which is what the declaration buys');
3547
+ }
3548
+ // The built-in type declares its own two, which is where this member came from.
3549
+ if ([...(distinct.keyResolvedRoutes ?? [])].join() !== 'first,duplicate') throw new Error('`distinct` declares first and duplicate');
3550
+
3551
+ // ---- 5. and it runs. ONE run, so the quota is shared by every branch. ----
3552
+ // `base` is the REGISTRY's own url, never the caller's: its worker urls are relative to the module that
3553
+ // declares them. Point it at the wrong base and the workers cannot load: every task of each such type fails
3554
+ // naming the worker URL and the load error — on a non-optional node it is parked user_action, so `done` waits
3555
+ // on a person (W141, Q266) — so this is a line to get right rather than to debug.
3556
+ const runtime = createRuntime(QUOTA_REGISTRY, { gateTypes: { quota: QUOTA } }, { base: QUOTA_BASE });
3557
+ const { done } = runtime.start(quotaFlow('owner.id'), { files: QUOTA_FILES });
3558
+ const state = await done;
3559
+
3560
+ console.log(state.outcome, state.gated.map((d) => `${d.key ?? '-'} ${d.route}`));
3561
+ // completed [ 'ada within', 'ada within', 'ada over', 'grace within', ' nokey', '- nokey' ]
3562
+ if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
3563
+ const routes = state.gated.map((d) => d.route).sort();
3564
+ if (routes.join() !== 'nokey,nokey,over,within,within,within') throw new Error(`two within the quota per owner, ada's third over, and two roads to nokey: ${routes.join()}`);
3565
+ const over = state.gated.filter((d) => d.route === 'over');
3566
+ if (over.length !== 1 || over[0]!.key !== 'ada') throw new Error('the owner with three files is the one who goes over');
3567
+ // BOTH roads to `nokey`: a path that RESOLVES to the empty string, which the TYPE refuses, and a path
3568
+ // that does not resolve at all, which the library never resolved.
3569
+ const nokeys = state.gated.filter((d) => d.route === 'nokey').map((d) => (d.key === '' ? 'empty' : d.key === undefined ? 'unresolved' : `?${String(d.key)}`)).sort();
3570
+ if (nokeys.join() !== 'empty,unresolved') throw new Error(`the two roads to nokey are both taken: ${nokeys.join()}`);
3571
+ // And the work the guarantee licensed actually HAPPENED: the run completing is not evidence on its own.
3572
+ const did = Object.values(state.nodes).filter((n) => n.nodeId === 'report').map((n) => (n.result as { owner: string }).owner).sort();
3573
+ if (did.join() !== 'ada,ada,grace') throw new Error(`three reports ran, and each read the owner the route guaranteed: ${did.join()}`);
3574
+
3575
+ // ---- and the run the MISSPELLED key gives, which is what being wrong costs ----
3576
+ const wrong = await (runtime.start(quotaFlow('ownr.id'), { files: QUOTA_FILES })).done;
3577
+ await runtime.shutdown();
3578
+ console.log(wrong.outcome, wrong.gated.map((d) => d.route)); // completed [ 'nokey', 'nokey', ... ]
3579
+ if (wrong.outcome !== 'completed') throw new Error(`the run COMPLETES, which is what makes this worth checking: ${wrong.outcome}`);
3580
+ if (wrong.gated.some((d) => d.route !== 'nokey')) throw new Error('every branch takes the unresolved route');
3581
+ if (Object.values(wrong.nodes).some((n) => n.nodeId === 'report')) throw new Error('and NONE of the work the gate guards runs');
3582
+ ```
3583
+
3584
+ **What the check does NOT say, each measured rather than assumed.** An **entry gate** takes no verdict at
3585
+ all — it has no arc, so a key path misspelled there is caught by nothing this report carries
3586
+ ([Q416](doc/questions/q416.md)). A gate wired into a **join** is neither refined nor refused, because a
3587
+ join's record is built from what arrived AT the gate ([Q415](doc/questions/q415.md)). A container declared
3588
+ by `enum` or `const` rather than by `properties` can forbid a step in silence ([Q417](doc/questions/q417.md)).
3589
+ A verdict carries **no `route` field** ([Q410](doc/questions/q410.md)), so which route an arc left by is
3590
+ read from the reason a refusal gives rather than from the report's shape — which is why step 3 asserts
3591
+ the reason and not the order. And a type that declares **no** `keyResolvedRoutes` is asked no question
3592
+ about its key path at all — which is step 4's control, and the reason the declaration is worth writing.
3593
+
3594
+ ## 21. A flow a UI could have emitted — what a document can say without code
3595
+
3596
+ **Sections 4 and 13 gave you the two halves: a `when` that may be DATA and a built-in type that only
3597
+ transforms. This is what they add up to.** A flow document plus a registry document, and the only
3598
+ TypeScript is the domain workers and one import of the language adapter the library itself ships.
3599
+
3600
+ **What that covers**, which is [P26](doc/plans/p26-what-a-flow-can-say-without-code.md) prediction 3
3601
+ and [W110](doc/work-items/w110-a-flow-a-ui-could-have-emitted.md)'s to answer:
3602
+
3603
+ | what a flow has to say | how, in data | what it costs in code |
3604
+ |---|---|---|
3605
+ | **sequence** | `edges: [{ to }]` | nothing |
3606
+ | **fan-out** at run-time arity | a `map` over a key | nothing |
3607
+ | **join** | a `join` naming its opener | nothing |
3608
+ | **selection** — give this node that node's output | `needs` on the consuming node (tier 1) | nothing |
3609
+ | **decision** | a `when`: a comparison or a flat `all`/`any` group | nothing |
3610
+ | the same decision, when the question is **outside the subset** | a `when` naming a **registered predicate** | **a function** — this is the half that is not data, which is P26's own framing |
3611
+ | **the join policy** a run is started with | nothing in the document: it is an argument to `runtime.start`, defaulted to `live-token` | the library's own, or a consumer's |
3612
+ | **computation feeding a worker** | `transform` on that worker's own node (tier 2) | the **language adapter** |
3613
+ | **computation feeding a DECISION or a join** | a `Transform` node (tier 3) | the **language adapter** |
3614
+
3615
+ **Two narrowings, both measured, because the table above is the interesting part and the honest part
3616
+ is underneath it.**
3617
+
3618
+ 1. **Every `transform` names a `lang`, and no language ships registered.** `Registry.languages` is
3619
+ EMPTY when absent, so a transform against a registry naming none is `FD-V35`. The adapter is the
3620
+ library's own sub-path — `import { jsonata } from 'fyflow-orchestration/jsonata'` — so it is one
3621
+ import and one object entry with no logic in it, but it is not nothing and it is not a domain
3622
+ worker. **A flow that only SELECTS needs no adapter; a flow that COMPUTES does.**
3623
+ 2. **A comparison reads ONE path and compares it to ONE literal.** It cannot name a second path and
3624
+ it cannot do arithmetic, so *is the mean below the threshold this run carries* is not a comparison.
3625
+ **The route around it is data**: a tier-three `Transform` node computes the answer and the `when`
3626
+ reads `outcome.result.<field>`. That is what the flow below does, and it is the one place the
3627
+ three-tier rule's *reshape for … a decision* is load-bearing rather than advisory — a `transform`
3628
+ transforms its node's INPUT and a decision reads its node's OUTPUT, so moving the computation onto
3629
+ the consuming node would leave the decision unable to see it.
3630
+
3631
+ > **A comparison whose `value` is meant to be a path is NOT refused** — `{ "op": "eq", "value":
3632
+ > "needs.cfg.threshold" }` is a perfectly good string comparison against the literal text, and it is
3633
+ > false for ever. The grammar catches a wrong ROOT (`FD-V43`) and nothing catches this
3634
+ > ([Q456](doc/questions/q456.md)). Write the literal, or compute the comparison in a `Transform` node.
3635
+
3636
+ > **This flow is `examples/flows/ui-emitted.flow.json` and this registry is
3637
+ > `examples/ui-registry.json`, with the SCHEMAS reduced to what these three verdicts turn on.** The
3638
+ > nodes, the edges, the `needs`, both transforms and both groups are the shipped document's; the
3639
+ > worker entries carry two schemas each instead of the corpus's full set, because the point here is
3640
+ > which arcs are checked and not what the corpus declares. **Nothing compares the two copies** — a
3641
+ > check would have to hold a TypeScript literal against a JSON document, where `X8` has byte-equality
3642
+ > to work with and this has none — so they are two copies and this sentence is what says so
3643
+ > ([Q458](doc/questions/q458.md)). The shipped pair is what ladder L6 RUNS, on real workers, in both
3644
+ > runtimes; this is what a reader can read.
3645
+
3646
+ ```ts run
3647
+ import { checkWiring, validateDefinition, type FlowDefinition, type Registry } from 'fyflow-orchestration';
3648
+ // **The ONE import that is code**, and it is the library's own sub-path rather than anything you write.
3649
+ import { jsonata } from 'fyflow-orchestration/jsonata';
3650
+
3651
+ // ---- the flow a UI emits. No predicate names it, and nothing here is a function. ----
3652
+ const flow: FlowDefinition = {
3653
+ flowId: 'ui-emitted', version: 1, entry: 'intake',
3654
+ nodes: [
3655
+ { id: 'intake', kind: 'task', type: 'SplitWorker', edges: [{ to: 'each' }] },
3656
+ { id: 'each', kind: 'map', over: 'items', branch: 'rate' },
3657
+ { id: 'rate', kind: 'task', type: 'ScoreWorker', edges: [{ to: 'gather' }] },
3658
+ { id: 'gather', kind: 'join', of: 'each', edges: [{ to: 'rollup' }] },
3659
+ // TIER THREE — the computation a comparison cannot express, as a node with NO worker.
3660
+ {
3661
+ id: 'rollup', kind: 'task', type: 'Transform',
3662
+ needs: { joined: '.', cfg: 'intake' },
3663
+ transform: {
3664
+ lang: 'jsonata',
3665
+ expr: '( $s := [joined.*.score]; { "n": $count($s), "total": $sum($s), "mean": $sum($s) / $count($s), '
3666
+ + '"weak": ($sum($s) / $count($s)) < (cfg.confidence * 100) } )',
3667
+ },
3668
+ edges: [
3669
+ // A flat GROUP, and both members decide: `weak` on one payload, `n` at one item.
3670
+ { when: { all: [{ path: 'outcome.result.weak', op: 'eq', value: true }, { path: 'outcome.result.n', op: 'gte', value: 2 }] }, to: 'remediate' },
3671
+ { when: { any: [{ path: 'outcome.result.n', op: 'eq', value: 1 }, { path: 'outcome.result.weak', op: 'eq', value: false }] }, to: 'publish' },
3672
+ // The third edge is a TRIPWIRE: a wrong group lands at a named `end` rather than falling through.
3673
+ { to: 'stalled' },
3674
+ ],
3675
+ },
3676
+ // TIER TWO — a transform on the node whose worker consumes it, so no extra task.
3677
+ {
3678
+ id: 'remediate', kind: 'task', type: 'WeighWorker',
3679
+ transform: { lang: 'jsonata', expr: '{ "item": { "word": "mean", "confidence": mean / 100 }, "run": { "confidence": 1 } }' },
3680
+ edges: [{ when: { path: 'outcome.result.score', op: 'lt', value: 20 }, to: 'escalated' }, { to: 'done' }],
3681
+ },
3682
+ { id: 'publish', kind: 'task', type: 'PublishWorker', edges: [{ to: 'done' }] },
3683
+ { id: 'escalated', kind: 'end', outcome: 'escalated-to-a-person' },
3684
+ { id: 'stalled', kind: 'end', outcome: 'no-branch-matched' },
3685
+ { id: 'done', kind: 'end', outcome: 'completed' },
3686
+ ],
3687
+ };
3688
+
3689
+ // ---- the registry it names. `Transform` is NOT in it: a built-in is merged at every read. ----
3690
+ const registry: Registry = {
3691
+ // **EMPTY, and that is the claim.** A declared `when` resolves nothing against this.
3692
+ predicates: [],
3693
+ workers: {
3694
+ split: {
3695
+ url: './split.ts',
3696
+ outputSchema: {
3697
+ type: 'object', required: ['confidence', 'items'],
3698
+ properties: {
3699
+ confidence: { type: 'number' },
3700
+ // The ITEM's own shape, so the arc into the map is checked — without it that arc is
3701
+ // `unchecked` for a reason that has nothing to do with the transform node.
3702
+ items: { type: 'array', items: { type: 'object', required: ['word', 'confidence'], properties: { word: { type: 'string' }, confidence: { type: 'number' } } } },
3703
+ },
3704
+ },
3705
+ },
3706
+ score: {
3707
+ url: './score.ts',
3708
+ inputSchema: { type: 'object', properties: { word: { type: 'string' }, confidence: { type: 'number' } } },
3709
+ outputSchema: { type: 'object', required: ['score'], properties: { score: { type: 'number' } } },
3710
+ },
3711
+ weigh: { url: './weigh.ts', inputSchema: { type: 'object', properties: { item: { type: 'object' }, run: { type: 'object' } } } },
3712
+ publish: { url: './publish.ts' },
3713
+ },
3714
+ nodeTypes: {
3715
+ SplitWorker: { worker: 'split' }, ScoreWorker: { worker: 'score' },
3716
+ WeighWorker: { worker: 'weigh' }, PublishWorker: { worker: 'publish' },
3717
+ },
3718
+ // The one thing that is code: the adapter for the language the transforms name.
3719
+ languages: { jsonata },
3720
+ };
3721
+
3722
+ const findings = validateDefinition(flow, registry);
3723
+ console.log(findings.length); // 0
3724
+ if (findings.length !== 0) throw new Error(`a UI-emitted flow validates with NO predicate registered: ${JSON.stringify(findings)}`);
3725
+
3726
+ // **The registry really is empty** — an unused empty map would prove nothing (W110's careful-point 1).
3727
+ if ((registry.predicates as readonly string[]).length !== 0) throw new Error('no predicate is registered anywhere');
3728
+
3729
+ // ---- and what the author gets told about the wiring, which is where the residue is ----
3730
+ const report = await checkWiring(registry, flow);
3731
+ console.log(report.counts); // { fits: 1, no: 0, ..., unchecked: 3 }
3732
+ // `intake -> rate` is the one arc neither end of which is the transform node, and it FITS.
3733
+ const leaving = report.verdicts.filter((v) => v.producer === 'rollup').map((v) => v.kind);
3734
+ console.log(leaving); // [ 'unchecked', 'unchecked' ]
3735
+ if (leaving.length !== 2 || leaving.some((v) => v !== 'unchecked')) {
3736
+ throw new Error(`both arcs LEAVING the transform-only node are unchecked — it declares no outputSchema, which is Q446: ${JSON.stringify(leaving)}`);
3737
+ }
3738
+ // The arc INTO it is unchecked for the mirror reason — no inputSchema — so the residue is THREE arcs.
3739
+ if (report.counts.unchecked !== 3) throw new Error(`three arcs touch the node and all three are unchecked: ${report.counts.unchecked}`);
3740
+ // And the arcs that do not touch it are checked as any other arc is, so the residue is LOCAL to the node.
3741
+ if (report.counts.fits !== 1 || report.counts.no !== 0) throw new Error(`the rest of the flow is checked as usual: ${JSON.stringify(report.counts)}`);
3742
+ ```
3743
+
3744
+ **What the check cannot tell you here, said rather than left to be discovered.** Every arc LEAVING a
3745
+ transform-only node is `unchecked`, because the node declares no `outputSchema` and a sample is
3746
+ evidence about one input where `fits` is a claim about every input
3747
+ ([Q446](doc/questions/q446.md), and [ADR 0075](doc/decisions/0075-a-node-that-only-transforms-is-a-built-in-type-with-no-inner-worker.md)
3748
+ rule 6 is why it is not inferred). `sampledOutput` in the assists gives you the shape the node
3749
+ actually emitted on the producer's own example, with those verdicts beside it — so the residue is
3750
+ reported rather than hidden. And a declared predicate's PATH is checked against no schema at all, so
3751
+ a path naming a key the producer cannot emit answers *no match* for ever (ADR 0074 consequence 6).
3752
+
3753
+ > **A `Transform` node's realm and its groups are the scheduler's default and nothing else.** The
3754
+ > built-in declares no `inline`, no `groups` and no `pool`, and a named pool instance carries only
3755
+ > `type` and `config` — so of the four things ADR 0075 rule 3 lists, a named instance reaches
3756
+ > `config`, a `retryPolicy` goes on the NODE, and `inline` and `groups` are reachable by no road a
3757
+ > consumer can write ([Q457](doc/questions/q457.md)). Measured at W110.