fyflow-orchestration 0.10.0 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENTS.md CHANGED
@@ -218,11 +218,12 @@ if (v48.join() !== 'FD-V48') throw new Error(`expected FD-V48, got ${v48.join()}
218
218
  >
219
219
  > **What each instance receives, and what the join hands on.** Each instance is dispatched with
220
220
  > **its own item** — `over`'s array at that instance's index, as its own copy, so no two instances
221
- > share an object — and the join carries **every item's result keyed by that item's branch path**. A
222
- > map of three items at the top level dispatches three tasks and its successor receives
223
- > `{ "0": …, "1": …, "2": … }`; **a map nested in a fan-out branch keys its collection with the whole
224
- > path**, so a map inside branch 1 gives `{ "1.0": …, "1.1": … }`. *(This said "keyed by its item
225
- > index", which is the depth-1 case stated as if it were general — W34's conformity reader.)*
221
+ > share an object — and the join carries **every item's result keyed by that item's INDEX**, one
222
+ > segment, wherever the map sits: a map of three items dispatches three tasks and its successor
223
+ > receives `{ "0": …, "1": …, "2": … }`, and so does a map nested in a fan-out branch. **Since 0.12.0**
224
+ > ([ADR 0138](doc/decisions/0138-a-join-hands-on-each-branch-under-its-own-key-and-a-checkpoint-of-the-old-keying-is-refused-by-format.md)): until then a nested map keyed its collection with the whole path — a map
225
+ > inside branch 1 gave `{ "1.0": …, "1.1": … }` — and *What a join carries downstream* in section 5 says
226
+ > what moved with it.
226
227
  > [ADR 0037](doc/decisions/0037-what-a-branch-receives-is-a-property-of-the-fan-out-and-what-a-join-carries-is-a-property-of-the-region.md),
227
228
  > and see *What a join carries downstream* in section 5 for what happens when an item's branch does
228
229
  > not arrive.
@@ -292,7 +293,7 @@ if (!noGroups.some((f) => f.rule === 'FD-V31' && f.at === 'score')) {
292
293
  | `join` | `of: '<fanout id>'`. Fires when its policy says so | it does not have **exactly one** edge, or that edge carries `when` or `on` (`FD-V7`) |
293
294
  | `end` | terminal. `outcome` is **your label**, never interpreted | — |
294
295
  | `ask` | waits for a PERSON — `edges`, and optionally `settings` (opaque), `answerSchema` (what a `done` answer must satisfy) and `needs` (what reaches it). Never dispatched, and has a task id; its answer is its output. A run PARKS at one and takes the person's answer at `resolve()` (section 7) | it has no `edges`, or **no edge of it is eligible for `done`** (`FD-V7`, as a task); `settings` that is not a plain object (`FD-V21`); a key only a task declares — `type`, `pool`, `optional`, `retryPolicy`, `workerGroups`, `transform` — one finding per key (`FD-V47`); an `answerSchema` outside the schema subset (`FD-V48`) |
295
- | `gate` | `type: '<gate type>'`, optional `settings`, and `edges` of `{ to, route? }` — routes the branch that reaches it and hands on what reached it. `settings.key` is a dotted key path over that value (`series.id`; no index). Built in: `distinct`, `if` and `switch`. Run by the driver where a branch enters it (section 19) | its `type` is not a registered gate type, or its type's `routesFor` throws or returns a malformed route set for this gate, or its `settingsProblem` throws or returns neither a message nor `null` (`FD-V39`); its `settings` are not a plain object its type's schema accepts, its type's `settingsProblem` returns a message, they cannot be copied for the type, or its `key` is not a key path (`FD-V40`); a route is unwired, wired twice, or not its type's, or a gate edge carries `when`, `on` or `needs` (`FD-V41`) — and `route` on any other node's edge is `FD-V41` too |
296
+ | `gate` | `type: '<gate type>'`, optional `settings`, and `edges` of `{ to, route? }` — routes the branch that reaches it and hands on what reached it. `settings.key` is a dotted key path over that value — `series.id`, and since 0.12.0 an array element by a bracket index, `files[0].series` (a plain `xs.1` is still a KEY, never an index). Built in: `distinct`, `if` and `switch`. Run by the driver where a branch enters it (section 19) | its `type` is not a registered gate type, or its type's `routesFor` throws or returns a malformed route set for this gate, or its `settingsProblem` throws or returns neither a message nor `null` (`FD-V39`); its `settings` are not a plain object its type's schema accepts, its type's `settingsProblem` returns a message, they cannot be copied for the type, or its `key` is not a key path (`FD-V40`); a route is unwired, wired twice, or not its type's, or a gate edge carries `when`, `on` or `needs` (`FD-V41`) — and `route` on any other node's edge is `FD-V41` too |
296
297
 
297
298
  **A `task` node — and an `ask` — must have an edge that can be taken when it succeeds.** A node whose only edge is
298
299
  `{ when: { predicate: 'p' }, to: 'x' }` is refused at construction — it is the mistake the format most invites, and
@@ -347,6 +348,7 @@ that object and nothing else — there is no wrapper and no reserved key.
347
348
  | `"someNode"` | that node's result, **on your own branch** |
348
349
  | `["someNode"]` | that node's results as an **array**, in ascending branch-path order. Legal only when you are OUTSIDE a region that node is in — which is what makes it more than one |
349
350
  | **a `?` on either of those** — `"someNode?"`, `["someNode?"]` | the same thing, except that **nothing to give leaves the key out** (or gives `[]` for the array form) instead of stopping the run. This is how you say *if it ran* |
351
+ | **`!error` after the node id, on any of those** — `"someNode!error"`, `["someNode!error?"]`, … | that node's **ERROR** instead of its result: the string its failure was reduced to, `null` where it succeeded (since 0.11.0, below) |
350
352
 
351
353
  **What a reference hands you is ONE shared, frozen copy — read it, never write it** — from 0.10.0 ([ADR 0126](doc/decisions/0126-a-references-value-is-one-frozen-copy-shared-by-every-reader.md),
352
354
  `spec/flow-definition.md` v0.47). The value under a `"someNode"` or `["someNode"]` key (either with `?`) is ONE deep copy
@@ -521,7 +523,7 @@ that passed it through.
521
523
  | where it is legal | every node and every edge — at the entry, on a task, it equals `"."`, and both spellings are accepted. **On an edge it is NOT refused** the way `"."` is: a predicate has no `ctx.payload`, so `needs: { run: '$payload' }` is how a decision routes on `needs.run.amount` with no task before it |
522
524
  | always there | no `?` and no array form: `["$payload"]` and `"$payload?"` are `FD-V26`, each naming the bare form. The whole `$` prefix is reserved — `$` is outside the node-id charset (`FD-V16`), so the token can never collide with a node — and any other `$` spelling (`"$input"`, a typo) is `FD-V26` naming the two `$` references |
523
525
  | `"$item"` | the prefix's second reference, the item of the map instance the reader is in — the next subsection |
524
- | on a resume | the same value, from the stored header: nothing new is written to the log, the header or the checkpoint (`CHECKPOINT_FORMAT` stays 1) |
526
+ | on a resume | the same value, from the stored header: nothing new is written to the log, the header or the checkpoint (`CHECKPOINT_FORMAT` does not move for it — it is 2 since 0.12.0, for the join's keys, section 5) |
525
527
  | at `start()` | `FD-R4` holds a task's composed `"$payload"` key to its worker's `inputSchema` like any other key (section 9) — composed from the value you passed, before the run exists |
526
528
  | in the wiring check | an arc composed through a `"$payload"` key is `unchecked`: the run's payload is a value no schema declares (section 15) |
527
529
  | what it costs | ONE copy of the whole payload per RUN, from 0.10.0 — however many instances of a `map` read it. *(Until 0.10.0 one copy per key per read: inside a `map`, once per instance — the cliff above at its largest.)* On an ASK the copy is also written into the run's LOG: the `asked` element keeps what reached the ask (section 7) |
@@ -616,7 +618,7 @@ passed it through.
616
618
  | always there | no `?` and no array form: `["$item"]` and `"$item?"` are `FD-V26`, each naming the bare form |
617
619
  | what it is | its OWN deep copy of the item, per key per read — two `"$item"` keys on one node are two objects, and a worker or a predicate writing into one reaches nothing else |
618
620
  | what an instance KEEPS | its item, by reference to the element the map received — and only where some node of that map's region reads `"$item"`: a map nobody reads it from keeps nothing. **Released when nothing of that instance may still be entered** — its branch has ended (arrived at the map's join, exited the region, or been lost) AND no branch of a region nested inside it is still live — told by those branches' terminal region events, never by a join: an ask parked in the instance keeps it until the person answers, and a nested fan-out whose join a policy fired early keeps it until its last branch ends. Released whole when the run halts, is cancelled or is stopped (`spec/run-state.md` `RN-R13`) |
619
- | on a resume | rebuilt by the replay, which re-enters the map: nothing new is written to the log, the header or the checkpoint (`CHECKPOINT_FORMAT` stays 1) |
621
+ | on a resume | rebuilt by the replay, which re-enters the map: nothing new is written to the log, the header or the checkpoint (`CHECKPOINT_FORMAT` does not move for it — it is 2 since 0.12.0, for the join's keys, section 5) |
620
622
  | at `start()` | `FD-R4` composes it on an ENTRY map's branch task from the element the walk reached — which IS that instance's item — and holds it to the worker's `inputSchema` like any key (section 9). *Until 0.8.0's W191 ANY `needs` on such a task — `"."` included — made `start()` throw an `UnresolvedBranchPath` instead of `BoundaryRefused`* |
621
623
  | in the wiring check | an arc composed through a `"$item"` key reads the enclosing map's ITEM schema, one sample per element; `unchecked` for an ENTRY map, whose items come from the run's payload, which no schema declares (section 15) |
622
624
  | what it costs | one copy of ONE element per key per read — the cheap end of section 3's cliff — and the retention above, at most the `over` array the log already holds |
@@ -691,6 +693,83 @@ const [finding] = validateDefinition(outside, registry);
691
693
  if (finding?.rule !== 'FD-V49' || !finding.detail.includes('move this node inside the map')) throw new Error(`expected FD-V49, got ${finding?.rule}`);
692
694
  ```
693
695
 
696
+ ### `"someNode!error"` — what FAILED, from the nodes after it
697
+
698
+ **`"lookup!error"` names node `lookup`'s ERROR** — the outcome's `error`, the string the scheduler reduced the failure to
699
+ (section 6) — where `"lookup"` names its result (`spec/flow-definition.md` v0.49,
700
+ [ADR 0131](doc/decisions/0131-a-reference-may-name-a-nodes-error.md), since 0.11.0). A failed task's result is `null`, so
701
+ until it a node after a failure edge — an ask asking a person *what now?* above all — could not say what failed.
702
+
703
+ | | |
704
+ |---|---|
705
+ | what it is | the effective outcome's `error`: the record's, or a RESOLUTION's where a person resolved the node (`resolve(id, 'failed', { error })` on a parked task, a declined ask; `'resolved as failed'` where they gave none). **`null` where the output SUCCEEDED** — the key is present, holding `null` |
706
+ | its forms | the plain reference's four: `"lookup!error"`, `"lookup!error?"`, `["lookup!error"]`, `["lookup!error?"]` — the `?` goes LAST. It names the same node's same outputs as `"lookup"`, so `FD-V27`-`FD-V29` and the `?` and array rules are the plain form's, and the fixes `FD-V28` and `FD-V29` name keep the `!error` (`FD-V27` names the node, as for `"lookup"`) |
707
+ | where it is legal | wherever `"lookup"` is: a `task`'s, a `map`'s or an `ask`'s `needs`, and an edge's (`ctx.needs`, section 4) |
708
+ | what it is NOT | a path: the error is a field of the OUTCOME beside `result`, and `error` is the one part a reference may name. `"lookup?!error"`, `"!error"` and `"lookup!err"` are `FD-V26`, each naming its fix |
709
+ | what it costs | nothing to copy — a string or `null`, never the shared frozen copy of the result (above) |
710
+ | in the wiring check | the key composes as `{ type: ['string', 'null'] }` with the sample `''`, whatever the producer declares (section 15) |
711
+ | `referenceTarget` | the error form parses to the plain form's object plus `part: 'error'`, present ONLY there (`AimedReference`), so every other spelling parses as before. A resolver of your own must read `part`; an aiming rule may ignore it |
712
+
713
+ **The default — a failure edge handing `{ error }` to its successor — was declined** (P43 (c2)): it would change what every
714
+ existing failure edge's successor receives. You name the error where you want it.
715
+
716
+ The app's case — `lookup` fails, an ask after its failure edge shows the person why, and the node both roads reach reads
717
+ the error beside the result:
718
+
719
+ ```ts run
720
+ import { Driver, fold, liveToken, validateDefinition, type FlowDefinition } from 'fyflow-orchestration';
721
+
722
+ const added: { id: string; payload: { input: unknown } }[] = [];
723
+ const listeners = new Map<string, ((e: { detail: unknown }) => void)[]>();
724
+ const scheduler = {
725
+ addTask(task: unknown) { added.push(task as { id: string; payload: { input: unknown } }); },
726
+ addEventListener(type: string, fn: (e: { detail: unknown }) => void) { listeners.set(type, [...(listeners.get(type) ?? []), fn]); },
727
+ };
728
+ const handedInput = (id: string) => added.find((t) => t.id === id)?.payload.input;
729
+ const fire = (type: string, detail: unknown) => { for (const fn of listeners.get(type) ?? []) fn({ detail }); };
730
+
731
+ const flow: FlowDefinition = {
732
+ flowId: 'lookup-or-ask', version: 1, entry: 'lookup',
733
+ nodes: [
734
+ // `optional`, so a failure SETTLES failed and routes the failure edge (a non-optional task would park, section 7).
735
+ { id: 'lookup', kind: 'task', type: 'Work', optional: true, edges: [{ on: ['failed'], to: 'resolve' }, { to: 'publish' }] },
736
+ { id: 'resolve', kind: 'ask', needs: { why: 'lookup!error' },
737
+ answerSchema: { type: 'object', required: ['value'], properties: { value: { type: 'string' } } },
738
+ edges: [{ on: ['failed'], to: 'skipped' }, { to: 'publish' }] },
739
+ { id: 'publish', kind: 'task', type: 'Work', needs: { found: 'lookup', why: 'lookup!error', given: 'resolve?' }, edges: [{ to: 'done' }] },
740
+ { id: 'skipped', kind: 'end', outcome: 'skipped' },
741
+ { id: 'done', kind: 'end', outcome: 'completed' },
742
+ ],
743
+ };
744
+ const types = { workers: { w: { url: 'worker.js' } }, nodeTypes: { Work: { worker: 'w' } } };
745
+ const registry = { predicates: [], ...types };
746
+ if (validateDefinition(flow, registry).length) throw new Error('"lookup!error" is legal on an ask and on a task');
747
+
748
+ const driver = new Driver(flow, { runId: 'r1', scheduler, createTask: (c) => c, predicates: new Map(), joinPolicy: liveToken, ...types });
749
+ driver.start({ invoice: 'INV-?' });
750
+ fire('task.failed', { id: 'r1:lookup:-:0', state: 'failed', result: null, error: 'lookup timed out after 30 s' });
751
+
752
+ // The ask parks with the error: what it shows the person.
753
+ const asked = driver.foldInputs().find((e) => e.kind === 'asked');
754
+ console.log(asked?.kind === 'asked' && asked.input); // { why: 'lookup timed out after 30 s' }
755
+ if (JSON.stringify(asked?.kind === 'asked' && asked.input) !== '{"why":"lookup timed out after 30 s"}') throw new Error('the ask is handed the error');
756
+
757
+ driver.resolve('r1:resolve:-:0', 'done', { result: { value: 'INV-9' }, actor: 'ana' });
758
+ console.log(handedInput('r1:publish:-:0')); // { found: null, why: 'lookup timed out after 30 s', given: { value: 'INV-9' } }
759
+ if (JSON.stringify(handedInput('r1:publish:-:0')) !== '{"found":null,"why":"lookup timed out after 30 s","given":{"value":"INV-9"}}') {
760
+ throw new Error(`publish received ${JSON.stringify(handedInput('r1:publish:-:0'))}`);
761
+ }
762
+ if (fold(driver.foldInputs()).outcome !== 'running') throw new Error('publish is in flight');
763
+
764
+ // A part other than `error`, or the `?` in the wrong place, is refused naming the fix.
765
+ const refuse = (ref: unknown) => validateDefinition({ ...flow, nodes: flow.nodes.map((n) =>
766
+ (n.id === 'publish' ? { ...n, needs: { why: ref } } : n)) } as FlowDefinition, registry).map((f) => `${f.rule}: ${f.detail}`);
767
+ for (const [ref, fix] of [['lookup!err', 'write "lookup!error"'], ['lookup?!error', 'write "lookup!error?"']] as const) {
768
+ const [finding] = refuse(ref);
769
+ if (!finding?.startsWith('FD-V26') || !finding.includes(fix)) throw new Error(`${ref}: ${finding}`);
770
+ }
771
+ ```
772
+
694
773
  ### Validation happens at construction, not at routing time
695
774
 
696
775
  `new Driver(...)` validates the whole definition before anything is queued. A typo cannot become a
@@ -724,6 +803,87 @@ It would have said *every path from a branch entry reaches the join* — which w
724
803
  that routes a branch away from its join invalid input, and that case is the one this library exists
725
804
  to get right (`ADR 0003`). Do not add it.
726
805
 
806
+ ### The definition's SHAPE, as data — `definitionSchema()` and `definitionSchema(registry)`
807
+
808
+ **`definitionSchema()` returns everything above as a JSON Schema, draft 2020-12** (`spec/flow-definition.md` v0.50,
809
+ [ADR 0132](doc/decisions/0132-a-definitions-shape-is-a-schema-built-from-the-validator.md), since 0.11.0): every node kind
810
+ and its keys, which are required, the edges of each kind, the three `when` shapes, the `needs` grammar (`"lookup!error"`
811
+ included), a `retryPolicy`, a `transform` and an `answerSchema` — each key with a `description`. Hand it to any 2020-12
812
+ validator, or to an agent writing a flow, BEFORE the findings. It is BUILT from the validator's own key sets, so its
813
+ keys cannot drift from them (the reference tokens and the operator partition are written beside them and held to the
814
+ validator by its tests); it is a new plain object on every call, yours to change. A join's `of` and a fanout's branch
815
+ entries are held to the id charset, a non-string refused — as `validateDefinition` refuses one under `FD-V21` **since
816
+ 0.12.0** (`spec/flow-definition.md` v0.52, [Q774](doc/questions/q774.md)). **That is a change in what is accepted**:
817
+ `of: null`, or `branches: ["b", "c", 5]`, passed 0.11.0's validator with no finding when nothing else in the flow
818
+ tripped over it, and is refused now. **And the other way, since 0.12.0** (`spec/flow-definition.md` v0.53,
819
+ [ADR 0137](doc/decisions/0137-a-key-path-indexes-an-array-by-a-bracket-suffix-and-a-plain-segment-keeps-its-meaning.md)):
820
+ a gate's key and a comparison's `path` may carry a bracket index — `files[0].series`, `outcome.result.files[0].series` — so both patterns accept it — and
821
+ `KEY_PATH_RE`, the gate key's, is the exported constant whose `source` moved. Only more is accepted.
822
+
823
+ | | |
824
+ |---|---|
825
+ | it refuses | what a node, an edge or a value gets wrong in its OWN shape: an undeclared or missing key, a wrong type, an id outside `[A-Za-z0-9_-]{1,64}`, an unknown kind, a malformed reference, `when`, `retryPolicy` or `transform`, a task's key on an ask, an `answerSchema` outside the subset |
826
+ | it does not | anything that needs ANOTHER node or the REGISTRY — whether `to` names a node, reachability, regions and joins, where a reference points, every registered name (types, pools, groups, languages, predicates, gate types and their routes) — the last is what `definitionSchema(registry)` adds, below |
827
+ | the authority | **`validateDefinition(flow, registry)`**, or `runtime.check(flow)` on a runtime — for all of that and for the whole. What the schema refuses, `validateDefinition` refuses under every registry; a document it accepts may still be refused there. The root `description` says so |
828
+ | not | a check the library runs (nothing evaluates it), nor in [ADR 0053](doc/decisions/0053-the-schema-validator-is-a-built-in-subset-and-a-keyword-outside-it-is-refused.md)'s subset — it uses `$defs`, `$ref`, `oneOf`, `if`/`then`, `pattern` |
829
+
830
+ **`definitionSchema(registry)` is the same shape SPECIALISED to the `Registry` you validate against** (`spec/flow-definition.md`
831
+ v0.51, since 0.11.0) — what an agent most often gets wrong, as data: a task's `type` is one of your node types, and per
832
+ type (an `if`/`then`) its `pool`, its `settings` against its worker's `settingsSchema`, and a `transform` where the type only
833
+ transforms; `workerGroups`, `transform.lang` and a predicate name from your sets; a gate's `type`, and per gate type its
834
+ `settings` and — for a static type — its edges' `route` names. A registry `checkRegistry` refuses (what `new Driver` would
835
+ refuse) is never specialised: the call throws that refusal. What it refuses, `validateDefinition(flow, registry)` refuses.
836
+
837
+ | | |
838
+ |---|---|
839
+ | it cannot cleanly carry | the `settings` check over the EFFECTIVE settings (your type's defaults merged under the node's): a key the defaults supply is not required of the node, and a default's own value — and a top-level keyword of your settingsSchema other than `properties`, `required` and `additionalProperties` — is not checked; a `minLength` above 1 inside one of your schemas, emitted as half of it rounded up (your schemas count UTF-16 units, JSON Schema counts code points); a DERIVED gate type's routes (`switch`'s are its cases); every route wired EXACTLY ONCE; a gate type's `settingsProblem` and a language's `compile` |
840
+ | annotation only | each node type's worker `inputSchema` — at `$defs.inputSchema` in its type's branch, constraining nothing (it is runtime input) |
841
+ | the authority | as above, against that registry. The root `description` names every row of this table |
842
+ | mind | a validator that tests a property with `in` (`@cfworker/json-schema` 4.1.1 does) misreads a schema property of yours NAMED as an `Object.prototype` member — `constructor`, `toString`, `__proto__` — and refuses or throws where 2020-12 accepts; name your settings otherwise |
843
+
844
+ ```ts run
845
+ import { definitionSchema, validateDefinition, type FlowDefinition } from 'fyflow-orchestration';
846
+
847
+ const schema = definitionSchema();
848
+ console.log(schema.$schema); // https://json-schema.org/draft/2020-12/schema
849
+ if (schema.$schema !== 'https://json-schema.org/draft/2020-12/schema') throw new Error('a 2020-12 document');
850
+ if (!String(schema.description).includes('validateDefinition')) throw new Error('it names the authority for the rest');
851
+
852
+ type Def = { properties: Record<string, { description?: string }>; required: string[]; additionalProperties: boolean };
853
+ const defs = schema.$defs as Record<string, Def> & { reference: { anyOf: { pattern?: string }[] } };
854
+ console.log(defs.askNode.required); // [ 'id', 'kind', 'edges' ]
855
+ if (defs.askNode.additionalProperties !== false || 'type' in defs.askNode.properties) throw new Error('an ask declares no type');
856
+ if (!defs.taskNode.properties.retryPolicy?.description) throw new Error('every key is described');
857
+
858
+ // The reference grammar, as a pattern any validator can apply — the error form accepted, a misplaced "?" not.
859
+ const reference = new RegExp(defs.reference.anyOf[0]!.pattern!);
860
+ if (!reference.test('lookup!error?') || reference.test('lookup?!error')) throw new Error('the "!error" grammar');
861
+
862
+ // A shape the schema accepts can still be refused: an edge to a node that does not exist is the VALIDATOR's.
863
+ const dangling = { flowId: 'f', version: 1, entry: 'a', nodes: [{ id: 'a', kind: 'task', type: 'Work', edges: [{ to: 'ghost' }] }] } as FlowDefinition;
864
+ const rules = validateDefinition(dangling, { predicates: [], workers: { w: { url: 'worker.js' } }, nodeTypes: { Work: { worker: 'w' } } }).map((f) => f.rule);
865
+ if (rules.join() !== 'FD-V2') throw new Error(`the graph is validateDefinition's: ${rules.join()}`);
866
+ if (definitionSchema() === schema) throw new Error('a new object per call');
867
+
868
+ // Specialised to a registry: its node types (the built-in Transform beside yours), each type's pools and settings.
869
+ const registry = {
870
+ predicates: ['isUrgent'],
871
+ workers: { w: { url: 'worker.js', settingsSchema: { type: 'object', properties: { depth: { type: 'integer', minimum: 1 } }, additionalProperties: false } } },
872
+ nodeTypes: { Work: { worker: 'w' } },
873
+ };
874
+ const specialised = definitionSchema(registry);
875
+ type Task = { properties: { type: { enum: string[] } }; allOf: { then: { properties: { pool: { enum: string[] }; settings?: unknown } } }[] };
876
+ const task = (specialised.$defs as { taskNode: Task }).taskNode;
877
+ console.log(task.properties.type.enum); // [ 'Work', 'Transform' ]
878
+ console.log(task.allOf[0]!.then.properties.pool.enum); // [ 'Work' ]
879
+ if (task.allOf[0]!.then.properties.settings === undefined) throw new Error('Work\'s settings are held to its worker\'s schema');
880
+ if (!String(specialised.description).includes('EXACTLY ONCE')) throw new Error('it names what it cannot carry');
881
+ // A registry new Driver would refuse is never specialised.
882
+ let refused = '';
883
+ try { definitionSchema({ ...registry, nodeTypes: { Lost: { worker: 'missing' } } }); } catch (e) { refused = (e as Error).name; }
884
+ if (refused !== 'WorkerMissing') throw new Error(`a refused registry throws its refusal: ${refused}`);
885
+ ```
886
+
727
887
  ---
728
888
 
729
889
  ## 4. Decisions and predicates
@@ -784,7 +944,7 @@ if (refuse({ all: [{ any: [{ path: 'outcome.result.a', op: 'eq', value: 1 }] }]
784
944
 
785
945
  | | |
786
946
  |---|---|
787
- | `path` | a key path over the context, **rooted at `outcome` or `needs`** — `outcome.result.x`, `outcome.taskId`, `needs.earlier.flag`. No array index, no wildcard, no expression |
947
+ | `path` | a key path over the context, **rooted at `outcome` or `needs`** — `outcome.result.x`, `outcome.taskId`, `needs.earlier.flag`, and an array element by a bracket index after a key, `outcome.result.items[0].score` (never on the root). No wildcard, no expression |
788
948
  | `op` | one of `eq`, `ne`, `lt`, `lte`, `gt`, `gte`, `startsWith`, and **nothing else** |
789
949
  | `value` | a scalar: `null`, a boolean, a finite number or a string |
790
950
  | `default` | optional — what is compared when the path does not resolve **or resolves to `null`**, which is exactly `??`. Without it an absent path is **no match**, and a resolved `null` is compared as the scalar it is |
@@ -860,7 +1020,7 @@ flagged low confidence* was not expressible until v0.12 —
860
1020
  answering [Q8](doc/questions/q8.md).
861
1021
 
862
1022
  **An edge may carry `needs`, and its predicate sees the result under `ctx.needs`.** Same grammar as
863
- a node's (section 3) — a node id, or a one-element array holding one, and since 0.8.0 `"$payload"` and `"$item"` —
1023
+ a node's (section 3) — a node id, or a one-element array holding one, since 0.8.0 `"$payload"` and `"$item"`, and since 0.11.0 a node's error, `"n!error"` —
864
1024
  **minus `"."`**, which an edge has no use for: *what a predicate would have received anyway* is `ctx.outcome`. `"$payload"`
865
1025
  and `"$item"` are not refused there: a predicate has no other road to the run's payload or to its map instance's item
866
1026
  (section 3's two subsections route on them).
@@ -1071,10 +1231,12 @@ is not a supported choice. See
1071
1231
 
1072
1232
  ### What a join carries downstream
1073
1233
 
1074
- **The arrived branches' results, keyed by branch path.** A satisfied join dispatches its successor
1075
- with an object whose keys are the `branchPath`s of the branches that **reached** it, and whose values
1076
- are what each of those branches carried in — so a fan-out's results are not thrown away, and the
1077
- value is the same whatever order the branches finished in.
1234
+ **The arrived branches' results, each under the branch's OWN key.** A satisfied join dispatches its
1235
+ successor with an object holding one entry per branch that **reached** it — keyed by the branch's
1236
+ **entry node id** for a `fanout` and by the **item index** for a `map`, one segment at every depth —
1237
+ whose values are what each of those branches carried in. So a fan-out's results are not thrown away,
1238
+ a key path reaches into any depth (`input.prep.1`), and the value is the same whatever order the
1239
+ branches finished in.
1078
1240
 
1079
1241
  ```ts run
1080
1242
  import { createRuntime } from 'fyflow-orchestration/scheduler';
@@ -1096,7 +1258,7 @@ const flow: FlowDefinition = {
1096
1258
  flowId: 'collect', version: 1, entry: 'split',
1097
1259
  nodes: [
1098
1260
  { id: 'split', kind: 'fanout', branches: ['scoreA', 'scoreB'] },
1099
- // `scoreA` is branch path "0" and is made SLOW on purpose, so "1" settles first.
1261
+ // `scoreA` is the FIRST entry and is made SLOW on purpose, so `scoreB` settles first.
1100
1262
  { id: 'scoreA', kind: 'task', type: 'Scorer', settings: { score: 0.9, ms: 40 }, edges: [{ to: 'collect' }] },
1101
1263
  { id: 'scoreB', kind: 'task', type: 'Scorer', settings: { score: 0.4 }, edges: [{ to: 'collect' }] },
1102
1264
  { id: 'collect', kind: 'join', of: 'split', edges: [{ to: 'publish' }] },
@@ -1115,7 +1277,7 @@ const inputs: FoldInput[] = [];
1115
1277
  const state = await runtime.start(flow, {}, { onFoldInput: (input) => { inputs.push(input); } }).done;
1116
1278
  await runtime.shutdown();
1117
1279
 
1118
- // **The order is READ, not assumed.** The delay above ASKS for branch "1" to settle first; this is
1280
+ // **The order is READ, not assumed.** The delay above ASKS for `scoreB` to settle first; this is
1119
1281
  // what the run actually did, and the claim below is about the collection being independent of it.
1120
1282
  const settledOrder = inputs.flatMap((i) =>
1121
1283
  (i.kind === 'outcome' && /:score[AB]:/.test(i.record.taskId) ? [i.record.taskId.split(':')[1]] : []));
@@ -1124,13 +1286,14 @@ console.log(settledOrder.join(' then ')); // 'scoreB then scoreA'
1124
1286
  // `publish` is the join's successor, so what it received IS the collection — and `tag` returns what
1125
1287
  // it was handed with its own label added, so the collection is readable in the result.
1126
1288
  const published = Object.values(state.nodes).find((n) => n.nodeId === 'publish')?.result;
1127
- console.log(JSON.stringify(published)); // {"0":{"score":0.9},"1":{"score":0.4},"publish":true}
1289
+ console.log(JSON.stringify(published)); // {"scoreA":{"score":0.9},"scoreB":{"score":0.4},"publish":true}
1128
1290
 
1129
- const want = '{"0":{"score":0.9},"1":{"score":0.4},"publish":true}';
1291
+ const want = '{"scoreA":{"score":0.9},"scoreB":{"score":0.4},"publish":true}';
1130
1292
  if (JSON.stringify(published) !== want) throw new Error(`the join carried ${JSON.stringify(published)}`);
1131
- // **B settled FIRST and is still under key "1".** The collection is keyed by BRANCH PATH, not by
1132
- // arrival order, which is the whole point — and the two assertions are a pair: the first says which
1133
- // order the run took, the second says the record did not follow it.
1293
+ // **B settled FIRST and is still under its own key, listed SECOND.** The collection is keyed by each
1294
+ // branch's ENTRY ID and inserted in `branches` order, not in arrival order, which is the whole point —
1295
+ // and the two assertions are a pair: the first says which order the run took, the second says the
1296
+ // record did not follow it.
1134
1297
  if (settledOrder.join() !== 'scoreB,scoreA') throw new Error(`settled ${settledOrder.join()} — the arrangement did not hold, so the claim below is untested`);
1135
1298
  ```
1136
1299
 
@@ -1143,20 +1306,32 @@ if (settledOrder.join() !== 'scoreB,scoreA') throw new Error(`settled ${settledO
1143
1306
  2. **`{}` is a real answer, not an error.** Under `liveToken` a join fires once its region has no live
1144
1307
  branches, and every branch reaching an `end` satisfies that with nothing arrived. If that should be
1145
1308
  a failure instead, pass `liveTokenRequiringArrival` — see above.
1146
- 3. **The keys are strings, in ascending branch-path order**, compared segment by segment as numbers —
1147
- so `"0"`, `"1"`, `"2"`, `"10"` at the top level, and `"1.0"`, `"1.1"`, `"1.10"` inside branch 1.
1148
- A nested region's inner join carries its own object, which becomes the value of the outer branch it
1149
- sits in. **Every key of one collection has the same number of segments**, because they are the
1150
- branch paths of one region. *(An earlier version of this bullet illustrated the order as `"1"`,
1151
- `"1.0"`, `"2"`, `"10"` — a mix of depths that no single collection can hold, and one that
1152
- `Object.keys` would not even return in that order, because JavaScript re-orders integer-like keys
1153
- ahead of string ones. W34's conformity reader.)*
1309
+ 3. **Every key is ONE segment, at every depth — the branch's own key.** A `fanout`'s branch is keyed by
1310
+ its entry node id (`FD-V9` keeps them distinct), a `map`'s instance by its item index — so a
1311
+ fan-out whose first branch holds a map gives `{"prep":{"0":1,"1":2},"side":{"xs":[1,2]}}`, never
1312
+ `"0.1"` under `"0"`. A nested region's inner join carries its own object, which becomes the value of
1313
+ the outer branch it sits in. The full branch path is not lost: it is the region's own path (`regionPath`) + `.` + the
1314
+ position — the position alone at the top level, where `regionPath` is `-` — on every `branch.*` event, task id and
1315
+ `inputOf` path, which are unchanged.
1316
+ 4. **The order is the object's own.** Entries are inserted in `branches` order for a fan-out and in
1317
+ ascending item index for a map, and **JavaScript lists integer-like keys first, ascending** — so a
1318
+ fan-out whose entries are `"b"` and `"2"` lists `"2"` first. Deterministic whatever order the
1319
+ branches finished in; no node id is refused for it. Each entry is an own property, so even an entry
1320
+ id `__proto__` is a key.
1321
+ 5. **BREAKING in 0.12.0, and a run in flight across the upgrade cannot be resumed.** Until 0.11.0 the
1322
+ keys were full branch paths (`{"0":{"0.0":1,"0.1":2},"1":{"xs":[1,2]}}` for the flow above) and a
1323
+ fan-out's were POSITIONS; a flow reading `"0.1"` or `"0"` reads nothing now. A collection reaches a
1324
+ checkpoint (an ask's kept input, a gate's decision, a worker's result), so **`CHECKPOINT_FORMAT` is
1325
+ 2** and a checkpoint 0.11.0 wrote — join or none — is refused `ResumeRefused('format')` by
1326
+ `resume()`, `runtime.resume` and `inputOf`; a 0.11.0 driver refuses a 0.12.0 checkpoint the same
1327
+ way. There is no migration: drain or abandon those runs first
1328
+ ([ADR 0138](doc/decisions/0138-a-join-hands-on-each-branch-under-its-own-key-and-a-checkpoint-of-the-old-keying-is-refused-by-format.md)).
1154
1329
 
1155
1330
  *(This section read **"`null`, currently … this is [Q36](doc/questions/q36.md) and it is open"**
1156
1331
  until [ADR 0037](doc/decisions/0037-what-a-branch-receives-is-a-property-of-the-fan-out-and-what-a-join-carries-is-a-property-of-the-region.md).
1157
1332
  The argument for the `null` was real and is unchanged — which branch finishes last is genuinely
1158
1333
  non-deterministic — and what answered it is that carrying **all** the arrivals keyed by branch is not
1159
- picking one. `spec/join-policy.md` section 4, `JP-R12`.)*
1334
+ picking one. `spec/join-policy.md` section 4, `JP-R12`; its keys are each branch's own since 0.12.0.)*
1160
1335
 
1161
1336
  ### The same on the RUNTIME — a decision, a fan-out and a join on real workers
1162
1337
 
@@ -1191,7 +1366,7 @@ const isLowConfidence: Predicate = (ctx) => {
1191
1366
 
1192
1367
  // THE FLOW — a DECISION, a FAN-OUT, a JOIN. `ingest` goes to `review` when the predicate holds,
1193
1368
  // otherwise to `split`. `split` runs two branches concurrently; `collect` waits for both and hands
1194
- // `publish` every branch's result, keyed by branch path.
1369
+ // `publish` every branch's result, each under its own entry id.
1195
1370
  const flow: FlowDefinition = {
1196
1371
  flowId: 'decide', version: 1, entry: 'ingest',
1197
1372
  nodes: [
@@ -1219,13 +1394,13 @@ const ran = (s: RunState) => Object.values(s.nodes).map((n) => n.nodeId).sort();
1219
1394
  const resultOf = (s: RunState, id: string) => Object.values(s.nodes).find((n) => n.nodeId === id)?.result;
1220
1395
  console.log(ran(high)); // [ 'classify', 'enrich', 'ingest', 'publish' ]
1221
1396
  console.log(ran(low)); // [ 'ingest', 'review' ]
1222
- console.log(resultOf(high, 'publish')); // { '0': { confidence: 0.9, ingest: true, enrich: true }, '1': { …, classify: true }, publish: true }
1397
+ console.log(resultOf(high, 'publish')); // { enrich: { confidence: 0.9, ingest: true, enrich: true }, classify: { …, classify: true }, publish: true }
1223
1398
 
1224
1399
  // Verified by the docs gate:
1225
1400
  if (high.outcome !== 'completed' || low.outcome !== 'completed') throw new Error(`outcomes ${high.outcome} / ${low.outcome}`);
1226
1401
  if (ran(high).join() !== 'classify,enrich,ingest,publish') throw new Error(`high ran ${ran(high).join()}`);
1227
1402
  if (ran(low).join() !== 'ingest,review') throw new Error(`low ran ${ran(low).join()}`);
1228
- const want = '{"0":{"confidence":0.9,"ingest":true,"enrich":true},"1":{"confidence":0.9,"ingest":true,"classify":true},"publish":true}';
1403
+ const want = '{"enrich":{"confidence":0.9,"ingest":true,"enrich":true},"classify":{"confidence":0.9,"ingest":true,"classify":true},"publish":true}';
1229
1404
  if (JSON.stringify(resultOf(high, 'publish')) !== want) throw new Error(`publish received ${JSON.stringify(resultOf(high, 'publish'))}`);
1230
1405
  ```
1231
1406
 
@@ -1468,10 +1643,18 @@ keeps waiting, and the run is not stuck.
1468
1643
  ([W180](doc/work-items/w180-asking-and-answering-in-a-run.md), [ADR 0109](doc/decisions/0109-an-ask-parks-with-nothing-dispatched-and-its-answer-is-its-result.md), `spec/run-state.md` `RN-R36`/`RN-R37`):
1469
1644
 
1470
1645
  - **Nothing is handed to the scheduler** — the scheduler never sees the ask's id. The driver appends ONE fold input,
1471
- `{ kind: 'asked', taskId, input }`; the fold reads the node `suspended` and the run `awaiting_action`, exactly the
1646
+ `{ kind: 'asked', taskId, input, observedAt? }`; the fold reads the node `suspended` and the run `awaiting_action`, exactly the
1472
1647
  words a parked task gives (so `done` waits, below), and the node's entry in `nodes` carries **`input`** — what reached
1473
1648
  the ask, its `needs` composed — kept after the answer. Tell an ask from a parked failure by the node's KIND, or by the
1474
1649
  `asked` element, never by an error string. In a fan-out the ask's branch is LIVE, so **its join waits for the person**.
1650
+ - **WHEN it was asked** (since 0.11.0, [ADR 0130](doc/decisions/0130-a-regenerated-asked-element-keeps-the-stored-runs-time.md)):
1651
+ the element's `observedAt` is the driver's `now` — the clock an outcome record's `observedAt` reads — and the node's
1652
+ entry carries it as **`askedAt`**, kept after the answer as `input` is. **A resume KEEPS the stored time**: the replay
1653
+ regenerates the element and takes its stored `observedAt`, so a wait measured across a resume does not restart; past
1654
+ the stored tail it is the resumed driver's `now`. *(An `asked` element written before 0.11.0 has none; since 0.12.0
1655
+ such a checkpoint is refused `format` before its events are read — section 5, `CHECKPOINT_FORMAT` 2.)* **A driver
1656
+ older than 0.11.0 refused a 0.11.0 checkpoint `diverged` at its first ask**; any driver before 0.12.0 refuses a 0.12.0
1657
+ checkpoint `format` before replaying anything — resume a run on a driver of the format that wrote it.
1475
1658
  - **The answer is `driver.resolve(taskId, …)`** — the same call as below. `'done'` with `result` the answer, held to the
1476
1659
  ask's `answerSchema` (refused as `BoundaryRefused` with NOTHING appended and the ask still parked — answer again;
1477
1660
  `validateBoundaries: false` turns it off); `'failed'` with an `error` is the person DECLINING, held to nothing.
@@ -1484,7 +1667,7 @@ keeps waiting, and the run is not stuck.
1484
1667
  run's log, once per ask — that element keeps what reached the ask, whatever its size. A predicate's `ctx.outcome` for an ask has `attempts: 0` and
1485
1668
  **`schedulerTimestamp`, `startTime`, `endTime` and `executionTime` all `null`** — no scheduler was involved.
1486
1669
  - **BREAKING in 0.8.0** (P35 decision 8): `FoldInput` gains the kind `'asked'`, and `OutcomeRecord.schedulerTimestamp`
1487
- is `number | null` — `null` only on an ask's effective outcome. `NodeRun.input` is optional and additive.
1670
+ is `number | null` — `null` only on an ask's effective outcome. `NodeRun.input` is optional and additive — as, since 0.11.0, are the element's `observedAt` and `NodeRun.askedAt`.
1488
1671
  - **An answer given from a callback on the driver's own step** — `onFoldInput` on the `asked` element included — is
1489
1672
  checked at once (a refusal is thrown or `false` returned right there) and, accepted, `resolve()` returns `true` and the
1490
1673
  resolution is appended when the step ends: so the run's checkpoint resumes. A second answer while it is pending is
@@ -1543,6 +1726,9 @@ console.log(added, parked.outcome, parked.nodes['r1:approve:-:0'].input);
1543
1726
  // [ 'r1:draft:-:0' ] awaiting_action { text: 'a draft' }
1544
1727
  if (added.join() !== 'r1:draft:-:0' || parked.outcome !== 'awaiting_action') throw new Error('the ask should park with nothing dispatched');
1545
1728
  if (JSON.stringify(parked.nodes['r1:approve:-:0'].input) !== '{"text":"a draft"}') throw new Error('the ask keeps what reached it');
1729
+ // And WHEN it was asked: the element's `observedAt` (the driver's clock, `Date.now()` by default) is the entry's `askedAt`.
1730
+ const askedEl = driver.foldInputs().find((e) => e.kind === 'asked');
1731
+ if (askedEl?.kind !== 'asked' || typeof askedEl.observedAt !== 'number' || parked.nodes['r1:approve:-:0'].askedAt !== askedEl.observedAt) throw new Error('the ask keeps when it was asked');
1546
1732
 
1547
1733
  // An answer its schema refuses is thrown at you BY NAME — nothing appended, the ask still parked.
1548
1734
  let refused: unknown = null;
@@ -2035,7 +2221,7 @@ AFTER the target — or one that moves only an envelope, even at the target: the
2035
2221
  answers with the definition you hand it. **It re-runs your predicates and gate types**, as a resume does. The value is
2036
2222
  the reading's own — a value a `needs` reference composed is frozen, as every reader's is (section 3), the rest yours —
2037
2223
  and nothing of a live run is reachable through it. **The input a task's worker sees AFTER its transform is not answered**
2038
- ([Q760](doc/questions/q760.md)): hand the envelope's `input` to `tryTransform` (section 15). **BREAKING in 0.10.0**:
2224
+ ([Q760](doc/questions/q760.md), resolved by P43: the road serves): hand the envelope's `input` to `tryTransform` (section 15). **BREAKING in 0.10.0**:
2039
2225
  `RunHandle` gains `inputOf`, which breaks only code that BUILDS a handle.
2040
2226
 
2041
2227
  ```ts run
@@ -2718,7 +2904,9 @@ After a refusal the run is **halted**: nothing routes for it again, and its stat
2718
2904
  and FOUR surfaces are renamed — the `RunOutcome` `stopped` is `halted`, `RunState.stoppedBy` is `haltedBy`, the fold
2719
2905
  input `{ kind: 'stopped', reason }` is `{ kind: 'halted', reason }`, and `ResumeRefusalReason`'s `stopped` is
2720
2906
  `halted` — so a comparison against an old name, or a `{ kind: 'stopped' }` you construct, no longer compiles. A 0.7.0 checkpoint or log holding a `{ kind: 'stopped' }` element still reads as `halted`:
2721
- `fold` folds it so, and a resume refuses it `halted` at its position.
2907
+ `fold` folds it so, and a resume refuses it `halted` at its position — but a 0.7.0 checkpoint is format 1, so since
2908
+ 0.12.0 a resume refuses it `format` first (section 5); the `halted` read is reached only by such an element under a
2909
+ format-2 header, which no driver writes.
2722
2910
 
2723
2911
  ---
2724
2912
 
@@ -2979,7 +3167,7 @@ if (!haltedRun.haltedBy?.startsWith('UncloneableResult')) throw new Error(`halte
2979
3167
  | ~~**map-style fan-out**~~ — **BUILT** since [W33](doc/work-items/w33-a-map-at-run-time-arity.md); see section 3 |
2980
3168
  | ~~**RUNNING an `ask`**~~ — a node that waits for a person, its answer at `resolve()` | **BUILT** since [W180](doc/work-items/w180-asking-and-answering-in-a-run.md): a run parks at an ask with nothing handed to the scheduler, and its answer is its result — section 7. Under a cancel, a stop, a pause and a resume an ask is a parked task — never in flight, so nothing is ever sent or handed on for it — since [W181](doc/work-items/w181-an-ask-under-run-control.md) (section 7) |
2981
3169
  | a **live, mutable view** of a run's state, or the state handed to `onFoldInput` beside each input | **no** — [P39](doc/plans/p39-what-a-step-costs-and-where-a-live-run-state-is-read.md) (b), [ADR 0122](doc/decisions/0122-a-runtimes-live-state-is-a-frozen-copy-on-its-handle.md): `handle.state()` is a frozen COPY, cached until the run moves, and `handle.outcome()` / `handle.node()` are the O(1) reads — section 8 |
2982
- | a task's INPUT kept in the run state — `NodeRun.input` for a task, or `pathEnded` carrying the value that reached an end | **no** — [ADR 0046](doc/decisions/0046-a-run-retains-what-its-tasks-returned-and-a-region-releases-its-collection-when-its-join-becomes-terminal.md): a run keeps what its tasks returned. **READ it instead**, since [W205](doc/work-items/w205-what-a-node-was-handed-read-by-replaying-the-run.md): `inputOf(flow, options, { header, events }, target)` or `handle.inputOf(target)` replays the run to that node — section 8. The input a worker sees AFTER its transform: **no** ([Q760](doc/questions/q760.md)) |
3170
+ | a task's INPUT kept in the run state — `NodeRun.input` for a task, or `pathEnded` carrying the value that reached an end | **no** — [ADR 0046](doc/decisions/0046-a-run-retains-what-its-tasks-returned-and-a-region-releases-its-collection-when-its-join-becomes-terminal.md): a run keeps what its tasks returned. **READ it instead**, since [W205](doc/work-items/w205-what-a-node-was-handed-read-by-replaying-the-run.md): `inputOf(flow, options, { header, events }, target)` or `handle.inputOf(target)` replays the run to that node — section 8. The input a worker sees AFTER its transform: **no** ([Q760](doc/questions/q760.md), resolved — the envelope's `input` through `tryTransform` serves) |
2983
3171
  | a **join timeout** | nothing. A stuck join is detected by the `stuck` outcome, with no clock |
2984
3172
  | **per-join policy choice** | one policy per driver |
2985
3173
  | a **declared event family** for `fyflow-react-components` | `onRegionEvent`, `onFoldInput` and `joinStatusOf()` — see below |
@@ -3911,7 +4099,7 @@ driver.start({ items: [1, 2, 3] });
3911
4099
 
3912
4100
  **What an expression sees, and what it may not.** `$` is the node's input — the previous node's
3913
4101
  result, the object `needs` described, a map instance's own item, or a join's collection keyed by
3914
- branch path — and the node's effective `settings` are bound as `$settings`, so one expression is
4102
+ each branch's own key — and the node's effective `settings` are bound as `$settings`, so one expression is
3915
4103
  parameterised per node. **Five names are unbound, by name, on every compiled expression**: `$now`,
3916
4104
  `$millis`, `$random` — the three every adapter must refuse, because a transform is a function of what
3917
4105
  the run produced and run state is a fold of the log — **`$shuffle`**, JSONata's own fourth, which
@@ -3966,10 +4154,10 @@ if (!publish('$sum([*.score])', { ...registry, languages: {} }).some((f) => f.ru
3966
4154
  // The envelope worker — the class the bootstrap helper puts around your worker — with the adapter's worker half by URL.
3967
4155
  const w = new EnvelopeWorker({ worker: echo.url, languages: { jsonata: jsonata.worker } });
3968
4156
  await w.setup();
3969
- const record = { '0': { score: 40 }, '1': { score: 20 } }; // JP-R12's collection, two branches
4157
+ const record = { scoreA: { score: 40 }, scoreB: { score: 20 } }; // JP-R12's collection: `split`'s two entry ids
3970
4158
  const total = await w.run({ settings: { factor: 2 }, input: record, transform: { lang: 'jsonata', expr: '$sum([*.score]) * $settings.factor' } });
3971
4159
  if (total !== 120) throw new Error(`the worker received ${JSON.stringify(total)} — expected the reduction, 120`);
3972
- const one = await w.run({ settings: {}, input: { '0': { score: 7 } }, transform: { lang: 'jsonata', expr: '$sum([*.score])' } });
4160
+ const one = await w.run({ settings: {}, input: { scoreA: { score: 7 } }, transform: { lang: 'jsonata', expr: '$sum([*.score])' } });
3973
4161
  if (one !== 7) throw new Error('arity one, the same expression: 7');
3974
4162
  const none = await w.run({ settings: {}, input: {}, transform: { lang: 'jsonata', expr: '$sum([*.score])' } });
3975
4163
  if (none !== 0) throw new Error('a join nothing reached: 0, because the reduction is over the ARRAY form');
@@ -4149,6 +4337,7 @@ whatever arcs reach the node, on the entry node too** (W166, P37 stop-and-ask (d
4149
4337
  | `shape` | `'given'` — not an arc; `checkWiring` never produces it |
4150
4338
  | `producer` | **absent** — the value came from no node (and `through` too) |
4151
4339
  | `sample` | `{ source: 'given', value, result }` — the value you gave, and what the transform made of it; no `arity`. **Never `'example'` or `'witness'`** |
4340
+ | `cause` | **on `transform-fails` only** (since 0.11.0, every road — `checkWiring`'s arcs too): `'threw'` — the worker half did not import, had no default export, or its `compile` or `evaluate` threw — or `'schema'`, a result the node's `inputSchema` refused — the type `TransformFailureCause`, a root export. Absent on every other kind. Read it rather than `why`'s prefix |
4152
4341
  | `kind` | `transform-passes`, or `transform-fails` with the finding against the node's worker `inputSchema` — or naming the error, where the expression does not compile or throws on this value; `unchecked` for a language, type, worker or worker schema the registry lacks, as on an arc, naming which — and for a `null` candidate, which takes away any transform the node declares too: there is nothing to run the value through, and holding that value to the `inputSchema` is `findings(schema, value)` |
4153
4342
 
4154
4343
  **The value is `$` as you give it: no `needs` is composed over it**, because a `needs` composes what
@@ -4205,6 +4394,7 @@ if (entry!.kind !== 'transform-passes') throw new Error(`expected the given valu
4205
4394
  const [wrong] = await tryTransform(registry, flow, 'read', rename, { input: { folder: 3 } });
4206
4395
  console.log(`${wrong!.kind}: ${wrong!.why}`); // transform-fails: the transform's result on the given input fails the consumer's inputSchema — $.source expected string, ...
4207
4396
  if (wrong!.kind !== 'transform-fails' || !wrong!.why.includes('$.source expected string')) throw new Error(`expected the finding, got ${wrong!.why}`);
4397
+ if (wrong!.cause !== 'schema') throw new Error(`a result the schema refused is cause 'schema', got ${wrong!.cause}`);
4208
4398
 
4209
4399
  // THE INNER NODE. Without an input its one arc runs on `reader`'s declared EXAMPLE; given a value, it runs on that.
4210
4400
  const join = { lang: 'jsonata', expr: '{ "text": $join(lines, " ") }' };
@@ -4217,6 +4407,8 @@ if (JSON.stringify(mine!.sample!.result) !== JSON.stringify({ text: 'typed by ha
4217
4407
  // An expression the engine refuses: `transform-fails`, naming the throw — the compile error, on the given input.
4218
4408
  const [broken] = await tryTransform(registry, flow, 'sum', { lang: 'jsonata', expr: '{{{' }, { input: { lines: [] } });
4219
4409
  if (broken!.kind !== 'transform-fails' || !broken!.why.startsWith('the transform threw on the given input')) throw new Error(`expected the compile error, got ${broken!.why}`);
4410
+ // The same, as DATA (since 0.11.0): `cause` is 'threw' — no result was produced — where the finding above is 'schema'.
4411
+ if (broken!.cause !== 'threw' || entry!.cause !== undefined) throw new Error(`a throw is cause 'threw', and a passing verdict has none — got ${broken!.cause}, ${entry!.cause}`);
4220
4412
  ```
4221
4413
 
4222
4414
  **BREAKING in 0.8.0** (P37 decision 10): **`SampleSource` gains `'given'`**, **`WiringShape` gains
@@ -4403,7 +4595,7 @@ Three things follow, and they are the whole of what you need:
4403
4595
  |---|---|
4404
4596
  | **`RunState.regions` is TWO-LEVEL** | `regions[openerId][openerPath]`. The opener's id is unchanged in meaning and in value; the second key is `-` for a region in no region, and inside a `map` region it is that map's item index |
4405
4597
  | **every `RegionEvent` carries `regionPath` beside `regionId`** | so an event says WHICH instance it is about. The same for `UnsatisfiableJoin` |
4406
- | **a join's collection is per INSTANCE** | `JP-R12` keys it by branch path, and a branch inside an instance carries that instance's path as its own prefix — so two instances of one opener hold DISJOINT key sets and cannot collide |
4598
+ | **a join's collection is per INSTANCE** | each instance of one opener holds its OWN collection, so two instances cannot collide — even though, since 0.12.0, their keys are each branch's own key (`JP-R12`) and two instances may hold the SAME key set ([ADR 0138](doc/decisions/0138-a-join-hands-on-each-branch-under-its-own-key-and-a-checkpoint-of-the-old-keying-is-refused-by-format.md): until 0.11.0 the keys were branch paths, carrying the instance's path as a prefix) |
4407
4599
 
4408
4600
  **Both arities come from data, at both levels, and neither is in the document.** That is what
4409
4601
  separates a `map` from a `fanout`: `FD-V9` requires a fan-out's branch entries to be written down, so
@@ -4489,11 +4681,12 @@ if (!arrivals.includes('perLine@0 <- 0.0') || !arrivals.includes('perLine@1 <- 1
4489
4681
  }
4490
4682
 
4491
4683
  // WHAT THE OUTER JOIN CARRIES IS TWO LEVELS DEEP. `JP-R12`: one entry per branch that arrived, keyed
4492
- // by branch path — and each of the outer entries IS that instance's own collection.
4684
+ // by its OWN key — an item index at both levels — and each of the outer entries IS that instance's own
4685
+ // collection. So `report.0.1` is order 0's second line: the key path a gate or an `if` reads.
4493
4686
  const report = Object.values(state.nodes).find((n) => n.nodeId === 'report')?.result as Record<string, Record<string, { sku: string }>>;
4494
4687
  console.log(Object.keys(report).filter((k) => k !== 'reported').sort()); // [ '0', '1' ]
4495
- console.log(report['0']!['0.1']!.sku); // ab-1
4496
- if (report['0']!['0.1']!.sku !== 'ab-1' || report['1']!['1.0']!.sku !== 'c-0') {
4688
+ console.log(report['0']!['1']!.sku); // ab-1
4689
+ if (report['0']!['1']!.sku !== 'ab-1' || report['1']!['0']!.sku !== 'c-0') {
4497
4690
  throw new Error(`the outer collection's values are the inner collections: ${JSON.stringify(report)}`);
4498
4691
  }
4499
4692
  ```
@@ -4626,7 +4819,7 @@ console.log(state.outcome); // completed
4626
4819
  if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
4627
4820
  // The same answer a run nobody killed gives (section 17's):
4628
4821
  const report = Object.values(state.nodes).find((n) => n.nodeId === 'report')?.result as Record<string, Record<string, { sku: string }>>;
4629
- if (report['0']!['0.1']!.sku !== 'ab-1' || report['1']!['1.0']!.sku !== 'c-0') {
4822
+ if (report['0']!['1']!.sku !== 'ab-1' || report['1']!['0']!.sku !== 'c-0') {
4630
4823
  throw new Error(`the resumed run reaches the same report: ${JSON.stringify(report)}`);
4631
4824
  }
4632
4825
 
@@ -4763,11 +4956,13 @@ if (JSON.stringify(store.header) !== JSON.stringify(driver.checkpoint().header))
4763
4956
  ([Q374](doc/questions/q374.md)) — and so is a stored `attempted` event's number, which a re-run task is handed as
4764
4957
  the attempts it SPENT (one naming a task the replay never dispatched, carrying no positive integer, or not RISING over
4765
4958
  the last stored for its task, is refused `diverged` at its position). A store you do not trust is a store you should not resume from.
4766
- - **`ResumeRefused` names its `reason`**: `format` (not a checkpoint this version writes), `run` (the
4959
+ - **`ResumeRefused` names its `reason`**: `format` (not a checkpoint this version writes — **every checkpoint 0.11.0
4960
+ or earlier wrote is format 1 and refused here**: `CHECKPOINT_FORMAT` is 2 since 0.12.0, section 5), `run` (the
4767
4961
  header's run is not the driver's), `flow` (another flow id or `version` — there is no migration),
4768
4962
  `halted`, `cancelled` and `diverged` — the last three with the `index` of the position. **A HALTED
4769
- run is not resumable**: the refusal that halted it would halt it again — and a 0.7.0 checkpoint whose
4770
- halt is stored under the old kind, `stopped`, is refused `halted` at that element's position (ADR 0102). **Nor is a CANCELLED one**
4963
+ run is not resumable**: the refusal that halted it would halt it again — and a halt stored under 0.7.0's kind,
4964
+ `stopped`, is refused `halted` at that element's position (ADR 0102; a 0.7.0 checkpoint itself is format 1, refused
4965
+ `format` first). **Nor is a CANCELLED one**
4771
4966
  (section 6): a cancel is its owner's act, and a resume would hand the scheduler again what the owner
4772
4967
  stopped — `index` is the first cancelled settlement or the cancel request, whichever is first. A completed or `stuck` run resumes to itself and
4773
4968
  dispatches nothing. **A STOPPED run is the one terminal run a resume TAKES** (below): its `interrupted` event is replayed, and `resumed` ends it.
@@ -4993,7 +5188,7 @@ rest, while every file gets a thumbnail* — had nowhere to keep that memory.
4993
5188
 
4994
5189
  | | |
4995
5190
  |---|---|
4996
- | **it names a gate TYPE** | `distinct` is built in: routes `first` (its DEFAULT), `duplicate` and `nokey`, keyed by `settings.key` — a DOTTED path of object keys over the value that reached the gate, no index, no wildcard, no expression. A key a flow needs COMPUTED is computed by the task before the gate. `if` and `switch` are built in too, and remember nothing (the end of this section) |
5191
+ | **it names a gate TYPE** | `distinct` is built in: routes `first` (its DEFAULT), `duplicate` and `nokey`, keyed by `settings.key` — a DOTTED path of object keys over the value that reached the gate, each key optionally followed by bracket indexes into an ARRAY (`files[0].series`, `m[0][2]`; `[n]` with no leading zero, at most 999,999,999, never leading the path — [ADR 0137](doc/decisions/0137-a-key-path-indexes-an-array-by-a-bracket-suffix-and-a-plain-segment-keeps-its-meaning.md)); no quoted key, no wildcard, no expression. **`[1]` and `.1` are two paths**: `xs[1]` is element 1 of a real array and nothing over an object, `xs.1` an object's own key `"1"` and nothing over an array. A key a flow needs COMPUTED is computed by the task before the gate. `if` and `switch` are built in too, and remember nothing (the end of this section) |
4997
5192
  | **every route is wired, exactly once** | an edge with no `route` carries the type's DEFAULT route, never every route; a DROP is an edge to an `end`, and each is refused at load if it is missing (`FD-V41`) |
4998
5193
  | **it computes nothing** | the branch leaves on the chosen route with EXACTLY the value that reached the gate |
4999
5194
  | **its memory is ONE per gate per RUN** | shared by every branch that reaches it, whatever map item or region instance it comes from — and it starts empty with the run and ends with it |
@@ -5084,12 +5279,13 @@ for (const d of firsts) {
5084
5279
  }
5085
5280
 
5086
5281
  // A DROP IS AN `end`, so it is a path end like any other — and the thumbnail ran for EVERY file: each file's
5087
- // `perFile` record holds its thumbnail, and nothing from the gate road, which ended at an `end` in every branch.
5282
+ // `perFile` record holds its thumbnail under `beside`'s entry id `thumbnail`, and nothing from the gate road,
5283
+ // which ended at an `end` in every branch.
5088
5284
  console.log(state.pathEnds.map((p) => p.at).sort());
5089
5285
  // [ 'done', 'dropDuplicate', 'dropDuplicate', 'dropUnidentified', 'seriesReady', 'seriesReady' ]
5090
5286
  const published = Object.values(state.nodes).find((n) => n.nodeId === 'publish')!.result as
5091
5287
  { joinPayload: Record<string, Record<string, { thumbnail?: string }>> };
5092
- const thumbs = Object.entries(published.joinPayload).map(([i, perFile]) => perFile[`${i}.0`]?.thumbnail);
5288
+ const thumbs = Object.values(published.joinPayload).map((perFile) => perFile['thumbnail']?.thumbnail);
5093
5289
  if (thumbs.length !== files.length || thumbs.some((t) => t === undefined)) throw new Error(`a thumbnail per file: ${thumbs.join()}`);
5094
5290
 
5095
5291
  // A GATE TYPE OF YOUR OWN is CODE, registered by name the way `distinct` is written. A route its `fn` returns that
@@ -5298,7 +5494,7 @@ its three routes, so the third is the control sitting in the same report.
5298
5494
  | the author writes | the check says |
5299
5495
  |---|---|
5300
5496
  | a key path the producer FORBIDS at any step | **`no`** on every arc leaving by a key-resolved route, naming the gate, the key, the STEP it stopped on, the word the schema itself uses, and the route |
5301
- | a key path the producer describes | those arcs carry `required` along the path, so a consumer requiring it **`fits`** |
5497
+ | a key path the producer describes | those arcs carry `required` along the path, so a consumer requiring it **`fits`** — and through a bracket index `[n]`, `minItems: n + 1` on that array and nothing about any element past it, since `items` speaks for every element ([ADR 0137](doc/decisions/0137-a-key-path-indexes-an-array-by-a-bracket-suffix-and-a-plain-segment-keeps-its-meaning.md)) |
5302
5498
  | the same arc, with the type declaring no guarantee | **`no`** again — *consumer requires "owner", producer does not guarantee it* |
5303
5499
  | an arc leaving by a route the type does NOT call key-resolved | its own verdict, unchanged — the refusal is the ROUTE's and not the gate's |
5304
5500