fyflow-orchestration 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +249 -53
- package/README.md +5 -5
- package/dist/browser/chunks/{chunk-NA6QQAZ2.js → chunk-4JFQX7DL.js} +164 -49
- package/dist/browser/index.js +546 -3
- package/dist/browser/scheduler/index.js +1 -1
- package/dist/node/chunks/{chunk-NA6QQAZ2.js → chunk-4JFQX7DL.js} +164 -49
- package/dist/node/index.js +546 -3
- package/dist/node/scheduler/index.js +1 -1
- package/dist/types/index.d.ts +1 -0
- package/dist/types/src/definition/assist.d.ts +1 -1
- package/dist/types/src/definition/condition.d.ts +21 -3
- package/dist/types/src/definition/definition-schema.d.ts +23 -0
- package/dist/types/src/definition/types.d.ts +8 -5
- package/dist/types/src/definition/validate.d.ts +50 -6
- package/dist/types/src/definition/wiring.d.ts +15 -5
- package/dist/types/src/driver/driver.d.ts +17 -6
- package/dist/types/src/fold/run-state.d.ts +17 -0
- package/package.json +2 -1
package/AGENTS.md
CHANGED
|
@@ -218,11 +218,12 @@ if (v48.join() !== 'FD-V48') throw new Error(`expected FD-V48, got ${v48.join()}
|
|
|
218
218
|
>
|
|
219
219
|
> **What each instance receives, and what the join hands on.** Each instance is dispatched with
|
|
220
220
|
> **its own item** — `over`'s array at that instance's index, as its own copy, so no two instances
|
|
221
|
-
> share an object — and the join carries **every item's result keyed by that item's
|
|
222
|
-
> map of three items
|
|
223
|
-
> `{ "0": …, "1": …, "2": … }
|
|
224
|
-
>
|
|
225
|
-
>
|
|
221
|
+
> share an object — and the join carries **every item's result keyed by that item's INDEX**, one
|
|
222
|
+
> segment, wherever the map sits: a map of three items dispatches three tasks and its successor
|
|
223
|
+
> receives `{ "0": …, "1": …, "2": … }`, and so does a map nested in a fan-out branch. **Since 0.12.0**
|
|
224
|
+
> ([ADR 0138](doc/decisions/0138-a-join-hands-on-each-branch-under-its-own-key-and-a-checkpoint-of-the-old-keying-is-refused-by-format.md)): until then a nested map keyed its collection with the whole path — a map
|
|
225
|
+
> inside branch 1 gave `{ "1.0": …, "1.1": … }` — and *What a join carries downstream* in section 5 says
|
|
226
|
+
> what moved with it.
|
|
226
227
|
> [ADR 0037](doc/decisions/0037-what-a-branch-receives-is-a-property-of-the-fan-out-and-what-a-join-carries-is-a-property-of-the-region.md),
|
|
227
228
|
> and see *What a join carries downstream* in section 5 for what happens when an item's branch does
|
|
228
229
|
> not arrive.
|
|
@@ -292,7 +293,7 @@ if (!noGroups.some((f) => f.rule === 'FD-V31' && f.at === 'score')) {
|
|
|
292
293
|
| `join` | `of: '<fanout id>'`. Fires when its policy says so | it does not have **exactly one** edge, or that edge carries `when` or `on` (`FD-V7`) |
|
|
293
294
|
| `end` | terminal. `outcome` is **your label**, never interpreted | — |
|
|
294
295
|
| `ask` | waits for a PERSON — `edges`, and optionally `settings` (opaque), `answerSchema` (what a `done` answer must satisfy) and `needs` (what reaches it). Never dispatched, and has a task id; its answer is its output. A run PARKS at one and takes the person's answer at `resolve()` (section 7) | it has no `edges`, or **no edge of it is eligible for `done`** (`FD-V7`, as a task); `settings` that is not a plain object (`FD-V21`); a key only a task declares — `type`, `pool`, `optional`, `retryPolicy`, `workerGroups`, `transform` — one finding per key (`FD-V47`); an `answerSchema` outside the schema subset (`FD-V48`) |
|
|
295
|
-
| `gate` | `type: '<gate type>'`, optional `settings`, and `edges` of `{ to, route? }` — routes the branch that reaches it and hands on what reached it. `settings.key` is a dotted key path over that value
|
|
296
|
+
| `gate` | `type: '<gate type>'`, optional `settings`, and `edges` of `{ to, route? }` — routes the branch that reaches it and hands on what reached it. `settings.key` is a dotted key path over that value — `series.id`, and since 0.12.0 an array element by a bracket index, `files[0].series` (a plain `xs.1` is still a KEY, never an index). Built in: `distinct`, `if` and `switch`. Run by the driver where a branch enters it (section 19) | its `type` is not a registered gate type, or its type's `routesFor` throws or returns a malformed route set for this gate, or its `settingsProblem` throws or returns neither a message nor `null` (`FD-V39`); its `settings` are not a plain object its type's schema accepts, its type's `settingsProblem` returns a message, they cannot be copied for the type, or its `key` is not a key path (`FD-V40`); a route is unwired, wired twice, or not its type's, or a gate edge carries `when`, `on` or `needs` (`FD-V41`) — and `route` on any other node's edge is `FD-V41` too |
|
|
296
297
|
|
|
297
298
|
**A `task` node — and an `ask` — must have an edge that can be taken when it succeeds.** A node whose only edge is
|
|
298
299
|
`{ when: { predicate: 'p' }, to: 'x' }` is refused at construction — it is the mistake the format most invites, and
|
|
@@ -347,6 +348,7 @@ that object and nothing else — there is no wrapper and no reserved key.
|
|
|
347
348
|
| `"someNode"` | that node's result, **on your own branch** |
|
|
348
349
|
| `["someNode"]` | that node's results as an **array**, in ascending branch-path order. Legal only when you are OUTSIDE a region that node is in — which is what makes it more than one |
|
|
349
350
|
| **a `?` on either of those** — `"someNode?"`, `["someNode?"]` | the same thing, except that **nothing to give leaves the key out** (or gives `[]` for the array form) instead of stopping the run. This is how you say *if it ran* |
|
|
351
|
+
| **`!error` after the node id, on any of those** — `"someNode!error"`, `["someNode!error?"]`, … | that node's **ERROR** instead of its result: the string its failure was reduced to, `null` where it succeeded (since 0.11.0, below) |
|
|
350
352
|
|
|
351
353
|
**What a reference hands you is ONE shared, frozen copy — read it, never write it** — from 0.10.0 ([ADR 0126](doc/decisions/0126-a-references-value-is-one-frozen-copy-shared-by-every-reader.md),
|
|
352
354
|
`spec/flow-definition.md` v0.47). The value under a `"someNode"` or `["someNode"]` key (either with `?`) is ONE deep copy
|
|
@@ -521,7 +523,7 @@ that passed it through.
|
|
|
521
523
|
| where it is legal | every node and every edge — at the entry, on a task, it equals `"."`, and both spellings are accepted. **On an edge it is NOT refused** the way `"."` is: a predicate has no `ctx.payload`, so `needs: { run: '$payload' }` is how a decision routes on `needs.run.amount` with no task before it |
|
|
522
524
|
| always there | no `?` and no array form: `["$payload"]` and `"$payload?"` are `FD-V26`, each naming the bare form. The whole `$` prefix is reserved — `$` is outside the node-id charset (`FD-V16`), so the token can never collide with a node — and any other `$` spelling (`"$input"`, a typo) is `FD-V26` naming the two `$` references |
|
|
523
525
|
| `"$item"` | the prefix's second reference, the item of the map instance the reader is in — the next subsection |
|
|
524
|
-
| on a resume | the same value, from the stored header: nothing new is written to the log, the header or the checkpoint (`CHECKPOINT_FORMAT`
|
|
526
|
+
| on a resume | the same value, from the stored header: nothing new is written to the log, the header or the checkpoint (`CHECKPOINT_FORMAT` does not move for it — it is 2 since 0.12.0, for the join's keys, section 5) |
|
|
525
527
|
| at `start()` | `FD-R4` holds a task's composed `"$payload"` key to its worker's `inputSchema` like any other key (section 9) — composed from the value you passed, before the run exists |
|
|
526
528
|
| in the wiring check | an arc composed through a `"$payload"` key is `unchecked`: the run's payload is a value no schema declares (section 15) |
|
|
527
529
|
| what it costs | ONE copy of the whole payload per RUN, from 0.10.0 — however many instances of a `map` read it. *(Until 0.10.0 one copy per key per read: inside a `map`, once per instance — the cliff above at its largest.)* On an ASK the copy is also written into the run's LOG: the `asked` element keeps what reached the ask (section 7) |
|
|
@@ -616,7 +618,7 @@ passed it through.
|
|
|
616
618
|
| always there | no `?` and no array form: `["$item"]` and `"$item?"` are `FD-V26`, each naming the bare form |
|
|
617
619
|
| what it is | its OWN deep copy of the item, per key per read — two `"$item"` keys on one node are two objects, and a worker or a predicate writing into one reaches nothing else |
|
|
618
620
|
| what an instance KEEPS | its item, by reference to the element the map received — and only where some node of that map's region reads `"$item"`: a map nobody reads it from keeps nothing. **Released when nothing of that instance may still be entered** — its branch has ended (arrived at the map's join, exited the region, or been lost) AND no branch of a region nested inside it is still live — told by those branches' terminal region events, never by a join: an ask parked in the instance keeps it until the person answers, and a nested fan-out whose join a policy fired early keeps it until its last branch ends. Released whole when the run halts, is cancelled or is stopped (`spec/run-state.md` `RN-R13`) |
|
|
619
|
-
| on a resume | rebuilt by the replay, which re-enters the map: nothing new is written to the log, the header or the checkpoint (`CHECKPOINT_FORMAT`
|
|
621
|
+
| on a resume | rebuilt by the replay, which re-enters the map: nothing new is written to the log, the header or the checkpoint (`CHECKPOINT_FORMAT` does not move for it — it is 2 since 0.12.0, for the join's keys, section 5) |
|
|
620
622
|
| at `start()` | `FD-R4` composes it on an ENTRY map's branch task from the element the walk reached — which IS that instance's item — and holds it to the worker's `inputSchema` like any key (section 9). *Until 0.8.0's W191 ANY `needs` on such a task — `"."` included — made `start()` throw an `UnresolvedBranchPath` instead of `BoundaryRefused`* |
|
|
621
623
|
| in the wiring check | an arc composed through a `"$item"` key reads the enclosing map's ITEM schema, one sample per element; `unchecked` for an ENTRY map, whose items come from the run's payload, which no schema declares (section 15) |
|
|
622
624
|
| what it costs | one copy of ONE element per key per read — the cheap end of section 3's cliff — and the retention above, at most the `over` array the log already holds |
|
|
@@ -691,6 +693,83 @@ const [finding] = validateDefinition(outside, registry);
|
|
|
691
693
|
if (finding?.rule !== 'FD-V49' || !finding.detail.includes('move this node inside the map')) throw new Error(`expected FD-V49, got ${finding?.rule}`);
|
|
692
694
|
```
|
|
693
695
|
|
|
696
|
+
### `"someNode!error"` — what FAILED, from the nodes after it
|
|
697
|
+
|
|
698
|
+
**`"lookup!error"` names node `lookup`'s ERROR** — the outcome's `error`, the string the scheduler reduced the failure to
|
|
699
|
+
(section 6) — where `"lookup"` names its result (`spec/flow-definition.md` v0.49,
|
|
700
|
+
[ADR 0131](doc/decisions/0131-a-reference-may-name-a-nodes-error.md), since 0.11.0). A failed task's result is `null`, so
|
|
701
|
+
until it a node after a failure edge — an ask asking a person *what now?* above all — could not say what failed.
|
|
702
|
+
|
|
703
|
+
| | |
|
|
704
|
+
|---|---|
|
|
705
|
+
| what it is | the effective outcome's `error`: the record's, or a RESOLUTION's where a person resolved the node (`resolve(id, 'failed', { error })` on a parked task, a declined ask; `'resolved as failed'` where they gave none). **`null` where the output SUCCEEDED** — the key is present, holding `null` |
|
|
706
|
+
| its forms | the plain reference's four: `"lookup!error"`, `"lookup!error?"`, `["lookup!error"]`, `["lookup!error?"]` — the `?` goes LAST. It names the same node's same outputs as `"lookup"`, so `FD-V27`-`FD-V29` and the `?` and array rules are the plain form's, and the fixes `FD-V28` and `FD-V29` name keep the `!error` (`FD-V27` names the node, as for `"lookup"`) |
|
|
707
|
+
| where it is legal | wherever `"lookup"` is: a `task`'s, a `map`'s or an `ask`'s `needs`, and an edge's (`ctx.needs`, section 4) |
|
|
708
|
+
| what it is NOT | a path: the error is a field of the OUTCOME beside `result`, and `error` is the one part a reference may name. `"lookup?!error"`, `"!error"` and `"lookup!err"` are `FD-V26`, each naming its fix |
|
|
709
|
+
| what it costs | nothing to copy — a string or `null`, never the shared frozen copy of the result (above) |
|
|
710
|
+
| in the wiring check | the key composes as `{ type: ['string', 'null'] }` with the sample `''`, whatever the producer declares (section 15) |
|
|
711
|
+
| `referenceTarget` | the error form parses to the plain form's object plus `part: 'error'`, present ONLY there (`AimedReference`), so every other spelling parses as before. A resolver of your own must read `part`; an aiming rule may ignore it |
|
|
712
|
+
|
|
713
|
+
**The default — a failure edge handing `{ error }` to its successor — was declined** (P43 (c2)): it would change what every
|
|
714
|
+
existing failure edge's successor receives. You name the error where you want it.
|
|
715
|
+
|
|
716
|
+
The app's case — `lookup` fails, an ask after its failure edge shows the person why, and the node both roads reach reads
|
|
717
|
+
the error beside the result:
|
|
718
|
+
|
|
719
|
+
```ts run
|
|
720
|
+
import { Driver, fold, liveToken, validateDefinition, type FlowDefinition } from 'fyflow-orchestration';
|
|
721
|
+
|
|
722
|
+
const added: { id: string; payload: { input: unknown } }[] = [];
|
|
723
|
+
const listeners = new Map<string, ((e: { detail: unknown }) => void)[]>();
|
|
724
|
+
const scheduler = {
|
|
725
|
+
addTask(task: unknown) { added.push(task as { id: string; payload: { input: unknown } }); },
|
|
726
|
+
addEventListener(type: string, fn: (e: { detail: unknown }) => void) { listeners.set(type, [...(listeners.get(type) ?? []), fn]); },
|
|
727
|
+
};
|
|
728
|
+
const handedInput = (id: string) => added.find((t) => t.id === id)?.payload.input;
|
|
729
|
+
const fire = (type: string, detail: unknown) => { for (const fn of listeners.get(type) ?? []) fn({ detail }); };
|
|
730
|
+
|
|
731
|
+
const flow: FlowDefinition = {
|
|
732
|
+
flowId: 'lookup-or-ask', version: 1, entry: 'lookup',
|
|
733
|
+
nodes: [
|
|
734
|
+
// `optional`, so a failure SETTLES failed and routes the failure edge (a non-optional task would park, section 7).
|
|
735
|
+
{ id: 'lookup', kind: 'task', type: 'Work', optional: true, edges: [{ on: ['failed'], to: 'resolve' }, { to: 'publish' }] },
|
|
736
|
+
{ id: 'resolve', kind: 'ask', needs: { why: 'lookup!error' },
|
|
737
|
+
answerSchema: { type: 'object', required: ['value'], properties: { value: { type: 'string' } } },
|
|
738
|
+
edges: [{ on: ['failed'], to: 'skipped' }, { to: 'publish' }] },
|
|
739
|
+
{ id: 'publish', kind: 'task', type: 'Work', needs: { found: 'lookup', why: 'lookup!error', given: 'resolve?' }, edges: [{ to: 'done' }] },
|
|
740
|
+
{ id: 'skipped', kind: 'end', outcome: 'skipped' },
|
|
741
|
+
{ id: 'done', kind: 'end', outcome: 'completed' },
|
|
742
|
+
],
|
|
743
|
+
};
|
|
744
|
+
const types = { workers: { w: { url: 'worker.js' } }, nodeTypes: { Work: { worker: 'w' } } };
|
|
745
|
+
const registry = { predicates: [], ...types };
|
|
746
|
+
if (validateDefinition(flow, registry).length) throw new Error('"lookup!error" is legal on an ask and on a task');
|
|
747
|
+
|
|
748
|
+
const driver = new Driver(flow, { runId: 'r1', scheduler, createTask: (c) => c, predicates: new Map(), joinPolicy: liveToken, ...types });
|
|
749
|
+
driver.start({ invoice: 'INV-?' });
|
|
750
|
+
fire('task.failed', { id: 'r1:lookup:-:0', state: 'failed', result: null, error: 'lookup timed out after 30 s' });
|
|
751
|
+
|
|
752
|
+
// The ask parks with the error: what it shows the person.
|
|
753
|
+
const asked = driver.foldInputs().find((e) => e.kind === 'asked');
|
|
754
|
+
console.log(asked?.kind === 'asked' && asked.input); // { why: 'lookup timed out after 30 s' }
|
|
755
|
+
if (JSON.stringify(asked?.kind === 'asked' && asked.input) !== '{"why":"lookup timed out after 30 s"}') throw new Error('the ask is handed the error');
|
|
756
|
+
|
|
757
|
+
driver.resolve('r1:resolve:-:0', 'done', { result: { value: 'INV-9' }, actor: 'ana' });
|
|
758
|
+
console.log(handedInput('r1:publish:-:0')); // { found: null, why: 'lookup timed out after 30 s', given: { value: 'INV-9' } }
|
|
759
|
+
if (JSON.stringify(handedInput('r1:publish:-:0')) !== '{"found":null,"why":"lookup timed out after 30 s","given":{"value":"INV-9"}}') {
|
|
760
|
+
throw new Error(`publish received ${JSON.stringify(handedInput('r1:publish:-:0'))}`);
|
|
761
|
+
}
|
|
762
|
+
if (fold(driver.foldInputs()).outcome !== 'running') throw new Error('publish is in flight');
|
|
763
|
+
|
|
764
|
+
// A part other than `error`, or the `?` in the wrong place, is refused naming the fix.
|
|
765
|
+
const refuse = (ref: unknown) => validateDefinition({ ...flow, nodes: flow.nodes.map((n) =>
|
|
766
|
+
(n.id === 'publish' ? { ...n, needs: { why: ref } } : n)) } as FlowDefinition, registry).map((f) => `${f.rule}: ${f.detail}`);
|
|
767
|
+
for (const [ref, fix] of [['lookup!err', 'write "lookup!error"'], ['lookup?!error', 'write "lookup!error?"']] as const) {
|
|
768
|
+
const [finding] = refuse(ref);
|
|
769
|
+
if (!finding?.startsWith('FD-V26') || !finding.includes(fix)) throw new Error(`${ref}: ${finding}`);
|
|
770
|
+
}
|
|
771
|
+
```
|
|
772
|
+
|
|
694
773
|
### Validation happens at construction, not at routing time
|
|
695
774
|
|
|
696
775
|
`new Driver(...)` validates the whole definition before anything is queued. A typo cannot become a
|
|
@@ -724,6 +803,87 @@ It would have said *every path from a branch entry reaches the join* — which w
|
|
|
724
803
|
that routes a branch away from its join invalid input, and that case is the one this library exists
|
|
725
804
|
to get right (`ADR 0003`). Do not add it.
|
|
726
805
|
|
|
806
|
+
### The definition's SHAPE, as data — `definitionSchema()` and `definitionSchema(registry)`
|
|
807
|
+
|
|
808
|
+
**`definitionSchema()` returns everything above as a JSON Schema, draft 2020-12** (`spec/flow-definition.md` v0.50,
|
|
809
|
+
[ADR 0132](doc/decisions/0132-a-definitions-shape-is-a-schema-built-from-the-validator.md), since 0.11.0): every node kind
|
|
810
|
+
and its keys, which are required, the edges of each kind, the three `when` shapes, the `needs` grammar (`"lookup!error"`
|
|
811
|
+
included), a `retryPolicy`, a `transform` and an `answerSchema` — each key with a `description`. Hand it to any 2020-12
|
|
812
|
+
validator, or to an agent writing a flow, BEFORE the findings. It is BUILT from the validator's own key sets, so its
|
|
813
|
+
keys cannot drift from them (the reference tokens and the operator partition are written beside them and held to the
|
|
814
|
+
validator by its tests); it is a new plain object on every call, yours to change. A join's `of` and a fanout's branch
|
|
815
|
+
entries are held to the id charset, a non-string refused — as `validateDefinition` refuses one under `FD-V21` **since
|
|
816
|
+
0.12.0** (`spec/flow-definition.md` v0.52, [Q774](doc/questions/q774.md)). **That is a change in what is accepted**:
|
|
817
|
+
`of: null`, or `branches: ["b", "c", 5]`, passed 0.11.0's validator with no finding when nothing else in the flow
|
|
818
|
+
tripped over it, and is refused now. **And the other way, since 0.12.0** (`spec/flow-definition.md` v0.53,
|
|
819
|
+
[ADR 0137](doc/decisions/0137-a-key-path-indexes-an-array-by-a-bracket-suffix-and-a-plain-segment-keeps-its-meaning.md)):
|
|
820
|
+
a gate's key and a comparison's `path` may carry a bracket index — `files[0].series`, `outcome.result.files[0].series` — so both patterns accept it — and
|
|
821
|
+
`KEY_PATH_RE`, the gate key's, is the exported constant whose `source` moved. Only more is accepted.
|
|
822
|
+
|
|
823
|
+
| | |
|
|
824
|
+
|---|---|
|
|
825
|
+
| it refuses | what a node, an edge or a value gets wrong in its OWN shape: an undeclared or missing key, a wrong type, an id outside `[A-Za-z0-9_-]{1,64}`, an unknown kind, a malformed reference, `when`, `retryPolicy` or `transform`, a task's key on an ask, an `answerSchema` outside the subset |
|
|
826
|
+
| it does not | anything that needs ANOTHER node or the REGISTRY — whether `to` names a node, reachability, regions and joins, where a reference points, every registered name (types, pools, groups, languages, predicates, gate types and their routes) — the last is what `definitionSchema(registry)` adds, below |
|
|
827
|
+
| the authority | **`validateDefinition(flow, registry)`**, or `runtime.check(flow)` on a runtime — for all of that and for the whole. What the schema refuses, `validateDefinition` refuses under every registry; a document it accepts may still be refused there. The root `description` says so |
|
|
828
|
+
| not | a check the library runs (nothing evaluates it), nor in [ADR 0053](doc/decisions/0053-the-schema-validator-is-a-built-in-subset-and-a-keyword-outside-it-is-refused.md)'s subset — it uses `$defs`, `$ref`, `oneOf`, `if`/`then`, `pattern` |
|
|
829
|
+
|
|
830
|
+
**`definitionSchema(registry)` is the same shape SPECIALISED to the `Registry` you validate against** (`spec/flow-definition.md`
|
|
831
|
+
v0.51, since 0.11.0) — what an agent most often gets wrong, as data: a task's `type` is one of your node types, and per
|
|
832
|
+
type (an `if`/`then`) its `pool`, its `settings` against its worker's `settingsSchema`, and a `transform` where the type only
|
|
833
|
+
transforms; `workerGroups`, `transform.lang` and a predicate name from your sets; a gate's `type`, and per gate type its
|
|
834
|
+
`settings` and — for a static type — its edges' `route` names. A registry `checkRegistry` refuses (what `new Driver` would
|
|
835
|
+
refuse) is never specialised: the call throws that refusal. What it refuses, `validateDefinition(flow, registry)` refuses.
|
|
836
|
+
|
|
837
|
+
| | |
|
|
838
|
+
|---|---|
|
|
839
|
+
| it cannot cleanly carry | the `settings` check over the EFFECTIVE settings (your type's defaults merged under the node's): a key the defaults supply is not required of the node, and a default's own value — and a top-level keyword of your settingsSchema other than `properties`, `required` and `additionalProperties` — is not checked; a `minLength` above 1 inside one of your schemas, emitted as half of it rounded up (your schemas count UTF-16 units, JSON Schema counts code points); a DERIVED gate type's routes (`switch`'s are its cases); every route wired EXACTLY ONCE; a gate type's `settingsProblem` and a language's `compile` |
|
|
840
|
+
| annotation only | each node type's worker `inputSchema` — at `$defs.inputSchema` in its type's branch, constraining nothing (it is runtime input) |
|
|
841
|
+
| the authority | as above, against that registry. The root `description` names every row of this table |
|
|
842
|
+
| mind | a validator that tests a property with `in` (`@cfworker/json-schema` 4.1.1 does) misreads a schema property of yours NAMED as an `Object.prototype` member — `constructor`, `toString`, `__proto__` — and refuses or throws where 2020-12 accepts; name your settings otherwise |
|
|
843
|
+
|
|
844
|
+
```ts run
|
|
845
|
+
import { definitionSchema, validateDefinition, type FlowDefinition } from 'fyflow-orchestration';
|
|
846
|
+
|
|
847
|
+
const schema = definitionSchema();
|
|
848
|
+
console.log(schema.$schema); // https://json-schema.org/draft/2020-12/schema
|
|
849
|
+
if (schema.$schema !== 'https://json-schema.org/draft/2020-12/schema') throw new Error('a 2020-12 document');
|
|
850
|
+
if (!String(schema.description).includes('validateDefinition')) throw new Error('it names the authority for the rest');
|
|
851
|
+
|
|
852
|
+
type Def = { properties: Record<string, { description?: string }>; required: string[]; additionalProperties: boolean };
|
|
853
|
+
const defs = schema.$defs as Record<string, Def> & { reference: { anyOf: { pattern?: string }[] } };
|
|
854
|
+
console.log(defs.askNode.required); // [ 'id', 'kind', 'edges' ]
|
|
855
|
+
if (defs.askNode.additionalProperties !== false || 'type' in defs.askNode.properties) throw new Error('an ask declares no type');
|
|
856
|
+
if (!defs.taskNode.properties.retryPolicy?.description) throw new Error('every key is described');
|
|
857
|
+
|
|
858
|
+
// The reference grammar, as a pattern any validator can apply — the error form accepted, a misplaced "?" not.
|
|
859
|
+
const reference = new RegExp(defs.reference.anyOf[0]!.pattern!);
|
|
860
|
+
if (!reference.test('lookup!error?') || reference.test('lookup?!error')) throw new Error('the "!error" grammar');
|
|
861
|
+
|
|
862
|
+
// A shape the schema accepts can still be refused: an edge to a node that does not exist is the VALIDATOR's.
|
|
863
|
+
const dangling = { flowId: 'f', version: 1, entry: 'a', nodes: [{ id: 'a', kind: 'task', type: 'Work', edges: [{ to: 'ghost' }] }] } as FlowDefinition;
|
|
864
|
+
const rules = validateDefinition(dangling, { predicates: [], workers: { w: { url: 'worker.js' } }, nodeTypes: { Work: { worker: 'w' } } }).map((f) => f.rule);
|
|
865
|
+
if (rules.join() !== 'FD-V2') throw new Error(`the graph is validateDefinition's: ${rules.join()}`);
|
|
866
|
+
if (definitionSchema() === schema) throw new Error('a new object per call');
|
|
867
|
+
|
|
868
|
+
// Specialised to a registry: its node types (the built-in Transform beside yours), each type's pools and settings.
|
|
869
|
+
const registry = {
|
|
870
|
+
predicates: ['isUrgent'],
|
|
871
|
+
workers: { w: { url: 'worker.js', settingsSchema: { type: 'object', properties: { depth: { type: 'integer', minimum: 1 } }, additionalProperties: false } } },
|
|
872
|
+
nodeTypes: { Work: { worker: 'w' } },
|
|
873
|
+
};
|
|
874
|
+
const specialised = definitionSchema(registry);
|
|
875
|
+
type Task = { properties: { type: { enum: string[] } }; allOf: { then: { properties: { pool: { enum: string[] }; settings?: unknown } } }[] };
|
|
876
|
+
const task = (specialised.$defs as { taskNode: Task }).taskNode;
|
|
877
|
+
console.log(task.properties.type.enum); // [ 'Work', 'Transform' ]
|
|
878
|
+
console.log(task.allOf[0]!.then.properties.pool.enum); // [ 'Work' ]
|
|
879
|
+
if (task.allOf[0]!.then.properties.settings === undefined) throw new Error('Work\'s settings are held to its worker\'s schema');
|
|
880
|
+
if (!String(specialised.description).includes('EXACTLY ONCE')) throw new Error('it names what it cannot carry');
|
|
881
|
+
// A registry new Driver would refuse is never specialised.
|
|
882
|
+
let refused = '';
|
|
883
|
+
try { definitionSchema({ ...registry, nodeTypes: { Lost: { worker: 'missing' } } }); } catch (e) { refused = (e as Error).name; }
|
|
884
|
+
if (refused !== 'WorkerMissing') throw new Error(`a refused registry throws its refusal: ${refused}`);
|
|
885
|
+
```
|
|
886
|
+
|
|
727
887
|
---
|
|
728
888
|
|
|
729
889
|
## 4. Decisions and predicates
|
|
@@ -784,7 +944,7 @@ if (refuse({ all: [{ any: [{ path: 'outcome.result.a', op: 'eq', value: 1 }] }]
|
|
|
784
944
|
|
|
785
945
|
| | |
|
|
786
946
|
|---|---|
|
|
787
|
-
| `path` | a key path over the context, **rooted at `outcome` or `needs`** — `outcome.result.x`, `outcome.taskId`, `needs.earlier.flag
|
|
947
|
+
| `path` | a key path over the context, **rooted at `outcome` or `needs`** — `outcome.result.x`, `outcome.taskId`, `needs.earlier.flag`, and an array element by a bracket index after a key, `outcome.result.items[0].score` (never on the root). No wildcard, no expression |
|
|
788
948
|
| `op` | one of `eq`, `ne`, `lt`, `lte`, `gt`, `gte`, `startsWith`, and **nothing else** |
|
|
789
949
|
| `value` | a scalar: `null`, a boolean, a finite number or a string |
|
|
790
950
|
| `default` | optional — what is compared when the path does not resolve **or resolves to `null`**, which is exactly `??`. Without it an absent path is **no match**, and a resolved `null` is compared as the scalar it is |
|
|
@@ -860,7 +1020,7 @@ flagged low confidence* was not expressible until v0.12 —
|
|
|
860
1020
|
answering [Q8](doc/questions/q8.md).
|
|
861
1021
|
|
|
862
1022
|
**An edge may carry `needs`, and its predicate sees the result under `ctx.needs`.** Same grammar as
|
|
863
|
-
a node's (section 3) — a node id, or a one-element array holding one,
|
|
1023
|
+
a node's (section 3) — a node id, or a one-element array holding one, since 0.8.0 `"$payload"` and `"$item"`, and since 0.11.0 a node's error, `"n!error"` —
|
|
864
1024
|
**minus `"."`**, which an edge has no use for: *what a predicate would have received anyway* is `ctx.outcome`. `"$payload"`
|
|
865
1025
|
and `"$item"` are not refused there: a predicate has no other road to the run's payload or to its map instance's item
|
|
866
1026
|
(section 3's two subsections route on them).
|
|
@@ -1071,10 +1231,12 @@ is not a supported choice. See
|
|
|
1071
1231
|
|
|
1072
1232
|
### What a join carries downstream
|
|
1073
1233
|
|
|
1074
|
-
**The arrived branches' results,
|
|
1075
|
-
with an object
|
|
1076
|
-
|
|
1077
|
-
|
|
1234
|
+
**The arrived branches' results, each under the branch's OWN key.** A satisfied join dispatches its
|
|
1235
|
+
successor with an object holding one entry per branch that **reached** it — keyed by the branch's
|
|
1236
|
+
**entry node id** for a `fanout` and by the **item index** for a `map`, one segment at every depth —
|
|
1237
|
+
whose values are what each of those branches carried in. So a fan-out's results are not thrown away,
|
|
1238
|
+
a key path reaches into any depth (`input.prep.1`), and the value is the same whatever order the
|
|
1239
|
+
branches finished in.
|
|
1078
1240
|
|
|
1079
1241
|
```ts run
|
|
1080
1242
|
import { createRuntime } from 'fyflow-orchestration/scheduler';
|
|
@@ -1096,7 +1258,7 @@ const flow: FlowDefinition = {
|
|
|
1096
1258
|
flowId: 'collect', version: 1, entry: 'split',
|
|
1097
1259
|
nodes: [
|
|
1098
1260
|
{ id: 'split', kind: 'fanout', branches: ['scoreA', 'scoreB'] },
|
|
1099
|
-
// `scoreA` is
|
|
1261
|
+
// `scoreA` is the FIRST entry and is made SLOW on purpose, so `scoreB` settles first.
|
|
1100
1262
|
{ id: 'scoreA', kind: 'task', type: 'Scorer', settings: { score: 0.9, ms: 40 }, edges: [{ to: 'collect' }] },
|
|
1101
1263
|
{ id: 'scoreB', kind: 'task', type: 'Scorer', settings: { score: 0.4 }, edges: [{ to: 'collect' }] },
|
|
1102
1264
|
{ id: 'collect', kind: 'join', of: 'split', edges: [{ to: 'publish' }] },
|
|
@@ -1115,7 +1277,7 @@ const inputs: FoldInput[] = [];
|
|
|
1115
1277
|
const state = await runtime.start(flow, {}, { onFoldInput: (input) => { inputs.push(input); } }).done;
|
|
1116
1278
|
await runtime.shutdown();
|
|
1117
1279
|
|
|
1118
|
-
// **The order is READ, not assumed.** The delay above ASKS for
|
|
1280
|
+
// **The order is READ, not assumed.** The delay above ASKS for `scoreB` to settle first; this is
|
|
1119
1281
|
// what the run actually did, and the claim below is about the collection being independent of it.
|
|
1120
1282
|
const settledOrder = inputs.flatMap((i) =>
|
|
1121
1283
|
(i.kind === 'outcome' && /:score[AB]:/.test(i.record.taskId) ? [i.record.taskId.split(':')[1]] : []));
|
|
@@ -1124,13 +1286,14 @@ console.log(settledOrder.join(' then ')); // 'scoreB then scoreA'
|
|
|
1124
1286
|
// `publish` is the join's successor, so what it received IS the collection — and `tag` returns what
|
|
1125
1287
|
// it was handed with its own label added, so the collection is readable in the result.
|
|
1126
1288
|
const published = Object.values(state.nodes).find((n) => n.nodeId === 'publish')?.result;
|
|
1127
|
-
console.log(JSON.stringify(published)); // {"
|
|
1289
|
+
console.log(JSON.stringify(published)); // {"scoreA":{"score":0.9},"scoreB":{"score":0.4},"publish":true}
|
|
1128
1290
|
|
|
1129
|
-
const want = '{"
|
|
1291
|
+
const want = '{"scoreA":{"score":0.9},"scoreB":{"score":0.4},"publish":true}';
|
|
1130
1292
|
if (JSON.stringify(published) !== want) throw new Error(`the join carried ${JSON.stringify(published)}`);
|
|
1131
|
-
// **B settled FIRST and is still under key
|
|
1132
|
-
//
|
|
1133
|
-
// order the run took, the second says the
|
|
1293
|
+
// **B settled FIRST and is still under its own key, listed SECOND.** The collection is keyed by each
|
|
1294
|
+
// branch's ENTRY ID and inserted in `branches` order, not in arrival order, which is the whole point —
|
|
1295
|
+
// and the two assertions are a pair: the first says which order the run took, the second says the
|
|
1296
|
+
// record did not follow it.
|
|
1134
1297
|
if (settledOrder.join() !== 'scoreB,scoreA') throw new Error(`settled ${settledOrder.join()} — the arrangement did not hold, so the claim below is untested`);
|
|
1135
1298
|
```
|
|
1136
1299
|
|
|
@@ -1143,20 +1306,32 @@ if (settledOrder.join() !== 'scoreB,scoreA') throw new Error(`settled ${settledO
|
|
|
1143
1306
|
2. **`{}` is a real answer, not an error.** Under `liveToken` a join fires once its region has no live
|
|
1144
1307
|
branches, and every branch reaching an `end` satisfies that with nothing arrived. If that should be
|
|
1145
1308
|
a failure instead, pass `liveTokenRequiringArrival` — see above.
|
|
1146
|
-
3. **
|
|
1147
|
-
|
|
1148
|
-
|
|
1149
|
-
|
|
1150
|
-
branch
|
|
1151
|
-
|
|
1152
|
-
`
|
|
1153
|
-
|
|
1309
|
+
3. **Every key is ONE segment, at every depth — the branch's own key.** A `fanout`'s branch is keyed by
|
|
1310
|
+
its entry node id (`FD-V9` keeps them distinct), a `map`'s instance by its item index — so a
|
|
1311
|
+
fan-out whose first branch holds a map gives `{"prep":{"0":1,"1":2},"side":{"xs":[1,2]}}`, never
|
|
1312
|
+
`"0.1"` under `"0"`. A nested region's inner join carries its own object, which becomes the value of
|
|
1313
|
+
the outer branch it sits in. The full branch path is not lost: it is the region's own path (`regionPath`) + `.` + the
|
|
1314
|
+
position — the position alone at the top level, where `regionPath` is `-` — on every `branch.*` event, task id and
|
|
1315
|
+
`inputOf` path, which are unchanged.
|
|
1316
|
+
4. **The order is the object's own.** Entries are inserted in `branches` order for a fan-out and in
|
|
1317
|
+
ascending item index for a map, and **JavaScript lists integer-like keys first, ascending** — so a
|
|
1318
|
+
fan-out whose entries are `"b"` and `"2"` lists `"2"` first. Deterministic whatever order the
|
|
1319
|
+
branches finished in; no node id is refused for it. Each entry is an own property, so even an entry
|
|
1320
|
+
id `__proto__` is a key.
|
|
1321
|
+
5. **BREAKING in 0.12.0, and a run in flight across the upgrade cannot be resumed.** Until 0.11.0 the
|
|
1322
|
+
keys were full branch paths (`{"0":{"0.0":1,"0.1":2},"1":{"xs":[1,2]}}` for the flow above) and a
|
|
1323
|
+
fan-out's were POSITIONS; a flow reading `"0.1"` or `"0"` reads nothing now. A collection reaches a
|
|
1324
|
+
checkpoint (an ask's kept input, a gate's decision, a worker's result), so **`CHECKPOINT_FORMAT` is
|
|
1325
|
+
2** and a checkpoint 0.11.0 wrote — join or none — is refused `ResumeRefused('format')` by
|
|
1326
|
+
`resume()`, `runtime.resume` and `inputOf`; a 0.11.0 driver refuses a 0.12.0 checkpoint the same
|
|
1327
|
+
way. There is no migration: drain or abandon those runs first
|
|
1328
|
+
([ADR 0138](doc/decisions/0138-a-join-hands-on-each-branch-under-its-own-key-and-a-checkpoint-of-the-old-keying-is-refused-by-format.md)).
|
|
1154
1329
|
|
|
1155
1330
|
*(This section read **"`null`, currently … this is [Q36](doc/questions/q36.md) and it is open"**
|
|
1156
1331
|
until [ADR 0037](doc/decisions/0037-what-a-branch-receives-is-a-property-of-the-fan-out-and-what-a-join-carries-is-a-property-of-the-region.md).
|
|
1157
1332
|
The argument for the `null` was real and is unchanged — which branch finishes last is genuinely
|
|
1158
1333
|
non-deterministic — and what answered it is that carrying **all** the arrivals keyed by branch is not
|
|
1159
|
-
picking one. `spec/join-policy.md` section 4, `JP-R12
|
|
1334
|
+
picking one. `spec/join-policy.md` section 4, `JP-R12`; its keys are each branch's own since 0.12.0.)*
|
|
1160
1335
|
|
|
1161
1336
|
### The same on the RUNTIME — a decision, a fan-out and a join on real workers
|
|
1162
1337
|
|
|
@@ -1191,7 +1366,7 @@ const isLowConfidence: Predicate = (ctx) => {
|
|
|
1191
1366
|
|
|
1192
1367
|
// THE FLOW — a DECISION, a FAN-OUT, a JOIN. `ingest` goes to `review` when the predicate holds,
|
|
1193
1368
|
// otherwise to `split`. `split` runs two branches concurrently; `collect` waits for both and hands
|
|
1194
|
-
// `publish` every branch's result,
|
|
1369
|
+
// `publish` every branch's result, each under its own entry id.
|
|
1195
1370
|
const flow: FlowDefinition = {
|
|
1196
1371
|
flowId: 'decide', version: 1, entry: 'ingest',
|
|
1197
1372
|
nodes: [
|
|
@@ -1219,13 +1394,13 @@ const ran = (s: RunState) => Object.values(s.nodes).map((n) => n.nodeId).sort();
|
|
|
1219
1394
|
const resultOf = (s: RunState, id: string) => Object.values(s.nodes).find((n) => n.nodeId === id)?.result;
|
|
1220
1395
|
console.log(ran(high)); // [ 'classify', 'enrich', 'ingest', 'publish' ]
|
|
1221
1396
|
console.log(ran(low)); // [ 'ingest', 'review' ]
|
|
1222
|
-
console.log(resultOf(high, 'publish')); // {
|
|
1397
|
+
console.log(resultOf(high, 'publish')); // { enrich: { confidence: 0.9, ingest: true, enrich: true }, classify: { …, classify: true }, publish: true }
|
|
1223
1398
|
|
|
1224
1399
|
// Verified by the docs gate:
|
|
1225
1400
|
if (high.outcome !== 'completed' || low.outcome !== 'completed') throw new Error(`outcomes ${high.outcome} / ${low.outcome}`);
|
|
1226
1401
|
if (ran(high).join() !== 'classify,enrich,ingest,publish') throw new Error(`high ran ${ran(high).join()}`);
|
|
1227
1402
|
if (ran(low).join() !== 'ingest,review') throw new Error(`low ran ${ran(low).join()}`);
|
|
1228
|
-
const want = '{"
|
|
1403
|
+
const want = '{"enrich":{"confidence":0.9,"ingest":true,"enrich":true},"classify":{"confidence":0.9,"ingest":true,"classify":true},"publish":true}';
|
|
1229
1404
|
if (JSON.stringify(resultOf(high, 'publish')) !== want) throw new Error(`publish received ${JSON.stringify(resultOf(high, 'publish'))}`);
|
|
1230
1405
|
```
|
|
1231
1406
|
|
|
@@ -1468,10 +1643,18 @@ keeps waiting, and the run is not stuck.
|
|
|
1468
1643
|
([W180](doc/work-items/w180-asking-and-answering-in-a-run.md), [ADR 0109](doc/decisions/0109-an-ask-parks-with-nothing-dispatched-and-its-answer-is-its-result.md), `spec/run-state.md` `RN-R36`/`RN-R37`):
|
|
1469
1644
|
|
|
1470
1645
|
- **Nothing is handed to the scheduler** — the scheduler never sees the ask's id. The driver appends ONE fold input,
|
|
1471
|
-
`{ kind: 'asked', taskId, input }`; the fold reads the node `suspended` and the run `awaiting_action`, exactly the
|
|
1646
|
+
`{ kind: 'asked', taskId, input, observedAt? }`; the fold reads the node `suspended` and the run `awaiting_action`, exactly the
|
|
1472
1647
|
words a parked task gives (so `done` waits, below), and the node's entry in `nodes` carries **`input`** — what reached
|
|
1473
1648
|
the ask, its `needs` composed — kept after the answer. Tell an ask from a parked failure by the node's KIND, or by the
|
|
1474
1649
|
`asked` element, never by an error string. In a fan-out the ask's branch is LIVE, so **its join waits for the person**.
|
|
1650
|
+
- **WHEN it was asked** (since 0.11.0, [ADR 0130](doc/decisions/0130-a-regenerated-asked-element-keeps-the-stored-runs-time.md)):
|
|
1651
|
+
the element's `observedAt` is the driver's `now` — the clock an outcome record's `observedAt` reads — and the node's
|
|
1652
|
+
entry carries it as **`askedAt`**, kept after the answer as `input` is. **A resume KEEPS the stored time**: the replay
|
|
1653
|
+
regenerates the element and takes its stored `observedAt`, so a wait measured across a resume does not restart; past
|
|
1654
|
+
the stored tail it is the resumed driver's `now`. *(An `asked` element written before 0.11.0 has none; since 0.12.0
|
|
1655
|
+
such a checkpoint is refused `format` before its events are read — section 5, `CHECKPOINT_FORMAT` 2.)* **A driver
|
|
1656
|
+
older than 0.11.0 refused a 0.11.0 checkpoint `diverged` at its first ask**; any driver before 0.12.0 refuses a 0.12.0
|
|
1657
|
+
checkpoint `format` before replaying anything — resume a run on a driver of the format that wrote it.
|
|
1475
1658
|
- **The answer is `driver.resolve(taskId, …)`** — the same call as below. `'done'` with `result` the answer, held to the
|
|
1476
1659
|
ask's `answerSchema` (refused as `BoundaryRefused` with NOTHING appended and the ask still parked — answer again;
|
|
1477
1660
|
`validateBoundaries: false` turns it off); `'failed'` with an `error` is the person DECLINING, held to nothing.
|
|
@@ -1484,7 +1667,7 @@ keeps waiting, and the run is not stuck.
|
|
|
1484
1667
|
run's log, once per ask — that element keeps what reached the ask, whatever its size. A predicate's `ctx.outcome` for an ask has `attempts: 0` and
|
|
1485
1668
|
**`schedulerTimestamp`, `startTime`, `endTime` and `executionTime` all `null`** — no scheduler was involved.
|
|
1486
1669
|
- **BREAKING in 0.8.0** (P35 decision 8): `FoldInput` gains the kind `'asked'`, and `OutcomeRecord.schedulerTimestamp`
|
|
1487
|
-
is `number | null` — `null` only on an ask's effective outcome. `NodeRun.input` is optional and additive.
|
|
1670
|
+
is `number | null` — `null` only on an ask's effective outcome. `NodeRun.input` is optional and additive — as, since 0.11.0, are the element's `observedAt` and `NodeRun.askedAt`.
|
|
1488
1671
|
- **An answer given from a callback on the driver's own step** — `onFoldInput` on the `asked` element included — is
|
|
1489
1672
|
checked at once (a refusal is thrown or `false` returned right there) and, accepted, `resolve()` returns `true` and the
|
|
1490
1673
|
resolution is appended when the step ends: so the run's checkpoint resumes. A second answer while it is pending is
|
|
@@ -1543,6 +1726,9 @@ console.log(added, parked.outcome, parked.nodes['r1:approve:-:0'].input);
|
|
|
1543
1726
|
// [ 'r1:draft:-:0' ] awaiting_action { text: 'a draft' }
|
|
1544
1727
|
if (added.join() !== 'r1:draft:-:0' || parked.outcome !== 'awaiting_action') throw new Error('the ask should park with nothing dispatched');
|
|
1545
1728
|
if (JSON.stringify(parked.nodes['r1:approve:-:0'].input) !== '{"text":"a draft"}') throw new Error('the ask keeps what reached it');
|
|
1729
|
+
// And WHEN it was asked: the element's `observedAt` (the driver's clock, `Date.now()` by default) is the entry's `askedAt`.
|
|
1730
|
+
const askedEl = driver.foldInputs().find((e) => e.kind === 'asked');
|
|
1731
|
+
if (askedEl?.kind !== 'asked' || typeof askedEl.observedAt !== 'number' || parked.nodes['r1:approve:-:0'].askedAt !== askedEl.observedAt) throw new Error('the ask keeps when it was asked');
|
|
1546
1732
|
|
|
1547
1733
|
// An answer its schema refuses is thrown at you BY NAME — nothing appended, the ask still parked.
|
|
1548
1734
|
let refused: unknown = null;
|
|
@@ -2035,7 +2221,7 @@ AFTER the target — or one that moves only an envelope, even at the target: the
|
|
|
2035
2221
|
answers with the definition you hand it. **It re-runs your predicates and gate types**, as a resume does. The value is
|
|
2036
2222
|
the reading's own — a value a `needs` reference composed is frozen, as every reader's is (section 3), the rest yours —
|
|
2037
2223
|
and nothing of a live run is reachable through it. **The input a task's worker sees AFTER its transform is not answered**
|
|
2038
|
-
([Q760](doc/questions/q760.md)): hand the envelope's `input` to `tryTransform` (section 15). **BREAKING in 0.10.0**:
|
|
2224
|
+
([Q760](doc/questions/q760.md), resolved by P43: the road serves): hand the envelope's `input` to `tryTransform` (section 15). **BREAKING in 0.10.0**:
|
|
2039
2225
|
`RunHandle` gains `inputOf`, which breaks only code that BUILDS a handle.
|
|
2040
2226
|
|
|
2041
2227
|
```ts run
|
|
@@ -2718,7 +2904,9 @@ After a refusal the run is **halted**: nothing routes for it again, and its stat
|
|
|
2718
2904
|
and FOUR surfaces are renamed — the `RunOutcome` `stopped` is `halted`, `RunState.stoppedBy` is `haltedBy`, the fold
|
|
2719
2905
|
input `{ kind: 'stopped', reason }` is `{ kind: 'halted', reason }`, and `ResumeRefusalReason`'s `stopped` is
|
|
2720
2906
|
`halted` — so a comparison against an old name, or a `{ kind: 'stopped' }` you construct, no longer compiles. A 0.7.0 checkpoint or log holding a `{ kind: 'stopped' }` element still reads as `halted`:
|
|
2721
|
-
`fold` folds it so, and a resume refuses it `halted` at its position.
|
|
2907
|
+
`fold` folds it so, and a resume refuses it `halted` at its position — but a 0.7.0 checkpoint is format 1, so since
|
|
2908
|
+
0.12.0 a resume refuses it `format` first (section 5); the `halted` read is reached only by such an element under a
|
|
2909
|
+
format-2 header, which no driver writes.
|
|
2722
2910
|
|
|
2723
2911
|
---
|
|
2724
2912
|
|
|
@@ -2979,7 +3167,7 @@ if (!haltedRun.haltedBy?.startsWith('UncloneableResult')) throw new Error(`halte
|
|
|
2979
3167
|
| ~~**map-style fan-out**~~ — **BUILT** since [W33](doc/work-items/w33-a-map-at-run-time-arity.md); see section 3 |
|
|
2980
3168
|
| ~~**RUNNING an `ask`**~~ — a node that waits for a person, its answer at `resolve()` | **BUILT** since [W180](doc/work-items/w180-asking-and-answering-in-a-run.md): a run parks at an ask with nothing handed to the scheduler, and its answer is its result — section 7. Under a cancel, a stop, a pause and a resume an ask is a parked task — never in flight, so nothing is ever sent or handed on for it — since [W181](doc/work-items/w181-an-ask-under-run-control.md) (section 7) |
|
|
2981
3169
|
| a **live, mutable view** of a run's state, or the state handed to `onFoldInput` beside each input | **no** — [P39](doc/plans/p39-what-a-step-costs-and-where-a-live-run-state-is-read.md) (b), [ADR 0122](doc/decisions/0122-a-runtimes-live-state-is-a-frozen-copy-on-its-handle.md): `handle.state()` is a frozen COPY, cached until the run moves, and `handle.outcome()` / `handle.node()` are the O(1) reads — section 8 |
|
|
2982
|
-
| a task's INPUT kept in the run state — `NodeRun.input` for a task, or `pathEnded` carrying the value that reached an end | **no** — [ADR 0046](doc/decisions/0046-a-run-retains-what-its-tasks-returned-and-a-region-releases-its-collection-when-its-join-becomes-terminal.md): a run keeps what its tasks returned. **READ it instead**, since [W205](doc/work-items/w205-what-a-node-was-handed-read-by-replaying-the-run.md): `inputOf(flow, options, { header, events }, target)` or `handle.inputOf(target)` replays the run to that node — section 8. The input a worker sees AFTER its transform: **no** ([Q760](doc/questions/q760.md)) |
|
|
3170
|
+
| a task's INPUT kept in the run state — `NodeRun.input` for a task, or `pathEnded` carrying the value that reached an end | **no** — [ADR 0046](doc/decisions/0046-a-run-retains-what-its-tasks-returned-and-a-region-releases-its-collection-when-its-join-becomes-terminal.md): a run keeps what its tasks returned. **READ it instead**, since [W205](doc/work-items/w205-what-a-node-was-handed-read-by-replaying-the-run.md): `inputOf(flow, options, { header, events }, target)` or `handle.inputOf(target)` replays the run to that node — section 8. The input a worker sees AFTER its transform: **no** ([Q760](doc/questions/q760.md), resolved — the envelope's `input` through `tryTransform` serves) |
|
|
2983
3171
|
| a **join timeout** | nothing. A stuck join is detected by the `stuck` outcome, with no clock |
|
|
2984
3172
|
| **per-join policy choice** | one policy per driver |
|
|
2985
3173
|
| a **declared event family** for `fyflow-react-components` | `onRegionEvent`, `onFoldInput` and `joinStatusOf()` — see below |
|
|
@@ -3911,7 +4099,7 @@ driver.start({ items: [1, 2, 3] });
|
|
|
3911
4099
|
|
|
3912
4100
|
**What an expression sees, and what it may not.** `$` is the node's input — the previous node's
|
|
3913
4101
|
result, the object `needs` described, a map instance's own item, or a join's collection keyed by
|
|
3914
|
-
branch
|
|
4102
|
+
each branch's own key — and the node's effective `settings` are bound as `$settings`, so one expression is
|
|
3915
4103
|
parameterised per node. **Five names are unbound, by name, on every compiled expression**: `$now`,
|
|
3916
4104
|
`$millis`, `$random` — the three every adapter must refuse, because a transform is a function of what
|
|
3917
4105
|
the run produced and run state is a fold of the log — **`$shuffle`**, JSONata's own fourth, which
|
|
@@ -3966,10 +4154,10 @@ if (!publish('$sum([*.score])', { ...registry, languages: {} }).some((f) => f.ru
|
|
|
3966
4154
|
// The envelope worker — the class the bootstrap helper puts around your worker — with the adapter's worker half by URL.
|
|
3967
4155
|
const w = new EnvelopeWorker({ worker: echo.url, languages: { jsonata: jsonata.worker } });
|
|
3968
4156
|
await w.setup();
|
|
3969
|
-
const record = {
|
|
4157
|
+
const record = { scoreA: { score: 40 }, scoreB: { score: 20 } }; // JP-R12's collection: `split`'s two entry ids
|
|
3970
4158
|
const total = await w.run({ settings: { factor: 2 }, input: record, transform: { lang: 'jsonata', expr: '$sum([*.score]) * $settings.factor' } });
|
|
3971
4159
|
if (total !== 120) throw new Error(`the worker received ${JSON.stringify(total)} — expected the reduction, 120`);
|
|
3972
|
-
const one = await w.run({ settings: {}, input: {
|
|
4160
|
+
const one = await w.run({ settings: {}, input: { scoreA: { score: 7 } }, transform: { lang: 'jsonata', expr: '$sum([*.score])' } });
|
|
3973
4161
|
if (one !== 7) throw new Error('arity one, the same expression: 7');
|
|
3974
4162
|
const none = await w.run({ settings: {}, input: {}, transform: { lang: 'jsonata', expr: '$sum([*.score])' } });
|
|
3975
4163
|
if (none !== 0) throw new Error('a join nothing reached: 0, because the reduction is over the ARRAY form');
|
|
@@ -4149,6 +4337,7 @@ whatever arcs reach the node, on the entry node too** (W166, P37 stop-and-ask (d
|
|
|
4149
4337
|
| `shape` | `'given'` — not an arc; `checkWiring` never produces it |
|
|
4150
4338
|
| `producer` | **absent** — the value came from no node (and `through` too) |
|
|
4151
4339
|
| `sample` | `{ source: 'given', value, result }` — the value you gave, and what the transform made of it; no `arity`. **Never `'example'` or `'witness'`** |
|
|
4340
|
+
| `cause` | **on `transform-fails` only** (since 0.11.0, every road — `checkWiring`'s arcs too): `'threw'` — the worker half did not import, had no default export, or its `compile` or `evaluate` threw — or `'schema'`, a result the node's `inputSchema` refused — the type `TransformFailureCause`, a root export. Absent on every other kind. Read it rather than `why`'s prefix |
|
|
4152
4341
|
| `kind` | `transform-passes`, or `transform-fails` with the finding against the node's worker `inputSchema` — or naming the error, where the expression does not compile or throws on this value; `unchecked` for a language, type, worker or worker schema the registry lacks, as on an arc, naming which — and for a `null` candidate, which takes away any transform the node declares too: there is nothing to run the value through, and holding that value to the `inputSchema` is `findings(schema, value)` |
|
|
4153
4342
|
|
|
4154
4343
|
**The value is `$` as you give it: no `needs` is composed over it**, because a `needs` composes what
|
|
@@ -4205,6 +4394,7 @@ if (entry!.kind !== 'transform-passes') throw new Error(`expected the given valu
|
|
|
4205
4394
|
const [wrong] = await tryTransform(registry, flow, 'read', rename, { input: { folder: 3 } });
|
|
4206
4395
|
console.log(`${wrong!.kind}: ${wrong!.why}`); // transform-fails: the transform's result on the given input fails the consumer's inputSchema — $.source expected string, ...
|
|
4207
4396
|
if (wrong!.kind !== 'transform-fails' || !wrong!.why.includes('$.source expected string')) throw new Error(`expected the finding, got ${wrong!.why}`);
|
|
4397
|
+
if (wrong!.cause !== 'schema') throw new Error(`a result the schema refused is cause 'schema', got ${wrong!.cause}`);
|
|
4208
4398
|
|
|
4209
4399
|
// THE INNER NODE. Without an input its one arc runs on `reader`'s declared EXAMPLE; given a value, it runs on that.
|
|
4210
4400
|
const join = { lang: 'jsonata', expr: '{ "text": $join(lines, " ") }' };
|
|
@@ -4217,6 +4407,8 @@ if (JSON.stringify(mine!.sample!.result) !== JSON.stringify({ text: 'typed by ha
|
|
|
4217
4407
|
// An expression the engine refuses: `transform-fails`, naming the throw — the compile error, on the given input.
|
|
4218
4408
|
const [broken] = await tryTransform(registry, flow, 'sum', { lang: 'jsonata', expr: '{{{' }, { input: { lines: [] } });
|
|
4219
4409
|
if (broken!.kind !== 'transform-fails' || !broken!.why.startsWith('the transform threw on the given input')) throw new Error(`expected the compile error, got ${broken!.why}`);
|
|
4410
|
+
// The same, as DATA (since 0.11.0): `cause` is 'threw' — no result was produced — where the finding above is 'schema'.
|
|
4411
|
+
if (broken!.cause !== 'threw' || entry!.cause !== undefined) throw new Error(`a throw is cause 'threw', and a passing verdict has none — got ${broken!.cause}, ${entry!.cause}`);
|
|
4220
4412
|
```
|
|
4221
4413
|
|
|
4222
4414
|
**BREAKING in 0.8.0** (P37 decision 10): **`SampleSource` gains `'given'`**, **`WiringShape` gains
|
|
@@ -4403,7 +4595,7 @@ Three things follow, and they are the whole of what you need:
|
|
|
4403
4595
|
|---|---|
|
|
4404
4596
|
| **`RunState.regions` is TWO-LEVEL** | `regions[openerId][openerPath]`. The opener's id is unchanged in meaning and in value; the second key is `-` for a region in no region, and inside a `map` region it is that map's item index |
|
|
4405
4597
|
| **every `RegionEvent` carries `regionPath` beside `regionId`** | so an event says WHICH instance it is about. The same for `UnsatisfiableJoin` |
|
|
4406
|
-
| **a join's collection is per INSTANCE** |
|
|
4598
|
+
| **a join's collection is per INSTANCE** | each instance of one opener holds its OWN collection, so two instances cannot collide — even though, since 0.12.0, their keys are each branch's own key (`JP-R12`) and two instances may hold the SAME key set ([ADR 0138](doc/decisions/0138-a-join-hands-on-each-branch-under-its-own-key-and-a-checkpoint-of-the-old-keying-is-refused-by-format.md): until 0.11.0 the keys were branch paths, carrying the instance's path as a prefix) |
|
|
4407
4599
|
|
|
4408
4600
|
**Both arities come from data, at both levels, and neither is in the document.** That is what
|
|
4409
4601
|
separates a `map` from a `fanout`: `FD-V9` requires a fan-out's branch entries to be written down, so
|
|
@@ -4489,11 +4681,12 @@ if (!arrivals.includes('perLine@0 <- 0.0') || !arrivals.includes('perLine@1 <- 1
|
|
|
4489
4681
|
}
|
|
4490
4682
|
|
|
4491
4683
|
// WHAT THE OUTER JOIN CARRIES IS TWO LEVELS DEEP. `JP-R12`: one entry per branch that arrived, keyed
|
|
4492
|
-
// by
|
|
4684
|
+
// by its OWN key — an item index at both levels — and each of the outer entries IS that instance's own
|
|
4685
|
+
// collection. So `report.0.1` is order 0's second line: the key path a gate or an `if` reads.
|
|
4493
4686
|
const report = Object.values(state.nodes).find((n) => n.nodeId === 'report')?.result as Record<string, Record<string, { sku: string }>>;
|
|
4494
4687
|
console.log(Object.keys(report).filter((k) => k !== 'reported').sort()); // [ '0', '1' ]
|
|
4495
|
-
console.log(report['0']!['
|
|
4496
|
-
if (report['0']!['
|
|
4688
|
+
console.log(report['0']!['1']!.sku); // ab-1
|
|
4689
|
+
if (report['0']!['1']!.sku !== 'ab-1' || report['1']!['0']!.sku !== 'c-0') {
|
|
4497
4690
|
throw new Error(`the outer collection's values are the inner collections: ${JSON.stringify(report)}`);
|
|
4498
4691
|
}
|
|
4499
4692
|
```
|
|
@@ -4626,7 +4819,7 @@ console.log(state.outcome); // completed
|
|
|
4626
4819
|
if (state.outcome !== 'completed') throw new Error(`expected completed, got ${state.outcome}`);
|
|
4627
4820
|
// The same answer a run nobody killed gives (section 17's):
|
|
4628
4821
|
const report = Object.values(state.nodes).find((n) => n.nodeId === 'report')?.result as Record<string, Record<string, { sku: string }>>;
|
|
4629
|
-
if (report['0']!['
|
|
4822
|
+
if (report['0']!['1']!.sku !== 'ab-1' || report['1']!['0']!.sku !== 'c-0') {
|
|
4630
4823
|
throw new Error(`the resumed run reaches the same report: ${JSON.stringify(report)}`);
|
|
4631
4824
|
}
|
|
4632
4825
|
|
|
@@ -4763,11 +4956,13 @@ if (JSON.stringify(store.header) !== JSON.stringify(driver.checkpoint().header))
|
|
|
4763
4956
|
([Q374](doc/questions/q374.md)) — and so is a stored `attempted` event's number, which a re-run task is handed as
|
|
4764
4957
|
the attempts it SPENT (one naming a task the replay never dispatched, carrying no positive integer, or not RISING over
|
|
4765
4958
|
the last stored for its task, is refused `diverged` at its position). A store you do not trust is a store you should not resume from.
|
|
4766
|
-
- **`ResumeRefused` names its `reason`**: `format` (not a checkpoint this version writes
|
|
4959
|
+
- **`ResumeRefused` names its `reason`**: `format` (not a checkpoint this version writes — **every checkpoint 0.11.0
|
|
4960
|
+
or earlier wrote is format 1 and refused here**: `CHECKPOINT_FORMAT` is 2 since 0.12.0, section 5), `run` (the
|
|
4767
4961
|
header's run is not the driver's), `flow` (another flow id or `version` — there is no migration),
|
|
4768
4962
|
`halted`, `cancelled` and `diverged` — the last three with the `index` of the position. **A HALTED
|
|
4769
|
-
run is not resumable**: the refusal that halted it would halt it again — and a 0.7.0
|
|
4770
|
-
|
|
4963
|
+
run is not resumable**: the refusal that halted it would halt it again — and a halt stored under 0.7.0's kind,
|
|
4964
|
+
`stopped`, is refused `halted` at that element's position (ADR 0102; a 0.7.0 checkpoint itself is format 1, refused
|
|
4965
|
+
`format` first). **Nor is a CANCELLED one**
|
|
4771
4966
|
(section 6): a cancel is its owner's act, and a resume would hand the scheduler again what the owner
|
|
4772
4967
|
stopped — `index` is the first cancelled settlement or the cancel request, whichever is first. A completed or `stuck` run resumes to itself and
|
|
4773
4968
|
dispatches nothing. **A STOPPED run is the one terminal run a resume TAKES** (below): its `interrupted` event is replayed, and `resumed` ends it.
|
|
@@ -4993,7 +5188,7 @@ rest, while every file gets a thumbnail* — had nowhere to keep that memory.
|
|
|
4993
5188
|
|
|
4994
5189
|
| | |
|
|
4995
5190
|
|---|---|
|
|
4996
|
-
| **it names a gate TYPE** | `distinct` is built in: routes `first` (its DEFAULT), `duplicate` and `nokey`, keyed by `settings.key` — a DOTTED path of object keys over the value that reached the gate, no
|
|
5191
|
+
| **it names a gate TYPE** | `distinct` is built in: routes `first` (its DEFAULT), `duplicate` and `nokey`, keyed by `settings.key` — a DOTTED path of object keys over the value that reached the gate, each key optionally followed by bracket indexes into an ARRAY (`files[0].series`, `m[0][2]`; `[n]` with no leading zero, at most 999,999,999, never leading the path — [ADR 0137](doc/decisions/0137-a-key-path-indexes-an-array-by-a-bracket-suffix-and-a-plain-segment-keeps-its-meaning.md)); no quoted key, no wildcard, no expression. **`[1]` and `.1` are two paths**: `xs[1]` is element 1 of a real array and nothing over an object, `xs.1` an object's own key `"1"` and nothing over an array. A key a flow needs COMPUTED is computed by the task before the gate. `if` and `switch` are built in too, and remember nothing (the end of this section) |
|
|
4997
5192
|
| **every route is wired, exactly once** | an edge with no `route` carries the type's DEFAULT route, never every route; a DROP is an edge to an `end`, and each is refused at load if it is missing (`FD-V41`) |
|
|
4998
5193
|
| **it computes nothing** | the branch leaves on the chosen route with EXACTLY the value that reached the gate |
|
|
4999
5194
|
| **its memory is ONE per gate per RUN** | shared by every branch that reaches it, whatever map item or region instance it comes from — and it starts empty with the run and ends with it |
|
|
@@ -5084,12 +5279,13 @@ for (const d of firsts) {
|
|
|
5084
5279
|
}
|
|
5085
5280
|
|
|
5086
5281
|
// A DROP IS AN `end`, so it is a path end like any other — and the thumbnail ran for EVERY file: each file's
|
|
5087
|
-
// `perFile` record holds its thumbnail
|
|
5282
|
+
// `perFile` record holds its thumbnail under `beside`'s entry id `thumbnail`, and nothing from the gate road,
|
|
5283
|
+
// which ended at an `end` in every branch.
|
|
5088
5284
|
console.log(state.pathEnds.map((p) => p.at).sort());
|
|
5089
5285
|
// [ 'done', 'dropDuplicate', 'dropDuplicate', 'dropUnidentified', 'seriesReady', 'seriesReady' ]
|
|
5090
5286
|
const published = Object.values(state.nodes).find((n) => n.nodeId === 'publish')!.result as
|
|
5091
5287
|
{ joinPayload: Record<string, Record<string, { thumbnail?: string }>> };
|
|
5092
|
-
const thumbs = Object.
|
|
5288
|
+
const thumbs = Object.values(published.joinPayload).map((perFile) => perFile['thumbnail']?.thumbnail);
|
|
5093
5289
|
if (thumbs.length !== files.length || thumbs.some((t) => t === undefined)) throw new Error(`a thumbnail per file: ${thumbs.join()}`);
|
|
5094
5290
|
|
|
5095
5291
|
// A GATE TYPE OF YOUR OWN is CODE, registered by name the way `distinct` is written. A route its `fn` returns that
|
|
@@ -5298,7 +5494,7 @@ its three routes, so the third is the control sitting in the same report.
|
|
|
5298
5494
|
| the author writes | the check says |
|
|
5299
5495
|
|---|---|
|
|
5300
5496
|
| a key path the producer FORBIDS at any step | **`no`** on every arc leaving by a key-resolved route, naming the gate, the key, the STEP it stopped on, the word the schema itself uses, and the route |
|
|
5301
|
-
| a key path the producer describes | those arcs carry `required` along the path, so a consumer requiring it **`fits`** |
|
|
5497
|
+
| a key path the producer describes | those arcs carry `required` along the path, so a consumer requiring it **`fits`** — and through a bracket index `[n]`, `minItems: n + 1` on that array and nothing about any element past it, since `items` speaks for every element ([ADR 0137](doc/decisions/0137-a-key-path-indexes-an-array-by-a-bracket-suffix-and-a-plain-segment-keeps-its-meaning.md)) |
|
|
5302
5498
|
| the same arc, with the type declaring no guarantee | **`no`** again — *consumer requires "owner", producer does not guarantee it* |
|
|
5303
5499
|
| an arc leaving by a route the type does NOT call key-resolved | its own verdict, unchanged — the refusal is the ROUTE's and not the gate's |
|
|
5304
5500
|
|