@parcha/agentrun-dsl 0.1.0-beta.21 → 0.1.0-beta.23
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +59 -4
- package/dist/author.d.ts +4 -1
- package/dist/author.js +4 -2
- package/dist/graph.d.ts +74 -5
- package/dist/graph.js +142 -5
- package/dist/recovery/driver.d.ts +11 -3
- package/dist/recovery/driver.js +52 -21
- package/dist/workflow-shape.js +30 -5
- package/dist/workflow.js +8 -6
- package/package.json +1 -1
- package/skills/author/references/language.md +1 -1
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @parcha/agentrun-dsl
|
|
2
2
|
|
|
3
|
-
Typed workflows that your agent writes. It drafts a workflow, checks it, dry-runs it for free, then runs it
|
|
3
|
+
Typed workflows that your agent writes. It drafts a workflow, checks it, dry-runs it for free, then runs it; `@parcha/agentrun-dsl/graph` turns the workflow into its graph. The core provides the v2 workflow types, validator, interpreter, events, schema, and an offline CLI. Supply agent, judgment, and effect functions only when a workflow needs them.
|
|
4
4
|
|
|
5
5
|
Beta, on npm under the `beta` dist-tag (`npm view @parcha/agentrun-dsl dist-tags`). Requires Node 22.19+; TypeScript consumers require TypeScript 5.4+.
|
|
6
6
|
|
|
@@ -27,7 +27,7 @@ A workflow is a JSON document of typed nodes: each node that asks a model names
|
|
|
27
27
|
2. **Check.** `validateWorkflow(workflow)` checks the structure, the schemas, the state paths between nodes and the predicates, with no model call. A refused workflow lists its `issues`: each names the node or schema at fault with a JSON Pointer `path`, and the rule it breaks with a `code`, so the agent repairs that node and checks again. `inspectWorkflow` shows the tree and the tools and adapters it needs, without running any code.
|
|
28
28
|
3. **Dry-run, for free.** `dryRunWorkflow(workflow)` runs the whole workflow on synthetic values made from its schemas, with no model, Jev or tool call. Its `status` is `passed` when every check ran, `failed` with the `problems`, or `not_run` when a check could not run, with the reasons in `skipped`. A `not_run` dry run proves nothing.
|
|
29
29
|
4. **Run.** `runWorkflow(workflow, input, deps)` runs it with your agent, judgment and effect functions, and returns an output that matches the output schema, or an escalation. `deps` is the object of adapters your application supplies (`runNode`, `runJudge`, `runEffect`); [Integrate with your application](https://github.com/Parcha-ai/agentrun/blob/main/docs/host-integration.md) says what each one owes the engine.
|
|
30
|
-
5. **Watch every node.** `workflowGraph(workflow)` from `@parcha/agentrun-dsl/graph` lists every node and edge from the JSON alone, and `nodeStatus` marks each node `done`, `working`, `failed` or `not-run` from what a durable run kept. Pi's `/agentrun` view draws the same graph, with each node's state as the run goes.
|
|
30
|
+
5. **Watch every node.** `workflowGraph(workflow)` from `@parcha/agentrun-dsl/graph` lists every node and edge from the JSON alone, and `nodeStatus` marks each node `done`, `working`, `failed` or `not-run` from what a durable run kept, and `escalated` where the run's terminal result says it stopped. Pi's `/agentrun` view draws the same graph, with each node's state as the run goes.
|
|
31
31
|
|
|
32
32
|
The same checks from a shell, in the project where you installed the package, on one of the author skill's examples that ships with it (an agent extracts the change in each release note, then summarizes the migration):
|
|
33
33
|
|
|
@@ -102,10 +102,65 @@ It prints `"status": "complete"` and an output with `"total": 6`. The shipped ex
|
|
|
102
102
|
|
|
103
103
|
The `agentrun` CLI provides `demo`, `example`, `inspect`, `validate`, `dry-run`, and deterministic `run`. `inspect workflow.json [--json]` displays structure without executing code. Validation/run require `--trusted` because workflow JavaScript can execute, including during validation. This package is not an isolation boundary.
|
|
104
104
|
|
|
105
|
-
`runWorkflow(workflow, input, deps)` returns a complete schema-validated output or an escalation; its TypeScript output type is `unknown`. `defineWorkflow` binds Standard JSON Schema contracts, including supported Zod schemas, to a frozen v2 document; `runTypedWorkflow` infers those input/output types and delegates to the same interpreter. Intermediate state paths remain runtime-validated. Execution and verification failures throw. Hosts own tools,
|
|
105
|
+
`runWorkflow(workflow, input, deps)` returns a complete schema-validated output or an escalation; its TypeScript output type is `unknown`. `defineWorkflow` binds Standard JSON Schema contracts, including supported Zod schemas, to a frozen v2 document; `runTypedWorkflow` infers those input/output types and delegates to the same interpreter. Intermediate state paths remain runtime-validated. Execution and verification failures throw. Hosts own tools, file delivery, idempotency and activation, and choose where a run's journal lives; the package ships the recovery driver (see Durable runs below). A durable run's store keeps each effect's arguments as given, so redact secrets before they reach an effect.
|
|
106
106
|
|
|
107
107
|
Raw JSON schemas retain `unknown` boundary types. Zod transforms, refinements, defaults and coercion are rejected rather than silently dropped. The core does not depend on Zod at runtime. Other Standard JSON Schema providers must faithfully represent their contracts. Typed execution accepts the original `defineWorkflow` result; copied or deserialized documents use `runWorkflow`.
|
|
108
108
|
|
|
109
|
+
### A judge on any model
|
|
110
|
+
|
|
111
|
+
`runJudge` is a plain function: it receives the node's `questions` and returns `answers`, so any model can stand behind it. This one answers every question with a fixed probability, so it runs with no key; replace its body with a call to your model that returns a probability for each question. Save this as `judge.mjs` next to `node_modules` and run `node judge.mjs`:
|
|
112
|
+
|
|
113
|
+
```js
|
|
114
|
+
import { readFileSync } from 'node:fs';
|
|
115
|
+
import { runWorkflow } from '@parcha/agentrun-dsl';
|
|
116
|
+
|
|
117
|
+
const dir = 'node_modules/@parcha/agentrun-dsl/skills/author/examples/';
|
|
118
|
+
const workflow = JSON.parse(readFileSync(dir + 'evidence-gate.json', 'utf8'));
|
|
119
|
+
const input = JSON.parse(readFileSync(dir + 'evidence-gate.input.json', 'utf8'));
|
|
120
|
+
|
|
121
|
+
// A yes-probability for every question, as a noul answer.
|
|
122
|
+
const runJudge = async ({ questions }) => ({
|
|
123
|
+
answers: Object.fromEntries(Object.keys(questions).map(id => [id, { type: 'noul', noul: 0.95 }])),
|
|
124
|
+
});
|
|
125
|
+
|
|
126
|
+
const result = await runWorkflow(workflow, input, { runJudge });
|
|
127
|
+
console.log(result.status, result.output?.accepted);
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
It prints `complete true`: the workflow's own code gate accepts a yes-probability of 0.90 or more. Set `noul` to `0.5` and it prints `escalated undefined` instead. The Jev adapter, `@parcha/agentrun-jev`, is this same function backed by TypeSafe's judgment service.
|
|
131
|
+
|
|
132
|
+
### Durable runs
|
|
133
|
+
|
|
134
|
+
`@parcha/agentrun-dsl/recovery` journals a run so a process that dies can continue it. `openRecovery(store, workflow, { key })` opens the run's journal (`fileStore(directory)` keeps it in a directory, `memoryStore()` in memory) and `withRecovery(driver, deps)` wraps your adapters. A run opened again with the same key answers each committed step from the journal and runs only the rest; a tool call that was in flight when the process died has an unknown outcome and is refused, not repeated, unless your `reconcile` option settles it. The same journal opened for a completed run runs nothing. Save this as `durable.mjs` and run it three times: `CRASH=1 node durable.mjs` (the process exits before the last step commits), then `node durable.mjs` (it runs one model step, the one that did not commit), then `node durable.mjs` again (zero steps):
|
|
135
|
+
|
|
136
|
+
```js
|
|
137
|
+
import { readFileSync } from 'node:fs';
|
|
138
|
+
import { runWorkflow } from '@parcha/agentrun-dsl';
|
|
139
|
+
import { openRecovery, withRecovery, fileStore } from '@parcha/agentrun-dsl/recovery';
|
|
140
|
+
|
|
141
|
+
const dir = 'node_modules/@parcha/agentrun-dsl/skills/author/examples/';
|
|
142
|
+
const workflow = JSON.parse(readFileSync(dir + 'review-release-notes.json', 'utf8'));
|
|
143
|
+
const input = JSON.parse(readFileSync(dir + 'review-release-notes.input.json', 'utf8'));
|
|
144
|
+
|
|
145
|
+
let calls = 0;
|
|
146
|
+
const runNode = async ({ label, user }) => {
|
|
147
|
+
calls += 1;
|
|
148
|
+
if (process.env.CRASH && label === 'summarize-migration') process.exit(1); // dies before the last step commits
|
|
149
|
+
const state = JSON.parse(user);
|
|
150
|
+
return label === 'extract-change'
|
|
151
|
+
? { breakingChange: state.text, migrationAction: 'Follow the release note.' }
|
|
152
|
+
: { summary: `${state.changes.length} breaking changes.`, changes: state.changes };
|
|
153
|
+
};
|
|
154
|
+
|
|
155
|
+
const driver = await openRecovery(fileStore('./run-journal'), workflow, { key: 'release-review' });
|
|
156
|
+
try {
|
|
157
|
+
const result = await runWorkflow(workflow, input, withRecovery(driver, { runNode }));
|
|
158
|
+
console.log(result.status, `model steps run this time: ${calls}`);
|
|
159
|
+
} finally { await driver.close(); }
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
The first run exits with code 1 and leaves `run-journal/`; the second prints `complete model steps run this time: 1`; the third prints `complete model steps run this time: 0`. Delete `run-journal/` to start over. The key names the run, so use one per run; the journal is bound to the workflow and input it was opened with, and a change to either is refused. Hosts keep the store, the key and the decision to resume: see [Integrate with your application](https://github.com/Parcha-ai/agentrun/blob/main/docs/host-integration.md#recovery-and-versions).
|
|
163
|
+
|
|
109
164
|
## Public API
|
|
110
165
|
|
|
111
166
|
The package root exports a deliberate API. The beta retains agent, structured-output, report, artifact, effect, child-workflow and focused-evaluation paths. Hosts supply their adapters and policy; a workflow cannot grant itself tool permissions.
|
|
@@ -128,7 +183,7 @@ A `code` node runs in the host process through the package's own executor, which
|
|
|
128
183
|
|
|
129
184
|
The schema helpers resolve the owning workflow's schema catalog; `stageSchemaForNode` and `submissionSchemaForNode` name the same submission contract. `mergeStageDelta` stores declared submission fields under the node's output key; callers must validate the submission first. Internal traversal, compilation, CSV, reference-file and input-normalization helpers are not package-root APIs.
|
|
130
185
|
|
|
131
|
-
`@parcha/agentrun-dsl/graph` imports nothing, so a static viewer bundles it and a reader with no Node runs it. `workflowGraph(workflow)` returns `{ nodes, edges }` for a workflow document, computed from its JSON alone: a node is `{ path, kind, label, via? }` with `path` a JSON Pointer into the document as written (`/root/steps/1`, `/root/branches/<name>/body`, `/root/body` for a map's or a loop's body, `/root/workflow/root` for a child; the pointers `inspectWorkflow` and a validation issue use), and an edge is `{ from, to, kind }` with `kind` one of `next` (a chain's steps in order, a loop into its body), `branch` (a parallel node's branches), `route` (a route's branches), `item` (a map into its body) and `child` (a workflow node into its child's root). `nodeStatus(graph, source)` says, for each node, `done`, `working`, `failed` or `not-run`, read from what the recovery driver kept: its frame (the `pin` of the frozen snapshot), the map items' records (`agentrun.items`, keyed layout), or both as `{ pin, items }`. A run's map item or loop iteration has no node of its own: its instances count toward the body's node, and a node inside a map's or loop's body is done only once the map or loop is. To read one item's record on its own, pass `{ item: "<map path>/items/<index>" }` as a third argument: that item is then the instance, and its body is done when the item is. `graphNodeOf(graph, executionPath)` names the node an execution path (an event's or a record's) ran at: `{ path, at }`, where `path` is the node's pointer and `at` lists the map items (`{ kind: "item", index }`) and loop iterations (`{ kind: "iteration", index }`) it ran in, outermost first. An execution path is the node's pointer with `items/<i>` before a map's `body` and `iterations/<i>` before a loop's; the path is read by walking the graph by node kind, so a route branch named `items` stays a branch, and a path the graph does not have is `undefined`. The pi session's text view draws the same graph.
|
|
186
|
+
`@parcha/agentrun-dsl/graph` imports nothing, so a static viewer bundles it and a reader with no Node runs it. `workflowGraph(workflow)` returns `{ nodes, edges }` for a workflow document, computed from its JSON alone: a node is `{ path, kind, label, via? }` with `path` a JSON Pointer into the document as written (`/root/steps/1`, `/root/branches/<name>/body`, `/root/body` for a map's or a loop's body, `/root/workflow/root` for a child; the pointers `inspectWorkflow` and a validation issue use), and an edge is `{ from, to, kind }` with `kind` one of `next` (a chain's steps in order, a loop into its body), `branch` (a parallel node's branches), `route` (a route's branches), `item` (a map into its body) and `child` (a workflow node into its child's root). `nodeStatus(graph, source)` says, for each node, `done`, `working`, `failed` or `not-run`, read from what the recovery driver kept: its frame (the `pin` of the frozen snapshot), the map items' records (`agentrun.items`, keyed layout), or both as `{ pin, items }`. A run's map item or loop iteration has no node of its own: its instances count toward the body's node, and a node inside a map's or loop's body is done only once the map or loop is. Once the run has ended, pass its terminal result as `{ pin, items, result }`: an escalation (`result.status` `escalated`, with `result.escalation.executionPath` as `runWorkflow` returns it) marks the escalate node that fired and every container on its path `escalated`, since the frame holds no mark for that node, and a map or loop on that path has ended, so what finished inside it is done. Without a result the status is what it was. To read one item's record on its own, pass `{ item: "<map path>/items/<index>" }` as a third argument: that item is then the instance, and its body is done when the item is. `graphNodeOf(graph, executionPath)` names the node an execution path (an event's or a record's) ran at: `{ path, at }`, where `path` is the node's pointer and `at` lists the map items (`{ kind: "item", index }`) and loop iterations (`{ kind: "iteration", index }`) it ran in, outermost first. An execution path is the node's pointer with `items/<i>` before a map's `body` and `iterations/<i>` before a loop's; the path is read by walking the graph by node kind, so a route branch named `items` stays a branch, and a path the graph does not have is `undefined`. `graphDiff(prev, next)` says what changed between two graphs of a workflow, such as two saved revisions of one: `{ added, removed, changed, moved, edges: { added, removed } }`. A node is the same in both when its `(kind, label)` is unique in each graph, so a step inserted before others is one `added` node (its path in `next`) and the steps after it `moved` (`{ prev, next }` paths), never a run of changes; a node whose `(kind, label)` repeats, or is in one graph only, is matched by its path, and the same path with a new kind or label is `changed` (`{ prev, next, was, now }`, `was` and `now` each `{ kind, label }`). `removed` holds nodes of `prev` only, at their paths there; `rewritten` holds matched nodes whose kind and label stayed but whose own fields changed (new instructions, a new budget: `{ prev, next, kind, label }`); `edges.added` and `edges.removed` hold the edges of one graph only, each in its own graph's paths. Each graph node carries a `changeCheck` for this: 8 hex digits, the FNV-1a hash of the node's own fields as canonical JSON, its children left out as the walk leaves them out. It detects a change; it is not an identity, and not a saved revision's sha256. The pi session's text view draws the same graph.
|
|
132
187
|
|
|
133
188
|
The separate `@parcha/agentrun-dsl/demo` entrypoint provides the support-triage example. `@parcha/agentrun-dsl/schema` provides the generated workflow JSON Schema, a JSON file: import it with `import schema from '@parcha/agentrun-dsl/schema' with { type: 'json' };` (a plain `import` throws `ERR_IMPORT_ATTRIBUTE_MISSING`), or load it with `require('@parcha/agentrun-dsl/schema')` in CommonJS.
|
|
134
189
|
|
package/dist/author.d.ts
CHANGED
|
@@ -39,7 +39,8 @@ export declare function authorContract(options?: {
|
|
|
39
39
|
export interface CandidatePolicyOptions {
|
|
40
40
|
/** Authoritative rubric text by section name. Every generative node must name every section. */
|
|
41
41
|
rubricSections?: Record<string, string>;
|
|
42
|
-
/** Admits code, call and artifact nodes.
|
|
42
|
+
/** Admits code, call and artifact nodes. A candidate's code is syntax-checked before acceptance and run only after
|
|
43
|
+
* its host accepts it; in-process code is not a sandbox (see the README's `code` node note). */
|
|
43
44
|
allowExecutableCandidates?: boolean;
|
|
44
45
|
/** Direct tool calls admitted without executable authorization. */
|
|
45
46
|
allowedEffectTools?: readonly string[];
|
|
@@ -66,6 +67,8 @@ export interface AuthorWorkflowOptions {
|
|
|
66
67
|
/** Keys available to a candidate without an input schema; defaults to the host's initial state keys. */
|
|
67
68
|
inputKeys?: string[];
|
|
68
69
|
rubricSections?: Record<string, string>;
|
|
70
|
+
/** Admits code, call and artifact nodes. A candidate's code is syntax-checked before acceptance and run only after
|
|
71
|
+
* its host accepts it; in-process code is not a sandbox (see the README's `code` node note). */
|
|
69
72
|
allowExecutableCandidates?: boolean;
|
|
70
73
|
maxCandidates?: number;
|
|
71
74
|
/** Host acceptance over a structurally valid candidate, in the interpreter's view (`applyHostOutputTypes`):
|
package/dist/author.js
CHANGED
|
@@ -36,7 +36,7 @@ A workflow is {"v":2,"name","schemas","input"?,"output","root"}.
|
|
|
36
36
|
- The input contract: the run starts from the input object and nothing else. ${code("input.schemaId")}, when present, is enforced before the first node. A node may read only input keys and keys an earlier node wrote.
|
|
37
37
|
- ${code("output.schemaId")} validates the final value; ${code("output.path")} selects it from state and must name an input key or a key an earlier node wrote.
|
|
38
38
|
- State is one object. Each step writes its result under one key: its ${code("as")}, or, for an agent, decide, extract or code node without ${code("as")}, its label. A report writes ${code("report_markdown")}. A code node without ${code("as")} that returns an object merges that object's top-level fields instead.
|
|
39
|
-
- Keys that
|
|
39
|
+
- Keys that hold a ${code("$")} are engine-owned: ${code("$host")} (host state), ${code("<as>$answers")} (raw judge answers and confidences) and ${code("<as>$verify")} (verifier results). Read them; never write one. No ${code("as")} may hold a ${code("$")}, and neither may the label of an agent, decide, extract or code node without ${code("as")}.
|
|
40
40
|
- ${code("{dot.path}")} placeholders are replaced from state in ${code("state")} maps (including ${code("verify.state")} and ${code("ask.state")}), call ${code("args")}, ${code("input")}, ${code("env")} and ${code("produces")}, a child workflow's ${code("input")}, and ${code("escalate.summary")}; ${code("pick")} and ${code("sift")} ${code("describe")} templates also see ${code("{item...}")}. A placeholder that is the whole string keeps its value's type. Every key a state map, call or child input interpolates must be produced upstream. ${code("instructions")}, ${code("code")} and ${code("command")} are literal and never interpolated.
|
|
41
41
|
|
|
42
42
|
## Node kinds
|
|
@@ -271,7 +271,9 @@ export async function authorWorkflow(options) {
|
|
|
271
271
|
const errors = candidatePolicyErrors(value, options);
|
|
272
272
|
if (!errors.length) {
|
|
273
273
|
try {
|
|
274
|
-
|
|
274
|
+
// executeCode: false — a candidate's code is syntax-checked here, never run, so a candidate runs nothing
|
|
275
|
+
// until its host accepts it. Runs and dry runs after acceptance still probe the code.
|
|
276
|
+
const verdict = validateWorkflow(applyHostOutputTypes(value, options.host), { inputKeys, executeCode: false });
|
|
275
277
|
if (!verdict.ok)
|
|
276
278
|
errors.push(...verdict.errors);
|
|
277
279
|
}
|
package/dist/graph.d.ts
CHANGED
|
@@ -1,12 +1,19 @@
|
|
|
1
1
|
/** One node of the workflow as written. `path` is a JSON Pointer (RFC 6901) into the document: `/root` is the root,
|
|
2
2
|
* `/root/steps/1` a chain's second step, `/root/branches/<name>/body` a route's branch, `/root/body` a map's or a
|
|
3
3
|
* loop's body, `/root/workflow/root` a child workflow's root: the pointers `inspectWorkflow` and a validation issue
|
|
4
|
-
* use for the same node. `label` is the node's label, or its kind when it has none. `via` is a `call` node's transport.
|
|
4
|
+
* use for the same node. `label` is the node's label, or its kind when it has none. `via` is a `call` node's transport.
|
|
5
|
+
* `changeCheck` detects a change to the node's own fields from one graph to another: 8 hex digits, the FNV-1a hash of
|
|
6
|
+
* the canonical JSON (keys sorted) of the node with its children left out as the walk leaves them out (a chain's steps,
|
|
7
|
+
* a parallel's branches, a map's or a loop's body, each route branch's body, a child workflow's root). A route's branch
|
|
8
|
+
* names are its own fields, since they decide which branch runs, so a route that gains or loses a branch is rewritten. It
|
|
9
|
+
* is not an identity and not a saved revision's sha256: two nodes with the same check are not the same node, and a check
|
|
10
|
+
* that stayed only means no change was seen. A graph built by hand may leave it out. */
|
|
5
11
|
export type GraphNode = {
|
|
6
12
|
path: string;
|
|
7
13
|
kind: string;
|
|
8
14
|
label: string;
|
|
9
15
|
via?: string;
|
|
16
|
+
changeCheck?: string;
|
|
10
17
|
};
|
|
11
18
|
/** How control reaches `to` from `from`.
|
|
12
19
|
* - `next`: a chain enters its first step and each step is followed by the next; a loop enters its body, which runs again
|
|
@@ -33,8 +40,9 @@ export declare function workflowGraph(workflow: unknown): WorkflowGraph;
|
|
|
33
40
|
* only those that started, so it is `done` only once the map or loop is: until then it is `working`, and a done node never
|
|
34
41
|
* goes back. A container (chain, parallel, map, loop, route, workflow) is `working` while some of
|
|
35
42
|
* its descendants are, or some are done and it is not; a node a run never reached, a branch a route did not take among
|
|
36
|
-
* them, is `not-run`.
|
|
37
|
-
|
|
43
|
+
* them, is `not-run`. `escalated` is where the run ended on an escalation: the escalate node that fired and each container
|
|
44
|
+
* on its path. It is read from the run's terminal result, never from the frame, which holds no mark for that node. */
|
|
45
|
+
export type NodeState = "done" | "working" | "failed" | "not-run" | "escalated";
|
|
38
46
|
type Frame = {
|
|
39
47
|
done?: unknown;
|
|
40
48
|
steps?: unknown;
|
|
@@ -48,11 +56,21 @@ export type ItemRecordLike = {
|
|
|
48
56
|
status?: unknown;
|
|
49
57
|
frame?: Frame;
|
|
50
58
|
};
|
|
59
|
+
/** The part of a run's terminal result the status reads: its `status` and, for an escalation, the execution path of the
|
|
60
|
+
* node it stopped at (`escalation.executionPath`, as `runWorkflow` returns it). */
|
|
61
|
+
export type TerminalResultLike = {
|
|
62
|
+
status?: unknown;
|
|
63
|
+
escalation?: {
|
|
64
|
+
executionPath?: unknown;
|
|
65
|
+
};
|
|
66
|
+
};
|
|
51
67
|
/** What `nodeStatus` reads: a driver frame (the `pin` of a frozen snapshot: `done`, `steps`, `inputs`), a frozen snapshot
|
|
52
|
-
* (its `pin`), or the map items' records, alone or together with the driver's record (`{ pin, items }`)
|
|
68
|
+
* (its `pin`), or the map items' records, alone or together with the driver's record (`{ pin, items }`), and with it the
|
|
69
|
+
* run's terminal `result` once the run has ended. */
|
|
53
70
|
export type StatusSource = Frame | {
|
|
54
71
|
pin: Frame;
|
|
55
72
|
items?: readonly ItemRecordLike[];
|
|
73
|
+
result?: TerminalResultLike | null;
|
|
56
74
|
} | readonly ItemRecordLike[];
|
|
57
75
|
/** `item`: the source is one map item's (or loop iteration's) record, read on its own, and this is its execution path
|
|
58
76
|
* (`<map path>/items/<index>`, `<loop path>/iterations/<index>`). That item is then the instance, not one of many: its body is done
|
|
@@ -62,7 +80,9 @@ export type StatusOptions = {
|
|
|
62
80
|
};
|
|
63
81
|
/** For every node of `graph`, the state the record shows. `source` is what the driver kept: its frame and, in the keyed
|
|
64
82
|
* layout, the records of the map items (a run's whole story is the frame and the items together). A source that is not
|
|
65
|
-
* one of those shows no node run.
|
|
83
|
+
* one of those shows no node run. With the run's terminal `result`, an escalation that names a node of the graph marks
|
|
84
|
+
* that node and every container on its path `escalated`, and the maps and loops on that path have ended, so what finished
|
|
85
|
+
* inside them is done; without one, or when it names no node of the graph, nothing else changes. */
|
|
66
86
|
export declare function nodeStatus(graph: WorkflowGraph, source: StatusSource, options?: StatusOptions): Record<string, NodeState>;
|
|
67
87
|
/** Where an execution path is in the graph: the node it ran at (`path`, a node of the graph) and the map items and loop
|
|
68
88
|
* iterations it ran in, outermost first. */
|
|
@@ -80,4 +100,53 @@ export type GraphNodeAt = {
|
|
|
80
100
|
* a path the graph does not have, or one that is not an execution path (a map's `body` without an item, an item without
|
|
81
101
|
* its `body`). */
|
|
82
102
|
export declare function graphNodeOf(graph: WorkflowGraph, executionPath: string): GraphNodeAt | undefined;
|
|
103
|
+
/** A node as `graphDiff` reports it: its path in the graph it is in, its kind and its label. */
|
|
104
|
+
export type GraphDiffNode = {
|
|
105
|
+
path: string;
|
|
106
|
+
kind: string;
|
|
107
|
+
label: string;
|
|
108
|
+
};
|
|
109
|
+
/** What changed from one graph to the next. `added` holds nodes of `next` only (paths in `next`), `removed` nodes of `prev`
|
|
110
|
+
* only (paths in `prev`); `changed` the same node at the same path with a new kind or label; `moved` the same node at a
|
|
111
|
+
* new path; `rewritten` the same node, its kind and label as they were, with other own fields changed (its `changeCheck`
|
|
112
|
+
* differs: new instructions, a new budget), at its path in each graph; `edges.added` edges of `next` only and
|
|
113
|
+
* `edges.removed` edges of `prev` only, each in its own graph's paths. */
|
|
114
|
+
export type GraphDiff = {
|
|
115
|
+
added: GraphDiffNode[];
|
|
116
|
+
removed: GraphDiffNode[];
|
|
117
|
+
changed: {
|
|
118
|
+
prev: string;
|
|
119
|
+
next: string;
|
|
120
|
+
was: {
|
|
121
|
+
kind: string;
|
|
122
|
+
label: string;
|
|
123
|
+
};
|
|
124
|
+
now: {
|
|
125
|
+
kind: string;
|
|
126
|
+
label: string;
|
|
127
|
+
};
|
|
128
|
+
}[];
|
|
129
|
+
moved: {
|
|
130
|
+
prev: string;
|
|
131
|
+
next: string;
|
|
132
|
+
}[];
|
|
133
|
+
rewritten: {
|
|
134
|
+
prev: string;
|
|
135
|
+
next: string;
|
|
136
|
+
kind: string;
|
|
137
|
+
label: string;
|
|
138
|
+
}[];
|
|
139
|
+
edges: {
|
|
140
|
+
added: GraphEdge[];
|
|
141
|
+
removed: GraphEdge[];
|
|
142
|
+
};
|
|
143
|
+
};
|
|
144
|
+
/** What changed between two graphs of a workflow, such as two saved revisions of one. A node is the same in both when its
|
|
145
|
+
* `(kind, label)` is unique in each graph, wherever it is: a step inserted before others is one `added` node and the rest
|
|
146
|
+
* `moved`, never a run of changes. A node whose `(kind, label)` repeats in either graph, or is in one only, is the same
|
|
147
|
+
* as the other graph's node at its path, unless that node is already matched by its identity; with a new kind or label
|
|
148
|
+
* it is `changed`. A matched node with its kind and label as they were and another `changeCheck` is `rewritten`; a node
|
|
149
|
+
* without a check in either graph is never reported rewritten. An edge is the same when both its ends are the same nodes
|
|
150
|
+
* and its kind is the same. Every list is in its graph's node or edge order. */
|
|
151
|
+
export declare function graphDiff(prev: WorkflowGraph, next: WorkflowGraph): GraphDiff;
|
|
83
152
|
export {};
|
package/dist/graph.js
CHANGED
|
@@ -2,12 +2,56 @@
|
|
|
2
2
|
// (the static viewer inlines it) and the pi session's text view draw the same graph; nothing here validates, runs or
|
|
3
3
|
// reads a store. `workflowGraph` reads the workflow JSON as written; `nodeStatus` reads the driver's record of a run
|
|
4
4
|
// (the `agentrun.driver` frame of the frozen snapshot, and the `agentrun.items` of a keyed-layout run) and says, for
|
|
5
|
-
// each node of that graph, whether it is done, working, failed or not run
|
|
6
|
-
//
|
|
5
|
+
// each node of that graph, whether it is done, working, failed or not run, or, given the run's terminal result, where an
|
|
6
|
+
// escalation stopped it; `graphNodeOf` names the node an execution path ran
|
|
7
|
+
// at, and the map items and loop iterations around it; `graphDiff` says what changed between two graphs of a workflow.
|
|
7
8
|
const isObject = (value) => typeof value === "object" && value !== null && !Array.isArray(value);
|
|
8
9
|
const pointer = (part) => part.replace(/~/g, "~0").replace(/\//g, "~1");
|
|
9
10
|
/** The nesting depth `validateWorkflow` and `inspectWorkflow` accept (128 JSON levels): a node nests at least one level deeper than its parent, so every node of such a document is within it. */
|
|
10
11
|
const MAX_DEPTH = 128;
|
|
12
|
+
/** A value as canonical JSON: object keys sorted, undefined members left out, other values as JSON writes them. Deeper
|
|
13
|
+
* than `MAX_DEPTH` a value is written as one marker, so a value that is not JSON (a cycle) still ends. A leaf JSON
|
|
14
|
+
* cannot encode (a bigint, a symbol) is written as a marker of its type, so changeCheck is total: `workflowGraph` never
|
|
15
|
+
* throws on a node the validator accepts, and such a leaf still changes the check when it changes value. */
|
|
16
|
+
const leaf = (value) => {
|
|
17
|
+
try {
|
|
18
|
+
return JSON.stringify(value) ?? "null";
|
|
19
|
+
}
|
|
20
|
+
catch {
|
|
21
|
+
return JSON.stringify(`[${typeof value}:${String(value)}]`);
|
|
22
|
+
}
|
|
23
|
+
};
|
|
24
|
+
const canonicalJson = (value, depth = 0) => {
|
|
25
|
+
if (depth > MAX_DEPTH)
|
|
26
|
+
return '"…"';
|
|
27
|
+
if (Array.isArray(value))
|
|
28
|
+
return `[${value.map((item) => canonicalJson(item, depth + 1)).join(",")}]`;
|
|
29
|
+
if (isObject(value))
|
|
30
|
+
return `{${Object.keys(value).filter((key) => value[key] !== undefined).sort().map((key) => `${JSON.stringify(key)}:${canonicalJson(value[key], depth + 1)}`).join(",")}}`;
|
|
31
|
+
return leaf(value);
|
|
32
|
+
};
|
|
33
|
+
/** FNV-1a (32 bits) over the UTF-16 code units of `text`, as 8 hex digits. */
|
|
34
|
+
const fnv1a = (text) => {
|
|
35
|
+
let hash = 0x811c9dc5;
|
|
36
|
+
for (let i = 0; i < text.length; i += 1)
|
|
37
|
+
hash = Math.imul(hash ^ text.charCodeAt(i), 0x01000193);
|
|
38
|
+
return (hash >>> 0).toString(16).padStart(8, "0");
|
|
39
|
+
};
|
|
40
|
+
const without = (value, key) => Object.fromEntries(Object.entries(value).filter(([name]) => name !== key));
|
|
41
|
+
/** A node's own fields: the node with the children the walk visits left out. */
|
|
42
|
+
const ownFields = (node) => {
|
|
43
|
+
switch (node.node) {
|
|
44
|
+
case "chain": return without(node, "steps");
|
|
45
|
+
case "parallel": return without(node, "branches");
|
|
46
|
+
case "map":
|
|
47
|
+
case "loop": return without(node, "body");
|
|
48
|
+
case "route": return isObject(node.branches)
|
|
49
|
+
? { ...node, branches: Object.fromEntries(Object.entries(node.branches).map(([name, branch]) => [name, isObject(branch) ? without(branch, "body") : branch])) }
|
|
50
|
+
: node;
|
|
51
|
+
case "workflow": return isObject(node.workflow) ? { ...node, workflow: without(node.workflow, "root") } : node;
|
|
52
|
+
default: return node;
|
|
53
|
+
}
|
|
54
|
+
};
|
|
11
55
|
/** The graph of `workflow`, computed from its JSON alone: nodes in document order (a node, then its descendants), and the
|
|
12
56
|
* edges between them. A value that is not a workflow, or a node the walk does not know, adds nothing and never throws:
|
|
13
57
|
* validation is `validateWorkflow`'s. */
|
|
@@ -18,7 +62,7 @@ export function workflowGraph(workflow) {
|
|
|
18
62
|
if (!isObject(node) || typeof node.node !== "string" || depth > MAX_DEPTH)
|
|
19
63
|
return;
|
|
20
64
|
nodes.push({ path, kind: node.node, label: typeof node.label === "string" ? node.label : node.node,
|
|
21
|
-
...(node.node === "call" && typeof node.via === "string" ? { via: node.via } : {}) });
|
|
65
|
+
...(node.node === "call" && typeof node.via === "string" ? { via: node.via } : {}), changeCheck: fnv1a(canonicalJson(ownFields(node))) });
|
|
22
66
|
const child = (target, at, kind, from = path) => {
|
|
23
67
|
if (!isObject(target) || typeof target.node !== "string" || depth + 1 > MAX_DEPTH)
|
|
24
68
|
return;
|
|
@@ -61,7 +105,9 @@ const definitionPath = (path) => path.replace(/\/items\/\d+\/body/g, "/body").re
|
|
|
61
105
|
const RANK = { working: 1, done: 2, failed: 3 };
|
|
62
106
|
/** For every node of `graph`, the state the record shows. `source` is what the driver kept: its frame and, in the keyed
|
|
63
107
|
* layout, the records of the map items (a run's whole story is the frame and the items together). A source that is not
|
|
64
|
-
* one of those shows no node run.
|
|
108
|
+
* one of those shows no node run. With the run's terminal `result`, an escalation that names a node of the graph marks
|
|
109
|
+
* that node and every container on its path `escalated`, and the maps and loops on that path have ended, so what finished
|
|
110
|
+
* inside them is done; without one, or when it names no node of the graph, nothing else changes. */
|
|
65
111
|
export function nodeStatus(graph, source, options = {}) {
|
|
66
112
|
const instances = new Map();
|
|
67
113
|
const note = (path, state) => {
|
|
@@ -108,6 +154,19 @@ export function nodeStatus(graph, source, options = {}) {
|
|
|
108
154
|
}
|
|
109
155
|
else
|
|
110
156
|
readFrame(raw);
|
|
157
|
+
// The node a run's escalation ended at, from its terminal result: by its execution path, as the node it ran as.
|
|
158
|
+
const result = isObject(raw) && isObject(raw.pin) && isObject(raw.result) ? raw.result : undefined;
|
|
159
|
+
const escalation = result?.status === "escalated" && isObject(result.escalation) ? result.escalation : undefined;
|
|
160
|
+
const escalatedExecPath = typeof escalation?.executionPath === "string" ? escalation.executionPath : undefined;
|
|
161
|
+
// When one map item or loop iteration is read on its own (`options.item`), the escalation belongs to this view only if
|
|
162
|
+
// it fired inside that same instance: another item's escalation, reduced to the shared body, must not mark this one's
|
|
163
|
+
// body escalated. The item index is in the raw execution path, which `definitionPath` then strips.
|
|
164
|
+
const itemScope = typeof options.item === "string" ? options.item.replace(/\/$/, "") : undefined;
|
|
165
|
+
const inThisItem = escalatedExecPath !== undefined
|
|
166
|
+
&& (itemScope === undefined || escalatedExecPath === itemScope || escalatedExecPath.startsWith(`${itemScope}/`));
|
|
167
|
+
const stoppedAt = escalatedExecPath !== undefined && inThisItem ? definitionPath(escalatedExecPath) : undefined;
|
|
168
|
+
const ended = stoppedAt !== undefined && graph.nodes.some((node) => node.path === stoppedAt) ? stoppedAt : undefined;
|
|
169
|
+
const onPath = (path) => ended !== undefined && (path === ended || ended.startsWith(`${path}/`));
|
|
111
170
|
// A done mark outranks a working one at the same path (a step's record stays after its node commits); a failure outranks both.
|
|
112
171
|
const byNode = new Map();
|
|
113
172
|
for (const [path, state] of instances) {
|
|
@@ -147,7 +206,8 @@ export function nodeStatus(graph, source, options = {}) {
|
|
|
147
206
|
// The map or loop an `item` belongs to is the exception, and so is every one that encloses it: that item is the instance, and
|
|
148
207
|
// the source holds nothing of the items around it. A map or loop inside the item still stands for its own items.
|
|
149
208
|
const owner = typeof options.item === "string" ? definitionPath(options.item.replace(/\/(?:items|iterations)\/\d+$/, "")) : undefined;
|
|
150
|
-
|
|
209
|
+
// A map or loop on an escalation's path has ended: no other item or iteration will run, so what finished inside it is done.
|
|
210
|
+
const containers = graph.nodes.filter((node) => (node.kind === "map" || node.kind === "loop") && !onPath(node.path)
|
|
151
211
|
&& !(owner !== undefined && (node.path === owner || owner.startsWith(`${node.path}/body`))));
|
|
152
212
|
for (const node of graph.nodes) {
|
|
153
213
|
if (out[node.path] !== "done")
|
|
@@ -155,6 +215,11 @@ export function nodeStatus(graph, source, options = {}) {
|
|
|
155
215
|
if (containers.some((container) => node.path.startsWith(`${container.path}/body`) && out[container.path] !== "done"))
|
|
156
216
|
out[node.path] = "working";
|
|
157
217
|
}
|
|
218
|
+
// The run ended at the escalate node that fired, and in each container on its path; a failure recorded below a container
|
|
219
|
+
// still shows.
|
|
220
|
+
for (const node of graph.nodes)
|
|
221
|
+
if (onPath(node.path) && (node.path === ended || out[node.path] !== "failed"))
|
|
222
|
+
out[node.path] = "escalated";
|
|
158
223
|
return out;
|
|
159
224
|
}
|
|
160
225
|
/** An item or iteration index as an execution path writes it: a whole number with no sign and no leading zero. */
|
|
@@ -229,3 +294,75 @@ export function graphNodeOf(graph, executionPath) {
|
|
|
229
294
|
}
|
|
230
295
|
return { path, at };
|
|
231
296
|
}
|
|
297
|
+
/** What changed between two graphs of a workflow, such as two saved revisions of one. A node is the same in both when its
|
|
298
|
+
* `(kind, label)` is unique in each graph, wherever it is: a step inserted before others is one `added` node and the rest
|
|
299
|
+
* `moved`, never a run of changes. A node whose `(kind, label)` repeats in either graph, or is in one only, is the same
|
|
300
|
+
* as the other graph's node at its path, unless that node is already matched by its identity; with a new kind or label
|
|
301
|
+
* it is `changed`. A matched node with its kind and label as they were and another `changeCheck` is `rewritten`; a node
|
|
302
|
+
* without a check in either graph is never reported rewritten. An edge is the same when both its ends are the same nodes
|
|
303
|
+
* and its kind is the same. Every list is in its graph's node or edge order. */
|
|
304
|
+
export function graphDiff(prev, next) {
|
|
305
|
+
const identity = (node) => JSON.stringify([node.kind, node.label]);
|
|
306
|
+
const counts = (graph) => {
|
|
307
|
+
const count = new Map();
|
|
308
|
+
for (const node of graph.nodes)
|
|
309
|
+
count.set(identity(node), (count.get(identity(node)) ?? 0) + 1);
|
|
310
|
+
return count;
|
|
311
|
+
};
|
|
312
|
+
const [inPrev, inNext] = [counts(prev), counts(next)];
|
|
313
|
+
const unique = (node) => inPrev.get(identity(node)) === 1 && inNext.get(identity(node)) === 1;
|
|
314
|
+
// The matched nodes, each way: a path of `next` to its node's path in `prev`, and back.
|
|
315
|
+
const prevOf = new Map();
|
|
316
|
+
const nextOf = new Map();
|
|
317
|
+
const match = (from, to) => { prevOf.set(to, from); nextOf.set(from, to); };
|
|
318
|
+
const uniqueInPrev = new Map(prev.nodes.filter(unique).map((node) => [identity(node), node.path]));
|
|
319
|
+
for (const node of next.nodes) {
|
|
320
|
+
const at = unique(node) ? uniqueInPrev.get(identity(node)) : undefined;
|
|
321
|
+
if (at !== undefined)
|
|
322
|
+
match(at, node.path);
|
|
323
|
+
}
|
|
324
|
+
const prevAt = new Map(prev.nodes.map((node) => [node.path, node]));
|
|
325
|
+
const changed = [];
|
|
326
|
+
for (const node of next.nodes) {
|
|
327
|
+
const old = prevAt.get(node.path);
|
|
328
|
+
if (prevOf.has(node.path) || old === undefined || nextOf.has(old.path))
|
|
329
|
+
continue;
|
|
330
|
+
match(old.path, node.path);
|
|
331
|
+
if (old.kind !== node.kind || old.label !== node.label) {
|
|
332
|
+
changed.push({ prev: old.path, next: node.path, was: { kind: old.kind, label: old.label }, now: { kind: node.kind, label: node.label } });
|
|
333
|
+
}
|
|
334
|
+
}
|
|
335
|
+
const rewritten = [];
|
|
336
|
+
for (const node of next.nodes) {
|
|
337
|
+
const from = prevOf.get(node.path);
|
|
338
|
+
const old = from === undefined ? undefined : prevAt.get(from);
|
|
339
|
+
if (old === undefined || old.kind !== node.kind || old.label !== node.label)
|
|
340
|
+
continue;
|
|
341
|
+
if (old.changeCheck !== undefined && node.changeCheck !== undefined && old.changeCheck !== node.changeCheck) {
|
|
342
|
+
rewritten.push({ prev: old.path, next: node.path, kind: node.kind, label: node.label });
|
|
343
|
+
}
|
|
344
|
+
}
|
|
345
|
+
const reported = ({ path, kind, label }) => ({ path, kind, label });
|
|
346
|
+
const edgeKey = (from, to, kind) => JSON.stringify([from, to, kind]);
|
|
347
|
+
const prevEdges = new Set(prev.edges.map((edge) => edgeKey(edge.from, edge.to, edge.kind)));
|
|
348
|
+
const nextEdges = new Set(next.edges.map((edge) => edgeKey(edge.from, edge.to, edge.kind)));
|
|
349
|
+
// Whether the other graph has this edge: both its ends matched there, joined by an edge of the same kind.
|
|
350
|
+
const theirs = (edge, map, edges) => {
|
|
351
|
+
const [from, to] = [map.get(edge.from), map.get(edge.to)];
|
|
352
|
+
return from !== undefined && to !== undefined && edges.has(edgeKey(from, to, edge.kind));
|
|
353
|
+
};
|
|
354
|
+
return {
|
|
355
|
+
added: next.nodes.filter((node) => !prevOf.has(node.path)).map(reported),
|
|
356
|
+
removed: prev.nodes.filter((node) => !nextOf.has(node.path)).map(reported),
|
|
357
|
+
changed,
|
|
358
|
+
moved: next.nodes.flatMap((node) => {
|
|
359
|
+
const from = prevOf.get(node.path);
|
|
360
|
+
return from !== undefined && from !== node.path ? [{ prev: from, next: node.path }] : [];
|
|
361
|
+
}),
|
|
362
|
+
rewritten,
|
|
363
|
+
edges: {
|
|
364
|
+
added: next.edges.filter((edge) => !theirs(edge, prevOf, prevEdges)).map(({ from, to, kind }) => ({ from, to, kind })),
|
|
365
|
+
removed: prev.edges.filter((edge) => !theirs(edge, nextOf, nextEdges)).map(({ from, to, kind }) => ({ from, to, kind })),
|
|
366
|
+
},
|
|
367
|
+
};
|
|
368
|
+
}
|
|
@@ -84,7 +84,12 @@ export type RecoveryOptions = {
|
|
|
84
84
|
/** What a host's effect adapter receives under the driver: the interpreter's parameters, `call`, which keeps each
|
|
85
85
|
* outside call the effect makes on its record as the call ends, and `effect`, the admitted effect: its journal id and
|
|
86
86
|
* the hash of its arguments, as the journal stores them and the same in every process of the run. A host that keeps
|
|
87
|
-
* what an outside call answered under them can settle an effect whose call was cut with `journal.complete`.
|
|
87
|
+
* what an outside call answered under them can settle an effect whose call was cut with `journal.complete`.
|
|
88
|
+
*
|
|
89
|
+
* What the adapter returns is the effect's result, whatever its fields. An adapter that cannot know whether the outside
|
|
90
|
+
* call happened throws: a `RecoveryError` with code `FROZEN_EFFECT_UNKNOWN`, its `cause` what it observed (kept for the
|
|
91
|
+
* operator in the checkpoint's `unknownResponses`), or an `EffectOutcomeUnknownError`. The effect then stays unknown,
|
|
92
|
+
* is never dispatched again, and the run resumes only once the host reconciles it, as after any call that throws. */
|
|
88
93
|
export type RecoveryEffectParams = Parameters<NonNullable<WorkflowDeps["runEffect"]>>[0] & {
|
|
89
94
|
call?: (record: unknown) => void;
|
|
90
95
|
effect?: {
|
|
@@ -201,8 +206,11 @@ export declare function openRecovery(store: RecoveryStore, workflow: Workflow, o
|
|
|
201
206
|
lastState(): Record<string, unknown>;
|
|
202
207
|
readonly started: boolean;
|
|
203
208
|
wrapEffect(run: (params: RecoveryEffectParams) => Promise<unknown>): NonNullable<WorkflowDeps["runEffect"]>;
|
|
204
|
-
/** Every commit the driver started has landed; a failed one is thrown. */
|
|
209
|
+
/** Every commit the driver started has landed; a failed one is thrown, a pause's or a cancel's own commit included. */
|
|
205
210
|
flush: () => Promise<void>;
|
|
206
|
-
/** Let go of the run once every commit the driver started has landed.
|
|
211
|
+
/** Let go of the run once every commit the driver started has landed. A commit the store refused after a pause or
|
|
212
|
+
* cancel is thrown once the journal is released: the stop is not durable, and the host must not take the run as
|
|
213
|
+
* stopped. It is thrown even when a later stop's commit lands, since a later commit does not carry every record the
|
|
214
|
+
* refused one held. A run's own failed commit is the run's failure and is not thrown again here. */
|
|
207
215
|
close(): Promise<void>;
|
|
208
216
|
}>;
|
package/dist/recovery/driver.js
CHANGED
|
@@ -5,6 +5,7 @@ import fs from "node:fs";
|
|
|
5
5
|
import path from "node:path";
|
|
6
6
|
import { createHash } from "node:crypto";
|
|
7
7
|
import { desugarWorkflow, isItemFailure } from "../workflow.js";
|
|
8
|
+
import { setLongTimeout } from "../long-timer.js";
|
|
8
9
|
import { chainStep, childrenOf, completedIteration, enclosingChains, enclosingIterations, isWithin, mapItemOf, nodeAt, topLevelCompletion } from "./execution-path.js";
|
|
9
10
|
import { bindingDigests, LEDGER_FORMAT } from "./binding.js";
|
|
10
11
|
import { canonicalHash as hash } from "./canonical-hash.js";
|
|
@@ -168,8 +169,11 @@ export async function openRecovery(store, workflow, options) {
|
|
|
168
169
|
const driver = options.key;
|
|
169
170
|
const controller = new AbortController();
|
|
170
171
|
let fatal;
|
|
171
|
-
/** A pause or cancel was applied: its commit still
|
|
172
|
+
/** A pause or cancel was applied: its commit is still made, and no admission after it. */
|
|
172
173
|
let stopping = false;
|
|
174
|
+
/** A commit the store refused after a pause or cancel was applied (the stop's own, say): the stop is not durable, the
|
|
175
|
+
* journal still reads as it did, and the host hears it from `flush` and `close`. */
|
|
176
|
+
let refusedAfterStop;
|
|
173
177
|
const journal = await store.open(bound);
|
|
174
178
|
/** Whether the store keeps parts beside the state: the keyed layout needs both reads. */
|
|
175
179
|
const keyed = typeof journal.items === "function" && typeof journal.stateOf === "function";
|
|
@@ -539,14 +543,19 @@ export async function openRecovery(store, workflow, options) {
|
|
|
539
543
|
const persist = (write) => {
|
|
540
544
|
const next = chain.then(() => { if (fatal && !stopping)
|
|
541
545
|
throw fatal; return write(); });
|
|
542
|
-
chain = next.catch((error) => {
|
|
543
|
-
fatal
|
|
544
|
-
|
|
545
|
-
|
|
546
|
+
chain = next.catch((error) => {
|
|
547
|
+
if (!fatal) {
|
|
548
|
+
fatal = error;
|
|
549
|
+
controller.abort(error);
|
|
550
|
+
}
|
|
551
|
+
else if (stopping && refusedAfterStop === undefined)
|
|
552
|
+
refusedAfterStop = error;
|
|
553
|
+
});
|
|
546
554
|
return next;
|
|
547
555
|
};
|
|
548
|
-
/** Every commit the driver started has landed; a failed one is thrown. */
|
|
549
|
-
const flush = async () => { await chain; if (
|
|
556
|
+
/** Every commit the driver started has landed; a failed one is thrown, a pause's or a cancel's own commit included. */
|
|
557
|
+
const flush = async () => { await chain; if (refusedAfterStop !== undefined)
|
|
558
|
+
throw refusedAfterStop; if (fatal && !stopping)
|
|
550
559
|
throw fatal; };
|
|
551
560
|
try {
|
|
552
561
|
// A driver an earlier process bound, with no checkpoint, is a run killed between its binding and its first
|
|
@@ -897,8 +906,9 @@ export async function openRecovery(store, workflow, options) {
|
|
|
897
906
|
// The wait's clock is durable before the wait starts, so a process that dies inside it resumes the same wait.
|
|
898
907
|
await flush();
|
|
899
908
|
await new Promise((resolve, reject) => {
|
|
900
|
-
const onAbort = () => {
|
|
901
|
-
|
|
909
|
+
const onAbort = () => { wait.clear(); reject(controller.signal.reason); };
|
|
910
|
+
// setLongTimeout: a wait past Node's 2^31-1 ms timer limit waits instead of firing at once.
|
|
911
|
+
const wait = setLongTimeout(() => { controller.signal.removeEventListener("abort", onAbort); resolve(); }, until - Date.now());
|
|
902
912
|
controller.signal.addEventListener("abort", onAbort, { once: true });
|
|
903
913
|
});
|
|
904
914
|
guard();
|
|
@@ -1161,6 +1171,7 @@ export async function openRecovery(store, workflow, options) {
|
|
|
1161
1171
|
// From its admission the effect is in flight: a pause waits for it to complete and commit, so the admission is
|
|
1162
1172
|
// never left without its dispatch.
|
|
1163
1173
|
inFlight += 1;
|
|
1174
|
+
let deadlineTimer;
|
|
1164
1175
|
try {
|
|
1165
1176
|
// The driver's state and the admission are one transaction.
|
|
1166
1177
|
const admission = writeOf(id);
|
|
@@ -1175,7 +1186,15 @@ export async function openRecovery(store, workflow, options) {
|
|
|
1175
1186
|
throw params.signal.reason;
|
|
1176
1187
|
if (Date.now() >= deadline)
|
|
1177
1188
|
throw failure("FROZEN_EFFECT_DEADLINE", "Effect exceeded its own deadline");
|
|
1178
|
-
|
|
1189
|
+
// The deadline is the clock the driver kept, so a fraction of a millisecond, or a delay past Node's 2^31-1 ms timer
|
|
1190
|
+
// limit, waits out its time; it aborts with the TimeoutError AbortSignal.timeout gives, which a host reads as a deadline.
|
|
1191
|
+
const timeout = new AbortController();
|
|
1192
|
+
deadlineTimer = setLongTimeout(() => timeout.abort(new DOMException("The operation was aborted due to timeout", "TimeoutError")), deadline - Date.now());
|
|
1193
|
+
const signal = AbortSignal.any([params.signal, controller.signal, timeout.signal]);
|
|
1194
|
+
// Clear the deadline timer the moment the effect is aborted (a cancel, a pause, or the deadline itself): a
|
|
1195
|
+
// `runEffect` that ignores its signal and stays pending would otherwise leave this process alive until the
|
|
1196
|
+
// deadline — hours or days out — because the `finally` runs only when the dispatch settles.
|
|
1197
|
+
signal.addEventListener("abort", () => deadlineTimer?.clear(), { once: true });
|
|
1179
1198
|
// A thrown/aborted call remains UNKNOWN, including transient failures: no blind retry. Each outside call the effect
|
|
1180
1199
|
// makes is kept on its record as it ends, on the driver's commit chain, so a process killed mid-effect keeps the
|
|
1181
1200
|
// calls it made and a commit that fails is the run's failure, which aborts the effect at once.
|
|
@@ -1185,20 +1204,27 @@ export async function openRecovery(store, workflow, options) {
|
|
|
1185
1204
|
try {
|
|
1186
1205
|
value = await run({ ...params, signal, call: keep, effect: { id, argsHash } });
|
|
1187
1206
|
}
|
|
1207
|
+
catch (error) {
|
|
1208
|
+
// The host's one signal that it cannot know the outcome: what it observed is kept for the operator, without
|
|
1209
|
+
// certifying completion. Any other throw leaves the effect unknown too, with nothing observed.
|
|
1210
|
+
if (error instanceof RecoveryError && error.code === "FROZEN_EFFECT_UNKNOWN") {
|
|
1211
|
+
if (calls.length)
|
|
1212
|
+
await flush();
|
|
1213
|
+
if (error.cause !== undefined) {
|
|
1214
|
+
(snapshot.unknownResponses ??= {})[id] = error.cause;
|
|
1215
|
+
await save();
|
|
1216
|
+
await flush();
|
|
1217
|
+
}
|
|
1218
|
+
throw failure("FROZEN_EFFECT_UNKNOWN", `Reconcile ${id}: the host reported an unknown outcome`);
|
|
1219
|
+
}
|
|
1220
|
+
throw error;
|
|
1221
|
+
}
|
|
1188
1222
|
finally {
|
|
1189
1223
|
if (calls.length)
|
|
1190
1224
|
await flush();
|
|
1191
1225
|
}
|
|
1192
1226
|
if (signal.aborted)
|
|
1193
1227
|
throw signal.reason;
|
|
1194
|
-
const returned = value;
|
|
1195
|
-
if (returned?.outcome === "unknown" || returned?.effect_status === "unknown"
|
|
1196
|
-
|| returned?.details?.outcome === "unknown" || returned?.details?.effect_status === "unknown") {
|
|
1197
|
-
(snapshot.unknownResponses ??= {})[id] = value;
|
|
1198
|
-
await save(); // Preserve the observation without certifying completion.
|
|
1199
|
-
await flush();
|
|
1200
|
-
throw failure("FROZEN_EFFECT_UNKNOWN", `Reconcile ${id}: tool returned an unknown outcome`);
|
|
1201
|
-
}
|
|
1202
1228
|
const files = fileHashes(params.produces);
|
|
1203
1229
|
// A tool effect's receipt names the external call it paid for (the tool and its arguments),
|
|
1204
1230
|
// so a continuation that inherits the receipt can answer the same call from it. Any other
|
|
@@ -1210,13 +1236,18 @@ export async function openRecovery(store, workflow, options) {
|
|
|
1210
1236
|
return value;
|
|
1211
1237
|
}
|
|
1212
1238
|
finally {
|
|
1239
|
+
deadlineTimer?.clear();
|
|
1213
1240
|
settled();
|
|
1214
1241
|
}
|
|
1215
1242
|
};
|
|
1216
1243
|
},
|
|
1217
|
-
/** Every commit the driver started has landed; a failed one is thrown. */
|
|
1244
|
+
/** Every commit the driver started has landed; a failed one is thrown, a pause's or a cancel's own commit included. */
|
|
1218
1245
|
flush,
|
|
1219
|
-
/** Let go of the run once every commit the driver started has landed.
|
|
1220
|
-
|
|
1246
|
+
/** Let go of the run once every commit the driver started has landed. A commit the store refused after a pause or
|
|
1247
|
+
* cancel is thrown once the journal is released: the stop is not durable, and the host must not take the run as
|
|
1248
|
+
* stopped. It is thrown even when a later stop's commit lands, since a later commit does not carry every record the
|
|
1249
|
+
* refused one held. A run's own failed commit is the run's failure and is not thrown again here. */
|
|
1250
|
+
async close() { await chain; await journal.close(); if (refusedAfterStop !== undefined)
|
|
1251
|
+
throw refusedAfterStop; },
|
|
1221
1252
|
};
|
|
1222
1253
|
}
|
package/dist/workflow-shape.js
CHANGED
|
@@ -8,11 +8,36 @@ const isRecord = (value) => value !== null && typeof value === "object" && !Arra
|
|
|
8
8
|
// Select the declared discriminator's schema, checking nested nodes separately so
|
|
9
9
|
// their own discriminators also get useful errors. Never evaluate authored code.
|
|
10
10
|
const diagnosticDefinitions = { ...documentSchema.definitions, WorkflowNode: {}, Workflow: {} };
|
|
11
|
-
|
|
11
|
+
// A node kind may have more than one schema (a judged route and a route by value are both "route"): every variant is kept.
|
|
12
|
+
const nodeSchemas = new Map();
|
|
13
|
+
for (const schema of documentSchema.definitions.WorkflowNode.anyOf) {
|
|
14
|
+
const kind = schema.properties.node.const;
|
|
15
|
+
nodeSchemas.set(kind, [...(nodeSchemas.get(kind) ?? []), schema]);
|
|
16
|
+
}
|
|
12
17
|
const diagnosticValidators = new Map();
|
|
13
|
-
for (const schema of [documentSchema.definitions.Workflow, ...nodeSchemas.values()]) {
|
|
18
|
+
for (const schema of [documentSchema.definitions.Workflow, ...[...nodeSchemas.values()].flat()]) {
|
|
14
19
|
diagnosticValidators.set(schema, Compile({ ...schema, definitions: diagnosticDefinitions }));
|
|
15
20
|
}
|
|
21
|
+
/** The variant of its kind a node is: the one that accepts it, else the one it has the fewest errors against, so a judged
|
|
22
|
+
* route missing its state is diagnosed as a judged route, not as a route by value missing its valuePath. Nested nodes are
|
|
23
|
+
* opaque to a variant's validator, so the count is of the node's own fields. A tie goes to the variant that comes first in
|
|
24
|
+
* the schema: a bare route is diagnosed as a judged route. */
|
|
25
|
+
const variantOf = (variants, entry) => {
|
|
26
|
+
if (variants.length === 1)
|
|
27
|
+
return variants[0];
|
|
28
|
+
let closest = variants[0], fewest = Infinity;
|
|
29
|
+
for (const variant of variants) {
|
|
30
|
+
const validator = diagnosticValidators.get(variant);
|
|
31
|
+
if (validator.Check(entry))
|
|
32
|
+
return variant;
|
|
33
|
+
const count = [...validator.Errors(entry)].length;
|
|
34
|
+
if (count < fewest) {
|
|
35
|
+
closest = variant;
|
|
36
|
+
fewest = count;
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
return closest;
|
|
40
|
+
};
|
|
16
41
|
const pointer = (key) => key.replace(/~/g, "~0").replace(/\//g, "~1");
|
|
17
42
|
function discriminatedErrors(value) {
|
|
18
43
|
const errors = [];
|
|
@@ -32,9 +57,9 @@ function discriminatedErrors(value) {
|
|
|
32
57
|
if (!schema || typeof schema !== "object" || errors.length >= 8)
|
|
33
58
|
return;
|
|
34
59
|
if (schema.$ref === "#/definitions/WorkflowNode") {
|
|
35
|
-
const
|
|
36
|
-
if (
|
|
37
|
-
check(
|
|
60
|
+
const variants = isRecord(entry) && typeof entry.node === "string" ? nodeSchemas.get(entry.node) : undefined;
|
|
61
|
+
if (variants)
|
|
62
|
+
check(variantOf(variants, entry), entry, path);
|
|
38
63
|
return; // Unknown/missing discriminators retain the full validator's fallback.
|
|
39
64
|
}
|
|
40
65
|
if (schema.$ref === "#/definitions/Workflow") {
|
package/dist/workflow.js
CHANGED
|
@@ -820,14 +820,14 @@ export function validateWorkflow(workflow, opts) {
|
|
|
820
820
|
const extraneous = Object.keys(node).filter((key) => !allowed.has(key));
|
|
821
821
|
if (extraneous.length)
|
|
822
822
|
fail("NODE_KEY_UNKNOWN", ptrOf(path), `${path} (${node.label || node.node}): unknown key(s) for a ${node.node} node: ${extraneous.join(", ")}`);
|
|
823
|
-
//
|
|
824
|
-
// is the state key whenever `as` is omitted, so labels are held to the same rule.
|
|
825
|
-
if (typeof node.as === "string" && node.as
|
|
823
|
+
// State keys that hold a `$` are engine-owned (`$host`, and a typed question's `<as>$answers` and `<as>$verify`): no node
|
|
824
|
+
// may write one, wherever the `$` is. A label is the state key whenever `as` is omitted, so labels are held to the same rule.
|
|
825
|
+
if (typeof node.as === "string" && holdsDollar(node.as))
|
|
826
826
|
fail("STATE_KEY_RESERVED", ptrOf(path), `${path} (${node.label || node.node}): "as" must not name an engine-owned "$" state key ("${node.as}")`);
|
|
827
827
|
// Only an unaliased generative or code node writes under its label; every other label is a name, not a key.
|
|
828
828
|
const labelIsKey = (node.node === "agent" || node.node === "decide" || node.node === "extract" || node.node === "code") && node.as === undefined;
|
|
829
|
-
if (labelIsKey && typeof node.label === "string" && node.label
|
|
830
|
-
fail("STATE_KEY_RESERVED", ptrOf(path), `${path} (${node.label}): label must not
|
|
829
|
+
if (labelIsKey && typeof node.label === "string" && holdsDollar(node.label))
|
|
830
|
+
fail("STATE_KEY_RESERVED", ptrOf(path), `${path} (${node.label}): label must not contain "$" — an unaliased ${node.node} node writes under its label, and "$" keys are engine-owned`);
|
|
831
831
|
}
|
|
832
832
|
switch (node.node) {
|
|
833
833
|
case "chain":
|
|
@@ -932,7 +932,7 @@ export function validateWorkflow(workflow, opts) {
|
|
|
932
932
|
if (node.onItemFailure === "collect") {
|
|
933
933
|
if (typeof node.failedAs !== "string" || !node.failedAs.trim())
|
|
934
934
|
fail("NODE_FIELD_INVALID", ptrOf(path), `${path} (${node.label}): onItemFailure "collect" needs failedAs, the state key the map writes its failed items under`);
|
|
935
|
-
else if (node.failedAs
|
|
935
|
+
else if (holdsDollar(node.failedAs))
|
|
936
936
|
fail("STATE_KEY_RESERVED", ptrOf(path), `${path} (${node.label}): failedAs must not name an engine-owned "$" state key ("${node.failedAs}")`);
|
|
937
937
|
else if (node.failedAs === node.as)
|
|
938
938
|
fail("NODE_FIELD_INVALID", ptrOf(path), `${path} (${node.label}): failedAs must differ from as: the map writes its results and its failed items under two keys`);
|
|
@@ -2292,6 +2292,8 @@ const SCOPED_EFFECTS = Symbol("workflow.scopedEffects");
|
|
|
2292
2292
|
const MAP_ITEM = Symbol("workflow.mapItem");
|
|
2293
2293
|
/** The reserved state key host policy writes under. Engine-owned like every `$`-prefixed key: no node may name it. */
|
|
2294
2294
|
export const HOST_STATE_KEY = "$host";
|
|
2295
|
+
/** Whether a state key is engine-owned: it holds a `$` (`$host`, `<as>$answers`, `<as>$verify`). No node may declare a write to one. */
|
|
2296
|
+
const holdsDollar = (key) => key.includes("$");
|
|
2295
2297
|
const hostStateOf = (state) => {
|
|
2296
2298
|
const host = state[HOST_STATE_KEY];
|
|
2297
2299
|
return host && typeof host === "object" && !Array.isArray(host) ? host : {};
|
package/package.json
CHANGED
|
@@ -12,7 +12,7 @@ A workflow is {"v":2,"name","schemas","input"?,"output","root"}.
|
|
|
12
12
|
- The input contract: the run starts from the input object and nothing else. `input.schemaId`, when present, is enforced before the first node. A node may read only input keys and keys an earlier node wrote.
|
|
13
13
|
- `output.schemaId` validates the final value; `output.path` selects it from state and must name an input key or a key an earlier node wrote.
|
|
14
14
|
- State is one object. Each step writes its result under one key: its `as`, or, for an agent, decide, extract or code node without `as`, its label. A report writes `report_markdown`. A code node without `as` that returns an object merges that object's top-level fields instead.
|
|
15
|
-
- Keys that
|
|
15
|
+
- Keys that hold a `$` are engine-owned: `$host` (host state), `<as>$answers` (raw judge answers and confidences) and `<as>$verify` (verifier results). Read them; never write one. No `as` may hold a `$`, and neither may the label of an agent, decide, extract or code node without `as`.
|
|
16
16
|
- `{dot.path}` placeholders are replaced from state in `state` maps (including `verify.state` and `ask.state`), call `args`, `input`, `env` and `produces`, a child workflow's `input`, and `escalate.summary`; `pick` and `sift` `describe` templates also see `{item...}`. A placeholder that is the whole string keeps its value's type. Every key a state map, call or child input interpolates must be produced upstream. `instructions`, `code` and `command` are literal and never interpolated.
|
|
17
17
|
|
|
18
18
|
## Node kinds
|