ai-engineering-loop 1.2.1 → 1.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +47 -4
- package/README.npm.md +7 -0
- package/SECURITY.md +3 -1
- package/agents/recipe-builder.md +28 -0
- package/bin/ai-engineering-loop.js +432 -11
- package/core/controlled-workflow-runtime.md +77 -0
- package/core/local-workflow-studio.md +77 -0
- package/core/workflow-recipes.md +140 -0
- package/lib/budget.js +42 -4
- package/lib/doctor.js +26 -2
- package/lib/handoff.js +209 -0
- package/lib/node-registry.js +155 -0
- package/lib/recipe-builder.js +200 -0
- package/lib/recipe.js +495 -0
- package/lib/run-state.js +50 -3
- package/lib/runtime-files.js +2 -0
- package/lib/studio-server.js +183 -0
- package/lib/workflow-runtime.js +718 -0
- package/package.json +3 -1
- package/recipes/audit.json +32 -0
- package/recipes/bugfix.json +63 -0
- package/recipes/default.json +49 -0
- package/recipes/docs-light.json +63 -0
- package/recipes/high-risk.json +69 -0
- package/recipes/refactor.json +63 -0
- package/schemas/compiled-plan.schema.json +19 -0
- package/schemas/handoff.schema.json +36 -0
- package/schemas/node-definition.schema.json +33 -0
- package/schemas/recipe.schema.json +26 -0
- package/schemas/workflow-event.schema.json +29 -0
- package/schemas/workflow-state.schema.json +29 -0
- package/studio/app.js +461 -0
- package/studio/index.html +31 -0
- package/studio/styles.css +75 -0
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Local Workflow Studio
|
|
2
|
+
|
|
3
|
+
Workflow Studio is a localhost-only visual control plane for declarative recipes and recipe-bound runs. It does not execute models, shell commands, or arbitrary node code. The existing recipe compiler and controlled workflow runtime remain the source of truth.
|
|
4
|
+
|
|
5
|
+
## Start
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
ai-engineering-loop studio
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
The CLI binds only to `127.0.0.1`, creates a random process-local session token, and opens a tokenized bootstrap URL. The server exchanges that URL for an `HttpOnly; SameSite=Strict` cookie and removes the token from the address bar. Use `--no-open` when you want to copy the printed URL manually, or `--port 0` to let the operating system choose an available port.
|
|
12
|
+
|
|
13
|
+
The server rejects non-local `Host` headers. Static assets are bundled in the npm package; there are no CDNs, analytics, remote fonts, or browser-side model calls. Responses use a restrictive Content Security Policy and are not cached.
|
|
14
|
+
|
|
15
|
+
## Authoring model
|
|
16
|
+
|
|
17
|
+
The canvas loads built-in and project recipes through the same APIs as the CLI:
|
|
18
|
+
|
|
19
|
+
1. Select a recipe.
|
|
20
|
+
2. Fork a protected built-in by changing its ID.
|
|
21
|
+
3. Add or edit allowlisted node types.
|
|
22
|
+
4. Connect nodes with dependency IDs.
|
|
23
|
+
5. Review graph validation and hashes for every compatible mode.
|
|
24
|
+
6. Install only after explicit confirmation.
|
|
25
|
+
|
|
26
|
+
Installation writes a private temporary candidate, invokes the existing atomic installer, and removes the candidate. Built-ins cannot be overwritten. Replacing a project recipe still requires the version policy enforced by `recipe install --replace`.
|
|
27
|
+
|
|
28
|
+
## Live execution
|
|
29
|
+
|
|
30
|
+
For a recipe-bound current run, Studio polls a small live snapshot rather than reloading the recipe catalog. It displays:
|
|
31
|
+
|
|
32
|
+
- current node states and the append-only event stream;
|
|
33
|
+
- an animated connector leaving the running node;
|
|
34
|
+
- provider-recorded run token spend and configured limit;
|
|
35
|
+
- explicit approval and retry controls;
|
|
36
|
+
- a short redacted activity message reported by the host:
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
ai-engineering-loop node activity <node-id> \
|
|
40
|
+
--message "Comparing the failing test with the implementation"
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Activity can only be attached to a `RUNNING` node. It is capped at 240 printable characters, secret-redacted before persistence, and added to the integrity-bound workflow event chain.
|
|
44
|
+
|
|
45
|
+
## Decision memory
|
|
46
|
+
|
|
47
|
+
Studio can record a concise decision and rationale against the current run. Decision memory is private runtime data under the run directory, is bounded to 100 entries, and is redacted before persistence. It becomes part of exported handoffs so the next developer or agent receives the “why,” not only the final state.
|
|
48
|
+
|
|
49
|
+
Do not use decision memory as a credential store or as a replacement for a repository ADR when a durable architectural decision belongs in version control.
|
|
50
|
+
|
|
51
|
+
## Knowledge handoff
|
|
52
|
+
|
|
53
|
+
The top bar provides two forms:
|
|
54
|
+
|
|
55
|
+
- **Copy Brief** creates a human-readable developer summary.
|
|
56
|
+
- **Export Handoff** downloads a machine-readable `.ael-handoff.json` bundle for another agent.
|
|
57
|
+
|
|
58
|
+
The bundle contains objective, workflow state, artifact descriptors, decisions, transitions, and ready next actions. It deliberately excludes source contents. Strings pass through secret and home-path redaction, and the complete canonical manifest is bound to `bundleHash`.
|
|
59
|
+
|
|
60
|
+
Verify a received bundle before trusting it:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
ai-engineering-loop handoff inspect run-id.ael-handoff.json
|
|
64
|
+
ai-engineering-loop handoff brief run-id.ael-handoff.json
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Inspection accepts only regular files smaller than 1 MiB inside the repository, rejects symlinks and path escapes, and fails if any field no longer matches the bundle hash.
|
|
68
|
+
|
|
69
|
+
## Trust boundary
|
|
70
|
+
|
|
71
|
+
Studio improves authoring and observability; it does not weaken runtime gates:
|
|
72
|
+
|
|
73
|
+
- graph validation does not imply execution permission;
|
|
74
|
+
- command nodes remain disabled;
|
|
75
|
+
- approvals remain explicit state transitions;
|
|
76
|
+
- token data comes only from the provider-recorded usage ledger;
|
|
77
|
+
- handoff integrity proves the bundle has not changed after export, not that every original claim was true.
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# Declarative Workflow Recipes
|
|
2
|
+
|
|
3
|
+
Workflow recipes describe a directed acyclic graph of engineering stages in JSON. They are an additive authoring layer over the existing AI Engineering Loop: `recipe` commands provide deterministic validation and inspection, while `run --recipe` opts into the controlled runtime. Runs without a recipe keep their established behavior.
|
|
4
|
+
|
|
5
|
+
This separation is deliberate. A visual editor or an AI agent may eventually author the JSON, but the compiler—not the authoring interface—owns safety invariants.
|
|
6
|
+
|
|
7
|
+
## Sources and precedence
|
|
8
|
+
|
|
9
|
+
Recipes are loaded from two locations:
|
|
10
|
+
|
|
11
|
+
1. Package presets in `recipes/*.json`
|
|
12
|
+
2. Project recipes in `.ai-engineering-loop/recipes/*.json`
|
|
13
|
+
|
|
14
|
+
Built-in IDs are reserved and cannot be shadowed by a project file. Recipe IDs and node IDs use lowercase letters, numbers, and hyphens. Recipe files and the project recipe directory must be regular filesystem entries; symlinks are rejected.
|
|
15
|
+
|
|
16
|
+
The package includes:
|
|
17
|
+
|
|
18
|
+
| Recipe | Purpose | Modes |
|
|
19
|
+
| --- | --- | --- |
|
|
20
|
+
| `default` | Existing production safety loop | `ASSISTED`, `UNATTENDED` |
|
|
21
|
+
| `audit` | Read-only structured analysis | `REPORT_ONLY` |
|
|
22
|
+
| `bugfix` | Root-cause-first repair | `ASSISTED`, `UNATTENDED` |
|
|
23
|
+
| `docs-light` | Bounded documentation change | `ASSISTED`, `UNATTENDED` |
|
|
24
|
+
| `high-risk` | Blast-radius analysis and pre-mutation approval | `ASSISTED`, `UNATTENDED` |
|
|
25
|
+
| `refactor` | Behavior-baseline-first refactor | `ASSISTED`, `UNATTENDED` |
|
|
26
|
+
|
|
27
|
+
## Safety backbone
|
|
28
|
+
|
|
29
|
+
The graph compiler fails closed.
|
|
30
|
+
|
|
31
|
+
- `REPORT_ONLY` requires `goal-contract → ... → report` and rejects every mutation or remote-side-effect node.
|
|
32
|
+
- `ASSISTED` and `UNATTENDED` require the ordered backbone `goal-contract → maker → verification → devil-advocate → judge → delivery`.
|
|
33
|
+
- `ASSISTED` additionally requires human approval after Judge and before delivery.
|
|
34
|
+
- Required safety nodes cannot be conditional.
|
|
35
|
+
- `report` and `delivery` are graph terminals. Every node must lead to one terminal.
|
|
36
|
+
- Cycles, duplicate singleton safety nodes, missing dependencies, unknown node types, and incompatible modes are invalid.
|
|
37
|
+
|
|
38
|
+
Dependencies may include additional nodes between required stages. A custom analysis node before `maker`, for example, does not weaken the mandatory backbone.
|
|
39
|
+
|
|
40
|
+
## Node catalog
|
|
41
|
+
|
|
42
|
+
| Type | Intended use | Model | Repository mutation | Remote side effect |
|
|
43
|
+
| --- | --- | ---: | ---: | ---: |
|
|
44
|
+
| `goal-contract` | Acceptance contract | No | No | No |
|
|
45
|
+
| `agent` | Bounded structured analysis | Yes | No | No |
|
|
46
|
+
| `command` | Trusted repository executable | No | No | No |
|
|
47
|
+
| `condition` | Structured branch predicate | No | No | No |
|
|
48
|
+
| `approval` | Human checkpoint | No | No | No |
|
|
49
|
+
| `artifact-check` | Deterministic artifact assertion | No | No | No |
|
|
50
|
+
| `maker` | Implementation | Yes | Yes | No |
|
|
51
|
+
| `verification` | Evidence collection | No | No | No |
|
|
52
|
+
| `devil-advocate` | Adversarial review | Yes | No | No |
|
|
53
|
+
| `judge` | Independent verdict | Yes | No | No |
|
|
54
|
+
| `report` | Read-only terminal | No | No | No |
|
|
55
|
+
| `delivery` | Delivery terminal | No | No | Yes |
|
|
56
|
+
|
|
57
|
+
An `agent` node must declare a role, bounded `maxFiles` and `maxTokens`, and a repository-relative output schema. A `command` node must use a bare executable name, an argument array, a bounded timeout, and expected exit codes. Shells and general-purpose interpreters are rejected; command nodes are for trusted repository tools, not arbitrary script text.
|
|
58
|
+
|
|
59
|
+
## Authoring example
|
|
60
|
+
|
|
61
|
+
Project recipe:
|
|
62
|
+
|
|
63
|
+
```json
|
|
64
|
+
{
|
|
65
|
+
"schemaVersion": 1,
|
|
66
|
+
"id": "api-bugfix",
|
|
67
|
+
"version": 1,
|
|
68
|
+
"description": "Analyze an API regression before entering the standard repair loop.",
|
|
69
|
+
"compatibleModes": ["ASSISTED"],
|
|
70
|
+
"nodes": [
|
|
71
|
+
{ "id": "goal", "type": "goal-contract" },
|
|
72
|
+
{
|
|
73
|
+
"id": "api-analysis",
|
|
74
|
+
"type": "agent",
|
|
75
|
+
"dependsOn": ["goal"],
|
|
76
|
+
"role": "api-regression-analyst",
|
|
77
|
+
"context": { "maxFiles": 25, "maxTokens": 8000 },
|
|
78
|
+
"output": {
|
|
79
|
+
"artifact": "api-analysis",
|
|
80
|
+
"schema": "schemas/report.schema.json"
|
|
81
|
+
}
|
|
82
|
+
},
|
|
83
|
+
{ "id": "maker", "type": "maker", "dependsOn": ["api-analysis"] },
|
|
84
|
+
{ "id": "verification", "type": "verification", "dependsOn": ["maker"] },
|
|
85
|
+
{ "id": "review", "type": "devil-advocate", "dependsOn": ["verification"] },
|
|
86
|
+
{ "id": "judge", "type": "judge", "dependsOn": ["review"] },
|
|
87
|
+
{
|
|
88
|
+
"id": "approval",
|
|
89
|
+
"type": "approval",
|
|
90
|
+
"dependsOn": ["judge"],
|
|
91
|
+
"message": "Approve delivery."
|
|
92
|
+
},
|
|
93
|
+
{ "id": "delivery", "type": "delivery", "dependsOn": ["approval"] }
|
|
94
|
+
]
|
|
95
|
+
}
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Validate and inspect it:
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
ai-engineering-loop recipe validate api-bugfix --mode assisted
|
|
102
|
+
ai-engineering-loop recipe explain api-bugfix --mode assisted
|
|
103
|
+
ai-engineering-loop recipe graph api-bugfix --mode assisted
|
|
104
|
+
ai-engineering-loop recipe simulate api-bugfix --mode assisted
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
`simulate` compiles the graph and reports capability counts. It never calls a model, mutates the repository, executes a command, or performs delivery.
|
|
108
|
+
|
|
109
|
+
## AI-assisted authoring protocol
|
|
110
|
+
|
|
111
|
+
Until a visual editor exists, an AI agent can safely act as the recipe builder:
|
|
112
|
+
|
|
113
|
+
1. Ask the user for the outcome, allowed run modes, required checkpoints, context limits, and delivery risk.
|
|
114
|
+
2. Start from the closest built-in preset rather than a blank graph.
|
|
115
|
+
3. Add only nodes that change the workflow meaningfully.
|
|
116
|
+
4. Write the candidate to `.ai-engineering-loop/recipes/<id>.json`.
|
|
117
|
+
5. Run `recipe validate`, `recipe explain`, `recipe graph`, and `recipe simulate`.
|
|
118
|
+
6. Present the explanation, graph hash, warnings, and capability counts for human confirmation.
|
|
119
|
+
7. Do not claim execution from validation or simulation. Execution is proven only by a recipe-bound run and its validated event chain.
|
|
120
|
+
|
|
121
|
+
Machine consumers should use `--json`. The compiled plan includes a canonical source hash and graph hash so execution and audit logs bind to the exact graph reviewed by the user. Runtime behavior is specified in `core/controlled-workflow-runtime.md`.
|
|
122
|
+
|
|
123
|
+
The CLI supports safe scaffolding and installation:
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
ai-engineering-loop recipe create api-bugfix --from bugfix
|
|
127
|
+
ai-engineering-loop recipe inspect api-bugfix --json
|
|
128
|
+
ai-engineering-loop recipe diff bugfix api-bugfix
|
|
129
|
+
ai-engineering-loop recipe install candidate-v2.json --replace
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
Creation never overwrites an existing recipe. Installation validates before writing, reserves built-in IDs, requires `--replace` plus a strictly higher version, and stores the previous document under `.ai-engineering-loop/recipes/.history/<id>/`. Candidate paths and recipe stores reject filesystem escape and symlink traversal.
|
|
133
|
+
|
|
134
|
+
## Schemas
|
|
135
|
+
|
|
136
|
+
- `schemas/recipe.schema.json`: top-level authoring document
|
|
137
|
+
- `schemas/node-definition.schema.json`: node shape and type catalog
|
|
138
|
+
- `schemas/compiled-plan.schema.json`: deterministic compiler output
|
|
139
|
+
|
|
140
|
+
Semantic safety checks are stricter than JSON Schema and are enforced by `recipe validate` and the compiler.
|
package/lib/budget.js
CHANGED
|
@@ -69,6 +69,9 @@ function readUsageFile(filePath) {
|
|
|
69
69
|
throw new Error('totalTokens does not equal inputTokens + outputTokens');
|
|
70
70
|
}
|
|
71
71
|
validateModel(entry.model);
|
|
72
|
+
if (entry.nodeId != null && !/^[a-z][a-z0-9-]{0,63}$/.test(entry.nodeId)) {
|
|
73
|
+
throw new Error('nodeId is invalid');
|
|
74
|
+
}
|
|
72
75
|
if (typeof entry.recordedAt !== 'string' || Number.isNaN(Date.parse(entry.recordedAt))) {
|
|
73
76
|
throw new Error('recordedAt is invalid');
|
|
74
77
|
}
|
|
@@ -107,7 +110,7 @@ function resolveRun(rootDir, runId) {
|
|
|
107
110
|
return current;
|
|
108
111
|
}
|
|
109
112
|
|
|
110
|
-
function budgetStatus(rootDir, { runId, estimatedTokens = 0, now = new Date() } = {}) {
|
|
113
|
+
function budgetStatus(rootDir, { runId, nodeId, estimatedTokens = 0, now = new Date() } = {}) {
|
|
111
114
|
validateTokenCount('estimatedTokens', estimatedTokens);
|
|
112
115
|
const run = resolveRun(rootDir, runId);
|
|
113
116
|
const policy = loadPolicy(rootDir);
|
|
@@ -120,6 +123,30 @@ function budgetStatus(rootDir, { runId, estimatedTokens = 0, now = new Date() }
|
|
|
120
123
|
const projectedToday = spentToday + estimatedTokens;
|
|
121
124
|
const projectedThisRun = spentThisRun + estimatedTokens;
|
|
122
125
|
const reasons = [];
|
|
126
|
+
let node = null;
|
|
127
|
+
let spentThisNode = 0;
|
|
128
|
+
let nodeTokenLimit = null;
|
|
129
|
+
if (nodeId) {
|
|
130
|
+
const { workflowStatus, NODE_STATES } = require('./workflow-runtime.js');
|
|
131
|
+
const workflow = workflowStatus(rootDir, { runId: run.runId });
|
|
132
|
+
node = workflow.plan.nodes.find((candidate) => candidate.id === nodeId);
|
|
133
|
+
if (!node) throw usageError(`Unknown workflow node: ${nodeId}`);
|
|
134
|
+
if (!node.capabilities.modelRequired) reasons.push(`node ${nodeId} does not permit model usage`);
|
|
135
|
+
const status = workflow.state.nodes[nodeId].status;
|
|
136
|
+
if (![NODE_STATES.READY, NODE_STATES.RUNNING].includes(status)) {
|
|
137
|
+
reasons.push(`node ${nodeId} is ${status}, not READY`);
|
|
138
|
+
}
|
|
139
|
+
nodeTokenLimit = node.context?.maxTokens || null;
|
|
140
|
+
spentThisNode = allEntries
|
|
141
|
+
.filter((entry) => entry.runId === run.runId && entry.nodeId === nodeId)
|
|
142
|
+
.reduce((sum, entry) => sum + entry.totalTokens, 0);
|
|
143
|
+
if (nodeTokenLimit != null && spentThisNode + estimatedTokens > nodeTokenLimit) {
|
|
144
|
+
reasons.push(`node ${nodeId} token limit would be exceeded`);
|
|
145
|
+
}
|
|
146
|
+
if (nodeTokenLimit != null && estimatedTokens === 0 && spentThisNode >= nodeTokenLimit) {
|
|
147
|
+
reasons.push(`node ${nodeId} token limit is exhausted`);
|
|
148
|
+
}
|
|
149
|
+
}
|
|
123
150
|
if (policy.killSwitch) reasons.push('kill switch is active');
|
|
124
151
|
if (projectedThisRun > policy.perRunTokenLimit) reasons.push('per-run token limit would be exceeded');
|
|
125
152
|
if (projectedToday > policy.dailyTokenLimit) reasons.push('daily token limit would be exceeded');
|
|
@@ -143,7 +170,13 @@ function budgetStatus(rootDir, { runId, estimatedTokens = 0, now = new Date() }
|
|
|
143
170
|
spentToday,
|
|
144
171
|
dailyTokenLimit: policy.dailyTokenLimit,
|
|
145
172
|
remainingToday: Math.max(0, policy.dailyTokenLimit - spentToday),
|
|
146
|
-
estimatedTokens
|
|
173
|
+
estimatedTokens,
|
|
174
|
+
...(node ? {
|
|
175
|
+
nodeId,
|
|
176
|
+
spentThisNode,
|
|
177
|
+
nodeTokenLimit,
|
|
178
|
+
remainingThisNode: nodeTokenLimit == null ? null : Math.max(0, nodeTokenLimit - spentThisNode)
|
|
179
|
+
} : {})
|
|
147
180
|
};
|
|
148
181
|
}
|
|
149
182
|
|
|
@@ -182,7 +215,7 @@ function appendPrivateLine(rootDir, filePath, line) {
|
|
|
182
215
|
|
|
183
216
|
function recordTokenUsage(
|
|
184
217
|
rootDir,
|
|
185
|
-
{ runId, inputTokens, outputTokens, model, now = new Date() } = {}
|
|
218
|
+
{ runId, nodeId, inputTokens, outputTokens, model, now = new Date() } = {}
|
|
186
219
|
) {
|
|
187
220
|
const run = resolveRun(rootDir, runId);
|
|
188
221
|
validateTokenCount('inputTokens', inputTokens);
|
|
@@ -197,10 +230,15 @@ function recordTokenUsage(
|
|
|
197
230
|
outputTokens,
|
|
198
231
|
totalTokens,
|
|
199
232
|
model: normalizedModel,
|
|
233
|
+
...(nodeId ? { nodeId } : {}),
|
|
200
234
|
recordedAt: now.toISOString()
|
|
201
235
|
};
|
|
236
|
+
if (nodeId) budgetStatus(rootDir, { runId: run.runId, nodeId, now });
|
|
202
237
|
appendPrivateLine(rootDir, usagePath(rootDir, now), `${JSON.stringify(entry)}\n`);
|
|
203
|
-
return {
|
|
238
|
+
return {
|
|
239
|
+
entry,
|
|
240
|
+
status: budgetStatus(rootDir, { runId: run.runId, nodeId, now })
|
|
241
|
+
};
|
|
204
242
|
}
|
|
205
243
|
|
|
206
244
|
function setKillSwitch(rootDir, active) {
|
package/lib/doctor.js
CHANGED
|
@@ -4,6 +4,7 @@ const fs = require('fs');
|
|
|
4
4
|
const path = require('path');
|
|
5
5
|
const { loadEvaluationCases } = require('./evaluation.js');
|
|
6
6
|
const { auditPackage } = require('./package-audit.js');
|
|
7
|
+
const { listRecipes, loadRecipe, validateRecipe } = require('./recipe.js');
|
|
7
8
|
|
|
8
9
|
const REQUIRED_HOST_ASSETS = [
|
|
9
10
|
'.claude/skills/ai-engineering-loop/SKILL.md',
|
|
@@ -24,7 +25,13 @@ const REQUIRED_SCHEMAS = [
|
|
|
24
25
|
'finding-ledger.schema.json',
|
|
25
26
|
'verdict.schema.json',
|
|
26
27
|
'delivery.schema.json',
|
|
27
|
-
'report.schema.json'
|
|
28
|
+
'report.schema.json',
|
|
29
|
+
'recipe.schema.json',
|
|
30
|
+
'node-definition.schema.json',
|
|
31
|
+
'compiled-plan.schema.json',
|
|
32
|
+
'workflow-state.schema.json',
|
|
33
|
+
'workflow-event.schema.json',
|
|
34
|
+
'handoff.schema.json'
|
|
28
35
|
];
|
|
29
36
|
|
|
30
37
|
function check(id, passed, detail) {
|
|
@@ -66,6 +73,23 @@ function runDoctor(rootDir = path.join(__dirname, '..')) {
|
|
|
66
73
|
schemaErrors.length ? schemaErrors.join('; ') : `${REQUIRED_SCHEMAS.length} schemas parse`
|
|
67
74
|
));
|
|
68
75
|
|
|
76
|
+
try {
|
|
77
|
+
const recipes = listRecipes(rootDir).filter(({ source }) => source === 'builtin');
|
|
78
|
+
const recipeErrors = recipes.flatMap(({ id }) => {
|
|
79
|
+
const { recipe } = loadRecipe(rootDir, id);
|
|
80
|
+
return recipe.compatibleModes.flatMap((mode) => (
|
|
81
|
+
validateRecipe(recipe, { mode }).errors.map((error) => `${id} (${mode}): ${error}`)
|
|
82
|
+
));
|
|
83
|
+
});
|
|
84
|
+
checks.push(check(
|
|
85
|
+
'workflow-recipes',
|
|
86
|
+
recipes.length > 0 && recipeErrors.length === 0,
|
|
87
|
+
recipeErrors.length ? recipeErrors.join('; ') : `${recipes.length} built-in recipes validate`
|
|
88
|
+
));
|
|
89
|
+
} catch (cause) {
|
|
90
|
+
checks.push(check('workflow-recipes', false, cause.message));
|
|
91
|
+
}
|
|
92
|
+
|
|
69
93
|
try {
|
|
70
94
|
const fixtures = loadEvaluationCases(path.join(rootDir, 'evals', 'cases'));
|
|
71
95
|
checks.push(check('evaluation-catalog', fixtures.length === 20, `${fixtures.length} valid cases; required 20`));
|
|
@@ -74,7 +98,7 @@ function runDoctor(rootDir = path.join(__dirname, '..')) {
|
|
|
74
98
|
}
|
|
75
99
|
|
|
76
100
|
const packageFiles = new Set(pkg.files || []);
|
|
77
|
-
const requiredPackageEntries = ['bin/', 'lib/', 'core/', 'policies/', 'schemas/', 'evals/'];
|
|
101
|
+
const requiredPackageEntries = ['bin/', 'lib/', 'core/', 'policies/', 'recipes/', 'schemas/', 'evals/', 'studio/'];
|
|
78
102
|
const missingPackageEntries = requiredPackageEntries.filter((entry) => !packageFiles.has(entry));
|
|
79
103
|
checks.push(check(
|
|
80
104
|
'package-assets',
|
package/lib/handoff.js
ADDED
|
@@ -0,0 +1,209 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
const fs = require('fs');
|
|
4
|
+
const path = require('path');
|
|
5
|
+
const { canonicalJson, sha256 } = require('./recipe.js');
|
|
6
|
+
const { atomicWriteJson, getCurrentRun, loadRun, runsDir } = require('./run-state.js');
|
|
7
|
+
const { redactSecrets } = require('./safe-context.js');
|
|
8
|
+
const { workflowStatus } = require('./workflow-runtime.js');
|
|
9
|
+
|
|
10
|
+
const AUDIENCES = new Set(['developer', 'agent', 'auditor']);
|
|
11
|
+
const MAX_BUNDLE_BYTES = 1024 * 1024;
|
|
12
|
+
|
|
13
|
+
function handoffError(message, code = 'HANDOFF_FAILED') {
|
|
14
|
+
const error = new Error(message);
|
|
15
|
+
error.code = code;
|
|
16
|
+
return error;
|
|
17
|
+
}
|
|
18
|
+
|
|
19
|
+
function resolveRun(rootDir, runId) {
|
|
20
|
+
const run = runId ? loadRun(rootDir, runId) : getCurrentRun(rootDir);
|
|
21
|
+
if (!run) throw handoffError('No current run. Pass --run <id>.', 'NO_CURRENT_RUN');
|
|
22
|
+
return run;
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
function redactValue(value, categories) {
|
|
26
|
+
if (typeof value === 'string') {
|
|
27
|
+
const result = redactSecrets(value);
|
|
28
|
+
result.categories.forEach((category) => categories.add(category));
|
|
29
|
+
const redacted = result.text
|
|
30
|
+
.replace(/(?:file:\/\/)?\/?Users\/[A-Za-z0-9._-]+/g, '[REDACTED HOME]')
|
|
31
|
+
.replace(/\b[A-Z]:\\Users\\[^\\\s]+/gi, '[REDACTED HOME]');
|
|
32
|
+
if (redacted !== result.text) categories.add('HOME_PATH');
|
|
33
|
+
return redacted;
|
|
34
|
+
}
|
|
35
|
+
if (Array.isArray(value)) return value.map((item) => redactValue(item, categories));
|
|
36
|
+
if (value && typeof value === 'object') {
|
|
37
|
+
return Object.fromEntries(Object.entries(value).map(([key, item]) => [key, redactValue(item, categories)]));
|
|
38
|
+
}
|
|
39
|
+
return value;
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
function loadOptionalJson(filePath) {
|
|
43
|
+
if (!fs.existsSync(filePath)) return null;
|
|
44
|
+
const stat = fs.lstatSync(filePath);
|
|
45
|
+
if (stat.isSymbolicLink() || !stat.isFile()) throw handoffError('Optional handoff input must be a regular file');
|
|
46
|
+
try {
|
|
47
|
+
return JSON.parse(fs.readFileSync(filePath, 'utf8'));
|
|
48
|
+
} catch (cause) {
|
|
49
|
+
throw handoffError(`Optional handoff input is invalid JSON: ${cause.message}`, 'INVALID_HANDOFF_INPUT');
|
|
50
|
+
}
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
function recordDecision(rootDir, {
|
|
54
|
+
runId,
|
|
55
|
+
summary,
|
|
56
|
+
rationale,
|
|
57
|
+
consequences = [],
|
|
58
|
+
now = new Date()
|
|
59
|
+
} = {}) {
|
|
60
|
+
const run = resolveRun(rootDir, runId);
|
|
61
|
+
if (typeof summary !== 'string' || !summary.trim() || summary.trim().length > 160) {
|
|
62
|
+
throw handoffError('Decision summary must be 1-160 characters', 'INVALID_DECISION');
|
|
63
|
+
}
|
|
64
|
+
if (typeof rationale !== 'string' || !rationale.trim() || rationale.trim().length > 1000) {
|
|
65
|
+
throw handoffError('Decision rationale must be 1-1000 characters', 'INVALID_DECISION');
|
|
66
|
+
}
|
|
67
|
+
if (!Array.isArray(consequences) || consequences.length > 20 || consequences.some(
|
|
68
|
+
(item) => typeof item !== 'string' || !item.trim() || item.length > 240
|
|
69
|
+
)) {
|
|
70
|
+
throw handoffError('Decision consequences must contain at most 20 short strings', 'INVALID_DECISION');
|
|
71
|
+
}
|
|
72
|
+
const filePath = path.join(runsDir(rootDir), run.runId, 'decisions.json');
|
|
73
|
+
const decisions = loadOptionalJson(filePath) || [];
|
|
74
|
+
if (!Array.isArray(decisions) || decisions.length >= 100) {
|
|
75
|
+
throw handoffError('Decision memory must be an array with fewer than 100 entries', 'INVALID_DECISION');
|
|
76
|
+
}
|
|
77
|
+
const categories = new Set();
|
|
78
|
+
const entry = redactValue({
|
|
79
|
+
id: `decision-${String(decisions.length + 1).padStart(3, '0')}`,
|
|
80
|
+
summary: summary.trim(),
|
|
81
|
+
rationale: rationale.trim(),
|
|
82
|
+
consequences: consequences.map((item) => item.trim()),
|
|
83
|
+
recordedAt: now.toISOString()
|
|
84
|
+
}, categories);
|
|
85
|
+
atomicWriteJson(filePath, [...decisions, entry]);
|
|
86
|
+
return { runId: run.runId, decision: entry, redactionCategories: [...categories].sort() };
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
function createHandoff(rootDir, { runId, audience = 'developer', now = new Date() } = {}) {
|
|
90
|
+
if (!AUDIENCES.has(audience)) throw handoffError(`Invalid audience: ${audience}`);
|
|
91
|
+
const run = resolveRun(rootDir, runId);
|
|
92
|
+
const directory = path.join(runsDir(rootDir), run.runId);
|
|
93
|
+
const workflow = run.workflow ? workflowStatus(rootDir, { runId: run.runId }) : null;
|
|
94
|
+
const decisions = loadOptionalJson(path.join(directory, 'decisions.json')) || [];
|
|
95
|
+
if (!Array.isArray(decisions)) throw handoffError('Decision memory must be an array', 'INVALID_HANDOFF_INPUT');
|
|
96
|
+
const categories = new Set();
|
|
97
|
+
const content = redactValue({
|
|
98
|
+
objective: run.task,
|
|
99
|
+
state: run.state,
|
|
100
|
+
mode: run.mode,
|
|
101
|
+
iteration: run.iteration,
|
|
102
|
+
recipe: run.workflow || null,
|
|
103
|
+
nodes: workflow ? workflow.plan.nodes.map((node) => ({
|
|
104
|
+
id: node.id,
|
|
105
|
+
type: node.type,
|
|
106
|
+
status: workflow.state.nodes[node.id].status,
|
|
107
|
+
attempts: workflow.state.nodes[node.id].attempts,
|
|
108
|
+
reason: workflow.state.nodes[node.id].reason || null,
|
|
109
|
+
activity: workflow.state.nodes[node.id].activity || null
|
|
110
|
+
})) : [],
|
|
111
|
+
artifacts: run.artifacts,
|
|
112
|
+
decisions,
|
|
113
|
+
transitions: run.history,
|
|
114
|
+
nextActions: workflow
|
|
115
|
+
? workflow.plan.nodes.filter((node) => workflow.state.nodes[node.id].status === 'READY').map((node) => node.id)
|
|
116
|
+
: []
|
|
117
|
+
}, categories);
|
|
118
|
+
const material = {
|
|
119
|
+
schemaVersion: 1,
|
|
120
|
+
audience,
|
|
121
|
+
runId: run.runId,
|
|
122
|
+
generatedAt: now.toISOString(),
|
|
123
|
+
graphHash: run.workflow?.graphHash || null,
|
|
124
|
+
finalState: run.state,
|
|
125
|
+
redacted: true,
|
|
126
|
+
sourceIncluded: false,
|
|
127
|
+
redactionCategories: [...categories].sort(),
|
|
128
|
+
content
|
|
129
|
+
};
|
|
130
|
+
return { ...material, bundleHash: sha256(material) };
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
function handoffBrief(bundle) {
|
|
134
|
+
const lines = [
|
|
135
|
+
'# Engineering Handoff',
|
|
136
|
+
'',
|
|
137
|
+
`Run: ${bundle.runId}`,
|
|
138
|
+
`State: ${bundle.finalState}`,
|
|
139
|
+
`Mode: ${bundle.content.mode}`,
|
|
140
|
+
`Graph: ${bundle.graphHash || 'legacy'}`,
|
|
141
|
+
'',
|
|
142
|
+
'## Objective',
|
|
143
|
+
bundle.content.objective || 'Not recorded.',
|
|
144
|
+
'',
|
|
145
|
+
'## Workflow',
|
|
146
|
+
...bundle.content.nodes.map((node) => `- ${node.id}: ${node.status}${node.reason ? ` — ${node.reason}` : ''}`),
|
|
147
|
+
'',
|
|
148
|
+
'## Decisions',
|
|
149
|
+
...(bundle.content.decisions.length ? bundle.content.decisions.map((decision) => `- ${decision.decision || decision.summary || JSON.stringify(decision)}`) : ['- None recorded.']),
|
|
150
|
+
'',
|
|
151
|
+
'## Evidence',
|
|
152
|
+
...Object.entries(bundle.content.artifacts).map(([name, artifact]) => (
|
|
153
|
+
`- ${name}: ${artifact && typeof artifact === 'object' ? artifact.sha256 || 'recorded' : 'recorded'}`
|
|
154
|
+
)),
|
|
155
|
+
'',
|
|
156
|
+
'## Next actions',
|
|
157
|
+
...(bundle.content.nextActions.length ? bundle.content.nextActions.map((action) => `- ${action}`) : ['- No READY nodes.'])
|
|
158
|
+
];
|
|
159
|
+
return lines.join('\n');
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
function verifyHandoff(bundle) {
|
|
163
|
+
if (!bundle || bundle.schemaVersion !== 1 || !AUDIENCES.has(bundle.audience)) {
|
|
164
|
+
throw handoffError('Handoff manifest is invalid', 'INVALID_HANDOFF');
|
|
165
|
+
}
|
|
166
|
+
const { bundleHash, ...material } = bundle;
|
|
167
|
+
if (bundleHash !== sha256(material)) throw handoffError('Handoff bundle hash does not match', 'HANDOFF_TAMPERED');
|
|
168
|
+
return {
|
|
169
|
+
valid: true,
|
|
170
|
+
runId: bundle.runId,
|
|
171
|
+
audience: bundle.audience,
|
|
172
|
+
bundleHash,
|
|
173
|
+
canonicalBytes: Buffer.byteLength(canonicalJson(bundle))
|
|
174
|
+
};
|
|
175
|
+
}
|
|
176
|
+
|
|
177
|
+
function readHandoff(rootDir, file) {
|
|
178
|
+
const absolute = path.resolve(rootDir, file);
|
|
179
|
+
const relative = path.relative(rootDir, absolute);
|
|
180
|
+
if (relative === '..' || relative.startsWith(`..${path.sep}`)) throw handoffError('Handoff path escapes repository');
|
|
181
|
+
let stat;
|
|
182
|
+
try {
|
|
183
|
+
stat = fs.lstatSync(absolute);
|
|
184
|
+
} catch (cause) {
|
|
185
|
+
throw handoffError(`Unable to inspect handoff: ${cause.message}`, 'INVALID_HANDOFF');
|
|
186
|
+
}
|
|
187
|
+
if (stat.isSymbolicLink() || !stat.isFile() || stat.size > MAX_BUNDLE_BYTES) {
|
|
188
|
+
throw handoffError('Handoff must be a regular file under 1 MiB');
|
|
189
|
+
}
|
|
190
|
+
const realRoot = fs.realpathSync(rootDir);
|
|
191
|
+
const realFile = fs.realpathSync(absolute);
|
|
192
|
+
if (realFile !== realRoot && !realFile.startsWith(`${realRoot}${path.sep}`)) {
|
|
193
|
+
throw handoffError('Handoff path escapes repository through a symlink');
|
|
194
|
+
}
|
|
195
|
+
try {
|
|
196
|
+
return JSON.parse(fs.readFileSync(absolute, 'utf8'));
|
|
197
|
+
} catch (cause) {
|
|
198
|
+
throw handoffError(`Handoff is invalid JSON: ${cause.message}`, 'INVALID_HANDOFF');
|
|
199
|
+
}
|
|
200
|
+
}
|
|
201
|
+
|
|
202
|
+
module.exports = {
|
|
203
|
+
AUDIENCES,
|
|
204
|
+
createHandoff,
|
|
205
|
+
handoffBrief,
|
|
206
|
+
recordDecision,
|
|
207
|
+
verifyHandoff,
|
|
208
|
+
readHandoff
|
|
209
|
+
};
|