faberun 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +193 -31
- package/package.json +1 -1
- package/src/campaign/chain.mjs +19 -0
- package/src/cli.mjs +4 -3
- package/src/contract/final-verification.mjs +41 -10
- package/src/contract/task-packet.mjs +12 -3
- package/src/engine/dispatch.mjs +17 -0
- package/src/engine/lifecycle.mjs +31 -60
- package/src/engine/scheduler.mjs +199 -26
- package/src/engine/settle-judge.mjs +141 -0
- package/src/engine/settle.mjs +1 -1
- package/src/engine/verify.mjs +57 -4
- package/src/harnesses/protocol.mjs +46 -10
- package/src/host/preflight.mjs +19 -0
- package/src/repo/integrate.mjs +6 -1
- package/src/report/render.mjs +58 -58
- package/src/web/index.html +3 -3
- package/src/web/server.mjs +2 -1
package/README.md
CHANGED
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
<h1 align="center">faberun</h1>
|
|
4
4
|
|
|
5
|
-
<p align="center">From intent to running software.</p>
|
|
5
|
+
<p align="center"><strong>From intent to running software.</strong></p>
|
|
6
|
+
|
|
7
|
+
<p align="center">Graph engineering for coding agents.</p>
|
|
6
8
|
|
|
7
9
|
<p align="center">
|
|
8
10
|
<a href="https://github.com/feliperun/faberun/actions/workflows/ci.yml"><img src="https://github.com/feliperun/faberun/actions/workflows/ci.yml/badge.svg" alt="CI status"></a>
|
|
@@ -10,21 +12,162 @@
|
|
|
10
12
|
<a href="https://www.npmjs.com/package/faberun"><img src="https://img.shields.io/npm/v/faberun" alt="npm version"></a>
|
|
11
13
|
</p>
|
|
12
14
|
|
|
13
|
-
Faberun is a
|
|
14
|
-
|
|
15
|
-
dependencies,
|
|
16
|
-
|
|
17
|
-
|
|
15
|
+
Faberun is a graph-engineering runtime for building software with coding agents.
|
|
16
|
+
It sits above the models and the harnesses and turns an implementation plan into
|
|
17
|
+
a durable graph of tasks, dependencies, workers, proofs, reviews, retries and
|
|
18
|
+
integration gates.
|
|
19
|
+
|
|
20
|
+
Harnesses are execution environments: Claude Code, Codex, the Gemini, DeepSeek
|
|
21
|
+
and GLM shells, and a generic `exec-jsonl` door for whatever comes next. Models
|
|
22
|
+
are the engines inside them. Faberun owns the graph, the state and the
|
|
23
|
+
definition of done around both.
|
|
24
|
+
|
|
25
|
+
The goal is not to make a group of agents talk to each other. It is to use the
|
|
26
|
+
right model for each kind of work, keep the process independent of any single
|
|
27
|
+
model or harness, and require evidence before software counts as done.
|
|
28
|
+
|
|
29
|
+
## Why Faberun
|
|
30
|
+
|
|
31
|
+
Coding agents are already very good at changing repositories. The remaining
|
|
32
|
+
problem is everything around that execution.
|
|
33
|
+
|
|
34
|
+
A frontier model earns its cost when it reasons about architecture, challenges a
|
|
35
|
+
design, writes a specification or reviews a difficult decision. The same model
|
|
36
|
+
is usually unnecessary for a small, well-defined implementation task a much
|
|
37
|
+
cheaper one performs well. Without an orchestration layer, though, the model
|
|
38
|
+
driving the current session becomes the model doing everything.
|
|
39
|
+
|
|
40
|
+
Faberun separates those concerns. One campaign can use a frontier model to
|
|
41
|
+
reason about the problem and author a plan, a second strong model to attack that
|
|
42
|
+
plan, lower-cost models to execute well-defined implementation nodes, a model
|
|
43
|
+
from another vendor to judge the work that genuinely needs judgment, and plain
|
|
44
|
+
deterministic commands wherever no model is needed at all. The expensive
|
|
45
|
+
intelligence goes where it matters, and the rest becomes an execution problem.
|
|
46
|
+
|
|
47
|
+
It also takes the campaign out of the lifetime of one chat. The work has durable
|
|
48
|
+
state on disk, so Claude Code can start a campaign and Codex, DeepSeek, GLM or
|
|
49
|
+
any other adapted harness can continue it later without reconstructing the
|
|
50
|
+
process from a conversation history.
|
|
51
|
+
|
|
52
|
+
## Graph engineering
|
|
18
53
|
|
|
19
|
-
Faberun
|
|
20
|
-
|
|
21
|
-
models are engines; Faberun sits above them. It keeps the intent, coordinates
|
|
22
|
-
the work, tracks what was actually completed, validates the result and decides
|
|
23
|
-
what should happen next.
|
|
54
|
+
Faberun treats development as an executable graph rather than a long
|
|
55
|
+
conversation.
|
|
24
56
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
57
|
+
```text
|
|
58
|
+
intent
|
|
59
|
+
└─ campaign
|
|
60
|
+
└─ contract
|
|
61
|
+
├─ node A ── proof ── review ──┐
|
|
62
|
+
├─ node B ── proof ── review ──┼── integration ── promotion
|
|
63
|
+
└─ node C ── proof ── review ──┘
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
A campaign carries the durable intent. A contract turns part of that intent into
|
|
67
|
+
a schema-versioned DAG. Each node carries a closed packet: what the worker may
|
|
68
|
+
read, what it may change, what done means, and how that claim has to be proven.
|
|
69
|
+
The graph decides what can run, what must wait, what may retry, what needs a
|
|
70
|
+
human and what is allowed to advance. The models are replaceable participants
|
|
71
|
+
inside it. The graph remains.
|
|
72
|
+
|
|
73
|
+
## What is different
|
|
74
|
+
|
|
75
|
+
### Use the right model for the job
|
|
76
|
+
|
|
77
|
+
A runtime is one harness running one model. Workers, judges and planning
|
|
78
|
+
sessions need not share a provider or a capability tier, so high-cost reasoning
|
|
79
|
+
and low-cost execution live in the same campaign instead of one model owning the
|
|
80
|
+
whole process. Changing providers is a routing decision, not a rewrite of the
|
|
81
|
+
workflow.
|
|
82
|
+
|
|
83
|
+
### Proof before opinion
|
|
84
|
+
|
|
85
|
+
A worker saying "done" is not evidence that it is. Every definition-of-done item
|
|
86
|
+
declares how it can be proven: a command that must succeed, a path that must
|
|
87
|
+
exist, or — only when the criterion cannot be mechanised — the judgment of a
|
|
88
|
+
judge. Mechanical verification always runs first, and a node whose proofs are
|
|
89
|
+
all mechanical settles without spending a token on review. A gate may also
|
|
90
|
+
declare `skipWhen`, so a change that is green and small enough skips the judge
|
|
91
|
+
by policy rather than by accident.
|
|
92
|
+
|
|
93
|
+
### Independent review
|
|
94
|
+
|
|
95
|
+
When judgment is required, the worker does not grade itself. Validation refuses
|
|
96
|
+
a gated node whose worker and judge resolve to the same vendor. The judge reads
|
|
97
|
+
the recorded result and its evidence, never the reasoning that produced it, and
|
|
98
|
+
never re-runs the work. The point is not to make two agents agree; it is to keep
|
|
99
|
+
implementation and acceptance as separate responsibilities.
|
|
100
|
+
|
|
101
|
+
### Isolated execution
|
|
102
|
+
|
|
103
|
+
Each attempt runs in its own git worktree, so independent nodes run at the same
|
|
104
|
+
time without sharing a mutable working tree, and accepted work reaches the
|
|
105
|
+
repository through controlled refs instead of being copied into the operator's
|
|
106
|
+
checkout. The operator keeps working while the campaign runs.
|
|
107
|
+
|
|
108
|
+
### Durable campaigns
|
|
109
|
+
|
|
110
|
+
A run is not tied to the terminal that launched it. The controller runs
|
|
111
|
+
detached, state is persisted under `.runs/`, `supervise` resumes a controller
|
|
112
|
+
that died, and a campaign carries its intent across many runs and sessions. If a
|
|
113
|
+
provider goes down or a harness session ends, the work does not have to be
|
|
114
|
+
re-authored.
|
|
115
|
+
|
|
116
|
+
### Safe integration
|
|
117
|
+
|
|
118
|
+
Passing a worker's own checks is not the end of the process. An accepted attempt
|
|
119
|
+
is sealed, integrated onto the run ref and verified again in the candidate
|
|
120
|
+
state, and the phase-terminal node runs the contract's final verification. If
|
|
121
|
+
combining individually valid changes breaks the system, the campaign does not
|
|
122
|
+
promote them.
|
|
123
|
+
|
|
124
|
+
### Measurable model economics
|
|
125
|
+
|
|
126
|
+
Every invocation records its usage, routing and outcome, and a closed campaign
|
|
127
|
+
keeps that ledger in its own record under `docs/campaigns/<id>/ledger/`. It makes
|
|
128
|
+
questions like these measurable instead of anecdotal: which model handles this
|
|
129
|
+
class of task reliably, where a frontier model is actually worth its price,
|
|
130
|
+
which nodes close mechanically with no judge at all, how many revisions a worker
|
|
131
|
+
needs before acceptance, what one closed checkpoint costs, and how often a
|
|
132
|
+
fallback saves a campaign. The objective is not to spend less. It is to know
|
|
133
|
+
where expensive intelligence creates value.
|
|
134
|
+
|
|
135
|
+
## Campaigns, not synthetic teams
|
|
136
|
+
|
|
137
|
+
Faberun does not model an agent process as a human organisation. It needs no "AI
|
|
138
|
+
architect", "AI developer", "AI QA engineer" and "AI product manager" merely
|
|
139
|
+
because human teams carry those titles: models are general enough that those
|
|
140
|
+
boundaries are mostly artificial. The roles it does define are the ones that
|
|
141
|
+
stay useful for agents.
|
|
142
|
+
|
|
143
|
+
| role | responsibility |
|
|
144
|
+
| --- | --- |
|
|
145
|
+
| worker | produces a result inside its packet. |
|
|
146
|
+
| judge | decides independently whether a result that needs judgment satisfies its contract. |
|
|
147
|
+
| controller | advances the graph deterministically. |
|
|
148
|
+
| operator | supplies the intent and intervenes when a decision genuinely needs a human. |
|
|
149
|
+
|
|
150
|
+
Everything else belongs in the contract.
|
|
151
|
+
|
|
152
|
+
## From spec to contract
|
|
153
|
+
|
|
154
|
+
Planning is part of the product, and it is adversarial on purpose. `faberun spec
|
|
155
|
+
validate` checks a specification's front matter, its `R<n>` requirements and the
|
|
156
|
+
proof each one declares, and invokes no model to do it. `faberun plan` then runs
|
|
157
|
+
the debate in budgeted, isolated runs: an author drafts, a reviewer from another
|
|
158
|
+
vendor attacks the draft holding only the spec, the repository facts and the
|
|
159
|
+
plan under review, and a revision round follows while a critical finding
|
|
160
|
+
remains. A converged draft is sized, routed and frozen into a contract. Freezing
|
|
161
|
+
never launches, and a plan that never converges ends `contested`, with its open
|
|
162
|
+
findings recorded as a campaign question instead of a contract.
|
|
163
|
+
|
|
164
|
+
None of it is mandatory. The traceability rules are advisory unless you ask for
|
|
165
|
+
`--strict-traceability`, a document with no front matter validates as `legacy`
|
|
166
|
+
rather than being rejected, and you can skip planning altogether and author the
|
|
167
|
+
contract by hand. What Faberun needs is the executable part of the plan:
|
|
168
|
+
objective, nodes, dependencies, read and write scope, definitions of done,
|
|
169
|
+
verification, runtime policy and integration behaviour. Your specification
|
|
170
|
+
process stays yours.
|
|
28
171
|
|
|
29
172
|
## Install
|
|
30
173
|
|
|
@@ -92,29 +235,35 @@ The operator writes the intent into a campaign and an authored contract, a
|
|
|
92
235
|
schema-versioned DAG whose nodes each carry a closed task packet. The controller
|
|
93
236
|
schedules every dependency-ready node and dispatches its packet to a worker
|
|
94
237
|
inside an attempt worktree, where the worker sees only the files the packet
|
|
95
|
-
names.
|
|
96
|
-
|
|
97
|
-
A passing attempt is sealed and integrated onto the run ref, a campaign promotes
|
|
98
|
-
each run onto its landing branch, and the orchestrator lands that branch.
|
|
99
|
-
Campaigns, handoffs and the `supervise` watchdog carry the work across sessions,
|
|
100
|
-
so an interrupted run is continued in place instead of being re-authored.
|
|
238
|
+
names. Faberun then judges the result in stages, and a free slot dispatches the
|
|
239
|
+
next node while another one is still being verified.
|
|
101
240
|
|
|
102
241
|
```text
|
|
103
|
-
|
|
104
|
-
└─
|
|
105
|
-
└─
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
242
|
+
packet
|
|
243
|
+
└─ worker attempt in its own worktree
|
|
244
|
+
└─ mechanical proof ── failure ─→ retry · route · attention
|
|
245
|
+
└─ judge, when required ── rejection ─→ revision
|
|
246
|
+
└─ seal ─→ integration ref ─→ candidate verification
|
|
247
|
+
└─ promotion onto the campaign's landing branch
|
|
109
248
|
```
|
|
110
249
|
|
|
111
|
-
|
|
112
|
-
|
|
250
|
+
A judge reviews recorded evidence rather than re-running arbitrary work. A
|
|
251
|
+
passing attempt is sealed and integrated onto the run ref, a campaign promotes
|
|
252
|
+
each finished run onto its landing branch, and the orchestrator lands that
|
|
253
|
+
branch. Campaigns, handoffs and the `supervise` watchdog carry the work across
|
|
254
|
+
sessions, so an interrupted run is continued in place instead of being
|
|
255
|
+
re-authored.
|
|
113
256
|
|
|
114
|
-
|
|
257
|
+
The vocabulary is in [Concepts](docs/CONCEPTS.md), and the layers, process model
|
|
258
|
+
and gates are in [Architecture](docs/ARCHITECTURE.md).
|
|
115
259
|
|
|
116
|
-
|
|
117
|
-
|
|
260
|
+
## Harnesses and models
|
|
261
|
+
|
|
262
|
+
Faberun keeps three things apart. A **model** is the reasoning engine, a
|
|
263
|
+
**harness** is the environment that gives a model access to code and tools, and
|
|
264
|
+
a **runtime** is one configured harness-and-model pair Faberun can dispatch. Any
|
|
265
|
+
runtime can act as a worker or as a judge, according to the contract and the
|
|
266
|
+
routing policy.
|
|
118
267
|
|
|
119
268
|
| Harness | Default vendor | Example model |
|
|
120
269
|
| --- | --- | --- |
|
|
@@ -126,6 +275,19 @@ judge of another vendor reviews what it produced.
|
|
|
126
275
|
| `exec-jsonl` | declared per runtime | the model its command names |
|
|
127
276
|
| `replay` | declared per runtime | the recorded model |
|
|
128
277
|
|
|
278
|
+
Harness and model support are adapter concerns. The orchestration above them
|
|
279
|
+
depends on no single vendor.
|
|
280
|
+
|
|
281
|
+
## Philosophy
|
|
282
|
+
|
|
283
|
+
Software should be built, not merely generated. Generating plausible code is
|
|
284
|
+
becoming cheap; the valuable part is the system around the generation —
|
|
285
|
+
preserving intent, decomposing work, choosing the appropriate intelligence,
|
|
286
|
+
isolating effects, proving outcomes, recovering from failure, and recording why a
|
|
287
|
+
change was accepted. A craftsman does not depend on one hammer, and Faberun does
|
|
288
|
+
not depend on one model. Models change, harnesses change, providers change. The
|
|
289
|
+
intent, the graph, the evidence and the finished work remain.
|
|
290
|
+
|
|
129
291
|
## Documentation
|
|
130
292
|
|
|
131
293
|
- [Documentation map](docs/README.md): every document and the question it answers.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "faberun",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"description": "Faberun is a development orchestration system that turns intent into verified software: harness- and model-agnostic, it keeps the intent, coordinates the work, verifies the result and decides what happens next.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
package/src/campaign/chain.mjs
CHANGED
|
@@ -290,6 +290,25 @@ export function validateContractAgainstRef(raw, contractPath, context = {}) {
|
|
|
290
290
|
}
|
|
291
291
|
}
|
|
292
292
|
|
|
293
|
+
/**
|
|
294
|
+
* Validate a contract for a launch, the one path the CLI's `run` command and
|
|
295
|
+
* the scheduler's own re-validation both go through: against `baseRef` when
|
|
296
|
+
* one is given, the checkout otherwise. `repo` defaults to the contract's own
|
|
297
|
+
* `cwd` when the caller does not know a better one (the campaign chain always
|
|
298
|
+
* passes its own), since a contract's `cwd` is always inside the repo it
|
|
299
|
+
* names.
|
|
300
|
+
*
|
|
301
|
+
* @param {Record<string, unknown>} raw
|
|
302
|
+
* @param {string} contractPath
|
|
303
|
+
* @param {{repo?: string, baseRef?: string}} [context]
|
|
304
|
+
* @returns {ValidatedContract}
|
|
305
|
+
*/
|
|
306
|
+
export function validateContractForLaunch(raw, contractPath, context = {}) {
|
|
307
|
+
if (!context.baseRef) return validateContract(raw, contractPath);
|
|
308
|
+
const repo = context.repo ?? resolve(dirname(contractPath), typeof raw.cwd === "string" ? raw.cwd : ".");
|
|
309
|
+
return validateContractAgainstRef(raw, contractPath, { repo, baseRef: context.baseRef });
|
|
310
|
+
}
|
|
311
|
+
|
|
293
312
|
/**
|
|
294
313
|
* Reduce a run's progress to the one decision the chain makes. An in-flight run
|
|
295
314
|
* is `unfinished` even though `reduceRunOutcome` already names its running nodes
|
package/src/cli.mjs
CHANGED
|
@@ -24,6 +24,7 @@ import {
|
|
|
24
24
|
validBootstrapNonce,
|
|
25
25
|
} from "./run/lock.mjs";
|
|
26
26
|
import { renderRunHandoff } from "./campaign/index.mjs";
|
|
27
|
+
import { validateContractForLaunch } from "./campaign/chain.mjs";
|
|
27
28
|
import { campaignCli } from "./cli/campaign.mjs";
|
|
28
29
|
import { seatCli } from "./cli/seat.mjs";
|
|
29
30
|
import { initCommand } from "./cli/init.mjs";
|
|
@@ -291,9 +292,9 @@ async function main(argv) {
|
|
|
291
292
|
if (command === "run") {
|
|
292
293
|
warnIfNoTransport();
|
|
293
294
|
const absolute = resolve(target);
|
|
294
|
-
const contract = validateContract(JSON.parse(readFileSync(absolute, "utf8")), absolute);
|
|
295
|
-
const runDir = join(contract.cwd, ".runs", contract.id);
|
|
296
295
|
const baseRef = typeof values["base-ref"] === "string" && values["base-ref"] ? values["base-ref"] : undefined;
|
|
296
|
+
const contract = validateContractForLaunch(JSON.parse(readFileSync(absolute, "utf8")), absolute, { baseRef });
|
|
297
|
+
const runDir = join(contract.cwd, ".runs", contract.id);
|
|
297
298
|
setLaunchBaseRef(baseRef);
|
|
298
299
|
// The base is what every worktree is cut from; a dirty tree only blocks
|
|
299
300
|
// when the cwd HEAD *is* that base. A `--base-ref` elsewhere leaves the
|
|
@@ -314,7 +315,7 @@ async function main(argv) {
|
|
|
314
315
|
return;
|
|
315
316
|
}
|
|
316
317
|
for (const warning of [...contract.warnings, ...reusedDoneWarnings(contract)]) process.stdout.write(`${advisoryToken()} ${warning}\n`);
|
|
317
|
-
const result = await runContract(target, { detachedBootstrap: hasDetachedBootstrapNonce() });
|
|
318
|
+
const result = await runContract(target, { detachedBootstrap: hasDetachedBootstrapNonce(), baseRef });
|
|
318
319
|
if (!result.ok) process.exitCode = 1;
|
|
319
320
|
return;
|
|
320
321
|
}
|
|
@@ -3,11 +3,13 @@
|
|
|
3
3
|
* rule 12).
|
|
4
4
|
*
|
|
5
5
|
* `contract.finalVerification` is the contract-wide proof that the phase as a
|
|
6
|
-
* whole closes: the controller runs it before the judge on
|
|
7
|
-
* node (
|
|
8
|
-
* approved on partial verification
|
|
9
|
-
*
|
|
10
|
-
*
|
|
6
|
+
* whole closes: the controller runs it before the judge, once, on whichever
|
|
7
|
+
* phase-terminal node (a node no other node depends on) is the last of them
|
|
8
|
+
* to settle, so no final checkpoint is ever approved on partial verification
|
|
9
|
+
* and a flake on one phase-terminal node cannot also cost its siblings a
|
|
10
|
+
* revision. `contract.sharedVerification` carries the same command schema but
|
|
11
|
+
* is appended to every node's attempt and integration candidate, for the fast
|
|
12
|
+
* repository ratchets a node's write set can break.
|
|
11
13
|
* The persisted node-snapshot shape for verification evidence lives here too,
|
|
12
14
|
* next to the schema it records.
|
|
13
15
|
*/
|
|
@@ -58,20 +60,49 @@ export function sharedVerificationCommands(contract) {
|
|
|
58
60
|
return contract.sharedVerification ?? [];
|
|
59
61
|
}
|
|
60
62
|
|
|
63
|
+
/**
|
|
64
|
+
* Whether no other node in the contract depends on this one. `finalVerification`
|
|
65
|
+
* is a candidate to run on a phase-terminal node only; a node with a dependant
|
|
66
|
+
* never carries it, no matter which of its siblings settles last.
|
|
67
|
+
*
|
|
68
|
+
* @param {{nodes: {id: string, dependsOn: string[]}[]}} contract
|
|
69
|
+
* @param {{id: string}} node
|
|
70
|
+
* @returns {boolean}
|
|
71
|
+
*/
|
|
72
|
+
export function phaseTerminalNode(contract, node) {
|
|
73
|
+
return !contract.nodes.some((candidate) => candidate.dependsOn.includes(node.id));
|
|
74
|
+
}
|
|
75
|
+
|
|
61
76
|
/**
|
|
62
77
|
* The contract's `finalVerification` commands when this node is the one that
|
|
63
|
-
* closes the phase, otherwise none. A
|
|
64
|
-
*
|
|
78
|
+
* closes the phase, otherwise none. A phase can have several phase-terminal
|
|
79
|
+
* nodes (several nodes no other node depends on); the suite runs once for the
|
|
80
|
+
* phase, on whichever of them is the last to settle, not on every one of them.
|
|
81
|
+
*
|
|
82
|
+
* `settledIds` is the set of sibling node ids (any status a node does not
|
|
83
|
+
* leave on its own -- see `SETTLED` in `engine/prompts.mjs`) already reached
|
|
84
|
+
* at the moment this call is made. A phase-terminal node carries the suite
|
|
85
|
+
* exactly when every *other* phase-terminal node is already in that set --
|
|
86
|
+
* whichever node that is true for, in whatever order nodes actually settle,
|
|
87
|
+
* including a node retried after every sibling has already closed. Omitting
|
|
88
|
+
* `settledIds` (the scheduler's budget estimate, which has no run to inspect)
|
|
89
|
+
* falls back to the old per-node-shape answer: any phase-terminal node may
|
|
90
|
+
* still turn out to be the one that carries it, so the estimate stays an
|
|
91
|
+
* upper bound.
|
|
65
92
|
*
|
|
66
93
|
* @param {{finalVerification?: VerificationCommand[], nodes: {id: string, dependsOn: string[]}[]}} contract
|
|
67
94
|
* @param {{id: string}} node
|
|
95
|
+
* @param {Set<string>} [settledIds]
|
|
68
96
|
* @returns {VerificationCommand[]}
|
|
69
97
|
*/
|
|
70
|
-
export function finalVerificationCommands(contract, node) {
|
|
98
|
+
export function finalVerificationCommands(contract, node, settledIds) {
|
|
71
99
|
const commands = contract.finalVerification ?? [];
|
|
72
100
|
if (commands.length === 0) return [];
|
|
73
|
-
|
|
74
|
-
|
|
101
|
+
if (!phaseTerminalNode(contract, node)) return [];
|
|
102
|
+
if (!settledIds) return commands;
|
|
103
|
+
const siblings = contract.nodes.filter((candidate) => candidate.id !== node.id && phaseTerminalNode(contract, candidate));
|
|
104
|
+
const isLastToClose = siblings.every((sibling) => settledIds.has(sibling.id));
|
|
105
|
+
return isLastToClose ? commands : [];
|
|
75
106
|
}
|
|
76
107
|
|
|
77
108
|
/**
|
|
@@ -27,6 +27,15 @@ const PROMPT_MAX_BYTES = 64 * 1024;
|
|
|
27
27
|
*/
|
|
28
28
|
const VERIFICATION_PARAGRAPH = "The controller runs every command below after you report; its recorded results are the proof of this node. Running a command yourself is optional and only for one that finishes in seconds and spawns no long-lived process. Keep output bounded (pipe through `| tail -n 200`). Never wait on a background job, never run the whole test suite, and never run tests that start and terminate other processes.";
|
|
29
29
|
|
|
30
|
+
/**
|
|
31
|
+
* The `## Required output` schema every mode states: literally every key
|
|
32
|
+
* `validateWorkerResult` requires, so a worker never loses a valid result to
|
|
33
|
+
* a field this paragraph failed to name. What belongs in `artifacts` (if
|
|
34
|
+
* anything) is this packet's own content contract, stated in its own
|
|
35
|
+
* `instructions`, not repeated or guessed here.
|
|
36
|
+
*/
|
|
37
|
+
const REQUIRED_OUTPUT_SCHEMA = 'Return exactly one JSON object, with no markdown or prose, matching this shape: {"status":"done"|"blocked_context","summary":"string","verification":["string"],"artifacts":["string"],"missingContext":["string"]}. status is "done" when the node is complete or "blocked_context" when required context is missing; summary is prose for a human reader; verification and artifacts are string arrays, sent as [] when this node ran no command or has nothing to put there; missingContext lists exactly what is absent, non-empty only for blocked_context. Use blocked_context only when missingContext is non-empty; use done only when missingContext is empty.';
|
|
38
|
+
|
|
30
39
|
/**
|
|
31
40
|
* `validateRelativePath`'s answer when a path is absent and the caller asked to
|
|
32
41
|
* defer the missing-path verdict rather than throw it. Only contract loading
|
|
@@ -128,7 +137,7 @@ export function renderWorkerPrompt(packet, nodeId) {
|
|
|
128
137
|
...packet.verification.map((command) => `- ${command.argv.join(" ")}`),
|
|
129
138
|
"",
|
|
130
139
|
"## Required output",
|
|
131
|
-
|
|
140
|
+
REQUIRED_OUTPUT_SCHEMA,
|
|
132
141
|
];
|
|
133
142
|
const prompt = `${lines.join("\n")}\n`;
|
|
134
143
|
if (Buffer.byteLength(prompt, "utf8") > PROMPT_MAX_BYTES) {
|
|
@@ -432,7 +441,7 @@ function renderDiscoveryPrompt(packet, nodeId) {
|
|
|
432
441
|
...bulletOrNone(packet.nonGoals),
|
|
433
442
|
"",
|
|
434
443
|
"## Required output",
|
|
435
|
-
|
|
444
|
+
`${REQUIRED_OUTPUT_SCHEMA} Put structured findings meant to inform this packet's own instructions in \`output\` instead (a JSON object, at most 65536 bytes).`,
|
|
436
445
|
"",
|
|
437
446
|
"## Verification",
|
|
438
447
|
VERIFICATION_PARAGRAPH,
|
|
@@ -481,7 +490,7 @@ function renderAutonomousPrompt(packet, nodeId) {
|
|
|
481
490
|
...packet.verification.map((command) => `- ${command.argv.join(" ")}`),
|
|
482
491
|
"",
|
|
483
492
|
"## Required output",
|
|
484
|
-
|
|
493
|
+
REQUIRED_OUTPUT_SCHEMA,
|
|
485
494
|
];
|
|
486
495
|
const prompt = `${lines.join("\n")}\n`;
|
|
487
496
|
if (Buffer.byteLength(prompt, "utf8") > PROMPT_MAX_BYTES) throw new TypeError(`worker prompt exceeds ${PROMPT_MAX_BYTES} bytes`);
|
package/src/engine/dispatch.mjs
CHANGED
|
@@ -668,6 +668,19 @@ export async function startJudge(contract, node, state, runDir, running, workerR
|
|
|
668
668
|
error.code = "judge_prompt_too_large";
|
|
669
669
|
throw error;
|
|
670
670
|
}
|
|
671
|
+
// Captured at the last possible instant before the judge can touch
|
|
672
|
+
// anything, so the settlement pass's comparison proves what the judge
|
|
673
|
+
// itself wrote rather than racing whatever ran just before dispatch. A
|
|
674
|
+
// capture failure must not block dispatch -- the write check is a
|
|
675
|
+
// controller invariant on top of whatever the judge does, not a
|
|
676
|
+
// precondition for running it -- so it degrades to unchecked instead of
|
|
677
|
+
// to a refusal.
|
|
678
|
+
let judgeBaseline;
|
|
679
|
+
try {
|
|
680
|
+
judgeBaseline = captureWorkspaceSnapshot(workspace);
|
|
681
|
+
} catch {
|
|
682
|
+
judgeBaseline = null;
|
|
683
|
+
}
|
|
671
684
|
const job = startProcess({
|
|
672
685
|
contract, node, state, runtime, workspace,
|
|
673
686
|
prompt: phasePlan.prompt,
|
|
@@ -678,6 +691,10 @@ export async function startJudge(contract, node, state, runDir, running, workerR
|
|
|
678
691
|
}),
|
|
679
692
|
onInvocation: (invocation, currentJob) => {
|
|
680
693
|
stampInvocation(invocation, contract, node, runtime, state, runDir, "judge", phasePlan.mode, phasePlan.continuationId);
|
|
694
|
+
// Reusing the worker phase's own scratch field: a job is never both a
|
|
695
|
+
// worker and a judge, and this field carries no persisted shape of
|
|
696
|
+
// its own that a judge borrowing it would have to match.
|
|
697
|
+
currentJob.scopeBaseline = judgeBaseline;
|
|
681
698
|
persistInvocation(runDir, state, invocation, currentJob, lock);
|
|
682
699
|
persistInvocationIntent(runDir, invocation, {
|
|
683
700
|
nodeId: node.id,
|
package/src/engine/lifecycle.mjs
CHANGED
|
@@ -17,17 +17,8 @@ import {
|
|
|
17
17
|
SETTLED,
|
|
18
18
|
} from "./prompts.mjs";
|
|
19
19
|
import {
|
|
20
|
-
judgeReaskOutstanding,
|
|
21
|
-
} from "./judge-gate.mjs";
|
|
22
|
-
import {
|
|
23
|
-
judgeVerdictEvidence,
|
|
24
|
-
} from "../contract/review-modes.mjs";
|
|
25
|
-
import {
|
|
26
|
-
JUDGE_MAX_FAILURES,
|
|
27
|
-
applyJudgeProtocolFailure,
|
|
28
20
|
applyJudgeResult,
|
|
29
21
|
applyJudgeRound,
|
|
30
|
-
settleUnavailableJudge,
|
|
31
22
|
} from "./review.mjs";
|
|
32
23
|
import { routeRuntimeForState, routingBackoffActive, runtimeSnapshot } from "./failover.mjs";
|
|
33
24
|
import {
|
|
@@ -65,6 +56,7 @@ import { startJudge, startResultMaterialization } from "./dispatch.mjs";
|
|
|
65
56
|
import { raiseNodeAttention, settleDone } from "./settle.mjs";
|
|
66
57
|
import { applyRejection, applyVerificationFailure } from "./settle.mjs";
|
|
67
58
|
import { emitNodeAdvisories } from "./notify-queue.mjs";
|
|
59
|
+
import { judgeWorkspaceWriteViolation, settleJudgeRound } from "./settle-judge.mjs";
|
|
68
60
|
|
|
69
61
|
/** @typedef {import("../repo/integrate.mjs").IntegrationResult} IntegrationResult */
|
|
70
62
|
/** @typedef {import("./backoff.mjs").Transition} Transition */
|
|
@@ -80,7 +72,6 @@ import { emitNodeAdvisories } from "./notify-queue.mjs";
|
|
|
80
72
|
/** @typedef {import("../contract/index.mjs").GateResult} GateResult */
|
|
81
73
|
/** @typedef {import("../contract/index.mjs").SnapshotError} SnapshotError */
|
|
82
74
|
/** @typedef {import("../contract/index.mjs").BoundedScope} BoundedScope */
|
|
83
|
-
/** @typedef {import("../repo/workspace.mjs").WorkspaceSnapshot} WorkspaceSnapshot */
|
|
84
75
|
/** @typedef {import("../run/lock.mjs").LockRecord} LockRecord */
|
|
85
76
|
/** @typedef {ReturnType<typeof acquireLock>} LockHandle */
|
|
86
77
|
/** @typedef {import("../harnesses/index.mjs").HarnessRuntime} HarnessRuntime */
|
|
@@ -90,7 +81,6 @@ import { emitNodeAdvisories } from "./notify-queue.mjs";
|
|
|
90
81
|
/** @typedef {import("../contract/verification.mjs").VerificationAttempt} VerificationAttempt */
|
|
91
82
|
/** @typedef {import("../contract/verification.mjs").VerificationAttemptResult} VerificationAttemptResult */
|
|
92
83
|
/** @typedef {import("../contract/verification.mjs").VerificationResult} VerificationResult */
|
|
93
|
-
/** @typedef {import("../repo/workspace.mjs").ScopeComparison} ScopeComparison */
|
|
94
84
|
/** @typedef {import("../contract/worker-result.mjs").WorkerResult} WorkerResult */
|
|
95
85
|
/** @typedef {import("../campaign/index.mjs").Campaign} Campaign */
|
|
96
86
|
/** @typedef {{path: string, campaign: Campaign}} CampaignRef */
|
|
@@ -277,7 +267,7 @@ export function autoRetryParkedNodes(contract, runDir, states, lock, previouslyP
|
|
|
277
267
|
*
|
|
278
268
|
* @param {NodeSnapshot} state
|
|
279
269
|
*/
|
|
280
|
-
function clearTierExhaustion(state) {
|
|
270
|
+
export function clearTierExhaustion(state) {
|
|
281
271
|
if (!state.routing || state.routing.tierExhaustion === undefined) return;
|
|
282
272
|
const routing = { ...state.routing };
|
|
283
273
|
delete routing.tierExhaustion;
|
|
@@ -393,6 +383,25 @@ export async function finalizeClosedJobs(contract, runDir, states, running, lock
|
|
|
393
383
|
appendUsageRecord(runDir, state.invocations.find((invocation) => invocation.id === job.invocation.id));
|
|
394
384
|
state.usage = invocationUsage(state);
|
|
395
385
|
state.costUsd = invocationCost(state);
|
|
386
|
+
// Checked before any other branch can act on how this invocation closed --
|
|
387
|
+
// a done verdict, a provider failure, a bounded re-dispatch, or exhaustion
|
|
388
|
+
// -- so a judge that wrote into its own workspace is caught here rather
|
|
389
|
+
// than laundered through a re-dispatch whose fresh baseline would already
|
|
390
|
+
// contain the write (the whole point of TECH-SPEC's write check: blocked
|
|
391
|
+
// outright, never re-asked). A comparison that cannot even be completed --
|
|
392
|
+
// an edited ignore source, a symlink escaping the tree, too many entries --
|
|
393
|
+
// is the same violation as an ordinary write, never silence: the worker
|
|
394
|
+
// path (`engine/scope.mjs`'s `checkWorkerScope`) already fails closed on
|
|
395
|
+
// exactly the same throw.
|
|
396
|
+
if (job.phase === "judge") {
|
|
397
|
+
const violation = judgeWorkspaceWriteViolation(job);
|
|
398
|
+
if (violation) {
|
|
399
|
+
clearTierExhaustion(state);
|
|
400
|
+
transition(runDir, state, "blocked", { phase: "judge", result: state.result, usage: state.usage, error: { code: "judge_protocol", message: violation.message } }, lock);
|
|
401
|
+
await raiseNodeAttention(campaignPath, runDir, state, "judge_protocol");
|
|
402
|
+
continue;
|
|
403
|
+
}
|
|
404
|
+
}
|
|
396
405
|
// A closed worker whose canonical result file is valid and whose scope
|
|
397
406
|
// passed is completed work, no matter what the provider envelope or the
|
|
398
407
|
// exit code said. The durable file was read above, before the scope gate,
|
|
@@ -425,55 +434,17 @@ export async function finalizeClosedJobs(contract, runDir, states, running, lock
|
|
|
425
434
|
continue;
|
|
426
435
|
}
|
|
427
436
|
// A judge provider that failed outright (its turn died, its tool host was
|
|
428
|
-
// gone) is a provider failure, never a verdict
|
|
429
|
-
//
|
|
430
|
-
//
|
|
431
|
-
//
|
|
432
|
-
//
|
|
433
|
-
//
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
// warmed, and spends none of the one re-dispatch counted below.
|
|
438
|
-
const network = networkTransition(contract, job.node, state, "judge", envelope, job.exitCode);
|
|
439
|
-
if (network && handleProviderExhaustion(contract, runDir, job.node, state, "judge", envelope, job.runtime.id, lock, states, campaignPath, network)) continue;
|
|
440
|
-
clearTierExhaustion(state);
|
|
441
|
-
// The provider died on the bounded re-ask itself, so the one permitted
|
|
442
|
-
// re-ask is spent: settle by review mode here rather than dispatch a
|
|
443
|
-
// third judge invocation behind a fresh failure count.
|
|
444
|
-
// The provider died on the bounded re-ask itself, so the one permitted
|
|
445
|
-
// re-ask is spent: settle by review mode here rather than dispatch a
|
|
446
|
-
// third judge invocation behind a fresh failure count.
|
|
447
|
-
if (judgeReaskOutstanding(state)) {
|
|
448
|
-
await applyJudgeProtocolFailure(contract, job.node, state, runDir, running, lock, states, campaignPath, envelope.error?.message ?? "judge provider failed");
|
|
449
|
-
continue;
|
|
450
|
-
}
|
|
451
|
-
state.judgeFailures = (state.judgeFailures ?? 0) + 1;
|
|
452
|
-
if (state.judgeFailures < JUDGE_MAX_FAILURES) {
|
|
453
|
-
writeNode(runDir, state, lock);
|
|
454
|
-
await applyJudgeRound(await startJudge(contract, job.node, state, runDir, running, state.result, lock, states, campaignPath),
|
|
455
|
-
contract, job.node, state, runDir, running, lock, states, campaignPath, state.result);
|
|
456
|
-
continue;
|
|
457
|
-
}
|
|
458
|
-
await settleUnavailableJudge(contract, job.node, state, runDir, lock, states, campaignPath, envelope.error?.message ?? "judge provider failed");
|
|
459
|
-
continue;
|
|
460
|
-
} // Whatever else this invocation produced, it is not exactly one usable
|
|
461
|
-
// verdict: no verdict at all, several of them in separate agent messages,
|
|
462
|
-
// an unparseable one, a stream cut off before its terminal envelope, or a
|
|
463
|
-
// phase killed on its wall clock. One bounded re-ask, then the review mode
|
|
464
|
-
// decides — advisory completes, blocking enters attention with the work
|
|
465
|
-
// preserved so a retry in place can re-judge it.
|
|
437
|
+
// gone) is a provider failure, never a verdict, and a verdict that arrived
|
|
438
|
+
// but is not exactly one usable one (none at all, several of them, an
|
|
439
|
+
// unparseable one, a stream cut off before its terminal envelope, or a
|
|
440
|
+
// phase killed on its wall clock) is a protocol defect rather than a
|
|
441
|
+
// pass. Both are settled by `settleJudgeRound`, moved out of this file
|
|
442
|
+
// for the same reason `engine/settle.mjs` was: this file and
|
|
443
|
+
// `test/engine/judge.test.mjs` both sit on the 800-line ceiling
|
|
444
|
+
// `test/repo/source-shape.test.mjs` enforces. The write check above has
|
|
445
|
+
// already run, so nothing here can adopt or launder a judge's own write.
|
|
466
446
|
if (job.phase === "judge") {
|
|
467
|
-
|
|
468
|
-
if (!evidence.ok) {
|
|
469
|
-
const network = networkTransition(contract, job.node, state, "judge", envelope, job.exitCode);
|
|
470
|
-
if (network && handleProviderExhaustion(contract, runDir, job.node, state, "judge", envelope, job.runtime.id, lock, states, campaignPath, network)) continue;
|
|
471
|
-
clearTierExhaustion(state);
|
|
472
|
-
await applyJudgeProtocolFailure(contract, job.node, state, runDir, running, lock, states, campaignPath, evidence.reason);
|
|
473
|
-
continue;
|
|
474
|
-
}
|
|
475
|
-
clearTierExhaustion(state);
|
|
476
|
-
await applyJudgeResult(contract, job.node, state, evidence.result, runDir, lock, running, states, campaignPath);
|
|
447
|
+
await settleJudgeRound(contract, job, state, runDir, running, lock, states, campaignPath, envelope, { clearTierExhaustion, handleProviderExhaustion });
|
|
477
448
|
continue;
|
|
478
449
|
}
|
|
479
450
|
// An empty final message is a missing worker result, not a no-op worker:
|