bare-agent 0.29.0 → 0.31.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/bareagent.context.md +9 -3
- package/package.json +1 -1
- package/src/loop.js +5 -0
- package/src/recurse.d.ts +125 -11
- package/src/recurse.js +454 -44
- package/src/refine.d.ts +23 -7
- package/src/refine.js +11 -5
package/README.md
CHANGED
|
@@ -94,7 +94,7 @@ Every piece works alone — take what you need, ignore the rest. Two axes: **Act
|
|
|
94
94
|
|
|
95
95
|
### Recurse — break a hard task into a tree *(the RLM primitive)*
|
|
96
96
|
|
|
97
|
-
`recurse(task, ctx, opts)` does **decompose → fan-out → verify → synthesize** in one call — Recursive Language Models as a single import, composed *around* the Loop (never a new engine). The default is **model-driven**: the worker is handed a `spawn_child` tool and decides whether to split, bounded by depth + bareguard (no second guard layer). Forced fan-out (`count` / `mode:'fanout'`) and data-driven width (`mode:'partition'`, measured from a corpus) are opt-in. Give workers a stance with `opts.persona` (prepended to every worker, carries down the tree, deliberately kept out of the isolated verifier), and tell them *where they are* with `opts.context` (a read-only paths/cwd blob threaded to every worker so a sliced child can locate its artifact — facts, not a stance). For a leaf that should self-correct, pass `opts.refineLeaf` (opt-in): a definite leaf becomes a bounded generate→sense→regenerate loop driven by *your* deterministic sensor (test/compile/lint), feeding the gap back (with escalating temperature on models that accept it; on a temperature-fixed model like `claude-sonnet-5` the gap critique carries recovery, and the receipt records the effective temps). The headline guarantee: **aggregation is code, never a model-stated number**, and a dead worker or exhausted guard returns an honest `{ incomplete, missingSlices }` — never a faked pass.
|
|
97
|
+
`recurse(task, ctx, opts)` does **decompose → fan-out → verify → synthesize** in one call — Recursive Language Models as a single import, composed *around* the Loop (never a new engine). The default is **model-driven**: the worker is handed a `spawn_child` tool and decides whether to split, bounded by depth + bareguard (no second guard layer). Forced fan-out (`count` / `mode:'fanout'`) and data-driven width (`mode:'partition'`, measured from a corpus) are opt-in. Give workers a stance with `opts.persona` (prepended to every worker, carries down the tree, deliberately kept out of the isolated verifier), and tell them *where they are* with `opts.context` (a read-only paths/cwd blob threaded to every worker so a sliced child can locate its artifact — facts, not a stance). For a leaf that should self-correct, pass `opts.refineLeaf` (opt-in): a definite leaf becomes a bounded generate→sense→regenerate loop driven by *your* deterministic sensor (test/compile/lint — one that judges the *returned* result, never a worker side-effect it could game), feeding the gap back (with escalating temperature on models that accept it; on a temperature-fixed model like `claude-sonnet-5` the gap critique carries recovery, and the receipt records the effective temps). On a temperature-fixed model, `refineLeaf.rejectedBuffer` adds a second lever — it feeds the model's own prior *failed attempts* back verbatim ("write something structurally different"), the directed-diversity complement to temperature's random diversity (adaptive by default; the two are antagonistic, so it holds temperature flat when it engages). A *broken* arbiter — a sensor or a caller `evaluate` that throws or returns a malformed verdict — is named, never blamed on the model: the node stops at the first broken close and returns `{ incomplete, blocker: 'broken-sensor' | 'broken-verifier' }` with the model's last output preserved. The headline guarantee: **aggregation is code, never a model-stated number**, and a dead worker or exhausted guard returns an honest `{ incomplete, missingSlices }` — never a faked pass.
|
|
98
98
|
|
|
99
99
|
Over a corpus, context reaches a worker as a **handle routed by question shape** (`opts.retrieval`):
|
|
100
100
|
|
package/bareagent.context.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# bareagent — Integration Guide
|
|
2
2
|
|
|
3
3
|
> For AI assistants and developers wiring bareagent into a project.
|
|
4
|
-
> v0.
|
|
4
|
+
> v0.31.0 | Node.js >= 18 | zero required deps (`bareguard >=0.9.0 <0.13.0` optional peer for governance) | Apache 2.0
|
|
5
5
|
>
|
|
6
6
|
> Full human guide with composition examples, design philosophy, and recipes: [Usage Guide](docs/02-features/usage-guide.md)
|
|
7
7
|
|
|
@@ -38,7 +38,7 @@ Eight entry points:
|
|
|
38
38
|
| Count / answer "how many / all" over a corpus, honestly | recurse(task, ctx, `{ corpus, retrieval: 'scan' }`) — scans every slice, CODE-counts |
|
|
39
39
|
| Give recurse workers a persona/role (senior-dev stance) | recurse(task, ctx, `{ persona }`) — prepended to every worker, carries down the tree; not applied to the verifier |
|
|
40
40
|
| Tell recurse workers WHERE they are (paths/cwd) so a slice can find its file | recurse(task, ctx, `{ context }`) — read-only blob on every worker's task message + the Planner + verifier; carries down (facts, not a stance) |
|
|
41
|
-
| Let a recurse LEAF retry its own failure with a deterministic check | recurse(task, ctx, `{ refineLeaf: { sensor } }`) — leaf becomes a bounded generate→sense→regenerate loop; your sensor (test/compile/lint), gap fed back, escalating temperature |
|
|
41
|
+
| Let a recurse LEAF retry its own failure with a deterministic check | recurse(task, ctx, `{ refineLeaf: { sensor } }`) — leaf becomes a bounded generate→sense→regenerate loop; your sensor (test/compile/lint) judges the RETURNED result, gap fed back, escalating temperature; `rejectedBuffer` feeds prior failed attempts back on a temp-fixed model |
|
|
42
42
|
| Track task state (pending/running/done/failed) | StateMachine |
|
|
43
43
|
| Run agent turns on a schedule (cron, timers) | Scheduler |
|
|
44
44
|
| Require human approval before dangerous actions | Checkpoint |
|
|
@@ -708,7 +708,13 @@ const out = await recurse('Audit auth.js, billing.js, gateway.js for authz bugs'
|
|
|
708
708
|
|
|
709
709
|
**Worker context (`opts.context`, v0.23.0):** a read-only working-context string (paths/cwd) PREPENDED to every worker's TASK message as a `Working context:` block — so a sliced child can **locate its artifact** (the Planner paraphrases the goal into subtasks and drops absolute paths; without this, workers guess `.`/`~`/`/tmp` and get denied). Forwarded to the Planner as `info` (path-aware slices) and shown to the verifier too (neutral FACTS, not a stance — distinct from `persona`, which is a privileged SYSTEM-prompt stance). Carries down the tree. **Security:** it still becomes part of the prompt, so pass caller-trusted run-state only, never untrusted/end-user text (lower-privilege than `persona` — user message, not system — but still an injection surface). Absent ⇒ the task message is unchanged.
|
|
710
710
|
|
|
711
|
-
**Leaf self-correction (`opts.refineLeaf`, v0.23.0, opt-in):** turn a **definite leaf** (a node offered no `spawn_child` — `simple` tier or at `maxDepth`) into a bounded generate→sense→regenerate loop instead of a single pass: `{ sensor, maxIterations?, temperatures? }`. `sensor(result, { task, context, contract }) → Verdict` is YOUR **deterministic** close (test/compile/lint — not a model judge); on a non-pass its `critique` (the gap, not the transcript) is fed FRESH into the next attempt and — **on models that accept `temperature`** — the **retry temperature ESCALATES** (default `[0.2, 0.7, 1.0]` — load-bearing there: a weak model at a flat temperature regenerates identical wrong code and ignores even crisp feedback). On a **temperature-fixed model** (e.g. `claude-sonnet-5`, which 400s any non-default temperature) the provider silently drops the param (see below), the escalation lever is inert, and the fed-back gap critique carries recovery alone; `receipts.refineLeaf.temperatures` then records the EFFECTIVE temps — a `null` marks an attempt that ran at the model's default (never the ignored requested value). Each attempt is gate-checked + metered; a HaltError mid-loop → clean `{ incomplete }`; honest non-recovery → `receipts.refineLeaf.passed === false` (never a faked pass); `receipts.tokens` sums all attempts. The error-keyed `recall` stays YOUR tool (`opts.tools`), keyed off the fed-back critique — bareagent stays litectx-agnostic. Carries down (engages at the leaves). Absent ⇒ a leaf is a single pass.
|
|
711
|
+
**Leaf self-correction (`opts.refineLeaf`, v0.23.0, opt-in):** turn a **definite leaf** (a node offered no `spawn_child` — `simple` tier or at `maxDepth`) into a bounded generate→sense→regenerate loop instead of a single pass: `{ sensor, maxIterations?, temperatures?, rejectedBuffer? }`. `sensor(result, { task, context, contract }) → Verdict` is YOUR **deterministic** close (test/compile/lint — not a model judge); on a non-pass its `critique` (the gap, not the transcript) is fed FRESH into the next attempt and — **on models that accept `temperature`** — the **retry temperature ESCALATES** (default `[0.2, 0.7, 1.0]` — load-bearing there: a weak model at a flat temperature regenerates identical wrong code and ignores even crisp feedback). On a **temperature-fixed model** (e.g. `claude-sonnet-5`, which 400s any non-default temperature) the provider silently drops the param (see below), the escalation lever is inert, and the fed-back gap critique carries recovery alone; `receipts.refineLeaf.temperatures` then records the EFFECTIVE temps — a `null` marks an attempt that ran at the model's default (never the ignored requested value). Each attempt is gate-checked + metered; a HaltError mid-loop → clean `{ incomplete }`; honest non-recovery → `receipts.refineLeaf.passed === false` (never a faked pass); `receipts.tokens` sums all attempts. The error-keyed `recall` stays YOUR tool (`opts.tools`), keyed off the fed-back critique — bareagent stays litectx-agnostic. Carries down (engages at the leaves). Absent ⇒ a leaf is a single pass.
|
|
712
|
+
|
|
713
|
+
> **Sensor integrity (`refineLeaf.sensor`):** the sensor must judge the **returned result** (tamper-proof — build/run the returned string in isolation), never a worker **side-effect** a worker with edit tools could game (writing a passing file then returning junk, or editing the failing test itself). The loop optimizes against whatever the sensor reads — keep the close outside what the worker can write. (RSI field lesson: reward-hacking appeared in every optimization loop with a gameable close. **Demonstrated in-repo** — `poc/sensor-gaming-blocked.mjs`: the moment the honest path was blocked, `claude-sonnet-5` faked a pass on an unsatisfiable on-disk check **5/5** — both by editing the test and by making the returned function non-pure — while a tamper-proof close that runs the returned artifact in isolation was not gameable.)
|
|
714
|
+
|
|
715
|
+
> **Broken arbiter ≠ failing model (BA-15):** a `refineLeaf.sensor` — or a caller `opts.evaluate` verifier — that **throws** (your test runner crashed — ENOENT, a harness syntax error) or returns a **malformed verdict** (anything with neither a usable `pass` nor a valid tri-state `status`) is a faulty **arbiter**, not a model failure. The loop stops at the **first** broken close (it never retries against a broken judge — each retry would carry zero feedback) and returns a labeled `{ incomplete, blocker: 'broken-sensor' | 'broken-verifier', blockerDetail }` (what the arbiter did; mirrored on `receipts`), with `best` preserving the model's last non-empty output (BA-5) — best-effort work the arbiter never graded, not a pass. The label names WHICH knob to fix. Check `out.blocker` before debugging the model: `'broken-sensor'` / `'broken-verifier'` mean fix your sensor / your `evaluate` and re-run; `'governance-deny'` means widen scope / re-gate. **`pass` may be any truthy/falsy value** (`true`, `1`, `0`) — only a verdict carrying *no* usable signal is malformed — and a verdict backed by a class/getters keeps its fields. **In a nested run** the label travels: `out.blocker` reports a descendant's fault with `out.blockerTask` naming which sub-task broke, while each ancestor's receipts record it as `blockerFrom` — an ancestor's own `blocker` always means *that* node broke, so it never accuses a node whose sensor never ran. The **default Evaluator rubric verifier is never labeled** (its failures are provider-class faults), and a `HaltError` thrown by either arbiter stays a clean governance halt. One thing stays YOURS: an arbiter that **hangs** hangs the node (no gate checkpoint fires inside your callback) — run untrusted or model-generated checks in an isolated child process **with a timeout**.
|
|
716
|
+
|
|
717
|
+
> **`rejectedBuffer` (BA-14, v0.30.0):** a second lever for a **temperature-fixed** model, where escalation is inert. Instead of only the latest critique, it surfaces the model's OWN prior failed attempts VERBATIM — *"you wrote these, they failed X — write something STRUCTURALLY DIFFERENT."* This is **directed** diversity (attack the specific repeated mistake); escalation is **random** diversity, and the two are **antagonistic** — temperature monotonically degrades the buffer (`poc/ba14b`: flat-0.2 100% → 0.7 70% → 1.0 50%), so when the buffer engages the retry temperature is **held flat** at `temperatures[0]`, never escalated. Trigger: `true` = force on (also on temperature-accepting models); `false` = force off (pure BA-8 escalation); **unset = adaptive** — engage only once a prior attempt's temperature was dropped (i.e. a temp-fixed model where escalation is inert and the buffer is the sole lever). On a temperature-accepting model the default leaves behavior byte-identical. `receipts.refineLeaf.rejectedBuffer` reports whether it engaged. Efficacy is a **weak-model / fixation** phenomenon (live on `claude-sonnet-5` it engaged 6/6 but recovered no better than critique-only — cost-neutral, hence adaptive-not-always-on).
|
|
712
718
|
|
|
713
719
|
```javascript
|
|
714
720
|
const out = await recurse('Fix the failing function in calc.js', ctx, {
|
package/package.json
CHANGED
package/src/loop.js
CHANGED
|
@@ -701,6 +701,11 @@ class Loop {
|
|
|
701
701
|
const generate = () => this.provider.generate(toSend, activeTools, options);
|
|
702
702
|
result = this.retry ? await this.retry.call(generate) : await generate();
|
|
703
703
|
} catch (err) {
|
|
704
|
+
// A HaltError is a governance exit, not a provider failure — re-throw it (like every other seam) so it
|
|
705
|
+
// reaches the outer handler, which seals dangling tool_calls and returns error:`halt:<rule>`. Without
|
|
706
|
+
// this, throwOnError:false laundered a provider-surfaced HaltError into a generic `error:<message>`,
|
|
707
|
+
// indistinguishable from a real fault (the retry never retries it — DEFAULT_RETRY_ON is false for it).
|
|
708
|
+
if (err instanceof HaltError) throw err;
|
|
704
709
|
this._reportError('provider', err, { round });
|
|
705
710
|
if (this.throwOnError) throw err;
|
|
706
711
|
// BA-5: a mid-run provider failure must not erase the work of the rounds that succeeded.
|
package/src/recurse.d.ts
CHANGED
|
@@ -94,13 +94,36 @@ export type RecurseOptions = {
|
|
|
94
94
|
* (`temperatures`, default `[0.2,0.7,1.0]`) — the live-validated lever that lets a weak model escape a
|
|
95
95
|
* repeat-the-same-mistake rut (`poc/ba8-leaf-refine.mjs`: 0/5 → 2-3/5; flat temp recovers 0/5). On a
|
|
96
96
|
* temperature-fixed model (BA-10) the provider drops the param, `receipts.refineLeaf.temperatures` records
|
|
97
|
-
* `null`, and the fed-back gap critique carries recovery alone
|
|
97
|
+
* `null`, and the fed-back gap critique carries recovery alone — UNLESS the rejected-attempt buffer engages
|
|
98
|
+
* (below). `maxIterations` defaults to
|
|
98
99
|
* `temperatures.length`; the REAL bound is bareguard (each attempt is gate-checked + metered). CARRIES DOWN the
|
|
99
100
|
* tree (preserved by `forChild`), so it engages at the leaves of a Family-A decomposition. Recovery is PARTIAL
|
|
100
101
|
* (a stubborn blind spot may persist) — `receipts.refineLeaf.passed` reports honestly. Does NOT apply to a node
|
|
101
102
|
* that delegates (its children + the tree verify own quality), nor to the scan/fanout/partition dispatch paths.
|
|
102
103
|
* Absent ⇒ a leaf is a single pass (byte-identical to pre-BA-8). An error-keyed `recall` is the CALLER's tool
|
|
103
104
|
* (`opts.tools`) keyed off the fed-back critique — bareagent stays litectx-agnostic.
|
|
105
|
+
* **Sensor integrity (RSI-learnings #1/#5, "audit the close"):** the `sensor` MUST judge the RETURNED result
|
|
106
|
+
* (tamper-proof — e.g. build/run the returned string in an isolated context, as `poc/ba8-leaf-refine.mjs` does),
|
|
107
|
+
* NEVER a worker side-effect a worker with edit tools could GAME (writing a passing file then returning junk, or
|
|
108
|
+
* editing the failing test itself). A gameable close is the reward-hacking surface every RSI system in the field
|
|
109
|
+
* got bitten by; the loop optimizes against WHATEVER the sensor reads, so keep it outside what the worker can write.
|
|
110
|
+
* **Broken sensor ≠ failing model (BA-15):** a sensor that THROWS (non-Halt) or returns a MALFORMED verdict
|
|
111
|
+
* (anything but `{pass: boolean}` or a valid tri-state `status`) is a faulty ARBITER — the loop stops at the
|
|
112
|
+
* FIRST broken close (never retries against it) and returns a labeled `{incomplete, blocker:'broken-sensor'}`
|
|
113
|
+
* (+ `receipts.blockerDetail`), with `best` preserving the model's last attempt. A `HaltError` thrown by the
|
|
114
|
+
* sensor stays a clean governance halt. The sensor's EXECUTION environment is the caller's: run untrusted /
|
|
115
|
+
* model-generated checks in an isolated child process WITH A TIMEOUT — a sensor that hangs forever hangs the
|
|
116
|
+
* leaf (no bareguard checkpoint fires between sensor start and return).
|
|
117
|
+
* **`rejectedBuffer` (BA-14):** a SkillOpt-shaped rejected-attempt buffer — instead of only the LATEST critique,
|
|
118
|
+
* surface the model's OWN prior failed attempts verbatim ("you wrote these, they failed X — write something
|
|
119
|
+
* STRUCTURALLY DIFFERENT"). This is DIRECTED diversity (attack the specific repeated mistake), where escalation
|
|
120
|
+
* is RANDOM diversity; the two are ANTAGONISTIC (`poc/ba14b-temp-with-buffer.mjs`: temperature monotonically
|
|
121
|
+
* degrades the buffer, 100%→70%→50% across 0.2/0.7/1.0), so when the buffer engages the retry temperature is
|
|
122
|
+
* HELD at `temperatures[0]` (flat-low), never escalated. `true` = force on (also on temperature-accepting
|
|
123
|
+
* models); `false` = force off (pure BA-8 escalation); UNSET = ADAPTIVE — engage only once a prior attempt's
|
|
124
|
+
* temperature was dropped (a temperature-fixed model, BA-10, where escalation is inert and the buffer is the
|
|
125
|
+
* sole lever — `poc/ba14-rejected-buffer.mjs`: flat-temp 50%→100%). `receipts.refineLeaf.rejectedBuffer`
|
|
126
|
+
* reports whether any iteration injected it. Bounded by `maxIterations` (the buffer never outgrows it).
|
|
104
127
|
*/
|
|
105
128
|
refineLeaf?: {
|
|
106
129
|
sensor: (result: any, ctx: {
|
|
@@ -110,6 +133,7 @@ export type RecurseOptions = {
|
|
|
110
133
|
}) => (Verdict | Promise<Verdict>);
|
|
111
134
|
maxIterations?: number;
|
|
112
135
|
temperatures?: number[];
|
|
136
|
+
rejectedBuffer?: boolean;
|
|
113
137
|
} | undefined;
|
|
114
138
|
/**
|
|
115
139
|
* - Definition of done (A3). When present, the verifier grades against THIS,
|
|
@@ -216,24 +240,48 @@ export type RecurseNode = {
|
|
|
216
240
|
incomplete: boolean;
|
|
217
241
|
halted: boolean;
|
|
218
242
|
/**
|
|
219
|
-
* -
|
|
220
|
-
* short-circuited a consecutive-policy-deny
|
|
243
|
+
* - Set when this node stopped for a specific non-model reason (mirrors
|
|
244
|
+
* `RecurseResult.blocker`): `'governance-deny'` (BA-11) — its Loop short-circuited a consecutive-policy-deny
|
|
245
|
+
* spin; `'broken-sensor'` (BA-15) — the caller's `refineLeaf.sensor` threw or returned a malformed verdict;
|
|
246
|
+
* `'broken-verifier'` (BA-15) — the caller's `opts.evaluate` did (the default Evaluator path is never labeled).
|
|
221
247
|
*/
|
|
222
248
|
blocker?: string | undefined;
|
|
249
|
+
/**
|
|
250
|
+
* - (BA-15) with a `broken-*` blocker: what the arbiter did (threw with
|
|
251
|
+
* which message, or which malformed shape it returned) — the actionable half of the label.
|
|
252
|
+
*/
|
|
253
|
+
blockerDetail?: string | undefined;
|
|
254
|
+
/**
|
|
255
|
+
* - (BA-15) a
|
|
256
|
+
* DESCENDANT's blocker, surfaced here so an aggregating node still reports the fault upward. Deliberately
|
|
257
|
+
* SEPARATE from this node's own `blocker` (which means "THIS node's arbiter/Loop broke"): stamping a
|
|
258
|
+
* descendant's label onto every ancestor made the receipts tree accuse nodes whose sensor never ran, and
|
|
259
|
+
* re-labelled a parent `governance-deny` when only one child was denied. `blockerTask` names the culprit.
|
|
260
|
+
*/
|
|
261
|
+
blockerFrom?: {
|
|
262
|
+
blocker: string;
|
|
263
|
+
blockerDetail?: string;
|
|
264
|
+
blockerTask?: string;
|
|
265
|
+
} | undefined;
|
|
223
266
|
/**
|
|
224
267
|
* - The worker Loop's `metrics.tokens`.
|
|
225
268
|
*/
|
|
226
269
|
tokens: object | null;
|
|
227
270
|
/**
|
|
228
|
-
* - (BA-8) when
|
|
229
|
-
* leaf ran as a bounded refine loop: how many attempts it took and whether the deterministic sensor finally
|
|
271
|
+
* - (BA-8) when
|
|
272
|
+
* this leaf ran as a bounded refine loop: how many attempts it took and whether the deterministic sensor finally
|
|
230
273
|
* passed (false = honest non-recovery, not a faked success). `temperatures` are the EFFECTIVE per-attempt temps
|
|
231
274
|
* (BA-10): a `null` marks an attempt the model ran at its DEFAULT because it rejected the requested temperature.
|
|
275
|
+
* `rejectedBuffer` (BA-14): whether any iteration injected the rejected-attempt buffer (prior failed attempts
|
|
276
|
+
* surfaced verbatim); when it engages the REQUESTED retry temperature is held flat at `temperatures[0]`, never
|
|
277
|
+
* escalated (ba14b antagonism) — but a temperature-fixed model still drops it, so the EFFECTIVE temp recorded
|
|
278
|
+
* above is `null`, not `temperatures[0]`.
|
|
232
279
|
*/
|
|
233
280
|
refineLeaf?: {
|
|
234
281
|
iterations: number;
|
|
235
282
|
passed: boolean;
|
|
236
283
|
temperatures: (number | null)[];
|
|
284
|
+
rejectedBuffer: boolean;
|
|
237
285
|
} | undefined;
|
|
238
286
|
model: string | null;
|
|
239
287
|
/**
|
|
@@ -296,8 +344,26 @@ export type RecurseResult = {
|
|
|
296
344
|
* - Present when `incomplete` for a specific, actionable reason. `'governance-deny'`
|
|
297
345
|
* (BA-11): the worker's Loop short-circuited after N consecutive policy denials rather than burn to the
|
|
298
346
|
* budget cap — the caller can widen scope / re-gate / escalate instead of reading it as a model failure.
|
|
347
|
+
* `'broken-sensor'` (BA-15): the caller's `refineLeaf.sensor` threw or returned a malformed verdict — the
|
|
348
|
+
* ARBITER is faulty, not the model; fix the sensor and re-run (`receipts.blockerDetail` says what it did).
|
|
349
|
+
* `'broken-verifier'` (BA-15): same fault class at the verify slot — the caller's `opts.evaluate` threw
|
|
350
|
+
* (non-Halt) or returned a malformed verdict; the default Evaluator path is never labeled (its failures are
|
|
351
|
+
* provider-class faults). For both `broken-*` blockers `best` preserves the model's last non-empty output
|
|
352
|
+
* (BA-5) — the arbiter broke, so treat it as best-effort work rather than a graded pass.
|
|
299
353
|
*/
|
|
300
354
|
blocker?: string | undefined;
|
|
355
|
+
/**
|
|
356
|
+
* - (BA-15) with a `broken-*` blocker: what the arbiter did — the
|
|
357
|
+
* ACTIONABLE half of the label, surfaced on the result (not only in `receipts`) so a caller branching on
|
|
358
|
+
* `blocker` can report the cause without walking the receipts tree.
|
|
359
|
+
*/
|
|
360
|
+
blockerDetail?: string | undefined;
|
|
361
|
+
/**
|
|
362
|
+
* - (BA-15) when the blocker was INHERITED from a descendant in a nested run:
|
|
363
|
+
* which sub-task actually broke. Without it a nested failure reports "a sensor broke" with no way to find
|
|
364
|
+
* which one.
|
|
365
|
+
*/
|
|
366
|
+
blockerTask?: string | undefined;
|
|
301
367
|
/**
|
|
302
368
|
* - The audit node for this call (RC-10).
|
|
303
369
|
*/
|
|
@@ -362,7 +428,7 @@ export type Slice = {
|
|
|
362
428
|
* TRUSTED run-state (paths/cwd); do NOT pass untrusted / end-user-controlled text here.
|
|
363
429
|
* @property {ToolDef[]} [tools] - Handle tools offered to EVERY worker (RC-5 pull-default: litectx
|
|
364
430
|
* `recall`/`get`, wired at build step 7). Workers query on demand; never the whole corpus.
|
|
365
|
-
* @property {{sensor: (result: any, ctx: {task: string, context: string|undefined, contract: string|null}) => (Verdict|Promise<Verdict>), maxIterations?: number, temperatures?: number[]}} [refineLeaf]
|
|
431
|
+
* @property {{sensor: (result: any, ctx: {task: string, context: string|undefined, contract: string|null}) => (Verdict|Promise<Verdict>), maxIterations?: number, temperatures?: number[], rejectedBuffer?: boolean}} [refineLeaf]
|
|
366
432
|
* (Opt-in, BA-8 / relayfact F17) Turn a DEFINITE LEAF (a node that is offered no `spawn_child` — `simple`
|
|
367
433
|
* tier or at `maxDepth`) into a bounded generate→sense→regenerate loop instead of a single pass, so a failed
|
|
368
434
|
* slice can self-correct. `sensor` is a DETERMINISTIC close (test/compile/lint — NOT a model judge, R-S8) that
|
|
@@ -371,13 +437,36 @@ export type Slice = {
|
|
|
371
437
|
* (`temperatures`, default `[0.2,0.7,1.0]`) — the live-validated lever that lets a weak model escape a
|
|
372
438
|
* repeat-the-same-mistake rut (`poc/ba8-leaf-refine.mjs`: 0/5 → 2-3/5; flat temp recovers 0/5). On a
|
|
373
439
|
* temperature-fixed model (BA-10) the provider drops the param, `receipts.refineLeaf.temperatures` records
|
|
374
|
-
* `null`, and the fed-back gap critique carries recovery alone
|
|
440
|
+
* `null`, and the fed-back gap critique carries recovery alone — UNLESS the rejected-attempt buffer engages
|
|
441
|
+
* (below). `maxIterations` defaults to
|
|
375
442
|
* `temperatures.length`; the REAL bound is bareguard (each attempt is gate-checked + metered). CARRIES DOWN the
|
|
376
443
|
* tree (preserved by `forChild`), so it engages at the leaves of a Family-A decomposition. Recovery is PARTIAL
|
|
377
444
|
* (a stubborn blind spot may persist) — `receipts.refineLeaf.passed` reports honestly. Does NOT apply to a node
|
|
378
445
|
* that delegates (its children + the tree verify own quality), nor to the scan/fanout/partition dispatch paths.
|
|
379
446
|
* Absent ⇒ a leaf is a single pass (byte-identical to pre-BA-8). An error-keyed `recall` is the CALLER's tool
|
|
380
447
|
* (`opts.tools`) keyed off the fed-back critique — bareagent stays litectx-agnostic.
|
|
448
|
+
* **Sensor integrity (RSI-learnings #1/#5, "audit the close"):** the `sensor` MUST judge the RETURNED result
|
|
449
|
+
* (tamper-proof — e.g. build/run the returned string in an isolated context, as `poc/ba8-leaf-refine.mjs` does),
|
|
450
|
+
* NEVER a worker side-effect a worker with edit tools could GAME (writing a passing file then returning junk, or
|
|
451
|
+
* editing the failing test itself). A gameable close is the reward-hacking surface every RSI system in the field
|
|
452
|
+
* got bitten by; the loop optimizes against WHATEVER the sensor reads, so keep it outside what the worker can write.
|
|
453
|
+
* **Broken sensor ≠ failing model (BA-15):** a sensor that THROWS (non-Halt) or returns a MALFORMED verdict
|
|
454
|
+
* (anything but `{pass: boolean}` or a valid tri-state `status`) is a faulty ARBITER — the loop stops at the
|
|
455
|
+
* FIRST broken close (never retries against it) and returns a labeled `{incomplete, blocker:'broken-sensor'}`
|
|
456
|
+
* (+ `receipts.blockerDetail`), with `best` preserving the model's last attempt. A `HaltError` thrown by the
|
|
457
|
+
* sensor stays a clean governance halt. The sensor's EXECUTION environment is the caller's: run untrusted /
|
|
458
|
+
* model-generated checks in an isolated child process WITH A TIMEOUT — a sensor that hangs forever hangs the
|
|
459
|
+
* leaf (no bareguard checkpoint fires between sensor start and return).
|
|
460
|
+
* **`rejectedBuffer` (BA-14):** a SkillOpt-shaped rejected-attempt buffer — instead of only the LATEST critique,
|
|
461
|
+
* surface the model's OWN prior failed attempts verbatim ("you wrote these, they failed X — write something
|
|
462
|
+
* STRUCTURALLY DIFFERENT"). This is DIRECTED diversity (attack the specific repeated mistake), where escalation
|
|
463
|
+
* is RANDOM diversity; the two are ANTAGONISTIC (`poc/ba14b-temp-with-buffer.mjs`: temperature monotonically
|
|
464
|
+
* degrades the buffer, 100%→70%→50% across 0.2/0.7/1.0), so when the buffer engages the retry temperature is
|
|
465
|
+
* HELD at `temperatures[0]` (flat-low), never escalated. `true` = force on (also on temperature-accepting
|
|
466
|
+
* models); `false` = force off (pure BA-8 escalation); UNSET = ADAPTIVE — engage only once a prior attempt's
|
|
467
|
+
* temperature was dropped (a temperature-fixed model, BA-10, where escalation is inert and the buffer is the
|
|
468
|
+
* sole lever — `poc/ba14-rejected-buffer.mjs`: flat-temp 50%→100%). `receipts.refineLeaf.rejectedBuffer`
|
|
469
|
+
* reports whether any iteration injected it. Bounded by `maxIterations` (the buffer never outgrows it).
|
|
381
470
|
* @property {string} [contract] - Definition of done (A3). When present, the verifier grades against THIS,
|
|
382
471
|
* not the loose task, and verification always runs.
|
|
383
472
|
* @property {(result: any, ctx: {contract: string|null, task: string}) => (Verdict|Promise<Verdict>)} [evaluate]
|
|
@@ -435,13 +524,26 @@ export type Slice = {
|
|
|
435
524
|
* @property {Verdict|null} verdict
|
|
436
525
|
* @property {boolean} incomplete
|
|
437
526
|
* @property {boolean} halted
|
|
438
|
-
* @property {string} [blocker] -
|
|
439
|
-
* short-circuited a consecutive-policy-deny
|
|
527
|
+
* @property {string} [blocker] - Set when this node stopped for a specific non-model reason (mirrors
|
|
528
|
+
* `RecurseResult.blocker`): `'governance-deny'` (BA-11) — its Loop short-circuited a consecutive-policy-deny
|
|
529
|
+
* spin; `'broken-sensor'` (BA-15) — the caller's `refineLeaf.sensor` threw or returned a malformed verdict;
|
|
530
|
+
* `'broken-verifier'` (BA-15) — the caller's `opts.evaluate` did (the default Evaluator path is never labeled).
|
|
531
|
+
* @property {string} [blockerDetail] - (BA-15) with a `broken-*` blocker: what the arbiter did (threw with
|
|
532
|
+
* which message, or which malformed shape it returned) — the actionable half of the label.
|
|
533
|
+
* @property {{blocker: string, blockerDetail?: string, blockerTask?: string}} [blockerFrom] - (BA-15) a
|
|
534
|
+
* DESCENDANT's blocker, surfaced here so an aggregating node still reports the fault upward. Deliberately
|
|
535
|
+
* SEPARATE from this node's own `blocker` (which means "THIS node's arbiter/Loop broke"): stamping a
|
|
536
|
+
* descendant's label onto every ancestor made the receipts tree accuse nodes whose sensor never ran, and
|
|
537
|
+
* re-labelled a parent `governance-deny` when only one child was denied. `blockerTask` names the culprit.
|
|
440
538
|
* @property {object|null} tokens - The worker Loop's `metrics.tokens`.
|
|
441
|
-
* @property {{iterations: number, passed: boolean, temperatures: (number|null)[]}} [refineLeaf] - (BA-8) when
|
|
442
|
-
* leaf ran as a bounded refine loop: how many attempts it took and whether the deterministic sensor finally
|
|
539
|
+
* @property {{iterations: number, passed: boolean, temperatures: (number|null)[], rejectedBuffer: boolean}} [refineLeaf] - (BA-8) when
|
|
540
|
+
* this leaf ran as a bounded refine loop: how many attempts it took and whether the deterministic sensor finally
|
|
443
541
|
* passed (false = honest non-recovery, not a faked success). `temperatures` are the EFFECTIVE per-attempt temps
|
|
444
542
|
* (BA-10): a `null` marks an attempt the model ran at its DEFAULT because it rejected the requested temperature.
|
|
543
|
+
* `rejectedBuffer` (BA-14): whether any iteration injected the rejected-attempt buffer (prior failed attempts
|
|
544
|
+
* surfaced verbatim); when it engages the REQUESTED retry temperature is held flat at `temperatures[0]`, never
|
|
545
|
+
* escalated (ba14b antagonism) — but a temperature-fixed model still drops it, so the EFFECTIVE temp recorded
|
|
546
|
+
* above is `null`, not `temperatures[0]`.
|
|
445
547
|
* @property {string|null} model
|
|
446
548
|
* @property {string|null} [retrieval] - (§10 step 7) the retrieval mode this node ran (`scan`/`search`/`exact`),
|
|
447
549
|
* or null/absent for a plain reasoning node.
|
|
@@ -465,6 +567,18 @@ export type Slice = {
|
|
|
465
567
|
* @property {string} [blocker] - Present when `incomplete` for a specific, actionable reason. `'governance-deny'`
|
|
466
568
|
* (BA-11): the worker's Loop short-circuited after N consecutive policy denials rather than burn to the
|
|
467
569
|
* budget cap — the caller can widen scope / re-gate / escalate instead of reading it as a model failure.
|
|
570
|
+
* `'broken-sensor'` (BA-15): the caller's `refineLeaf.sensor` threw or returned a malformed verdict — the
|
|
571
|
+
* ARBITER is faulty, not the model; fix the sensor and re-run (`receipts.blockerDetail` says what it did).
|
|
572
|
+
* `'broken-verifier'` (BA-15): same fault class at the verify slot — the caller's `opts.evaluate` threw
|
|
573
|
+
* (non-Halt) or returned a malformed verdict; the default Evaluator path is never labeled (its failures are
|
|
574
|
+
* provider-class faults). For both `broken-*` blockers `best` preserves the model's last non-empty output
|
|
575
|
+
* (BA-5) — the arbiter broke, so treat it as best-effort work rather than a graded pass.
|
|
576
|
+
* @property {string} [blockerDetail] - (BA-15) with a `broken-*` blocker: what the arbiter did — the
|
|
577
|
+
* ACTIONABLE half of the label, surfaced on the result (not only in `receipts`) so a caller branching on
|
|
578
|
+
* `blocker` can report the cause without walking the receipts tree.
|
|
579
|
+
* @property {string} [blockerTask] - (BA-15) when the blocker was INHERITED from a descendant in a nested run:
|
|
580
|
+
* which sub-task actually broke. Without it a nested failure reports "a sensor broke" with no way to find
|
|
581
|
+
* which one.
|
|
468
582
|
* @property {RecurseNode} receipts - The audit node for this call (RC-10).
|
|
469
583
|
*/
|
|
470
584
|
/**
|
package/src/recurse.js
CHANGED
|
@@ -216,7 +216,7 @@ function auditSafeCtx(ctx, overrides = {}) {
|
|
|
216
216
|
* TRUSTED run-state (paths/cwd); do NOT pass untrusted / end-user-controlled text here.
|
|
217
217
|
* @property {ToolDef[]} [tools] - Handle tools offered to EVERY worker (RC-5 pull-default: litectx
|
|
218
218
|
* `recall`/`get`, wired at build step 7). Workers query on demand; never the whole corpus.
|
|
219
|
-
* @property {{sensor: (result: any, ctx: {task: string, context: string|undefined, contract: string|null}) => (Verdict|Promise<Verdict>), maxIterations?: number, temperatures?: number[]}} [refineLeaf]
|
|
219
|
+
* @property {{sensor: (result: any, ctx: {task: string, context: string|undefined, contract: string|null}) => (Verdict|Promise<Verdict>), maxIterations?: number, temperatures?: number[], rejectedBuffer?: boolean}} [refineLeaf]
|
|
220
220
|
* (Opt-in, BA-8 / relayfact F17) Turn a DEFINITE LEAF (a node that is offered no `spawn_child` — `simple`
|
|
221
221
|
* tier or at `maxDepth`) into a bounded generate→sense→regenerate loop instead of a single pass, so a failed
|
|
222
222
|
* slice can self-correct. `sensor` is a DETERMINISTIC close (test/compile/lint — NOT a model judge, R-S8) that
|
|
@@ -225,13 +225,36 @@ function auditSafeCtx(ctx, overrides = {}) {
|
|
|
225
225
|
* (`temperatures`, default `[0.2,0.7,1.0]`) — the live-validated lever that lets a weak model escape a
|
|
226
226
|
* repeat-the-same-mistake rut (`poc/ba8-leaf-refine.mjs`: 0/5 → 2-3/5; flat temp recovers 0/5). On a
|
|
227
227
|
* temperature-fixed model (BA-10) the provider drops the param, `receipts.refineLeaf.temperatures` records
|
|
228
|
-
* `null`, and the fed-back gap critique carries recovery alone
|
|
228
|
+
* `null`, and the fed-back gap critique carries recovery alone — UNLESS the rejected-attempt buffer engages
|
|
229
|
+
* (below). `maxIterations` defaults to
|
|
229
230
|
* `temperatures.length`; the REAL bound is bareguard (each attempt is gate-checked + metered). CARRIES DOWN the
|
|
230
231
|
* tree (preserved by `forChild`), so it engages at the leaves of a Family-A decomposition. Recovery is PARTIAL
|
|
231
232
|
* (a stubborn blind spot may persist) — `receipts.refineLeaf.passed` reports honestly. Does NOT apply to a node
|
|
232
233
|
* that delegates (its children + the tree verify own quality), nor to the scan/fanout/partition dispatch paths.
|
|
233
234
|
* Absent ⇒ a leaf is a single pass (byte-identical to pre-BA-8). An error-keyed `recall` is the CALLER's tool
|
|
234
235
|
* (`opts.tools`) keyed off the fed-back critique — bareagent stays litectx-agnostic.
|
|
236
|
+
* **Sensor integrity (RSI-learnings #1/#5, "audit the close"):** the `sensor` MUST judge the RETURNED result
|
|
237
|
+
* (tamper-proof — e.g. build/run the returned string in an isolated context, as `poc/ba8-leaf-refine.mjs` does),
|
|
238
|
+
* NEVER a worker side-effect a worker with edit tools could GAME (writing a passing file then returning junk, or
|
|
239
|
+
* editing the failing test itself). A gameable close is the reward-hacking surface every RSI system in the field
|
|
240
|
+
* got bitten by; the loop optimizes against WHATEVER the sensor reads, so keep it outside what the worker can write.
|
|
241
|
+
* **Broken sensor ≠ failing model (BA-15):** a sensor that THROWS (non-Halt) or returns a MALFORMED verdict
|
|
242
|
+
* (anything but `{pass: boolean}` or a valid tri-state `status`) is a faulty ARBITER — the loop stops at the
|
|
243
|
+
* FIRST broken close (never retries against it) and returns a labeled `{incomplete, blocker:'broken-sensor'}`
|
|
244
|
+
* (+ `receipts.blockerDetail`), with `best` preserving the model's last attempt. A `HaltError` thrown by the
|
|
245
|
+
* sensor stays a clean governance halt. The sensor's EXECUTION environment is the caller's: run untrusted /
|
|
246
|
+
* model-generated checks in an isolated child process WITH A TIMEOUT — a sensor that hangs forever hangs the
|
|
247
|
+
* leaf (no bareguard checkpoint fires between sensor start and return).
|
|
248
|
+
* **`rejectedBuffer` (BA-14):** a SkillOpt-shaped rejected-attempt buffer — instead of only the LATEST critique,
|
|
249
|
+
* surface the model's OWN prior failed attempts verbatim ("you wrote these, they failed X — write something
|
|
250
|
+
* STRUCTURALLY DIFFERENT"). This is DIRECTED diversity (attack the specific repeated mistake), where escalation
|
|
251
|
+
* is RANDOM diversity; the two are ANTAGONISTIC (`poc/ba14b-temp-with-buffer.mjs`: temperature monotonically
|
|
252
|
+
* degrades the buffer, 100%→70%→50% across 0.2/0.7/1.0), so when the buffer engages the retry temperature is
|
|
253
|
+
* HELD at `temperatures[0]` (flat-low), never escalated. `true` = force on (also on temperature-accepting
|
|
254
|
+
* models); `false` = force off (pure BA-8 escalation); UNSET = ADAPTIVE — engage only once a prior attempt's
|
|
255
|
+
* temperature was dropped (a temperature-fixed model, BA-10, where escalation is inert and the buffer is the
|
|
256
|
+
* sole lever — `poc/ba14-rejected-buffer.mjs`: flat-temp 50%→100%). `receipts.refineLeaf.rejectedBuffer`
|
|
257
|
+
* reports whether any iteration injected it. Bounded by `maxIterations` (the buffer never outgrows it).
|
|
235
258
|
* @property {string} [contract] - Definition of done (A3). When present, the verifier grades against THIS,
|
|
236
259
|
* not the loose task, and verification always runs.
|
|
237
260
|
* @property {(result: any, ctx: {contract: string|null, task: string}) => (Verdict|Promise<Verdict>)} [evaluate]
|
|
@@ -290,13 +313,26 @@ function auditSafeCtx(ctx, overrides = {}) {
|
|
|
290
313
|
* @property {Verdict|null} verdict
|
|
291
314
|
* @property {boolean} incomplete
|
|
292
315
|
* @property {boolean} halted
|
|
293
|
-
* @property {string} [blocker] -
|
|
294
|
-
* short-circuited a consecutive-policy-deny
|
|
316
|
+
* @property {string} [blocker] - Set when this node stopped for a specific non-model reason (mirrors
|
|
317
|
+
* `RecurseResult.blocker`): `'governance-deny'` (BA-11) — its Loop short-circuited a consecutive-policy-deny
|
|
318
|
+
* spin; `'broken-sensor'` (BA-15) — the caller's `refineLeaf.sensor` threw or returned a malformed verdict;
|
|
319
|
+
* `'broken-verifier'` (BA-15) — the caller's `opts.evaluate` did (the default Evaluator path is never labeled).
|
|
320
|
+
* @property {string} [blockerDetail] - (BA-15) with a `broken-*` blocker: what the arbiter did (threw with
|
|
321
|
+
* which message, or which malformed shape it returned) — the actionable half of the label.
|
|
322
|
+
* @property {{blocker: string, blockerDetail?: string, blockerTask?: string}} [blockerFrom] - (BA-15) a
|
|
323
|
+
* DESCENDANT's blocker, surfaced here so an aggregating node still reports the fault upward. Deliberately
|
|
324
|
+
* SEPARATE from this node's own `blocker` (which means "THIS node's arbiter/Loop broke"): stamping a
|
|
325
|
+
* descendant's label onto every ancestor made the receipts tree accuse nodes whose sensor never ran, and
|
|
326
|
+
* re-labelled a parent `governance-deny` when only one child was denied. `blockerTask` names the culprit.
|
|
295
327
|
* @property {object|null} tokens - The worker Loop's `metrics.tokens`.
|
|
296
|
-
* @property {{iterations: number, passed: boolean, temperatures: (number|null)[]}} [refineLeaf] - (BA-8) when
|
|
297
|
-
* leaf ran as a bounded refine loop: how many attempts it took and whether the deterministic sensor finally
|
|
328
|
+
* @property {{iterations: number, passed: boolean, temperatures: (number|null)[], rejectedBuffer: boolean}} [refineLeaf] - (BA-8) when
|
|
329
|
+
* this leaf ran as a bounded refine loop: how many attempts it took and whether the deterministic sensor finally
|
|
298
330
|
* passed (false = honest non-recovery, not a faked success). `temperatures` are the EFFECTIVE per-attempt temps
|
|
299
331
|
* (BA-10): a `null` marks an attempt the model ran at its DEFAULT because it rejected the requested temperature.
|
|
332
|
+
* `rejectedBuffer` (BA-14): whether any iteration injected the rejected-attempt buffer (prior failed attempts
|
|
333
|
+
* surfaced verbatim); when it engages the REQUESTED retry temperature is held flat at `temperatures[0]`, never
|
|
334
|
+
* escalated (ba14b antagonism) — but a temperature-fixed model still drops it, so the EFFECTIVE temp recorded
|
|
335
|
+
* above is `null`, not `temperatures[0]`.
|
|
300
336
|
* @property {string|null} model
|
|
301
337
|
* @property {string|null} [retrieval] - (§10 step 7) the retrieval mode this node ran (`scan`/`search`/`exact`),
|
|
302
338
|
* or null/absent for a plain reasoning node.
|
|
@@ -321,6 +357,18 @@ function auditSafeCtx(ctx, overrides = {}) {
|
|
|
321
357
|
* @property {string} [blocker] - Present when `incomplete` for a specific, actionable reason. `'governance-deny'`
|
|
322
358
|
* (BA-11): the worker's Loop short-circuited after N consecutive policy denials rather than burn to the
|
|
323
359
|
* budget cap — the caller can widen scope / re-gate / escalate instead of reading it as a model failure.
|
|
360
|
+
* `'broken-sensor'` (BA-15): the caller's `refineLeaf.sensor` threw or returned a malformed verdict — the
|
|
361
|
+
* ARBITER is faulty, not the model; fix the sensor and re-run (`receipts.blockerDetail` says what it did).
|
|
362
|
+
* `'broken-verifier'` (BA-15): same fault class at the verify slot — the caller's `opts.evaluate` threw
|
|
363
|
+
* (non-Halt) or returned a malformed verdict; the default Evaluator path is never labeled (its failures are
|
|
364
|
+
* provider-class faults). For both `broken-*` blockers `best` preserves the model's last non-empty output
|
|
365
|
+
* (BA-5) — the arbiter broke, so treat it as best-effort work rather than a graded pass.
|
|
366
|
+
* @property {string} [blockerDetail] - (BA-15) with a `broken-*` blocker: what the arbiter did — the
|
|
367
|
+
* ACTIONABLE half of the label, surfaced on the result (not only in `receipts`) so a caller branching on
|
|
368
|
+
* `blocker` can report the cause without walking the receipts tree.
|
|
369
|
+
* @property {string} [blockerTask] - (BA-15) when the blocker was INHERITED from a descendant in a nested run:
|
|
370
|
+
* which sub-task actually broke. Without it a nested failure reports "a sensor broke" with no way to find
|
|
371
|
+
* which one.
|
|
324
372
|
* @property {RecurseNode} receipts - The audit node for this call (RC-10).
|
|
325
373
|
*/
|
|
326
374
|
|
|
@@ -566,16 +614,16 @@ async function recurse(task, ctx = {}, opts = {}) {
|
|
|
566
614
|
const missingSlices = node.spawned.filter(c => c.incomplete).map(c => c.task);
|
|
567
615
|
if (missingSlices.length > 0) {
|
|
568
616
|
node.incomplete = true;
|
|
569
|
-
|
|
617
|
+
// BA-15: carry a child's blocker up, so a nested broken sensor still names itself at the top.
|
|
618
|
+
return incompleteWithBlocker(node, result, { missingSlices });
|
|
570
619
|
}
|
|
571
620
|
|
|
572
621
|
// Verify: a SEPARATE-context judge, never the generator grading itself. Runs when a contract is given, the
|
|
573
622
|
// caller supplied a verifier, OR the task is critical (the forced-verify safety rail).
|
|
574
623
|
const wantVerify = critical || typeof opts.contract === 'string' || typeof opts.evaluate === 'function';
|
|
575
624
|
if (wantVerify) {
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
return { result, verdict, receipts: node };
|
|
625
|
+
// `return await` (not a bare `return`) is load-bearing here — see verifyOrBlock's JSDoc.
|
|
626
|
+
return await verifyOrBlock(task, result, ctx, opts, node);
|
|
579
627
|
}
|
|
580
628
|
|
|
581
629
|
return { result, verdict: null, receipts: node };
|
|
@@ -583,12 +631,221 @@ async function recurse(task, ctx = {}, opts = {}) {
|
|
|
583
631
|
if (err instanceof HaltError) {
|
|
584
632
|
node.halted = true;
|
|
585
633
|
node.incomplete = true;
|
|
586
|
-
|
|
634
|
+
// BA-15: a halt AFTER the children ran (e.g. mid-synthesize) must still name a child's broken sensor —
|
|
635
|
+
// otherwise the caller reads a governance problem where their own arbiter crashed.
|
|
636
|
+
return incompleteWithBlocker(node, result);
|
|
587
637
|
}
|
|
588
638
|
throw err;
|
|
589
639
|
}
|
|
590
640
|
}
|
|
591
641
|
|
|
642
|
+
/** Valid tri-state `Verdict.status` values a caller arbiter (sensor/verifier) may return in lieu of a boolean `pass`. */
|
|
643
|
+
const SENSOR_STATUS = new Set(['satisfied', 'needs_revision', 'failed']);
|
|
644
|
+
|
|
645
|
+
/**
|
|
646
|
+
* BA-15 — a TYPED signal that a caller arbiter (the `refineLeaf.sensor` or the `opts.evaluate` verifier)
|
|
647
|
+
* broke (threw non-Halt, or returned a malformed verdict). Classified by `instanceof` + `.tag`, NOT by
|
|
648
|
+
* re-parsing an `Error.message` prefix — so an intermediate layer that rewords the message (the exact class
|
|
649
|
+
* of the prior loop.js HaltError-wrapping bug) or a Loop/provider error whose text happens to begin
|
|
650
|
+
* `broken-sensor: ` can never be mis-labeled. Module-local: it is always thrown AND caught inside recurse.js
|
|
651
|
+
* (never propagates to a caller), so it needs no `errors.js` entry or public export.
|
|
652
|
+
*/
|
|
653
|
+
class BrokenArbiterError extends Error {
|
|
654
|
+
/**
|
|
655
|
+
* @param {'broken-sensor'|'broken-verifier'} tag - which seam broke.
|
|
656
|
+
* @param {string} detail - what the arbiter did (surfaced as `receipts.blockerDetail`).
|
|
657
|
+
* @param {{cause?: any}} [options] - `cause` preserves the original error (stack/type) for debugging.
|
|
658
|
+
*/
|
|
659
|
+
constructor(tag, detail, options = {}) {
|
|
660
|
+
super(`${tag}: ${detail}`, options.cause !== undefined ? { cause: options.cause } : undefined);
|
|
661
|
+
this.name = 'BrokenArbiterError';
|
|
662
|
+
this.tag = tag;
|
|
663
|
+
this.detail = detail;
|
|
664
|
+
}
|
|
665
|
+
}
|
|
666
|
+
|
|
667
|
+
/**
|
|
668
|
+
* BA-15 — validate a caller arbiter's return at a close seam (the `refineLeaf.sensor` AND the caller
|
|
669
|
+
* `opts.evaluate` verifier). A verdict is well-formed iff it is an object carrying a boolean `pass` OR a
|
|
670
|
+
* valid tri-state `status` (the two shapes `refine`/callers branch on). Returns `null` when well-formed,
|
|
671
|
+
* else a short description of the malformation ("named, never coerced" — a garbage verdict otherwise reads
|
|
672
|
+
* as pass:false with critique:null at the sensor seam, or rides a converged-shaped return at the verify slot).
|
|
673
|
+
* @param {any} v
|
|
674
|
+
* @returns {string|null}
|
|
675
|
+
*/
|
|
676
|
+
function verdictShapeFault(v) {
|
|
677
|
+
if (v === null || typeof v !== 'object' || Array.isArray(v)) {
|
|
678
|
+
return `returned ${v === null ? 'null' : Array.isArray(v) ? 'an array' : `a ${typeof v}`}`;
|
|
679
|
+
}
|
|
680
|
+
// A PRESENT `pass` counts whatever its type (`1`/`0`/`'yes'` are a long-standing yes-no convention, and
|
|
681
|
+
// `refine` has always branched on its TRUTHINESS). BA-15 exists to catch a verdict carrying NO usable
|
|
682
|
+
// signal — not to reject one that answers clearly in a different dialect: demanding a strict boolean
|
|
683
|
+
// silently flipped previously-CONVERGING adopter sensors to a permanent first-attempt block.
|
|
684
|
+
if (v.pass != null || SENSOR_STATUS.has(v.status)) return null;
|
|
685
|
+
const keys = Object.keys(v).slice(0, 5).join(', ');
|
|
686
|
+
return `returned an object with neither a usable \`pass\` nor a valid \`status\` (keys: ${keys || 'none'})`;
|
|
687
|
+
}
|
|
688
|
+
|
|
689
|
+
/** Upper bound on a `blockerDetail` fragment — it rides into receipts and, via a wired gate, onto disk. */
|
|
690
|
+
const DETAIL_MAX = 200;
|
|
691
|
+
|
|
692
|
+
/**
|
|
693
|
+
* BA-15 — a readable one-liner for ANYTHING thrown by a caller arbiter. `String(err)` on a non-Error throw
|
|
694
|
+
* (a test harness rejecting with a raw `{code:'ENOENT', path}` result is the common case) yields the useless
|
|
695
|
+
* `[object Object]`, defeating `blockerDetail`'s entire purpose: naming what broke so the operator can fix it.
|
|
696
|
+
* @param {any} err
|
|
697
|
+
* @returns {string}
|
|
698
|
+
*/
|
|
699
|
+
function describeThrown(err) {
|
|
700
|
+
const clamp = (s) => (s.length > DETAIL_MAX ? `${s.slice(0, DETAIL_MAX)}… (truncated)` : s);
|
|
701
|
+
if (err instanceof Error && typeof err.message === 'string' && err.message) return clamp(err.message);
|
|
702
|
+
if (err === null || typeof err !== 'object') return clamp(String(err));
|
|
703
|
+
// Deliberately NOT a whole-object dump. A thrown non-Error is typically a spawn/exec RESULT, which routinely
|
|
704
|
+
// carries a full stdout buffer and an env snapshot — and `blockerDetail` rides into `receipts`, which a wired
|
|
705
|
+
// gate serializes VERBATIM into a plaintext audit log (the F16/BA-1 lesson: never let caller data of unknown
|
|
706
|
+
// shape reach the audit unfiltered). Take only the conventional diagnostic fields, clamped.
|
|
707
|
+
try {
|
|
708
|
+
const picked = ['name', 'code', 'errno', 'syscall', 'path', 'status', 'signal', 'message']
|
|
709
|
+
.filter(k => typeof err[k] === 'string' || typeof err[k] === 'number')
|
|
710
|
+
.map(k => `${k}=${String(err[k])}`)
|
|
711
|
+
.join(' ');
|
|
712
|
+
if (picked) return clamp(picked);
|
|
713
|
+
} catch { /* a throwing accessor / Proxy trap — fall through to the type tag */ }
|
|
714
|
+
return Object.prototype.toString.call(err);
|
|
715
|
+
}
|
|
716
|
+
|
|
717
|
+
/**
|
|
718
|
+
* BA-15 — run a caller arbiter (the `refineLeaf.sensor` or the `opts.evaluate` verifier) and NAME a broken one
|
|
719
|
+
* rather than let it launder. A non-`HaltError` throw or a malformed return (per {@link verdictShapeFault}) is
|
|
720
|
+
* re-thrown as a typed {@link BrokenArbiterError} the call sites classify by `instanceof` + `.tag` (NEVER by
|
|
721
|
+
* re-parsing the message text); a `HaltError` passes straight through (clean
|
|
722
|
+
* governance exit, BA-2). One helper so the sensor and verifier seams stay byte-identical (same guidance
|
|
723
|
+
* string, same fault taxonomy) — a divergence between them was the copy-paste risk this replaces.
|
|
724
|
+
* A verdict that carries a valid `status` but no boolean `pass` is NORMALIZED to add `pass = status ===
|
|
725
|
+
* 'satisfied'` (the same derivation as `evaluator.js`): the advertised `{status}` contract must actually
|
|
726
|
+
* WORK — `refine.js` stops the leaf on `verdict.pass`, so a bare `{status:'satisfied'}` would otherwise never
|
|
727
|
+
* satisfy, burn every iteration, and report `passed:false` (a satisfied close mislabeled as non-recovery).
|
|
728
|
+
* @template {{pass?: boolean, status?: string}} T
|
|
729
|
+
* @param {'broken-sensor'|'broken-verifier'} tag - the fault channel, carried on the thrown error's `.tag`.
|
|
730
|
+
* @param {() => (T | Promise<T>)} call - invokes the arbiter (already bound to its result/ctx args).
|
|
731
|
+
* @returns {Promise<T>} the arbiter's well-formed verdict (with `pass` derived from `status` when absent).
|
|
732
|
+
*/
|
|
733
|
+
async function runArbiter(tag, call) {
|
|
734
|
+
const who = tag === 'broken-sensor' ? 'sensor' : 'evaluate';
|
|
735
|
+
const shape = `a ${who === 'sensor' ? 'sensor' : 'verifier'} must return {pass} or {status: 'satisfied'|'needs_revision'|'failed'}`;
|
|
736
|
+
/** A throw from the arbiter's own body (sync or async rejection) — "the sensor threw". */
|
|
737
|
+
const threw = (err) => {
|
|
738
|
+
if (err instanceof HaltError || err instanceof BrokenArbiterError) return err;
|
|
739
|
+
// `cause` keeps the original error (stack + type) reachable for debugging; `detail` stays the short label.
|
|
740
|
+
return new BrokenArbiterError(tag, `${who} threw: ${describeThrown(err)}`, { cause: err });
|
|
741
|
+
};
|
|
742
|
+
/** A throw from READING the value the arbiter returned — "the verdict is unreadable", a different fault. */
|
|
743
|
+
const unreadable = (err) => {
|
|
744
|
+
if (err instanceof HaltError || err instanceof BrokenArbiterError) return err;
|
|
745
|
+
return new BrokenArbiterError(tag, `${who} returned a verdict whose properties could not be read: ${describeThrown(err)} — ${shape}`, { cause: err });
|
|
746
|
+
};
|
|
747
|
+
|
|
748
|
+
// Typed `any` deliberately: this block probes an UNTRUSTED caller return (it may be a promise, a plain
|
|
749
|
+
// verdict, or a hostile Proxy), so the generic `T | Promise<T>` narrowing does not apply until it is settled.
|
|
750
|
+
/** @type {any} */
|
|
751
|
+
let raw;
|
|
752
|
+
try {
|
|
753
|
+
raw = call();
|
|
754
|
+
} catch (err) { throw threw(err); }
|
|
755
|
+
|
|
756
|
+
// `await`ing directly would conflate two different faults: the await PROBES `.then` on the returned value,
|
|
757
|
+
// so an accessor-backed/Proxy verdict throws during the await and gets reported as "the sensor threw" —
|
|
758
|
+
// sending the operator hunting a `throw` in a sensor that returned perfectly normally. Probe the thenable
|
|
759
|
+
// separately (a throw HERE is the returned value being unreadable) and only then await (a rejection THERE
|
|
760
|
+
// is genuinely the arbiter's body failing).
|
|
761
|
+
let thenable;
|
|
762
|
+
try {
|
|
763
|
+
thenable = raw !== null && (typeof raw === 'object' || typeof raw === 'function') && typeof raw.then === 'function';
|
|
764
|
+
} catch (err) { throw unreadable(err); }
|
|
765
|
+
|
|
766
|
+
/** @type {any} */
|
|
767
|
+
let v = raw;
|
|
768
|
+
if (thenable) {
|
|
769
|
+
try {
|
|
770
|
+
v = await raw;
|
|
771
|
+
} catch (err) { throw threw(err); }
|
|
772
|
+
}
|
|
773
|
+
// The SHAPE INSPECTION is guarded SEPARATELY: reading `.status`/`.pass`/`Object.keys` on a returned Proxy or
|
|
774
|
+
// accessor-backed object can itself throw. Unguarded that escapes untyped (the very uncaught crash this seam
|
|
775
|
+
// prevents, one step later) — but folding it into the call's own catch is also wrong: it reports "the sensor
|
|
776
|
+
// threw" for a sensor that RETURNED NORMALLY, sending the operator hunting a `throw` that does not exist.
|
|
777
|
+
// The fault is in the returned VALUE, so it is named as such.
|
|
778
|
+
try {
|
|
779
|
+
const fault = verdictShapeFault(v);
|
|
780
|
+
if (fault) throw new BrokenArbiterError(tag, `${who} ${fault} — ${shape}`);
|
|
781
|
+
// Derive `pass` from a status-only verdict so the advertised `{status}` shape actually gates `refine`, which
|
|
782
|
+
// branches on `verdict.pass`. NEVER mutate the caller's object — and never flatten it either: an object
|
|
783
|
+
// spread copies OWN enumerable properties only, so a class-instance verdict whose `status`/`critique` are
|
|
784
|
+
// PROTOTYPE getters passes the shape check above and then comes out the other side with those fields
|
|
785
|
+
// ERASED (critique lost ⇒ every retry re-sends the plain task with zero feedback — the exact burn BA-15
|
|
786
|
+
// exists to prevent). Copy descriptors onto the SAME prototype so accessor-backed fields survive.
|
|
787
|
+
if (v.pass == null && typeof v.status === 'string') {
|
|
788
|
+
const copy = Object.create(Object.getPrototypeOf(v), Object.getOwnPropertyDescriptors(v));
|
|
789
|
+
Object.defineProperty(copy, 'pass', {
|
|
790
|
+
value: v.status === 'satisfied', writable: true, enumerable: true, configurable: true,
|
|
791
|
+
});
|
|
792
|
+
return /** @type {T} */ (copy);
|
|
793
|
+
}
|
|
794
|
+
return v;
|
|
795
|
+
} catch (err) { throw unreadable(err); }
|
|
796
|
+
}
|
|
797
|
+
|
|
798
|
+
/**
|
|
799
|
+
* BA-15 — surface a CHILD's blocker at the PARENT. A nested tree aggregates a dead child into
|
|
800
|
+
* `{incomplete, missingSlices}`; without this the child's `broken-sensor` (or `governance-deny`) label is
|
|
801
|
+
* dropped at the first parent, so a top-level caller branching on `result.blocker` sees nothing and debugs
|
|
802
|
+
* the model instead of its own sensor — the exact laundering BA-15 exists to close, reintroduced one level up.
|
|
803
|
+
* `broken-sensor` wins over `governance-deny`: it names a fault in the CALLER's own code, which is both more
|
|
804
|
+
* actionable and cheaper to fix than a gate decision.
|
|
805
|
+
*
|
|
806
|
+
* `blockerTask` names WHICH descendant broke. Without it a nested run reports "a sensor broke" with no way to
|
|
807
|
+
* find which one — half a fix. (It is also why the label is NOT stamped onto the parent's own `blocker`; see
|
|
808
|
+
* {@link incompleteWithBlocker}.)
|
|
809
|
+
*
|
|
810
|
+
* `'broken-verifier'` is deliberately NOT matched here: `forChild` strips `evaluate`, so a child never runs a
|
|
811
|
+
* caller verifier and no child node can carry that label. Matching it would be unreachable code asserting a
|
|
812
|
+
* capability that does not exist.
|
|
813
|
+
* @param {RecurseNode[]} spawned - this node's child receipts.
|
|
814
|
+
* @returns {{blocker: string, blockerDetail?: string, blockerTask?: string}|null}
|
|
815
|
+
*/
|
|
816
|
+
function inheritedBlocker(spawned) {
|
|
817
|
+
const pick = spawned.find(c => c.incomplete && c.blocker === 'broken-sensor')
|
|
818
|
+
|| spawned.find(c => c.incomplete && c.blocker);
|
|
819
|
+
if (!pick || !pick.blocker) return null;
|
|
820
|
+
return {
|
|
821
|
+
blocker: pick.blocker,
|
|
822
|
+
...(pick.blockerDetail && { blockerDetail: pick.blockerDetail }),
|
|
823
|
+
...(pick.task && { blockerTask: pick.task }),
|
|
824
|
+
};
|
|
825
|
+
}
|
|
826
|
+
|
|
827
|
+
/**
|
|
828
|
+
* BA-15 — the ONE shared "this node is incomplete because a descendant was" return. Used by every aggregating
|
|
829
|
+
* path (worker / partition / fanout) on BOTH its `missingSlices` branch AND its `HaltError` catch: the halt
|
|
830
|
+
* branches originally skipped the inherit entirely, so a gate tripping AFTER a broken-sensor child (e.g. mid-
|
|
831
|
+
* synthesize) dropped the label and the caller read a governance problem where their own sensor had crashed.
|
|
832
|
+
*
|
|
833
|
+
* The inherited label rides the RETURNED result (the caller-facing API — that surfacing is the whole point)
|
|
834
|
+
* but is recorded on the node as `blockerFrom`, NOT by overwriting `node.blocker`. `node.blocker` means "THIS
|
|
835
|
+
* node's own arbiter/Loop broke"; stamping a descendant's label onto every ancestor made the receipts tree
|
|
836
|
+
* accuse nodes whose sensor never ran, and re-labelled a parent `governance-deny` when only one child was
|
|
837
|
+
* denied — pointing the operator at the wrong node to re-gate.
|
|
838
|
+
* @param {RecurseNode} node
|
|
839
|
+
* @param {any} best
|
|
840
|
+
* @param {{missingSlices?: string[]}} [extra] - extra fields for the non-halt aggregation branch.
|
|
841
|
+
* @returns {RecurseResult}
|
|
842
|
+
*/
|
|
843
|
+
function incompleteWithBlocker(node, best, extra = {}) {
|
|
844
|
+
const inherited = inheritedBlocker(node.spawned);
|
|
845
|
+
if (inherited) node.blockerFrom = inherited;
|
|
846
|
+
return /** @type {RecurseResult} */ ({ incomplete: true, best, ...extra, ...(inherited || {}), receipts: node });
|
|
847
|
+
}
|
|
848
|
+
|
|
592
849
|
/**
|
|
593
850
|
* BA-8 leaf-refine — run a DEFINITE leaf as a bounded generate→sense→regenerate loop (relayfact F17). Reuses the
|
|
594
851
|
* existing `refine.js` primitive (the Outcomes iterate→grade→revise port): each attempt is a FRESH leaf Loop
|
|
@@ -608,10 +865,25 @@ async function recurse(task, ctx = {}, opts = {}) {
|
|
|
608
865
|
async function recurseRefineLeaf(task, ctx, opts, state) {
|
|
609
866
|
const { provider, system, handleTools, depth, critical, node, sensor } = state;
|
|
610
867
|
node.model = provider.model || null;
|
|
611
|
-
const cfg = /** @type {{maxIterations?: number, temperatures?: number[]}} */ (opts.refineLeaf || {});
|
|
868
|
+
const cfg = /** @type {{maxIterations?: number, temperatures?: number[], rejectedBuffer?: boolean}} */ (opts.refineLeaf || {});
|
|
612
869
|
const temps = Array.isArray(cfg.temperatures) && cfg.temperatures.length ? cfg.temperatures : DEFAULT_REFINE_TEMPS;
|
|
613
870
|
const maxIterations = Number.isInteger(cfg.maxIterations) && /** @type {number} */ (cfg.maxIterations) > 0
|
|
614
871
|
? /** @type {number} */ (cfg.maxIterations) : temps.length;
|
|
872
|
+
// BA-14 rejected-attempt buffer: surface the model's OWN prior failed attempts verbatim ("you wrote these,
|
|
873
|
+
// they failed X — write something DIFFERENT") — a SkillOpt-shaped directed-diversity lever. `rejectedBuffer`:
|
|
874
|
+
// `true` = force on (also on temperature-accepting models); `false` = force off; unset = ADAPTIVE (engage only
|
|
875
|
+
// once a prior attempt's temperature was DROPPED, i.e. a temperature-fixed model where BA-8 escalation is inert
|
|
876
|
+
// and the buffer is the only lever — ba14 D>C). Escalation and the buffer are ANTAGONISTIC (ba14b: temp
|
|
877
|
+
// monotonically degrades the buffer 100→70→50% across 0.2/0.7/1.0), so when the buffer engages we HOLD temps[0].
|
|
878
|
+
const bufferForced = cfg.rejectedBuffer === true;
|
|
879
|
+
const bufferDisabled = cfg.rejectedBuffer === false;
|
|
880
|
+
let bufferUsed = false; // receipt: did any iteration actually inject the ledger?
|
|
881
|
+
const LEDGER_ENTRY_CAP = 600, LEDGER_WHY_CAP = 400;
|
|
882
|
+
const formatLedger = (/** @type {Array<{result: any, verdict: any}>} */ history) => history.map((h, i) => {
|
|
883
|
+
const code = String(h.result == null ? '' : h.result).replace(/```[a-zA-Z]*\n?/g, '').trim().slice(0, LEDGER_ENTRY_CAP);
|
|
884
|
+
const why = h.verdict && typeof h.verdict.critique === 'string' ? h.verdict.critique.slice(0, LEDGER_WHY_CAP) : '';
|
|
885
|
+
return `--- Rejected attempt ${i + 1} (already failed — do NOT reproduce) ---\n${code}${why ? `\nFailed: ${why}` : ''}`;
|
|
886
|
+
}).join('\n\n');
|
|
615
887
|
|
|
616
888
|
// A refine leaf runs N Loops, so its receipts.tokens SUMS every attempt's spend (not just the last) — the
|
|
617
889
|
// honest cost of the node. The 4-tier tokens object (`{input,output,cacheCreation,cacheRead}`, loop.js) is flat
|
|
@@ -630,10 +902,23 @@ async function recurseRefineLeaf(task, ctx, opts, state) {
|
|
|
630
902
|
// a dropped attempt is stored as `null` ("provider default"). Indexed by iteration (refine calls once each).
|
|
631
903
|
/** @type {(number|null)[]} */
|
|
632
904
|
const effectiveTemps = [];
|
|
905
|
+
// BA-15/BA-5: the last attempt's text, kept OUTSIDE refine so a broken-sensor stop can still preserve the
|
|
906
|
+
// model's work — when the ARBITER breaks, the work was never judged; destroying it would punish the model
|
|
907
|
+
// for the caller's fault (refine's own history is lost on the throw).
|
|
908
|
+
/** @type {string|null} */
|
|
909
|
+
let lastAttemptText = null;
|
|
633
910
|
// One attempt = a fresh leaf Loop (no spawn tool: a retry is a direct correction, not a re-decomposition) at the
|
|
634
911
|
// iteration's temperature, with the GAP fed forward as fresh feedback. A governance halt → throw so refine stops.
|
|
635
|
-
const attempt = async ({ iteration, critique }) => {
|
|
636
|
-
const
|
|
912
|
+
const attempt = async ({ iteration, critique, history }) => {
|
|
913
|
+
const hist = Array.isArray(history) ? history : [];
|
|
914
|
+
// ADAPTIVE trigger: a prior attempt whose temperature the model rejected (BA-10) records `null` in
|
|
915
|
+
// effectiveTemps → escalation is inert on this (temperature-fixed) model, so engage the buffer. Forced-on
|
|
916
|
+
// engages regardless (incl. temperature-accepting models). Needs ≥1 prior attempt to have something to buffer.
|
|
917
|
+
const tempDropped = effectiveTemps.some((t) => t === null);
|
|
918
|
+
const useBuffer = hist.length > 0 && !bufferDisabled && (bufferForced || tempDropped);
|
|
919
|
+
// ba14b: temperature is antagonistic to the buffer's directed diversity — HOLD temps[0] when it engages;
|
|
920
|
+
// otherwise escalate (BA-8, the no-memory lever). On a temperature-fixed model both collapse to the default.
|
|
921
|
+
const temperature = useBuffer ? temps[0] : temps[Math.min(iteration, temps.length - 1)];
|
|
637
922
|
const loop = new Loop({
|
|
638
923
|
provider, system,
|
|
639
924
|
policy: ctx.policy || undefined,
|
|
@@ -642,24 +927,44 @@ async function recurseRefineLeaf(task, ctx, opts, state) {
|
|
|
642
927
|
throwOnError: false,
|
|
643
928
|
});
|
|
644
929
|
const base = withContext(task, opts.context);
|
|
645
|
-
|
|
646
|
-
|
|
647
|
-
|
|
930
|
+
let userText;
|
|
931
|
+
if (useBuffer) {
|
|
932
|
+
bufferUsed = true;
|
|
933
|
+
userText = `${base}\n\nYou have already tried the following and each FAILED. Do NOT reproduce them — write a STRUCTURALLY DIFFERENT result that passes ALL checks:\n\n${formatLedger(hist)}`;
|
|
934
|
+
} else if (critique) {
|
|
935
|
+
userText = `${base}\n\nYour previous attempt FAILED these checks:\n${critique}\n\nReturn a corrected result that passes ALL of them.`;
|
|
936
|
+
} else {
|
|
937
|
+
userText = base;
|
|
938
|
+
}
|
|
648
939
|
const out = await loop.run([{ role: 'user', content: userText }], handleTools, { ctx: auditSafeCtx(ctx, { depth }), temperature });
|
|
649
940
|
// `temperatureDropped` is set on the Loop result only when the model rejected the requested temperature
|
|
650
941
|
// (BA-10); it's absent on the error/halt return shapes, so read it through a narrow cast.
|
|
651
942
|
const dropped = /** @type {{temperatureDropped?: boolean}} */ (out).temperatureDropped;
|
|
652
943
|
effectiveTemps[iteration] = dropped ? null : temperature;
|
|
653
944
|
accrueTokens(out.metrics ? out.metrics.tokens : null);
|
|
945
|
+
// BA-5: capture BEFORE the throws. The Loop returns its last non-empty text on EVERY terminating path
|
|
946
|
+
// (halt/deny/truncation/refusal), so a throw placed above this line would discard the text of the very
|
|
947
|
+
// attempt that terminated — leaving `best:null` on a FIRST-attempt halt, the exact loss the catch-branch
|
|
948
|
+
// preservation exists to prevent. (Caught by review: the earlier tests only halted on attempt 2, where a
|
|
949
|
+
// prior clean attempt had already populated this.)
|
|
950
|
+
lastAttemptText = out.text || lastAttemptText;
|
|
654
951
|
if (typeof out.error === 'string' && out.error.startsWith('halt:')) throw new HaltError('refine-leaf attempt halted', { rule: out.error.slice('halt:'.length) });
|
|
655
952
|
if (out.error) throw new Error(out.error); // a non-halt worker fault → honest incomplete
|
|
656
953
|
return out.text;
|
|
657
954
|
};
|
|
658
955
|
|
|
956
|
+
// BA-15: the sensor call is WRAPPED (via runArbiter) so a broken arbiter is NAMED, never coerced. A non-Halt
|
|
957
|
+
// throw (the caller's test runner crashed — ENOENT, syntax error in the harness) and a malformed return are
|
|
958
|
+
// the same fault class: "didn't judge", which must never collapse into "judged-and-failed" (the model's
|
|
959
|
+
// fault) or a bare {incomplete} (indistinguishable from a provider death). The throw stops refine at the
|
|
960
|
+
// FIRST broken close — retrying against a broken arbiter burns every remaining attempt for nothing (each
|
|
961
|
+
// retry would carry critique:null, i.e. the plain task again). HaltError passes through (governance, BA-2).
|
|
962
|
+
const evaluate = (result, c) => runArbiter('broken-sensor', () => sensor(result, { task, context: opts.context, contract: c.contract }));
|
|
963
|
+
|
|
659
964
|
try {
|
|
660
965
|
const outcome = await refine({
|
|
661
966
|
attempt,
|
|
662
|
-
evaluate
|
|
967
|
+
evaluate,
|
|
663
968
|
contract: typeof opts.contract === 'string' ? opts.contract : undefined,
|
|
664
969
|
maxIterations,
|
|
665
970
|
});
|
|
@@ -667,35 +972,59 @@ async function recurseRefineLeaf(task, ctx, opts, state) {
|
|
|
667
972
|
// `temperatures` = the EFFECTIVE temps (BA-10): a `null` marks an attempt whose requested temperature the
|
|
668
973
|
// model rejected and ran at its default — so the receipt never claims a value the model ignored. On a
|
|
669
974
|
// temperature-accepting model this equals the requested `temps.slice(0, iterations)` (byte-identical receipt).
|
|
670
|
-
node.refineLeaf = { iterations: outcome.iterations, passed: !!(outcome.verdict && outcome.verdict.pass), temperatures: effectiveTemps.slice(0, outcome.iterations) };
|
|
975
|
+
node.refineLeaf = { iterations: outcome.iterations, passed: !!(outcome.verdict && outcome.verdict.pass), temperatures: effectiveTemps.slice(0, outcome.iterations), rejectedBuffer: bufferUsed };
|
|
671
976
|
const result = outcome.result;
|
|
672
977
|
|
|
673
978
|
// Optional rubric layer on top of the deterministic sensor (RC-7): forced for critical, or a contract/override.
|
|
674
979
|
const wantVerify = critical || typeof opts.contract === 'string' || typeof opts.evaluate === 'function';
|
|
675
980
|
if (wantVerify) {
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
return { result, verdict, receipts: node };
|
|
981
|
+
// `return await` (not a bare `return`) is load-bearing here — see verifyOrBlock's JSDoc.
|
|
982
|
+
return await verifyOrBlock(task, result, ctx, opts, node);
|
|
679
983
|
}
|
|
680
984
|
// No rubric layer ⇒ the sensor's final verdict IS the node verdict (a non-pass is surfaced, not hidden).
|
|
681
985
|
node.verdict = outcome.verdict || null;
|
|
682
986
|
return { result, verdict: outcome.verdict || null, receipts: node };
|
|
683
987
|
} catch (err) {
|
|
684
988
|
node.tokens = tokensSum; // record whatever attempts DID spend, on both the halt and fault paths
|
|
989
|
+
// The refineLeaf receipt must ride EVERY terminating path, not just the clean one (same invariant as BA-10's
|
|
990
|
+
// `temperatureDropped`): a leaf that ran attempts then halted/faulted still spent tokens and may have engaged
|
|
991
|
+
// the buffer. `effectiveTemps[iteration]` is set BEFORE each attempt's throw, so it reflects every attempt made.
|
|
992
|
+
// NOT unconditional: the refine loop may have COMPLETED (receipt already written, possibly `passed:true`) and
|
|
993
|
+
// the throw come from the verify layer below it — clobbering that to `passed:false` would report a sensor that
|
|
994
|
+
// never closed when it did, misattributing a verify-slot halt to a non-converging leaf.
|
|
995
|
+
if (!node.refineLeaf) {
|
|
996
|
+
node.refineLeaf = { iterations: effectiveTemps.length, passed: false, temperatures: effectiveTemps.slice(), rejectedBuffer: bufferUsed };
|
|
997
|
+
}
|
|
998
|
+
// BA-5: EVERY terminating branch preserves the model's last non-empty attempt (`lastAttemptText`), never
|
|
999
|
+
// `null` — matching the plain-worker path (`best: out.text || null`) and the "bounds PRESERVE work"
|
|
1000
|
+
// invariant. A refine leaf is the sole cross-attempt text channel in a ralph-style retry; dropping it on a
|
|
1001
|
+
// halt/deny/fault (as this path did pre-BA-15) loses attempt N's only bridge to attempt N+1.
|
|
685
1002
|
if (err instanceof HaltError) {
|
|
686
1003
|
node.halted = true;
|
|
687
1004
|
node.incomplete = true;
|
|
688
|
-
return { incomplete: true, best:
|
|
1005
|
+
return { incomplete: true, best: lastAttemptText, receipts: node };
|
|
689
1006
|
}
|
|
690
1007
|
// BA-11: a deny-spin inside a refine attempt (the Loop short-circuited after N consecutive governance
|
|
691
1008
|
// denials, rethrown at recurse.js as `denied:<tool>`) is a LABELED governance block, not a model fault.
|
|
692
1009
|
if (typeof err?.message === 'string' && err.message.startsWith('denied:')) {
|
|
693
1010
|
node.incomplete = true;
|
|
694
1011
|
node.blocker = 'governance-deny';
|
|
695
|
-
return { incomplete: true, best:
|
|
1012
|
+
return { incomplete: true, best: lastAttemptText, blocker: 'governance-deny', receipts: node };
|
|
1013
|
+
}
|
|
1014
|
+
// BA-15: the caller's SENSOR broke — a faulty arbiter, not a model failure. Named (like BA-11's
|
|
1015
|
+
// governance-deny) so the caller fixes the sensor instead of debugging the model; `best` preserves the
|
|
1016
|
+
// model's last non-empty attempt (BA-5) — the arbiter, not the model, is at fault.
|
|
1017
|
+
if (err instanceof BrokenArbiterError && err.tag === 'broken-sensor') {
|
|
1018
|
+
node.incomplete = true;
|
|
1019
|
+
node.blocker = 'broken-sensor';
|
|
1020
|
+
node.blockerDetail = err.detail;
|
|
1021
|
+
// `blockerDetail` rides the RESULT too, not just receipts: it is the actionable half of the label (WHAT
|
|
1022
|
+
// the sensor did), and a parent reading a child's return — or a caller branching on the documented
|
|
1023
|
+
// `{blocker}` shape — should not have to walk the receipts tree to get it.
|
|
1024
|
+
return { incomplete: true, best: lastAttemptText, blocker: 'broken-sensor', blockerDetail: err.detail, receipts: node };
|
|
696
1025
|
}
|
|
697
1026
|
node.incomplete = true;
|
|
698
|
-
return { incomplete: true, best:
|
|
1027
|
+
return { incomplete: true, best: lastAttemptText, receipts: node };
|
|
699
1028
|
}
|
|
700
1029
|
}
|
|
701
1030
|
|
|
@@ -746,6 +1075,10 @@ async function recurseScan(task, ctx, opts, state) {
|
|
|
746
1075
|
};
|
|
747
1076
|
}
|
|
748
1077
|
|
|
1078
|
+
// BA-5: hoisted so a halt thrown BELOW the scan (e.g. from the verify slot) still returns the finished,
|
|
1079
|
+
// code-counted result as `best` instead of destroying it — re-running a scan re-pays every window judge call.
|
|
1080
|
+
/** @type {{count: number, matchedIds: string[]}|null} */
|
|
1081
|
+
let scanResult = null;
|
|
749
1082
|
try {
|
|
750
1083
|
const scan = await scanCount(task, corpus, {
|
|
751
1084
|
provider,
|
|
@@ -758,6 +1091,7 @@ async function recurseScan(task, ctx, opts, state) {
|
|
|
758
1091
|
node.scan = { window: scan.window, passes: scan.passes, scanned: scan.scanned, matched: scan.count };
|
|
759
1092
|
// Structured, CODE-counted result — the count is authoritative; matchedIds carry the evidence (RC-10).
|
|
760
1093
|
const result = { count: scan.count, matchedIds: scan.matchedIds };
|
|
1094
|
+
scanResult = result;
|
|
761
1095
|
|
|
762
1096
|
// RC-9: a dead window means we did NOT see every slice → the count is a floor, not the answer. Report it
|
|
763
1097
|
// incomplete with the partial as `best`, never a clean pass over a hole.
|
|
@@ -770,16 +1104,21 @@ async function recurseScan(task, ctx, opts, state) {
|
|
|
770
1104
|
// structured count against the goal/contract (an isolated grader, never the scanner itself).
|
|
771
1105
|
const wantVerify = critical || typeof opts.contract === 'string' || typeof opts.evaluate === 'function';
|
|
772
1106
|
if (wantVerify) {
|
|
773
|
-
|
|
774
|
-
|
|
775
|
-
return { result, verdict, receipts: node };
|
|
1107
|
+
// `return await` (not a bare `return`) is load-bearing here — see verifyOrBlock's JSDoc.
|
|
1108
|
+
return await verifyOrBlock(task, result, ctx, opts, node);
|
|
776
1109
|
}
|
|
777
1110
|
return { result, verdict: null, receipts: node };
|
|
778
1111
|
} catch (err) {
|
|
779
1112
|
if (err instanceof HaltError) {
|
|
780
1113
|
node.halted = true;
|
|
781
1114
|
node.incomplete = true;
|
|
782
|
-
|
|
1115
|
+
// BA-5, PARTIAL by construction: a halt from the VERIFY slot lands here with the scan already finished,
|
|
1116
|
+
// so the counted result is returned instead of null (it previously survived only in receipts, forcing a
|
|
1117
|
+
// full re-scan). A halt DURING `scanCount` — the likelier case, since a token cap trips after many window
|
|
1118
|
+
// judge calls — still yields `null`: `scanCount` throws without surfacing the windows it did judge, so
|
|
1119
|
+
// there is nothing here to preserve. Fixing that needs `scanCount` to return its partial union on halt
|
|
1120
|
+
// (a retrieval-side change, not this seam's); tracked as a known limit rather than papered over here.
|
|
1121
|
+
return { incomplete: true, best: scanResult, receipts: node };
|
|
783
1122
|
}
|
|
784
1123
|
throw err;
|
|
785
1124
|
}
|
|
@@ -837,6 +1176,8 @@ async function recursePartition(task, ctx, opts, state) {
|
|
|
837
1176
|
// (never `undefined`) across every dispatch path.
|
|
838
1177
|
|
|
839
1178
|
const childResults = [];
|
|
1179
|
+
/** @type {{count: number, matchedIds: string[]}|null} */
|
|
1180
|
+
let partitionResult = null;
|
|
840
1181
|
try {
|
|
841
1182
|
// 2b) Pre-wave checkpoint — width (the cost) is now known. A governance HaltError halts BEFORE any worker
|
|
842
1183
|
// spends (bounds the burst to zero); a plain deny is advisory (allowlist-safe), same contract as fanout.
|
|
@@ -881,25 +1222,30 @@ async function recursePartition(task, ctx, opts, state) {
|
|
|
881
1222
|
if (child.incomplete) missingSlices.push(label);
|
|
882
1223
|
}
|
|
883
1224
|
const result = { count: matched.size, matchedIds: [...matched] };
|
|
1225
|
+
partitionResult = result; // BA-5: a halt below this (e.g. verify) returns the full result, not a count-only rebuild
|
|
884
1226
|
node.partition.matched = matched.size;
|
|
885
1227
|
|
|
886
1228
|
if (missingSlices.length > 0) {
|
|
887
1229
|
node.incomplete = true;
|
|
888
|
-
|
|
1230
|
+
// BA-15: carry a child's blocker up (see inheritedBlocker).
|
|
1231
|
+
return incompleteWithBlocker(node, result, { missingSlices });
|
|
889
1232
|
}
|
|
890
1233
|
const wantVerify = critical || typeof opts.contract === 'string' || typeof opts.evaluate === 'function';
|
|
891
1234
|
if (wantVerify) {
|
|
892
|
-
|
|
893
|
-
|
|
894
|
-
return { result, verdict, receipts: node };
|
|
1235
|
+
// `return await` (not a bare `return`) is load-bearing here — see verifyOrBlock's JSDoc.
|
|
1236
|
+
return await verifyOrBlock(task, result, ctx, opts, node);
|
|
895
1237
|
}
|
|
896
1238
|
return { result, verdict: null, receipts: node };
|
|
897
1239
|
} catch (err) {
|
|
898
1240
|
if (err instanceof HaltError) {
|
|
899
1241
|
node.halted = true;
|
|
900
1242
|
node.incomplete = true;
|
|
901
|
-
|
|
902
|
-
|
|
1243
|
+
// Prefer the finished result (a halt from the verify slot lands here with it already computed); else
|
|
1244
|
+
// rebuild a best-effort count from whatever slices did return (BA-5, never a bare null).
|
|
1245
|
+
const best = partitionResult
|
|
1246
|
+
|| (childResults.length ? { count: new Set(childResults.flatMap((v) => (v && Array.isArray(v.matchedIds) ? v.matchedIds : []))).size } : null);
|
|
1247
|
+
// BA-15: a halt must not swallow a child's broken-sensor label (see incompleteWithBlocker).
|
|
1248
|
+
return incompleteWithBlocker(node, best);
|
|
903
1249
|
}
|
|
904
1250
|
throw err;
|
|
905
1251
|
}
|
|
@@ -933,6 +1279,11 @@ async function recurseFanout(task, ctx, opts, state) {
|
|
|
933
1279
|
|
|
934
1280
|
const childResults = [];
|
|
935
1281
|
const contract = typeof opts.contract === 'string' ? opts.contract : null;
|
|
1282
|
+
// BA-5: declared OUTSIDE the try so a halt from the verify slot (which lands in the catch below, with the
|
|
1283
|
+
// reduce already computed and PAID FOR) returns the finished reduce instead of a lossy re-join of the raw
|
|
1284
|
+
// child strings — a different TYPE from the documented reduce output, and an 'merge' strategy's LLM call
|
|
1285
|
+
// thrown away. Mirrors `scanResult`/`partitionResult` on the sibling paths.
|
|
1286
|
+
let result;
|
|
936
1287
|
|
|
937
1288
|
try {
|
|
938
1289
|
// 1) Decompose into exactly `count` independent parallel steps (the NB-2 Planner seam). A non-Halt planner
|
|
@@ -1003,7 +1354,6 @@ async function recurseFanout(task, ctx, opts, state) {
|
|
|
1003
1354
|
// 4) NB-3 reduce over the slice results. Unlike Family A there is no parent closing turn, so we ALWAYS
|
|
1004
1355
|
// reduce: a `synthesize` FUNCTION is the deterministic code-reduce (§9.1); a string runs the built-in
|
|
1005
1356
|
// reducer; unset defaults to lossless `'concat'`. (`childResults` always has `count` entries.)
|
|
1006
|
-
let result;
|
|
1007
1357
|
if (typeof opts.synthesize === 'function') {
|
|
1008
1358
|
result = await opts.synthesize({ task, text: null, results: childResults, children: node.spawned, ctx });
|
|
1009
1359
|
} else {
|
|
@@ -1023,24 +1373,27 @@ async function recurseFanout(task, ctx, opts, state) {
|
|
|
1023
1373
|
// 5) Honest completeness (RC-9): any missing slice → incomplete, with the partial reduce as `best`.
|
|
1024
1374
|
if (missingSlices.length > 0) {
|
|
1025
1375
|
node.incomplete = true;
|
|
1026
|
-
|
|
1376
|
+
// BA-15: carry a child's blocker up (see inheritedBlocker).
|
|
1377
|
+
return incompleteWithBlocker(node, result, { missingSlices });
|
|
1027
1378
|
}
|
|
1028
1379
|
|
|
1029
1380
|
// 6) Verify (RC-7): forced for critical, or when a contract/override is supplied.
|
|
1030
1381
|
const wantVerify = critical || contract != null || typeof opts.evaluate === 'function';
|
|
1031
1382
|
if (wantVerify) {
|
|
1032
|
-
|
|
1033
|
-
|
|
1034
|
-
return { result, verdict, receipts: node };
|
|
1383
|
+
// `return await` (not a bare `return`) is load-bearing here — see verifyOrBlock's JSDoc.
|
|
1384
|
+
return await verifyOrBlock(task, result, ctx, opts, node);
|
|
1035
1385
|
}
|
|
1036
1386
|
return { result, verdict: null, receipts: node };
|
|
1037
1387
|
} catch (err) {
|
|
1038
1388
|
if (err instanceof HaltError) {
|
|
1039
1389
|
node.halted = true;
|
|
1040
1390
|
node.incomplete = true;
|
|
1041
|
-
//
|
|
1042
|
-
|
|
1043
|
-
|
|
1391
|
+
// BA-5: prefer the FINISHED reduce when the halt landed after it (the verify slot); only fall back to a
|
|
1392
|
+
// lossless join of the raw slices when the halt tripped before/during the reduce itself. Never a bare null.
|
|
1393
|
+
const rejoined = childResults.length ? childResults.filter(v => v !== '').join('\n\n') : null;
|
|
1394
|
+
const best = result !== undefined ? result : (rejoined || null);
|
|
1395
|
+
// BA-15: a halt must not swallow a child's broken-sensor label (see incompleteWithBlocker).
|
|
1396
|
+
return incompleteWithBlocker(node, best);
|
|
1044
1397
|
}
|
|
1045
1398
|
throw err;
|
|
1046
1399
|
}
|
|
@@ -1107,6 +1460,16 @@ function buildSpawnTool(ctx, opts, depth, maxDepth, node, childResults) {
|
|
|
1107
1460
|
// silently dropped or faked. The same declared value is collected for the NB-3 reducer.
|
|
1108
1461
|
const value = child.incomplete ? (child.best == null ? '' : child.best) : (child.result == null ? '' : child.result);
|
|
1109
1462
|
childResults.push(value);
|
|
1463
|
+
// BA-15: a child blocked by a BROKEN ARBITER must not read to the parent model as a generic failure.
|
|
1464
|
+
// Collapsed into a bare `[incomplete]`, the parent cannot tell "the model failed" from "the judge is
|
|
1465
|
+
// broken", so it re-spawns the identical subtask against the identical broken sensor — re-running a
|
|
1466
|
+
// full leaf attempt each time, up to the Loop's round limit. That is BA-15's own spend-burn ("retrying
|
|
1467
|
+
// against a broken arbiter carries zero feedback") reintroduced one level up, so the tool result says
|
|
1468
|
+
// so explicitly and tells the model not to retry.
|
|
1469
|
+
if (child.incomplete && child.blocker === 'broken-sensor') {
|
|
1470
|
+
const where = child.blockerTask ? ` (in sub-task: ${child.blockerTask})` : '';
|
|
1471
|
+
return `[blocked: broken-sensor] The CHECK that judges this subtask is itself broken${where} — ${child.blockerDetail || 'no detail available'}. Retrying will hit the same broken check: do NOT re-delegate this subtask; report it as blocked. Partial output: ${String(value)}`.trim();
|
|
1472
|
+
}
|
|
1110
1473
|
if (child.incomplete) return `[incomplete] ${String(value)}`.trim();
|
|
1111
1474
|
return String(value);
|
|
1112
1475
|
},
|
|
@@ -1130,7 +1493,19 @@ function verify(task, result, ctx, opts) {
|
|
|
1130
1493
|
// and an agentic critic needs the path to exercise the artifact. A caller `evaluate` gets the RAW task (it owns
|
|
1131
1494
|
// its own context); only the default isolated grader is contextualized.
|
|
1132
1495
|
if (typeof opts.evaluate === 'function') {
|
|
1133
|
-
|
|
1496
|
+
// BA-15 (verifier seam): the CALLER-supplied verifier is wrapped exactly like the refineLeaf sensor — a
|
|
1497
|
+
// non-Halt throw or a malformed return is a faulty ARBITER, tagged so the call sites label it (pre-fix a
|
|
1498
|
+
// throw crashed the whole run on the plain-worker path / laundered to a bare {incomplete} under refineLeaf,
|
|
1499
|
+
// and a garbage verdict rode a CONVERGED-shaped {result, verdict} out). The default Evaluator path below is
|
|
1500
|
+
// NOT wrapped: it constructs well-formed Verdicts by design, and its failures are provider-class faults.
|
|
1501
|
+
// Capture the narrowed reference in a const: `typeof opts.evaluate === 'function'` does NOT survive into the
|
|
1502
|
+
// nested async closure (TS re-widens `opts.evaluate` to possibly-undefined there → TS2722/TS18048).
|
|
1503
|
+
// BOUND to `opts`, because a bare `const evaluate = opts.evaluate` DETACHES the method: the call used to be
|
|
1504
|
+
// `opts.evaluate(...)` (receiver `opts`), and a caller passing a class method (`evaluate: grader.check`)
|
|
1505
|
+
// would suddenly get `this === undefined` and throw on its first `this.x` read — a working (if degraded)
|
|
1506
|
+
// integration flipped to a hard `broken-verifier` by an unrelated typecheck fix.
|
|
1507
|
+
const evaluate = opts.evaluate.bind(opts);
|
|
1508
|
+
return runArbiter('broken-verifier', () => evaluate(result, { contract, task }));
|
|
1134
1509
|
}
|
|
1135
1510
|
const provider = ctx.provider || opts.provider;
|
|
1136
1511
|
const evaluator = new Evaluator({ provider });
|
|
@@ -1145,4 +1520,39 @@ function verify(task, result, ctx, opts) {
|
|
|
1145
1520
|
);
|
|
1146
1521
|
}
|
|
1147
1522
|
|
|
1523
|
+
/**
|
|
1524
|
+
* BA-15 (verifier seam) — run the verify slot, converting a `BrokenArbiterError` into a LABELED
|
|
1525
|
+
* `{ incomplete, blocker:'broken-verifier' }` return with `best` preserving the result the arbiter failed to
|
|
1526
|
+
* judge (BA-5: the work exists — best-effort, not a graded pass). One helper so all five dispatch paths (worker /
|
|
1527
|
+
* refineLeaf / scan / partition / fanout) get identical semantics. Anything else (HaltError, a default-
|
|
1528
|
+
* Evaluator provider fault) rethrows to the caller's own catch, exactly as before.
|
|
1529
|
+
*
|
|
1530
|
+
* MUST be called as `return await verifyOrBlock(...)` from inside each caller's `try` — a bare
|
|
1531
|
+
* `return verifyOrBlock(...)` returns the promise and exits the `try` before it settles, so a verifier
|
|
1532
|
+
* `HaltError` would escape the caller's own catch instead of landing as a clean `{ incomplete, halted }`
|
|
1533
|
+
* (proven by `poc/ba15-broken-sensor.mjs` [E4]).
|
|
1534
|
+
* @param {string} task
|
|
1535
|
+
* @param {any} result
|
|
1536
|
+
* @param {RecurseCtx} ctx
|
|
1537
|
+
* @param {RecurseOptions} opts
|
|
1538
|
+
* @param {RecurseNode} node
|
|
1539
|
+
* @returns {Promise<RecurseResult>}
|
|
1540
|
+
*/
|
|
1541
|
+
async function verifyOrBlock(task, result, ctx, opts, node) {
|
|
1542
|
+
try {
|
|
1543
|
+
const verdict = await verify(task, result, ctx, opts);
|
|
1544
|
+
node.verdict = verdict;
|
|
1545
|
+
return { result, verdict, receipts: node };
|
|
1546
|
+
} catch (err) {
|
|
1547
|
+
// Only a typed BrokenArbiterError is a caller-verifier fault; everything else (HaltError, a default-
|
|
1548
|
+
// Evaluator provider fault) rethrows to the caller's own catch. Classified by type, never by message text.
|
|
1549
|
+
if (!(err instanceof BrokenArbiterError)) throw err;
|
|
1550
|
+
node.incomplete = true;
|
|
1551
|
+
node.blocker = 'broken-verifier';
|
|
1552
|
+
node.blockerDetail = err.detail;
|
|
1553
|
+
// `blockerDetail` rides the RESULT too (see the sensor-side sibling) — the actionable half of the label.
|
|
1554
|
+
return { incomplete: true, best: result, blocker: 'broken-verifier', blockerDetail: err.detail, receipts: node };
|
|
1555
|
+
}
|
|
1556
|
+
}
|
|
1557
|
+
|
|
1148
1558
|
module.exports = { recurse };
|
package/src/refine.d.ts
CHANGED
|
@@ -1,15 +1,25 @@
|
|
|
1
1
|
export type Verdict = import("./evaluator").Verdict;
|
|
2
2
|
export type RefineOptions = {
|
|
3
3
|
/**
|
|
4
|
-
* Build one generation. On iteration 0, `lastResult`/`critique` are null. Fresh-feedback
|
|
5
|
-
* consumer's to realize here: seed a NEW Loop with `{goal + critique}` rather than continuing the
|
|
6
|
-
* transcript — anchoring on a wrong answer defeats the independent verifier.
|
|
4
|
+
* Build one generation. On iteration 0, `lastResult`/`critique` are null and `history` is empty. Fresh-feedback
|
|
5
|
+
* (D6/A1) is the consumer's to realize here: seed a NEW Loop with `{goal + critique}` rather than continuing the
|
|
6
|
+
* failed transcript — anchoring on a wrong answer defeats the independent verifier. `history` is a fresh
|
|
7
|
+
* shallow COPY of every prior `{result, verdict}` in order (a SkillOpt rejected-attempt buffer, BA-14): a
|
|
8
|
+
* consumer MAY surface the failed attempts verbatim ("you already wrote these, they failed X — write something
|
|
9
|
+
* different"), the lever that recovers a temperature-fixed model's fixation rut where escalation is inert
|
|
10
|
+
* (`poc/ba14-rejected-buffer.mjs`: flat-temp 50%→100%). *Structurally* mutating the copy (push/splice/reorder)
|
|
11
|
+
* is safe; the `{result, verdict}` entries are SHARED references into refine's internal history (and the returned
|
|
12
|
+
* `outcome.history`), so treat them as READ-ONLY — deep-mutating an entry corrupts the outcome.
|
|
7
13
|
*/
|
|
8
14
|
attempt: (args: {
|
|
9
15
|
iteration: number;
|
|
10
16
|
lastResult: any;
|
|
11
17
|
critique: string | null;
|
|
12
18
|
contract: string | null;
|
|
19
|
+
history: Array<{
|
|
20
|
+
result: any;
|
|
21
|
+
verdict: Verdict;
|
|
22
|
+
}>;
|
|
13
23
|
}) => any;
|
|
14
24
|
/**
|
|
15
25
|
* Judge a result — typically `evaluator.evaluate(goal, result, { rubric, contract })`.
|
|
@@ -56,10 +66,16 @@ export type RefineOutcome = {
|
|
|
56
66
|
/** @typedef {import('./evaluator').Verdict} Verdict */
|
|
57
67
|
/**
|
|
58
68
|
* @typedef {object} RefineOptions
|
|
59
|
-
* @property {(args: {iteration: number, lastResult: any, critique: string|null, contract: string|null}) => any} attempt
|
|
60
|
-
* Build one generation. On iteration 0, `lastResult`/`critique` are null. Fresh-feedback
|
|
61
|
-
* consumer's to realize here: seed a NEW Loop with `{goal + critique}` rather than continuing the
|
|
62
|
-
* transcript — anchoring on a wrong answer defeats the independent verifier.
|
|
69
|
+
* @property {(args: {iteration: number, lastResult: any, critique: string|null, contract: string|null, history: Array<{result: any, verdict: Verdict}>}) => any} attempt
|
|
70
|
+
* Build one generation. On iteration 0, `lastResult`/`critique` are null and `history` is empty. Fresh-feedback
|
|
71
|
+
* (D6/A1) is the consumer's to realize here: seed a NEW Loop with `{goal + critique}` rather than continuing the
|
|
72
|
+
* failed transcript — anchoring on a wrong answer defeats the independent verifier. `history` is a fresh
|
|
73
|
+
* shallow COPY of every prior `{result, verdict}` in order (a SkillOpt rejected-attempt buffer, BA-14): a
|
|
74
|
+
* consumer MAY surface the failed attempts verbatim ("you already wrote these, they failed X — write something
|
|
75
|
+
* different"), the lever that recovers a temperature-fixed model's fixation rut where escalation is inert
|
|
76
|
+
* (`poc/ba14-rejected-buffer.mjs`: flat-temp 50%→100%). *Structurally* mutating the copy (push/splice/reorder)
|
|
77
|
+
* is safe; the `{result, verdict}` entries are SHARED references into refine's internal history (and the returned
|
|
78
|
+
* `outcome.history`), so treat them as READ-ONLY — deep-mutating an entry corrupts the outcome.
|
|
63
79
|
* @property {(result: any, ctx: {iteration: number, contract: string|null}) => (Verdict | Promise<Verdict>)} evaluate
|
|
64
80
|
* Judge a result — typically `evaluator.evaluate(goal, result, { rubric, contract })`.
|
|
65
81
|
* @property {string} [contract] - The shared definition of done (A3/D10), forwarded to BOTH `attempt` and
|
package/src/refine.js
CHANGED
|
@@ -4,10 +4,16 @@
|
|
|
4
4
|
|
|
5
5
|
/**
|
|
6
6
|
* @typedef {object} RefineOptions
|
|
7
|
-
* @property {(args: {iteration: number, lastResult: any, critique: string|null, contract: string|null}) => any} attempt
|
|
8
|
-
* Build one generation. On iteration 0, `lastResult`/`critique` are null. Fresh-feedback
|
|
9
|
-
* consumer's to realize here: seed a NEW Loop with `{goal + critique}` rather than continuing the
|
|
10
|
-
* transcript — anchoring on a wrong answer defeats the independent verifier.
|
|
7
|
+
* @property {(args: {iteration: number, lastResult: any, critique: string|null, contract: string|null, history: Array<{result: any, verdict: Verdict}>}) => any} attempt
|
|
8
|
+
* Build one generation. On iteration 0, `lastResult`/`critique` are null and `history` is empty. Fresh-feedback
|
|
9
|
+
* (D6/A1) is the consumer's to realize here: seed a NEW Loop with `{goal + critique}` rather than continuing the
|
|
10
|
+
* failed transcript — anchoring on a wrong answer defeats the independent verifier. `history` is a fresh
|
|
11
|
+
* shallow COPY of every prior `{result, verdict}` in order (a SkillOpt rejected-attempt buffer, BA-14): a
|
|
12
|
+
* consumer MAY surface the failed attempts verbatim ("you already wrote these, they failed X — write something
|
|
13
|
+
* different"), the lever that recovers a temperature-fixed model's fixation rut where escalation is inert
|
|
14
|
+
* (`poc/ba14-rejected-buffer.mjs`: flat-temp 50%→100%). *Structurally* mutating the copy (push/splice/reorder)
|
|
15
|
+
* is safe; the `{result, verdict}` entries are SHARED references into refine's internal history (and the returned
|
|
16
|
+
* `outcome.history`), so treat them as READ-ONLY — deep-mutating an entry corrupts the outcome.
|
|
11
17
|
* @property {(result: any, ctx: {iteration: number, contract: string|null}) => (Verdict | Promise<Verdict>)} evaluate
|
|
12
18
|
* Judge a result — typically `evaluator.evaluate(goal, result, { rubric, contract })`.
|
|
13
19
|
* @property {string} [contract] - The shared definition of done (A3/D10), forwarded to BOTH `attempt` and
|
|
@@ -49,7 +55,7 @@ async function refine(options) {
|
|
|
49
55
|
let lastVerdict = null;
|
|
50
56
|
|
|
51
57
|
for (let iteration = 0; iteration < maxIterations; iteration++) {
|
|
52
|
-
const result = await attempt({ iteration, lastResult, critique: lastVerdict ? lastVerdict.critique : null, contract });
|
|
58
|
+
const result = await attempt({ iteration, lastResult, critique: lastVerdict ? lastVerdict.critique : null, contract, history: history.slice() });
|
|
53
59
|
const verdict = await evaluate(result, { iteration, contract });
|
|
54
60
|
history.push({ result, verdict });
|
|
55
61
|
lastResult = result;
|