bullswarm 0.13.2 → 0.14.1

package/CHANGELOG.md

@@ -1,5 +1,85 @@
 # bullswarm changelog
 
+## 0.14.1 — the TUI survives its writer; steering lands or expires truthfully
+
+Proven on a goal-3 re-run (`d7xyg2`): 1 planner turn / 269 s (baseline 0.13.1: 1 / 294 s), planner context 6.2 k chars (from 32.7 k), auto-completed, deliverable verified, zero observation crashes — `docs/experiments/2026-08-29-dogfood-bullswarm-builds-bullswarm.md`.
+
+- Workflow `state.json`/`report.json`/`workflow.json` writes are atomic
+  (temp + rename, new `src/workflow/fsjson.js`): a concurrent reader can never
+  observe a half-written file. Earned: `workflow tui` crashed with
+  "Unterminated string in JSON at position 138968" parsing `state.json`
+  mid-write (observed twice, 2026-08-29).
+- Observation readers tolerate torn or missing JSON: the TUI keeps painting
+  the last good frame of the same run, and a render or key-handler error is
+  shown in the message line instead of killing the process and stranding the
+  terminal in alt-screen raw mode. Mutating commands (stop, approval) retry
+  the read once and then refuse loudly instead of silently dropping the
+  operator's command. `runs delete` treats an unreadable `state.json` as
+  ongoing (refuses without `--force`) rather than deleting a possibly-live run.
+- An action being re-run (repair round, re-verify, schema retry) reads as
+  `running` and its phase as `active` even when its previous round recorded
+  `ok:false`; a failed mark now means failed-and-not-being-retried. (User
+  report: the TUI showed ✗ "2/2 complete" beside a live spinner.)
+- Pending operator steering defers program self-completion: a clean program
+  with `completion: all-actions-ok` returns to the planner gate (event
+  `decision.completion_deferred`), which delivers the steer — instead of
+  auto-completing and silently discarding it (defect observed live:
+  0 `steering.delivered` events for a queued steer). Steering that can no
+  longer reach any gate is marked `expired_undelivered` with event
+  `steering.expired` at the terminal transition; interrupted runs keep their
+  queue for the resumed run's next gate.
+- Resume re-runs an action the interruption cancelled mid-flight instead of
+  re-planning around a phantom failure: cancelled actions and the dependents
+  blocked only by them are reopened (event `action.reopened`) and the accepted
+  program continues from where it stopped. Observed on a SIGTERM-interrupted
+  run: 1 cancelled action → 4 "blocked" → a spurious planner turn.
+- Planner contract: a verify with several `dependsOn` must set `review`
+  (rule 4); the program's last worker must be covered by a successful verify
+  (rule 8) — both were the causes of extra planner gates on the goal-3 proof
+  run. A corrective turn's `validationFeedback.rejectedResponseExcerpt` is
+  capped at 2 000 chars, and the rejected proposal is resent as a skeleton
+  (ids, shapes, dependsOn; prompts elided) — the planner's thread already
+  holds it verbatim.
+- A verify without `review` is no longer grounds to reject a whole program:
+  it reviews its single (or last) dependency's artifact, or audits the
+  repository directly when it has no `dependsOn` (`reviewScope: repository`).
+  Observed on two proof runs: a 9-action program bounced for one field,
+  costing a 5-minute correction turn each time.
+
+## 0.14.0 — structured worker output, compact planner contract
+
+- A verify whose reply cannot be parsed as the verdict JSON gets ONE bounded
+  re-ask (event `verify.verdict_retry`) before its failure can reach a planner
+  boundary — observed on run `ejk9w2`: one unparseable verdict cost a full
+  planner turn plus ~8 minutes of re-proving a passing state.
+- Planner contract amendments from the same run's observations: a verify is
+  scoped to what can be true at its point in the graph (later-scheduled work is
+  not a defect; cosmetic mismatches are concerns, never ok:false); when the
+  goal's acceptance checks pass the planner returns complete instead of adding
+  polish actions; restored the shared-working-tree, redundant-verification,
+  and operatorSteering guidance dropped by the contract merge.
+- "Full" planner-context excerpts (scout, new-since-last-decision, failing
+  verifies) obey the per-excerpt and total budgets again; the compaction must
+  never rebuild the 163 k-char contexts it replaced.
+- Planner context and contract compacted: complete emitted planner task text up to the durable-context marker, worktree-isolation suffix included **OBSERVED** `16,316 -> 5,208` characters, and a sample turn-2 durable context **COMPUTED** `163,000 -> 23,547` characters by replacing full attempt records with compact ledger rows and retaining full output excerpts only for new/scout or `ok:false` verify actions.
+- Planner `run` actions and fan-out `stepTemplate`s may declare an optional
+  `outputSchema`, an object-typed JSON-Schema subset. The runtime tells the
+  worker to end its output with one matching JSON object, parses and validates
+  it, and persists a `run` result as `outputs.<id>.data` with `schemaOk: true`;
+  fan-out results store those fields inside each `outputs.<fanoutId>.items[]`
+  entry. Successful validation emits `action.output_validated`.
+- Schema failures emit `action.output_schema_retry` and receive exactly one
+  bounded retry with the validation errors and the previous output tail. If
+  that retry also fails, the action remains `ok:false`, records
+  `schemaOk:false` and `schemaErrors`, and keeps the output text with the
+  reason `output did not match outputSchema: <errors>`. Resumed runs do not
+  re-dispatch actions already marked `schemaOk:true`.
+- Dependent prompts can render `{{outputs.<id>.data.<field>}}`, and
+  `fanout.itemsFrom` accepts `outputs.<id>.data.items` without an extraction
+  agent when the array is already present. Planner decision validation rejects
+  `outputSchema` on a proposed `verify` because verify has a fixed verdict
+  shape.
+
 ## 0.13.2 — user text is never a template
 
 - `workflow goal` failed before anything ran when the goal text quoted

package/docs/claude-dynamic-workflow-mechanics.md

@@ -234,7 +234,7 @@ from `docs/experiments/2026-08-29-ultracode-vs-bullswarm.md`, never projected.
 | Phases | Labels for grouping; never synchronise | Forward-only kebab-case names per action; also just labels | None |
 | Parallelism | `pipeline` default, `parallel` barrier; cap min(16, CPUs−2) | `executeActions` ran dependency-ready siblings **serially** (`runner.js:558`); only `fanout` items ran concurrently; goal default concurrency 3 | **Fixed in 0.11.0** — ready-set scheduler + default 8 |
 | Planner bias | Script author is told to fan out and default to pipeline | Goal prompt said "return needs_more_work with the **smallest useful set** of bounded … actions" (`goal.js:18`) and planner prompt said "keep actions cohesive" | **Fixed in 0.11.0** — "propose the COMPLETE dependency graph", per-item fix→verify chains, file ownership, self-contained prompts |
-| Per-agent prompt | Self-contained, plus JSON schema enforced at tool layer | Planner-authored prompt; free-text answer, content-verified by heuristics; `verify` returns JSON verdict | Partial. Schema-enforced worker output is a candidate, not adopted yet |
+| Per-agent prompt | Self-contained, plus JSON schema enforced at tool layer | Planner-authored prompt; `outputSchema` validates structured worker data, while `verify` retains its fixed JSON verdict | Adopted for declared schemas; tool-layer enforcement remains a difference |
 | Failure handling | Loops in code; `null` on agent death | Planner replans (costly); 0.10.9 added corrective turns for invalid decisions and 0.11.0 recovers mis-shaped `verify.review` before dispatch | Improved; retry-in-code per action still absent |
 | Determinism / resume | Journal of return values; prefix cache | Durable `state.json` + `events.jsonl` + action ledger; resume skips durable outputs | Equivalent |
 | Data-driven fan-out | `pipeline(discovered.items, …)` — count unknown when the script is written | Decision schema forced inline `items`; the planner spent a turn waiting for discovery | **Fixed in 0.12.0** — `itemsFrom` on proposed fan-outs + one bounded extraction retry |
@@ -288,8 +288,7 @@ author and the `Workflow` runtime.
    if the output still has no array the runtime runs ONE bounded, read-only
    extraction action over it (never re-running the producer, which may have
    mutated files). That is the "schema retry" of Claude's `StructuredOutput`,
-   done as a second cheap agent instead of a tool-layer retry. A general
-   `outputSchema` on run actions is still open (§5).
+   done as a second cheap agent instead of a tool-layer retry.
 3. **Pre-authored repair** — shipped. `repair: { prompt, maxRounds }` on a
    verify: verify-fail → `<verifyId>-repair-<n>` (concerns verbatim) →
    re-verify, inside the executor. Claude's fix-loop as code.
@@ -350,6 +349,21 @@ author and the `Workflow` runtime.
    exist — the script's `while (!ok)` loop *is* the evidence — which is the
    general lesson: every piece of control flow bullswarm moves from planner
    into runtime needs its evidence rule moved with it.
+11. **[SPEC] Schema-enforced worker output** — a planner `run` action or fan-out
+    `stepTemplate` may declare an object-typed `outputSchema` subset. The
+    runtime appends instructions for one trailing matching JSON object, with no
+    prose or markdown fences after it, then parses and validates the object.
+    A successful `run` persists `outputs.<id>.data` and `schemaOk: true`; a
+    fan-out stores those schema results inside each
+    `outputs.<fanoutId>.items[]` entry. Both emit `action.output_validated`. A mismatch emits
+    `action.output_schema_retry` and gets exactly one bounded retry carrying
+    the validation errors and the previous output tail; a second mismatch
+    fails the action while retaining its output text and recording
+    `schemaOk:false` and `schemaErrors`. Dependent prompts can render data
+    fields, and `fanout.itemsFrom` can consume `outputs.<id>.data.items` without
+    extraction when it is already an array. Planner decision validation rejects
+    `outputSchema` on a proposed `verify` because verify has a fixed verdict
+    shape.
 
 **Honest limitation.** `itemsFrom` removes the planner *turn*, not the stage
 *barrier*: a verify depending on a data-driven fan-out waits for all items,
@@ -361,10 +375,6 @@ them.
 
 ## 5. Not adopted (yet), and why
 
-- **Schema-enforced worker output.** bullswarm's content verification and the
-  JSON `verify` verdict cover the failure mode today; adding per-action
-  `outputSchema` is the next step if planners keep re-asking workers for
-  structure.
 - **Per-action worktree isolation.** File ownership declared by the planner is
   cheaper and matches Claude's own guidance ("EXPENSIVE … use ONLY when agents
   mutate files in parallel and would otherwise conflict").

package/docs/experiments/2026-08-29-dogfood-bullswarm-builds-bullswarm.md

@@ -0,0 +1,171 @@
+# Dogfood 2026-08-29 — bullswarm builds bullswarm (outputSchema + planner refactor)
+
+Observation log of the two dogfood runs that produced 0.14.0, kept verbatim
+from the driving session's notes; the post-run defects below produced 0.14.1.
+
+
+Goal file: `goal4.txt` (7 deliverables, single-implementer constraint). Repo branch: `feat/output-schema`.
+Runtime: local worktree `bullswarm-rt` pinned at main (user: redispatch with local runtime, keep committing; no npm wait).
+Home: default `~/.bullswarm` (heterogeneous pools) — user can `bullswarm workflow tui <shortId>`.
+
+## Attempt 1 — 01:43:37 Z, installed 0.13.1 — failed at validation, nothing ran
+`autonomous workflow invalid (nothing ran): phases[0].steps[0](scout): template ref "{{outputs.x.data.field}}" cannot resolve`.
+Cause: goal text spliced into the scout prompt; validator parsed a quoted ref in user text.
+Fix (bullswarm defect #1): goal → declared `inputs.goal`, inserted at render time; unresolved grammar-valid refs are
+left literal + `template.unresolved_ref` event instead of fatal. Commit `7badea3`, released 0.13.2 (`a0f0965`).
+Verified: scout task file of attempt 3 contains `{{outputs.x.data.field}}` verbatim; only `{{inputs.goal}}` resolved.
+
+## Attempt 2 — 01:47:37 Z — launcher bug (mine, not bullswarm): zsh does not word-split `$BS="node path"`. Fixed script.
+
+## Attempt 3 — 01:48:19 Z, runtime 0.13.2 @ a0f0965 — run `wf-mtdq1l9v-ed22fe` / `75t4n2`
+- 01:48:22 scout → pool `opencode2` model `kaihk/gpt-5.6-luna`. Routing: "most-behind capable pool (surplus 0)";
+  candidates opencode2 pace 0 (unmetered) > grok −7.5 > claude-code −9.4 > codex −49.1.
+  Observation: an unmetered pool reads as exactly on pace (0) and therefore outranks every metered pool that is
+  ahead of pace. Not a crash, but quota-unknown pools capture all work whenever metered pools are burning ahead.
+- 01:48:22 → 01:49:54 scout ok (92 s, opencode2).
+- 01:50:06 → 02:00:40 planner turn 1 (**~634 s** — roughly 2× the goal-2/3 turns; goal text is 6.5 k chars and the
+  program is 12 actions). Decision: `needs_more_work`, 12 actions, `completion: all-actions-ok` attached.
+  Program honours the single-implementer constraint: `impl-src` (all src) ∥ `docs`; `tests-schema`/`tests-adaptive`/
+  `tests-gaps` + `verify-impl(repair 2)` fan out after impl-src; per-writer verifies with repair; `final-report` →
+  `verify-suite(repair 2)`. Note: `verify-impl`'s repair may edit src while the test writers read it — accepted risk,
+  `verify-suite` runs the whole suite at the end.
+- 02:00:31 `impl-src` and `docs` → opencode2 `kaihk/gpt-5.6-luna` (medium effort, "most-behind capable pool (surplus 0)").
+  All build work lands on the unmetered pool while claude-code/grok/codex are ahead of pace. Orchestrator stayed on
+  claude-code (pinned).
+- 02:00:31 → 02:07:09 `impl-src` ok (398 s, opencode2); `docs` ok (140 s). 02:07:11 five actions started at once
+  (3 test writers + verify-impl + verify-docs).
+- 02:08:20 `verify-impl` ok:false → `verify-impl-repair-1` (concerns were concrete and correct: schema JSON omitted from
+  the retry task text; fan-out item resume skipped schemaOk; no combined run+fanout skeleton). 02:08:36 `verify-docs`
+  ok:false → `verify-docs-repair-1` (doc claimed fan-out persists top-level data/schemaOk; doc claimed static validator
+  rejects outputSchema on verify but only decision.js did). Repairs 121 s / 124 s. Both are repair-loop live uses; if the
+  program then self-completes this is the first live exercise of the 0.13.1 fix path.
+- 02:12 → 02:18 second round of repairs: `verify-impl-repair-2` (escalation could allow a 2nd schema retry — legit design
+  nit; "schema.js untracked so absent from git diff --stat" — verifier misreading), `verify-adaptive-tests-repair-1/2`
+  (rejected twice for the same reason: "diff is not purely additive: modifies an existing assertion" — the
+  programFeatures assertion HAD to change; an unrepairable process criterion). Both verifies ended ok:false after
+  maxRounds → `final-report`/`verify-suite` blocked (`failed_terminal: dynamic actions blocked by failed or unresolved
+  dependencies`) → program boundary → planner turn 2 at 02:18:52.
+  Observation (behaviour): a verifier that judges process criteria ("purely additive", "untracked file") instead of the
+  goal's acceptance checks produces rejections no repair can satisfy; two repair rounds (~10 min) were spent before the
+  boundary. Candidate for the prompt audit: verify doctrine "ok:false only for failed acceptance checks; process
+  observations are concerns" and/or runtime: identical concerns after a repair → boundary immediately.
+- 02:18:52 → ~02:25:20 planner turn 2 (~390 s): `needs_more_work`, 3 sequential actions — `recheck-src` (verify, repair 2,
+  depends on verify-impl-repair-2) → `full-suite-report` (run) → `verify-final` (verify, repair 2). Reason correctly
+  notes no whole-suite evidence existed after the repairs (impl-src's npm test predated them).
+- 02:25:52 → 02:32:36 follow-up program: `recheck-src` ok:false → `recheck-src-repair-1` → re-verify ok; `full-suite-report`
+  ok; `verify-final` ok → **`decision.auto_completed` (program-completion)** at 02:32:39.
+## Result (attempt 3)
+- Wall **44 min 17 s** (2 654 s), 28 dispatches (26 opencode2 + 2 planner on claude-code), max concurrent 5,
+  parallelism 1.5, planner 2 turns / 1 045 s (39 % — turn 1 alone 637 s), repairs 6 (2 repaired ok, 4 re-verify
+  rejected), tokens ≈ 108 k (estimate).
+- Deliverable: 10 files changed + 2 new (489+/38−); `npm test` **318/318** (299 + 19) on my own run; validator module,
+  decision/validate/runtime/runner/template changes all present; docs + changelog written.
+- My review: sound; one robustness flaw fixed by hand — `readTrailingObject` used a reverse brace/quote scanner whose
+  escape handling is wrong scanning backwards (a `\"` inside a string could derail it and waste the single retry);
+  replaced with the parse-candidates approach `hasStructuredAnswer` already uses. Double failure now reports the
+  retry's errors. Escalation concern from verify-impl judged mistaken (escalation follows failed dispatches only).
+- 0.13.1 fix path: NOT exercised here either — the latest worker at completion was `full-suite-report`, verified by a
+  direct edge (`verify-final`).
+## Adversarial review via `bullswarm run --lane analyze` (02:36 → 02:39 Z, 206 s, opencode2 kaihk/gpt-5.6-luna)
+- Verdict "do not release" with **two confirmed, reproduced defects** — both in code I had reviewed and passed:
+  (1) my rewritten `readTrailingObject` returned the first schema-valid `{…}` from the right, so a nested object
+  could be recorded (`{"wrapper":{"ok":"inner"}}` → data `{"ok":"inner"}`); (2) `schema.js` used `in`, so
+  `toString`/`constructor`/`__proto__` counted as present/declared. Cleared: escaped quotes, fenced JSON, resume rules,
+  exactly-one retry, validator paths. Fixed + regression tests (320/320); my fix also had an infinite loop when output
+  starts with `{` (lastIndexOf clamps negative fromIndex) — caught by the suite hanging, fixed.
+- Evidence for open item "adversarial verification by default": a 3-minute refute-framed review found what the
+  run's own verifies (6 rounds) and my manual review both missed.
+## Goal 5 — planner context/contract refactor — run `wf-mtds7tzx-95ab05` / `ejk9w2`, 02:49:09 Z, runtime 0.13.2 local
+- scout 72 s (opencode2); planner turn 1 02:50:55 → 02:58:56 (~480 s): 6 actions, completion attached; noticed the goal's
+  stale "318 passing" and used the real 320. Program: impl-src → verify-src(repair 2) → update-tests ∥ update-docs →
+  verify-tests(repair 2) → verify-suite(repair 1).
+- 02:58:56 → 03:09:04 impl-src (~610 s). verify-src rejected twice, both times on SUBSTANTIVE spec points (obsolete
+  skeleton text left in a comment; 6 JSON examples instead of 2; `<item>` instead of `{{item}}` in examples; excerpt
+  policy). One misread to check in the final diff: it called the existing 3 000/36 000-char excerpt caps a violation of
+  "full excerpt" although the goal said "(existing budget logic)" — the repair may have removed the caps.
+  Ordering tension: verify-src runs before update-tests, so it necessarily sees 5 failing old assertions; the planner
+  should either fold assertion updates into impl-src or make verify-src judge src only.
+- Goal 5 finished 04:00:40 Z (71.5 min, 3 planner turns + program-completion, 15 dispatches, plannerSec 1 105+175).
+  Where the time went: 18–20 min planner turns; 12.4 min verify-src repair loop enforcing my over-exacting spec and
+  judging intermediate state; ~8 min decision-3 nit round (`align-prefix-number` + `confirm-docs`) triggered by an
+  UNPARSEABLE final-check verdict; the queued steer (03:57:32, "converge now") was NEVER delivered — 0 steering.delivered events; the run auto-completed (program-completion, 04:00:40) and deliverSteering only runs at planner gates, so auto-completion silently discards pending operator steering. Known issue; convergence came from the runtime, not the steer.
+  Deliverable reviewed + committed `15f1534`: 10-rule contract (2.2 k chars) + 2 examples (1.9 k) replace 16.3 k prefix;
+  compact ledger rows; id-only failures; 200-char stale excerpts; `decision.context_built` size event; 323/323.
+- My follow-up (commit after 15f1534): verify verdict parse failure → ONE bounded re-ask (`verify.verdict_retry`,
+  test with a garbled-once verifier, 324/324); contract amendments (verify scoping, converge-not-polish, restored
+  shared-tree/redundant-verification/operatorSteering lines the merge dropped); re-budgeted "full" excerpts (the
+  uncapped version could have rebuilt the 163 k contexts).
+- Speed answer to the user: ~35 of 66 min (at question time) was real work; fixes target the rest — re-ask (−8 min),
+  verify scoping (−12 min), convergence rule (−nit rounds). Remaining lever: planner turn latency itself (Opus
+  high-effort per boundary; context compaction cuts cost ~7×, latency is model thinking time).
+
+## Post-run defects → 0.14.1 (fixed directly, dogfooding paused by user direction)
+- `workflow tui` crashed twice (`detailRow` dashboard.js:809 `JSON.parse` of
+  state.json mid-write; the throw escaped the repaint timer and killed the TUI,
+  stranding the terminal in alt-screen raw mode). Fix: atomic temp+rename
+  writes for state/report/workflow.json + torn-read-tolerant observation
+  readers + guarded paint/key handlers with last-good-frame fallback.
+- TUI showed phase ✗ "2/2 complete" while a re-verify attempt was live. Fix:
+  an action with an active agent reads as running; phase precedence
+  active > failed > completed.
+- The goal-5 steer was never delivered: auto-completion bypassed the planner
+  gate and silently discarded pending steering (0 steering.delivered events).
+  Fix: pending steering defers self-completion to the planner
+  (`decision.completion_deferred`); undeliverable steering is marked
+  `expired_undelivered` (`steering.expired`) at the terminal transition.
+- Routing concentration on the unmetered pool (26/28 dispatches) confirmed as
+  design intent (quota protection outranks diversity) and documented in the
+  skill rather than changed.
+
+## Proof run 1 — goal-3 re-run `d8pr8s` (wf-mtdvuk9m), 0.14.1-pre @ 0bbe78c
+Baseline (0.13.1, same fixture/goal/pool/flags): 28m42s wall, 1 planner turn, 294 s planner.
+- **completed + verified, deliverable exactly right** (csv+slugify guarded, existing tests byte-identical, 63/63),
+  zero crashes, 8 workers, no repairs.
+- Wall **32m19s**, planner **5 dispatches / 463 s** (195+114+58+66+30). Per-turn latency DOWN (max 195 s vs 294 s);
+  context per turn 6.2k–48k chars (`decision.context_built` measuring itself) vs the old 16.3k prefix + up-to-178k contexts.
+- The 3 extra gates, each diagnosed and fixed in `c0ff947`:
+  1. first proposal rejected — verify with several dependsOn lacked `review` → contract rule 4 now states it;
+  2. evidence-policy boundary + rejected `complete` — final-report left as last unverified worker → rule 8 now
+     states the LAST worker must be covered by a verify;
+  3. one deliberate operator steer — which **live-proved the 0.14.1 steering fix**: `decision.completion_deferred`
+     → `steering.delivered` (first ever observed; the goal-5 defect showed 0) → planner turn honoured it.
+- Also observed and fixed: a corrective turn re-inflated validationFeedback to 24k chars (raw response duplicated
+  the parsed proposal) → excerpt capped at 2k.
+- 0.13.1 completion-evidence policy exercised live for the first time: it refused auto-completion twice, correctly.
+
+## Proof run 2 — goal-3 re-run `djnjka` (wf-mtdx5htt), 0.14.1-pre @ c0ff947 — interrupted, then cancelled
+- Turn-1 proposal **accepted first try** (no validation rejection, no correction turn): contract fix #1 confirmed.
+  Turn-1 context 6,201 chars.
+- At 05:21 the driving session's background task was killed by the harness (not the user); SIGTERM reached the
+  runner, which persisted `interrupted` + resumable (1/3 steps, in-flight `triage` cancelled) — the 0.13 interruption
+  path working as designed.
+- Resume at 05:54 exposed a **resume defect**: the cancelled `triage` was persisted `ok:false` ("workflow
+  cancellation requested"), so its 4 dependents were marked "blocked by failed or unresolved dependencies" and the
+  planner was asked to re-plan around a failure that never happened. Cancelled the run; fixed in `459c58c`
+  (cancelled actions + dependents blocked only by them are reopened on resume, event `action.reopened`; regression
+  test drives a cancel marker into a slow in-flight action and asserts exactly one further planner turn).
+
+## Proof run 3 — goal-3 re-run `4t6m5a` (wf-mtdyyqkw), 0.14.1-pre @ 459c58c — cancelled after diagnosis
+- Turn-1 proposal (9 actions, sound shape: 3 disjoint impl workers ∥ audit of untouched modules → per-module
+  verifies → final-report → verify-final, completion attached) was **rejected** for one field: a zero-dependsOn
+  audit verify carried no `review`. Correction turn cost ~5 min and re-inflated context to 47.8k chars
+  (validationFeedback 38k — the 2k cap on rejectedResponseExcerpt was insufficient because rejectedProposal
+  itself is 37k). Root cause is the validator's posture, not the planner: rejecting a whole program for a field the
+  runtime can default. Fixed in `548eabe`: review defaults to the single/last dependency's artifact, or
+  `reviewScope: repository` for a no-dependency audit; contract rule 4 reworded. Run cancelled to re-prove cleanly.
+
+## Proof run 4 — goal-3 re-run `d7xyg2` (wf-mtdzhw88), 0.14.1-pre @ 548eabe — **PASS**
+| metric | 0.13.1 baseline (x3x2a2-era, same fixture) | 0.14.1-pre run 4 |
+| --- | ---: | ---: |
+| outcome | completed, verified | completed, verified, **auto-completed** (program-completion) |
+| wall | 28 min 42 s | 30 min 12 s |
+| planner turns / plannerSec | 1 / 294 s | **1 / 269 s** |
+| planner context (turn 1) | 32.7 k chars (measured on a sibling run) | **6.2 k chars** |
+| dispatches (workers) | — | 8 (7) · max concurrent 9 |
+| corrections / rejections / repairs / verdict re-asks | 0 / 0 / 0 / 0 | 0 / 0 / 0 / 0 |
+| deliverable | csv + slugify guarded, 63/63 | csv + slugify guarded, **63/63**, existing test files byte-identical |
+Program: scout → probe (all exports, wrong-type matrix) → fanout guards over the discovered modules → verify-guards →
+report → verify-final, `completion: all-actions-ok`. Wall is within noise of baseline (+90 s, dominated by worker
+model time: probe 4.7 min, guards 5.1 min, verify-guards 4.7 min); the planner side is faster and 5× smaller.
+Zero observation crashes across four runs of TUI/watch/runs/result/static-tui polling and a 20 s stress loop
+(1,681 paints against the live writer, 0 torn, 0 throws).

package/docs/planner-prompt-audit-2026-08-29.md

@@ -0,0 +1,157 @@
+# Planner prompt and context audit — 2026-08-29
+
+Question from the user: after the 0.11 → 0.14 iterations, does the instruction
+set given to the orchestrator/planner still make sense, or should it be
+simplified or refactored?
+
+Method: measure what the planner actually receives, not what the source files
+look like. The two planner task files of dogfood run `wf-mtdq1l9v-ed22fe`
+(`75t4n2`, bullswarm building `outputSchema` in its own repo, runtime 0.13.2)
+are the specimens; sizes are characters of the task text
+(`task-orchestrator-*.md`), tokens ≈ chars / 4.
+
+## 1. What one planner turn receives
+
+| section | source | turn 1 | turn 2 |
+| --- | --- | ---: | ---: |
+| orchestrator prompt + worktree line | `goal.js` `AUTONOMOUS_ORCHESTRATOR_PROMPT` | 5 000 | 5 000 |
+| PLANNING DOCTRINE (11 bullets) | `runtime.js` runDecision | 3 605 | 3 605 |
+| Action skeletons (6 shapes + verify semantics) | `runtime.js` | 1 342 | 1 342 |
+| Program skeleton (discovery → fan-out → verify → suite) | `runtime.js` | 1 217 | 1 217 |
+| Graph skeleton (two chains + suite) + fanout paragraph + runtime-owned line | `runtime.js` | 4 054 | 4 054 |
+| durable context (JSON) | `runtime.js` plannerContext | 17 474 | 162 946 |
+| **total** | | **32 730** (~8 k tokens) | **178 452** (~45 k tokens) |
+
+The fixed prefix is 15.2 k chars on every turn. The durable context grew
+**9×** between turn 1 and turn 2 of the same run.
+
+### Where the 163 k of turn 2 went
+
+| key | chars | what it is |
+| --- | ---: | --- |
+| `completedActions` (19 entries) | 66 700 | every finished action **with its full attempt records**: routing candidates and pace numbers, usage, pricing table, child pid, timings — ~3.5 k per action |
+| `outputs` (19 entries) | 41 560 | `outputExcerpt` of ~3 k chars for *every* finished action, including ones that finished ok and were already verified in the previous program |
+| `intent` | 6 657 | the goal text (6.5 k) + cwd + policy — needed, once |
+| `failures` | 1 073 | the two blocked actions — duplicates of `completedActions` entries |
+| `availablePools`, `budget`, `executionConstraints`, `closedPhases`, … | ~1 800 | fine |
+
+## 2. What is said more than once
+
+Reading the prefix as the planner does, the same rules appear two or three
+times in different words:
+
+| rule | orchestrator prompt | doctrine | skeletons |
+| --- | --- | --- | --- |
+| propose the whole program, one round trip costs minutes | item 2 | bullets 1, 2 | "all in ONE decision" ×2 |
+| N items → N run + N verify + one suite verify | item 5 | "Per-item chains" | Graph skeleton |
+| unknown items → discovery + `itemsFrom` fan-out | item 5 | "Unknown item count" | Program skeleton + fanout paragraph |
+| every verify gets a `repair` policy | item 5 | "Verification failures" | verify skeleton |
+| `completion: all-actions-ok` on a clean program | item 5 | "Self-completing programs" | — |
+| self-contained worker prompts, file ownership | item 3 | "File ownership", "Self-contained prompts" | — |
+| don't propose pool/addDir/taskFile | closing line | — | final line |
+
+Item 5 of the orchestrator prompt alone is 1 050 chars and restates four
+doctrine bullets. The two program skeletons both end in the same
+`verify-items → verify-suite` tail.
+
+## 3. Does it matter? Measured
+
+- Turn 1 (32.7 k chars) took **637 s**; turn 2 (178 k chars) took **390 s**.
+  Latency is therefore dominated by the model's reasoning on the goal, not by
+  context size — the 6.5 k-char goal and a 12-action program cost more thinking
+  than reading 45 k tokens. Trimming context is a **cost** and **attention**
+  lever, not primarily a latency lever.
+- Cost: turn 2 read ~45 k tokens to emit a ~1 k-token decision. At Opus
+  prices that is ~$0.25 per boundary; a run with four boundaries (goal 2 on
+  0.12.1) spends more on re-reading attempt metadata than on the decisions.
+- Attention: the planner's turn-2 reason correctly diagnosed the blocked
+  graph, so quality did not visibly suffer here — but 64 k chars of pricing
+  tables and routing candidates are noise it must skip to find the two
+  `ok:false` concerns that matter.
+- Behaviour observed in three runs (goal 2, goal 3, dogfood): every rule the
+  prefix repeats was followed on the first turn (whole program, per-item
+  chains, repair policies, `completion`). No observed decision needed a rule
+  to be stated twice.
+
+## 4. Recommendation
+
+Yes — refactor, in two independent pieces, both measurable:
+
+**A. Compact the durable context (the 9× growth).** Planner-facing ledger rows
+instead of raw ledger entries: `{ id, type, phase, status, pool, durationSec,
+attempts, why }` (~150 chars; 19 actions → ~3 k instead of 66.7 k). Keep a
+full `outputExcerpt` only for actions finished **since the last decision** and
+for every `ok:false` verify; older ok actions get a one-line summary (id, ok,
+first 200 chars). Replace `failures` with the ids of failing actions (their
+full entry already sits in the ledger). Expected turn-2 context: ~25 k chars
+instead of 163 k. Pure runtime change; no planner behaviour change intended.
+
+**B. One contract instead of three overlapping texts.** Merge
+`AUTONOMOUS_ORCHESTRATOR_PROMPT` and the doctrine bullets into a single ordered
+list of ~10 rules (target ≤ 4 k chars, from 8.1 k), each stated once with its
+reason; keep exactly two JSON examples — the action shapes list and one
+complete program (discovery → data-driven fan-out → per-item verify with
+repair → suite verify, with `completion`) — and delete the second program
+skeleton (target ≤ 3 k, from 6.6 k). Total prefix ≤ 7 k chars, from 15.2 k.
+
+Acceptance for both: unit tests on the context builder (row shape, excerpt
+policy by decision sequence) and on the prompt (each rule appears once; the
+skeleton assertions in `tests/workflow-adaptive.test.js` updated); then one
+re-run of goal 3 on the same fixture (baseline 0.13.1: 28 min 42 s, 1 planner
+turn, 294 s) to confirm the decision shape is unchanged and record the new
+per-turn size.
+
+Not recommended: cutting the goal text or the scout excerpt from the context —
+both were used verbatim by every first-turn program observed.
+
+## 5. Outcome
+
+Measurements below were taken after the refactor from the current source and
+from the committed source saved into `/tmp/goal-before.mjs` and
+`/tmp/runtime-before.js`, using the same temporary measurement script. The
+canonical prefix is the complete emitted planner task text counted from its
+first character up to (not including) the durable-context marker, with the
+worktree-isolation suffix included. The committed baseline predates the named
+section exports, so its emitted prefix was reconstructed from the committed
+`runtime.js` task-text assembly and the committed `AUTONOMOUS_ORCHESTRATOR_PROMPT`.
+
+- Complete emitted planner task text up to the durable-context marker,
+  worktree-isolation suffix included: **OBSERVED**, `16,316` characters before
+  and `5,208` characters after. Command: `node /tmp/measure-planner.mjs`.
+- Static planner task-prefix array through the durable-context boundary:
+  **OBSERVED**, `16,209` characters before and `5,101` characters after. The
+  after value is 107 characters shorter because it excludes the unchanged
+  worktree-isolation suffix; this is a secondary source-level measurement, not
+  the canonical emitted-prefix headline. Command: `node /tmp/measure-planner.mjs`.
+- `PLANNER_RULES_SECTION`: **OBSERVED**, `2,202` characters after. Command:
+  `node /tmp/measure-planner.mjs`.
+- `PLANNER_EXAMPLES_SECTION`: **OBSERVED**, `1,867` characters after. Command:
+  `node /tmp/measure-planner.mjs`.
+- `AUTONOMOUS_ORCHESTRATOR_PROMPT`: **OBSERVED**, `4,670` characters after;
+  the committed before source had no separately exported rules or examples
+  sections, so separate before-section sizes are **NOT AVAILABLE**, not
+  inferred. Command: `node /tmp/measure-planner.mjs`.
+- Sample durable context: **OBSERVED** baseline `163,000` characters in the
+  audit's rounded turn-2 durable-context total (the detailed table records
+  `162,946`; the full turn-2 task was `178,452`), and **COMPUTED** `23,547`
+  characters after. The computed sample applies 19
+  compact ledger rows at 150 characters each, keeps a 3,000-character scout
+  excerpt and two 3,000-character failing-verify excerpts, truncates the
+  other 16 action excerpts to 200 characters, represents two failures as
+  20-character IDs, and retains the audit's 6,657-character intent and
+  1,800-character other-context components. Command: `node /tmp/compute-context.mjs`.
+
+Deliverables:
+
+- Durable planner context shrank because completed actions are compact ledger
+  rows, failures are IDs, and stale successful output is truncated.
+- Planner contract shrank because overlapping prompt/doctrine/skeleton text is
+  now one ordered rules section plus exactly two JSON examples, single-sourced
+  in `src/workflow/goal.js`.
+- Runtime prompt construction shrank because `src/workflow/runtime.js` imports
+  the shared contract instead of carrying a duplicate doctrine and graph
+  skeleton.
+- `skill/SKILL.md` was left unchanged: it documents the general durable
+  context and the separate run-state/TUI attempt view, but does not document a
+  renamed/dropped planner-context field shape such as the old attempt records
+  or an old `failures` representation.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bullswarm",
-  "version": "0.13.2",
+  "version": "0.14.1",
   "description": "Route work across coding-agent CLI subscriptions — paced by live quota meters, verified by content, never trusting exit codes.",
   "type": "module",
   "bin": {

package/skill/SKILL.md

@@ -246,7 +246,12 @@ bullswarm workflow steer <shortId> --message "guidance for the next planner chec
 `capabilities` reports available pools, supported lanes, configured models,
 meter readings, burst gates, quarantine state, retry limits, and the important
 routing rule. Automatic routing chooses the highest time-adjusted quota surplus
-among capable pools. For strategic model selection, first run:
+among capable pools. An unmetered pool reads as exactly on pace (surplus 0), so
+whenever every metered pool is burning ahead of its window (negative
+surplus) the unmetered pool wins ALL work — by design: quota protection
+outranks provider diversity (observed 2026-08-29: 26 of 28 dispatches on
+one unmetered pool). If that concentration is unwanted, meter the pool or
+exclude its models via `strategy exclude-model`. For strategic model selection, first run:
 
 ```bash
 bullswarm strategy refresh
@@ -361,6 +366,11 @@ that expressible without extra turns:
   (`source: "program-completion"`, event `decision.auto_completed`) and the run
   ends without another planner turn; a failing action emits
   `decision.completion_predicate_unmet` and the boundary returns to the planner.
+- `outputSchema` on a `run` or fan-out `stepTemplate` — declare it when a
+  downstream action needs reliable structured data, such as an object to render
+  into a dependent prompt or an `items` array for `fanout.itemsFrom`; leave it
+  off for ordinary prose; planner proposals must not put it on `verify`, whose
+  verdict shape is fixed.
 
 Every fan-out records a summary artifact as `outputs.<id>.outFile` and a
 boolean `ok` (item count in `succeeded`), so a verify may depend on a fan-out
@@ -401,6 +411,17 @@ Resume by shortId:
 bullswarm workflow draft run my-audit --resume <shortId> --json --quiet
 ```
 
+## Writing goals that converge
+
+State outcomes, not measurements. A goal that fixes character counts, exact
+event names, or cosmetic layout turns every verifier into a nit machine:
+observed 2026-08-29 (run `ejk9w2`), a spec with hard numeric limits cost a
+12-minute verify/repair loop enforcing them against an intermediate state.
+Say what must be true at the end (`npm test` passes, the planner receives one
+contract stated once, context stays bounded) and let workers pick the numbers;
+put any hard limit in ONE final acceptance check, not on every intermediate
+verify.
+
 ## Writing prompts that the verify gate will accept
 
 The verify gate (`src/lib/verify.js`) flags outputs as `intent_only`

package/src/workflow/cli.js

@@ -340,7 +340,7 @@ async function wfGoal(opts) {
     try {
       doc = existsSync(workflowPath)
         ? JSON.parse(readFileSync(workflowPath, 'utf8'))
-        : JSON.parse(readFileSync(statePath, 'utf8'))._doc;
+        : readJsonForUpdate(statePath, 'workflow state')._doc;
     } catch (err) {
       console.error(`✗ cannot load durable workflow for ${resumeRunId}: ${err.message}`);
       return 1;