bullswarm 0.13.1 → 0.14.0

package/CHANGELOG.md

@@ -1,5 +1,53 @@
 # bullswarm changelog
 
+## 0.14.0 — structured worker output, compact planner contract
+
+- A verify whose reply cannot be parsed as the verdict JSON gets ONE bounded
+  re-ask (event `verify.verdict_retry`) before its failure can reach a planner
+  boundary — observed on run `ejk9w2`: one unparseable verdict cost a full
+  planner turn plus ~8 minutes of re-proving a passing state.
+- Planner contract amendments from the same run's observations: a verify is
+  scoped to what can be true at its point in the graph (later-scheduled work is
+  not a defect; cosmetic mismatches are concerns, never ok:false); when the
+  goal's acceptance checks pass the planner returns complete instead of adding
+  polish actions; restored the shared-working-tree, redundant-verification,
+  and operatorSteering guidance dropped by the contract merge.
+- "Full" planner-context excerpts (scout, new-since-last-decision, failing
+  verifies) obey the per-excerpt and total budgets again; the compaction must
+  never rebuild the 163 k-char contexts it replaced.
+- Planner context and contract compacted: complete emitted planner task text up to the durable-context marker, worktree-isolation suffix included **OBSERVED** `16,316 -> 5,208` characters, and a sample turn-2 durable context **COMPUTED** `163,000 -> 23,547` characters by replacing full attempt records with compact ledger rows and retaining full output excerpts only for new/scout or `ok:false` verify actions.
+- Planner `run` actions and fan-out `stepTemplate`s may declare an optional
+  `outputSchema`, an object-typed JSON-Schema subset. The runtime tells the
+  worker to end its output with one matching JSON object, parses and validates
+  it, and persists a `run` result as `outputs.<id>.data` with `schemaOk: true`;
+  fan-out results store those fields inside each `outputs.<fanoutId>.items[]`
+  entry. Successful validation emits `action.output_validated`.
+- Schema failures emit `action.output_schema_retry` and receive exactly one
+  bounded retry with the validation errors and the previous output tail. If
+  that retry also fails, the action remains `ok:false`, records
+  `schemaOk:false` and `schemaErrors`, and keeps the output text with the
+  reason `output did not match outputSchema: <errors>`. Resumed runs do not
+  re-dispatch actions already marked `schemaOk:true`.
+- Dependent prompts can render `{{outputs.<id>.data.<field>}}`, and
+  `fanout.itemsFrom` accepts `outputs.<id>.data.items` without an extraction
+  agent when the array is already present. Planner decision validation rejects
+  `outputSchema` on a proposed `verify` because verify has a fixed verdict
+  shape.
+
+## 0.13.2 — user text is never a template
+
+- `workflow goal` failed before anything ran when the goal text quoted
+  something shaped like a template ref (`{{outputs.x.data.field}}` in a goal
+  *about* templates): the goal was spliced into the scout prompt and the
+  workflow validator rejected the ref as unresolvable — "autonomous workflow
+  invalid (nothing ran)". The goal is now a declared input (`inputs.goal`)
+  inserted at render time, so nothing in user text is ever parsed.
+- A grammar-valid ref with nothing behind it no longer kills the action at
+  render time. It is left literally in the prompt and reported as
+  `template.unresolved_ref { actionId, ref }`; planner-authored prompts may
+  quote refs as text, and a worker can usually still act on the literal.
+  `renderTemplate(str, scope, { strict: true })` keeps the old hard failure.
+
 ## 0.13.1 — a repaired verify counts as verification of its repair
 
 - `completionEvidenceGaps` accepted a verify as evidence for the latest worker

package/docs/claude-dynamic-workflow-mechanics.md

@@ -216,7 +216,15 @@ from `docs/experiments/2026-08-29-ultracode-vs-bullswarm.md`, never projected.
   policy and race on the barrel file." The policy it cites was a caution line
   in the planner prompt; a concurrency cap of 8 was available and unused.
   Discovery alone then ran 458 s. (Final numbers: experiment report.)
-- (comparison fixture final results: pending)
+- Final comparison on the 6-module fixture (experiment report for the full
+  tables). Goal 2 (known items): Claude 58 min, 24 agents, 0 orchestrator
+  turns during execution, parallelism 3.1 · bullswarm 0.11.1 41 min, 3 planner
+  turns (27 %), max 6 concurrent, parallelism 2.74 · bullswarm 0.12.1 48 min of
+  execution after a 3 h quota wait it survived, 4 planner turns (two caused by
+  the bug fixed in 0.13.1), max 7 concurrent, one live repair round. Goal 3
+  (discovered items) on 0.13.1: 28 min 42 s, **one planner turn**, the runtime
+  recorded `complete` itself, exactly the three unguarded modules fixed,
+  75/75 tests.
 
 ## 3. bullswarm today, mechanic by mechanic
 
@@ -226,7 +234,7 @@ from `docs/experiments/2026-08-29-ultracode-vs-bullswarm.md`, never projected.
 | Phases | Labels for grouping; never synchronise | Forward-only kebab-case names per action; also just labels | None |
 | Parallelism | `pipeline` default, `parallel` barrier; cap min(16, CPUs−2) | `executeActions` ran dependency-ready siblings **serially** (`runner.js:558`); only `fanout` items ran concurrently; goal default concurrency 3 | **Fixed in 0.11.0** — ready-set scheduler + default 8 |
 | Planner bias | Script author is told to fan out and default to pipeline | Goal prompt said "return needs_more_work with the **smallest useful set** of bounded … actions" (`goal.js:18`) and planner prompt said "keep actions cohesive" | **Fixed in 0.11.0** — "propose the COMPLETE dependency graph", per-item fix→verify chains, file ownership, self-contained prompts |
-| Per-agent prompt | Self-contained, plus JSON schema enforced at tool layer | Planner-authored prompt; free-text answer, content-verified by heuristics; `verify` returns JSON verdict | Partial. Schema-enforced worker output is a candidate, not adopted yet |
+| Per-agent prompt | Self-contained, plus JSON schema enforced at tool layer | Planner-authored prompt; `outputSchema` validates structured worker data, while `verify` retains its fixed JSON verdict | Adopted for declared schemas; tool-layer enforcement remains a difference |
 | Failure handling | Loops in code; `null` on agent death | Planner replans (costly); 0.10.9 added corrective turns for invalid decisions and 0.11.0 recovers mis-shaped `verify.review` before dispatch | Improved; retry-in-code per action still absent |
 | Determinism / resume | Journal of return values; prefix cache | Durable `state.json` + `events.jsonl` + action ledger; resume skips durable outputs | Equivalent |
 | Data-driven fan-out | `pipeline(discovered.items, …)` — count unknown when the script is written | Decision schema forced inline `items`; the planner spent a turn waiting for discovery | **Fixed in 0.12.0** — `itemsFrom` on proposed fan-outs + one bounded extraction retry |
@@ -280,8 +288,7 @@ author and the `Workflow` runtime.
    if the output still has no array the runtime runs ONE bounded, read-only
    extraction action over it (never re-running the producer, which may have
    mutated files). That is the "schema retry" of Claude's `StructuredOutput`,
-   done as a second cheap agent instead of a tool-layer retry. A general
-   `outputSchema` on run actions is still open (§5).
+   done as a second cheap agent instead of a tool-layer retry.
 3. **Pre-authored repair** — shipped. `repair: { prompt, maxRounds }` on a
    verify: verify-fail → `<verifyId>-repair-<n>` (concerns verbatim) →
    re-verify, inside the executor. Claude's fix-loop as code.
@@ -316,7 +323,7 @@ author and the `Workflow` runtime.
    sees `outputs.scout.ok=false` with the reason, and a run where only the
    scout succeeded is `blocked`, never "delivered".
 
-8. **Program-level completion** (0.13.0, unreleased at the time of writing) —
+8. **Program-level completion** (0.13.0) —
    `completion: { when: "all-actions-ok", reason }` on a program. Claude's
    script simply returns when its code is done; bullswarm still spent a final
    planner turn (110–250 s measured) to say `complete` after a clean run. Now
@@ -324,10 +331,39 @@ author and the `Workflow` runtime.
    never below the completion policy) and consults the planner only when
    something failed. With 0.12.0's repair-in-program this makes a clean run
    **one planner turn**: compile, execute, done — Claude's "0 orchestrator turns
-   during execution" for the passing case.
+   during execution" for the passing case. **[OBSERVED]** goal-3 run
+   `wf-mtdkvx0k` (0.13.1): the planner attached the predicate on its own, the
+   runtime emitted `decision.auto_completed` (`source: program-completion`),
+   one planner process for the whole 28 min run.
 9. **Rate limits are waited for** (0.12.1): a burst-gated provider parks the
    dispatch in `waiting_for_quota` until the window resets instead of failing
    the run in 4 s, which is what the first 0.12.0 comparison launch did.
+   **[OBSERVED]** goal-2 run `wf-mtdcghw0`: parked at 95 % for 3 h 2 min,
+   dispatched 17 s after the provider reset.
+10. **A repair is verified by its verify's re-run** (0.13.1). The executor's
+   repair loop creates `<verify>-repair-N` *depending on* the verify, then runs
+   the verify again; the completion-evidence check only followed
+   `verify.dependsOn` and so never saw a repair as verified. **[OBSERVED]** on
+   `wf-mtdcghw0`: a clean `complete` rejected, three more planner turns
+   (~11 min) to re-prove a passed re-verify. In Claude's model this bug cannot
+   exist — the script's `while (!ok)` loop *is* the evidence — which is the
+   general lesson: every piece of control flow bullswarm moves from planner
+   into runtime needs its evidence rule moved with it.
+11. **[SPEC] Schema-enforced worker output** — a planner `run` action or fan-out
+    `stepTemplate` may declare an object-typed `outputSchema` subset. The
+    runtime appends instructions for one trailing matching JSON object, with no
+    prose or markdown fences after it, then parses and validates the object.
+    A successful `run` persists `outputs.<id>.data` and `schemaOk: true`; a
+    fan-out stores those schema results inside each
+    `outputs.<fanoutId>.items[]` entry. Both emit `action.output_validated`. A mismatch emits
+    `action.output_schema_retry` and gets exactly one bounded retry carrying
+    the validation errors and the previous output tail; a second mismatch
+    fails the action while retaining its output text and recording
+    `schemaOk:false` and `schemaErrors`. Dependent prompts can render data
+    fields, and `fanout.itemsFrom` can consume `outputs.<id>.data.items` without
+    extraction when it is already an array. Planner decision validation rejects
+    `outputSchema` on a proposed `verify` because verify has a fixed verdict
+    shape.
 
 **Honest limitation.** `itemsFrom` removes the planner *turn*, not the stage
 *barrier*: a verify depending on a data-driven fan-out waits for all items,
@@ -339,10 +375,6 @@ them.
 
 ## 5. Not adopted (yet), and why
 
-- **Schema-enforced worker output.** bullswarm's content verification and the
-  JSON `verify` verdict cover the failure mode today; adding per-action
-  `outputSchema` is the next step if planners keep re-asking workers for
-  structure.
 - **Per-action worktree isolation.** File ownership declared by the planner is
   cheaper and matches Claude's own guidance ("EXPENSIVE … use ONLY when agents
   mutate files in parallel and would otherwise conflict").

package/docs/experiments/2026-08-29-ultracode-vs-bullswarm.md

@@ -325,7 +325,7 @@ together with its instructions. (2) The remediation round spent two of its
 three fixes on "non-blocking" nits from verifiers that had returned
 `ok:true`; 0.12.0's doctrine tells the planner those are informational.
 
-### bullswarm 0.12.0 (installed binary) — same goal, fresh copy `g2-bs-v3`
+### bullswarm 0.12.0 → 0.12.1 (installed binary) — same goal, fresh copy `g2-bs-v3`
 
 **First launch, 19:09:48 Z, installed 0.12.0 — failed in 4 s.** Both the
 scout and the orchestrator were `failed_terminal` with `no eligible pool`
@@ -393,27 +393,73 @@ Take the bug and the dependency slip out and this run is ~29 min of execution wi
 The originally planned 0.10.9 goal-2 run was dropped at the user's request
 (2026-08-29): the installed latest is the only baseline that matters.
 
+### bullswarm 0.13.1 (installed binary) — goal 3, discovery-shaped, fresh copy `g3-bs-v3`
+
+Goal 3 was written to exercise what goal 2 cannot: an **unknown item list**.
+"Some — not all — of the exported functions accept a wrong-typed argument and
+misbehave. Find out which modules actually have this problem (probe every
+export; keep only the misbehaving modules), then for EACH affected module only:
+add top-of-function argument validation (TypeError naming function, parameter,
+expected type; behaviour for valid input unchanged) and `tests/<module>.guards.test.js`
+(node:test, one test per guard). Do not modify existing tests or touch modules
+that already validate. Finish with `npm test` passing and report exactly which
+modules you changed and which you left alone, with evidence." Same fixture
+family, same single pool (`claude-opus-5`), 0.13.1 installed after the goal-2
+run ended so the binary each run used is unambiguous. Run `wf-mtdkvx0k-c40480`,
+23:23:56 → 23:52:41 Z.
+
+| when (Z) | what |
+|---|---|
+| 23:23:59 → 23:28:05 | scout (246 s): probed all six modules; found exactly three misbehaving (csv, slugify, semver) with per-function evidence |
+| 23:28:05 → 23:32:59 | planner turn 1 (294 s): **one 9-action program with `completion: {when: "all-actions-ok"}`** — `fix-{csv,slugify,semver}` + `audit-remaining` (independently re-probe duration/intervals/lru/index) in parallel, each with its own `verify-*` carrying `repair {maxRounds: 2}`, then `verify-suite` |
+| 23:32:59 | 4 workers started in the same second |
+| 23:37:04 → 23:39:31 | each `verify-<m>` started as its own fix finished (pipeline, no barrier) |
+| 23:42:44 → 23:52:41 | `verify-suite` (597 s) ok:true |
+| 23:52:41 | **runtime recorded `complete` itself** — `decision.auto_completed`, `source: program-completion`; no second planner process |
+
+| metric | value |
+|---|---|
+| wall | **28 min 42 s** (1 722 s), no quota wait |
+| planner turns / seconds | **1 / 294 s (17 %)** |
+| dispatches / max concurrent / parallelism | 11 / 4 / 1.82 (four items → four chains; width was item-bound, cap 8 unused) |
+| repairs | 0 needed (every verify ok:true first time) |
+| actions by source | planner 9; completion recorded by the runtime |
+| result (audit-fixture.sh + `npm test`) | `src/csv.js`, `src/semver.js`, `src/slugify.js` modified (24/9/9 non-comment lines); duration/intervals/lru/index untouched; 3 new `*.guards.test.js`; existing tests byte-identical; **75/75** (52 + 23) |
+| tokens (estimate) | 51 422 |
+
+Two notes. First, the planner did **not** use `fanout.itemsFrom` — it inlined
+the three modules the scout had already named and gave the "not yet confirmed"
+half of the repo to one `audit-remaining` worker. That is the right call (the
+scout had done the discovery), and it is exactly what Claude's author does when
+it discovers the list inline before writing the script; `itemsFrom` stays the
+tool for lists that only exist after a worker runs. Second, `verify-others` and
+`verify-suite` both flagged `lru` throwing `RangeError` rather than `TypeError`
+for a wrong-typed capacity and both correctly treated it as informational (the
+existing test pins `RangeError`): passing-with-nits produced no extra work,
+as the doctrine intends.
+
 ## Behaviour differences observed
 
 Same goal, same fixture, same model (Opus for every worker and for bullswarm's
 planner; the Claude session's author was Opus too). Read left to right: what
 Claude did, what bullswarm 0.11.1 did on the identical run, and what 0.12.x
-now does about it (built and unit-tested; the live re-run below is the
-confirmation).
+now does about it. Every 0.12.x/0.13.x cell is unit-tested; cells marked
+**observed** were also seen live in the `g2-bs-v3` (0.12.1) and `g3-bs-v3`
+(0.13.1) runs above.
 
-| Dimension | Claude Code `Workflow` (ultracode) — observed | bullswarm 0.11.1 — observed | bullswarm 0.12.0 / 0.12.1 — built |
+| Dimension | Claude Code `Workflow` (ultracode) — observed | bullswarm 0.11.1 — observed | bullswarm 0.12.1 / 0.13.1 |
 | --- | --- | --- | --- |
-| Who plans, and when | The session author read every file and ran the tests inline (4 min), then wrote **one script** (23 k chars, 5 `agent()` sites). **0 orchestrator turns during the 48 min 51 s of execution.** | The planner compiled the **whole 14-action graph in one decision** (253 s) — but blind: goal text + cwd only, no repo survey, no worker output text in its context. Consulted **3 times** (253 s, 304 s, 110 s) = **27 % of wall**. | Read-only `scout` action before the planner; `outputExcerpt` of every finished action in the planner context; prompt reframed as "compile the goal into a PROGRAM"; planner told it is consulted only at the program boundary. |
+| Who plans, and when | The session author read every file and ran the tests inline (4 min), then wrote **one script** (23 k chars, 5 `agent()` sites). **0 orchestrator turns during the 48 min 51 s of execution.** | The planner compiled the **whole 14-action graph in one decision** (253 s) — but blind: goal text + cwd only, no repo survey, no worker output text in its context. Consulted **3 times** (253 s, 304 s, 110 s) = **27 % of wall**. | Read-only `scout` action before the planner; `outputExcerpt` of every finished action in the planner context; prompt reframed as "compile the goal into a PROGRAM"; planner told it is consulted only at the program boundary. **Observed:** both runs compiled the whole program on turn 1 from the scout's survey; goal 3 ran on **one planner turn** (0.13.0 self-completion). |
 | Item discovery | `pipeline(MODULES, probe, author, verify, fix-loop)` over a known list; when a list is unknown Claude discovers it inline *before* writing the script. | Goal named the six modules → inlined them. Nothing to discover here. | `fanout.itemsFrom: "outputs.<discovery>.outFile"` resolved at run time (+ one bounded read-only extraction retry), so an unknown item count never costs a planner turn. |
 | Parallel width and overlap | **6 concurrent** (= six items, cap 8), mean parallelism 3.1. Per-item pipeline: author-B starts the second probe-B ends; no barriers. | **6 concurrent**, mean parallelism 2.74. Ready-set scheduler: each `verify-<m>` started the second its own `module-<m>` finished; `docs-index` waited for all six by design. | Unchanged for known items. Limitation stays: a verify on a *discovered* fan-out waits for all items (no per-item chain inside a fan-out yet). |
-| Verify → fix | Fix loops **pre-authored in code** (`while (!verdict.ok && rounds < N)`): slugify ×2, intervals ×1, all inside the script; 9 verifies, 3 fixes, 0 planner involvement. | A failed/blocked verify came back to the **planner** (turn 2, 304 s), which authored `slugify-recheck` + `verify-slugify-2`. Round trip ≈ 5 min before the fix even started. | `verify.repair { prompt, maxRounds 1–3 }` — the executor runs `<verify>-repair-<n>` with the concerns verbatim and re-runs the same verify; only still-failing verifies return to the planner. |
-| Passing verifies with nits | Schema-forced `{ok, issues}`; the script fixes only when `!ok`. Nits on passing modules were ignored. | Planner spent **2 of 3 remediation fixes** (`polish-semver`, `polish-lru`) on "non-blocking" notes from verifiers that had returned `ok:true` — an extra ~10 min program round. | Doctrine line: an `ok:true` verify is accepted; its concerns are informational. |
-| Robustness to content | Prompts are JS strings; the runtime substitutes nothing. A parse error in the *script* was caught by the harness and corrected inline in 94 s. | The template renderer parsed **any** `{{…}}` — in a planner prompt *and* in the review artifact it appended. `verify-slugify` died at render time with **0 attempts**, blocked `verify-suite`, cost a planner round, and the fix **rewrote fixture source** (JSDoc) to dodge the bug. | Only a known root + dotted identifiers is a template ref; other double braces are text. `verify` appends the reviewed artifact verbatim, never rendered. |
-| Provider rate limits | Agents retry on API errors; a terminal error resolves the agent to `null`, the script keeps going. The session waits. | Pool burst-gated (5h window 91 %) → the whole run **failed in 4 s** with `no eligible pool`, no reset time named (first 0.12.0 launch, 19:09 Z). | 0.12.1: `waiting_for_quota` stage, meter re-read every 60 s, continue when the window resets; fail only after reset + 10 min grace, naming pool / usage / reset time. |
+| Verify → fix | Fix loops **pre-authored in code** (`while (!verdict.ok && rounds < N)`): slugify ×2, intervals ×1, all inside the script; 9 verifies, 3 fixes, 0 planner involvement. | A failed/blocked verify came back to the **planner** (turn 2, 304 s), which authored `slugify-recheck` + `verify-slugify-2`. Round trip ≈ 5 min before the fix even started. | `verify.repair { prompt, maxRounds 1–3 }` — the executor runs `<verify>-repair-<n>` with the concerns verbatim and re-runs the same verify; only still-failing verifies return to the planner. **Observed** (goal 2): `verify-full-delivery` failed on a missing `docs/README.md`, the repair wrote it and the re-verify passed, ~9 min, no planner turn. Exposed the 0.13.1 bug (repair never counted as verified). |
+| Passing verifies with nits | Schema-forced `{ok, issues}`; the script fixes only when `!ok`. Nits on passing modules were ignored. | Planner spent **2 of 3 remediation fixes** (`polish-semver`, `polish-lru`) on "non-blocking" notes from verifiers that had returned `ok:true` — an extra ~10 min program round. | Doctrine line: an `ok:true` verify is accepted; its concerns are informational. **Observed:** 6 + 4 passing verifies with concerns in the two runs, zero polish actions. |
+| Robustness to content | Prompts are JS strings; the runtime substitutes nothing. A parse error in the *script* was caught by the harness and corrected inline in 94 s. | The template renderer parsed **any** `{{…}}` — in a planner prompt *and* in the review artifact it appended. `verify-slugify` died at render time with **0 attempts**, blocked `verify-suite`, cost a planner round, and the fix **rewrote fixture source** (JSDoc) to dodge the bug. | Only a known root + dotted identifiers is a template ref; other double braces are text. `verify` appends the reviewed artifact verbatim, never rendered. **Observed:** same pristine `slugify.js` with `{{maxLength?: number}}`, `verify-slugify` ran and passed. |
+| Provider rate limits | Agents retry on API errors; a terminal error resolves the agent to `null`, the script keeps going. The session waits. | Pool burst-gated (5h window 91 %) → the whole run **failed in 4 s** with `no eligible pool`, no reset time named (first 0.12.0 launch, 19:09 Z). | 0.12.1: `waiting_for_quota` stage, meter re-read every 60 s, continue when the window resets; fail only after reset + 10 min grace, naming pool / usage / reset time. **Observed:** waited 3 h 2 min at 95 %, resumed 17 s after the reset, no operator action. |
 | Structured worker output | `schema:` forces a `StructuredOutput` tool call; mismatches retry at the tool layer, so the script never parses prose. | Prose "content gate" (`looksLikeWork`); a bare JSON array answer was rejected as an "announcement"; the verify verdict is the only structured channel. | Content gate accepts JSON; `parseJsonArray` prefers the trailing array; one extraction action when discovery output has no array. A general `outputSchema` on run actions is still open. |
 | Failure semantics | In code: `parallel()` never rejects, a throwing stage drops its item to `null`, `.filter(Boolean)`. | Runtime `onError: continue` per step; failed dependencies block dependents; blocked graph → planner. | Same, plus: a failed scout is non-fatal; fan-out `ok` is a boolean so dependents can wait on a whole fan-out. |
-| Outcome quality (audits, read-only) | 168/168 tests (52 + 116 new); existing tests byte-identical; `src` comment-only; every deliverable present. | 130/130 tests (52 + 78 new); existing tests byte-identical; `src` comment-only; every deliverable present. | — (re-run below) |
-| Time and agents | 58 min end to end; 24 agents. | 41 min end to end; 22 dispatches. Faster because it wrote fewer tests per module (11–15 vs 16–24) and skipped Claude's probe stage — not because it orchestrated better. | — |
+| Outcome quality (audits, read-only) | 168/168 tests (52 + 116 new); existing tests byte-identical; `src` comment-only; every deliverable present. | 130/130 tests (52 + 78 new); existing tests byte-identical; `src` comment-only; every deliverable present. | Goal 2 on 0.12.1: 120/120 (52 + 68); existing tests byte-identical; `src` comment-only; every deliverable present. Goal 3 on 0.13.1: exactly the three unguarded modules changed, 75/75. |
+| Time and agents | 58 min end to end; 24 agents. | 41 min end to end; 22 dispatches. Faster because it wrote fewer tests per module (11–15 vs 16–24) and skipped Claude's probe stage — not because it orchestrated better. | Goal 2 on 0.12.1: 48 min of execution (+3 h quota wait), 22 dispatches, max 7 concurrent — ~11 min of it spent on the 0.13.1 bug and ~9 min on one repair round. Goal 3 on 0.13.1: 28 min 42 s, 11 dispatches, 1 planner turn. |
 
 The short version: after 0.11.x the *shape* already matched (one decision =
 whole graph, six in parallel, per-item verify overlap). What still separated
@@ -425,8 +471,9 @@ the runtime.
 
 ## What to change in bullswarm
 
-**Shipped in this cycle** (0.12.0 `c1b71a8`, 0.12.1 `beeed94`; all covered by
-unit tests, 295/295):
+**Shipped in this cycle** (0.12.0 `c1b71a8`, 0.12.1 `beeed94`, 0.13.0
+`6e5f620`, 0.13.1 `1bf0840` — all released to npm and installed; unit suite
+299/299):
 
 1. Orchestrator as **compiler**: prompt reframed; planner consulted only at the
    program boundary; `programFeatures: ['itemsFrom', 'repair']` advertised.
@@ -442,11 +489,18 @@ unit tests, 295/295):
 8. Template refs are grammar-checked; review artifacts are never rendered.
 9. Doctrine: `ok:true` verifies are accepted; concerns are informational.
 10. Burst-gated providers are waited for (`waiting_for_quota`), not failed.
-11. **Program-level completion predicate** (`0bcfd21`, 0.13.0 unreleased while
-    the 0.12.1 observation run is in flight): `completion: { when:
+11. **Program-level completion predicate** (0.13.0): `completion: { when:
     "all-actions-ok", reason }` lets a clean program record its own `complete`
-    decision — no final planner turn (110 s here) just to say so. Anything
-    failing still returns to the planner.
+    decision — no final planner turn just to say so. Anything failing still
+    returns to the planner. **Observed** on goal 3: the planner attached it
+    unprompted, the runtime recorded `complete` (`source: program-completion`)
+    at 23:52:41 Z, one planner turn for the whole run.
+12. **A repaired verify counts as verification of its repair** (0.13.1): the
+    completion-evidence check only followed `verify.dependsOn`; a repair action
+    depends on its verify (reverse edge), so after a clean repair round every
+    `complete` was rejected as "missing a successful verification of latest
+    worker <verify>-repair-1" — observed on goal 2 (three extra planner turns,
+    ~11 min) and it would have blocked item 11 in the same situation.
 
 **Still open, in priority order** (each is a measured gap, not a guess):
 
@@ -467,3 +521,14 @@ unit tests, 295/295):
    whatever the planner wrote. A default skeptic framing in the verify wrapper
    is cheap and would have caught nothing extra here — listed for parity, not
    urgency.
+5. **Dependency slips by the planner.** Goal 2's `write-docs-index` was
+   compiled with `dependsOn: []` although it reads the six `docs/<m>.md` files
+   the builders create; it ran first and found nothing. The verify + repair
+   loop recovered it (~9 min). Claude has the same failure class (a mis-ordered
+   `pipeline` stage) and the same recovery. A doctrine line — "an action that
+   reads another proposed action's deliverable must depend on it" — is free;
+   a deterministic check is not possible without declared outputs, which would
+   be a small schema addition (`produces: [paths]`).
+6. **Record `completion` on the decision.** The planner artifact carries the
+   predicate but `state.decisions[]` does not, so `watch`/metrics cannot show
+   that a program declared itself self-completing until it does. One field.

package/docs/planner-prompt-audit-2026-08-29.md

@@ -0,0 +1,157 @@
+# Planner prompt and context audit — 2026-08-29
+
+Question from the user: after the 0.11 → 0.14 iterations, does the instruction
+set given to the orchestrator/planner still make sense, or should it be
+simplified or refactored?
+
+Method: measure what the planner actually receives, not what the source files
+look like. The two planner task files of dogfood run `wf-mtdq1l9v-ed22fe`
+(`75t4n2`, bullswarm building `outputSchema` in its own repo, runtime 0.13.2)
+are the specimens; sizes are characters of the task text
+(`task-orchestrator-*.md`), tokens ≈ chars / 4.
+
+## 1. What one planner turn receives
+
+| section | source | turn 1 | turn 2 |
+| --- | --- | ---: | ---: |
+| orchestrator prompt + worktree line | `goal.js` `AUTONOMOUS_ORCHESTRATOR_PROMPT` | 5 000 | 5 000 |
+| PLANNING DOCTRINE (11 bullets) | `runtime.js` runDecision | 3 605 | 3 605 |
+| Action skeletons (6 shapes + verify semantics) | `runtime.js` | 1 342 | 1 342 |
+| Program skeleton (discovery → fan-out → verify → suite) | `runtime.js` | 1 217 | 1 217 |
+| Graph skeleton (two chains + suite) + fanout paragraph + runtime-owned line | `runtime.js` | 4 054 | 4 054 |
+| durable context (JSON) | `runtime.js` plannerContext | 17 474 | 162 946 |
+| **total** | | **32 730** (~8 k tokens) | **178 452** (~45 k tokens) |
+
+The fixed prefix is 15.2 k chars on every turn. The durable context grew
+**9×** between turn 1 and turn 2 of the same run.
+
+### Where the 163 k of turn 2 went
+
+| key | chars | what it is |
+| --- | ---: | --- |
+| `completedActions` (19 entries) | 66 700 | every finished action **with its full attempt records**: routing candidates and pace numbers, usage, pricing table, child pid, timings — ~3.5 k per action |
+| `outputs` (19 entries) | 41 560 | `outputExcerpt` of ~3 k chars for *every* finished action, including ones that finished ok and were already verified in the previous program |
+| `intent` | 6 657 | the goal text (6.5 k) + cwd + policy — needed, once |
+| `failures` | 1 073 | the two blocked actions — duplicates of `completedActions` entries |
+| `availablePools`, `budget`, `executionConstraints`, `closedPhases`, … | ~1 800 | fine |
+
+## 2. What is said more than once
+
+Reading the prefix as the planner does, the same rules appear two or three
+times in different words:
+
+| rule | orchestrator prompt | doctrine | skeletons |
+| --- | --- | --- | --- |
+| propose the whole program, one round trip costs minutes | item 2 | bullets 1, 2 | "all in ONE decision" ×2 |
+| N items → N run + N verify + one suite verify | item 5 | "Per-item chains" | Graph skeleton |
+| unknown items → discovery + `itemsFrom` fan-out | item 5 | "Unknown item count" | Program skeleton + fanout paragraph |
+| every verify gets a `repair` policy | item 5 | "Verification failures" | verify skeleton |
+| `completion: all-actions-ok` on a clean program | item 5 | "Self-completing programs" | — |
+| self-contained worker prompts, file ownership | item 3 | "File ownership", "Self-contained prompts" | — |
+| don't propose pool/addDir/taskFile | closing line | — | final line |
+
+Item 5 of the orchestrator prompt alone is 1 050 chars and restates four
+doctrine bullets. The two program skeletons both end in the same
+`verify-items → verify-suite` tail.
+
+## 3. Does it matter? Measured
+
+- Turn 1 (32.7 k chars) took **637 s**; turn 2 (178 k chars) took **390 s**.
+  Latency is therefore dominated by the model's reasoning on the goal, not by
+  context size — the 6.5 k-char goal and a 12-action program cost more thinking
+  than reading 45 k tokens. Trimming context is a **cost** and **attention**
+  lever, not primarily a latency lever.
+- Cost: turn 2 read ~45 k tokens to emit a ~1 k-token decision. At Opus
+  prices that is ~$0.25 per boundary; a run with four boundaries (goal 2 on
+  0.12.1) spends more on re-reading attempt metadata than on the decisions.
+- Attention: the planner's turn-2 reason correctly diagnosed the blocked
+  graph, so quality did not visibly suffer here — but 64 k chars of pricing
+  tables and routing candidates are noise it must skip to find the two
+  `ok:false` concerns that matter.
+- Behaviour observed in three runs (goal 2, goal 3, dogfood): every rule the
+  prefix repeats was followed on the first turn (whole program, per-item
+  chains, repair policies, `completion`). No observed decision needed a rule
+  to be stated twice.
+
+## 4. Recommendation
+
+Yes — refactor, in two independent pieces, both measurable:
+
+**A. Compact the durable context (the 9× growth).** Planner-facing ledger rows
+instead of raw ledger entries: `{ id, type, phase, status, pool, durationSec,
+attempts, why }` (~150 chars; 19 actions → ~3 k instead of 66.7 k). Keep a
+full `outputExcerpt` only for actions finished **since the last decision** and
+for every `ok:false` verify; older ok actions get a one-line summary (id, ok,
+first 200 chars). Replace `failures` with the ids of failing actions (their
+full entry already sits in the ledger). Expected turn-2 context: ~25 k chars
+instead of 163 k. Pure runtime change; no planner behaviour change intended.
+
+**B. One contract instead of three overlapping texts.** Merge
+`AUTONOMOUS_ORCHESTRATOR_PROMPT` and the doctrine bullets into a single ordered
+list of ~10 rules (target ≤ 4 k chars, from 8.1 k), each stated once with its
+reason; keep exactly two JSON examples — the action shapes list and one
+complete program (discovery → data-driven fan-out → per-item verify with
+repair → suite verify, with `completion`) — and delete the second program
+skeleton (target ≤ 3 k, from 6.6 k). Total prefix ≤ 7 k chars, from 15.2 k.
+
+Acceptance for both: unit tests on the context builder (row shape, excerpt
+policy by decision sequence) and on the prompt (each rule appears once; the
+skeleton assertions in `tests/workflow-adaptive.test.js` updated); then one
+re-run of goal 3 on the same fixture (baseline 0.13.1: 28 min 42 s, 1 planner
+turn, 294 s) to confirm the decision shape is unchanged and record the new
+per-turn size.
+
+Not recommended: cutting the goal text or the scout excerpt from the context —
+both were used verbatim by every first-turn program observed.
+
+## 5. Outcome
+
+Measurements below were taken after the refactor from the current source and
+from the committed source saved into `/tmp/goal-before.mjs` and
+`/tmp/runtime-before.js`, using the same temporary measurement script. The
+canonical prefix is the complete emitted planner task text counted from its
+first character up to (not including) the durable-context marker, with the
+worktree-isolation suffix included. The committed baseline predates the named
+section exports, so its emitted prefix was reconstructed from the committed
+`runtime.js` task-text assembly and the committed `AUTONOMOUS_ORCHESTRATOR_PROMPT`.
+
+- Complete emitted planner task text up to the durable-context marker,
+  worktree-isolation suffix included: **OBSERVED**, `16,316` characters before
+  and `5,208` characters after. Command: `node /tmp/measure-planner.mjs`.
+- Static planner task-prefix array through the durable-context boundary:
+  **OBSERVED**, `16,209` characters before and `5,101` characters after. The
+  after value is 107 characters shorter because it excludes the unchanged
+  worktree-isolation suffix; this is a secondary source-level measurement, not
+  the canonical emitted-prefix headline. Command: `node /tmp/measure-planner.mjs`.
+- `PLANNER_RULES_SECTION`: **OBSERVED**, `2,202` characters after. Command:
+  `node /tmp/measure-planner.mjs`.
+- `PLANNER_EXAMPLES_SECTION`: **OBSERVED**, `1,867` characters after. Command:
+  `node /tmp/measure-planner.mjs`.
+- `AUTONOMOUS_ORCHESTRATOR_PROMPT`: **OBSERVED**, `4,670` characters after;
+  the committed before source had no separately exported rules or examples
+  sections, so separate before-section sizes are **NOT AVAILABLE**, not
+  inferred. Command: `node /tmp/measure-planner.mjs`.
+- Sample durable context: **OBSERVED** baseline `163,000` characters in the
+  audit's rounded turn-2 durable-context total (the detailed table records
+  `162,946`; the full turn-2 task was `178,452`), and **COMPUTED** `23,547`
+  characters after. The computed sample applies 19
+  compact ledger rows at 150 characters each, keeps a 3,000-character scout
+  excerpt and two 3,000-character failing-verify excerpts, truncates the
+  other 16 action excerpts to 200 characters, represents two failures as
+  20-character IDs, and retains the audit's 6,657-character intent and
+  1,800-character other-context components. Command: `node /tmp/compute-context.mjs`.
+
+Deliverables:
+
+- Durable planner context shrank because completed actions are compact ledger
+  rows, failures are IDs, and stale successful output is truncated.
+- Planner contract shrank because overlapping prompt/doctrine/skeleton text is
+  now one ordered rules section plus exactly two JSON examples, single-sourced
+  in `src/workflow/goal.js`.
+- Runtime prompt construction shrank because `src/workflow/runtime.js` imports
+  the shared contract instead of carrying a duplicate doctrine and graph
+  skeleton.
+- `skill/SKILL.md` was left unchanged: it documents the general durable
+  context and the separate run-state/TUI attempt view, but does not document a
+  renamed/dropped planner-context field shape such as the old attempt records
+  or an old `failures` representation.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bullswarm",
-  "version": "0.13.1",
+  "version": "0.14.0",
   "description": "Route work across coding-agent CLI subscriptions — paced by live quota meters, verified by content, never trusting exit codes.",
   "type": "module",
   "bin": {

package/skill/SKILL.md

@@ -361,6 +361,11 @@ that expressible without extra turns:
   (`source: "program-completion"`, event `decision.auto_completed`) and the run
   ends without another planner turn; a failing action emits
   `decision.completion_predicate_unmet` and the boundary returns to the planner.
+- `outputSchema` on a `run` or fan-out `stepTemplate` — declare it when a
+  downstream action needs reliable structured data, such as an object to render
+  into a dependent prompt or an `items` array for `fanout.itemsFrom`; leave it
+  off for ordinary prose; planner proposals must not put it on `verify`, whose
+  verdict shape is fixed.
 
 Every fan-out records a summary artifact as `outputs.<id>.outFile` and a
 boolean `ok` (item count in `succeeded`), so a verify may depend on a fan-out

package/src/workflow/decision.js

@@ -3,6 +3,7 @@
 // or dispatch any new action.
 
 export const DECISION_SCHEMA_VERSION = 'bullswarm.workflow.decision.v1';
+import { isValidOutputSchema } from './schema.js';
 export const DECISIONS = new Set([
   'proceed', 'complete', 'needs_more_work', 'retry', 'escalate',
   'wait_for_approval', 'stop',
@@ -40,7 +41,7 @@ export function parseDecisionText(text) {
 export const REVIEW_PATH_RE = /^outputs\.([A-Za-z0-9_-]+(?:\[\d+\])?)\.outFile$/;
 // Data-driven fan-out source: the artifact of an earlier (or co-proposed)
 // action whose output ends with a JSON array of items.
-export const ITEMS_FROM_RE = /^outputs\.([A-Za-z0-9_-]+)(?:\.outFile)?$/;
+export const ITEMS_FROM_RE = /^outputs\.([A-Za-z0-9_-]+)(?:\.data\.([A-Za-z0-9_-]+)|\.outFile)?$/;
 export const REPAIR_MAX_ROUNDS = 3;
 // Program-level completion predicates a planner may attach to a program so the
 // runtime can record completion itself when every action finishes ok.
@@ -179,7 +180,15 @@ export function validateDecisionProposal(proposal, {
     if (action.type === 'run' && typeof action.prompt !== 'string') {
       issues.push(`${at} needs a prompt`);
     }
-    if (action.type === 'fanout') {
+       if (action.outputSchema !== undefined) {
+         if (action.type !== 'run') issues.push(`${at}.outputSchema is only allowed on run actions`);
+         else {
+           const schema = isValidOutputSchema(action.outputSchema);
+           if (!schema.ok) issues.push(...schema.issues.map((issue) => `${at}.outputSchema: ${issue}`));
+           else if (action.outputSchema.type !== 'object') issues.push(`${at}.outputSchema.type must be "object"`);
+         }
+       }
+       if (action.type === 'fanout') {
       const hasItems = Array.isArray(action.items);
       const hasItemsFrom = action.itemsFrom != null;
       if (!hasItems && !hasItemsFrom) {
@@ -198,10 +207,15 @@ export function validateDecisionProposal(proposal, {
         }
       }
       if (!action.stepTemplate || typeof action.stepTemplate !== 'object') issues.push(`${at}.stepTemplate is required`);
-      for (const runtimeOwned of ['pool', 'addDir', 'taskFile']) {
+       for (const runtimeOwned of ['pool', 'addDir', 'taskFile']) {
         if (action.stepTemplate?.[runtimeOwned] != null) {
           issues.push(`${at}.stepTemplate.${runtimeOwned} is runtime-owned and cannot be proposed by a planner`);
-        }
+       }
+       if (action.stepTemplate?.outputSchema !== undefined) {
+         const schema = isValidOutputSchema(action.stepTemplate.outputSchema);
+         if (!schema.ok) issues.push(...schema.issues.map((issue) => `${at}.stepTemplate.outputSchema: ${issue}`));
+         else if (action.stepTemplate.outputSchema.type !== 'object') issues.push(`${at}.stepTemplate.outputSchema.type must be "object"`);
+       }
       }
     }
     if (action.repair != null && action.type !== 'verify') {