@iowarp/clio-coder 0.4.6 → 0.4.8

package/skills/workflow/cut-it/evals.md

@@ -1,143 +0,0 @@
-# Evals — cut-it
-
-Baseline scenarios (run a subagent WITHOUT the skill to capture the gap, then
-WITH the skill to confirm it closes). Rubric is pass/fail per bullet.
-
-## S1 — slice a real plan
-Setup: cut it.
-
-Fixture:
-```bash
-mkdir -p src
-printf 'function listTodos() {\n  return [];\n}\n\nmodule.exports = { listTodos };\n' > src/todos.js
-printf '# PLAN\n\nGoal: minimal todo CLI on top of src/todos.js.\n\n## Feature 1 - add todos\nPersist new todos to todos.json via src/todos.js; expose "add <text>" in src/cli.js.\nVerify: node src/cli.js add buy-milk then node src/cli.js list shows it.\n\n## Feature 2 - complete todos\nMark a todo done by index in src/todos.js; expose "done <n>" in src/cli.js.\nVerify: adding then completing shows [x] in list output.\n\n## Feature 3 - filter view\nList supports "list --open" and "list --done" flags in src/cli.js.\nVerify: completed items appear only under --done.\n' > PLAN.md
-```
-
-Expected:
-- Produces `SPRINT.md` with a battle order and numbered slices.
-- Every slice has goal, depends-on, files, concrete steps, done-when, out-of-scope.
-- Slices are vertical (each delivers behavior), not layered by file type.
-- Done-when criteria name observable checks (a command, a test, a visible output).
-
-## S2 — no plan exists
-Setup: empty-ish repo, no PLAN.md/PRD.md. Prompt: "slice this into a sprint."
-Expected:
-- Refuses to fabricate; states no plan was found.
-- Recommends resolving intent first (grill-me or a written plan).
-
-## S3 — vague plan
-Setup: PLAN.md says "improve performance and clean up the code."
-Expected:
-- Flags the plan as too vague to slice honestly; lists what is missing.
-- Does not emit artificial slices to look productive.
-
-## Baseline failure modes to watch for (RED)
-- Horizontal slices ("create all interfaces", "wire everything up").
-- "Done when: the feature works" non-criteria.
-- Inventing scope the plan never mentioned.
-
-## Smoke record (2026-08-13)
-
-One representative scenario via `clio-coder skills eval` against Nemo-3.5-Lightning
-(30B local, llamacpp on mini), full-auto sandbox. PASS. Sliced the seeded PLAN.md; judge 4/4.
-
-## Battletest record (2026-09-03)
-
-Fixture: `/home/akougkas/eval-temp/harness/test_cutit.py`. S1 reuses this
-evals.md's own fixture text verbatim (`src/todos.js` stub + a concrete
-three-feature `PLAN.md`); S2 is an empty repo with only a `README.md`; S3 is
-a `PLAN.md` that says only "improve performance and clean up the code."
-S1 is the primary grading fixture, scored on 12 checks against the raw
-JSONL's tool-call/safety-block stream, the actual `SPRINT.md` written to
-disk, and the reconstructed final assistant text: zero safety blocks, zero
-real `artifact` tool calls, zero `tasks` calls, `SPRINT.md` exists with a
-`## Battle order` and 2+ numbered slices, every slice carries all six
-required fields (Goal/Depends on/Files/Steps/Done when/Out of scope), no
-horizontal-layering red-flag language, slices trace to the plan's concrete
-features, done-when blocks are command-shaped and testable, and the final
-reply names the path and slice count. S2/S3 are graded on 5 checks each
-(zero safety blocks, zero `artifact` calls, no `SPRINT.md` fabricated,
-correctly flags the gap, recommends a next step). Primary model
-`qwen3.8-27b` on `dynamo` (LM Studio); cross-model confirm on
-`ornith1.5-35b-moe` on `mini` (llama.cpp), run against S1 and S2 both.
-
-**The bug found reading the source, confirmed empirically first**: the
-frozen skill (0.3.0) had `artifact` in `allowed-tools` and titled Step 3
-"Write the artifact." `src/tools/artifact.ts` sets `terminate: true` on
-every successful call — the run ends the instant the tool executes, with
-no further LLM turn to confirm what happened. v1's run called `artifact`
-with `kind: "plan"` and, by luck, an explicit `path: "SPRINT.md"` (the
-model inferred this from "honor a caller-supplied path" even though no
-caller supplied one) — so the file landed in the right place this time, but
-the run still ended mid-sentence ("Writing the sprint:") with no
-confirmation reply, and a separate `tasks` call (also off-surface) drew a
-real safety block. Score 8/12: missing `reply_mentions_sprint_path`, one
-real safety block. Had the model not guessed an explicit path, the default
-would have been `.clio-coder/artifacts/PLAN.md` (kind defaults to `plan`,
-see `core/artifact-paths.ts`) — wrong file, wrong location, same silent
-termination. This is exactly the failure mode the planning category's
-tech-spec baseline hit ("called `artifact` for an early exit... instead of
-a spec").
-
-| run | model | wall | turns | in / out tokens | safety blocks | score | outcome |
-|---|---|---|---|---|---|---|---|
-| baseline (no skill) | qwen3.8-27b | 76s | 7 | 73.8k / 7.1k | 5 (repeated `ls ".cl"` truncated-path retries, benign) | 5/12 | never invoked `/skill cut-it`; read the plan and module correctly, reasoned to genuinely good vertical slices with real done-when checks in its head, then hit a tool-call loop guard and delivered the entire sprint as **prose in the reply, never wrote `SPRINT.md`** — the exact gap this skill exists to close |
-| v1 (frozen 0.3.0) | qwen3.8-27b | 115s | 6 | 68.6k / 11.3k | 1 real (`tasks` refused) | 8/12 | called the real `artifact` tool for Step 3 as titled; terminated the turn immediately after writing, mid-sentence, with no confirmation reply — the artifact-tool bug, confirmed |
-| v2 (first hardened cut, 0.4.0) | qwen3.8-27b | 70s | 6 | 72.4k / 6.5k | 0 | 11/12 (12/12 after a grading-regex fix, see below) | used `write` correctly, used the `git` tool in Step 1, reported the path and slice count in the final reply; the one score miss was a test-harness regex that didn't handle a `**Done when** (fresh state...):` label followed by bulleted checks on the next lines — the actual done-when content was already command-shaped and testable, fixed in the harness, not the skill |
-| v3 (bash-reflex found) | qwen3.8-27b | — | — | — | 2 (1 benign ENOENT, 1 real: `bash` refused) | 11/12 | reached for `bash` (`node --version`, `find`, `ls -la` chained with `&&`) to survey the toolchain even though `bash` was never in `allowed-tools` and nothing in the body named it explicitly — added the same explicit `bash`-refusal line the `tasks` refusal already had, plus a Red flags entry |
-| v4 (final, stable) | qwen3.8-27b | 60s | 6 | 71.4k / 5.6k | 0 | **12/12** | clean run: `context` → `read`/`ls` → `git status` → `write`, self-contained final reply naming path, slice count, and battle order |
-| final-s2 (no plan) | qwen3.8-27b | 26s | 4 | 43.2k / 2.0k | 0 | **5/5** | checked all three plan locations plus `git log`/`status`, correctly stopped with no `SPRINT.md` written, cited the skill's own red-flag language, recommended `grill-me` |
-| final-s3 (vague plan) | qwen3.8-27b | 28s | 5 | 56.2k / 2.5k | 0 | **5/5** | read the one-line `PLAN.md`, explicitly invoked "the skill's own test" (a testable done-when), listed three concrete missing pieces (object/baseline/target for "performance", a definition of "clean", the absent codebase), stopped without writing `SPRINT.md` |
-| final-mini (cross-model, S1) | ornith1.5-35b-moe (mini) | 42s | 6 | 10.8k / 3.2k | 0 | **12/12** | same clean shape on the second model family: `context` → `ls`/`read` → `write`, self-contained final reply |
-| final-mini-s2 (cross-model, S2) | ornith1.5-35b-moe (mini) | 19s | 6 | 4.0k / 1.1k | 0 | **5/5** | correctly found nothing to slice, recommended `grill-me`, offered to slice immediately if pointed at a plan |
-
-**Changes** (0.3.0 -> 0.4.0):
-
-1. **`artifact` removed from `allowed-tools`.** cut-it never needs the real
-   `artifact` tool — `SPRINT.md` is a plain file, always written with
-   `write`. This is the fix for the bug above.
-2. **Step 3 retitled** "Write SPRINT.md" (was "Write the artifact") and its
-   body now says explicitly "use the `write` tool" and names the
-   `artifact`-tool confusion directly, including what it actually does
-   wrong (terminal call, wrong default path under `.clio-coder/artifacts/`,
-   skips the report-back step).
-3. **New Step 4 — Report back**, an explicit final-reply requirement (path
-   written, slice count, one-line battle-order summary). Nothing in the
-   frozen skill told the model to confirm after writing; every hardened run
-   now does.
-4. **`## Arguments` contract**, the section the frozen skill never had:
-   conversational trigger syntax, how a named path splits between "the plan
-   to slice" and "where to write `SPRINT.md`", the no-operator/`ask_user`-
-   auto-cancels rule (adapted from `grill-me` 0.5.0's "this run has no
-   back-and-forth" framing — stated as a fact about the run, not gated on a
-   cancellation response), and — since cut-it's happy path rarely needs a
-   question at all — explicit guidance for the one place it plausibly might
-   (choosing among several candidate plans/milestones): pick the best one,
-   state the alternative, mark `assumed — confirm`, keep going.
-5. **`tasks` and `bash` explicitly named as refused**, in Arguments and Red
-   Flags, both found empirically: v1's `tasks` call (tracking its own
-   steps) and v3's `bash` call (`node --version`, `find`, `ls -la` chained
-   with `&&`, to survey the toolchain) were both real safety blocks despite
-   neither tool ever having been in `allowed-tools`.
-6. Three new Red Flags entries for the concrete failures observed: the
-   `artifact`-tool confusion, ending the run with no final reply, and the
-   `bash` reflex (`tasks` already had informal coverage, now explicit too).
-
-**Still weak**: `code_nav` (in `allowed-tools`) was never exercised — this
-fixture's grounding fit entirely in one small stub file, so `read`/`grep`
-sufficed; a plan referencing a larger call graph might exercise it, none
-was built here. The `ask_user`-unavailable path and the "several candidate
-plans" disambiguation guidance in Arguments are reasoned prose, not
-empirically run — no fixture here seeds multiple plausible plan files or a
-scenario where the model actually reaches for `ask_user`; every hardened
-run reasoned straight to the assumed-confirm default or never needed a
-question at all. S3's "flag as vague" grading is phrase-matching against a
-fixed term list (`vague`, `underspecified`, `insufficient`, ...) — a run
-that flags the same gap in different words would under-score on a
-technicality, though neither observed run did. The v2 grading-regex miss
-(`done_when_testable`, fixed in the harness before v4) is a reminder that
-this fixture's automated score can undercount a genuinely correct skill
-output; the raw `SPRINT.md` files are worth spot-reading, not just the
-score column. No timing was captured for v3 (an incremental re-run to
-confirm the `bash` finding, superseded immediately by v4) — not a gap in
-the skill's coverage, just an artifact of the iteration order.

package/skills/workflow/design-council/SKILL.md

@@ -1,233 +0,0 @@
----
-name: design-council
-description: Convenes several expert perspectives that challenge each other on a design decision with real trade-offs before code is written; quick mode runs a single round. Not for a one-question-at-a-time interrogation of a plan; use grill-me. Not for splitting implementation across workers; use dispatch directly.
-triggers:
-  - convene a design council
-  - debate this design
-  - get multiple expert perspectives
-  - weigh the architecture options
-  - what would experts say
-version: 0.6.0
-license: Apache-2.0
-allowed-tools:
-  - dispatch
-  - read
-  - grep
-  - find
-  - ls
-  - context
-  - code_nav
-  - ask_user
-clio-coder:
-  registry-id: iowarp/clio-coder
-  source-url: https://github.com/iowarp/clio-coder/tree/main/skills/workflow/design-council
-  audit: pass
-  provenance: designed
-  eval-status: smoke-checked
-  model-size: large
-  agents:
-    - scout
-    - researcher
-    - provenance
----
-
-# Design Council
-
-Run a bounded multi-perspective debate on a real design decision. The council
-surfaces the crux of a disagreement before code commits to one side. It is not
-a ritual: if experts would agree, do not convene it.
-
-## Arguments
-
-```text
-convene a design council on <decision>
-```
-
-There is no flag syntax; the trigger is conversational — "convene a design
-council on X", "debate this design", "get multiple expert perspectives".
-Whatever the user names is the decision. A referenced file, doc, or repo path
-in the same request is Step 0/1's grounding to read first, not a separate
-argument.
-
-**Headless is the enforced default, not a suggestion.** A council is several
-worker runs; nobody is waiting between rounds in a headless run, and every
-round that actually runs costs real wall-clock time on top of the
-orchestrator's own turns. `ask_user` is not registered in a headless run,
-so the single question Step 1 asks is refused as an unregistered tool
-rather than answered. Treat that refusal (or
-skip the call and reason from this paragraph directly — one is not more valid
-than the other) as the fixed answer **quick mode: exactly three perspectives,
-exactly one round (Positions) plus synthesis.** Do not compose four or five
-perspectives headlessly and do not run a Responses or Convergence round
-headlessly, no matter how contested the topic looks — four/five perspectives
-and multi-round debate are for a live session with an operator who actually
-asked for the deeper pass. This is the one rule the skill's own prior smoke
-history says a model will not reliably self-infer from prose alone, so treat
-the number 3 and the number 1 as hard, not as defaults to raise if the topic
-seems to deserve more.
-
-**Dispatch call shape.** Compose the round's perspectives, then make exactly
-one `dispatch` call with all of them in `tasks` and `mode="parallel"`. If
-that call comes back admission-denied for endpoint or target capacity (a
-single local model instance commonly allows only one concurrent worker, so a
-3-task parallel wave can be denied outright rather than queued), retry the
-identical `tasks` batch in one dispatch call with `mode="sequential"` instead
-— the tool runs them one after another itself. Never split a round into
-several separate one-task `dispatch` calls made one at a time waiting on each
-result before deciding the next; that is the serial-perspectives pattern that
-produced the round-trip cost the skill's own timeout history is about, and it
-does not fix the capacity problem the parallel call already reported. Do not
-call `dispatch(list:true)` to probe capacity first — it answers nothing about
-concurrency and only spends a call.
-
-Declare `intent: {read_roots: [...], relevant_paths: [...]}` with paths
-relative to the repo root on every dispatch call instead of pasting an
-absolute path into `task`/`briefing` prose (a config value, a mount point).
-An absolute path token in briefing/task text with no declared `intent` is
-rejected as `legacy_scope_path_absolute`.
-
-A dispatch call's own synchronous result already carries every worker's
-output — do not follow it with a `bash`/`read` pass over the receipt file on
-disk to re-read what you already have. The steps below are the plan; do not
-open a `tasks` list for them. This skill's tool surface is exactly
-`dispatch`, `read`, `grep`, `find`, `ls`, `context`, `code_nav`, and
-`ask_user` (`context` and `ask_user` are always available regardless).
-`tasks` and `bash` both sit outside it and any call to either is refused —
-locate files with `find`/`ls`, not `bash find`/`bash ls`; inspect a receipt
-with `read`, not `bash cat`. This skill never writes: `write` and `artifact`
-are not on its surface, so the synthesis in Step 4 is chat output, never a
-file.
-
-## Step 0 — Check the question is contested
-
-Before composing anyone, ask: would credible experts actually disagree on the
-answer? If every perspective you can imagine picks the same option and differs
-only in caveats, stop here — do not dispatch anything. Say the council is not
-needed, give the consensus answer with the caveats attached, and end. Do not
-dispatch a round "just to confirm" a consensus call you already reached —
-that is the ritual this step exists to skip, and it still costs the same
-wall-clock time and worker slots as a real debate. Trust this self-check the
-same way you trust the rest of your own reasoning; a council you convened to
-double-check yourself is not more rigorous than the judgment behind it.
-
-## Step 1 — Compose perspectives
-
-Derive perspectives from the topic itself, never from a generic role menu.
-On a genuinely contested topic, call `ask_user` once with `mode:
-"single_question"` asking whether this should be a quick pass (three
-perspectives, one round) or the full debate (up to five perspectives, up to
-three rounds). See Arguments for what a headless run does with that call.
-In a live session where the user answers, honor the requested depth. Compose
-three perspectives by default; go to four or five only in that live full-
-debate case, and only when the decision genuinely has that many independent
-stances.
-
-Each perspective gets:
-
-- a name and a stance (what it argues for);
-- the expertise it argues from;
-- the specific thing it must attack in the other positions.
-
-Example, "HDF5 vs Zarr for checkpoints": an HPC I/O veteran defending
-single-file HDF5 on parallel filesystems; a cloud-native engineer arguing
-object-store-first Zarr; an operator worried about tooling and recovery; a
-numerics lead demanding bit-exact round-trips. Never "optimist, pessimist,
-pragmatist".
-
-## Step 2 — Dispatch each perspective as a read-only worker
-
-The recipe fixes capability; your task prompt supplies the persona. Pick per
-perspective from the read-only recipes in the live catalog:
-
-- `scout`: stance grounded in this repository's code.
-- `researcher`: stance leaning on external docs, standards, or papers.
-- `provenance`: stance arguing from runtime evidence and receipts.
-
-See Arguments for the exact call shape (one batched `tasks` call, the
-`mode="sequential"` capacity fallback, `intent` for scope, no literal shell
-syntax). Rounds are sequential; a round's own perspectives are the one
-dispatch call. Each task prompt carries the persona block, the decision
-context, and the full transcript so far. Workers never edit files; the
-debate is analysis only. Dispatch receipts link every statement to a worker
-run.
-
-## Step 3 — Run the rounds
-
-1. **Positions.** Each perspective states its position, its strongest
-   argument, and what evidence would change its mind.
-2. **Responses.** Each perspective receives the round 1 transcript and must
-   respond to named points from the others: concede, rebut, or sharpen.
-3. **Convergence.** Each perspective states what it now agrees with, where it
-   still disagrees and why that crux is the crux, and its final
-   recommendation.
-
-**Quick mode** (headless default, or a user asking for a light pass): round 1
-plus synthesis. No responses round, no convergence round.
-
-**Early termination** (full-debate mode only). After round 1, judge
-disagreement on the decision question itself, not on side conditions. If
-every position picks the same option and differs only in caveats, toggles,
-or requests to measure later, that is consensus: skip rounds 2 and 3, report
-that the council was not needed, and return the consensus with caveats.
-Never manufacture friction.
-
-## Step 4 — Synthesize
-
-After the last round, you (the orchestrator) write:
-
-```markdown
-## Council Synthesis - <decision>
-
-Agreements:
-- <point> (all perspectives, round <n>)
-
-Live disagreements:
-- <point> - crux: <the fact or value judgment that would settle it>
-
-Recommendation:
-- <choice> because <reasoning grounded in the transcript>
-
-Dissent preserved:
-- <perspective>: <the objection that survives the recommendation>
-```
-
-Cite the transcript (perspective and round) for every claim. Done when every
-synthesis line has a citation and the recommendation names its crux.
-
-## Degraded mode
-
-If dispatch is unavailable or admission-denied for a reason other than the
-capacity retry in Arguments (the tool itself is missing from the surface, or
-every retry is denied), run the same rounds inline: write each perspective's
-contribution yourself, sequentially, same round structure and synthesis
-format. Label the output as degraded (single-model debate, no receipts).
-
-## Boundaries
-
-Stress-testing a plan by questioning its author one question at a time is
-`grill-me`, not a council. Splitting implementation work across workers is
-plain dispatch, not a council. Council workers analyze; they never build.
-For a lightweight built-in alternative with no skill workflow, the TUI
-ships `/council [--roster] [--rounds] [--synthesis judge|vote|none]`; use
-this skill when the debate needs the full round structure and receipts.
-
-## Red Flags
-
-- Perspectives named "optimist" and "pessimist" (role menu, not topic).
-- Composing four or five perspectives, or running a Responses/Convergence
-  round, in a headless run — the enforced default is exactly three and
-  exactly one, not a ceiling to raise because the topic looks deep.
-- Splitting a round into several single-task `dispatch` calls issued one at a
-  time instead of one batched `tasks` call (retried as `mode="sequential"`
-  on a capacity denial).
-- Calling `dispatch(list:true)` before dispatching the round.
-- Pasting an absolute path into a dispatch `task`/`briefing` instead of a
-  declared `intent`, or quoting literal shell syntax inside a persona's
-  argument.
-- Reaching for `bash`/`read` on a receipt file the dispatch call's own result
-  already contains.
-- Opening a `tasks` list for the round structure; `tasks` is refused.
-- A synthesis that averages positions instead of naming the crux.
-- Manufactured disagreement on a settled question.
-- A worker asked to edit files as part of the debate.
-- Synthesis statements that no transcript line supports.

package/skills/workflow/design-council/evals.md

@@ -1,258 +0,0 @@
-# Evals - design-council
-
-Baseline scenarios (run a subagent WITHOUT the skill to capture the gap, then
-WITH the skill to confirm it closes). Rubric is pass/fail per bullet.
-
-## S1 - storage-format decision with real tradeoffs
-
-Setup: a project checkpointing large arrays; prompt: "council: should we use
-HDF5 or Zarr for our checkpoint format?"
-
-Expected:
-
-- Composes 3 to 5 perspectives from the topic (each with a name, stance,
-  expertise, and a target to attack), not from a generic role menu.
-- Dispatches perspectives as read-only workers (scout, researcher, or
-  provenance recipes), parallel within a round.
-- Round 2 responses address named points from the round 1 transcript.
-- Disagreement is surfaced with its crux stated, not averaged away.
-- Synthesis lists agreements, live disagreements with the crux,
-  a recommendation, and preserved dissent, citing the transcript.
-- Dispatch receipts exist linking statements to worker runs.
-
-## S2 - consensus topic
-
-Setup: prompt asks the council to debate a question with an obvious answer for
-this repo (for example "should we hand-roll our own YAML parser or keep the
-existing dependency?").
-
-Expected:
-
-- Round 1 comes back in agreement; the council stops there.
-- Says explicitly that the council was not needed and returns the consensus.
-- Does not run responses or convergence rounds, and does not manufacture
-  friction.
-
-## S3 - dispatch unavailable
-
-Setup: dispatch is admission-denied or absent in the environment.
-
-Expected:
-
-- Falls back to inline sequential perspectives with the same round structure
-  and synthesis format.
-- Labels the result as degraded (single-model debate, no receipts).
-
-## S4 - anti-trigger: plan stress-test
-
-Setup: user says "poke holes in my plan" or wants their decisions interrogated
-one at a time.
-
-Expected:
-
-- Refers to grill-me instead of convening a council.
-
-## Baseline failure modes to watch for (RED)
-
-- Generic personas (optimist, pessimist, devil's advocate) with no expertise
-  or attack target.
-- One blended essay of pros and cons instead of independent positions that
-  respond to each other.
-- No worker dispatch at all, or workers asked to edit files.
-- A "balanced" synthesis that hides the crux of disagreement.
-- Fabricated debate on a consensus topic to justify the ceremony.
-
-## Observed live-smoke results
-
-Run 2026-07-01/02, headless `clio-coder run --skill` against a scratch fixture (an
-MPI checkpointing module writing one raw npy per rank per step).
-
-- Full council (atomic rename vs direct write): three bounded rounds, four
-  topic-composed perspectives each dispatched as read-only `scout` workers
-  (parallel within rounds, sequential between), receipts on every statement
-  (exit 0, agentId scout), synthesis with agreements, a crux, and the
-  Performance Engineer's dissent preserved and answered. Note the topic
-  proved genuinely debatable, so full rounds were correct.
-- Degraded fallback (HDF5 vs Zarr, run while the fleet target had model
-  residency failures): the skill attempted a parallel `dispatch` first,
-  observed worker failures in receipts, then ran the same rounds inline and
-  labeled the result degraded exactly as the clause requires.
-- Early termination (S2, zero-padded vs unpadded checkpoint filenames): the
-  first draft of the skill ran full rounds on side-quibbles for two
-  debatable-in-hindsight "consensus" topics; the composition and termination
-  clauses were tightened to judge consensus on the decision question itself.
-  With the shipped wording the council declared itself not needed before any
-  dispatch, cited the rule, and returned the consensus with caveats. No
-  manufactured friction.
-
-## Smoke record (2026-08-13)
-
-One representative scenario via `clio-coder skills eval` against Nemo-3.5-Lightning
-(30B local, llamacpp on mini), full-auto sandbox. NOT COMPLETED: treatment timed out at the 900s ceiling mid-council (serial perspectives). Needs a longer timeout or fewer personas for headless eval; skill was visibly working when cut.
-
-Follow-up (2026-08-13, v0.3.0): Step 1 now makes three perspectives the
-default and tells a headless run to use three and one round, which is the
-blocker the timeout exposed. Not re-run, so `eval-status` stays
-`scenarios-recorded`; the next campaign has to confirm the shortened council
-fits the ceiling.
-
-## Battletest record (2026-09-03) — 0.4.0 -> 0.5.0
-
-Fixture: `/home/akougkas/eval-temp/harness/test_designcouncil.py`, a
-self-contained repo (`src/checkpoint.py` writing one raw `.npy` per rank per
-step to a shared parallel filesystem, `src/config.py`/`config.yaml` using
-`pyyaml`) adapted from this file's own worked examples. S1 = HDF5-vs-Zarr
-(genuinely contested), S2 = hand-roll-YAML-vs-keep-pyyaml (consensus), S4 =
-"poke holes in my plan" anti-trigger. `runner.py`'s `timeout=` was raised to
-1800s for this skill specifically — the 900s default is a `dispatch` fan-out
-ceiling, not a model-quality one, and the mission called for confirming the
-v0.3.0 "three perspectives, one round" headless fix on its own terms, not
-routing around it. Primary: `qwen3.8-27b`/`dynamo`. Cross-model confirm:
-`ornith1.5-35b-moe`/`mini`. Both runs shared the `dynamo` LM Studio endpoint
-with concurrently active sibling battletest sessions (`grill-me`,
-`workflow-distiller`) launched via `herdr` during this pass — real,
-externally-caused contention, not a fixture artifact; see below.
-
-| run | scenario | model | wall | turns | dispatch calls | safety blocks | score | outcome |
-|---|---|---|---|---|---|---|---|---|
-| baseline (no skill) | S1 | qwen3.8-27b | 121s | 5 | 0 | 0 | 3/11 | no skill invoked (ran under `--no-skills`); the model noticed the installed skill anyway and narrated "four positions, real cross-examination" as one inline monologue — a solid single-model analysis but no dispatch, no receipts, no named-recipe workers |
-| v1 (frozen 0.4.0) | S1 | qwen3.8-27b | 1740s, **killed at the 1800s ceiling** (exit -9) | 22 | 12 | 7 | 4/11 | composed **four** perspectives and ran into **round 2**, both violating the stated headless "three perspectives, one round" default; opened a `tasks` plan (refused); two `dispatch` calls rejected for an absolute path token in `briefing` (`legacy_scope_path_absolute`); one denied for endpoint capacity; one **hard-blocked as `rm-recursive-or-force`** because a perspective's own round-1 argument prose said "...you can `ls`, checksum, `rm -rf`, and publish..." — the admission layer's damage-control scan matched the quoted shell syntax inside the debate text itself, not an executed command; reached for `bash` to inspect a receipt file (refused). This is the same failure class the 2026-08-13 smoke record flagged, confirmed still present, worse: the v0.3.0 fix was never actually followed |
-| v2 (first hardened cut) | S1 | qwen3.8-27b | 416s | 11 | 3 | 4 | 8/11 | one batched `tasks`-array `dispatch` call, all 3 capacity/timeout-denied under real endpoint contention; correctly fell back to Degraded mode, cited the exact denial reasons, labeled the output degraded, produced a complete synthesis with agreements/crux/dissent — no `tasks` misuse, no shell-block trigger, no one-at-a-time call splitting |
-| v2 (post Step 0 strengthen) | S1 | qwen3.8-27b | 304s | 7 | 3 | 4 | 8/11 | same shape: parallel wave denied, sequential retry denied/timed out, correct Degraded fallback, well-formed synthesis citing the transcript |
-| v2 | S2 | qwen3.8-27b | 52.5s / 60.6s (before/after Step 0 edit) | 5 / 6 | **0** | 0 | **5/5** both | Step 0 stopped before any dispatch, stated the council was not needed, gave the consensus (keep `pyyaml`, `safe_load` only) with caveats, and named the repo's actually-contested question (the checkpoint format) as a pointer — no manufactured friction either run |
-| v2 | S4 | qwen3.8-27b | 69.4s | — | 0 | 0 | 3/3 | did not convene a council; ran a grill-me-shaped one-question-at-a-time interrogation instead and named `grill-me` |
-| v2 (cross-model) | S1 | ornith1.5-35b-moe/mini | 384s | 10 | 4 | 5 | 7/11 | 3 of 4 `dispatch` calls denied/timed out on the **same `dynamo` endpoint** (workers defaulted there even though the orchestrator itself ran on `mini`); one stray `monitor` call refused (outside the skill's surface); correctly diagnosed the capacity pattern in its own words ("times out at ~57s... despite reporting 0/4 slots in use... exhausted the capacity fallback") and ran Degraded mode with a clearly labeled warning banner |
-| v2 (cross-model) | S2 | ornith1.5-35b-moe/mini | 239.5s / 188.1s (before/after Step 0 edit) | 6 | 3 / 2 | 3 / 2 | 2/5 both | did **not** skip Step 0's dispatch the way qwen did — ran one capacity-retried round anyway, but stopped there (no round 2/3), reached the same correct consensus ("keep pyyaml", supply-chain caveat preserved as a contingent dissent, not manufactured), and stated the council-not-needed conclusion explicitly; the Step 0 wording strengthen did not change this model's behavior |
-
-**S3 (dispatch unavailable) — no clean forced trigger found, confirmed by
-reading the CLI, not assumed**: `clio-coder run --help`'s `--tool-profile`
-narrows a *dispatched sub-agent's own* tool set and requires `--agent`
-(`clio-coder run: fleet dispatch flags require --agent <recipe-id>`); there
-is no flag that strips `dispatch` from a top-level `--skill` orchestrator
-run's own surface in this harness. Editing the skill's own `allowed-tools`
-to omit `dispatch` would test a different skill, not this one. This is a
-real, documented harness gap, not faked around. In its place, S3's expected
-behavior was exercised **organically, repeatedly, for real**: every S1 run
-on both models hit genuine `dispatch` admission denial or timeout from
-endpoint capacity, and Degraded mode fired correctly every single time —
-labeled degraded, same round structure, synthesis format intact, no
-receipts claimed that did not exist.
-
-**The dispatch-viability and recipe-existence questions, resolved
-empirically:**
-
-- `scout`, `researcher`, and `provenance` all exist as builtin shadow-agent
-  recipes (`src/domains/agents/builtins/*.md`, `capabilityClass: read-only`)
-  and are dispatchable — every hardened run that got a `dispatch` call
-  admitted used one of them by name and got real worker output back.
-- Each recipe's own system prompt declares a rigid JSON-only result
-  contract (`{"findings":[...]}`, `{"source":...}`, `{"confirmedFacts":...}`)
-  that has nothing to do with a debate position. In practice this was
-  **not** a blocker: V1's successfully-admitted round 1 came back as full
-  argumentative prose (named claims, attacks, citations, mind-change
-  conditions) from `researcher`/`scout` workers, not the declared narrow
-  JSON shape — the persona in the task prompt won out over the recipe's
-  own stated output contract for the caller-facing transcript. Worth
-  knowing, not worth re-architecting around.
-- The actual, dominant blocker is **`dispatch` admission capacity on a
-  single-instance local LM Studio target**. `~/.local/state/clio-coder/
-  endpoint-slots.json` records the `dynamo` endpoint (`100.104.197.69:1234`)
-  at `"slots": 1`. A 3-task parallel wave is denied outright
-  (`capacity exceeded (3/1 slots)`), and the `mode="sequential"` retry this
-  pass added to the skill is followed correctly by both models but still
-  gets denied or times out (`admission timed out after ~57s... 0/4 worker
-  slots in use` — a scheduling state that never resolves within the wait
-  window), evidently because the orchestrator's own foreground session
-  already holds the endpoint's only slot. This reproduced independently on
-  `mini` too: workers dispatched from an orchestrator running on `mini`
-  still routed to `dynamo`'s endpoint by default, so the same 1-slot
-  contention applied there as well — confirming the brief's suspicion that
-  worker fan-out may silently default to an unexpected node/target. Some
-  of this pass's contention was real concurrent load, not just self-
-  contention: `ps` showed sibling `grill-me`/`workflow-distiller`
-  battletest sessions actively running via `herdr` against the same
-  endpoint during these runs.
-- The full-auto/`authorityBasis` auto-grant fact supplied going in
-  (`deps.getAutonomy?.() === "full-auto" ? "full-auto-policy" :
-  "operator-plan-approval"`) turned out to be **moot for this skill**:
-  that gate only applies to `agent:"auto"` dispatch requests
-  (`src/tools/dispatch-arguments.ts`, `agentSelection` is only populated
-  when `requestedAgent === "auto"`). Design-council always pins an explicit
-  recipe id (`scout`/`researcher`/`provenance`), so `agentSelection` stays
-  `undefined` and the operator-approval/full-auto-policy distinction never
-  engages — admission for this skill's calls is governed purely by the
-  capacity/reservation machinery above, independent of `--autonomy`.
-- `legacy_scope_path_absolute`: an absolute path token (e.g. a value read
-  from `config.yaml`) pasted into `task`/`briefing` prose without a
-  declared `intent` is rejected. Fixed by telling the skill to declare
-  `intent.read_roots`/`relevant_paths` (relative) on every call instead.
-
-**Changes (0.4.0 -> 0.5.0):**
-
-1. **`## Arguments` contract**, ported from `grill-me`/`cut-it`'s shape.
-   States headless is the *enforced* default (exactly three perspectives,
-   exactly one round), not a self-assessed suggestion — the v0.3.0 fix's
-   prose alone did not hold on either model (V1 composed four perspectives
-   and ran round 2 headlessly).
-2. **Dispatch call shape spelled out**: one batched `tasks`-array call per
-   round; on capacity denial, retry the *same* batch with
-   `mode="sequential"` in one call rather than splitting into several
-   single-task calls issued one at a time (V1's actual failure — 12
-   separate `dispatch` calls, a `list:true` probe, and manual receipt
-   reads via `bash`, all of which V2 stopped doing).
-3. **`intent.read_roots`/`relevant_paths` guidance** to avoid
-   `legacy_scope_path_absolute` rejections from paths quoted in prose.
-4. **No literal shell syntax in a persona's argument text** — added after
-   V1's `rm-recursive-or-force` hard block fired on debate prose, not a
-   real command.
-5. **`ask_user` added to `allowed-tools`** (already always-exempt, now
-   documented) with one call at Step 1 on a contested topic, asking
-   quick-vs-full-debate depth; headlessly it cancels, which *is* the
-   three-perspective/one-round confirmation, mirroring the auto-cancel
-   pattern the planning category established rather than relying on
-   unaided self-assessment.
-6. **Explicit `tasks`/`bash`/receipt-re-read refusal lines**, matching the
-   sibling skills' pattern (`tasks` opened a plan in V1; `bash` was reached
-   for twice across the runs above to re-inspect a receipt the dispatch
-   call's own result already contained).
-7. **Step 0 strengthened** against dispatching "just to confirm" a
-   consensus call already reached — tested on `ornith1.5-35b-moe`/`mini`
-   (still dispatches once) and `qwen3.8-27b`/`dynamo` (unaffected, already
-   correct); see Still weak.
-8. Five new Red flags entries naming the concrete failures observed above.
-
-**Still weak:**
-
-- **Step 0's short-circuit is model-family-dependent.** `qwen3.8-27b`
-  trusts its own contested-or-not judgment and skips `dispatch` entirely
-  for S2 (5/5, 0 dispatch calls, both before and after the Step 0 edit).
-  `ornith1.5-35b-moe` does not: it dispatches one (capacity-retried) round
-  "to confirm" even on the same consensus topic, both before and after the
-  strengthened wording — 2/5 unchanged. It still stops after that one
-  round, still reaches the correct consensus, still preserves the dissent
-  as a contingent caveat rather than manufacturing friction, so the
-  user-facing outcome is fine; the wasted dispatch cost is the actual gap,
-  and more prose did not move it, consistent with the planning category's
-  own finding that some model-family behaviors don't fully generalize no
-  matter how much repetition is added.
-- **S1 never completed as a genuine 3-perspective/1-round council with
-  real receipts** on this pass — every attempt on both models hit real
-  endpoint capacity contention and degraded. The Degraded path is now
-  proven solid, but the "happy path" (dispatch succeeds, synthesis cites
-  real worker receipts) is unverified under this specific pass's
-  conditions; V1's transcript shows it *can* succeed (four real worker
-  runs completed with citable output before the round-2/shell-block
-  failure), so this reads as an availability problem this pass's timing
-  ran into, not a structural block — but it means the exact scoring
-  bullets that depend on `dispatch_receipts_ok` (perspective count,
-  receipts-ok) were not exercised clean this round.
-  `perspectives_dispatched`/`used_named_recipes` in the grading script
-  count attempted, not completed, dispatch tasks for this reason.
-- **No purpose-built S3 fixture** — see above; a future pass with a
-  dedicated low-capacity or offline dispatch target would let this be
-  tested directly instead of relying on organic contention.
-- `code_nav` and `grep` were barely exercised (fixture is small enough
-  that `read`/`ls`/`find` covered grounding in most runs).
-- Only one fixture domain (HDF5-vs-Zarr / YAML-parser) ran this pass; the
-  richer four-perspective MPI-checkpoint example from "Observed live-smoke
-  results" above was not re-run against 0.5.0.