@iowarp/clio-coder 0.4.1 → 0.4.3

package/skills/planning/architecture/SKILL.md

@@ -1,13 +1,13 @@
 ---
 name: architecture
-description: 'Use when an intent (PRD, epic, brief, or idea) needs its engineering approach decided — "how should we build this", "pick the stack", "architecture for this feature". An interactive working session: investigates, proposes 2-3 genuinely different approaches with trade-offs, recommends with reasoning, lets the user decide, and writes a high-level architecture decision doc. Not a task-by-task plan; use cut-it for that. Not a multi-perspective debate; use design-council. Not product intent; use product-intent. Not a typed implementation handoff with code-shaped contracts; use tech-spec.'
+description: "Decides the engineering approach for an intent in an interactive session: investigates, proposes two or three genuinely different approaches with trade-offs, recommends, lets the user decide, and writes the decision doc. Not a task-by-task plan; use cut-it. Not a multi-perspective debate; use design-council. Not product intent; use product-intent. Not a typed implementation handoff; use tech-spec."
 triggers:
   - how should we build this
   - pick the stack
   - architecture for this feature
   - decide the engineering approach
   - compare architecture options
-version: 0.2.2
+version: 0.4.0
 license: Apache-2.0
 allowed-tools:
   - read
@@ -51,17 +51,48 @@ stack they know beats a "better" one they don't), leanness (decide only
 what is needed to move), and reversibility (spend deliberation on the
 expensive calls only).
 
-When asking is unavailable (headless or worker run), do not skip the loop
-and do not go quiet: for each decision, state the options, the
-recommendation, and the reasoning inline, adopt the recommendation, and
-mark it `assumed — confirm`. Every other rule still applies, including the
-output path below.
+## Arguments
+
+```text
+/skill architecture <the intent — a PRD path, a brief, or a few sentences>
+```
+
+- The text is what to decide the engineering approach for. A PRD path,
+  brief, or reference doc named or pathed in the request is Step 0's
+  grounding to read first, not more arguments.
+- Nothing is required beyond some text; a blank invocation falls straight
+  to Step 0's own question — what are we building, and is there a written
+  intent to read — rather than inventing a project to architect.
+
+There is no operator in a headless run: `ask_user` still executes, but with
+nothing to answer it every call returns immediately with no answers, every
+time — calling it again will not produce a different result. From wherever
+the first empty response lands — Step 0's "ask whether reference docs
+exist", Step 1's greenfield/brownfield call, or any decision in Step 2 —
+switch immediately to the treatment already described above (state the
+options, the recommendation, the reasoning, adopt it, mark it `assumed —
+confirm`) and keep running every remaining step through Step 3 in the same
+turn. Never stop at the first unanswered gate and never go silent; a
+one-shot document produced without ever attempting the loop is also wrong
+in the other direction — always investigate and state the loop out loud
+first, degrading only once a gate actually goes unanswered. Never invent a
+fact, evidence, or a codebase detail to back an assumption; anything
+genuinely uncertain becomes an open question or a spike, never a plausible
+guess.
+
+The steps below (Step 0 through Step 3) are the plan; do not open a task
+list for them. `tasks` sits outside this skill's tool surface and any call
+to it is refused. `bash` is also outside this skill's tool surface —
+investigate with `read`, `grep`, `find`, `git`, and `code_nav` instead;
+reaching for `bash` to explore or verify is refused.
 
 ## Step 0 — Ground
 
 Read the intent (PRD path, brief, or the user's words). Read any reference
 docs passed alongside; if none were passed, ask whether any exist — the
-context needed is usually already written down.
+context needed is usually already written down. To see what exists in the
+repo before reading it, use the `find` tool (e.g. pattern `**/*`) or `ls`,
+never `bash find`/`bash ls` — `bash` is refused outright, see Arguments.
 
 ## Step 1 — Greenfield or brownfield
 
@@ -99,11 +130,17 @@ user's call. Skip what does not apply and say that you skipped it:
 
 ## Step 3 — Write the decision doc
 
-Only after the calls are made. Default location: `docs/architecture-<slug>.md`
-(or folded into the PRD as an `## Architecture` section when the user
-prefers one document; a tracker page when the user names one and an
-integration exists). Never invent another filename — `final_report.md`,
-`REPORT.md`, and similar are wrong. Shape:
+Only after the calls are made. There are exactly three valid locations, in
+order of default preference — pick one, never a fourth:
+
+1. `docs/architecture-<slug>.md` — the default, no exceptions needed.
+2. An `## Architecture` section folded into the named PRD, only when the
+   user said they prefer one document.
+3. A tracker page, only when the user names one and an integration exists.
+
+No other filename or location is ever correct. `final_report.md`,
+`REPORT.md`, `summary.md`, and anything similar are hard misses, not
+stylistic variants — if none of the three above fit, use option 1. Shape:
 
 ```markdown
 # Architecture — <intent name>
@@ -133,3 +170,15 @@ sprint (`cut-it`), spike a flagged risk now, or keep refining here.
 - One foregone conclusion instead of real alternatives.
 - File-by-file edit lists (that is cut-it's altitude).
 - A one-way door decided by vibe instead of a spike.
+- A decision doc at any filename other than the three valid locations
+  above — `final_report.md`, `REPORT.md`, and similar are wrong every time.
+- A headless run that stops at the first unanswered `ask_user` call instead
+  of running the assumed-confirm treatment through every remaining step, or
+  one that skips straight to Step 3 without ever running the loop out loud.
+- Opening a task list for Steps 0-3; `tasks` is refused. Reaching for
+  `bash` to explore the repo or verify the written doc; `bash` is not in
+  this skill's tool surface and the call is refused — use `read`, `grep`,
+  `find`, `git`, and `code_nav` instead.
+- A claim in the doc that traces to neither the read intent/codebase nor a
+  fact marked `assumed — confirm` — an invented detail reads as confident
+  and is the hardest failure to catch after the fact.

package/skills/planning/architecture/evals.md

@@ -34,3 +34,68 @@ Expected:
 
 One representative scenario via `clio-coder skills eval` against Nemo-3.5-Lightning
 (30B local, llamacpp on mini), full-auto sandbox. SMOKE ACTED but substance bullets failed (wrote final_report.md, skipped the loop). Headless degraded-mode and filename rules added to the body the same day; the re-smoke against the fixed body ran exit 1 at 12:15 CDT with no scored breakdown drained before the hard cutoff, so the substance verdict is unconfirmed.
+
+## Battletest record (2026-09-03)
+
+Fixture: `/home/akougkas/eval-temp/harness/test_architecture.py`, continuing
+the planning category's shared HPC log-triage domain from `product-intent`/
+`prd`. Brownfield: seeds the actual `docs/hpc-log-triage.prd.md` intent doc
+and a partial codebase (`src/scanner.py`: working `FailureEvent` + OOM-only
+`scan_oom`, ECC/Xid not implemented; `requirements.txt` with click/pyyaml)
+inside a git repo. The prompt asks for the v1 engineering approach for
+dragon-cluster and blade-cluster and names the one genuine one-way-door
+tension from the fixture family: an always-on log-ingestion pipeline vs.
+on-demand reads. Combines S1 (brownfield: read existing surfaces, reuse
+`FailureEvent`/`scan_oom` rather than re-spec) with a spike-worthy
+one-way-door call into one gradable run. Graded 13 checks against real
+post-run disk state (`docs/architecture-<slug>.md` at the exact default
+path — no `final_report.md`/`REPORT.md` miss, all seven required sections,
+>=2 genuinely distinct approaches inside "Approaches considered", the
+pipeline-vs-on-demand tension named *and* backed by a spike with a decision
+rule in "Spikes & experiments", grounding in the seeded PRD/code) plus the
+reconstructed final assistant text and the raw JSONL's tool-call/safety-block
+stream. Ran on `mini`/`ornith1.5-35b-moe` only this pass (see Still weak).
+
+| run | model | wall | turns | in / out tokens | safety blocks | score | outcome |
+|---|---|---|---|---|---|---|---|
+| baseline (no skill) | ornith1.5-35b-moe | 110s | 5 | 11.1k / 8.5k | 0 | 3/13 | never invoked `/skill architecture`; investigated correctly (read the PRD and scanner.py) but then called the terminal `artifact` tool mid-reasoning with no arguments and ended the run before writing anything — no decision doc, no alternatives, no spike |
+| v1 (frozen 0.3.0) | ornith1.5-35b-moe | 145s | 9 | 9.7k / 10.0k | 2 real + 1 benign | 10/13 | wrote the correct `docs/architecture-hpc-log-triage.md` with all sections, 3 real alternatives (a table: on-demand / always-on / hybrid-with-gate), the pipeline tension named with a spike and an explicit decision rule, and ran the full headless assumed-confirm loop entirely on its own initiative (the existing body prose already covers this well) — but reached for `bash` once (refused) and opened a `tasks` plan once (refused); a third "error" was a harmless `ls` on a directory the fixture never created |
+| v2 (first hardened cut, 0.4.0) | ornith1.5-35b-moe | 125s | 9 | 5.5k / 7.6k | 1 real + 2 benign | 10/13 | `tasks` call eliminated entirely by the new explicit refusal line; still reached for `bash find .` once before self-recovering with the `find` tool — the added Red flags/Arguments refusal line reduced but did not eliminate the bash reflex, matching the pattern `prd`/`product-intent` saw on this same model family; 2 benign `read` ENOENTs (evidence docs the PRD cites but the fixture never seeded) also counted as errors under this harness's blanket isError grading |
+| v3 (final 0.4.0) | ornith1.5-35b-moe | 159s | 7 | 8.5k / 9.7k | 0 real + 1 benign | 12/13 | zero `bash` calls, zero `tasks` calls, correct path, all 7 sections, 2 distinct approaches, pipeline tension with a spike and decision rule; the one remaining "error" is a harmless duplicate `context` call the harness nudged back onto the loaded skill (not a tool-surface refusal, `context` is always exempt) |
+
+**Changes** (0.3.0 -> 0.4.0): (1) an `## Arguments` contract — the skill had
+solid headless-degradation prose in its intro already, but no formal
+contract section; added slash-invocation syntax, what's required vs.
+inferred, and the explicit no-operator/`ask_user`-returns-empty rule ported
+from `product-intent`/`prd`, extended to name Step 0's evidence-check and
+Step 1's greenfield/brownfield call explicitly (not just Step 2's decisions)
+as places the degradation applies from; (2) explicit `tasks` and `bash`
+refusal lines in Arguments and Red flags — these were v1's two real safety
+blocks; (3) Step 3's filename rule tightened from a permissive "default
+location... never invent another" into an enumerated three-item list with
+"no other filename or location is ever correct," closing the exact gap the
+2026-08-13 smoke record flagged (`final_report.md` on a different model);
+(4) a one-line `find`/`ls`-not-`bash` hint added directly in Step 0, where
+the exploration happens, which is what took the bash-reach from 1/run (v2)
+to 0/run (v3); (5) five new Red flags entries naming the concrete failures
+observed: wrong filename, a stalled or skipped headless loop, `tasks`/`bash`
+reaches, and ungrounded claims.
+
+**Still weak**: per this pass's coordinator note, only `ornith1.5-35b-moe`
+on `mini` was run — no `qwen3.8-27b`/dynamo confirmation this session, so
+cross-model-family generalization is unverified (the sibling `prd`/
+`product-intent` runs found the bash-reflex fix that worked on `qwen3.8-27b`
+did *not* fully generalize to `ornith-1.5-35b-a3b`, so the reverse — a fix
+tuned on ornith not holding on qwen — is a live risk here too, untested).
+The v3 run's one remaining logged "error" is a benign duplicate `context`
+call, not a real defect, but the harness's blanket isError-counting means a
+literal 13/13 may not be reachable without also silencing genuinely benign
+tool responses — not worth chasing. `web_fetch` and `code_nav` (both in
+allowed-tools) were never exercised — this fixture never needed them
+(brownfield, no external research). The baseline's specific failure (an
+unprompted terminal `artifact` call mid-investigation, with no arguments)
+is a skill-selection/tool-choice issue outside this SKILL.md's own body.
+Only one fixture shape ran (brownfield + one-way-door); a pure-greenfield
+scenario (S2's familiarity-vs-fashion pull) and a mid-session altitude
+check (S3, "just list the files to change") from the scenario list above
+were not exercised standalone against 0.4.0.

package/skills/planning/backlog/SKILL.md

@@ -1,18 +1,19 @@
 ---
 name: backlog
-description: Use when a finished PRD or architecture doc must become a real ticket backlog — "create the stories", "turn this PRD into issues", "build the backlog". Decomposes phases and user stories into small tickets with verifiable acceptance criteria, confirms the list, then creates them as GitHub issues (or in another tracker when an integration exists). Not for local sprint slicing into a SPRINT.md; use cut-it.
+description: Turns a finished PRD or architecture doc into a ticket backlog of small stories with verifiable acceptance criteria, confirmed, then created as GitHub issues or in another configured tracker. Not for local sprint slicing into a SPRINT.md; use cut-it.
 triggers:
   - create the stories
   - turn this PRD into issues
   - build the backlog
   - decompose this plan into tickets
   - create GitHub issues from this architecture
-version: 0.2.1
+version: 0.4.0
 license: Apache-2.0
 allowed-tools:
   - read
   - grep
   - ls
+  - git
   - bash
   - tasks
   - ask_user
@@ -33,12 +34,93 @@ clio-coder:
 Turn a finished planning doc into small, engineer-ready tickets on a real
 tracker. Decomposition is tracker-agnostic; creation branches on the target.
 
+## Arguments
+
+```text
+/skill backlog <path to the finished PRD or planning doc>
+```
+
+- Required: the doc path. Named inline, or clearly the file just discussed
+  in context — never a doc invented from memory. If no path can be found or
+  inferred, say so at Step 1 and stop; do not decompose without a doc.
+- Optional, inferred rather than asked for by default: target platform
+  (Step 1's own rule picks it — see below) and a milestone/epic to attach
+  tickets to.
+
+There is no operator in a headless run: `ask_user` resolves immediately
+with no answer, every time it is called — not a stall, and calling it again
+will not produce a different result. **This skill treats headless
+degradation differently at each step below, because Step 3 gates an
+outward, not-cleanly-reversible action — real tickets, GitHub issues or
+persisted local tasks — not a document write:**
+
+- **Step 1 (platform).** The default rule (`gh` when a remote and the `gh`
+  binary are both present; the `tasks` fallback otherwise) is a fact to
+  detect, not a judgment call, and runs the same whether or not anyone is
+  watching — no confirmation needed either way. It is genuinely ambiguous
+  only when the user names a platform with no available integration; if
+  that `ask_user` call goes unanswered, do not guess a fallback and do not
+  silently substitute `gh` or `tasks` for the platform actually requested —
+  state plainly that the named platform is unavailable and confirmation of
+  what to use instead was not obtainable, then stop exactly as Step 3 does.
+- **Step 2 (decompose).** Always run to completion, headless or not.
+  Decomposition itself creates nothing yet, and every source-doc gap still
+  needs surfacing whether or not a human is present to read it.
+- **Step 3 (confirm before creating).** An unanswered `ask_user` here is
+  never a yes, and this gate does not get the assumed-confirm-and-proceed
+  treatment the other planning skills use for their document gates. Finish
+  Step 2, print the full proposed ticket list and target platform exactly
+  as Step 3 already requires, then stop: say explicitly that confirmation
+  is required before any ticket is created, that it was not obtainable in
+  this run, and that Step 4 did not execute. This is the one gate in the
+  planning category's headless story that ends in a stop rather than an
+  assumed yes — filing unrequested public GitHub issues, or persisting
+  local tickets nobody confirmed, on a guess is worse than an incomplete
+  run; the full decomposed list is still delivered as the answer.
+
+The steps below are the plan; hold it in your head, not in a tool — do not
+call `tasks` with `plan`/`add`/`done`/`block`/`drop` to track your own
+progress through Steps 1–5 as if they were a workflow board. What actually
+matters when Step 3 stops without confirmation: the board must end this
+run holding zero net-open items. `tasks` calls here are Step 4's — one
+entry per ticket *already confirmed at Step 3* — and Step 4 never runs
+before that yes. If you do reach for `tasks` before confirmation (to
+stage the proposal, or to check the board is clean), any item you opened
+with `plan`/`add` must be `drop`ped again before you finish, in the same
+run — an item left open on the board is exactly "created without
+confirmation," whether or not you called it that. A read-only
+`tasks(action="list")` changes nothing and is always fine. Do not re-invoke
+`context(scope="skills")` once the skill is already loaded this turn — it
+is refused as a redundant call and wastes a turn; if you need to recheck
+what you already read, use `read`/`grep` on the files themselves.
+
+Shell rules for every `bash` call: one command per call, plain and direct.
+Never use `$(...)` or backticks; they trigger an approval gate a headless
+run cannot answer, and the call is refused outright. Never redirect output
+to `/tmp` (`> /tmp/...`) to stage or capture something for a later step —
+that write needs an approval a headless run cannot give either, and the
+call is refused the same way; just run the command and read its output
+directly, nothing needs staging on disk first. The `git` tool only covers
+`status`/`diff`/`log`; it has no `remote` op, so Step 1's remote and
+`gh`-availability check still goes through `bash` (e.g. `git remote -v`,
+`command -v gh`) — use `git` for a plain status check, `bash` for
+everything else this skill needs from git or `gh`.
+
 ## Step 1 — Inputs
 
-Required: the path to the PRD or planning doc. Target platform: GitHub
-issues via `gh` by default; another tracker only when the user names it and
-an integration for it is available. Platform genuinely ambiguous → ask,
-never guess. Optional: a milestone or epic to attach tickets to.
+Required: the path to the PRD or planning doc (see Arguments). Target
+platform: GitHub issues via `gh` when a git remote exists and the `gh`
+binary is present — check both in one `bash` call (`git remote -v;
+command -v gh`) and trust the result; empty output from `git remote -v` is
+a definitive "no remote", not an inconclusive check that needs a second or
+third method (`git config`, reading `.git/config` directly — the latter is
+a hard-blocked path regardless of this skill). Fall back to local tracking
+automatically (see Step 4) when there is no remote, `gh` is unavailable, or
+the user asked for local tracking, and say which was detected and why. A
+platform the user names with no integration available is genuinely
+ambiguous → ask, never guess, and never silently substitute a different
+platform than the one requested (see Arguments for the headless case).
+Optional: a milestone or epic to attach tickets to.
 
 ## Step 2 — Decompose
 
@@ -55,17 +137,23 @@ each user story becomes one or more tickets. Per ticket, draft:
 Sizing rules: a ticket is at most about a day of work; a ticket that needs
 more than one screen to describe is two tickets. A phase too vague to
 decompose is a gap in the source doc — stop and flag it rather than
-inventing tickets.
+inventing tickets. Flag it by name (which phase, what is missing) in the
+Step 3 list and again in the Step 5 report; do not paper over it with a
+plausible-sounding ticket the source doc never actually specified.
 
 ## Step 3 — Confirm before creating
 
-Print the full proposed list (titles grouped by phase) and the target
-platform, and get explicit confirmation via `ask_user`. Ticket creation is
-outward-facing and not one-click reversible; nothing is created before the
-yes.
+Print the full proposed list, grouped by phase — title *and* acceptance
+criteria per ticket, not titles alone, plus any flagged gaps from Step 2 —
+and the target platform, and get explicit confirmation via `ask_user`.
+Ticket creation is outward-facing and not one-click reversible; nothing is
+created before the yes. When that confirmation cannot be obtained (see
+Arguments), stop here — do not proceed to Step 4.
 
 ## Step 4 — Create
 
+Runs only after Step 3's explicit yes.
+
 GitHub:
 
 ```bash
@@ -82,15 +170,43 @@ or the user asks for local tracking, create each confirmed ticket with the
 `tasks` tool instead (one task per ticket, acceptance criteria in the
 description) and capture the task ids. Say which target was used and why.
 
+Every report in this skill — confirmed-and-created or stopped-before-
+confirmation — is delivered as a chat message, never a file. Do not call
+`write` to save the proposal or the report to disk as a backlog/proposal
+markdown file; `write` is not in this skill's tool surface and the call is
+refused. If the user wants the backlog persisted as a document, that is a
+different, file-producing skill (`prd`, `architecture`), not this one.
+
 ## Step 5 — Report
 
-A table: title → phase → issue number/URL, plus the source doc path and any
-failures. Done when every proposed ticket is either created with its URL
-captured or listed as failed with the error.
+Confirmed-and-created run: a table of title → phase → issue number/URL,
+plus the source doc path and any failures. Done when every proposed ticket
+is either created with its URL captured or listed as failed with the
+error.
+
+Stopped-before-confirmation run (see Arguments): one self-contained final
+message — never split across turns and never "see above" — that repeats
+the full proposed ticket list with title *and* acceptance criteria per
+ticket (a status column of "proposed" is not a substitute for the
+criteria themselves), plus one explicit line stating that confirmation was
+required and unavailable and no ticket was created. A reader who sees only
+this last message must get the complete list and the complete status;
+never let the table alone imply creation happened, and never make the
+stop notice depend on an earlier turn still being visible.
 
 ## Red flags
 
-- Tickets created before the Step 3 confirmation.
+- Tickets created before the Step 3 confirmation — including a headless
+  run that treats an unanswered `ask_user` there as a yes and proceeds to
+  Step 4 anyway; that gate stops, it does not assume.
+- A final report whose ticket table doesn't say, in words, whether creation
+  happened or confirmation was unavailable.
 - Acceptance criteria that restate the title.
 - A mega-ticket hiding a week of work.
 - Tickets with no trace back to a phase or story in the source doc.
+- A vague phase decomposed into invented tickets instead of flagged as a
+  source-doc gap.
+- Any item left net-open on the `tasks` board when a run stops without
+  Step 3 confirmation; `tasks` here is Step 4's confirmed-ticket store, and
+  anything opened before that as a scratchpad must be dropped again in the
+  same run.

package/skills/planning/backlog/evals.md

@@ -41,3 +41,145 @@ Expected:
 
 One representative scenario via `clio-coder skills eval` against Nemo-3.5-Lightning
 (30B local, llamacpp on mini), full-auto sandbox. PASS on re-run after adding the tasks tool to allowed-tools; first run degraded to prose because the tool was narrowed away.
+
+## Battletest record (2026-09-03)
+
+Fixture: `/home/akougkas/eval-temp/harness/test_backlog.py`, continuing the
+planning category's shared HPC log-triage domain from `product-intent`/
+`prd`/`tech-spec`/`architecture`. Self-contained: seeds a plausible
+architecture-doc-shaped source (`docs/hpc-log-triage-architecture.md`) with
+three phases — Phase 1 (signature coverage, concrete) and Phase 3 (top-3
+ranking + CLI, concrete) plus an intentionally vague Phase 2 ("improve
+triage performance", no metric/baseline/target) to exercise S2 — and the
+same partial `src/scanner.py` (`FailureEvent` + OOM-only `scan_oom`) used
+by the sibling fixtures. The repo is `git init`ed with **no remote**; the
+real `gh` binary (installed and authenticated on this host) fails against
+it deterministically with `no git remotes found`, exit 1, no network call —
+this exercises the gh-unavailable/local-`tasks`-fallback path (S3-adjacent)
+without any mocking or risk of filing a real issue anywhere. `ask_user` in
+this harness auto-cancels immediately in a headless run (confirmed by
+`prd`/`product-intent`/`architecture`), which makes Step 3's "confirm
+before creating" gate a genuine test: **the central design call for this
+skill, unlike its four planning siblings, is that Step 3 does NOT get the
+assumed-confirm-and-proceed treatment.** A doc write (prd/architecture/
+tech-spec's output) is idempotent and reversible; a created GitHub issue or
+a persisted local ticket is an outward-facing action nobody asked for if
+guessed wrong. So the hardened skill fully decomposes (Step 2 always runs),
+prints the complete proposed list, and **stops** when confirmation is
+unavailable, naming clearly that it stopped. Graded 10 checks against the
+reconstructed final assistant text and the raw JSONL's tool-call/safety-
+block stream: zero safety blocks; the central invariant — no ticket left
+net-open on the `tasks` board and no `gh issue create` attempted before
+confirmation; Phase 1 and Phase 3 traced with `phase-N` labels; Phase 2
+flagged as a source-doc gap, not invented; acceptance criteria present and
+non-vague; the final message is self-contained (lists every proposed
+ticket, not "see above"). Ran on `dynamo`/`ornith-1.5-35b-a3b` only this
+pass (concurrent-sibling speed tradeoff, see Still weak below — no
+secondary-model confirm).
+
+| run | wall | turns | in / out tokens | safety blocks | score | outcome |
+|---|---|---|---|---|---|---|
+| baseline (no skill) | 71s | 15 | 89.1k / 2.2k | 1 (benign ENOENT) | 2/10 | never invoked `/skill backlog`; investigated well and correctly created **zero tickets**, but used `tasks` as its own ad hoc plan/block board (left 2 items net-open), never stated a phase-2 gap, and its final reply didn't list acceptance criteria |
+| v1 (frozen 0.3.0) | 38s | 5 | 52.8k / 6.4k | 1 real (`$(...)` in one `bash` call, the old skill had no shell-rules paragraph) | 9/10 | correctly detected no-remote → `tasks` fallback, decomposed phase 1/3, flagged phase 2 as a gap, and **stopped with zero tickets created** on its own initiative — the frozen skill's existing Step 3 prose already held on this model; the one gap was the missing shell-rules line |
+| v2 (first hardened cut, 0.4.0) | 81s | 14 | 245.5k / 12.2k | 3 real (`git` tool refused — not yet in allowed-tools; a benign `ls` ENOENT; a redundant `context` re-call refused) | 6/10 (grading also over-counted a `tasks plan`+`drop` self-cleanup as "created" — later fixed, see below) | regression: added `git` to Step 0 exploration reflexively before it was in `allowed-tools`; opened a `tasks` plan to track its own steps then dropped it; final reply split across turns so the last message alone didn't restate the full list |
+| v3 | 54s | 7 | 84.6k / 8.1k | 2 real (`$(...)` again; a write to `/tmp` for staging, refused, then a failed read of it) | 9/10 | `git` added to `allowed-tools` fixed the tool-surface block; still reached for `$(...)` once and staged output via `> /tmp/...` once — both new Red-flag/shell-rules gaps closed after this run |
+| v4 (0.4.0, stable) | 55s | 7 | 93.4k / 8.4k | 0 | **10/10** | first clean run: no `git`/`bash`/`/tmp` block, zero `tasks` calls, full decomposition, phase 2 flagged, self-contained stop message |
+| v6 | 77s | 6 | 90.7k / 13.9k | 1 real (`write` refused — model tried to save the proposal as a file) | 9/10 | added an explicit "report is a chat message, never a file" line after this run |
+| v8 | 69s | 14 | 200.3k / 10.9k | 2 real (`$(...)` recurred; a hard-blocked `read .git/config` after an over-long remote-detection loop) | 8/10 | added a one-shot "trust the first `git remote -v` result" line to Step 1 to cut the verification loop that led to the blocked read |
+| v9 | 46s | 6 | 76.9k / 7.2k | 1 (benign `grep`-no-match) | 9/10 | |
+| v10 | 77s | 8 | 76.8k / 2.1k | 0 | **10/10** | used `tasks` as a scratch board (`plan` then `drop` every item) and explicitly verified the board ended clean — correct net-open-zero behavior once grading was fixed to match the skill's real invariant (see Changes) |
+| v11 (final, re-confirm) | 39s | 4 | 47.6k / 6.5k | 0 | **10/10** | |
+| vfinal (post-cleanup re-confirm) | 61s | 6 | 92.5k / 8.9k | 2 real (`tasks(action="plan")` with an empty list, then `tasks(action="ask_user", ...)` — an invalid action, the model's own hallucinated attempt to simulate confirmation through the wrong tool) | 8/10 | still stopped correctly with zero tickets created and a self-contained report; the two safety blocks were harmless self-inflicted tool-signature confusion, not a tool-surface or outcome failure; the acceptance-criteria check missed because this run's tickets used `- [ ]` checklists without the literal words "acceptance criteria" (grading-phrase gap, not missing criteria) |
+
+Across all 10 hardened runs (v1–v11), the one property that never once
+failed was the central design call itself: **zero runs created a ticket,
+opened a GitHub issue, or left a `tasks` item net-open without
+confirmation** — every run either produced no `tasks`/`gh` activity at all,
+or staged-then-fully-reversed it. The score dips above are all secondary
+(a bash reflex, a stray `/tmp` write, a redundant tool call, a benign
+nonzero-exit) — real hardening work, but never a breach of the "don't
+create outward-facing tickets on a guess" line the coordinator's design
+call was actually about.
+
+**Changes** (0.3.0 -> 0.4.0):
+
+1. **`## Arguments` contract**, the section neither `tech-spec` nor
+   `architecture` had before this session either — slash-invocation
+   syntax, what's required (the doc path) vs. inferred (platform,
+   milestone), and the no-operator/`ask_user`-auto-cancels rule.
+2. **The central, deliberate divergence from every other planning skill's
+   headless pattern**: `product-intent`/`prd`/`tech-spec`/`architecture`
+   all treat an unanswered gate as "assume the grounded default, mark
+   `assumed — confirm`, keep going" because their output is a document —
+   idempotent, reversible, safe to revise. `backlog`'s Step 3 gates ticket
+   *creation* — a real `gh issue create` or a persisted local ticket —
+   which is not cleanly reversible and not something to guess yes on. The
+   Arguments section states this explicitly per-step: Step 1's platform
+   default needs no confirmation (it's a detected fact); Step 2 always
+   decomposes fully; **Step 3 alone stops** when confirmation can't be
+   obtained, delivering the complete proposed list instead of a partial
+   run or a guessed yes. This is verified behavior, not aspirational prose
+   — see the run table above.
+3. **`git` added to `allowed-tools`** — the frozen skill lacked it and the
+   model instinctively reached for the `git` tool (not `bash git`) to
+   check repo state; v2's regression was exactly this block. `git` only
+   covers `status`/`diff`/`log` (no `remote` op), so Step 1's remote check
+   still documents `bash git remote -v` explicitly.
+4. **Shell rules paragraph** (one command per `bash` call, never `$(...)`
+   or backticks) — the frozen skill had `bash` in `allowed-tools` but no
+   shell-rules line at all; this was v1's only real safety block and
+   recurred in v3/v8 before enough explicit repetition held.
+5. **Explicit `/tmp` write refusal** — v3 staged a remote/gh check via
+   `> /tmp/platform.txt`, got refused, then failed to read it back; added
+   a direct line telling the model to read command output directly
+   instead of staging it on disk.
+6. **`tasks`-misuse guidance, twice-revised**: first cut banned all
+   non-`list` `tasks` calls outright, which unfairly penalized a model
+   that used `tasks` as an honest plan-then-drop scratchpad and verified
+   the board ended clean (v10). Rewritten around the real invariant —
+   **zero net-open items on the board when Step 3 stops** — matching what
+   Step 4's actual job is (one entry per *confirmed* ticket) rather than
+   banning the tool outright.
+7. **Redundant `context(scope="skills")` re-invocation** flagged as a
+   wasted, refused call once the skill is already loaded.
+8. **"Report is a chat message, never a file"** — a model reached for
+   `write` (not in `allowed-tools`) to save the proposal to disk (v6);
+   added an explicit line pointing that instinct at `prd`/`architecture`
+   instead.
+9. **Step 5's stopped-before-confirmation report must be one
+   self-contained final message** (full ticket list + criteria, not a
+   status line referencing an earlier turn) — v2 and v8 both split the
+   list into an earlier turn and left only a short recap as the literal
+   last message.
+10. Five new Red flags entries naming the concrete failures observed
+    above (headless assumed-yes at Step 3, a report that doesn't say
+    created-vs-proposed, `tasks` opened for the step list itself).
+
+**Still weak**: per this pass's coordinator note, only
+`ornith-1.5-35b-a3b`/`dynamo` was run — no `qwen3.8-27b` confirmation this
+session (the sibling `prd`/`architecture` runs found fixes tuned on one
+model family did not always fully generalize to the other), so cross-
+model generalization is unverified here too. The `$(...)` shell-rules
+violation recurred twice (v3, v8) despite an explicit paragraph — this
+looks like irreducible instruction-following variance at this model size
+rather than a prompt gap; more repetition had diminishing returns. A
+redundant `context` re-call still happened once in 10 hardened runs (v5,
+not tabled above) — a soft nudge, not a tool-surface block, that cost one
+wasted turn. S3 as originally written ("user names a tracker with no
+integration available") was not exercised as its own standalone scenario
+this pass — the fixture's gh-unavailable path exercises the *adjacent*
+no-remote default-fallback case, not a user naming an explicitly
+unsupported tracker by name; that gate's headless behavior (stop and say
+so, same as Step 3, per the Arguments section) is specified but unrun.
+`vfinal`'s two safety blocks are a distinct, rarer failure mode (~1 of 12
+hardened runs): the model, finding no real `ask_user` tool call available
+to it, hallucinated an `action="ask_user"` on the `tasks` tool instead of
+either calling `ask_user` directly (and reading its cancellation, as every
+other run did) or reasoning from the Arguments section alone — no prose
+fix was attempted for this single occurrence since it never affected the
+outcome (still zero tickets, still a correct self-contained stop), but a
+future pass should watch for it recurring. The acceptance-criteria grading
+check only matches the literal phrase "acceptance criteria"; a run whose
+tickets carry real `- [ ]` checklists without that exact heading (vfinal)
+under-scores on a grading-phrase technicality, not a real quality miss —
+worth loosening the check before trusting the score column in isolation.

package/skills/planning/prd/SKILL.md

@@ -1,13 +1,13 @@
 ---
 name: prd
-description: Use when the user wants to turn an idea into a product requirements document through a phase-gated interview — each phase locks before the next opens — ending in PRD.md plus per-milestone prompt files ready to drive a coding agent. Triggers on "write a PRD", "spec this out", "help me define this feature/product", or a brain dump that needs structure before planning.
+description: Turns an idea into a product requirements document through a phase-gated interview, ending in PRD.md plus per-milestone prompt files ready to drive a coding agent. Not for the problem-first product thesis; use product-intent.
 triggers:
   - write a PRD
   - spec this product out
   - define this feature
   - structure this product brain dump
   - create milestone prompts
-version: 0.2.2
+version: 0.4.0
 license: Apache-2.0
 allowed-tools:
   - read
@@ -37,13 +37,47 @@ milestone prompts. The discipline is the phase gate: each phase produces a
 small locked artifact that the next phase builds on. No phase reopens without
 the user saying so. Markdown only, no external templates, repo-aware.
 
+## Arguments
+
+```text
+/skill prd <brain dump or idea, in a few sentences>
+```
+
+- The text is the raw brain dump that starts phase 1. An existing intent
+  doc, PRD, or evidence file named or pathed in the request is material to
+  read first (see "Read the repo before asking" below), not more arguments.
+- Nothing is required beyond some text; a blank invocation gets phase 1's
+  own prompt — let the user describe the idea raw — rather than an invented
+  idea.
+
+There is no operator in a headless run: `ask_user` is either not registered
+or nothing answers it, and stalling a gate to wait for it never resolves.
+When a gate goes unanswered, do not skip the phase and do not go quiet: run
+it as a monologue instead — state the phase's question, your
+recommendation (grounded in the repo and any evidence read, or the most
+defensible product default when nothing grounds it), and the reasoning,
+adopt the recommendation, mark it `assumed — confirm`, and move to the next
+phase. All nine phases still run, end to end, in one turn — the phase list
+below is the plan to execute, not an outline to abbreviate because no one
+answered the first gate. Never invent evidence or a fact to back an
+assumption; anything genuinely unknown stays an open item, marked as such,
+not a plausible guess.
+
+The nine phases below are the plan; do not open a task list for them.
+`tasks` sits outside this skill's tool surface and any call to it is
+refused. `bash` is also outside this skill's tool surface — verify what you
+wrote with `grep`, `read`, and `find`, never `bash`.
+
 ## Interview mechanics
 
 - Use the `ask_user` tool for every confirmation and choice, with your
-  recommendation as the first option. Without `ask_user`, ask in plain text,
-  one focused exchange per phase.
+  recommendation as the first option: post the question, stop, wait for the
+  answer. See Arguments above for what a gate that goes unanswered means and
+  how to carry every phase through anyway.
 - **Read the repo before asking.** Stack, conventions, existing entities, and
-  integrations are facts; discover them and *confirm*, never ask cold.
+  integrations are facts; discover them and *confirm*, never ask cold. An
+  entity or module that already exists gets reused and marked as such, never
+  re-specced as new work.
 - Keep each phase to one or two exchanges. Synthesize, propose, lock, move on.
 
 ## The phases (in order, each locks before the next)
@@ -71,7 +105,8 @@ the user saying so. Markdown only, no external templates, repo-aware.
 
 - **`PRD.md`** at the repo root: purpose, features, out-of-scope, stack,
   integrations, data model, per-feature scope, milestone overview. Markdown
-  only.
+  only. Never write it anywhere else or under another name — a nested
+  `docs/PRD.md`, a slugged filename, or a report-style name are all wrong.
 - **`milestones/N-<slug>/prompt.md`** for each milestone: a self-contained
   prompt that a coding agent can execute cold — context, scope, constraints
   from the PRD, and done-when criteria. A reader must not need the PRD open
@@ -83,6 +118,11 @@ into a sprint.
 ## Red flags (you are doing it wrong)
 
 - Asking about the stack when package.json answers it.
-- A phase "locked" without the user confirming it.
+- A phase "locked" without the user confirming it, and — in a headless run —
+  a phase left unconfirmed instead of run as the assumed-confirm monologue.
 - Out-of-scope list that is empty or generic ("no mobile app").
 - Milestone prompts that say "see PRD for details".
+- An existing entity or module re-specced as new work instead of reused.
+- Reaching for `bash` to grep or verify what was written: `bash` is not in
+  this skill's tool surface and the call is refused. Use `grep`/`read`/`find`.
+- Opening a task list for the nine phases; `tasks` is refused.