@iowarp/clio-coder 0.4.2 → 0.4.3

package/skills/planning/tech-spec/SKILL.md

@@ -7,7 +7,7 @@ triggers:
   - code-shaped contracts
   - implementation-ready technical specification
   - specify execution flows
-version: 0.2.0
+version: 0.3.0
 license: Apache-2.0
 disable-model-invocation: true
 allowed-tools:
@@ -42,6 +42,46 @@ TypeScript pseudocode plus end-to-end execution flows. Prose explains why;
 types and call stacks define what changes. Design only — never implement,
 and save a file only when the user asks; otherwise return the spec inline.
 
+## Arguments
+
+```text
+/skill tech-spec <the change, in a few sentences, or a path to read first>
+```
+
+- The text is the design problem: what's changing and why. A doc or file
+  path named in the request (a PRD, an architecture decision, a module) is
+  context to read, not more arguments — see "Load local context" below.
+- Nothing is required beyond some text; a blank invocation falls straight
+  to Path B's first question rather than inventing a change to spec.
+- **Output defaults to inline.** Write a file only when the request says
+  so explicitly — "save it", "write it to `<path>`", "put it in `docs/`".
+  Absent that, the finished spec is the reply itself: no file, in this run
+  or a prior one in the same session, gets created for it. This holds
+  regardless of which path below runs or how long the spec is — length is
+  never itself a reason to write a file.
+- Disabled for model self-invocation and requires the `tdd` skill be
+  installed to reference in the TDD Test Plan section; both are frontmatter
+  facts, not something to explain to the user unless asked.
+
+There is no operator in a headless run: `ask_user` either isn't registered
+or nothing answers it, and a call that goes unanswered will not resolve
+differently on a second try. In Path B (below), that means: state the
+question, your recommendation grounded in the codebase and any docs read
+(or the most defensible engineering default when nothing grounds it), and
+the reasoning; adopt the recommendation; mark it `assumed — confirm`; move
+to the next question. Run every question this way, end to end, not just
+the first — the interview is the plan to execute, not an outline to
+abbreviate because no one answered the opening question. Never invent a
+fact or a codebase detail to back an assumption; anything genuinely
+unknown becomes an Open Question in the spec, not a plausible guess. This
+degrades the interview only — it never licenses writing a file that
+wasn't asked for.
+
+The steps below are the plan; do not open a task list for them. `tasks`
+sits outside this skill's tool surface and any call to it is refused.
+`bash` is also outside this skill's tool surface — verify what you wrote
+with `grep`, `read`, and `find`, never `bash`.
+
 ## Choose the path
 
 - **Path A — convert context to spec**: the conversation, docs, or codebase
@@ -51,6 +91,8 @@ and save a file only when the user asks; otherwise return the spec inline.
   with a recommended answer per question (the grill-me posture); anything
   answerable by exploring the codebase is explored, not asked. When context
   suffices, run Path A. Never invent requirements to skip the interview.
+  See Arguments above for how a headless run carries every question
+  through instead of stalling on the first one.
 
 ## Path A
 
@@ -110,7 +152,8 @@ contracts, seams, call stacks, or the test plan for being hard):
 
 The spec follows the outline, every boundary has a typed contract or a
 stated reason it needs none, every behavior has a call stack, unknowns are
-open questions rather than invented design, and nothing was implemented.
+open questions rather than invented design, nothing was implemented, and
+no file was written unless the request asked for one.
 
 ## Red flags
 
@@ -119,3 +162,11 @@ open questions rather than invented design, and nothing was implemented.
 - Speculative seams no invariant, boundary, or test earns.
 - The same rule restated in three sections.
 - "While I'm here" implementation.
+- Writing the spec to a file when nothing in the request asked for one —
+  the default output is always the inline reply.
+- A Path B question left unanswered instead of run as the assumed-confirm
+  monologue, or an interview skipped straight into Path A without ever
+  asking the first question.
+- Opening a task list for the steps above; `tasks` is refused. Reaching for
+  `bash` to grep or verify the spec; `bash` is not in this skill's tool
+  surface and the call is refused — use `grep`/`read`/`find`.

package/skills/planning/tech-spec/evals.md

@@ -45,3 +45,76 @@ Expected:
 
 One representative scenario via `clio-coder skills eval` against Nemo-3.5-Lightning
 (30B local, llamacpp on mini), full-auto sandbox. PASS. Spec written and its claims exercised with node -e; judge 4/4.
+
+## Battletest record (2026-09-03)
+
+Fixture: `/home/akougkas/eval-temp/harness/test_techspec.py`, continuing the
+planning category's shared HPC log-triage domain (`product-intent` -> `prd`
+-> `tech-spec`). Seeds a plausible root `PRD.md` (purpose, features,
+out-of-scope, stack, integrations, data model, milestones, and the
+always-on-vs-on-demand ingestion tension explicitly marked as *not this
+document's decision*) plus the existing partial codebase (`src/scanner.py`:
+a working `FailureEvent` + `scan_oom`, OOM only) and two sample dmesg logs
+carrying real OOM/ECC/Xid line formats, inside a git repo. Three task
+variants, one fixture:
+
+- **base** (S1, Path A): "spec ECC + Xid detection and cross-signature
+  top-3 ranking" — sufficient context, no save request. Graded on 9 checks
+  against the *reconstructed final assistant text* (this skill's default
+  output is inline, not a file): all 11 outline-derived sections present,
+  >=5 domain grounding terms, >=2 materially different alternatives,
+  `FailureEvent` reused not respecced, nothing implemented (`scanner.py`
+  byte-identical to seed), the ingestion trade-off left unresolved, zero
+  safety blocks, no `tasks` call, and — the check this run exists to catch —
+  **no file written when nothing asked for one**.
+- **save** (S1 variant, confirmation only, run once on the final version):
+  same task plus an explicit "save it to docs/tech-spec-log-triage.md" —
+  10 checks, same 8 plus the file existing at exactly that path and no
+  other new file appearing.
+- **thin** (S2, Path B, confirmation only, run once on the final version):
+  a genuinely vague "improve our failure detection, you'll need to ask me
+  stuff" request with 5 lighter checks — zero safety blocks, no silent
+  stall, no `tasks` call, and either a real `ask_user` exchange or the
+  assumed-confirm monologue (`assumed` + `confirm` both present), with a
+  real spec still produced.
+
+| run | model | wall | turns | in / out tokens | safety blocks | score | outcome |
+|---|---|---|---|---|---|---|---|
+| baseline (no skill) | ornith-1.5-35b-a3b | 80s | 5 | 54.9k / 12.1k | 0 | 4/9 | never invoked `/skill tech-spec`; discovered the installed skill itself via `context(scope="skills")`, read its SKILL.md directly, then called `artifact` and terminated early with a `.clio-coder/artifacts/PLAN.md` instead of a spec — no alternatives, no sections, wrong output shape |
+| v1 (frozen 0.2.0) | ornith-1.5-35b-a3b | 95s | 8 | 115.0k / 15.1k | 1 | 6/9 | ran Path A correctly and produced a genuinely strong spec (11/11 sections, 3 material alternatives, `FailureEvent` reused, nothing implemented) but opened a `tasks` plan (refused, safety block) and **wrote the spec to `docs/tech-spec-scanner-ecc-xid.md` without being asked to** — the exact Path-A/B default-output risk flagged going in |
+| v2 (live 0.3.0) | ornith-1.5-35b-a3b | 68s | 5 | 57.2k / 10.4k | 0 | 9/9 | same spec quality, zero safety blocks, no `tasks` call, correctly returned inline with no file written; final text states explicitly "I did **not** write a file, since nothing in the request asked to save it" |
+| v2 confirm — save | ornith-1.5-35b-a3b | 110s | 11 | 184.8k / 16.5k | 0 | 10/10 | explicit "save it to docs/tech-spec-log-triage.md" correctly produces exactly that file at that path, nothing else |
+| v2 confirm — thin (Path B) | ornith-1.5-35b-a3b | 69s | 9 | 111.8k / 11.7k | 0 | 5/5 | correctly identified insufficient context, ran Path B, and carried all five scope decisions (S1-S5) through as an explicit assumed-confirm monologue headlessly instead of stalling or silently skipping to Path A |
+
+**Changes**: (1) `## Arguments` contract with the slash-invocation syntax,
+what's required vs. inferred, and — the section that mattered most here —
+an explicit "output defaults to inline" rule stated as its own bullet
+before the headless-monologue prose, so the fix for Path B's ask_user gap
+can't be misread as license to always write a file; (2) the headless
+no-operator paragraph, ported from `product-intent`/`prd`, applied to Path
+B's grill-me interview: every question runs as state-question /
+grounded-recommendation / reasoning / adopt / mark `assumed — confirm`,
+end to end, not just the first one; (3) explicit `tasks` and `bash`
+refusal lines — `tasks` was v1's only safety block; (4) `Done when` and
+`Red flags` both gained a line naming the unrequested-file failure and the
+unanswered-Path-B-question failure by name, plus the existing `tasks`/`bash`
+refusal repeated as a red flag (matching `prd`'s and `product-intent`'s
+pattern of naming the exact observed failure, not a generic reminder).
+Version 0.2.0 -> 0.3.0.
+
+**Still weak**: per this pass's coordinator note, no secondary-model
+confirmation was run (qwen3.8-27b was skipped in favor of running one full
+cycle on ornith-1.5-35b-a3b at speed, concurrently with a sibling agent
+hardening `architecture` on `mini`); the fix is validated on one model
+class only. The baseline's failure mode (discovering and improvising from
+the installed skill file directly, without ever invoking it, then calling
+`artifact` for an unrelated early exit) is a skill-selection/tool-scoping
+gap this SKILL.md cannot fix from inside its own body. `code_nav` (in
+`allowed-tools`) was never exercised — the fixture's one-file codebase
+never needed it. `requires: [skill:tdd]` is a diagnostic-only reference in
+this harness (unmet requires warn, never block `--skill`-path invocation);
+the TDD Test Plan section reads fine without the `tdd` skill installed, but
+that was not tested with `tdd` actually present to see if the reference
+changes. Genuine unknowns (S3 from the original evals) were exercised only
+incidentally via the Xid-severity and ECC-correctable open questions, not
+as an isolated scenario.

package/skills/registry.yaml

@@ -6,54 +6,58 @@ skills:
   # ── coding ──
   - name: ast-grep
     path: coding/ast-grep
-    version: 0.2.0
-    sha256: d21a4b330d1488c43348eaee7fbeec24bb8a1d7c4e536059db02fd9377c62dfa
+    version: 0.3.0
+    sha256: 0a72f4c906303f550be78f80a927da7e24e5c2076f058ccafdec80dfbed275af
   - name: coding-standards
     path: coding/coding-standards
-    version: 0.2.0
-    sha256: da21ad373252575934ad484a2926e8c7827880c9d91b4e0c656fe92d71335063
+    version: 0.3.0
+    sha256: 3ee3481430591a8f8d041bd1c5fa078becd301d88eb49610e41d056494d3413d
   - name: prototype
     path: coding/prototype
-    version: 0.3.0
-    sha256: 151bd153752874d5970dd06bc15df14c0037631283d148195a8fa75c5a59a45b
+    version: 0.4.0
+    sha256: 0f82df386c2ee565b0e968216ece746b1a11b4acd79676812074c1a406b698e1
   - name: tdd
     path: coding/tdd
-    version: 0.3.0
-    sha256: f3a03655864d4981791218b085d09f8fe836d0795012cf4c8c07b430730abb2a
+    version: 0.4.0
+    sha256: 63b88f29424091a92a8d8c0cc94474ae7e78af76491aea17e6f54af66f32db2d
   # ── context ──
   - name: context-handoff
     path: context/context-handoff
-    version: 0.4.0
-    sha256: ba0568719d0b58bb3e1631eee72094ec0165bf2fb7f69e1e67d223ff50b364d8
+    version: 0.5.0
+    sha256: e69adb1533a6a850cb83babe7e580dd260e034781d7c35e65921453e3fa191a5
   - name: context-prime
     path: context/context-prime
-    version: 0.3.0
-    sha256: d8413255688f1697d40231df830ce3c8947ac2a551f3e6fbb0d10158c4fa8d0e
+    version: 0.4.0
+    sha256: 21587263297a7ee9e81a2d66f6fc800a1ee6db15f76de1e650563b71fdf19d8e
   # ── git ──
+  - name: branch-closeout
+    path: git/branch-closeout
+    version: 0.1.0
+    sha256: 3227f31b0693bb6428abb20a2d4f449aeba1d8e45a79cf0c2c7c333c30c10664
   - name: file-ticket
     path: git/file-ticket
-    version: 0.2.0
-    sha256: 07b5d996d459c6c8c99ea44f44421d418f4738075cccd0e4b759a6f8a7daf720
+    version: 0.3.0
+    sha256: 65029c2d9d545728750c8a713f92035bd6bdbfa9874de442df72bd059846cb7a
   - name: fix-issue
     path: git/fix-issue
-    version: 0.2.0
-    sha256: ec5e6a00056041494f9b100da8f0775455124aebc7000ed0edb7c9b8493f74c8
+    version: 0.3.0
+    sha256: 626e3aa8ca95bc603f2e0cdd2e7502aa96be85af6bfa92b635c39ef99d3845a5
   - name: resolve-merge-conflicts
     path: git/resolve-merge-conflicts
-    version: 0.3.0
-    sha256: a1df7dee85ef788f257f8b8fbf4c2564cac245c3010d3f60d3b62d721d185f5e
+    version: 0.4.0
+    sha256: 901f482aa2231552d64a721e0d1c58dc8f211347543b5a91fd2ab3c7e413c11c
   - name: ship
     path: git/ship
-    version: 0.2.0
-    sha256: 8a22f74e56a6e98c85a38051550d1e1a70c9ca5bb5cf271f4e6448f0f791e0ed
+    version: 0.5.0
+    sha256: 4d626718fdb8b67cf3f22d09ac5a228e5c8103befa1295a672587a6605f038b1
   - name: worktree-create
     path: git/worktree-create
-    version: 0.3.0
-    sha256: d40bb0bb9152c4f2e391de1285b6cbd2e67a45492e2cf865b1f82910e30a6bcc
+    version: 0.6.0
+    sha256: a4ae790d916a170977814b4334e14aef4f96fb304e3289f7beb5e31e8fb83881
   - name: worktree-merge
     path: git/worktree-merge
-    version: 0.3.0
-    sha256: bca803a2484b0eac90939377c81f9e66928242f36ac2d86cef9381b1ff997f3e
+    version: 0.6.0
+    sha256: 0fbe2297622ed955b2e4fb14306d75a51bd36db7eebf59e8e92fac1da5b6e64e
   # ── meta ──
   - name: clio-coder-dev
     path: meta/clio-coder-dev
@@ -80,57 +84,61 @@ skills:
     version: 0.3.0
     sha256: ba81e09412f26647a9384b07359201efb08949865c50934778a4ab3173d43096
   # ── planning ──
+  - name: archify
+    path: planning/archify
+    version: 0.1.0
+    sha256: d9e891eb5f3c27678de14165a0eb35d1eb2289be480e55009f9a57bcfe55be2d
   - name: architecture
     path: planning/architecture
-    version: 0.3.0
-    sha256: 0da15631b3a60946e90b63047047e0ee1d84b6705e4f9f5405036b87fc849454
+    version: 0.4.0
+    sha256: e9017412b6261492b987fe27f7573b72013a340c0d3eb0e7e568fd588b22a214
   - name: backlog
     path: planning/backlog
-    version: 0.3.0
-    sha256: af92244d2ba8723b656ce00d531df88850c7886915653751e7ba59957b5454be
+    version: 0.4.0
+    sha256: e634e2fb5035731e7daa2ef246ae4ab3ef5d792b0b604f4216dfe10335ee10c0
   - name: prd
     path: planning/prd
-    version: 0.3.0
-    sha256: 505460dcac23b762a87fd0f7f3aaa99806e3b0fbedeaa54f9c48608c49c16356
+    version: 0.4.0
+    sha256: 39fe417f495be4153541dd70c886cf773265636dc9fa1844cfc7012b71cbbec1
   - name: product-intent
     path: planning/product-intent
-    version: 0.3.0
-    sha256: d34300e6e2422fc1071e352f3e0f1b13a4e65dbcfd4eaf6dd9ef2b61f18d72ad
+    version: 0.4.0
+    sha256: d41689db9a8c9b2dad0cd630412bebaed5792f9d22d4edb18770e4b1b3308699
   - name: tech-spec
     path: planning/tech-spec
-    version: 0.2.0
-    sha256: 28b1b0a0a3e9b48b29f679ab2377ee66d719c9a670eed622a38ac69be8b72b2e
+    version: 0.3.0
+    sha256: 5f9add0d01e43feb6808b2eaeae9ac68075cf369cbfc793e020181a642d4d2c4
   # ── research ──
   - name: arxiv-literature
     path: research/arxiv-literature
-    version: 0.4.0
-    sha256: b0b1bb80dc8d15e52ab180ab4f5645859310b45d7ebdf87649e0e5a7f7bca987
+    version: 0.5.0
+    sha256: 0bc6b39d12934568804fc11b23219de4c6c2172e28c4bfedcee3104e2217aae6
   - name: experiment-protocol
     path: research/experiment-protocol
-    version: 0.2.0
-    sha256: 126e0f6dc3534a141978f2ccd1c6ea659df447beb1015bc92e41856d318584a2
+    version: 0.3.0
+    sha256: 235256cbdf44ebdd25d87cce60940e00ad1fca42a1b2b4e534bfc245f0d1135c
   - name: scientific-debugging
     path: research/scientific-debugging
-    version: 0.2.0
-    sha256: f2b2b6442740ef23270fad94f75ba95ce522da1877a468ba112cbaff68e5fa2a
+    version: 0.3.0
+    sha256: 9c29d70b4b82a440640b41665c23731b4112125915bde0e91016fc60aec67207
   - name: scientific-modernization
     path: research/scientific-modernization
-    version: 0.3.0
-    sha256: 66a99f38b38232735bb70d7924a85d0ab5addabfe3935cc82c0aaa3303d68773
+    version: 0.4.0
+    sha256: 491522d547fa0bddef19d2ec92bc35236b169787bded6ecc3a29dfec19892886
   # ── workflow ──
   - name: cut-it
     path: workflow/cut-it
-    version: 0.3.0
-    sha256: 6429a4725acf874379c1df7736dd95db07acd97ad5bd6f6226649370cd03d177
+    version: 0.4.0
+    sha256: d3742221f0ace1f1b62c07f5902030e478496059a58ef194e7cb3303ccdff05e
   - name: design-council
     path: workflow/design-council
-    version: 0.4.0
-    sha256: 1ecd53397ce97ec78294cc9e8c7291b20e2e7d740bcde8ea999739d359c91055
+    version: 0.5.0
+    sha256: c279a94a4cd66d46980f9a5024aedac3e673144fadc9d36b53ad87fc2ee63501
   - name: grill-me
     path: workflow/grill-me
-    version: 0.4.0
-    sha256: 9cc07b89a4a8a3ecd4fb73d2cd9f7893d272d26f592ea3eb9883ff44069a2cea
+    version: 0.5.0
+    sha256: f48687566a8419aa52f30f8234f8dcfd050eaefdfdcfa281723d3603f3e08fcc
   - name: workflow-distiller
     path: workflow/workflow-distiller
-    version: 0.3.0
-    sha256: 6aa6bb3088f0e4925946aaba3f217dfc5a6e4b8e31f2f051abeda599da91cd48
+    version: 0.4.0
+    sha256: dd7c312d6904f861fa105334c7c2f18c9d6e26a70a76874c85858cee90bede58

package/skills/remote.yaml

@@ -0,0 +1,13 @@
+# Skills whose content lives in another repository at a pinned ref.
+# Clio never vendors these. `npm run skills:pin` publishes each entry into
+# skill-marketplace.json with the upstream tree as its sourceUrl, the catalog
+# overlay whose files land on top of that tree at install, and the upstream
+# top-level members the install drops. The overlay SKILL.md is pinned in
+# registry.yaml like every other catalog skill.
+version: 1
+skills:
+  - name: archify
+    category: planning
+    sourceUrl: https://github.com/tt-a1i/archify/tree/v2.16.0/archify
+    overlay: skills/planning/archify
+    exclude: [test, package-lock.json]

package/skills/research/arxiv-literature/SKILL.md

@@ -7,7 +7,7 @@ triggers:
   - compare these papers
   - find recent research papers
   - build a literature survey
-version: 0.4.0
+version: 0.5.0
 license: Apache-2.0
 allowed-tools:
   - web_fetch
@@ -16,7 +16,6 @@ allowed-tools:
   - grep
   - find
   - ls
-  - artifact
 clio-coder:
   registry-id: iowarp/clio-coder
   source-url: https://github.com/iowarp/clio-coder/tree/main/skills/research/arxiv-literature
@@ -34,6 +33,20 @@ Find, summarize, or compare academic papers without flooding the main context
 window. Raw search results and paper text stay in a worker or get compressed
 immediately; only paper cards reach the user.
 
+## Arguments
+
+```text
+/skill arxiv-literature <request>
+```
+
+The request is free text: a paper URL/ID, a topic, or two or more IDs to
+compare. There is no operator in a headless run — `ask_user` is not in this
+skill's tool surface and nothing answers it. If the request is ambiguous (no
+clear topic, an ID that doesn't resolve), state your best reading and
+proceed; never stall waiting for clarification. `tasks` sits outside this
+skill's tool surface and is refused; the steps below are the whole plan, do
+not open a task list for them.
+
 ## Step 1 — Classify the request
 
 Pick exactly one:
@@ -45,29 +58,49 @@ Pick exactly one:
 
 ## Step 2 — Pick the vehicle
 
-- **Search, compare, survey**: dispatch the `researcher` shadow agent. Task
-  prompt:
+Default every request — single paper, search, compare, and survey alike — to
+`web_fetch` directly against arXiv:
 
-  ```text
-  Research arXiv literature for: <user goal>.
-  Return only compact source-linked paper cards, comparison/synthesis,
-  caveats, and read/skim/skip recommendations.
-  ```
-
-- **Single paper, or dispatch unavailable**: use `web_fetch` directly.
-  - Paper URL/ID: fetch the arXiv page. Clio normalizes it into structured
-    metadata plus AlphaXiv enrichment when available.
-  - Search query: fetch the arXiv Atom API; Clio compacts the XML into paper
-    cards:
+- Paper URL/ID: fetch the arXiv page. Clio normalizes it into structured
+  metadata plus AlphaXiv enrichment when available.
+- Topic, compare, or survey: fetch the arXiv Atom API; Clio compacts the XML
+  into paper cards:
 
   ```text
   https://export.arxiv.org/api/query?search_query=all:QUERY&sortBy=submittedDate&sortOrder=descending&start=0&max_results=10
   ```
 
+  For a compare request, run one query per paper ID (`id_list=ID` instead of
+  `search_query`) or one broader query covering all of them — whichever stays
+  inside the fetch cap in Step 3.
+
 Useful categories for `search_query`: `cs.AI` (AI), `cs.LG` (ML), `cs.CL`
 (NLP/LLMs), `cs.CR` (security), `cs.SE` (software engineering), `cs.MA`
 (multi-agent), `cs.IR` (retrieval/RAG), `cs.CV` (vision), `cs.RO` (robotics).
 
+**Only dispatch the `researcher` shadow agent when the user explicitly asks
+for a deep or broad survey** ("survey the field", "don't just skim arXiv, go
+wide") — never as the default for an ordinary search or compare. Left
+unbounded, a dispatched worker has no arXiv-only restriction and no fetch
+budget of its own, and will wander into Semantic Scholar, DBLP, OpenAlex, and
+general web search, taking several minutes to return nothing useful. When you
+do dispatch, state the same bound this skill uses directly, in the task
+prompt itself:
+
+```text
+Research arXiv literature for: <user goal>.
+Search arXiv only (export.arxiv.org Atom API or arxiv.org paper pages) — do
+not query Semantic Scholar, DBLP, OpenAlex, or general web search. One
+round, at most two fetch attempts total (successes and failures both count).
+On a timeout or HTTP error, do not retry with a different host, scheme, or
+protocol — retry the identical URL at most once, then stop and report the
+failure. Build cards from whatever you have; do not keep escalating.
+Return only compact source-linked paper cards, comparison/synthesis,
+caveats, and read/skim/skip recommendations.
+```
+
+If dispatch is unavailable, fall back to the direct `web_fetch` path above.
+
 ## Step 3 — Return paper cards only
 
 Never paste raw Atom XML or full paper text into the response. Output format:
@@ -96,9 +129,14 @@ Never paste raw Atom XML or full paper text into the response. Output format:
 Done when every returned paper has a card with a working link and the
 recommendation section is filled in. Stop after one search round unless the
 user asks to go deeper; do not keep fetching to "be thorough". One round
-means at most two Atom API fetches: the initial query plus one refinement.
-Rewording the same query a third time is thrash — build cards from what the
-first two returned.
+means at most two Atom API fetch attempts total: the initial query plus one
+refinement, or one retry of a failed fetch. Failed attempts (timeout, 429,
+connection error) count against this cap the same as successful ones — a
+timeout is not a free retry. Rewording the same query a third time, or
+retrying through a different host/scheme (`http` vs `https`,
+`export.arxiv.org` vs `arxiv.org/search`, an unofficial JSON mirror) after a
+failure, is thrash: build cards from whatever the attempts returned, or
+report the network failure plainly and stop.
 
 ## Gotchas
 
@@ -108,3 +146,23 @@ first two returned.
 - AlphaXiv is AI-generated enrichment: useful for scanning, never citable as
   authoritative.
 - Fetch or enrich only the top candidates, not every result.
+- `export.arxiv.org` rate-limits (HTTP 429) under repeated hits; that is a
+  reason to stop at the fetch cap, not a reason to retry against a mirror.
+- This skill's tool surface has no file-writing tool and no `artifact`. The
+  paper cards are the chat reply, never a document; do not go looking for a
+  way to save one.
+
+## Red flags
+
+- Dispatching `researcher` for an ordinary search or compare "just in case"
+  it does a better job than a direct fetch — it is slower and unbounded by
+  default; reserve it for an explicit deep-survey ask.
+- A dispatch task prompt without the arXiv-only, fetch-capped instruction —
+  that omission is what let a prior run wander into Semantic Scholar, DBLP,
+  and OpenAlex for minutes with nothing to show for it.
+- Treating a timeout or 429 as free of the fetch cap and retrying against a
+  different host, scheme, or unofficial mirror instead of stopping.
+- Raw Atom XML, a full abstract dump, or more than the top few candidates
+  reaching the final reply.
+- Reaching for `artifact` or any write tool to "save" the result — it is not
+  in this skill's tool surface; the reply is the deliverable.

package/skills/research/arxiv-literature/evals.md

@@ -56,3 +56,53 @@ Expected:
 
 One representative scenario via `clio-coder skills eval` against Nemo-3.5-Lightning
 (30B local, llamacpp on mini), full-auto sandbox. PASS (driver FAIL overturned on transcript review): skill loaded, skill_surface blocked two curl attempts, retrieval ran through web_fetch on the Atom API per the designed dispatch-unavailable fallback. Judge misread the OR in bullet 1. Query thrash (8 fetches) led to the two-fetch cap now in the body.
+
+## Battletest record (2026-09-03)
+
+S1 (topic search, speculative decoding) and S2 (single known paper,
+`arxiv.org/abs/1706.03762`), `ornith1.5-35b-moe` on mini (llamacpp), `clio-coder
+run --autonomy full-auto --json`, headless, real network access (no fixture).
+
+| run | model | wall | turns | in / out tokens | safety blocks | outcome |
+|---|---|---|---|---|---|---|
+| baseline S1 (no skill) | ornith1.5-35b-moe | 286s (killed at timeout) | 9 | n/a (killed before `agent_end`) | 5 | never finished: fired raw `curl` via `bash` (permission-gated, refused) before falling back to `web_fetch`; hit real timeouts against `export.arxiv.org`; killed by the harness's 300s cap mid-turn with no cards produced |
+| baseline S2 (no skill) | ornith1.5-35b-moe | 24s | 2 | 3.6k / 1.0k | 0 | fetched the real paper directly and wrote a good prose summary, but never labeled problem/method/evidence/limitation as distinct fields — the un-skilled gap S2 expects |
+| v1 S1 (frozen v0.4.0, dispatch runaway) | ornith1.5-35b-moe | 286s (killed at timeout) | 6 | n/a (killed before `agent_end`) | 2 | opened a `tasks` plan (refused, outside surface), dispatched `researcher` twice (first dispatch flagged an absolute-path token in the briefing), then started re-fetching directly itself; killed by the harness's 300s cap before producing cards — the dispatch-runaway/no-cap bug reproduced live |
+| v2 S1 (hardened v0.5.0) | ornith1.5-35b-moe | 109s | 4 | 2.9k / 3.9k | 2 (real `export.arxiv.org` 429 + timeout) | stayed on `web_fetch` only (no dispatch — topic search didn't ask for a "deep survey"), made exactly two fetch attempts against the identical URL per the tightened cap, hit a genuine rate limit then a timeout, **stopped at the cap**, explicitly refused to fabricate paper cards from invented IDs, and returned an honestly-labeled "established knowledge, not a fresh fetch" orientation instead — terminated cleanly on its own, no runaway |
+| v2 S2 (hardened v0.5.0) | ornith1.5-35b-moe | 28s | 3 | 6.2k / 1.5k | 0 | direct `web_fetch` on the real paper page, full problem/method/evidence/limitation/relevance card, explicit read/skim/skip recommendation, AlphaXiv linked and labeled as enrichment, no `artifact` call — 5/5 on the S2 rubric |
+
+Changes: removed `artifact` from `allowed-tools` — nothing in the procedure
+ever called it, and it is a terminal tool that would end the run the moment
+the model reached for it to "save" a result. Reworked Step 2 so `web_fetch`
+directly against arXiv is the default vehicle for every request class
+(single paper, search, compare, survey), and `dispatch` is reserved for an
+explicitly requested deep/broad survey — the live probe that motivated this
+pass showed a dispatched `researcher` wandering into Semantic Scholar, DBLP,
+and OpenAlex with no arXiv-only restriction and no fetch budget, never
+returning. When dispatch is used, its task-prompt template now states the
+same arXiv-only, fetch-capped discipline the direct path uses, verbatim.
+Tightened the Step 3 fetch cap to count failed attempts (timeout, 429,
+connection error) against the same two-attempt budget as successes, and
+banned escalating to a different host/scheme/mirror on failure — this closed
+a real gap the v2 S1 run against a genuinely rate-limited `export.arxiv.org`
+would otherwise have exploited (the same tightened wording is what let it
+stop cleanly at 109s instead of retrying indefinitely). Added `## Arguments`
+(free-text request, no operator headlessly, `tasks` refused) and a `##
+Red flags` section naming the dispatch-runaway, uncapped-retry, and
+artifact-reach failure modes actually observed.
+
+Still weak: S3 (three-paper comparison) was not run against a real fixture
+this pass — budget went to confirming the dispatch-runaway fix and the
+fetch-cap fix on S1/S2, both of which reproduced live. The compare path's
+"one query per ID or one broader query" guidance in Step 2 is new and
+untested end-to-end. Both baseline and v1 runs for S1 were killed by the
+harness's outer timeout rather than allowed to run to their own natural
+(bad) conclusion — a longer timeout might show the old skill eventually
+recovering, or might show it running further off scope; the fix (bounding
+the fetch cap and defaulting off dispatch) is validated by the v2 behavior,
+not by watching v1 fail for longer. `export.arxiv.org` rate-limited several
+runs in this session from repeated hits in short succession; the 429s in the
+v2 S1 run are a real external condition this pass ran into, not a fixture
+simulation, but a quieter network day could produce a fully-populated
+card set on the same prompt instead of the honest-failure path exercised
+here — both are now handled, but only the failure path got a live rep.

package/skills/research/experiment-protocol/SKILL.md

@@ -8,7 +8,7 @@ triggers:
   - define numerical tolerances
   - reproduce these results
   - compare solver accuracy
-version: 0.2.0
+version: 0.3.0
 license: Apache-2.0
 allowed-tools:
   - read
@@ -39,6 +39,25 @@ mode this protocol exists to prevent.
 Anti-trigger: if the question is "why is this output wrong", that is a
 diagnosis, not an experiment; use scientific-debugging.
 
+## Arguments
+
+```text
+/skill experiment-protocol <what to benchmark, compare, or sweep>
+```
+
+There is no operator in a headless run — `ask_user` is not in this skill's
+tool surface. If a threshold, tolerance, or environment detail is unstated,
+pick the most defensible default, record it as an explicit assumption in the
+pre-registration, and proceed; never stall Phase 0 waiting for confirmation.
+
+The phases below are the plan; do not open a task list for them — `tasks`
+sits outside this skill's tool surface and any call to it is refused.
+
+Shell rules for every `bash` call: one command per call, plain and direct.
+Never use `$(...)` or backticks; they trigger an approval gate that ends a
+headless run. Capture checksums and environment facts with direct calls
+(`sha256sum data/mesh.h5`), never command substitution.
+
 ## Phase 0 - Pre-register
 
 Before any measurement, write the protocol into the repository validation

package/skills/research/experiment-protocol/evals.md

@@ -89,3 +89,26 @@ Prompt: "Make this kernel faster."
 
 One representative scenario via `clio-coder skills eval` against Nemo-3.5-Lightning
 (30B local, llamacpp on mini), full-auto sandbox. PASS. Pre-registration written before touching the seeded kernel; judge 5/5.
+
+## Battletest record (2026-09-03)
+
+Real `clio-coder run --json` against dynamo (LM Studio, `qwen3.8-27b`), the S1
+kernel.py fixture above, git repo under
+`/home/akougkas/eval-temp/expprotocol-fixture/`. Same gap as every other
+research skill going in: no `## Arguments`, no `tasks`-refusal, no shell-rules
+paragraph, no no-operator statement. Added all four.
+
+| run | model | outcome |
+|---|---|---|
+| v0.3.0 (hardened) | qwen3.8-27b | environment capture (`python3 --version`, `uname -a`, CPU model), `numpy` version check, `sha256sum` on the input array and the frozen baseline copy, `.clio-coder/validation.yaml` written with thresholds/tolerance semantics before any benchmark, a real 20-rep baseline vs. a vectorized candidate, a bit-exact accuracy check between them, then a correctness spot-check on a second slice-based candidate before timing it — 34 tool calls, zero safety blocks, zero `$(...)`, zero `tasks`, `write`/`edit` used correctly (in this skill's surface, unlike scientific-debugging) instead of a heredoc |
+
+The run did not reach a final Phase 3 verdict/report inside the 280s box used
+this pass (31 API calls, ~750k cumulative input tokens — the box closes on
+context-reprocessing volume, not model slowness); everything observed up to
+that point followed Phase 0-2 exactly as specified, including registering a
+100x stretch target, measuring ~13x, and correctly continuing to iterate
+rather than declaring victory early.
+
+**Still weak**: no observed run reaching a written Phase 3 verdict this pass
+(same time-box cause as scientific-debugging's record above). No cross-model
+confirmation this pass.

package/skills/research/scientific-debugging/SKILL.md

@@ -8,7 +8,7 @@ triggers:
   - diagnose NaNs
   - debug with falsifiable hypotheses
   - scientific root cause
-version: 0.2.0
+version: 0.3.0
 license: Apache-2.0
 allowed-tools:
   - read
@@ -38,6 +38,28 @@ Anti-trigger: if the failure is a typo, a missing import, or an error message
 that names its own cause, fix it directly and skip this workflow. The loop
 below is for failures that survived the first obvious fix.
 
+## Arguments
+
+```text
+/skill scientific-debugging <failure description>
+```
+
+Everything after the skill name is the failure report: the observed wrong
+behavior and whatever has already been tried. There is no operator in a
+headless run — `ask_user` is not in this skill's tool surface. If the goal,
+a fault-class split, or a ranking call is ambiguous, state your best reading
+in Step 1 or Step 3 and proceed; never stall a step waiting for confirmation.
+
+The Loop below is the plan; do not open a task list for it — `tasks` sits
+outside this skill's tool surface and any call to it is refused.
+
+Shell rules for every `bash` call: one command per call, plain and direct.
+Never use `$(...)` or backticks; they trigger an approval gate that ends a
+headless run. This skill has no `write`/`edit` tool — the structured-
+investigation file in the Tiers section below is written with a `bash`
+heredoc (`cat > file <<'EOF' ... EOF`), never through an edit tool that
+isn't in this skill's surface.
+
 ## The Loop
 
 1. **Goal.** One sentence stating the observable "fixed" state. "The regression