@iowarp/clio-coder 0.4.6 → 0.4.8

package/library/skills/research/arxiv-literature/SKILL.md

@@ -0,0 +1,168 @@
+---
+name: arxiv-literature
+description: Searches arXiv, summarizes or compares papers, and builds a compact literature survey as citation-ready, source-linked paper cards. Prefer the Researcher shadow agent for noisy multi-paper retrieval.
+triggers:
+  - search arXiv
+  - summarize an arXiv paper
+  - compare these papers
+  - find recent research papers
+  - build a literature survey
+version: 0.5.1
+license: Apache-2.0
+allowed-tools:
+  - web_fetch
+  - dispatch
+  - read
+  - grep
+  - find
+  - ls
+clio-coder:
+  registry-id: iowarp/clio-coder
+  source-url: https://github.com/iowarp/clio-coder/tree/main/library/skills/research/arxiv-literature
+  audit: pass
+  provenance: designed
+  eval-status: smoke-checked
+  model-size: any
+  agents:
+    - researcher
+---
+
+# ArXiv Literature
+
+Find, summarize, or compare academic papers without flooding the main context
+window. Raw search results and paper text stay in a worker or get compressed
+immediately; only paper cards reach the user.
+
+## Arguments
+
+```text
+/skill arxiv-literature <request>
+```
+
+The request is free text: a paper URL/ID, a topic, or two or more IDs to
+compare. There is no operator in a headless run: `ask_user` is not registered, so
+any call is refused as an unregistered tool rather than answered. If the request is ambiguous (no
+clear topic, an ID that doesn't resolve), state your best reading and
+proceed; never stall waiting for clarification. `tasks` sits outside this
+skill's tool surface and is refused; the steps below are the whole plan, do
+not open a task list for them.
+
+## Step 1 — Classify the request
+
+Pick exactly one:
+
+- **Single paper**: one arXiv URL/ID, "summarize this paper", "explain this paper".
+- **Search**: "find papers about X", "search arxiv for X", "latest papers on X".
+- **Compare**: two or more paper URLs/IDs, "compare these papers".
+- **Survey**: "what is the literature on X", "best papers for X".
+
+## Step 2 — Pick the vehicle
+
+Default every request — single paper, search, compare, and survey alike — to
+`web_fetch` directly against arXiv:
+
+- Paper URL/ID: fetch the arXiv page. Clio normalizes it into structured
+  metadata plus AlphaXiv enrichment when available.
+- Topic, compare, or survey: fetch the arXiv Atom API; Clio compacts the XML
+  into paper cards:
+
+  ```text
+  https://export.arxiv.org/api/query?search_query=all:QUERY&sortBy=submittedDate&sortOrder=descending&start=0&max_results=10
+  ```
+
+  For a compare request, run one query per paper ID (`id_list=ID` instead of
+  `search_query`) or one broader query covering all of them — whichever stays
+  inside the fetch cap in Step 3.
+
+Useful categories for `search_query`: `cs.AI` (AI), `cs.LG` (ML), `cs.CL`
+(NLP/LLMs), `cs.CR` (security), `cs.SE` (software engineering), `cs.MA`
+(multi-agent), `cs.IR` (retrieval/RAG), `cs.CV` (vision), `cs.RO` (robotics).
+
+**Only dispatch the `researcher` shadow agent when the user explicitly asks
+for a deep or broad survey** ("survey the field", "don't just skim arXiv, go
+wide") — never as the default for an ordinary search or compare. Left
+unbounded, a dispatched worker has no arXiv-only restriction and no fetch
+budget of its own, and will wander into Semantic Scholar, DBLP, OpenAlex, and
+general web search, taking several minutes to return nothing useful. When you
+do dispatch, state the same bound this skill uses directly, in the task
+prompt itself:
+
+```text
+Research arXiv literature for: <user goal>.
+Search arXiv only (export.arxiv.org Atom API or arxiv.org paper pages) — do
+not query Semantic Scholar, DBLP, OpenAlex, or general web search. One
+round, at most two fetch attempts total (successes and failures both count).
+On a timeout or HTTP error, do not retry with a different host, scheme, or
+protocol — retry the identical URL at most once, then stop and report the
+failure. Build cards from whatever you have; do not keep escalating.
+Return only compact source-linked paper cards, comparison/synthesis,
+caveats, and read/skim/skip recommendations.
+```
+
+If dispatch is unavailable, fall back to the direct `web_fetch` path above.
+
+## Step 3 — Return paper cards only
+
+Never paste raw Atom XML or full paper text into the response. Output format:
+
+```markdown
+## Literature Result
+
+### Query
+...
+
+### Best Matches / Papers
+1. **Title** — authors, date, `arxiv:id`
+   - Problem:
+   - Method:
+   - Evidence:
+   - Limitation:
+   - Relevance:
+   - Links:
+
+### Recommendation
+- Read:
+- Skim:
+- Skip:
+```
+
+Done when every returned paper has a card with a working link and the
+recommendation section is filled in. Stop after one search round unless the
+user asks to go deeper; do not keep fetching to "be thorough". One round
+means at most two Atom API fetch attempts total: the initial query plus one
+refinement, or one retry of a failed fetch. Failed attempts (timeout, 429,
+connection error) count against this cap the same as successful ones — a
+timeout is not a free retry. Rewording the same query a third time, or
+retrying through a different host/scheme (`http` vs `https`,
+`export.arxiv.org` vs `arxiv.org/search`, an unofficial JSON mirror) after a
+failure, is thrash: build cards from whatever the attempts returned, or
+report the network failure plainly and stop.
+
+## Gotchas
+
+- arXiv Atom is XML, not JSON; fetch it through `web_fetch` so Clio compacts it.
+- `lastUpdatedDate` surfaces old papers with trivial edits; use `submittedDate`
+  for newly submitted work.
+- AlphaXiv is AI-generated enrichment: useful for scanning, never citable as
+  authoritative.
+- Fetch or enrich only the top candidates, not every result.
+- `export.arxiv.org` rate-limits (HTTP 429) under repeated hits; that is a
+  reason to stop at the fetch cap, not a reason to retry against a mirror.
+- This skill's tool surface has no file-writing tool and no `artifact`. The
+  paper cards are the chat reply, never a document; do not go looking for a
+  way to save one.
+
+## Red flags
+
+- Dispatching `researcher` for an ordinary search or compare "just in case"
+  it does a better job than a direct fetch — it is slower and unbounded by
+  default; reserve it for an explicit deep-survey ask.
+- A dispatch task prompt without the arXiv-only, fetch-capped instruction —
+  that omission is what let a prior run wander into Semantic Scholar, DBLP,
+  and OpenAlex for minutes with nothing to show for it.
+- Treating a timeout or 429 as free of the fetch cap and retrying against a
+  different host, scheme, or unofficial mirror instead of stopping.
+- Raw Atom XML, a full abstract dump, or more than the top few candidates
+  reaching the final reply.
+- Reaching for `artifact` or any write tool to "save" the result — it is not
+  in this skill's tool surface; the reply is the deliverable.

package/library/skills/research/arxiv-literature/evals.md

@@ -0,0 +1,108 @@
+# Evals - arxiv-literature
+
+Baseline scenarios (run a subagent WITHOUT the skill to capture the gap, then
+WITH the skill to confirm it closes). Rubric is pass/fail per bullet.
+
+## S1 - topic search
+
+Setup: Prompt: "find recent arXiv papers on speculative decoding for LLM
+inference and tell me which are worth reading."
+
+Expected:
+
+- Retrieval happens through the `researcher` shadow agent or the arXiv Atom
+  API via `web_fetch`, not through generic web browsing.
+- No raw Atom XML or full paper text appears in the final answer.
+- Each paper is a compact card carrying title, authors, date, and an
+  `arxiv:id` or link.
+- The answer closes with explicit read/skim/skip recommendations.
+
+## S2 - single known paper
+
+Setup: Prompt: "summarize https://arxiv.org/abs/1706.03762 for an engineer
+deciding whether to read it."
+
+Expected:
+
+- Fetches the paper page directly with `web_fetch` instead of dispatching a
+  search.
+- The summary states problem, method, evidence, and limitation distinctly
+  rather than paraphrasing the abstract in one block.
+- Any AlphaXiv or AI-generated enrichment is labeled as such, not presented
+  as the paper's own claims.
+
+## S3 - comparison without context flooding
+
+Setup: Prompt gives three arXiv IDs and asks "compare these approaches and
+recommend one for low-latency serving."
+
+Expected:
+
+- Fetching and digestion is delegated or batched so full paper contents do
+  not accumulate in the main transcript.
+- The comparison is criterion-based (problem fit, evidence, limitations),
+  citing each paper by id.
+- A single recommendation is made and justified against the alternatives.
+
+## Baseline failure modes to watch for (RED)
+
+- Raw Atom XML or multi-page paper text pasted into the answer.
+- Searches by `lastUpdatedDate`, surfacing old papers with trivial edits as
+  "recent work".
+- Fetching every search hit instead of the top candidates.
+- Uncited claims: summaries with no ids or links a reader can follow.
+
+## Smoke record (2026-08-13)
+
+One representative scenario via `clio-coder eval skill` against Nemo-3.5-Lightning
+(30B local, llamacpp on mini), full-auto sandbox. PASS (driver FAIL overturned on transcript review): skill loaded, skill_surface blocked two curl attempts, retrieval ran through web_fetch on the Atom API per the designed dispatch-unavailable fallback. Judge misread the OR in bullet 1. Query thrash (8 fetches) led to the two-fetch cap now in the body.
+
+## Battletest record (2026-09-03)
+
+S1 (topic search, speculative decoding) and S2 (single known paper,
+`arxiv.org/abs/1706.03762`), `ornith1.5-35b-moe` on mini (llamacpp), `clio-coder
+run --autonomy full-auto --json`, headless, real network access (no fixture).
+
+| run | model | wall | turns | in / out tokens | safety blocks | outcome |
+|---|---|---|---|---|---|---|
+| baseline S1 (no skill) | ornith1.5-35b-moe | 286s (killed at timeout) | 9 | n/a (killed before `agent_end`) | 5 | never finished: fired raw `curl` via `bash` (permission-gated, refused) before falling back to `web_fetch`; hit real timeouts against `export.arxiv.org`; killed by the harness's 300s cap mid-turn with no cards produced |
+| baseline S2 (no skill) | ornith1.5-35b-moe | 24s | 2 | 3.6k / 1.0k | 0 | fetched the real paper directly and wrote a good prose summary, but never labeled problem/method/evidence/limitation as distinct fields — the un-skilled gap S2 expects |
+| v1 S1 (frozen v0.4.0, dispatch runaway) | ornith1.5-35b-moe | 286s (killed at timeout) | 6 | n/a (killed before `agent_end`) | 2 | opened a `tasks` plan (refused, outside surface), dispatched `researcher` twice (first dispatch flagged an absolute-path token in the briefing), then started re-fetching directly itself; killed by the harness's 300s cap before producing cards — the dispatch-runaway/no-cap bug reproduced live |
+| v2 S1 (hardened v0.5.0) | ornith1.5-35b-moe | 109s | 4 | 2.9k / 3.9k | 2 (real `export.arxiv.org` 429 + timeout) | stayed on `web_fetch` only (no dispatch — topic search didn't ask for a "deep survey"), made exactly two fetch attempts against the identical URL per the tightened cap, hit a genuine rate limit then a timeout, **stopped at the cap**, explicitly refused to fabricate paper cards from invented IDs, and returned an honestly-labeled "established knowledge, not a fresh fetch" orientation instead — terminated cleanly on its own, no runaway |
+| v2 S2 (hardened v0.5.0) | ornith1.5-35b-moe | 28s | 3 | 6.2k / 1.5k | 0 | direct `web_fetch` on the real paper page, full problem/method/evidence/limitation/relevance card, explicit read/skim/skip recommendation, AlphaXiv linked and labeled as enrichment, no `artifact` call — 5/5 on the S2 rubric |
+
+Changes: removed `artifact` from `allowed-tools` — nothing in the procedure
+ever called it, and it is a terminal tool that would end the run the moment
+the model reached for it to "save" a result. Reworked Step 2 so `web_fetch`
+directly against arXiv is the default vehicle for every request class
+(single paper, search, compare, survey), and `dispatch` is reserved for an
+explicitly requested deep/broad survey — the live probe that motivated this
+pass showed a dispatched `researcher` wandering into Semantic Scholar, DBLP,
+and OpenAlex with no arXiv-only restriction and no fetch budget, never
+returning. When dispatch is used, its task-prompt template now states the
+same arXiv-only, fetch-capped discipline the direct path uses, verbatim.
+Tightened the Step 3 fetch cap to count failed attempts (timeout, 429,
+connection error) against the same two-attempt budget as successes, and
+banned escalating to a different host/scheme/mirror on failure — this closed
+a real gap the v2 S1 run against a genuinely rate-limited `export.arxiv.org`
+would otherwise have exploited (the same tightened wording is what let it
+stop cleanly at 109s instead of retrying indefinitely). Added `## Arguments`
+(free-text request, no operator headlessly, `tasks` refused) and a `##
+Red flags` section naming the dispatch-runaway, uncapped-retry, and
+artifact-reach failure modes actually observed.
+
+Still weak: S3 (three-paper comparison) was not run against a real fixture
+this pass — budget went to confirming the dispatch-runaway fix and the
+fetch-cap fix on S1/S2, both of which reproduced live. The compare path's
+"one query per ID or one broader query" guidance in Step 2 is new and
+untested end-to-end. Both baseline and v1 runs for S1 were killed by the
+harness's outer timeout rather than allowed to run to their own natural
+(bad) conclusion — a longer timeout might show the old skill eventually
+recovering, or might show it running further off scope; the fix (bounding
+the fetch cap and defaulting off dispatch) is validated by the v2 behavior,
+not by watching v1 fail for longer. `export.arxiv.org` rate-limited several
+runs in this session from repeated hits in short succession; the 429s in the
+v2 S1 run are a real external condition this pass ran into, not a fixture
+simulation, but a quieter network day could produce a fully-populated
+card set on the same prompt instead of the honest-failure path exercised
+here — both are now handled, but only the failure path got a live rep.

package/library/skills/research/arxiv-literature/plugin.json

@@ -0,0 +1,23 @@
+{
+	"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
+	"name": "arxiv-literature",
+	"version": "0.5.1",
+	"description": "Searches arXiv, summarizes or compares papers, and builds a compact literature survey as citation-ready, source-linked paper cards. Prefer the Researcher shadow agent for noisy multi-paper retrieval.",
+	"license": "Apache-2.0",
+	"extensions": {
+		"ai.iowarp.clio": {
+			"manifestVersion": 1,
+			"kind": "skill",
+			"resources": {
+				"skills": "."
+			},
+			"components": [
+				{
+					"kind": "skill",
+					"id": "arxiv-literature",
+					"path": "SKILL.md"
+				}
+			]
+		}
+	}
+}

package/library/skills/research/experiment-protocol/SKILL.md

@@ -0,0 +1,148 @@
+---
+name: experiment-protocol
+description: "Pre-registers a performance study, numerical comparison, parameter sweep, or benchmark: thresholds, tolerances, environment pins, and verdict conditions locked into the validation contract before any measurement. Not for diagnosing a stalled bug; use scientific-debugging."
+triggers:
+  - benchmark these implementations
+  - pre-register a performance experiment
+  - run a parameter sweep
+  - define numerical tolerances
+  - reproduce these results
+  - compare solver accuracy
+version: 0.3.1
+license: Apache-2.0
+allowed-tools:
+  - read
+  - write
+  - grep
+  - ls
+  - find
+  - git
+  - context
+  - code_nav
+  - bash
+clio-coder:
+  registry-id: iowarp/clio-coder
+  source-url: https://github.com/iowarp/clio-coder/tree/main/library/skills/research/experiment-protocol
+  audit: pass
+  provenance: designed
+  eval-status: smoke-checked
+  model-size: any
+---
+
+# Experiment Protocol
+
+Lock the success criteria before any result exists. Pre-registration that
+happens after the first measurement is worthless: it can only ratify what was
+already seen. Moving the threshold after seeing results is the exact failure
+mode this protocol exists to prevent.
+
+Anti-trigger: if the question is "why is this output wrong", that is a
+diagnosis, not an experiment; use scientific-debugging.
+
+## Arguments
+
+```text
+/skill experiment-protocol <what to benchmark, compare, or sweep>
+```
+
+There is no operator in a headless run: `ask_user` is not registered, so
+any call is refused as an unregistered tool rather than answered. If a threshold, tolerance, or environment detail is unstated,
+pick the most defensible default, record it as an explicit assumption in the
+pre-registration, and proceed; never stall Phase 0 waiting for confirmation.
+
+The phases below are the plan; do not open a task list for them — `tasks`
+sits outside this skill's tool surface and any call to it is refused.
+
+Shell rules for every `bash` call: one command per call, plain and direct.
+Never use `$(...)` or backticks; they trigger an approval gate that ends a
+headless run. Capture checksums and environment facts with direct calls
+(`sha256sum data/mesh.h5`), never command substitution.
+
+## Phase 0 - Pre-register
+
+Before any measurement, write the protocol into the repository validation
+contract: `.clio-coder/validation.yaml` (preferred) or `VALIDATION.md` at the repo
+root. Creating this file raises Clio's repo-derived rigor to high immediately;
+from the next turn onward, completion claims in this session must carry
+validation evidence or state a limitation. That escalation is the point:
+the contract arms the finish gate with the criteria you are about to commit to.
+
+The pre-registration must contain:
+
+- **Outcome**: one sentence, e.g. "the fused kernel reaches at least 1.8x the
+  baseline throughput on the pinned input at equal accuracy".
+- **Thresholds**: minimum (below this is REFUTED), target, stretch.
+- **Tolerance semantics per metric**: absolute for near-zero quantities,
+  relative elsewhere; a mixed scheme must say which applies where. "1e-6" with
+  no stated semantics is not a tolerance.
+- **Environment pin**: compiler and flags, modules or package versions, node
+  class, scheduler context (partition, exclusivity).
+- **Input identity**: paths plus checksums (`sha256sum`).
+- **Verdict conditions**: what observation makes the result CONFIRMED, REFUTED,
+  or INCONCLUSIVE. Committed now, immutable after the first measurement.
+
+## Phase 1 - Baseline
+
+Capture current behavior under the pinned environment before changing
+anything. Store raw outputs and timings as artifacts; never edit them. A
+speedup claim without a baseline captured under the same pin is a guess.
+
+## Phase 2 - Experiment
+
+- One independent variable per run. A run that changes the algorithm and the
+  compiler flags answers no question.
+- Size repetitions to the noise: shared nodes and networked filesystems need
+  more repetitions and a reported variance, not a single lucky run.
+- Record scheduler identity (job id, node list) alongside every measurement.
+
+## Phase 3 - Analysis
+
+Compare against the pre-registered thresholds only. If the protocol was
+deviated from, log the deviation next to the result; do not silently absorb
+it. Findings outside the registered outcome are marked exploratory and get
+their own pre-registration if pursued.
+
+## Phase 4 - Iterate
+
+Keep a dead-ends ledger in the contract file or beside it: one line per
+rejected approach with the reason it was rejected. Read it before proposing
+the next approach; re-proposing a ledger entry wastes a run. Stop when one of
+the pre-registered stop conditions holds: target met, budget exhausted, or all
+candidate strategies rejected.
+
+## Worked Example
+
+Request: "make the halo exchange faster."
+
+```yaml
+# .clio-coder/validation.yaml
+experiment: halo-exchange-overlap
+outcome: overlap communication with interior compute; >= 1.5x step throughput
+thresholds: { minimum: 1.2x, target: 1.5x, stretch: 2.0x }
+metrics:
+  step_time: { semantics: relative, tolerance: 5% run-to-run variance }
+  solution_l2: { semantics: absolute, tolerance: 1e-12 vs baseline }
+environment: gcc 13.2 -O3, openmpi 4.1.6, 4x cpu-bind=cores, exclusive nodes
+inputs: { mesh: data/mesh-256.h5, sha256: "<checksum>" }
+verdicts:
+  confirmed: step_time speedup >= 1.2x AND solution_l2 within tolerance
+  refuted: speedup < 1.2x with variance < 5%, or accuracy loss
+  inconclusive: run-to-run variance > 5% (resize repetitions first)
+dead_ends: []
+```
+
+Baseline: 20 repetitions on exclusive nodes, mean and variance recorded with
+job ids. Experiment: nonblocking exchange only, flags untouched. Result: 1.3x,
+accuracy within 1e-12: CONFIRMED at minimum, target not met; ledger gains
+"persistent requests: no gain over nonblocking here, latency-bound", and the
+next variable (message aggregation) gets its own run.
+
+## Red Flags
+
+- Any measurement taken before the contract file exists.
+- A threshold, tolerance, or verdict condition edited after results appeared.
+- Wall-clock numbers with no environment pin attached.
+- A single-run victory claim on a shared machine.
+- A REFUTED result reported as "promising"; refuted plus a ledger entry is a
+  successful experiment, say so plainly.
+- Deleting or rewriting baseline artifacts.

package/library/skills/research/experiment-protocol/evals.md

@@ -0,0 +1,114 @@
+# Evals - experiment-protocol
+
+Baseline scenarios (run a subagent WITHOUT the skill to capture the gap, then
+WITH the skill to confirm it closes). Rubric is pass/fail per bullet.
+
+## S1 - "make this kernel faster"
+
+Setup: make the smoothing kernel in kernel.py faster.
+
+Fixture:
+```bash
+printf 'import time\n\ndef smooth(values, window):\n    out = []\n    for i in range(len(values)):\n        lo = max(0, i - window)\n        hi = min(len(values), i + window + 1)\n        out.append(sum(values[lo:hi]) / (hi - lo))\n    return out\n\nif __name__ == "__main__":\n    data = [float(i %% 97) for i in range(200000)]\n    t0 = time.perf_counter()\n    smooth(data, 25)\n    print("seconds:", round(time.perf_counter() - t0, 3))\n' > kernel.py
+```
+
+Expected:
+
+- Writes a pre-registration into `.clio-coder/validation.yaml` or `VALIDATION.md`
+  before running any benchmark or editing any code.
+- The pre-registration contains thresholds (minimum/target/stretch) and
+  tolerance semantics stated per metric (absolute vs relative).
+- Pins the environment (compiler, flags, versions, node/scheduler context)
+  and identifies inputs by path plus checksum.
+- Captures a baseline under the same pin before changing anything.
+- Changes one independent variable per experiment run.
+
+## S2 - results miss the target
+
+Setup: the pre-registered target was 1.5x; the measured result is 1.1x, below
+the registered minimum of 1.2x.
+
+Expected:
+
+- Reports REFUTED against the original pre-registered threshold.
+- Does not restate the goal, lower the threshold, or reframe 1.1x as success.
+- Adds a dead-ends ledger entry naming the rejected approach and the reason.
+- Proposes the next approach only after reading the ledger.
+
+## S3 - noisy measurements on a shared machine
+
+Setup: benchmark runs on a shared node; run-to-run variance exceeds the gap
+being measured.
+
+Expected:
+
+- Sizes repetitions to the observed noise instead of reporting a single run.
+- Reports variance alongside the mean, with scheduler identity recorded.
+- Declares INCONCLUSIVE if variance swamps the effect, rather than picking
+  the best run.
+
+## S4 - anti-trigger: wrong output
+
+Setup: user asks "why is this solver producing wrong values?"
+
+Expected:
+
+- Refers to scientific-debugging instead of starting a benchmark protocol.
+- Does not write a validation contract for a diagnosis task.
+
+## Baseline failure modes to watch for (RED)
+
+- Benchmarks first, defines success afterward from whatever the numbers show.
+- "Faster" claimed from one run, no baseline, no environment pin.
+- Threshold quietly adjusted after seeing results.
+- Tolerance given as a bare number with no absolute/relative semantics.
+- Rejected approaches vanish; the next session re-proposes them.
+- Raw baseline artifacts edited or overwritten.
+
+## Observed gap closure
+
+S1 run 2026-07-01, headless `clio-coder run` against a scratch git fixture (a pure
+Python O(n^2) nearest-neighbor kernel with a single-shot bench script).
+Prompt: "Make this kernel faster."
+
+- RED (no skill): the agent edited the kernel immediately, reported a 7.3x
+  speedup from one timing run each way, wrote no contract, stated no
+  thresholds or tolerance semantics, and pinned nothing. Correctness was
+  checked ad hoc after the fact.
+- GREEN (skill via `--skill` and `/skill <name>` invocation): the tool-call ledger
+  shows sha256 and environment capture, then `.clio-coder/validation.yaml` written
+  with min/target/stretch thresholds, per-metric tolerance semantics, and
+  verdict conditions, then the repetition-sized warm-up baseline, then
+  experiments, with the kernel edited only after a variant met the contract.
+  A float32 variant that beat the target but missed the accuracy tolerance
+  was rejected and logged in the dead-ends ledger. Writing the contract
+  raised repo rigor to high mid-session and the finish gate demanded
+  validation evidence before the turn settled. All five S1 bullets pass.
+
+## Smoke record (2026-08-13)
+
+One representative scenario via `clio-coder eval skill` against Nemo-3.5-Lightning
+(30B local, llamacpp on mini), full-auto sandbox. PASS. Pre-registration written before touching the seeded kernel; judge 5/5.
+
+## Battletest record (2026-09-03)
+
+Real `clio-coder run --json` against dynamo (LM Studio, `qwen3.8-27b`), the S1
+kernel.py fixture above, git repo under
+`/home/akougkas/eval-temp/expprotocol-fixture/`. Same gap as every other
+research skill going in: no `## Arguments`, no `tasks`-refusal, no shell-rules
+paragraph, no no-operator statement. Added all four.
+
+| run | model | outcome |
+|---|---|---|
+| v0.3.0 (hardened) | qwen3.8-27b | environment capture (`python3 --version`, `uname -a`, CPU model), `numpy` version check, `sha256sum` on the input array and the frozen baseline copy, `.clio-coder/validation.yaml` written with thresholds/tolerance semantics before any benchmark, a real 20-rep baseline vs. a vectorized candidate, a bit-exact accuracy check between them, then a correctness spot-check on a second slice-based candidate before timing it — 34 tool calls, zero safety blocks, zero `$(...)`, zero `tasks`, `write`/`edit` used correctly (in this skill's surface, unlike scientific-debugging) instead of a heredoc |
+
+The run did not reach a final Phase 3 verdict/report inside the 280s box used
+this pass (31 API calls, ~750k cumulative input tokens — the box closes on
+context-reprocessing volume, not model slowness); everything observed up to
+that point followed Phase 0-2 exactly as specified, including registering a
+100x stretch target, measuring ~13x, and correctly continuing to iterate
+rather than declaring victory early.
+
+**Still weak**: no observed run reaching a written Phase 3 verdict this pass
+(same time-box cause as scientific-debugging's record above). No cross-model
+confirmation this pass.

package/library/skills/research/experiment-protocol/plugin.json

@@ -0,0 +1,23 @@
+{
+	"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
+	"name": "experiment-protocol",
+	"version": "0.3.1",
+	"description": "Pre-registers a performance study, numerical comparison, parameter sweep, or benchmark: thresholds, tolerances, environment pins, and verdict conditions locked into the validation contract before any measurement. Not for diagnosing a stalled bug; use scientific-debugging.",
+	"license": "Apache-2.0",
+	"extensions": {
+		"ai.iowarp.clio": {
+			"manifestVersion": 1,
+			"kind": "skill",
+			"resources": {
+				"skills": "."
+			},
+			"components": [
+				{
+					"kind": "skill",
+					"id": "experiment-protocol",
+					"path": "SKILL.md"
+				}
+			]
+		}
+	}
+}