@iowarp/clio-coder 0.4.6 → 0.4.8

package/skills/research/arxiv-literature/SKILL.md

@@ -1,168 +0,0 @@
----
-name: arxiv-literature
-description: Searches arXiv, summarizes or compares papers, and builds a compact literature survey as citation-ready, source-linked paper cards. Prefer the Researcher shadow agent for noisy multi-paper retrieval.
-triggers:
-  - search arXiv
-  - summarize an arXiv paper
-  - compare these papers
-  - find recent research papers
-  - build a literature survey
-version: 0.5.1
-license: Apache-2.0
-allowed-tools:
-  - web_fetch
-  - dispatch
-  - read
-  - grep
-  - find
-  - ls
-clio-coder:
-  registry-id: iowarp/clio-coder
-  source-url: https://github.com/iowarp/clio-coder/tree/main/skills/research/arxiv-literature
-  audit: pass
-  provenance: designed
-  eval-status: smoke-checked
-  model-size: any
-  agents:
-    - researcher
----
-
-# ArXiv Literature
-
-Find, summarize, or compare academic papers without flooding the main context
-window. Raw search results and paper text stay in a worker or get compressed
-immediately; only paper cards reach the user.
-
-## Arguments
-
-```text
-/skill arxiv-literature <request>
-```
-
-The request is free text: a paper URL/ID, a topic, or two or more IDs to
-compare. There is no operator in a headless run: `ask_user` is not registered, so
-any call is refused as an unregistered tool rather than answered. If the request is ambiguous (no
-clear topic, an ID that doesn't resolve), state your best reading and
-proceed; never stall waiting for clarification. `tasks` sits outside this
-skill's tool surface and is refused; the steps below are the whole plan, do
-not open a task list for them.
-
-## Step 1 — Classify the request
-
-Pick exactly one:
-
-- **Single paper**: one arXiv URL/ID, "summarize this paper", "explain this paper".
-- **Search**: "find papers about X", "search arxiv for X", "latest papers on X".
-- **Compare**: two or more paper URLs/IDs, "compare these papers".
-- **Survey**: "what is the literature on X", "best papers for X".
-
-## Step 2 — Pick the vehicle
-
-Default every request — single paper, search, compare, and survey alike — to
-`web_fetch` directly against arXiv:
-
-- Paper URL/ID: fetch the arXiv page. Clio normalizes it into structured
-  metadata plus AlphaXiv enrichment when available.
-- Topic, compare, or survey: fetch the arXiv Atom API; Clio compacts the XML
-  into paper cards:
-
-  ```text
-  https://export.arxiv.org/api/query?search_query=all:QUERY&sortBy=submittedDate&sortOrder=descending&start=0&max_results=10
-  ```
-
-  For a compare request, run one query per paper ID (`id_list=ID` instead of
-  `search_query`) or one broader query covering all of them — whichever stays
-  inside the fetch cap in Step 3.
-
-Useful categories for `search_query`: `cs.AI` (AI), `cs.LG` (ML), `cs.CL`
-(NLP/LLMs), `cs.CR` (security), `cs.SE` (software engineering), `cs.MA`
-(multi-agent), `cs.IR` (retrieval/RAG), `cs.CV` (vision), `cs.RO` (robotics).
-
-**Only dispatch the `researcher` shadow agent when the user explicitly asks
-for a deep or broad survey** ("survey the field", "don't just skim arXiv, go
-wide") — never as the default for an ordinary search or compare. Left
-unbounded, a dispatched worker has no arXiv-only restriction and no fetch
-budget of its own, and will wander into Semantic Scholar, DBLP, OpenAlex, and
-general web search, taking several minutes to return nothing useful. When you
-do dispatch, state the same bound this skill uses directly, in the task
-prompt itself:
-
-```text
-Research arXiv literature for: <user goal>.
-Search arXiv only (export.arxiv.org Atom API or arxiv.org paper pages) — do
-not query Semantic Scholar, DBLP, OpenAlex, or general web search. One
-round, at most two fetch attempts total (successes and failures both count).
-On a timeout or HTTP error, do not retry with a different host, scheme, or
-protocol — retry the identical URL at most once, then stop and report the
-failure. Build cards from whatever you have; do not keep escalating.
-Return only compact source-linked paper cards, comparison/synthesis,
-caveats, and read/skim/skip recommendations.
-```
-
-If dispatch is unavailable, fall back to the direct `web_fetch` path above.
-
-## Step 3 — Return paper cards only
-
-Never paste raw Atom XML or full paper text into the response. Output format:
-
-```markdown
-## Literature Result
-
-### Query
-...
-
-### Best Matches / Papers
-1. **Title** — authors, date, `arxiv:id`
-   - Problem:
-   - Method:
-   - Evidence:
-   - Limitation:
-   - Relevance:
-   - Links:
-
-### Recommendation
-- Read:
-- Skim:
-- Skip:
-```
-
-Done when every returned paper has a card with a working link and the
-recommendation section is filled in. Stop after one search round unless the
-user asks to go deeper; do not keep fetching to "be thorough". One round
-means at most two Atom API fetch attempts total: the initial query plus one
-refinement, or one retry of a failed fetch. Failed attempts (timeout, 429,
-connection error) count against this cap the same as successful ones — a
-timeout is not a free retry. Rewording the same query a third time, or
-retrying through a different host/scheme (`http` vs `https`,
-`export.arxiv.org` vs `arxiv.org/search`, an unofficial JSON mirror) after a
-failure, is thrash: build cards from whatever the attempts returned, or
-report the network failure plainly and stop.
-
-## Gotchas
-
-- arXiv Atom is XML, not JSON; fetch it through `web_fetch` so Clio compacts it.
-- `lastUpdatedDate` surfaces old papers with trivial edits; use `submittedDate`
-  for newly submitted work.
-- AlphaXiv is AI-generated enrichment: useful for scanning, never citable as
-  authoritative.
-- Fetch or enrich only the top candidates, not every result.
-- `export.arxiv.org` rate-limits (HTTP 429) under repeated hits; that is a
-  reason to stop at the fetch cap, not a reason to retry against a mirror.
-- This skill's tool surface has no file-writing tool and no `artifact`. The
-  paper cards are the chat reply, never a document; do not go looking for a
-  way to save one.
-
-## Red flags
-
-- Dispatching `researcher` for an ordinary search or compare "just in case"
-  it does a better job than a direct fetch — it is slower and unbounded by
-  default; reserve it for an explicit deep-survey ask.
-- A dispatch task prompt without the arXiv-only, fetch-capped instruction —
-  that omission is what let a prior run wander into Semantic Scholar, DBLP,
-  and OpenAlex for minutes with nothing to show for it.
-- Treating a timeout or 429 as free of the fetch cap and retrying against a
-  different host, scheme, or unofficial mirror instead of stopping.
-- Raw Atom XML, a full abstract dump, or more than the top few candidates
-  reaching the final reply.
-- Reaching for `artifact` or any write tool to "save" the result — it is not
-  in this skill's tool surface; the reply is the deliverable.

package/skills/research/arxiv-literature/evals.md

@@ -1,108 +0,0 @@
-# Evals - arxiv-literature
-
-Baseline scenarios (run a subagent WITHOUT the skill to capture the gap, then
-WITH the skill to confirm it closes). Rubric is pass/fail per bullet.
-
-## S1 - topic search
-
-Setup: Prompt: "find recent arXiv papers on speculative decoding for LLM
-inference and tell me which are worth reading."
-
-Expected:
-
-- Retrieval happens through the `researcher` shadow agent or the arXiv Atom
-  API via `web_fetch`, not through generic web browsing.
-- No raw Atom XML or full paper text appears in the final answer.
-- Each paper is a compact card carrying title, authors, date, and an
-  `arxiv:id` or link.
-- The answer closes with explicit read/skim/skip recommendations.
-
-## S2 - single known paper
-
-Setup: Prompt: "summarize https://arxiv.org/abs/1706.03762 for an engineer
-deciding whether to read it."
-
-Expected:
-
-- Fetches the paper page directly with `web_fetch` instead of dispatching a
-  search.
-- The summary states problem, method, evidence, and limitation distinctly
-  rather than paraphrasing the abstract in one block.
-- Any AlphaXiv or AI-generated enrichment is labeled as such, not presented
-  as the paper's own claims.
-
-## S3 - comparison without context flooding
-
-Setup: Prompt gives three arXiv IDs and asks "compare these approaches and
-recommend one for low-latency serving."
-
-Expected:
-
-- Fetching and digestion is delegated or batched so full paper contents do
-  not accumulate in the main transcript.
-- The comparison is criterion-based (problem fit, evidence, limitations),
-  citing each paper by id.
-- A single recommendation is made and justified against the alternatives.
-
-## Baseline failure modes to watch for (RED)
-
-- Raw Atom XML or multi-page paper text pasted into the answer.
-- Searches by `lastUpdatedDate`, surfacing old papers with trivial edits as
-  "recent work".
-- Fetching every search hit instead of the top candidates.
-- Uncited claims: summaries with no ids or links a reader can follow.
-
-## Smoke record (2026-08-13)
-
-One representative scenario via `clio-coder skills eval` against Nemo-3.5-Lightning
-(30B local, llamacpp on mini), full-auto sandbox. PASS (driver FAIL overturned on transcript review): skill loaded, skill_surface blocked two curl attempts, retrieval ran through web_fetch on the Atom API per the designed dispatch-unavailable fallback. Judge misread the OR in bullet 1. Query thrash (8 fetches) led to the two-fetch cap now in the body.
-
-## Battletest record (2026-09-03)
-
-S1 (topic search, speculative decoding) and S2 (single known paper,
-`arxiv.org/abs/1706.03762`), `ornith1.5-35b-moe` on mini (llamacpp), `clio-coder
-run --autonomy full-auto --json`, headless, real network access (no fixture).
-
-| run | model | wall | turns | in / out tokens | safety blocks | outcome |
-|---|---|---|---|---|---|---|
-| baseline S1 (no skill) | ornith1.5-35b-moe | 286s (killed at timeout) | 9 | n/a (killed before `agent_end`) | 5 | never finished: fired raw `curl` via `bash` (permission-gated, refused) before falling back to `web_fetch`; hit real timeouts against `export.arxiv.org`; killed by the harness's 300s cap mid-turn with no cards produced |
-| baseline S2 (no skill) | ornith1.5-35b-moe | 24s | 2 | 3.6k / 1.0k | 0 | fetched the real paper directly and wrote a good prose summary, but never labeled problem/method/evidence/limitation as distinct fields — the un-skilled gap S2 expects |
-| v1 S1 (frozen v0.4.0, dispatch runaway) | ornith1.5-35b-moe | 286s (killed at timeout) | 6 | n/a (killed before `agent_end`) | 2 | opened a `tasks` plan (refused, outside surface), dispatched `researcher` twice (first dispatch flagged an absolute-path token in the briefing), then started re-fetching directly itself; killed by the harness's 300s cap before producing cards — the dispatch-runaway/no-cap bug reproduced live |
-| v2 S1 (hardened v0.5.0) | ornith1.5-35b-moe | 109s | 4 | 2.9k / 3.9k | 2 (real `export.arxiv.org` 429 + timeout) | stayed on `web_fetch` only (no dispatch — topic search didn't ask for a "deep survey"), made exactly two fetch attempts against the identical URL per the tightened cap, hit a genuine rate limit then a timeout, **stopped at the cap**, explicitly refused to fabricate paper cards from invented IDs, and returned an honestly-labeled "established knowledge, not a fresh fetch" orientation instead — terminated cleanly on its own, no runaway |
-| v2 S2 (hardened v0.5.0) | ornith1.5-35b-moe | 28s | 3 | 6.2k / 1.5k | 0 | direct `web_fetch` on the real paper page, full problem/method/evidence/limitation/relevance card, explicit read/skim/skip recommendation, AlphaXiv linked and labeled as enrichment, no `artifact` call — 5/5 on the S2 rubric |
-
-Changes: removed `artifact` from `allowed-tools` — nothing in the procedure
-ever called it, and it is a terminal tool that would end the run the moment
-the model reached for it to "save" a result. Reworked Step 2 so `web_fetch`
-directly against arXiv is the default vehicle for every request class
-(single paper, search, compare, survey), and `dispatch` is reserved for an
-explicitly requested deep/broad survey — the live probe that motivated this
-pass showed a dispatched `researcher` wandering into Semantic Scholar, DBLP,
-and OpenAlex with no arXiv-only restriction and no fetch budget, never
-returning. When dispatch is used, its task-prompt template now states the
-same arXiv-only, fetch-capped discipline the direct path uses, verbatim.
-Tightened the Step 3 fetch cap to count failed attempts (timeout, 429,
-connection error) against the same two-attempt budget as successes, and
-banned escalating to a different host/scheme/mirror on failure — this closed
-a real gap the v2 S1 run against a genuinely rate-limited `export.arxiv.org`
-would otherwise have exploited (the same tightened wording is what let it
-stop cleanly at 109s instead of retrying indefinitely). Added `## Arguments`
-(free-text request, no operator headlessly, `tasks` refused) and a `##
-Red flags` section naming the dispatch-runaway, uncapped-retry, and
-artifact-reach failure modes actually observed.
-
-Still weak: S3 (three-paper comparison) was not run against a real fixture
-this pass — budget went to confirming the dispatch-runaway fix and the
-fetch-cap fix on S1/S2, both of which reproduced live. The compare path's
-"one query per ID or one broader query" guidance in Step 2 is new and
-untested end-to-end. Both baseline and v1 runs for S1 were killed by the
-harness's outer timeout rather than allowed to run to their own natural
-(bad) conclusion — a longer timeout might show the old skill eventually
-recovering, or might show it running further off scope; the fix (bounding
-the fetch cap and defaulting off dispatch) is validated by the v2 behavior,
-not by watching v1 fail for longer. `export.arxiv.org` rate-limited several
-runs in this session from repeated hits in short succession; the 429s in the
-v2 S1 run are a real external condition this pass ran into, not a fixture
-simulation, but a quieter network day could produce a fully-populated
-card set on the same prompt instead of the honest-failure path exercised
-here — both are now handled, but only the failure path got a live rep.

package/skills/research/experiment-protocol/SKILL.md

@@ -1,148 +0,0 @@
----
-name: experiment-protocol
-description: "Pre-registers a performance study, numerical comparison, parameter sweep, or benchmark: thresholds, tolerances, environment pins, and verdict conditions locked into the validation contract before any measurement. Not for diagnosing a stalled bug; use scientific-debugging."
-triggers:
-  - benchmark these implementations
-  - pre-register a performance experiment
-  - run a parameter sweep
-  - define numerical tolerances
-  - reproduce these results
-  - compare solver accuracy
-version: 0.3.1
-license: Apache-2.0
-allowed-tools:
-  - read
-  - write
-  - grep
-  - ls
-  - find
-  - git
-  - context
-  - code_nav
-  - bash
-clio-coder:
-  registry-id: iowarp/clio-coder
-  source-url: https://github.com/iowarp/clio-coder/tree/main/skills/research/experiment-protocol
-  audit: pass
-  provenance: designed
-  eval-status: smoke-checked
-  model-size: any
----
-
-# Experiment Protocol
-
-Lock the success criteria before any result exists. Pre-registration that
-happens after the first measurement is worthless: it can only ratify what was
-already seen. Moving the threshold after seeing results is the exact failure
-mode this protocol exists to prevent.
-
-Anti-trigger: if the question is "why is this output wrong", that is a
-diagnosis, not an experiment; use scientific-debugging.
-
-## Arguments
-
-```text
-/skill experiment-protocol <what to benchmark, compare, or sweep>
-```
-
-There is no operator in a headless run: `ask_user` is not registered, so
-any call is refused as an unregistered tool rather than answered. If a threshold, tolerance, or environment detail is unstated,
-pick the most defensible default, record it as an explicit assumption in the
-pre-registration, and proceed; never stall Phase 0 waiting for confirmation.
-
-The phases below are the plan; do not open a task list for them — `tasks`
-sits outside this skill's tool surface and any call to it is refused.
-
-Shell rules for every `bash` call: one command per call, plain and direct.
-Never use `$(...)` or backticks; they trigger an approval gate that ends a
-headless run. Capture checksums and environment facts with direct calls
-(`sha256sum data/mesh.h5`), never command substitution.
-
-## Phase 0 - Pre-register
-
-Before any measurement, write the protocol into the repository validation
-contract: `.clio-coder/validation.yaml` (preferred) or `VALIDATION.md` at the repo
-root. Creating this file raises Clio's repo-derived rigor to high immediately;
-from the next turn onward, completion claims in this session must carry
-validation evidence or state a limitation. That escalation is the point:
-the contract arms the finish gate with the criteria you are about to commit to.
-
-The pre-registration must contain:
-
-- **Outcome**: one sentence, e.g. "the fused kernel reaches at least 1.8x the
-  baseline throughput on the pinned input at equal accuracy".
-- **Thresholds**: minimum (below this is REFUTED), target, stretch.
-- **Tolerance semantics per metric**: absolute for near-zero quantities,
-  relative elsewhere; a mixed scheme must say which applies where. "1e-6" with
-  no stated semantics is not a tolerance.
-- **Environment pin**: compiler and flags, modules or package versions, node
-  class, scheduler context (partition, exclusivity).
-- **Input identity**: paths plus checksums (`sha256sum`).
-- **Verdict conditions**: what observation makes the result CONFIRMED, REFUTED,
-  or INCONCLUSIVE. Committed now, immutable after the first measurement.
-
-## Phase 1 - Baseline
-
-Capture current behavior under the pinned environment before changing
-anything. Store raw outputs and timings as artifacts; never edit them. A
-speedup claim without a baseline captured under the same pin is a guess.
-
-## Phase 2 - Experiment
-
-- One independent variable per run. A run that changes the algorithm and the
-  compiler flags answers no question.
-- Size repetitions to the noise: shared nodes and networked filesystems need
-  more repetitions and a reported variance, not a single lucky run.
-- Record scheduler identity (job id, node list) alongside every measurement.
-
-## Phase 3 - Analysis
-
-Compare against the pre-registered thresholds only. If the protocol was
-deviated from, log the deviation next to the result; do not silently absorb
-it. Findings outside the registered outcome are marked exploratory and get
-their own pre-registration if pursued.
-
-## Phase 4 - Iterate
-
-Keep a dead-ends ledger in the contract file or beside it: one line per
-rejected approach with the reason it was rejected. Read it before proposing
-the next approach; re-proposing a ledger entry wastes a run. Stop when one of
-the pre-registered stop conditions holds: target met, budget exhausted, or all
-candidate strategies rejected.
-
-## Worked Example
-
-Request: "make the halo exchange faster."
-
-```yaml
-# .clio-coder/validation.yaml
-experiment: halo-exchange-overlap
-outcome: overlap communication with interior compute; >= 1.5x step throughput
-thresholds: { minimum: 1.2x, target: 1.5x, stretch: 2.0x }
-metrics:
-  step_time: { semantics: relative, tolerance: 5% run-to-run variance }
-  solution_l2: { semantics: absolute, tolerance: 1e-12 vs baseline }
-environment: gcc 13.2 -O3, openmpi 4.1.6, 4x cpu-bind=cores, exclusive nodes
-inputs: { mesh: data/mesh-256.h5, sha256: "<checksum>" }
-verdicts:
-  confirmed: step_time speedup >= 1.2x AND solution_l2 within tolerance
-  refuted: speedup < 1.2x with variance < 5%, or accuracy loss
-  inconclusive: run-to-run variance > 5% (resize repetitions first)
-dead_ends: []
-```
-
-Baseline: 20 repetitions on exclusive nodes, mean and variance recorded with
-job ids. Experiment: nonblocking exchange only, flags untouched. Result: 1.3x,
-accuracy within 1e-12: CONFIRMED at minimum, target not met; ledger gains
-"persistent requests: no gain over nonblocking here, latency-bound", and the
-next variable (message aggregation) gets its own run.
-
-## Red Flags
-
-- Any measurement taken before the contract file exists.
-- A threshold, tolerance, or verdict condition edited after results appeared.
-- Wall-clock numbers with no environment pin attached.
-- A single-run victory claim on a shared machine.
-- A REFUTED result reported as "promising"; refuted plus a ledger entry is a
-  successful experiment, say so plainly.
-- Deleting or rewriting baseline artifacts.

package/skills/research/experiment-protocol/evals.md

@@ -1,114 +0,0 @@
-# Evals - experiment-protocol
-
-Baseline scenarios (run a subagent WITHOUT the skill to capture the gap, then
-WITH the skill to confirm it closes). Rubric is pass/fail per bullet.
-
-## S1 - "make this kernel faster"
-
-Setup: make the smoothing kernel in kernel.py faster.
-
-Fixture:
-```bash
-printf 'import time\n\ndef smooth(values, window):\n    out = []\n    for i in range(len(values)):\n        lo = max(0, i - window)\n        hi = min(len(values), i + window + 1)\n        out.append(sum(values[lo:hi]) / (hi - lo))\n    return out\n\nif __name__ == "__main__":\n    data = [float(i %% 97) for i in range(200000)]\n    t0 = time.perf_counter()\n    smooth(data, 25)\n    print("seconds:", round(time.perf_counter() - t0, 3))\n' > kernel.py
-```
-
-Expected:
-
-- Writes a pre-registration into `.clio-coder/validation.yaml` or `VALIDATION.md`
-  before running any benchmark or editing any code.
-- The pre-registration contains thresholds (minimum/target/stretch) and
-  tolerance semantics stated per metric (absolute vs relative).
-- Pins the environment (compiler, flags, versions, node/scheduler context)
-  and identifies inputs by path plus checksum.
-- Captures a baseline under the same pin before changing anything.
-- Changes one independent variable per experiment run.
-
-## S2 - results miss the target
-
-Setup: the pre-registered target was 1.5x; the measured result is 1.1x, below
-the registered minimum of 1.2x.
-
-Expected:
-
-- Reports REFUTED against the original pre-registered threshold.
-- Does not restate the goal, lower the threshold, or reframe 1.1x as success.
-- Adds a dead-ends ledger entry naming the rejected approach and the reason.
-- Proposes the next approach only after reading the ledger.
-
-## S3 - noisy measurements on a shared machine
-
-Setup: benchmark runs on a shared node; run-to-run variance exceeds the gap
-being measured.
-
-Expected:
-
-- Sizes repetitions to the observed noise instead of reporting a single run.
-- Reports variance alongside the mean, with scheduler identity recorded.
-- Declares INCONCLUSIVE if variance swamps the effect, rather than picking
-  the best run.
-
-## S4 - anti-trigger: wrong output
-
-Setup: user asks "why is this solver producing wrong values?"
-
-Expected:
-
-- Refers to scientific-debugging instead of starting a benchmark protocol.
-- Does not write a validation contract for a diagnosis task.
-
-## Baseline failure modes to watch for (RED)
-
-- Benchmarks first, defines success afterward from whatever the numbers show.
-- "Faster" claimed from one run, no baseline, no environment pin.
-- Threshold quietly adjusted after seeing results.
-- Tolerance given as a bare number with no absolute/relative semantics.
-- Rejected approaches vanish; the next session re-proposes them.
-- Raw baseline artifacts edited or overwritten.
-
-## Observed gap closure
-
-S1 run 2026-07-01, headless `clio-coder run` against a scratch git fixture (a pure
-Python O(n^2) nearest-neighbor kernel with a single-shot bench script).
-Prompt: "Make this kernel faster."
-
-- RED (no skill): the agent edited the kernel immediately, reported a 7.3x
-  speedup from one timing run each way, wrote no contract, stated no
-  thresholds or tolerance semantics, and pinned nothing. Correctness was
-  checked ad hoc after the fact.
-- GREEN (skill via `--skill` and `/skill <name>` invocation): the tool-call ledger
-  shows sha256 and environment capture, then `.clio-coder/validation.yaml` written
-  with min/target/stretch thresholds, per-metric tolerance semantics, and
-  verdict conditions, then the repetition-sized warm-up baseline, then
-  experiments, with the kernel edited only after a variant met the contract.
-  A float32 variant that beat the target but missed the accuracy tolerance
-  was rejected and logged in the dead-ends ledger. Writing the contract
-  raised repo rigor to high mid-session and the finish gate demanded
-  validation evidence before the turn settled. All five S1 bullets pass.
-
-## Smoke record (2026-08-13)
-
-One representative scenario via `clio-coder skills eval` against Nemo-3.5-Lightning
-(30B local, llamacpp on mini), full-auto sandbox. PASS. Pre-registration written before touching the seeded kernel; judge 5/5.
-
-## Battletest record (2026-09-03)
-
-Real `clio-coder run --json` against dynamo (LM Studio, `qwen3.8-27b`), the S1
-kernel.py fixture above, git repo under
-`/home/akougkas/eval-temp/expprotocol-fixture/`. Same gap as every other
-research skill going in: no `## Arguments`, no `tasks`-refusal, no shell-rules
-paragraph, no no-operator statement. Added all four.
-
-| run | model | outcome |
-|---|---|---|
-| v0.3.0 (hardened) | qwen3.8-27b | environment capture (`python3 --version`, `uname -a`, CPU model), `numpy` version check, `sha256sum` on the input array and the frozen baseline copy, `.clio-coder/validation.yaml` written with thresholds/tolerance semantics before any benchmark, a real 20-rep baseline vs. a vectorized candidate, a bit-exact accuracy check between them, then a correctness spot-check on a second slice-based candidate before timing it — 34 tool calls, zero safety blocks, zero `$(...)`, zero `tasks`, `write`/`edit` used correctly (in this skill's surface, unlike scientific-debugging) instead of a heredoc |
-
-The run did not reach a final Phase 3 verdict/report inside the 280s box used
-this pass (31 API calls, ~750k cumulative input tokens — the box closes on
-context-reprocessing volume, not model slowness); everything observed up to
-that point followed Phase 0-2 exactly as specified, including registering a
-100x stretch target, measuring ~13x, and correctly continuing to iterate
-rather than declaring victory early.
-
-**Still weak**: no observed run reaching a written Phase 3 verdict this pass
-(same time-box cause as scientific-debugging's record above). No cross-model
-confirmation this pass.