@iceinvein/agent-skills 0.21.5 → 0.23.0

package/README.md

@@ -29,6 +29,7 @@ Auto-detects Claude Code, Cursor, Codex, or Gemini CLI. Use `--tool claude` to t
 | Skill | What it does |
 |-------|--------------|
 | **migrate** | Walks a legacy codebase through probe, enumerate, seam, extract, parity, queue, adjudicate and handoff: measured surface coverage, an empirically derived capability seam, cited requirements, a parity plan, a batch decision queue, and work items emitted to markdown, GitHub or a flow target with coverage and forecast read back. Bundles a Bun CLI that enforces the coverage arithmetic and phase ordering instead of trusting it. |
+| **scope-estimate** | The lighter path for a proposal: read-only discovery, a capability register with status and disposition, sized functional requirements, contracts-first work packages, and a build schedule across engineers x worktrees at fast, planning and floor rates. Bundles a Bun scheduler that refuses a plan missing or double-counting a requirement, so every figure comes from one reproducible run. |
 
 ### Code Architecture
 
@@ -77,6 +78,14 @@ Skills for reviewing high-level architecture, messaging patterns, and data flow.
 | **composability-auditor** | McIlroy, Pike, Thompson (Unix Philosophy) | Identifies reuse blockers (hidden I/O, non-standard interfaces, statefulness) and decomposes self-sufficient units into composable pieces. |
 | **temporal-coupling-detector** | Beck, Fowler (software engineering canon) | Detects hidden ordering dependencies (two-phase init, method order, invisible preconditions) and enforces them through types, parameters, and factory patterns. |
 
+### Verification
+
+Skills that check code does what it was meant to do, with evidence rather than opinion.
+
+| Skill | Based on | What it does |
+|-------|----------|--------------|
+| **bug-hunter** | QA scenario testing; intent-first reproduction | Proves logic bugs with failing repros checked against stated intent. Writes scenarios before reading the code, and reports bugs, ambiguities, and covered scenarios. |
+
 ### Communication
 
 Skills that control how the agent communicates: output compression, token efficiency, and response style.
@@ -112,7 +121,7 @@ Skills that compose the other audit skills into higher-level workflows.
 | Skill | What it does |
 |-------|--------------|
 | **improve-my-codebase** | Runs every applicable audit skill in parallel against your codebase, ranks findings by convergence (multiple audits agreeing) and per-file rollup, and writes a prioritized improvement report. Default sweeps everything; positional args switch modes (`quick`, `diff`, `interactive`) or narrow scope (`focus <area>`, `module <path>`). |
-| **magpie** | Interactive PR review pipeline. Runs five parallel specialist subagents (security, bugs, performance, code-smells, architecture), dedupes findings, applies a critic rubric, peer-reviews via `codex exec`, and serves an interactive HTML report for selecting findings to post via `gh`. |
+| **magpie** | Interactive PR review pipeline. Runs five parallel specialist subagents (security, bugs, performance, code-smells, architecture), dedupes findings, applies a critic rubric, peer-reviews via `codex exec`, and serves an interactive HTML report for selecting findings to post via `gh`. A local mode reviews the current branch diff before a PR exists and hands the findings back in the terminal. |
 
 ## Commands
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@iceinvein/agent-skills",
-  "version": "0.21.5",
+  "version": "0.23.0",
   "description": "Install agent skills into AI coding tools",
   "author": "iceinvein",
   "license": "MIT",
@@ -12,6 +12,7 @@
     "dist",
     "skills",
     "!skills/*/evals/results",
+    "!skills/*/evals/*/plugins",
     "!skills/terse/bench/results"
   ],
   "scripts": {
@@ -23,6 +24,8 @@
     "skill:bump:check": "bun run src/cli/index.ts bump --all --dry-run",
     "build:index": "bun run scripts/build-index.ts",
     "skill:audit": "bun run scripts/audit-skills.ts",
+    "eval:fresh": "bun run scripts/eval-freshness.ts",
+    "eval:plugins": "bun run scripts/eval-plugins.ts",
     "release": "bash scripts/release.sh"
   },
   "keywords": [

package/skills/bug-hunter/SKILL.md

@@ -0,0 +1,87 @@
+---
+name: bug-hunter
+description: Use when asked to hunt for bugs or edge cases in a change or area, to say what QA will find before QA does, or to prove a change correct. Runs QA-style scenarios as failing reproductions in the repo's own test runner and reports only bugs it can prove, plus the questions intent leaves open.
+---
+
+# Bug hunter
+
+Agent-written code passes its own tests and still fails when QA hits a
+scenario nobody wrote down. This skill runs the code and proves each bug with
+a failing reproduction, checked against what the change was meant to do.
+
+## Method
+
+1. **Name the target** in one line: a diff (a commit, a range, the working
+   changes) or a set of paths. Every later step is scoped to it.
+2. **Collect intent** into `intent.md`: numbered claims (`I1`, `I2`, ...),
+   each one checkable statement with its source. Sources are the request or
+   ticket, commit messages, specs and designs, docs, `CLAUDE.md`/`AGENTS.md`,
+   invariants stated in unchanged code, sibling implementations the target
+   says it mirrors, and existing tests on unchanged code. Read the target's
+   code as much as you need, but a claim never comes from what a changed body
+   does: that is the code confirming itself.
+3. **Hunt.** Write scenarios the way QA would into `scenarios.md` (`S1`,
+   `S2`, ..., each with a one-line label, the expected result and the claim
+   it rests on), working through `references/scenario-catalog.md`, adding
+   more as the code and failures suggest. For each one worth trying write a
+   repro in the repo's own test framework that asserts the expected result.
+   Find the runner from the repo's manifests and `CLAUDE.md`/`AGENTS.md`; do
+   not install a test dependency. Favour many cheap probes over a few
+   elaborate ones, and chase what a failure suggests.
+4. **Check each failure before reporting it.** Run its repro alone, clean,
+   with only its own files, and confirm it fails with the observed value.
+   Confirm the repro tests the target and not its own setup: a mock returning
+   what the real dependency never would, or a fixture breaking an invariant
+   the caller guarantees, is not a bug.
+5. **Sort** each result with the rules below, and write the output.
+
+Work in a scratch copy (a `git worktree`, or the copy the caller names). Never
+modify the caller's tree: no repro, install or build artefact lands there.
+
+## What counts as a bug
+
+No repro, no finding: a bug carries a reproduction that fails in a clean run
+and names the claim it violates.
+
+- **A concrete claim, contradicted,** is a bug citing that claim.
+- **Claims have a rank.** The request, ticket, spec or design outranks a
+  docstring or comment inside the target's changed code, which is the change
+  describing itself. A target that follows its own docstring against the spec
+  is a bug citing the spec. Only claims of the same rank that disagree make an
+  ambiguity.
+- **Some results are wrong under every reading,** and need no stated claim: a
+  crash, an error or a blank page from input the target accepts; a call that
+  reports success and did not do the work; an earlier action's result landing
+  over a later one's; a value shown in a form no reading allows, such as a
+  seconds field of 60; one user's data reaching or blocking another user. Add
+  the one you rely on to `intent.md` with source `baseline`.
+- **An ambiguity is a question with more than one acceptable answer,** where
+  the code picked one and no claim says which: where a cap sits, whether a
+  match ignores case. "No spec says so" is not enough; if every acceptable
+  answer makes the observed result wrong, it is a bug.
+- **Latent is not a bug.** A defect on a path no production caller reaches
+  today is noted in `report.md`, not filed in `bugs`.
+
+## Output
+
+Write to `.bug-hunter/<YYYY-MM-DD>-<slug>/` at the repo root: `intent.md`,
+`scenarios.md`, `report.md` for a human, and `findings.json` for a grader or calling skill,
+in the shape `references/findings-format.md` gives. Three buckets: **bugs**
+(scenario, violated claim, expected against observed, repro files and
+command), **ambiguities** (the question for product), **covered** (scenarios
+that passed, with the claim each checked). Write `findings.json` and
+`report.md` with the Write tool, never from a shell command or script.
+
+The `.bug-hunter/` directory is the one path in the caller's tree the skill
+writes. Promoting a repro into the tree as a regression test is the caller's
+job.
+
+The directory ignores itself, so no repo has to add a line for it: if
+`.bug-hunter/.gitignore` is absent, write it holding `*` before anything else
+in the run. Leave an existing one alone; deleting it is how a repo chooses to
+commit its runs.
+
+## Hand-back
+
+The run directory, the counts per bucket, then each bug in one line:
+scenario, violated claim, repro command. Name anything not done.

package/skills/bug-hunter/evals/README.md

@@ -0,0 +1,287 @@
+# bug-hunter evals
+
+Seven cases for `claude plugin eval`. Each scaffolds a small Node repo with a
+`SPEC.md`, one module and a passing `node --test` suite, committed in two steps
+(the spec, then the module) so the target reads as a change. The prompt asks
+what QA will find in the module, the way a user would, and never names the
+skill.
+
+| Case | Fixture | What it pins |
+|---|---|---|
+| `proves-a-spec-boundary-bug` | Usernames are 3 to 20 characters inclusive; the code rejects 20 | A bug in `findings.json` citing the limit, with a repro that fails against the fixture |
+| `silent-spec-is-an-ambiguity` | Quantities are "capped"; the spec never says where, the code uses 99 | The cap lands in `ambiguities` and `bugs` is empty |
+| `correct-code-has-no-bugs` | Pagination that meets every stated range at its edge | `bugs` is empty |
+| `callers-tree-unchanged` | A backoff whose first delay is doubled | Nothing changes in the caller's tree outside `.bug-hunter/`, and `.bug-hunter/.gitignore` is written holding `*` |
+| `bug-cites-intent-from-spec` | A coupon that floors where the spec rounds half up | `intent.md` is written, and the bug cites a claim sourced from `SPEC.md`, not from the code's floor |
+| `spec-outranks-target-docstring` | The spec keeps a sealed export on offer when its rewrite fails; the module's docstring and code drop it | A bug citing the spec claim, not an ambiguity between spec and docstring |
+| `impossible-value-is-a-bug` | The spec is silent on rounding; the code shows `60s` and `1m 60s` | A bug for the seconds field of 60, which no rounding rule gives |
+
+Every case also has `bug-hunter-fired`, a `tool_used: Skill` grader, so a
+score can be told apart from the model's own habits.
+
+## Running
+
+From the repo root. Pin the model, because the eval child does not inherit
+it, and pin the judge, because `--model` does not reach it:
+
+```bash
+claude plugin eval skills/bug-hunter --scaffold --allow-tools Bash Write Edit \
+  --model claude-opus-5-5 --judge-model claude-opus-5-5 --runs 1 --ablation none
+```
+
+Add `--keep-temp` to keep each run's sandbox and trace, which is the only way
+to read back why a grader failed. `-j 7` runs the seven cases at once.
+
+To check the skill against its last accepted baseline, use `eval:fresh`:
+
+```bash
+bun run eval:fresh run skills/bug-hunter --model claude-opus-5-5
+bun run eval:fresh diff skills/bug-hunter <result.json> [--against <model>]
+bun run eval:fresh accept skills/bug-hunter <result.json>
+```
+
+- `run` runs both arms on your Claude plan (default `--runs 3 -j 2`), then
+  diffs the result against `evals/baselines/<model>.json`.
+- `diff` is free: it re-reads an existing result and runs nothing.
+- `accept` writes the baseline. It is always a deliberate step, never part of
+  `run`.
+
+What each verdict means is in `skills/sluice/evals/README.md`.
+`callers-tree-unchanged` is listed in `vacuousWithout` in `rules.json`:
+`tree-unchanged` passes on a run that writes nothing, so the no-plugin arm
+scores on it without hunting, and the case is never called redundant or
+skill-worse.
+
+## What the graders can and cannot see
+
+The run directory is `.bug-hunter/<date>-<slug>/` and the slug is the skill's
+choice. Only `file_exists` resolves a glob; a `regex` or `llm` grader pointed at
+a file needs its exact path and throws on a glob. So:
+
+- `findings-written` and `intent-written` read the harness's `files` target,
+  one created path per line, and match
+  `.bug-hunter/<YYYY-MM-DD>-<slug>/findings.json` (or `intent.md`) there. A
+  `file_exists` glob would accept a directory with no date; this holds the
+  prefix to the shape SKILL.md states, though not to the day the run ran.
+- `written-with-write-tool` (in `silent-spec-is-an-ambiguity` and
+  `correct-code-has-no-bugs`) fails a Bash command that redirects or tees into
+  `intent.md`, `findings.json` or `report.md` under `.bug-hunter/`, which is
+  what holds the Write-tool rule: the content graders below read a heredoc's
+  JSON as readily as a Write call's. A script that opens the file itself
+  (`python3`, `node -e`) is not caught by it; the content graders fail that.
+- Every grader that reads the *content* of `findings.json` reads the trace
+  instead: the Write tool call, or the shell command, that created it. The
+  regex graders (`bugs-empty`, `cap-is-an-ambiguity`) need the JSON to appear
+  there as a literal, which holds for a Write call or a heredoc. **They cannot
+  see a `findings.json` that a script builds** (`python3 -c` with `json.dump`,
+  `node -e` with `JSON.stringify`): the trace then holds the script, not the
+  JSON, and the grader fails whatever the file says. The baseline hit this.
+  Both judge only the **last** write of `findings.json`: a closing lookahead
+  fails the match if any later write of it follows (a Write, an Edit, or a
+  shell redirect). Before 0.2.1 they passed on any write, so an empty first
+  draft rewritten with bugs passed `bugs-empty`. A later Edit fails them,
+  since an Edit's trace entry does not carry the whole file.
+- `bug-cites-the-limit` is an `llm` judge over the trace, so it can read a
+  script that builds the file as well as a literal. It sees a head-and-tail
+  window of a long trace (about 80k characters of head, 20k of tail), so a
+  findings write that lands in the dropped middle is invisible to it. It cannot
+  run the repro; "fails in the fixture" is judged against the fixture's code,
+  which the rubric quotes. Its rubric also says to judge only the last write.
+- `tree-unchanged` cannot run `git status --porcelain`. It reads the harness's
+  `files` target, the list of paths the run created in its working directory,
+  and fails on any path outside `.bug-hunter/` or `.git/`. A file `.gitignore`
+  hides still counts. The `files` target never lists a file the run only
+  modified, so on its own it passes an edit to the scaffold's files.
+  `source-unchanged` fails if the buggy line in `src/backoff.js` was
+  rewritten, and `tests-unchanged` if `tests/backoff.test.js` gained a test.
+  An edit to `package.json` or `SPEC.md` is still unseen. A run that writes
+  nothing passes all three, which is why the case also has
+  `bug-hunter-fired`.
+- `output-ignores-itself` in the same case reads the trace for a Write of
+  `.bug-hunter/.gitignore` whose content is `*`, or a shell command that
+  writes a `*` into it. The `files` target would show the path but not its
+  content, and an empty `.gitignore` ignores nothing. A script that opens the
+  file itself is not caught.
+- `bug-cites-the-spec` in `bug-cites-intent-from-spec` is an `llm` judge over
+  the trace, with the same window and last-write limits as
+  `bug-cites-the-limit`. It resolves the bug's `claim` through the `intent`
+  array and fails a claim sourced from `src/discount.js`. It cannot tell a
+  claim the skill read out of the body but labelled `SPEC.md`; the spec's
+  wording ("half cent rounding up") is what makes such a mislabel visible to
+  it. This case replaced `scenarios-before-bodies` in 0.4.0, which pinned the
+  scenario-before-body order that 0.4.0 dropped.
+
+## Baseline
+
+### Skill 0.4.0, Opus 5.5, 2026-10-13
+
+Same command, `-j 7`. Skill 0.4.0 drops the three dispatched roles for one
+agent that may read the code freely, keeping intent collection, the scenario
+catalog, a clean re-run of each failure and the 0.3.0 rules, and reporting
+latent defects without filing them. It is the `variants/thin` arm of the
+pilot benchmark, which proved or judged 3 of 12 answer keys against 2 for the
+plain baseline and 2 for 0.3.0, with every extra finding triaged real, at USD
+1.83 a run against 3.19 for 0.3.0. USD 3.74 in all, 102s,
+`results/2026-10-13T03-48-17-313Z`. Overall 1.00. One run per case.
+
+| Case | 0.3.0 | 0.4.0 | USD (0.4.0) |
+|---|---|---|---|
+| `proves-a-spec-boundary-bug` | 1.00 | 1.00 | 0.77 |
+| `silent-spec-is-an-ambiguity` | 1.00 | 1.00 | 0.44 |
+| `correct-code-has-no-bugs` | 1.00 | 1.00 | 0.34 |
+| `callers-tree-unchanged` | 1.00 | 1.00 | 0.43 |
+| `scenarios-before-bodies` | 1.00 | 1.00 | 0.49 |
+| `spec-outranks-target-docstring` | 1.00 | 1.00 | 0.58 |
+| `impossible-value-is-a-bug` | 1.00 | 1.00 | 0.70 |
+
+`scenarios-before-bodies` passed, but 0.4.0 no longer asks for that order: the
+run happened to write `scenarios.md` before reading `src/discount.js`. The case
+pinned a rule the skill dropped, so it was rewritten as
+`bug-cites-intent-from-spec` on the same fixture.
+
+### Skill 0.3.0, Opus 5.5, 2026-10-12
+
+Same command, `-j 7`. Skill 0.3.0 ranks claims (a spec, design or request
+outranks a docstring in the changed code), lets a result wrong under every
+reading count as a bug against a `baseline` claim, and keeps ambiguities for
+questions with more than one acceptable answer. It answers the pilot
+benchmark, where 0.2.1 filed two real defects it had reproduced as
+ambiguities. USD 6.55 in all, 228s, `results/2026-10-12T14-46-50-101Z`.
+Overall 1.00. One run per case.
+
+The two new cases also scored 1.00 on 0.2.1 (USD 0.91 and 1.07,
+`results/2026-10-12T14-43-31-*`): at fixture size the conflict is plain enough
+that the old rules got it right, so neither reproduces the pilot's miss. They
+stay as guards against the opposite failure. The cases that pin over-reporting,
+`silent-spec-is-an-ambiguity` and `correct-code-has-no-bugs`, still hold on
+0.3.0: the cap stays a question and the correct module has no bugs.
+
+| Case | 0.2.1 | 0.3.0 | USD (0.3.0) |
+|---|---|---|---|
+| `proves-a-spec-boundary-bug` | 1.00 | 1.00 | 1.29 |
+| `silent-spec-is-an-ambiguity` | 1.00 | 1.00 | 1.07 |
+| `correct-code-has-no-bugs` | 1.00 | 1.00 | 0.69 |
+| `callers-tree-unchanged` | 1.00 | 1.00 | 0.62 |
+| `scenarios-before-bodies` | 1.00 | 1.00 | 0.65 |
+| `spec-outranks-target-docstring` | 1.00 | 1.00 | 1.35 |
+| `impossible-value-is-a-bug` | 1.00 | 1.00 | 0.88 |
+
+### Skill 0.2.1, Opus 5.5, 2026-10-12
+
+Same command as 0.2.0 below. Skill 0.2.1 writes each findings `scenario` as
+`"<id>: <label>"`, lets the single-agent fallback add a late claim from a
+source other than the changed body, and the `findings.json` graders now judge
+only the last write. USD 5.39 in all, 361s,
+`results/2026-10-12T09-23-52-868Z`. Overall 1.00. One run per case.
+
+| Case | 0.2.0 | 0.2.1 | USD (0.2.1) | Reading |
+|---|---|---|---|---|
+| `proves-a-spec-boundary-bug` | 1.00 | 1.00 | 1.01 | Judge PASS PASS PASS. `B1` is `"S2: a 20-character name is accepted"`, citing I1 |
+| `silent-spec-is-an-ambiguity` | 1.00 | 1.00 | 1.04 | One `findings.json` Write, `bugs` empty; the cap question is `"S13: Quantity exactly at the cap, one below and one above"` |
+| `correct-code-has-no-bugs` | 1.00 | 1.00 | 1.03 | One Write, `bugs` empty, 16 covered, two ambiguities (numeric strings, `Infinity`) |
+| `callers-tree-unchanged` | 1.00 | 1.00 | 0.70 | No path outside `.bug-hunter/` changed |
+| `scenarios-before-bodies` | 1.00 | 1.00 | 1.61 | Scenario author dispatched; `scenarios.md` written before any Read of `src/discount.js` |
+
+Every `scenario` in every bucket of all five runs carried its label. Every
+run wrote `findings.json` once, so the last-write rule changed no verdict
+here; it was checked on doctored copies of the 0.2.0 traces (an empty draft
+followed by a write with bugs, a later Edit, a later heredoc), where the old
+patterns passed and the new ones fail. Every run dispatched at least one role
+and wrote `intent.md` once, so the fallback's late-claim rule was not
+exercised; no fixture has a sibling implementation or a stated invariant.
+
+### Skill 0.2.0, Opus 5.5, 2026-10-12
+
+Same command plus `--trust-plugin --no-publish`, CLI 2.1.296. Skill 0.2.0
+counts intent stated in the repo itself (invariants the code states, sibling
+implementations it says it mirrors, existing tests, docs) and requires
+`findings.json` and `report.md` to be written with the Write tool. USD 5.08 in
+all (judging USD 0.40), 265s, `results/2026-10-12T07-49-17-746Z`. Overall
+1.00, against 0.84 for 0.1.0 below. One run per case, so every score is a
+single sample.
+
+| Case | 0.1.0 | 0.2.0 | USD (0.2.0) | Reading |
+|---|---|---|---|---|
+| `proves-a-spec-boundary-bug` | 1.00 | 1.00 | 1.33 | Judge PASS PASS PASS |
+| `silent-spec-is-an-ambiguity` | 0.20 | 1.00 | 1.12 | `findings.json` written with Write, so both regexes see it. `bugs` empty; `A1` asks what the cap is and names the code's 99 |
+| `correct-code-has-no-bugs` | 1.00 | 1.00 | 0.38 | `bugs` empty, 13 covered, one ambiguity (sparse arrays, which `SPEC.md` does not address) |
+| `callers-tree-unchanged` | 1.00 | 1.00 | 1.17 | No path outside `.bug-hunter/` changed |
+| `scenarios-before-bodies` | 1.00 | 1.00 | 1.08 | The dispatched scenario author wrote `scenarios.md` before any Read of `src/discount.js` |
+
+No failures. The 0.1.0 failure is closed by the Write rule, not by a grader
+change: all five runs wrote both files with the Write tool. The new intent
+rule did not promote anything in the two cases it could have over-promoted:
+the ambiguity run's five questions all stay ambiguities with `bugs` empty, and
+the correct-code run's one open question (sparse arrays) stays an ambiguity.
+None of the five fixtures has a sibling implementation or a stated invariant,
+so these cases do not exercise the new confirm path; they only show it does
+not leak into silent-spec cases.
+
+Dispatch is still not reliable. The correct-code run ran all three roles in
+the main thread with the Agent tool available (70s, the cheapest run); its
+`intent.md` and `scenarios.md` were on disk before it ran `cat` on
+`src/paginate.js`, so the fallback's order held. It hunted in a `$TMPDIR` copy
+rather than a git worktree. The ambiguity run dispatched the scenario author
+and executor and no refuter, which fits: no repro failed.
+
+### Skill 0.1.0, Opus 5.5, 2026-10-12
+
+CLI 2.1.296, `--model claude-opus-5-5 --judge-model claude-opus-5-5 --runs 1
+--ablation none --keep-temp -j 5`. One run per case, so every score is a single
+sample. USD 5.30 in all (judging USD 0.39), 317s,
+`results/2026-10-12T05-33-00-761Z`. Overall 0.84, four of five cases at 1.00.
+
+| Case | Score | USD | Reading |
+|---|---|---|---|
+| `proves-a-spec-boundary-bug` | 1.00 | 0.77 | Judge PASS PASS PASS. One bug citing I1 (3 to 20 inclusive) with a repro asserting a 20-character name is accepted |
+| `silent-spec-is-an-ambiguity` | 0.20 | 0.97 | `bugs-empty` and `cap-is-an-ambiguity` fail on the grader's blind spot, not the skill (below) |
+| `correct-code-has-no-bugs` | 1.00 | 0.98 | `bugs` empty; findings written with the Write tool |
+| `callers-tree-unchanged` | 1.00 | 1.19 | No path outside `.bug-hunter/` changed; a look at the kept sandbox agrees |
+| `scenarios-before-bodies` | 1.00 | 1.39 | The scenario author (a dispatched agent) wrote `scenarios.md`; the first Read of `src/discount.js` came later |
+
+**`silent-spec-is-an-ambiguity` failed, and the skill did what the case
+asks.** The run wrote `findings.json` with `python3 -I -` and `json.dump`, so
+the trace holds a Python list, not the JSON the two regexes look for; both
+report "pattern not found in trace". The file kept in the sandbox has
+`"bugs": []` and, first among five ambiguities, `A1`: "What is the cap?
+SPEC.md never gives a number; the code uses 99 (MAX_QUANTITY). Is 99 the
+intended value, and should the spec say so?" The final message says the same.
+The graders stay as written. The fix is not to widen them: either the skill
+writes `findings.json` with the Write tool, which `findings-format.md` does not
+ask for today, or the harness lets a file target take a glob. Which one is
+open.
+
+**Other things the traces show, which no grader scores.**
+
+- Dispatch is not reliable. Four of the five baseline runs (ambiguity,
+  correct, tree, order) dispatched the scenario author, executor and refuter
+  as separate agents. The boundary-bug run, and the sanity run of the same
+  case before it, ran all three roles in the main thread with the Agent tool
+  available, which the skill reserves for when dispatch is unavailable. In
+  both, `intent.md` and `scenarios.md` were on disk before the body was read,
+  so the fallback's order held.
+- Git inside the eval sandbox: a bare `git` is Apple's xcrun shim, which fails
+  there (as in `skills/sluice/evals/README.md`), and so does the checkout step
+  of `git worktree add` even from `/opt/homebrew/bin/git`. The tree run worked
+  around it with `worktree add --no-checkout` then `reset --hard`, so it hunted
+  in a real scratch worktree and removed it after; its own closing
+  `git status --short` showed only `?? .bug-hunter/`. The order run fell back
+  to `git archive` into `$TMPDIR`, and the boundary-bug, ambiguity and correct
+  runs copied the files into `$TMPDIR` without git. The caller's tree stayed
+  clean either way, but the skill's scratch worktree held the repros in one
+  run of five.
+- Before writing scenarios, the order run's main thread ran Grep with content
+  output over `src/discount.js` for `export` and comment lines. That shows
+  declaration lines, which the skill allows, but the same call shape could
+  show a body, and the order grader does not count it.
+
+**Sanity run before the baseline.** `proves-a-spec-boundary-bug` alone (USD
+0.79, `results/2026-10-12T05-30-48-463Z`): 1.00. Its trace was then used to
+check that each regex grader can fail. `scenarios-before-bodies`, pointed at
+`src/username.js`, matched the real trace and did not match a copy with the
+body's Read moved ahead of the `scenarios.md` write. `bugs-empty` and
+`cap-is-an-ambiguity` did not match the real findings write (one bug, no
+ambiguities) and did match a copy with `bugs` emptied and a cap question added.
+`tree-unchanged` passed on a listing of only `.bug-hunter/` paths and failed
+once `tests/backoff-hunt.test.js` was added. The `llm` grader's FAIL path was
+not exercised: both runs of its case found the bug.

package/skills/bug-hunter/evals/baselines/claude-opus-5-5.json

@@ -0,0 +1,93 @@
+{
+  "model": "claude-opus-5-5",
+  "judgeModel": "claude-opus-5-5",
+  "claudeVersion": "2.1.300",
+  "skillVersion": "0.4.0",
+  "date": "2026-10-15T13:04:21.155Z",
+  "cases": {
+    "bug-cites-intent-from-spec": {
+      "with": [
+        1,
+        1,
+        1
+      ],
+      "without": [
+        0,
+        0,
+        0
+      ]
+    },
+    "callers-tree-unchanged": {
+      "with": [
+        1,
+        1,
+        1
+      ],
+      "without": [
+        1,
+        1,
+        1
+      ]
+    },
+    "correct-code-has-no-bugs": {
+      "with": [
+        1,
+        1,
+        1
+      ],
+      "without": [
+        0.5,
+        0.5,
+        0.5
+      ]
+    },
+    "impossible-value-is-a-bug": {
+      "with": [
+        1,
+        1,
+        1
+      ],
+      "without": [
+        0,
+        0,
+        0
+      ]
+    },
+    "proves-a-spec-boundary-bug": {
+      "with": [
+        1,
+        1,
+        1
+      ],
+      "without": [
+        0,
+        0,
+        0
+      ]
+    },
+    "silent-spec-is-an-ambiguity": {
+      "with": [
+        1,
+        1,
+        1
+      ],
+      "without": [
+        0.3333333333333333,
+        0.3333333333333333,
+        0.3333333333333333
+      ]
+    },
+    "spec-outranks-target-docstring": {
+      "with": [
+        1,
+        1,
+        1
+      ],
+      "without": [
+        0,
+        0,
+        0
+      ]
+    }
+  }
+}

package/skills/bug-hunter/evals/bug-cites-intent-from-spec/case.yaml

@@ -0,0 +1,4 @@
+schema_version: "1.1"
+name: bug-cites-intent-from-spec
+context:
+  scaffold_script: fixture.sh

package/skills/bug-hunter/evals/bug-cites-intent-from-spec/fixture.sh

@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+# A coupon helper whose only bug is in its rounding (the spec says halves
+# round up, the code floors). The case is about where the expected value comes
+# from: the bug has to cite a claim intent.md took from SPEC.md, not one read
+# out of the floor in src/discount.js, which would confirm the code against
+# itself.
+set -euo pipefail
+
+mkdir -p src tests
+
+cat > package.json <<'JSON'
+{
+  "name": "checkout",
+  "version": "3.1.0",
+  "type": "module",
+  "scripts": {
+    "test": "node --test \"tests/*.test.js\""
+  }
+}
+JSON
+
+cat > SPEC.md <<'MD'
+# Checkout coupons
+
+`applyCoupon(totalCents, coupon)` in `src/discount.js` returns the order total
+after a percentage coupon, in cents.
+
+- `totalCents` is a whole number of 0 or more; anything else throws a
+  `RangeError`.
+- A coupon is `{ percentOff }`, where `percentOff` is a whole number from 1 to
+  100, inclusive; anything else throws a `RangeError`.
+- The result is the total less `percentOff` percent, rounded to the nearest
+  cent, with a half cent rounding up.
+MD
+
+git init --quiet
+git add -A
+git -c user.email=fixture@example.com -c user.name=fixture commit --quiet -m "spec: checkout coupons"
+
+cat > src/discount.js <<'JS'
+export function applyCoupon(totalCents, coupon) {
+  if (!Number.isInteger(totalCents) || totalCents < 0) {
+    throw new RangeError(`totalCents must be a whole number of 0 or more, got ${totalCents}`);
+  }
+
+  const percentOff = coupon?.percentOff;
+
+  if (!Number.isInteger(percentOff) || percentOff < 1 || percentOff > 100) {
+    throw new RangeError(`percentOff must be a whole number from 1 to 100, got ${percentOff}`);
+  }
+
+  return Math.floor((totalCents * (100 - percentOff)) / 100);
+}
+JS
+
+cat > tests/discount.test.js <<'JS'
+import assert from "node:assert/strict";
+import { test } from "node:test";
+import { applyCoupon } from "../src/discount.js";
+
+test("ten percent off a round total", () => {
+  assert.equal(applyCoupon(10000, { percentOff: 10 }), 9000);
+});
+
+test("a coupon over 100 percent throws", () => {
+  assert.throws(() => applyCoupon(10000, { percentOff: 101 }), RangeError);
+});
+JS
+
+git add -A
+git -c user.email=fixture@example.com -c user.name=fixture commit --quiet -m "feat: apply percentage coupons at checkout"

package/skills/bug-hunter/evals/bug-cites-intent-from-spec/graders/bug-cites-the-spec.md

@@ -0,0 +1,36 @@
+---
+type: llm
+# As in proves-a-spec-boundary-bug: the judge cannot run the repro, so "fails
+# in the fixture" is judged against the fixture's code, quoted below, and it
+# reads the trace because findings.json sits under a slug no file focus names.
+focus: trace
+weight: 3
+---
+
+The agent was asked what QA will find in `src/discount.js`, going by
+`SPEC.md`. `SPEC.md` says the result is the total less `percentOff` percent,
+"rounded to the nearest cent, with a half cent rounding up". The fixture's
+code floors instead:
+
+```js
+return Math.floor((totalCents * (100 - percentOff)) / 100);
+```
+
+So `applyCoupon(1005, { percentOff: 10 })` returns `904` (904.5 floored)
+where the spec requires `905`, and any result with a fractional part of a
+half cent or more is a cent short. Nothing else in the code departs from the
+spec.
+
+Find the `findings.json` the agent wrote under `.bug-hunter/` (its content
+appears in a Write tool call or a shell command in the trace). If it was
+written more than once, judge only the last write, with any later Edit to it
+applied.
+
+PASS if its `bugs` array has an entry whose `claim` resolves, through the
+`intent` array, to a claim whose `source` is `SPEC.md` and which states the
+round-to-nearest, half-up rule, and whose `repro` asserts a half-up rounded
+result that the flooring code above does not give, so that it fails.
+FAIL if no findings.json content is visible in the trace, if `bugs` is empty,
+if the bug's claim resolves to an intent entry whose source is
+`src/discount.js` or that describes the floor as intended, or if the repro
+asserts something the code above already does.

package/skills/bug-hunter/evals/bug-cites-intent-from-spec/graders/bug-hunter-fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:[\w-]+:)?bug-hunter"'
+---

package/skills/bug-hunter/evals/bug-cites-intent-from-spec/graders/intent-written.md

@@ -0,0 +1,8 @@
+---
+type: regex
+# As findings-written in proves-a-spec-boundary-bug: the files target holds
+# the run directory's date prefix to the shape SKILL.md states.
+pattern: '^\.bug-hunter/\d{4}-\d{2}-\d{2}-[^/]+/intent\.md$'
+flags: m
+target: files
+---

package/skills/bug-hunter/evals/bug-cites-intent-from-spec/prompt.md

@@ -0,0 +1,11 @@
+---
+name: bug-cites-intent-from-spec
+description: The skill may read the code, but a claim never comes from what a changed body does. Pins the rounding bug citing an intent claim sourced from SPEC.md, with a repro expecting half-up rounding.
+tags: [intent, arithmetic, scaffold, write]
+max_turns: 80
+timeout_seconds: 2400
+allowed_tools: [Read, Glob, Grep, Skill, Write, Edit, Bash, Agent]
+expected_outcome: intent.md exists, and findings.json under .bug-hunter/ has a bug whose claim resolves to SPEC.md's half-cent-rounds-up rule, with a repro that fails against the flooring code.
+---
+
+What will QA find in `src/discount.js`, going by `SPEC.md`?