@iceinvein/agent-skills 0.23.9 → 0.23.13

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@iceinvein/agent-skills",
-  "version": "0.23.9",
+  "version": "0.23.13",
   "description": "Install agent skills into AI coding tools",
   "author": "iceinvein",
   "license": "MIT",

package/skills/index.json

@@ -227,7 +227,7 @@
     "name": "magpie",
     "description": "Interactive PR review pipeline. Runs five parallel specialist subagents (security, bugs, performance, code-smells, architecture) plus a once-per-PR checks pass that finds tests and controls able to pass without proving their purpose, drops findings whose quoted evidence is not in the code, dedupes, runs critic subagents that re-check each finding against the worktree and set its risk label, peer-reviews via codex exec (falling back to a Claude second opinion when codex is unavailable), and serves an interactive HTML report for selecting or dismissing findings to post via gh, recording the outcome per finding as labels. Bundles a Bun CLI put on PATH by the skill's postinstall step under a bunx install, or by the plugin's bin/ under a Claude Code plugin install. Use when the user asks to review a GitHub pull request. Local mode (setup --local) runs the same pipeline on the current branch's diff from its merge base, uncommitted and untracked work included, against the caller's live tree, with no server or posting, and hands the ranked findings back in the terminal for an implementing or orchestrating agent to address. Splits oversized diffs into budgeted shards and rebuilds the diff from the local clone when gh pr diff refuses it.",
     "type": "prompt",
-    "version": "0.15.0"
+    "version": "0.16.1"
   },
   {
     "name": "migrate",
@@ -295,7 +295,7 @@
     "name": "sluice",
     "description": "Routes work by change shape into four channels (bypass, fast, main, deep) and applies only the rules each channel needs, so a one-line fix does not pay the cost of a multi-subsystem build. Carries nine rules as one-liners in the router and the full treatment in references read only on friction. Deep runs read the repo before designing, map the spec's intent to its decisions, and check spec against intent and plan against spec with a fresh reader, with plan.sh validate enforcing that every decision is covered. Checks the finished plan with plan.sh validate rather than trusting it to memory, seeds the run state from it, keeps a deep run's task breakdown in .sluice/run.json so a statusline segment, one status command and a SessionStart hook can answer where the run is (the hook prints a live run at every session start, compaction included), lists every live deep run and routed fast or main session on the machine in one board.sh view (drawn once, watched, or as JSON), and closes each run with a ledger read out of the session transcript: elapsed, tools, tokens, and what each dispatched agent cost where the transcript recorded it. Two Stop hooks hold a handback: one refuses a tree that moved since its last recorded check, the other refuses a changed existing test not proven to fail on the session's starting code (or recorded with a reason). Claude Code only; stands down where the superpowers pipeline governs the repo.",
     "type": "prompt",
-    "version": "0.28.0"
+    "version": "0.28.2"
   },
   {
     "name": "temporal-coupling-detector",

package/skills/magpie/SKILL.md

@@ -16,7 +16,7 @@ Stop reading and follow these steps in order. Do not skip stages. Use the exact
 
 ### 0. Identify the PR
 
-A bare invocation (no arguments and no PR named), a request with no PR that asks to review the current branch, the working tree, or work in progress, or arguments starting `local` or `--` (any `--base` or `--intent-file` go to setup), is local mode: follow "Local mode" below in place of this step and stages 1 and 2, then run stages 3 to 7 as written, taking `TARGET` from Local mode.
+A bare invocation (no arguments and no PR named), a request with no PR that asks to review the current branch, the working tree, or work in progress, or arguments starting `local` or `--` (any `--base` or `--intent-file` go to setup), is local mode: follow "Local mode" below in place of this step and stages 1 and 2, then run stages 3 to 7b as written, taking `TARGET` from Local mode.
 
 Parse the user's request for a PR number, URL, or "this PR" (current branch). If ambiguous, ask one clarifying terminal question. Capture the PR number into `$PR_NUMBER` and the repo path into `$REPO` (default: the current working directory).
 
@@ -54,14 +54,12 @@ Setup then splits the filtered diff into shards, writing `$RUN_DIR/shards/manife
 and, when more than one shard results, `$RUN_DIR/shards/shard-<n>.patch`. `diff.patch`
 itself is never modified: shards are views over it. Re-split with a different budget
 using `magpie shard "$RUN_DIR" --budget <lines> --max-files <n>` (defaults: 6000 patch
-lines, 80 files). Re-splitting invalidates every existing
-`findings/<focus>.shard-<n>.json`, since a shard id then names a different file set:
-delete those files first, or stage 4's resume rule counts a pair as covered that
-nothing reviewed.
+lines, 80 files). Delete every `findings/<focus>.shard-<n>.json` before re-splitting:
+shard ids then name different files, and stage 4's resume rule would trust them.
 
-When a prior run exists for the same PR (active or archived under `~/.magpie/`), setup writes `$RUN_DIR/incremental.json` with `{previousRunId, previousSha, currentSha, sameSha}`. The post stage surfaces this as a "Incremental review since `<sha>`" trailer on the summary comment.
+Setup writes `pr-activity.json` (checks, review threads, comments) and, on a re-review, `incremental.json` and `previous-findings.json` (last run's posted findings, whether their quoted code changed, and its dismissals).
 
-Setup also runs a deterministic test-coverage check: when the diff contains zero test or spec files anywhere, each non-test source file with `>= 10` added code lines gets a `domain: "tests"` finding in `$RUN_DIR/findings/tests.json`. That domain flows through dedupe/critic/peer-review like the LLM findings; no subagent is dispatched for it.
+Setup also runs a deterministic test-coverage check: when the diff contains zero test or spec files anywhere, each non-test source file with `>= 10` added code lines gets a `domain: "tests"` finding in `$RUN_DIR/findings/tests.json`, reviewed downstream like the rest; no subagent is dispatched for it.
 
 ### 2. Serve
 
@@ -71,9 +69,9 @@ Start the HTML server in the background using the Bash tool with `run_in_backgro
 magpie serve "$RUN_DIR"
 ```
 
-Read `$RUN_DIR/state/server-info` for the URL; the server writes it asynchronously, so if it is missing, wait a moment and re-read (it appears within ~1s). Print to the user: "Open <url> in your browser to follow along."
+Read `$RUN_DIR/state/server-info` for the URL (written within ~1s; re-read if missing). Print to the user: "Open <url> in your browser to follow along."
 
-The server shuts down after 30 idle minutes (an open report tab keeps it up) and deletes `state/server-info`. Nothing in the pipeline depends on it: re-run `magpie serve "$RUN_DIR"` to bring the report back.
+The server stops after 30 idle minutes (an open report tab keeps it up); nothing depends on it, so re-run `magpie serve "$RUN_DIR"` to bring the report back.
 
 Render the first progress paint:
 
@@ -96,9 +94,11 @@ Append `{stage: context, status: done}` and re-render progress. If the scout wro
 Read `references/specialists.md` now, before dispatching anything: it holds the five focus blocks and the output contract every specialist prompt is built from. Assemble prompts from that file verbatim; written from memory they drift off the JSON contract, and `magpie dedupe` drops findings it cannot parse.
 
 Append `{stage: specialists, status: running}` to `$RUN_DIR/log.jsonl` and re-render
-progress, so the served page shows the stage as active rather than "Paused". Then read
+progress. Then read
 `$RUN_DIR/shards/manifest.json`.
 
+Run `magpie history "$RUN_DIR"` once; any output is part 6 of every specialist prompt.
+
 **One shard, zero shards, or no manifest** (a diff filtered down to nothing, e.g. a
 lockfile-only PR, yields `shards: []` in an otherwise normal manifest; a run predating
 this feature has no manifest at all): dispatch the five specialists in a single message,
@@ -118,14 +118,11 @@ many small files is split by the 80-file cap, so a larger `--budget` alone chang
 nothing); or review only the highest-risk shards, which means appending
 `{stage: shard-coverage, status: partial, reviewed: [<ids>], skipped: [<ids>]}` to
 `$RUN_DIR/log.jsonl` and telling the user in the terminal which shards go unreviewed.
-That log entry is the only record of the gap: the report has no unreviewed marker.
-Wait for the answer. This is the only interactive gate in the pipeline before the
-report, and it exists so that neither the cost nor a coverage gap is ever silent.
+The report, summary comment and handoff name the skipped shards from that entry.
+Wait for the answer; neither cost nor a coverage gap is ever silent.
 
 Dispatch by wave, one shard per wave, the five focuses in parallel within a wave,
-re-rendering progress between waves. That holds in-flight agents at five and makes a
-crash cheap to resume: only the `(focus, shard)` pairs whose findings file is missing
-need re-dispatching.
+re-rendering progress between waves. That holds in-flight agents at five.
 
 After each subagent returns, append
 `{stage: specialist, focus: <focus>, shard: <n>, status: done, findings: <count>}` to
@@ -204,13 +201,17 @@ If codex returns non-zero on a batch, do not abort: record `{stage: peer-review,
 
 **Apply the verdicts (both paths).** Parse the merged verdicts and apply the `update` / `add` entries (an empty array means no change). A peer `fields.severity` (or `finding.severity` on an `add`) is ignored: severity is derived from `risk.impact`. Mint each `add`'s `id` as above before merging, since the peer contract does not carry ids. Then write `findings.final.json`. Re-render progress.
 
+### 7b. Proof
+
+Append `{stage: proof, status: running}` and re-render progress. `magpie proof-prompt "$RUN_DIR"` fills `references/proof.md` for up to 8 must-fix or should-fix findings, printing `<prompt path>\t<output path>` each. On a PR run with any printed, first install `$WORKTREE`'s dependencies once with the repo's frozen-lockfile command (`npm ci`, `uv sync --frozen`, ...); on failure log `{stage: proof-deps, status: skipped, error}` and go on. Dispatch the prompts as stage 6 does its batches, then `magpie proof-apply "$RUN_DIR"`: it drops disproved findings to `proof-dropped.json`, marks the rest and logs `done`. On a non-zero exit, delete only the `proof-<id>.json` files or leftovers stderr names, then re-dispatch or re-apply.
+
 ### 8. Report
 
 ```
 magpie render "$RUN_DIR" findings
 ```
 
-The report shows the top 10 recommended (`must-fix`/`should-fix`, highest score first) and folds the rest. `--top <n>` is not saved (the stage 9 re-render and `magpie serve`'s auto-refresh show 10 again), so use it only when nothing will re-render.
+The report shows the top 10 recommended (`must-fix`/`should-fix`, highest score first) and folds the rest. `--top <n>` is not saved, so use it only when nothing will re-render.
 
 Append `{stage: report, status: done}` to `$RUN_DIR/log.jsonl` and re-render progress (the render CLI does not log this itself, and `magpie status` needs the `done` entry to resume past `report`).
 
@@ -294,9 +295,9 @@ The JSON output tells you `lastCompleted` and `next`. Resume from `next`:
 - Local run at `next: cleanup`: if the caller has not been sent the findings, re-run `magpie handoff` first (it only reads files).
 - If a specialist focus has no findings file but its sibling stages are done,
   re-dispatch only that focus. On a sharded run the unit is the `(focus, shard)` pair:
-  read `shards/manifest.json`, and re-dispatch only the pairs with no
-  `findings/<focus>.shard-<n>.json`. Never re-shard mid-run without deleting those
-  files first (stage 1): the check would otherwise trust ids that moved.
+  re-dispatch only the pairs with no `findings/<focus>.shard-<n>.json`, skipping
+  shards a `shard-coverage: partial` entry skipped. Never re-shard mid-run without
+  deleting those files first (stage 1): shard ids move.
 - Non-null `error` means the run stopped on a failed stage. Report which stage to the user and confirm before re-running it.
 
 The original server is gone. Restart it with `magpie serve "$RUN_DIR"` (step 2) before re-rendering, so the user gets a live URL again.

package/skills/magpie/bin/magpie.ts

@@ -13,6 +13,8 @@ Subcommands:
                              (default origin/HEAD), committed and uncommitted
   target <run-dir>           Print the directory findings are anchored against
   handoff <run-dir>          Print local review findings as a ranked list
+  history <run-dir>          Print what is already on the PR (open threads, comments,
+                             dismissals, failing checks) for the specialist prompts
   serve <run-dir-or-id>      Start the HTML server (accepts active or archived run id)
   dedupe <run-dir>           Merge specialist findings into deduped set
   critic-prompt <run-dir> [--batch-size N]
@@ -20,13 +22,20 @@ Subcommands:
   critic-apply <run-dir> [--design-cap N]
                              Apply critic verdicts, write findings.kept.json (no cap on
                              code-smells + architecture keeps unless --design-cap is given)
+  proof-prompt <run-dir>     Write one proof prompt per must-fix or should-fix finding, print
+                             prompt and output paths
+  proof-apply <run-dir>      Fold proof results into findings.final.json, drop the disproved
   shard <run-dir> [--budget N] [--max-files N]
-                             Re-split diff.patch into budgeted shards
+                             Re-split diff.patch into budgeted shards (and the
+                             impact map's per-shard files, when impact.json exists)
+  impact <run-dir>           Write impact.json and impact*.md: what outside the diff
+                             the change touches
   render <run-dir> <page> [--top N]
                              Render progress.html or findings.html (findings recommends
                              the top N, default 10, and folds the rest)
   labels <run-dir>           Write labels.json (posted, dismissed, ignored per finding)
-  cleanup <run-dir>          Remove worktree, stop server, write labels, archive run
+  cleanup <run-dir>          Remove worktree, stop server, write labels, archive run,
+                             delete saved refs of closed PRs
   status <run-dir>           Print highest completed stage
   open [id]                  Open findings.html in your browser (defaults to latest run)
   post <run-dir> --ids a,b   Post the given finding ids via gh (rich body + optional summary)
@@ -244,6 +253,24 @@ const HANDLERS: Record<string, Handler> = {
     const { runCriticApply } = await import('../scripts/critic-cmd.ts')
     return runCriticApply(runDir, designCap !== undefined ? { designCap } : {})
   },
+  'proof-prompt': async (args) => {
+    const runDir = args[0]
+    if (!runDir) {
+      process.stderr.write('proof-prompt: missing <run-dir>\n')
+      return 2
+    }
+    const { runProofPrompt } = await import('../scripts/proof-cmd.ts')
+    return runProofPrompt(runDir)
+  },
+  'proof-apply': async (args) => {
+    const runDir = args[0]
+    if (!runDir) {
+      process.stderr.write('proof-apply: missing <run-dir>\n')
+      return 2
+    }
+    const { runProofApply } = await import('../scripts/proof-cmd.ts')
+    return runProofApply(runDir)
+  },
   shard: async (args) => {
     const runDir = args[0]
     if (!runDir) {
@@ -283,8 +310,25 @@ const HANDLERS: Record<string, Handler> = {
     process.stdout.write(
       `${manifest.shards.length} shard(s), ${manifest.totalFiles} files, ${manifest.totalLines} patch lines\n`,
     )
+    // The impact map's per-shard files name shard ids, which the re-split just renumbered.
+    // runImpact reports its own outcome; the split itself has already succeeded.
+    const { existsSync } = await import('node:fs')
+    const { join } = await import('node:path')
+    if (existsSync(join(runDir, 'impact.json'))) {
+      const { runImpact } = await import('../scripts/impact-cmd.ts')
+      await runImpact(runDir)
+    }
     return 0
   },
+  impact: async (args) => {
+    const runDir = args[0]
+    if (!runDir || runDir.startsWith('-')) {
+      process.stderr.write('impact: missing <run-dir>\n')
+      return 2
+    }
+    const { runImpact } = await import('../scripts/impact-cmd.ts')
+    return runImpact(runDir)
+  },
   render: async (args) => {
     const runDir = args[0]
     const page = args[1]
@@ -306,6 +350,22 @@ const HANDLERS: Record<string, Handler> = {
     const { runRender } = await import('../scripts/render-cmd.ts')
     return runRender(runDir, page, topN !== undefined ? { topN } : {})
   },
+  history: async (args) => {
+    const runDir = args[0]
+    if (!runDir || runDir.startsWith('-')) {
+      process.stderr.write('history: missing <run-dir>\n')
+      return 2
+    }
+    const { readHistory, renderHistory } = await import('../scripts/history.ts')
+    try {
+      const text = renderHistory(await readHistory(runDir))
+      if (text !== '') process.stdout.write(`${text}\n`)
+    } catch (err) {
+      process.stderr.write(`history: ${err instanceof Error ? err.message : String(err)}\n`)
+      return 1
+    }
+    return 0
+  },
   labels: async (args) => {
     const runDir = args[0]
     if (!runDir) {
@@ -335,6 +395,7 @@ const HANDLERS: Record<string, Handler> = {
       runDir,
       repoPath,
       gitBin: process.env.MAGPIE_GIT_BIN || 'git',
+      ghBin: process.env.MAGPIE_GH_BIN || 'gh',
       killGraceMs,
     })
   },

package/skills/magpie/evals/quality/README.md

@@ -138,7 +138,8 @@ Re-runs stages 3 to 6 (context, specialists, dedupe, critic) on the run's `diff.
    `results/<ts>-full-<runId>.json` with `recall`, `recallVerdicts`, `judgeModel` and
    `judgeCostUsd` all null, plus `log`, `shardManifest`, `findingsFiles`, `dedupedCount`,
    `criticDropped`, `evidenceDropped` and `thresholdDropped` read from the scratch run dir
-   before it is deleted. `shardCoverage` is derived from the manifest and the findings file
+   before it is deleted, and `impactBytes`, the size of `impact.md`. It is null unless
+   `magpie impact` was run on the replay's run dir, which no review stage does now. `shardCoverage` is derived from the manifest and the findings file
    names: a shard counts as reviewed when any specialist wrote a file for it, so it overstates
    a shard only some focuses reviewed. The printed line ends with `, shards <reviewed>/<total>`.
 4. When the run has a recall set (see [Recall](#recall)), judges the new kept findings
@@ -153,6 +154,78 @@ spend cap; `claude -p` has no `--max-cost-usd` flag. Hitting the cap stops the s
 and the replay then fails without scoring, on Claude's error result or on the missing
 `findings.kept.json`.
 
+## Specialist probe
+
+```
+bun evals/quality/probe-specialist.ts --corpus <runId> --repo <path-to-local-clone> \
+  --focus <focus> --file <diff path> --misses <id,id> --arm base|impact --repeats <n> \
+  [--max-cost-usd N] [--judge-model M] [--dry-run]
+```
+
+Runs one specialist on one shard, with (`impact`) or without (`base`) the impact map in
+its prompt, and judges its findings against the named misses from the run's recall set.
+It asks a narrower question than a full replay, for about a tenth of the cost.
+
+The `impact` arm needs a `references/specialists.md` that carries a `magpie-impact-part`
+block, as it did at commit 731a686 (since reverted). Without that block the probe exits
+before any paid session, naming the missing block.
+
+Repeats run one after another, each in its own scratch run dir and worktree:
+
+1. Scaffolds the run dir as the full replay does (worktree at the PR head, `pr.json`,
+   `diff.patch` and `brief.json` copied, a `setup` done line, then `magpie shard`), and
+   picks the shard whose files include `--file`; no such shard exits 1, naming the file.
+2. Impact arm only: `magpie impact`. A skipped impact stage fails the probe, since the arm
+   would then be a base arm. The prompt then carries `Outside the diff: <path>` under
+   `Diff:` and the `magpie-impact-part` block, with `<top entries>` filled from the
+   shard's `impact.shard-<n>.brief.md` and `<the path above>` with the path
+   (`impact.md` and `impact.brief.md` on an unsharded run). A shard with no impact file
+   exits 1, naming the shard, before any session is paid for.
+3. Builds the prompt from this checkout's `references/specialists.md` the way stage 4
+   does: focus block, run header, Output Contract, the excluded-files block when the
+   corpus run has `excluded-files.json`, and the brief rendered from `brief.json`.
+4. Runs `claude -p --output-format stream-json --verbose --allowedTools
+   Bash,Read,Grep,Glob,Write --max-budget-usd <N>` from the worktree, prompt on stdin.
+5. Writes `results/<ts>-probe-<runId>-<focus>-s<shard>-<arm>-r<k>.json` as soon as the
+   session ends, with `costUsd` and `sessionError` (a non-zero exit, an `is_error` result
+   such as a spent budget, or no result event; the probe then exits 1 after finishing).
+   `transcriptBadLines` counts transcript lines that are not JSON (a killed session cuts
+   its last line off); they are skipped, not fatal. The results dir is checked writable
+   before the session starts. Each later step fills its fields and rewrites the file; a
+   step that throws records `error`, rewrites it, prints one stderr line naming the
+   repeat and moves on to the next repeat, and the command exits 1 at the end. A failure
+   before the session (setup, worktree, a missing impact file) stops every repeat. In
+   order:
+   - `findings`, from the focus's findings file for the shard, numbering any finding
+     with no id as `<focus>-s<n>-<i>`. A missing file records `findings: 0` and
+     `missingFindings` with the reason rather than failing. `readImpact`: the session
+     opened the impact file with the Read tool (a Bash `cat` is not counted); null on
+     the base arm.
+   - `citesOutside`: findings whose description cites a `file:line` the shard's impact
+     Markdown lists outside the shard. The base arm runs `magpie impact` after its
+     session ends, so both arms count against the same map. It is a lower bound: only an
+     exact `path:line` match counts, so a citation by line range or by a nearby line is
+     missed.
+   - `judged`, `caught`, `verdicts` and `judgeCostUsd` from `judgeRecall`. With no
+     findings the judge is not called, `judged` stays false and `notJudged` says why.
+6. Removes the worktree and scratch dir whatever happened. A probe killed before its
+   cleanup ran leaves a stale worktree registered in the clone: run
+   `git -C <repo> worktree prune` there.
+
+`--dry-run` stops after step 3, prints the chosen shard and the assembled prompt, cleans
+up and exits 0. It calls no model and spends nothing.
+
+**Reading results:** a phase A tally counts only runs with `judged: true` or with zero
+findings (`findings: 0`); a run with an `error`, or with findings that were never judged,
+is left out and re-run. Findings are judged straight from the specialist, before dedupe's
+evidence filter, on both arms alike, so a finding dedupe would drop can still count as a
+catch.
+
+**Cost:** one specialist session per repeat, about $1.1 to $1.3 on the corpus PRs, capped
+by `--max-cost-usd` (default 3, passed as `--max-budget-usd`), plus one judge call when
+there are findings. A session that hits the cap is recorded with its cost and
+`sessionError`, and its findings are still read and judged.
+
 ## Recall
 
 The labels above only score findings magpie raised. The recall set scores the other side: