@iceinvein/agent-skills 0.21.1 → 0.21.2

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@iceinvein/agent-skills",
-  "version": "0.21.1",
+  "version": "0.21.2",
   "description": "Install agent skills into AI coding tools",
   "author": "iceinvein",
   "license": "MIT",

package/skills/index.json

@@ -219,9 +219,9 @@
   },
   {
     "name": "magpie",
-    "description": "Interactive PR review pipeline. Runs five parallel specialist subagents (security, bugs, performance, code-smells, architecture), dedupes findings, applies a critic rubric, peer-reviews via codex exec (falling back to a Claude second opinion when codex is unavailable), and serves an interactive HTML report for selecting findings to post via gh. Bundles a Bun CLI installed onto PATH via the skill's postinstall step. Use when the user asks to review a GitHub pull request. Splits oversized diffs into budgeted shards and rebuilds the diff from the local clone when gh pr diff refuses it.",
+    "description": "Interactive PR review pipeline. Runs five parallel specialist subagents (security, bugs, performance, code-smells, architecture), drops findings whose quoted evidence is not in the code, dedupes, runs critic subagents that re-check each finding against the worktree and set its risk label, peer-reviews via codex exec (falling back to a Claude second opinion when codex is unavailable), and serves an interactive HTML report for selecting or dismissing findings to post via gh, recording the outcome per finding as labels. Bundles a Bun CLI installed onto PATH via the skill's postinstall step. Use when the user asks to review a GitHub pull request. Splits oversized diffs into budgeted shards and rebuilds the diff from the local clone when gh pr diff refuses it.",
     "type": "prompt",
-    "version": "0.12.0"
+    "version": "0.13.0"
   },
   {
     "name": "migrate",

package/skills/magpie/README.md

@@ -4,7 +4,7 @@ Interactive Claude Code skill that runs a multi-stage PR review pipeline inside
 
 ## What it does
 
-Given a GitHub PR number, dispatches five specialist subagents in parallel (security, bugs, performance, code-smells, architecture), dedupes their findings, applies a critic rubric, peer-reviews via `codex exec` (falling back to a Claude second-opinion subagent when codex is unavailable), serves an interactive HTML report, and posts the findings the user selects via `gh`.
+Given a GitHub PR number, dispatches five specialist subagents in parallel (security, bugs, performance, code-smells, architecture), each quoting an evidence snippet for every anchored finding. Dedupe drops findings whose snippet is not in the file and groups nearby findings from different domains as merge candidates. Critic subagents then re-check each surviving finding against the worktree, keep, drop or merge it, and set its risk label, from which the severity is derived. Peer review runs via `codex exec` (falling back to a Claude second-opinion subagent when codex is unavailable). The interactive HTML report puts the top recommended findings first, lets the user dismiss findings with a reason, and posts the ones they select via `gh`; cleanup records what was posted, dismissed or ignored in `labels.json`.
 
 ## Requirements
 
@@ -54,7 +54,7 @@ The fixture lives at `fixtures/example-pr/` (pr.json + findings.final.json + pos
 ## Layout
 
 - `SKILL.md` is the agent-facing prompt: the stage walkthrough and nothing else. Installed by the agent-skills CLI.
-- `references/` holds the prompt bodies the walkthrough loads on demand, one file per stage that needs one: `scout.md` (stage 3, the PR-brief prompt and the `brief.json` contract), `specialists.md` (stage 4, the five focus blocks plus the shared output contract), `critic.md` (stage 6), `peer-review.md` (stage 7, including the Claude-fallback preamble). They ship in the bundle and sit next to `SKILL.md` once installed.
+- `references/` holds the prompt bodies the walkthrough loads on demand, one file per stage that needs one: `scout.md` (stage 3, the PR-brief prompt and the `brief.json` contract, including the repository review rules), `specialists.md` (stage 4, the five focus blocks plus the shared output contract), `critic.md` (stage 6, the critic subagent prompt `magpie critic-prompt` fills), `peer-review.md` (stage 7, including the Claude-fallback preamble). They ship in the bundle and sit next to `SKILL.md` once installed.
 - `skill.json` is the agent-skills manifest.
 - `bin/magpie` is the CLI invoked by the agent during stages; symlinked onto PATH by `install.sh`.
 - `scripts/` holds the implementation (server, dedupe, render, setup, cleanup, etc.).
@@ -66,4 +66,4 @@ The fixture lives at `fixtures/example-pr/` (pr.json + findings.final.json + pos
 
 ## Run directory layout
 
-Each invocation creates `~/.magpie/pr-<n>-<ts>/` with `pr.json`, `diff.patch`, `findings/`, `findings.deduped.json`, `findings.kept.json`, `findings.final.json`, `screen/`, `state/`, `log.jsonl`. On completion the directory is renamed to `<run-dir>.archived-<timestamp>` rather than deleted, so logs survive for postmortem.
+Each invocation creates `~/.magpie/pr-<n>-<ts>/` with `pr.json`, `diff.patch`, `findings/`, `findings.deduped.json`, `merge-candidates.json`, `critic-prompt.md` and `critic.json` (numbered per batch when there is more than one), `findings.kept.json`, `critic-dropped.json`, `findings.final.json`, `labels.json`, `screen/`, `state/`, `log.jsonl`. On completion the directory is renamed to `<run-dir>.archived-<timestamp>` rather than deleted, so logs survive for postmortem.

package/skills/magpie/SKILL.md

@@ -148,7 +148,9 @@ If every specialist fails (no findings files written), log
 magpie dedupe "$RUN_DIR" [--threshold <0-10>]
 ```
 
-`magpie dedupe` also runs a deterministic evidence check against the worktree: findings whose `file` is missing or whose `line` is out of range are dropped, logged, and recorded to `$RUN_DIR/evidence-dropped.json`. The check is skipped when the worktree is gone (archived run replay).
+`magpie dedupe` also checks evidence against the worktree, dropping an anchored finding whose file is missing (`hallucinated-file`), whose line is out of range (`invented-line`), that has no `evidence` snippet (`missing-evidence`), or whose snippet is not near its line (`evidence-not-found`); a snippet found exactly once elsewhere in the file re-anchors it. Both go to `$RUN_DIR/evidence-dropped.json` as `{dropped, reanchored}`, written only when non-empty, so no file is normal. The check is skipped when the worktree is gone (archived run replay).
+
+It always writes `$RUN_DIR/merge-candidates.json`: ids from different domains anchored close together in one file, for the critic to merge or keep apart.
 
 Each finding gets a derived 0-10 `score` from its risk fields; those below `--threshold` (default 3) are dropped before the critic LLM runs and recorded to `$RUN_DIR/threshold-dropped.json`. Pass `--threshold 0` to keep everything.
 
@@ -156,12 +158,21 @@ Re-render progress.
 
 ### 6. Critic
 
-Read `references/critic.md` and `$RUN_DIR/findings.deduped.json`. Substitute both placeholders in the critic rubric (the compact candidate list including each finding's `onChangedLine`, and the `<<DIFF_EXCERPT>>` hunks for the referenced files), then apply the rubric verbatim (one verdict per finding). Write the kept subset to `$RUN_DIR/findings.kept.json`. Append `{stage: critic, status: done}` and re-render progress.
+The critic runs as worktree-reading subagents. The CLI fills its prompt from `references/critic.md`; substitute nothing by hand. Append `{stage: critic, status: running}` to `$RUN_DIR/log.jsonl`, re-render progress, then:
+
+```
+magpie critic-prompt "$RUN_DIR"
+```
+
+It prints `<prompt path>\t<output path>` per batch of about 30 candidates (a merge group always stays in one batch). Dispatch one subagent (Agent tool, `general-purpose`) per line, all in one message, each one's entire task the verbatim contents of its prompt file. Each writes its output path and returns one summary line. No lines printed means no candidates: go straight to `critic-apply`.
 
-When `findings.deduped.json` holds more than 40 findings, run the rubric in batches of
-30 rather than one prompt: a sharded run can produce more candidates than fit alongside
-their diff excerpts. Apply the same rubric verbatim per batch and concatenate the kept
-subsets into `findings.kept.json`.
+Confirm every output path exists; re-dispatch any batch whose file is missing. On a resume, dispatch only the batches whose output file is missing. Apply the verdicts:
+
+```
+magpie critic-apply "$RUN_DIR"
+```
+
+It writes `findings.kept.json` and `critic-dropped.json` and logs the critic `done` entry itself; do not append another. On a non-zero exit, stderr names offending ids or a file: delete the output file of each batch whose ids or file it names, re-dispatch only those, and re-run `critic-apply`. If a batch fails twice after re-dispatch, stop and show the user stderr verbatim rather than looping. Re-render progress.
 
 ### 7. Peer review
 
@@ -169,7 +180,7 @@ Append `{stage: peer-review, status: running}` to `$RUN_DIR/log.jsonl` and re-re
 
 Build the peer-review prompt first: read `references/peer-review.md`, take the `magpie-peer-review` block from it, and substitute the placeholders listed in that file's `## Substitute before use` preamble.
 
-One batch carries up to 40 findings; above that, split them 30 at a time, as in stage 6.
+One batch carries up to 40 findings; above that, split them 30 at a time.
 Write each batch's prompt, its `<<KEPT_FINDINGS_COMPACT>>` narrowed to that batch, to
 `$RUN_DIR/peer-prompt-<k>.md`, `<k>` counting from 1. **When there is a single batch, drop `-<k>` throughout** (`peer-prompt.md`,
 `peer.out`), which is the common case. Keep the `add` id counter running across batches
@@ -189,7 +200,7 @@ If codex returns non-zero on a batch, do not abort: record `{stage: peer-review,
 
 **Claude path (fallback).** When `codex` is unavailable or failed, get the second opinion from a Claude subagent instead, one per batch. Set `<<PEER_PROVIDER>>` to `claude`, then prepend the `magpie-peer-review-claude-preamble` block from `references/peer-review.md` to each batch's substituted prompt (the preamble forces genuine independence, since the reviewer shares a model family with the primary reviewers). Dispatch one subagent (Agent tool, `general-purpose`) per batch whose entire task is that combined prompt, and instruct it to return only the fenced `review-peer-review` JSON block. Write each output to `$RUN_DIR/peer-<k>.out`, extract each `review-peer-review` block, merge into `$RUN_DIR/peer.json` after the last batch as above, and append `{stage: peer-review, status: done, provider: claude}` (`provider: mixed` if codex handled some batches).
 
-**Apply the verdicts (both paths).** Parse the merged verdicts and apply the `update` / `add` entries (an empty array means no change). Mint each `add`'s `id` as above before merging, since the peer contract does not carry ids. Then write `findings.final.json`. Re-render progress.
+**Apply the verdicts (both paths).** Parse the merged verdicts and apply the `update` / `add` entries (an empty array means no change). A peer `fields.severity` (or `finding.severity` on an `add`) is ignored: severity is derived from `risk.impact`. Mint each `add`'s `id` as above before merging, since the peer contract does not carry ids. Then write `findings.final.json`. Re-render progress.
 
 ### 8. Report
 
@@ -197,6 +208,8 @@ If codex returns non-zero on a batch, do not abort: record `{stage: peer-review,
 magpie render "$RUN_DIR" findings
 ```
 
+The report shows the top 10 recommended (`must-fix`/`should-fix`, highest score first) and folds the rest. `--top <n>` is not saved (the stage 9 re-render and `magpie serve`'s auto-refresh show 10 again), so use it only when nothing will re-render.
+
 Append `{stage: report, status: done}` to `$RUN_DIR/log.jsonl` and re-render progress (the render CLI does not log this itself, and `magpie status` needs the `done` entry to resume past `report`).
 
 Print to the terminal: "Findings ready at <url>. Tick the ones you want and click **Post Selected**, or reply `post` here and I'll post whatever you've ticked."
@@ -205,9 +218,9 @@ End the turn.
 
 ### 9. Post
 
-Most users tick the checkboxes in the served report and click **Post Selected** (or **Post Recommended**, which takes every `must-fix`/`should-fix` finding and skips the `consider`/`optional` ones); the server posts that batch as one GitHub review with inline threads. The agent posts only when the user types `post` (optionally `post 1,3,7` for indices), which takes the CLI path below: separate inline comments plus a top-level summary comment. Either path records posted ids in `post-status.json`, so the two cannot double-post the same finding.
+Most users tick the checkboxes in the served report and click **Post Selected** (or **Post Recommended**, which takes only the top N above the fold; **Select recommended** ticks the same set); the server posts that batch as one GitHub review with inline threads. **Dismiss** on a finding records a reason (`wrong`, `not-worth-it`, `duplicate`, `style`) and drops it from the recommended set. The agent posts only when the user types `post` (optionally `post 1,3,7` for indices), which takes the CLI path below: separate inline comments plus a top-level summary comment. Either path records posted ids in `post-status.json`, so the two cannot double-post the same finding.
 
-When the user types `post`, read `$RUN_DIR/state/events` and fold them in order, keeping the LAST event per finding id; ids whose last event is `select` are selected. (Not union-minus: the UI emits one event per toggle, so select, deselect, select again resolves to selected.) Merge any explicit indices the user named (1-based, against `findings.final.json` in file order). If nothing is selected, say so and ask rather than posting an empty batch. Then post via the CLI:
+When the user types `post`, read `$RUN_DIR/state/events` and fold them in order, keeping the LAST event per finding id; ids whose last event is `select` are selected (a later `dismiss` therefore unselects). (Not union-minus: the UI emits one event per toggle, so select, deselect, select again resolves to selected.) Merge any explicit indices the user named (1-based, against `findings.final.json` in file order). If nothing is selected, say so and ask rather than posting an empty batch. Then post via the CLI:
 
 ```
 magpie post "$RUN_DIR" --ids id1,id2,id3
@@ -232,7 +245,7 @@ magpie render "$RUN_DIR" findings
 magpie cleanup "$RUN_DIR" --repo "$REPO"
 ```
 
-The run directory is renamed to `<run-dir>.archived-<timestamp>` and the worktree is removed. The CLI prints two lines on success: `archived to <path>` and `view later: magpie open <archived-id>`. Surface that second line verbatim so the user has a one-command path back to the report.
+Cleanup first writes `labels.json` (each final finding posted, dismissed or ignored, with the post route or dismiss reason; `magpie labels "$RUN_DIR"` writes it on demand). The run directory is then renamed to `<run-dir>.archived-<timestamp>` and the worktree is removed. The CLI prints two lines on success: `archived to <path>` and `view later: magpie open <archived-id>`. Surface that second line verbatim so the user has a one-command path back to the report.
 
 The archived `findings.html` is self-contained and auto-switches to read-only "archived" mode when opened, so:
 

package/skills/magpie/bin/magpie.ts

@@ -10,10 +10,18 @@ Subcommands:
   setup <run-dir> --pr <n>   Pre-flight, fetch PR, create worktree
   serve <run-dir-or-id>      Start the HTML server (accepts active or archived run id)
   dedupe <run-dir>           Merge specialist findings into deduped set
+  critic-prompt <run-dir> [--batch-size N]
+                             Write one critic prompt per batch, print prompt and output paths
+  critic-apply <run-dir> [--design-cap N]
+                             Apply critic verdicts, write findings.kept.json (no cap on
+                             code-smells + architecture keeps unless --design-cap is given)
   shard <run-dir> [--budget N] [--max-files N]
                              Re-split diff.patch into budgeted shards
-  render <run-dir> <page>    Render progress.html or findings.html
-  cleanup <run-dir>          Remove worktree, stop server, archive run
+  render <run-dir> <page> [--top N]
+                             Render progress.html or findings.html (findings recommends
+                             the top N, default 10, and folds the rest)
+  labels <run-dir>           Write labels.json (posted, dismissed, ignored per finding)
+  cleanup <run-dir>          Remove worktree, stop server, write labels, archive run
   status <run-dir>           Print highest completed stage
   open [id]                  Open findings.html in your browser (defaults to latest run)
   post <run-dir> --ids a,b   Post the given finding ids via gh (rich body + optional summary)
@@ -92,9 +100,16 @@ const HANDLERS: Record<string, Handler> = {
       }
     }
     // Auto-refresh: re-render findings.html with the currently-shipped CSS/JS
-    // so serving an old archive picks up new report features. Best-effort.
+    // so serving an old archive picks up new report features. A failed refresh
+    // leaves the previous page in place, so serve keeps going after reporting why.
     const { refreshFindings } = await import('../scripts/refresh.ts')
-    await refreshFindings(runDir).catch(() => {})
+    try {
+      await refreshFindings(runDir)
+    } catch (err) {
+      process.stderr.write(
+        `serve: refresh failed, serving the previous page: ${err instanceof Error ? err.message : String(err)}\n`,
+      )
+    }
     const { runServe } = await import('../scripts/serve-cmd.ts')
     return runServe({ runDir, idleMs, host })
   },
@@ -118,6 +133,48 @@ const HANDLERS: Record<string, Handler> = {
     const { runDedupe } = await import('../scripts/dedupe-cmd.ts')
     return runDedupe(runDir, threshold !== undefined ? { threshold } : {})
   },
+  'critic-prompt': async (args) => {
+    const runDir = args[0]
+    if (!runDir) {
+      process.stderr.write('critic-prompt: missing <run-dir> [--batch-size <n>]\n')
+      return 2
+    }
+    const flag = args.indexOf('--batch-size')
+    const raw = flag !== -1 ? args[flag + 1] : undefined
+    let batchSize: number | undefined
+    if (flag !== -1) {
+      const n = Number(raw)
+      if (!Number.isInteger(n) || n <= 0) {
+        process.stderr.write(
+          `critic-prompt: invalid --batch-size ${raw} (want a positive integer)\n`,
+        )
+        return 2
+      }
+      batchSize = n
+    }
+    const { runCriticPrompt } = await import('../scripts/critic-cmd.ts')
+    return runCriticPrompt(runDir, batchSize !== undefined ? { batchSize } : {})
+  },
+  'critic-apply': async (args) => {
+    const runDir = args[0]
+    if (!runDir) {
+      process.stderr.write('critic-apply: missing <run-dir> [--design-cap <n>]\n')
+      return 2
+    }
+    const flag = args.indexOf('--design-cap')
+    const raw = flag !== -1 ? args[flag + 1] : undefined
+    let designCap: number | undefined
+    if (flag !== -1) {
+      const n = Number(raw)
+      if (!Number.isInteger(n) || n < 0) {
+        process.stderr.write(`critic-apply: invalid --design-cap ${raw} (want an integer >= 0)\n`)
+        return 2
+      }
+      designCap = n
+    }
+    const { runCriticApply } = await import('../scripts/critic-cmd.ts')
+    return runCriticApply(runDir, designCap !== undefined ? { designCap } : {})
+  },
   shard: async (args) => {
     const runDir = args[0]
     if (!runDir) {
@@ -163,11 +220,31 @@ const HANDLERS: Record<string, Handler> = {
     const runDir = args[0]
     const page = args[1]
     if (!runDir || (page !== 'progress' && page !== 'findings')) {
-      process.stderr.write('render: missing <run-dir> <progress|findings>\n')
+      process.stderr.write('render: missing <run-dir> <progress|findings> [--top <n>]\n')
       return 2
     }
+    const flag = args.indexOf('--top')
+    const raw = flag !== -1 ? args[flag + 1] : undefined
+    let topN: number | undefined
+    if (flag !== -1) {
+      const n = Number(raw)
+      if (!Number.isInteger(n) || n <= 0) {
+        process.stderr.write(`render: invalid --top ${raw} (want a positive integer)\n`)
+        return 1
+      }
+      topN = n
+    }
     const { runRender } = await import('../scripts/render-cmd.ts')
-    return runRender(runDir, page)
+    return runRender(runDir, page, topN !== undefined ? { topN } : {})
+  },
+  labels: async (args) => {
+    const runDir = args[0]
+    if (!runDir) {
+      process.stderr.write('labels: missing <run-dir>\n')
+      return 2
+    }
+    const { runLabels } = await import('../scripts/labels-cmd.ts')
+    return runLabels(runDir)
   },
   cleanup: async (args) => {
     const runDir = args[0]
@@ -248,6 +325,7 @@ const HANDLERS: Record<string, Handler> = {
       runDir,
       findingIds,
       dryRun,
+      via: 'cli',
       ...(includeSummary ? { includeSummary } : {}),
     })
     process.stdout.write(`${JSON.stringify(outcome)}\n`)

package/skills/magpie/evals/README.md

@@ -80,6 +80,21 @@ file is what most of the graders read: "posted exactly these ids", "never ran
 invoked, and the call log answers them without depending on how the reply is
 worded.
 
+Cases that start past stage 6 hand the agent what the subagent critic would
+have left. `codex-missing-falls-back` writes the whole chain: specialist files
+with an `evidence` snippet per finding and no `severity`, `findings.deduped.json`
+with severity derived from `risk.impact`, `threshold-dropped.json`,
+`merge-candidates.json`, `critic.json` with one verdict per candidate,
+`critic-dropped.json`, and the `findings.kept.json` that `magpie critic-apply`
+produces from them. The log carries the critic `done` entry with the counts
+critic-apply writes. Running the real `magpie dedupe` and `critic-apply` over
+that run directory gives the same `merge-candidates.json`,
+`threshold-dropped.json` and `critic-dropped.json`, and the same kept ids and
+log counts. `findings.deduped.json` and `findings.kept.json` differ only in
+`onChangedLine`, which the fixture leaves out, and in order; no grader reads
+either. Repeat that check after changing either command. The report and post cases start from
+`findings.final.json` and only need the same log entry and the evidence field.
+
 `fixture.sh` is duplicated across the cases rather than shared, because
 `context.scaffold_script` reads only from the case's own directory.
 

package/skills/magpie/evals/codex-missing-falls-back/fixture.sh

@@ -111,7 +111,8 @@ cat > "$RUN_DIR/log.jsonl" <<'LOG'
 {"stage":"context","status":"done"}
 {"stage":"specialists","status":"done"}
 {"stage":"dedupe","status":"done"}
-{"stage":"critic","status":"done"}
+{"stage":"critic","status":"running"}
+{"stage":"critic","status":"done","kept":3,"dropped":1,"merged":0,"capped":0}
 LOG
 
 # The PR under review, as setup would have left it: the filtered diff, and a
@@ -187,9 +188,10 @@ export async function load(tenantId: string) {
 TS
 
 # The findings the run already has: one file per specialist focus, the deduped
-# set derived from them, and the subset the critic kept. Generated together so
-# the chain holds: nothing is kept that was never deduped, and every finding
-# cites a line its hunk carries.
+# set derived from them, the critic's verdicts, and the subset critic-apply kept.
+# Generated together so the chain holds: nothing is kept that was never deduped,
+# every finding cites a line its hunk carries, and every evidence snippet is the
+# worktree line it points at.
 python3 - "$RUN_DIR" <<'FINDINGS'
 import json, pathlib, sys
 
@@ -202,9 +204,9 @@ FINDINGS = [
         'domain': 'security',
         'file': 'src/settings/cache.ts',
         'line': 4,
-        'severity': 'high',
+        'evidence': 'store.set(tenantId, settings)',
         'risk': {'impact': 'high', 'likelihood': 'likely', 'confidence': 'high', 'action': 'must-fix'},
-        'score': 8,
+        'score': 8.8,
         'title': 'Tenant settings cache is a process-global Map with no eviction',
         'description': """Observation: put() writes into a module-level Map keyed by tenant id (src/settings/cache.ts:4), with no size bound and no TTL.
 
@@ -218,9 +220,9 @@ Suggested direction: bound the map and give entries a TTL, or key the cache per
         'domain': 'bugs',
         'file': 'src/settings/loader.ts',
         'line': 12,
-        'severity': 'medium',
+        'evidence': 'const fresh = await fetchSettings(tenantId)',
         'risk': {'impact': 'medium', 'likelihood': 'possible', 'confidence': 'medium', 'action': 'should-fix'},
-        'score': 6,
+        'score': 5.1,
         'title': 'Concurrent loads for the same tenant each hit the network',
         'description': """Observation: load() checks the cache, then awaits fetchSettings before writing back (src/settings/loader.ts:12).
 
@@ -234,9 +236,9 @@ Suggested direction: cache the in-flight promise rather than the resolved value.
         'domain': 'architecture',
         'file': 'src/settings/loader.ts',
         'line': 10,
-        'severity': 'medium',
+        'evidence': 'const hit = get(tenantId)',
         'risk': {'impact': 'medium', 'likelihood': 'possible', 'confidence': 'medium', 'action': 'consider'},
-        'score': 5,
+        'score': 4.7,
         'title': 'The loader owns the cache rather than being handed one',
         'description': """Observation: load() calls the cache module's free functions directly (src/settings/loader.ts:10).
 
@@ -250,9 +252,9 @@ Suggested direction: take the cache as a parameter.""",
         'domain': 'performance',
         'file': 'src/settings/cache.ts',
         'line': 12,
-        'severity': 'low',
+        'evidence': 'store.clear()',
         'risk': {'impact': 'low', 'likelihood': 'possible', 'confidence': 'medium', 'action': 'consider'},
-        'score': 3,
+        'score': 3.5,
         'title': 'clear() evicts every tenant, not the one whose settings changed',
         'description': """Observation: clear() calls store.clear() (src/settings/cache.ts:12) and is the only invalidation the module offers.
 
@@ -266,9 +268,9 @@ Suggested direction: add delete(tenantId) and leave clear() for shutdown.""",
         'domain': 'code-smells',
         'file': 'src/settings/cache.ts',
         'line': 8,
-        'severity': 'low',
-        'risk': {'impact': 'low', 'likelihood': 'unlikely', 'confidence': 'medium', 'action': 'optional'},
-        'score': 2,
+        'evidence': 'return store.get(tenantId)',
+        'risk': {'impact': 'low', 'likelihood': 'edge-case', 'confidence': 'medium', 'action': 'optional'},
+        'score': 2.3,
         'title': 'get() hands back the stored object, so a caller can mutate the cache',
         'description': """Observation: get() returns store.get(tenantId) directly (src/settings/cache.ts:8).
 
@@ -278,12 +280,22 @@ Suggested direction: freeze the value on put, or return a copy.""",
     },
 ]
 
-# The critic kept the three above its bar and dropped the two below it.
+# smell-1 scored under the default threshold of 3, so dedupe set it aside and
+# the critic never saw it. Of the four it did see, it kept three and dropped one.
+BELOW_THRESHOLD = {'smell-1'}
 KEPT = {'security-1', 'bugs-1', 'arch-1'}
 
+# Severity is derived from risk.impact; specialists do not write it.
+SEVERITY = {'critical': 'blocker', 'high': 'high', 'medium': 'medium', 'low': 'low'}
+
 def without(finding, *keys):
     return {k: v for k, v in finding.items() if k not in keys}
 
+def derived(finding):
+    return {**finding, 'severity': SEVERITY[finding['risk']['impact']]}
+
+deduped = [derived(without(f, 'focus')) for f in FINDINGS if f['id'] not in BELOW_THRESHOLD]
+
 findings_dir = run / 'findings'
 findings_dir.mkdir(parents=True, exist_ok=True)
 for focus in ('security', 'bugs', 'performance', 'code-smells', 'architecture'):
@@ -291,11 +303,38 @@ for focus in ('security', 'bugs', 'performance', 'code-smells', 'architecture'):
     (findings_dir / f'{focus}.json').write_text(json.dumps(mine, indent=2) + '\n')
 (findings_dir / 'tests.json').write_text('[]\n')
 
-(run / 'findings.deduped.json').write_text(
-    json.dumps([without(f, 'focus') for f in FINDINGS], indent=2) + '\n'
+(run / 'findings.deduped.json').write_text(json.dumps(deduped, indent=2) + '\n')
+(run / 'threshold-dropped.json').write_text(
+    json.dumps(
+        [{'id': f['id'], 'score': f['score'], 'title': f['title']} for f in FINDINGS if f['id'] in BELOW_THRESHOLD],
+        indent=2,
+    ) + '\n'
+)
+# Same file within 8 lines, more than one domain: what dedupe groups for the critic.
+(run / 'merge-candidates.json').write_text(
+    json.dumps([['security-1', 'perf-1'], ['arch-1', 'bugs-1']], indent=2) + '\n'
+)
+
+CHECKED = {
+    'security-1': ['src/settings/cache.ts:1-9', 'src/settings/loader.ts:9-15'],
+    'bugs-1': ['src/settings/loader.ts:9-15'],
+    'arch-1': ['src/settings/loader.ts:1-15'],
+    'perf-1': ['src/settings/cache.ts:11-13'],
+}
+verdicts = []
+for f in deduped:
+    if f['id'] in KEPT:
+        verdicts.append({'id': f['id'], 'verdict': 'keep', 'reason': 'confirmed in the worktree',
+                         'risk': f['risk'], 'checked': CHECKED[f['id']]})
+    else:
+        verdicts.append({'id': f['id'], 'verdict': 'drop', 'reason': 'one flush per settings change is not a measurable cost',
+                         'checked': CHECKED[f['id']]})
+(run / 'critic.json').write_text(json.dumps(verdicts, indent=2) + '\n')
+(run / 'critic-dropped.json').write_text(
+    json.dumps([{'id': v['id'], 'reason': v['reason']} for v in verdicts if v['verdict'] == 'drop'], indent=2) + '\n'
 )
 (run / 'findings.kept.json').write_text(
-    json.dumps([without(f, 'focus', 'score') for f in FINDINGS if f['id'] in KEPT], indent=2) + '\n'
+    json.dumps([f for f in deduped if f['id'] in KEPT], indent=2) + '\n'
 )
 FINDINGS
 
@@ -303,8 +342,10 @@ echo "http://127.0.0.1:4599" > "$RUN_DIR/state/server-info"
 
 cat > "$RUN_DIR/brief.json" <<'JSON'
 {
-  "summary": "Adds a process-global cache in front of tenant settings loads.",
-  "riskAreas": ["tenant isolation", "cache invalidation"],
-  "conventions": []
+  "purpose": "Adds a process-global cache in front of tenant settings loads.",
+  "changes": ["cache module gains put and get", "loader reads through the cache"],
+  "watchItems": [],
+  "unclear": ["how a settings change is meant to invalidate the cache"],
+  "reviewRules": []
 }
 JSON

package/skills/magpie/evals/post-folds-selection-events/fixture.sh

@@ -142,7 +142,8 @@ cat > "$RUN_DIR/log.jsonl" <<'LOG'
 {"stage":"context","status":"done"}
 {"stage":"specialists","status":"done"}
 {"stage":"dedupe","status":"done"}
-{"stage":"critic","status":"done"}
+{"stage":"critic","status":"running"}
+{"stage":"critic","status":"done","kept":5,"dropped":0,"merged":0,"capped":0}
 {"stage":"peer-review","status":"done","provider":"claude"}
 {"stage":"report","status":"done"}
 LOG
@@ -225,6 +226,7 @@ cat > "$RUN_DIR/findings.final.json" <<'JSON'
     "id": "security-1",
     "file": "src/settings/cache.ts",
     "line": 4,
+    "evidence": "store.set(tenantId, settings)",
     "severity": "high",
     "risk": { "impact": "high", "likelihood": "likely", "confidence": "high", "action": "must-fix" },
     "domain": "security",
@@ -235,6 +237,7 @@ cat > "$RUN_DIR/findings.final.json" <<'JSON'
     "id": "bugs-1",
     "file": "src/settings/loader.ts",
     "line": 12,
+    "evidence": "const fresh = await fetchSettings(tenantId)",
     "severity": "medium",
     "risk": { "impact": "medium", "likelihood": "possible", "confidence": "medium", "action": "should-fix" },
     "domain": "bugs",
@@ -245,6 +248,7 @@ cat > "$RUN_DIR/findings.final.json" <<'JSON'
     "id": "perf-1",
     "file": "src/settings/cache.ts",
     "line": 12,
+    "evidence": "store.clear()",
     "severity": "low",
     "risk": { "impact": "low", "likelihood": "possible", "confidence": "medium", "action": "consider" },
     "domain": "performance",
@@ -255,6 +259,7 @@ cat > "$RUN_DIR/findings.final.json" <<'JSON'
     "id": "arch-1",
     "file": "src/settings/loader.ts",
     "line": 10,
+    "evidence": "const hit = get(tenantId)",
     "severity": "medium",
     "risk": { "impact": "medium", "likelihood": "possible", "confidence": "medium", "action": "consider" },
     "domain": "architecture",
@@ -265,8 +270,9 @@ cat > "$RUN_DIR/findings.final.json" <<'JSON'
     "id": "smell-1",
     "file": "src/settings/cache.ts",
     "line": 8,
+    "evidence": "return store.get(tenantId)",
     "severity": "low",
-    "risk": { "impact": "low", "likelihood": "unlikely", "confidence": "medium", "action": "optional" },
+    "risk": { "impact": "low", "likelihood": "edge-case", "confidence": "medium", "action": "optional" },
     "domain": "code-smells",
     "title": "get() hands back the stored object, so a caller can mutate the cache",
     "description": "Observation: get() returns store.get(tenantId) directly (src/settings/cache.ts:8).\n\nWhy it matters: a caller that edits the returned settings edits every later reader's copy.\n\nSuggested direction: freeze the value on put, or return a copy."