vigiles 4.1.0 → 5.0.0

package/README.md

@@ -27,10 +27,10 @@ deterministic layer for the harness: it **lints** the references your instructio
 files make and **tests** that your hooks and skills actually fire. Two independent
 pillars — adopt either, or both:
 
-|       | Pillar                          | What it does                                                                                                                                                                                                |
-| ----- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **①** | **Lint your instruction files** | Every linter rule, file path, script, and code symbol your CLAUDE.md cites is checked against reality, so stale references can't silently mislead the agent. → [guide](docs/verifying-instruction-files.md) |
-| **②** | **Test your harness**           | Your hooks and skills are code — vigiles tests they actually fire, **deterministically and free** (no model, no API key) before you pay for an eval. → [guide](docs/harness-testing.md)                     |
+|       | Pillar                          | What it does                                                                                                                                                                                                                                                              |
+| ----- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **①** | **Lint your instruction files** | Every linter rule, file path, script, and code symbol your CLAUDE.md cites is checked against reality, so stale references can't silently mislead the agent. → [guide](docs/verifying-instruction-files.md)                                                               |
+| **②** | **Test your harness**           | Your hooks and skills are code — vigiles tests they actually fire, **deterministically and free** (no model, no API key); and when a question _does_ need a real-model eval, it runs on your **Claude subscription**, not metered API. → [guide](docs/harness-testing.md) |
 
 Neither pillar depends on the other — pick the one that hurts today. **Works with
 Claude Code and Codex** ([`vigiles/codex`](docs/harnesses.md)) behind a five-port
@@ -93,10 +93,42 @@ assert(r.blocked); // a red ✗ here means your hook silently lets it through
 ```
 
 Three tiers, cheapest first: **`runHook`** (a hook's logic), **`runHarnessTest`**
-(the real agent CLI against a scripted mock model), **`runEval`** (the real model
-A/B with a significance gate). **Testing a skill?** `measureTriggerRate` checks
-its description actually **fires** across varied prompts (recall) without
-hijacking unrelated ones (precision). **[Full guide →](docs/harness-testing.md)**
+(the real agent CLI against a scripted mock model), and the real-model scored tier
+(**`measure`** / **`runEval`**). **Testing a skill?** Two questions, both covered:
+does its description **fire** (`measureTriggerRate` — recall across varied prompts
+without hijacking unrelated ones, precision), **and** does its guidance actually
+**work**. For "is this exact skill any good?" score the output directly —
+`measure({ checks: [judged(rubric)] })` + `assertRates` (the **absolute** oracle,
+what promptfoo/DeepEval lead with; no on/off baseline needed). When you need the
+**relative** lift over no-skill — regression, or proving the change isn't noise —
+A/B it on-vs-off with `runEval` + `assertSignificant`. Description _and_ behavior,
+not just one. **Need a safety property** — that the agent
+**didn't** push to the wrong branch or call a paid API? `notTool` + `interceptTools`
+intercept the tool in the real hook layer, so the attempt is caught and the side
+effect never happens. **[Full guide →](docs/harness-testing.md)** · vigiles runs
+foreign code (and a real model) safely by default — **[safety model →](docs/safety.md)**
+
+Most of what real plugins do is testable cheaply — fire / trigger / contract /
+safety, plus **record-replay** for the tool/API results a skill consumes (recorded
+once from the real tool, replayed deterministically — no live service, no Docker).
+That covers ~90%+ of real plugin surface on your subscription; the rare case that
+needs a real browser or database **composes with Docker** rather than us
+reinventing the sandbox. **[What we test, how →](research/eval-coverage-and-isolation.md)**
+
+**Affordable by design — the eval you can actually run.** Almost nobody evals
+their harness, because the usual tools (promptfoo, DeepEval, …) hit the API SDK
+and bill **per token on every run**. vigiles inverts that: most questions are
+answered with **no model at all** (free, every commit), and when you do reach for
+a real-model eval, vigiles drives your `claude` CLI — so it runs on the **Pro/Max
+subscription you already pay for**, not metered API billing. CI runs only the free
+deterministic tiers; you run the real-model eval where the subscription already is
+— a Claude Code session or locally — when it's worth it, not on every PR.
+
+This affordability story is **ToS-clean**: vigiles drives _your own_ `claude` CLI
+to test _your own_ harness on _your own_ subscription — the same thing you do when
+you run Claude Code. (The Claude Agent SDK's ToS restricts _productizing_ claude.ai
+login/limits in a third-party offering; running your own tests on your own sub is
+exactly the supported posture, not that.)
 
 ## Quick start
 

package/dist/adapters/claude-code/agent-runtime.d.ts

@@ -3,7 +3,7 @@
  *
  * A subagent declares an allowed-tools contract in its frontmatter (`tools:`).
  * But that field is documentation, not a hard runtime boundary (Claude Code
- * issue #54898): permissions are session-wide, a subagent inherits the parent
+ * #4740/#21460, SDK #172): permissions are session-wide, a subagent inherits the parent
  * session's grants, and `tools:` only filters what's *offered* — it can't deny
  * what the session allows. The deterministic layer that actually closes the gap
  * is a **PreToolUse hook** that blocks any tool the active agent's contract

package/dist/adapters/claude-code/agent-runtime.js

@@ -4,7 +4,7 @@
  *
  * A subagent declares an allowed-tools contract in its frontmatter (`tools:`).
  * But that field is documentation, not a hard runtime boundary (Claude Code
- * issue #54898): permissions are session-wide, a subagent inherits the parent
+ * #4740/#21460, SDK #172): permissions are session-wide, a subagent inherits the parent
  * session's grants, and `tools:` only filters what's *offered* — it can't deny
  * what the session allows. The deterministic layer that actually closes the gap
  * is a **PreToolUse hook** that blocks any tool the active agent's contract

package/dist/arg-match.d.ts

@@ -0,0 +1,28 @@
+/**
+ * `ArgMatcher` — a small, serializable matcher over a tool call's `input`.
+ *
+ * Shared by the check vocabulary (`toolWith` / `notTool` in `src/check.ts`) and
+ * the tool-interception seam (`src/tool-intercept.ts`), so "did the agent call
+ * this tool with these arguments?" means the same thing whether you're *asserting*
+ * on a captured call or *intercepting* one before it runs. Pure + model-free.
+ */
+/**
+ * A declarative matcher over a tool call's `input`, keyed by **dot-path** (e.g.
+ * `"body.prompt"`). Each value is matched against the value at that path: a
+ * `RegExp` is a pattern over the stringified value (use this for "contains"), and
+ * a `string`/`number`/`boolean` is an **exact** match (use this for "equals", e.g.
+ * a push target). All keys must match (AND). Serializable, so anything carrying
+ * one still round-trips through `toJSON`.
+ */
+export type ArgMatcher = Record<string, string | number | boolean | RegExp>;
+/** Render any tool-input value as a string for matching / messages. */
+export declare function stringifyValue(v: unknown): string;
+/** Resolve a dot-path (`"a.b.c"`) within an arbitrary value, or undefined. */
+export declare function getPath(obj: unknown, path: string): unknown;
+/** Does `input` satisfy every entry of `matcher`? (RegExp = pattern, else exact.) */
+export declare function matchesArgs(input: unknown, matcher: ArgMatcher): boolean;
+/** A human-readable form of a matcher for failure messages. */
+export declare function describeArgs(matcher: ArgMatcher): string;
+/** Serialize a matcher for `toJSON` (RegExp → its string form). */
+export declare function serializeArgs(matcher: ArgMatcher): Record<string, string | number | boolean>;
+//# sourceMappingURL=arg-match.d.ts.map

package/dist/arg-match.js

@@ -0,0 +1,61 @@
+"use strict";
+/**
+ * `ArgMatcher` — a small, serializable matcher over a tool call's `input`.
+ *
+ * Shared by the check vocabulary (`toolWith` / `notTool` in `src/check.ts`) and
+ * the tool-interception seam (`src/tool-intercept.ts`), so "did the agent call
+ * this tool with these arguments?" means the same thing whether you're *asserting*
+ * on a captured call or *intercepting* one before it runs. Pure + model-free.
+ */
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.stringifyValue = stringifyValue;
+exports.getPath = getPath;
+exports.matchesArgs = matchesArgs;
+exports.describeArgs = describeArgs;
+exports.serializeArgs = serializeArgs;
+/** Render any tool-input value as a string for matching / messages. */
+function stringifyValue(v) {
+    if (typeof v === "string")
+        return v;
+    if (typeof v === "object" && v !== null) {
+        try {
+            return JSON.stringify(v) ?? "[object]";
+        }
+        catch {
+            return "[object]";
+        }
+    }
+    return String(v);
+}
+/** Resolve a dot-path (`"a.b.c"`) within an arbitrary value, or undefined. */
+function getPath(obj, path) {
+    let cur = obj;
+    for (const part of path.split(".")) {
+        if (cur === null || typeof cur !== "object")
+            return undefined;
+        cur = cur[part];
+    }
+    return cur;
+}
+/** Does `input` satisfy every entry of `matcher`? (RegExp = pattern, else exact.) */
+function matchesArgs(input, matcher) {
+    return Object.entries(matcher).every(([key, m]) => {
+        const value = getPath(input, key);
+        return m instanceof RegExp ? m.test(stringifyValue(value)) : value === m;
+    });
+}
+/** A human-readable form of a matcher for failure messages. */
+function describeArgs(matcher) {
+    return Object.entries(matcher)
+        .map(([k, m]) => `${k}=${m instanceof RegExp ? String(m) : JSON.stringify(m)}`)
+        .join(", ");
+}
+/** Serialize a matcher for `toJSON` (RegExp → its string form). */
+function serializeArgs(matcher) {
+    const out = {};
+    for (const [k, m] of Object.entries(matcher)) {
+        out[k] = m instanceof RegExp ? String(m) : m;
+    }
+    return out;
+}
+//# sourceMappingURL=arg-match.js.map

package/dist/check.d.ts

@@ -18,6 +18,8 @@
  */
 import type { Trace } from "./harness-test.js";
 import type { HookRunResult } from "./run-hook.js";
+import { type ArgMatcher } from "./arg-match.js";
+export type { ArgMatcher };
 /** The outcome of evaluating one check against one result. */
 export interface CheckResult {
     /** Did the check hold? */
@@ -54,6 +56,28 @@ export declare function evalChecks<T>(target: T, checks: readonly Check<T>[]): C
 export declare function assertChecks<T>(target: T, checks: readonly Check<T>[]): void;
 /** The agent invoked a tool by this name (regardless of result). */
 export declare function tool(name: string): Check<Trace>;
+/**
+ * The agent used tool `name` with at least one call whose `input` matches `args`
+ * — the **argument** half of the tool-call spy. Asserts not just *that* a tool was
+ * reached but *how* it was called (the image-request body carried the style suffix,
+ * the panel spawn requested a non-expert), which a completion grader can't see.
+ *
+ * Cross-reference: ≈ promptfoo `is-valid-function-call` / DeepEval `ToolCorrectnessMetric`.
+ */
+export declare function toolWith(name: string, args: ArgMatcher): Check<Trace>;
+/**
+ * The agent did NOT use tool `name` — the **safety / negative** assertion. With
+ * `args`, only calls whose `input` matches are forbidden (so "did not push to
+ * `main`" still allows pushing elsewhere; "no paid API call" forbids it outright).
+ * This is the highest-value, most-overlooked check, and the one a
+ * completion-grading eval structurally cannot make: it sees the agent's *decision*
+ * to act, not just its final text.
+ *
+ * Cross-reference: this negative/safety assertion of a *decision not to act* has
+ * no promptfoo / DeepEval / Inspect equivalent — they grade what the agent DID,
+ * not what it correctly refrained from doing.
+ */
+export declare function notTool(name: string, args?: ArgMatcher): Check<Trace>;
 /** A skill resolved to this id (`<plugin>:<skill>`) without erroring. */
 export declare function skill(id: string): Check<Trace>;
 /** The agent's final output contains a substring / matches a RegExp. */
@@ -112,8 +136,13 @@ interface UsageTrace {
     readonly usage: {
         readonly costUsd: number;
         readonly durationMs: number;
+        /** Fresh (uncached) input tokens, billed at full input price. */
         readonly inputTokens: number;
         readonly outputTokens: number;
+        /** Tokens written to the prompt cache this run (~1.25× input price). */
+        readonly cacheCreationTokens: number;
+        /** Tokens served from the prompt cache this run (~0.1× input price). */
+        readonly cacheReadTokens: number;
     };
 }
 /** The run cost at most `maxUsd`. */
@@ -128,5 +157,27 @@ export declare function latency(opts: {
 export declare function tokens(opts: {
     max: number;
 }): Check<UsageTrace>;
-export {};
+/**
+ * The run used at most `max` **fresh (uncached) input** tokens. The honest input
+ * side of a cost claim: a skill or CLAUDE.md injection adds input every turn, so
+ * a "compression" win on output can be erased here. (Cache reads are separate —
+ * see `cacheTokens`.)
+ */
+export declare function inputTokens(opts: {
+    max: number;
+}): Check<UsageTrace>;
+/** The run used at most `max` **output** tokens — the generation side. */
+export declare function outputTokens(opts: {
+    max: number;
+}): Check<UsageTrace>;
+/**
+ * Bound the prompt-cache token classes a run uses. `maxCreation` caps tokens
+ * **written** to the cache (the ~1.25× write premium — a fresh/cold prompt);
+ * `maxRead` caps tokens **served** from cache (~0.1× input). Each constraint is
+ * checked only when provided; the check passes when every provided bound holds.
+ */
+export declare function cacheTokens(opts: {
+    maxCreation?: number;
+    maxRead?: number;
+}): Check<UsageTrace>;
 //# sourceMappingURL=check.d.ts.map

package/dist/check.js

@@ -3,6 +3,8 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.evalChecks = evalChecks;
 exports.assertChecks = assertChecks;
 exports.tool = tool;
+exports.toolWith = toolWith;
+exports.notTool = notTool;
 exports.skill = skill;
 exports.output = output;
 exports.hookFired = hookFired;
@@ -17,7 +19,11 @@ exports.judged = judged;
 exports.cost = cost;
 exports.latency = latency;
 exports.tokens = tokens;
+exports.inputTokens = inputTokens;
+exports.outputTokens = outputTokens;
+exports.cacheTokens = cacheTokens;
 const judge_js_1 = require("./judge.js");
+const arg_match_js_1 = require("./arg-match.js");
 /** Evaluate every check against a target. Pure — the shared core of `expect`
  * (strict) and `measure` (scored). */
 function evalChecks(target, checks) {
@@ -71,6 +77,63 @@ function tool(name) {
         toJSON: () => ({ kind: "tool", name }),
     };
 }
+/**
+ * The agent used tool `name` with at least one call whose `input` matches `args`
+ * — the **argument** half of the tool-call spy. Asserts not just *that* a tool was
+ * reached but *how* it was called (the image-request body carried the style suffix,
+ * the panel spawn requested a non-expert), which a completion grader can't see.
+ *
+ * Cross-reference: ≈ promptfoo `is-valid-function-call` / DeepEval `ToolCorrectnessMetric`.
+ */
+function toolWith(name, args) {
+    return {
+        kind: "toolWith",
+        eval: (t) => {
+            const calls = t.toolCalls.filter((c) => c.name === name);
+            const want = (0, arg_match_js_1.describeArgs)(args);
+            if (calls.length === 0) {
+                return no(`expected the agent to use tool "${name}" (with ${want}), but it used ${distinctToolNames(t.toolCalls)}`);
+            }
+            return calls.some((c) => (0, arg_match_js_1.matchesArgs)(c.input, args))
+                ? ok(`agent used "${name}" with ${want}`)
+                : no(`agent used "${name}" but never with ${want} (saw ${calls
+                    .map((c) => truncate((0, arg_match_js_1.stringifyValue)(c.input), 60))
+                    .join("; ")})`);
+        },
+        toJSON: () => ({ kind: "toolWith", name, args: (0, arg_match_js_1.serializeArgs)(args) }),
+    };
+}
+/**
+ * The agent did NOT use tool `name` — the **safety / negative** assertion. With
+ * `args`, only calls whose `input` matches are forbidden (so "did not push to
+ * `main`" still allows pushing elsewhere; "no paid API call" forbids it outright).
+ * This is the highest-value, most-overlooked check, and the one a
+ * completion-grading eval structurally cannot make: it sees the agent's *decision*
+ * to act, not just its final text.
+ *
+ * Cross-reference: this negative/safety assertion of a *decision not to act* has
+ * no promptfoo / DeepEval / Inspect equivalent — they grade what the agent DID,
+ * not what it correctly refrained from doing.
+ */
+function notTool(name, args) {
+    return {
+        kind: "notTool",
+        eval: (t) => {
+            const calls = t.toolCalls.filter((c) => c.name === name);
+            const offending = args
+                ? calls.filter((c) => (0, arg_match_js_1.matchesArgs)(c.input, args))
+                : calls;
+            const what = args ? `"${name}" with ${(0, arg_match_js_1.describeArgs)(args)}` : `"${name}"`;
+            const first = offending[0];
+            if (!first)
+                return ok(`agent never used ${what}`);
+            return no(`expected the agent NOT to use ${what}, but it did (${truncate((0, arg_match_js_1.stringifyValue)(first.input), 60)})`);
+        },
+        toJSON: () => args
+            ? { kind: "notTool", name, args: (0, arg_match_js_1.serializeArgs)(args) }
+            : { kind: "notTool", name },
+    };
+}
 /** A skill resolved to this id (`<plugin>:<skill>`) without erroring. */
 function skill(id) {
     return {
@@ -315,4 +378,62 @@ function tokens(opts) {
         toJSON: () => ({ kind: "tokens", max: opts.max }),
     };
 }
+/**
+ * The run used at most `max` **fresh (uncached) input** tokens. The honest input
+ * side of a cost claim: a skill or CLAUDE.md injection adds input every turn, so
+ * a "compression" win on output can be erased here. (Cache reads are separate —
+ * see `cacheTokens`.)
+ */
+function inputTokens(opts) {
+    return {
+        kind: "inputTokens",
+        eval: (t) => {
+            const v = t.usage.inputTokens;
+            return v <= opts.max
+                ? ok(`${String(v)} input tokens ≤ ${String(opts.max)}`)
+                : no(`expected ≤ ${String(opts.max)} input tokens, got ${String(v)}`);
+        },
+        toJSON: () => ({ kind: "inputTokens", max: opts.max }),
+    };
+}
+/** The run used at most `max` **output** tokens — the generation side. */
+function outputTokens(opts) {
+    return {
+        kind: "outputTokens",
+        eval: (t) => {
+            const v = t.usage.outputTokens;
+            return v <= opts.max
+                ? ok(`${String(v)} output tokens ≤ ${String(opts.max)}`)
+                : no(`expected ≤ ${String(opts.max)} output tokens, got ${String(v)}`);
+        },
+        toJSON: () => ({ kind: "outputTokens", max: opts.max }),
+    };
+}
+/**
+ * Bound the prompt-cache token classes a run uses. `maxCreation` caps tokens
+ * **written** to the cache (the ~1.25× write premium — a fresh/cold prompt);
+ * `maxRead` caps tokens **served** from cache (~0.1× input). Each constraint is
+ * checked only when provided; the check passes when every provided bound holds.
+ */
+function cacheTokens(opts) {
+    return {
+        kind: "cacheTokens",
+        eval: (t) => {
+            const created = t.usage.cacheCreationTokens;
+            const read = t.usage.cacheReadTokens;
+            if (opts.maxCreation !== undefined && created > opts.maxCreation) {
+                return no(`expected ≤ ${String(opts.maxCreation)} cache-creation tokens, got ${String(created)}`);
+            }
+            if (opts.maxRead !== undefined && read > opts.maxRead) {
+                return no(`expected ≤ ${String(opts.maxRead)} cache-read tokens, got ${String(read)}`);
+            }
+            return ok(`cache tokens within bounds (created ${String(created)}, read ${String(read)})`);
+        },
+        toJSON: () => ({
+            kind: "cacheTokens",
+            ...(opts.maxCreation !== undefined && { maxCreation: opts.maxCreation }),
+            ...(opts.maxRead !== undefined && { maxRead: opts.maxRead }),
+        }),
+    };
+}
 //# sourceMappingURL=check.js.map

package/dist/claude-code.d.ts

@@ -7,6 +7,7 @@
  */
 export * from "./adapters/claude-code/plugin-loader.js";
 export * from "./mock-model.js";
+export { claudeCodeDriver, buildClaudeArgs, parseClaudeRun, claudeAvailable, } from "./harness-test.js";
 export * from "./adapters/claude-code/dialect.js";
 export * from "./adapters/claude-code/layout.js";
 export * from "./adapters/claude-code/runtime.js";

package/dist/claude-code.js

@@ -14,6 +14,7 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
     for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
 };
 Object.defineProperty(exports, "__esModule", { value: true });
+exports.claudeAvailable = exports.parseClaudeRun = exports.buildClaudeArgs = exports.claudeCodeDriver = void 0;
 /**
  * `vigiles/claude-code` — the Claude Code-specific harness pieces a *different*
  * harness would swap out: the plugin/repo loader (reads real Claude Code plugin
@@ -23,6 +24,16 @@ Object.defineProperty(exports, "__esModule", { value: true });
  */
 __exportStar(require("./adapters/claude-code/plugin-loader.js"), exports);
 __exportStar(require("./mock-model.js"), exports);
+// The Claude-Code harness-test transport — the default driver + its argv/parse
+// helpers + the `claude` capability probe. Agnostic users never need these
+// (`runHarnessTest` defaults to the CC driver), but they're exposed here — beside
+// the Codex driver in `vigiles/codex` — for CC-specific tests/tooling. They are
+// deliberately NOT on the agnostic `vigiles/testing` surface.
+var harness_test_js_1 = require("./harness-test.js");
+Object.defineProperty(exports, "claudeCodeDriver", { enumerable: true, get: function () { return harness_test_js_1.claudeCodeDriver; } });
+Object.defineProperty(exports, "buildClaudeArgs", { enumerable: true, get: function () { return harness_test_js_1.buildClaudeArgs; } });
+Object.defineProperty(exports, "parseClaudeRun", { enumerable: true, get: function () { return harness_test_js_1.parseClaudeRun; } });
+Object.defineProperty(exports, "claudeAvailable", { enumerable: true, get: function () { return harness_test_js_1.claudeAvailable; } });
 __exportStar(require("./adapters/claude-code/dialect.js"), exports);
 __exportStar(require("./adapters/claude-code/layout.js"), exports);
 __exportStar(require("./adapters/claude-code/runtime.js"), exports);

package/dist/cli.js

@@ -32,6 +32,7 @@ const compose_js_1 = require("./core/compose.js");
 const compile_generator_js_1 = require("./core/compile-generator.js");
 const action_gate_js_1 = require("./action-gate.js");
 const agent_runtime_js_1 = require("./adapters/claude-code/agent-runtime.js");
+const tool_intercept_js_1 = require("./tool-intercept.js");
 const refs_js_1 = require("./core/refs.js");
 const mcp_js_1 = require("./core/mcp.js");
 const skill_runtime_js_1 = require("./adapters/claude-code/skill-runtime.js");
@@ -909,8 +910,8 @@ async function runLint(restArgs, flags, config) {
         }
     }
     // 7b. Untested-surface check — skills/agents/hooks shipping without a test or
-    // eval. Warning by default (a nudge, exit 0); set rules.untested-surface to
-    // "error" to gate CI. See src/test-coverage.ts and docs/rules/untested-surface.md.
+    // eval. Warning by default (a nudge, exit 0); set rules.untested-{skill,agent,
+    // hook} to "error" to gate CI. See src/test-coverage.ts and docs/rules/.
     const untested = checkUntestedSurfaces(config, silent);
     // 8. Validate vigiles builder calls inside markdown code blocks. Default
     // is to validate every ref; illustrative blocks opt out via
@@ -924,6 +925,16 @@ async function runLint(restArgs, flags, config) {
             console.log(`  ${line}`);
         }
     }
+    // Per-line GitHub annotations for each broken doc ref — each carries file+line,
+    // so GitHub renders it INLINE on the PR diff (not just in the summary blob).
+    // Previously this check reported to stdout only; the inline/spec checks already
+    // annotate per-line, so this closes the gap that left doc-ref findings invisible
+    // on the PR. CI-only (isGitHubActions); skipped under --json/--summary.
+    if (isGitHubActions() && !silent) {
+        for (const e of docRefReport.errors) {
+            ghAnnotate("error", `${e.kind}("${e.value}") — ${e.message}`, e.file, e.line);
+        }
+    }
     // 9. Verify code-shaped symbol references live (see src/refs.ts).
     const symbolRefErrors = verifyMarkdownSymbols(files, silent);
     // 10. Verify `vigiles:mcp server#tool` marks against live MCP servers
@@ -1877,29 +1888,47 @@ function checkIntegrityForFiles(files, severity, silent) {
     return severity === "error" ? errorCount : 0;
 }
 /**
- * Apply the `untested-surface` rule: find skills/agents/hooks with no test or
- * eval (see src/test-coverage.ts). Returns the raw untested count plus the
- * severity-gated error count — "warn" prints but never fails CI (errors=0),
- * "error" fails (exit 2), mirroring the integrity check.
+ * Apply the per-kind `untested-skill` / `untested-agent` / `untested-hook` rules:
+ * find skills/agents/hooks with no test or eval (see src/test-coverage.ts). Each
+ * kind is gated by its OWN rule severity — a kind set to `false` is not scanned;
+ * "warn" prints but never fails CI; "error" fails (exit 2). Returns the raw
+ * untested count plus the severity-gated error count.
  */
 function checkUntestedSurfaces(config, silent) {
-    const severity = (0, types_js_1.ruleSeverity)(config?.rules["untested-surface"]);
-    if (!severity)
+    const rules = config?.rules;
+    const skillSev = (0, types_js_1.ruleSeverity)(rules?.["untested-skill"]);
+    const agentSev = (0, types_js_1.ruleSeverity)(rules?.["untested-agent"]);
+    const hookSev = (0, types_js_1.ruleSeverity)(rules?.["untested-hook"]);
+    if (!skillSev && !agentSev && !hookSev)
         return { untested: 0, errors: 0 };
-    const opts = (0, types_js_1.ruleOptions)(config?.rules["untested-surface"]);
-    const report = (0, test_coverage_js_1.findUntestedSurfaces)({ basePath: process.cwd(), ...opts });
+    const sevFor = (kind) => kind === "skill" ? skillSev : kind === "agent" ? agentSev : hookSev;
+    // Test-discovery options (testGlobs/exclude) are shared; merge them from
+    // whichever of the three rules carries them.
+    const opts = {
+        ...(0, types_js_1.ruleOptions)(rules?.["untested-skill"]),
+        ...(0, types_js_1.ruleOptions)(rules?.["untested-agent"]),
+        ...(0, types_js_1.ruleOptions)(rules?.["untested-hook"]),
+    };
+    const report = (0, test_coverage_js_1.findUntestedSurfaces)({
+        basePath: process.cwd(),
+        skills: skillSev !== false,
+        agents: agentSev !== false,
+        hooks: hookSev !== false,
+        testGlobs: opts.testGlobs,
+        exclude: opts.exclude,
+    });
     if (!silent) {
         console.log("\nUntested surfaces:\n");
         for (const line of (0, test_coverage_js_1.formatUntestedReport)(report).split("\n")) {
             console.log(`  ${line}`);
         }
         for (const s of report.untested) {
-            ghAnnotate(severity === "error" ? "error" : "warning", `${s.kind} ${s.path} ships without a test or eval`, s.path);
+            ghAnnotate(sevFor(s.kind) === "error" ? "error" : "warning", `${s.kind} ${s.path} ships without a test or eval`, s.path);
         }
     }
     return {
         untested: report.untested.length,
-        errors: severity === "error" ? report.untested.length : 0,
+        errors: report.untested.filter((s) => sevFor(s.kind) === "error").length,
     };
 }
 /**
@@ -2081,6 +2110,18 @@ function handleRunScripts(kind, args, restArgs) {
     // Harness/eval scripts may be authored in JS or TS (see run-scripts.ts).
     const defaultGlob = (0, run_scripts_js_1.scriptGlob)(kind === "test" ? "harness" : "eval");
     const files = (0, run_scripts_js_1.discoverScripts)(restArgs, defaultGlob, cwd);
+    // `--min=N`: a CI gate asserts at least N scripts actually RAN — so a bad path,
+    // a renamed file, or a glob that matched nothing fails LOUD instead of passing
+    // green with zero evals executed. Default 0 (off) keeps local runs ergonomic.
+    const minFlag = args.find((a) => a.startsWith("--min="));
+    const minRequired = minFlag
+        ? Math.max(0, Number.parseInt(minFlag.split("=")[1] ?? "", 10) || 0)
+        : 0;
+    if (files.length < minRequired) {
+        console.error(`✗ vigiles ${kind}: --min=${String(minRequired)} but only ${String(files.length)} ${kind} file(s) matched — ` +
+            "evals never executed (check the paths/globs, or that the run was reached).");
+        process.exit(1);
+    }
     if (files.length === 0) {
         console.log(`No ${defaultGlob} files found.`);
         return;
@@ -2091,6 +2132,10 @@ function handleRunScripts(kind, args, restArgs) {
     if (kind === "test" && !(0, harness_test_js_1.claudeAvailable)()) {
         console.log("ℹ `claude` CLI not found — unit-tier tests run; tests that need it report SKIPPED.\n");
     }
+    // `--trials=N` (a run knob: cost/precision, doesn't change WHAT is measured) is
+    // forwarded to scripts via env. The MODEL is deliberately NOT a CLI/env knob —
+    // it's part of the measurement definition, so it belongs in the spec
+    // (`model` / `minModel`), version-controlled, not a hidden override.
     const trialsFlag = args.find((a) => a.startsWith("--trials="));
     const env = {};
     if (trialsFlag)
@@ -2117,7 +2162,7 @@ function printUsage(command) {
     console.log("  vigiles compile [files...]     Compile .spec.ts → .md");
     console.log("  vigiles lint [files...]        Verify references, find gaps in instruction files");
     console.log("  vigiles test [files...]        Run *.harness.mjs deterministic harness tests");
-    console.log("  vigiles eval [files...]        Run *.eval.mjs real-model harness evals (--trials=N)");
+    console.log("  vigiles eval [files...]        Run *.eval.mjs real-model harness evals (--trials=N, --min=N, --no-skip)");
     console.log("");
     console.log("Examples:");
     console.log("  vigiles init                 Auto-detect project, create specs, wire CI");
@@ -2231,7 +2276,7 @@ function skillStartCommand(target) {
  * PreToolUse-hook entrypoint: enforce the active subagent's allowed-tools
  * contract. Reads the tool event on stdin, parses the active agent's compiled
  * `.md` tool rail, and blocks (exit 2 + reason on stderr) any tool outside it —
- * the deterministic boundary `tools:` alone can't provide (Claude Code #54898).
+ * the deterministic boundary `tools:` alone can't provide (Claude Code #4740/#21460, SDK #172).
  */
 function agentHookCommand() {
     let raw = "";
@@ -2256,6 +2301,30 @@ function agentHookCommand() {
         process.exit(2);
     }
 }
+/**
+ * `vigiles intercept-tool-hook` — the PreToolUse interception hook for the
+ * tool-call spy. Reads the intercept list from `VIGILES_INTERCEPT_TOOLS`, decides
+ * whether the called tool should be intercepted, and if so denies the real
+ * execution (exit 2) with a block message — the call is intercepted (prevented),
+ * NOT executed. Allowing (return) lets the tool run for real. The model still
+ * emits the `tool_use`, so its arguments land in the Trace for `toolWith` /
+ * `notTool` to assert on. See src/tool-intercept.ts.
+ */
+function interceptToolHookCommand() {
+    let raw = "";
+    try {
+        raw = (0, node_fs_1.readFileSync)(0, "utf-8");
+    }
+    catch {
+        /* no stdin */
+    }
+    const intercepts = (0, tool_intercept_js_1.parseIntercepts)(process.env[tool_intercept_js_1.INTERCEPT_TOOLS_ENV] ?? "");
+    const decision = (0, tool_intercept_js_1.interceptHookDecision)(raw, intercepts);
+    if (decision.intercept) {
+        console.error(decision.denyReason);
+        process.exit(2);
+    }
+}
 /** Mark a subagent active so the PreToolUse hook enforces its tool contract. */
 function agentStartCommand(target) {
     if (!target) {
@@ -2289,6 +2358,9 @@ function handleSkillCommand(command, restArgs) {
         case "agent-hook":
             agentHookCommand();
             return true;
+        case "intercept-tool-hook":
+            interceptToolHookCommand();
+            return true;
         case "action-hook":
             actionHookCommand();
             return true;