vigiles 2.3.0 → 2.4.0

package/README.md

@@ -21,6 +21,29 @@
 
 ---
 
+<details>
+<summary><b>Contents</b></summary>
+
+**Pillar 1 — verify your instruction files** · references your CLAUDE.md makes that a linter, the filesystem, and package.json can prove
+
+- Three adoption levels: [inline comments](#level-0--inline-comments-30-seconds-no-new-files) → [YAML frontmatter](#level-1--yaml-frontmatter-editor-autocomplete-still-no-typescript) → [typed spec](#level-2--typed-spec-compiler-grade-guarantees)
+- [What changes with vigiles](#what-changes-with-vigiles)
+- [Quick start](#quick-start)
+- [Three rule types](#three-rule-types) — `enforce` / `guidance` / `guard`
+- [Verified references](#verified-references) — `file` / `cmd` / `symbol` / `ref`
+
+**Pillar 2 — [test your Claude Code harness](#test-your-claude-code-harness)** · eval whether your hooks, skills, and CLAUDE.md actually change what the agent does
+
+- [Level 1 — unit-test a hook (no AI)](#level-1--test-a-hook-by-itself-no-ai-milliseconds)
+- [Level 2 — does it fire in a real session?](#level-2--does-it-fire-in-a-real-session-free-scripted-ai)
+- [Level 3 — does it change behaviour?](#level-3--does-it-change-what-claude-does-real-ai-occasional)
+- [Test skills for real + assert on actions](#test-your-skills-for-real--and-assert-on-what-claude-did)
+- [Run them in CI](#run-them-in-ci)
+
+**More** — [CLI & CI](#cli--ci) · [Skills](#skills) · [Maturity levels](#maturity-levels) · [Related tools](#related-tools)
+
+</details>
+
 Your CLAUDE.md lies to your agent. Here's the fix.
 
 Hand-written CLAUDE.md files rot silently. Here's what a typical one looks like:
@@ -47,6 +70,8 @@ Reads fine. Four things are wrong:
 
 The agent reads this, trusts it, and writes code based on stale claims nobody verified. vigiles **verifies the references in your instruction files** — that each linter rule exists and is enabled, that every file path and script is real, and that referenced **code symbols** (functions, classes, constants) actually exist in the files that define them — and meets you at whatever commitment level you want.
 
+> **See it in 60 seconds:** `npm run demo` runs `vigiles audit` against a deliberately-broken instruction file and catches a renamed symbol and a missing MCP tool (_"did you mean `purge`?"_), while the truthful references pass silently. [examples/demo →](examples/demo)
+
 Three levels. Each is independently useful; adopt as far up as you like.
 
 ### Level 0 — inline comments (30 seconds, no new files)
@@ -168,15 +193,7 @@ npx vigiles init
 
 The wizard auto-detects your project, creates a spec, scans your linters, compiles to markdown, adds a CI step, and installs Claude Code hooks. After install: the agent edits the spec (hooks block direct CLAUDE.md edits), the spec auto-compiles on save, and `vigiles audit` catches drift in CI.
 
-Start with `guidance()` rules (zero config). When you're ready, run `/strengthen` to find rules that can be upgraded to compile-verified `enforce()`. Already have a hand-written CLAUDE.md? The wizard detects it and offers migration.
-
-| Flag                 | Effect                                                |
-| -------------------- | ----------------------------------------------------- |
-| `--strict`           | Sets require-spec and require-skill-spec to `"error"` |
-| `--target=AGENTS.md` | Creates AGENTS.md spec instead of CLAUDE.md           |
-| `--no-gha`           | Skip adding CI step to GHA workflow                   |
-
-Works the same for humans and agents — fully non-interactive. [Agent setup guide →](docs/agent-setup.md) | [Agent workflows →](docs/agent-workflows.md)
+Start with `guidance()` rules (zero config). When you're ready, run `/strengthen` to find rules that can be upgraded to compile-verified `enforce()`. Already have a hand-written CLAUDE.md? The wizard detects it and offers migration. Flags (`--strict`, `--target=AGENTS.md`, `--no-gha`) and non-interactive agent usage are in the [CLI reference](docs/cli.md) and [agent setup guide](docs/agent-setup.md).
 
 ## Three Rule Types
 
@@ -198,20 +215,7 @@ Supports ESLint, Stylelint, Ruff, Clippy, Pylint, RuboCop, and Cedar policies. [
 "research-first": guidance("Google unfamiliar APIs first."),
 ```
 
-**`guard()`** — reactive: runs a command when watched files change. One declaration emits hooks for every supported system (Claude Code PostToolUse, husky pre-commit, etc.). Eliminates copy-pasting the same trigger across `.claude/settings.json`, `.husky/`, and CI configs.
-
-```typescript
-"recompile-specs": guard(
-  { watch: "*.spec.ts", run: "npx vigiles compile" },
-  "Recompile instruction files when any spec changes.",
-),
-"regen-types": guard(
-  { watch: ["eslint.config.*", "package.json"], run: "npx vigiles generate-types" },
-  "Regenerate types when linter config or deps change.",
-),
-```
-
-Same monotonicity guarantees as `enforce()` — guards can't be silently removed.
+**`guard()`** — reactive: runs a command when watched files change (e.g. `*.spec.ts` → `npx vigiles compile`). One declaration emits hooks for every supported system (Claude Code PostToolUse, husky pre-commit, etc.) — no copy-pasting the same trigger across `.claude/settings.json`, `.husky/`, and CI. Same monotonicity guarantees as `enforce()`. [Full spec format →](docs/spec-format.md)
 
 ## Verified References
 
@@ -234,220 +238,193 @@ export default claude({
 });
 ```
 
-Skill specs use the same helpers for verified references inside instructions. [Full spec format →](docs/spec-format.md)
+There's a small family of inline **marks** that `audit` checks, each binding a reference to its real source:
 
-### Symbol references (cross-language)
+- `` `vigiles:symbol file#name` `` — the named file actually **defines** that symbol (function, class, method, constant), parsed with [ast-grep](https://ast-grep.github.io) across **JS/TS, Python, Ruby, Rust, and CSS**. Rename it and `audit` fails; in markdown mode the `refs-hook` **forces the mark**, blocking edits that leave a code reference bare. [Details →](research/symbol-verification.md)
+- `` `vigiles:mcp server#tool` `` — the referenced **MCP tool exists** on its server. `audit` reads `.mcp.json`, starts the server, lists its tools, and flags a renamed/removed one with a "did you mean" — catching e.g. the GitHub MCP server renaming `create_issue` → `issue_write`, which otherwise fails silently.
 
-`symbol("file", "name")` (and the markdown mark `` `vigiles:symbol file#name` ``) verify that the named file actually **defines** the symbol — a function, class, method, or constant — parsed with [ast-grep](https://ast-grep.github.io) across **JS/TS, Python, Ruby, Rust, and CSS**. Rename the function and `audit` fails; no project-wide index, no autoloader guessing — it parses the one named file.
+**Typo-safe at authoring time, too.** `vigiles generate-types` emits a `.vigiles/generated.d.ts` so `enforce("eslint/no-consolee")` red-squiggles in your editor; `generate-schema` gives Level 1 frontmatter the same via your YAML language server. Both have `--check` CI freshness modes. [How it works →](docs/linter-support.md#generate-types)
 
-In markdown mode the `refs-hook` (PostToolUse) **forces the mark**: it blocks an edit that leaves a code reference bare, telling the agent to write `` `vigiles:symbol path#name` `` or opt out with `<!-- vigiles:ignore -->`. The harness makes the agent mark its references at write time, with full context; `audit` re-verifies them. [Symbol verification →](research/symbol-verification.md)
-
-## Type-Safe Rule References
-
-`vigiles generate-types` scans your linter configs and emits `.vigiles/generated.d.ts`. With this file, `enforce("eslint/no-consolee")` is a red squiggle in your editor — a typo caught at authoring time, not a runtime surprise. Without it, everything falls back to broad types and still works.
-
-```bash
-$ npx vigiles generate-types
-  eslint: 64 enabled rules  |  ruff: 12  |  npm scripts: 5  |  project files: 42
-✓ Generated .vigiles/generated.d.ts
-```
-
-Commit the file to git. CI can verify it's fresh: `npx vigiles generate-types --check`. [How it works →](docs/linter-support.md#generate-types)
-
-For markdown frontmatter (Level 1), `vigiles generate-schema` gives the same authoring-time feedback without TypeScript: it emits a JSON Schema from your enabled rules, and your editor's YAML language server autocompletes rule names and squiggles typos. CI freshness check: `npx vigiles generate-schema --check`.
-
-## CLI
+## Test your Claude Code harness
 
-```bash
-npx vigiles init [--target=X.md]    # Scaffold a spec (runs full setup wizard by default)
-npx vigiles compile [files...]      # Compile .spec.ts → .md
-npx vigiles audit [files...]        # Verify hashes + inline/frontmatter/spec rules + symbols + coverage
-npx vigiles refs <file.md>          # Check the symbol references in an instruction file
-npx vigiles test [files...]         # Run *.harness.mjs deterministic harness tests (no API key)
-npx vigiles eval [files...]         # Run *.eval.mjs real-model harness evals (--trials=N)
-npx vigiles generate-types          # Emit .d.ts from project state (for spec mode)
-npx vigiles generate-types --check  # Verify .d.ts is up to date
-npx vigiles generate-schema         # Emit JSON Schema for vigiles: frontmatter (Level 1)
-npx vigiles generate-schema --check # Verify schema.json is up to date
-```
+You wrote hooks, a skill, a CLAUDE.md rule — how do you know they work, beyond
+running Claude and eyeballing it? vigiles ships a library to **test the harness
+itself**, at three levels, cheapest first. It's plain async functions, so it
+drops into **node:test / vitest / jest**, or a zero-setup `vigiles test`.
 
-## GitHub Action
+### Level 1 — test a hook by itself (no AI, milliseconds)
 
-```yaml
-- uses: zernie/vigiles@main # runs `audit` by default
-- uses: zernie/vigiles@main
-  with:
-    command: compile # compile specs in CI
-```
+A hook is just a process that's handed a "Claude is about to do X" event and
+answers block/allow. Hand it a fake event and check the answer — no `claude`, no
+model, and **every** event type is reachable (incl. Edit/Write, PreCompact,
+SessionEnd):
 
-To verify generated types are fresh in CI:
+```typescript
+import { runHook } from "vigiles/run-hook";
 
-```yaml
-- run: npx vigiles generate-types --check
+const r = runHook(guardCommand, {
+  hook_event_name: "PreToolUse",
+  tool_name: "Bash",
+  tool_input: { command: "git commit --no-verify" },
+});
+assert(r.blocked); // exit 2 / decision:"block" / permissionDecision:"deny"
 ```
 
-## Claude Code Plugin
-
-**Install the plugin.** Without it, you're responsible for manually running `compile` and `generate-types`. With it, the agent works with fresh instruction files automatically.
+The same shape governs **MCP tools** — the dominant real MCP test — with no
+server running, because the hook only sees the tool _name_:
 
-```bash
-npx skills add zernie/vigiles
+```typescript
+// block the destructive github-MCP tool; read-only ones pass
+runHook(guard, {
+  hook_event_name: "PreToolUse",
+  tool_name: "mcp__github__merge_pull_request",
+  tool_input: { pull_number: 42 },
+}).blocked; // true
 ```
 
-The plugin provides two hooks:
-
-- **PreToolUse** (Edit/Write) — blocks direct edits to compiled `.md` files and redirects the agent to the `.spec.ts` source
-- **PostToolUse** (Edit/Write) — auto-runs `generate-types` on linter config changes, `compile` on `.spec.ts` changes
-
-## Validation
-
-`vigiles audit` validates instruction files with four rules:
+### Level 2 — does it fire in a real session? (free, scripted "AI")
 
-| Rule                                                     | Default  | What it checks                                                               |
-| -------------------------------------------------------- | -------- | ---------------------------------------------------------------------------- |
-| [`require-spec`](docs/rules/require-spec.md)             | `"warn"` | Every CLAUDE.md/AGENTS.md has a spec, inline rule, or `vigiles:` frontmatter |
-| [`require-skill-spec`](docs/rules/require-skill-spec.md) | `"warn"` | Every SKILL.md has a `.spec.ts`                                              |
-| [`integrity`](docs/rules/integrity.md)                   | `"warn"` | Compiled markdown wasn't hand-edited (SHA-256 check)                         |
-| [`coverage`](docs/rules/coverage.md)                     | `false`  | Spec covers enough of the project surface                                    |
+Right logic ≠ wired in correctly. `runHarnessTest` runs the **real** `claude`
+against a **scripted mock model** you control — your hooks fire for real, the
+agent's turns are fixed, no API key, same result every time. Covers the
+governance shapes: SessionStart, Stop, UserPromptSubmit, and Bash **and
+Edit/Write** Pre/PostToolUse.
 
-Configure in `.vigilesrc.json`:
+```typescript
+import { runHarnessTest, scriptModel } from "vigiles/harness-test";
 
-```json
-{
-  "rules": {
-    "require-spec": "error",
-    "integrity": "error",
-    "coverage": ["warn", { "scripts": 50, "linterRules": 5 }]
-  }
-}
+const r = await runHarnessTest({
+  settings: {
+    hooks: {
+      Stop: [
+        { hooks: [{ type: "command", command: "test -f DONE || exit 2" }] },
+      ],
+    },
+  },
+  model: scriptModel([
+    { text: "I'm done" }, // tries to stop → blocked (no DONE)
+    { tool: "Bash", input: { command: "touch DONE" } },
+    { text: "now done" },
+  ]),
+});
+assert(JSON.parse(r.stdout).num_turns > 1); // the Stop hook forced more work
 ```
 
-Disable per-file with `<!-- vigiles-disable require-spec -->` at the top of the markdown.
-
-## Skills
-
-Install with [Vercel Skills](https://github.com/vercel-labs/skills): `npx skills add zernie/vigiles`
-
-| Skill                  | What it does                                                            |
-| ---------------------- | ----------------------------------------------------------------------- |
-| `strengthen`           | Upgrade `guidance()` → `enforce()` using linter-specific reference docs |
-| `edit-spec`            | Edit a spec file — guided workflow with compile step                    |
-| `migrate-to-spec`      | Convert a hand-written CLAUDE.md to a typed `.spec.ts`                  |
-| `generate-rule`        | Add a new `enforce()` / `guidance()` rule to a spec                     |
-| `pr-to-lint-rule`      | Turn a recurring PR review comment into a lint rule + spec entry        |
-| `enforce-rules-format` | Validate all rules have enforcement classification                      |
-| `audit-feedback-loop`  | Score your repo's feedback loop maturity                                |
-
-## Test your Claude Code harness
-
-vigiles also ships a library for **testing the harness itself** — your hooks,
-settings, skills, and instruction files. `Agent = Model + Harness`; this tests
-the harness, at three levels.
+### Level 3 — does it change what Claude does? (real AI, occasional)
 
-**Evals — does my change actually move agent behaviour?** Define a fixture, a set
-of **arms** (a hook on vs off, with/without a CLAUDE.md rule), a task, and a
-metric; `runEval` drives the real `claude` CLI N trials per arm and aggregates.
+`runEval` runs the **real** model N times with your change **on vs off** and
+reports the gap. Costs tokens, so you run it now and then — not on every save:
 
 ```typescript
 import { runEval, formatEvalReport } from "vigiles/eval";
 
 const report = await runEval({
-  fixture: { "src/billing.ts": "export function chargeCard() {}" },
-  arms: {
-    vanilla: {},
-    gated: { settings: { hooks: { PostToolUse: [refsHook] } } },
-  },
+  arms: { off: {}, on: { settings: { hooks: { PostToolUse: [refsHook] } } } },
   task: "Document chargeCard in SKILL.md, referencing it by name.",
   measure: (ctx) => ({
     marked: ctx.sh("grep -c vigiles:symbol SKILL.md") !== "0",
   }),
   trials: 6,
 });
-console.log(formatEvalReport(report)); //  vanilla marked=0.00   gated marked=0.50
+console.log(formatEvalReport(report)); // off marked=0.00   on marked=0.50
 ```
 
-**Deterministic tests — does my hook fire correctly?** No API key, no cost.
-`runHarnessTest` runs real `claude` against a **scripted mock model**
-(`vigiles/mock-model`), so your real hooks fire but the agent's turns are fixed.
+### Test your skills for real — and assert on what Claude _did_
+
+Install a plugin the way Claude actually does (`pluginDir` → `--plugin-dir`) so
+its **skills genuinely activate**, then assert on the agent's _actions_, not a
+stdout grep:
 
 ```typescript
-import { runHarnessTest, scriptModel } from "vigiles/harness-test";
+import { assertSkillResolved, assertToolNotUsed } from "vigiles/harness-assert";
 
 const r = await runHarnessTest({
-  settings: {
-    hooks: {
-      Stop: [
-        {
-          hooks: [
-            {
-              type: "command",
-              command: "test -f DONE || { echo 'not done' >&2; exit 2; }",
-            },
-          ],
-        },
-      ],
-    },
-  },
+  pluginDir: "./my-plugin",
+  transcript: true, // populate r.toolCalls
+  allowedTools: ["Read", "Write", "Bash", "Skill"],
   model: scriptModel([
-    { text: "I'm done" }, // tries to stop → blocked
-    { tool: "Bash", input: { command: "touch DONE" } },
-    { text: "now done" },
+    { tool: "Skill", input: { skill: "my-plugin:greet" } },
+    { text: "ok" },
   ]),
 });
-assert(JSON.parse(r.stdout).num_turns > 1); // the Stop hook forced more work
+assertSkillResolved(r, "my-plugin:greet"); // the skill fired, no error
+assertToolNotUsed(r, /^mcp__github__merge/); // the safety negative: the scary tool was never called
 ```
 
-The deterministic tier is reliable for **SessionStart, Stop, UserPromptSubmit,
-and Bash PreToolUse/PostToolUse** hooks — the governance/policy shapes most real
-plugins use; Edit/Write tool-event hooks are headless-gated, so test those at the
-unit tier or via the eval tier.
+`assertToolNotUsed` is how you test a safety rule **honestly** — _proving_ the
+dangerous tool was never used, which "the file looks unchanged" can't. It works
+on **real third-party plugins** too: the suite confirms real `obra/superpowers`
+and `wshobson/agents` skills resolve this way, with no markers injected.
 
-**Unit-test a hook — no `claude` at all.** A hook is just a process: `runHook`
-pipes an event JSON to its stdin and reports the block/allow decision —
-milliseconds, and the only tier that reaches **every** event (incl. Edit/Write,
-PreCompact, SessionEnd, which the deterministic mock can't trigger).
+### Run them in CI
 
-```typescript
-import { runHook } from "vigiles/run-hook";
+`vigiles test` runs `*.harness.mjs` files (free, no key); `vigiles eval` runs
+`*.eval.mjs` files (real model). Point a test at a whole plugin (or `"./"` for
+your repo) to load **what ships** — hooks (with `${CLAUDE_PLUGIN_ROOT}`
+resolved), CLAUDE.md, skills, subagents, commands — and `loadPlugin().warnings`
+flags anything only a real model can drive, so you never silently test an empty
+machine.
 
-const r = runHook(guardCommand, {
-  hook_event_name: "PreToolUse",
-  tool_name: "Bash",
-  tool_input: { command: "git commit --no-verify" },
-});
-assert(r.blocked); // exit 2, decision:"block", or permissionDecision:"deny"
+```bash
+npx vigiles test examples/harness/policy-gate.harness.mjs
+npx vigiles eval --trials=6 examples/harness/skill-outcome.eval.mjs
 ```
 
-**Run them as a CI command.** `vigiles test` discovers `*.harness.mjs` files
-(deterministic, no API key) and `vigiles eval` discovers `*.eval.mjs` files
-(real model). Canonical, real-plugin-shaped examples to copy:
+### What's covered today — surface × tier
+
+| Surface                                                       | Unit / static                | Integration (no API key)    | Eval (real model) |
+| ------------------------------------------------------------- | ---------------------------- | --------------------------- | ----------------- |
+| Hooks — Bash / SessionStart / Stop / UserPromptSubmit         | ✅ logic                     | ✅ fires                    | ✅                |
+| Hooks — Edit / Write                                          | ✅ logic                     | ✅ fires                    | ✅                |
+| Hooks — PreCompact / Notification / SessionEnd / SubagentStop | ✅ logic                     | — (mock can't trigger)      | 🟡                |
+| CLAUDE.md / instructions                                      | ✅ refs                      | 🟡 present, not behaviour   | ✅ behaviour      |
+| Skills                                                        | 🟡 refs                      | ✅ resolves via `pluginDir` | ✅ activation     |
+| Subagents (`agents/`)                                         | 🟡 refs                      | 🔴 hard                     | ✅ via Task       |
+| Slash commands (`commands/`)                                  | 🟡 refs                      | 🟡 needs prompt capture     | ✅ via `/cmd`     |
+| MCP servers                                                   | ✅ tool refs (`vigiles:mcp`) | 🔴                          | 🔴                |
+| settings.json                                                 | 🟡 assert merged             | ✅ applied                  | ✅                |
 
-- [`examples/harness/policy-gate.harness.mjs`](examples/harness/policy-gate.harness.mjs) — a `PreToolUse` Bash policy gate (block `git commit --no-verify`) and a `SessionStart` setup hook, deterministic.
-- [`examples/harness/skill-outcome.eval.mjs`](examples/harness/skill-outcome.eval.mjs) — does a skill change the agent's output? (the question you ask of any `SKILL.md`).
+✅ shipped · 🟡 partial · 🔴 gap · — n/a. Full detail + roadmap: [`research/harness-testing-coverage-matrix.md`](research/harness-testing-coverage-matrix.md).
+
+[Full guide → `docs/harness-testing.md`](docs/harness-testing.md) · [benchmarks](research/benchmarks-runtime-gates.md).
+
+## CLI & CI
 
 ```bash
-npx vigiles test examples/harness/policy-gate.harness.mjs
-npx vigiles eval --trials=6 examples/harness/skill-outcome.eval.mjs
+npx vigiles init        # Scaffold a spec (full setup wizard)
+npx vigiles compile     # Compile .spec.ts → .md
+npx vigiles audit       # Verify hashes + inline/frontmatter/spec rules + symbols + coverage
+npx vigiles test        # Run *.harness.mjs deterministic harness tests (no API key)
+npx vigiles eval        # Run *.eval.mjs real-model harness evals (--trials=N)
 ```
 
-**Test the whole machine.** Point `plugin` at a plugin (or `"./"` for your repo)
-and the real harness — hooks (with `${CLAUDE_PLUGIN_ROOT}` resolved), CLAUDE.md,
-skills, subagents and commands — is loaded into the sandbox, so you test what
-ships, not a retyped subset. `loadPlugin(...).warnings` flags surfaces only a
-real model can drive (subagents, slash commands, MCP), so loading a whole plugin
-never silently tests an empty machine. The library is plain async functions, so
-it runs in **node:test, vitest, or jest** unchanged (shared `expect.extend`
-matchers for the latter two).
-
-[Full guide → `docs/harness-testing.md`](docs/harness-testing.md). The design
-rationale and a coverage assessment against real plugins (protect-mcp,
-obra/superpowers, block-no-verify, 156 wshobson skills) are in
-[`research/harness-testing.md`](research/harness-testing.md); findings from
-running this harness in anger live in [`research/benchmarks-runtime-gates.md`](research/benchmarks-runtime-gates.md).
+`vigiles audit` enforces four rules — `require-spec`, `require-skill-spec`, `integrity`, `coverage` — configurable in `.vigilesrc.json`. The GitHub Action runs `audit` by default; the Claude Code plugin (`npx skills add zernie/vigiles`) adds the Pre/PostToolUse hooks that block direct `.md` edits and auto-compile specs. [Full CLI, Action, plugin & validation reference →](docs/cli.md)
+
+## Skills
+
+Install with [Vercel Skills](https://github.com/vercel-labs/skills): `npx skills add zernie/vigiles`
+
+<details>
+<summary><b>The 7 skills</b></summary>
+
+| Skill                  | What it does                                                            |
+| ---------------------- | ----------------------------------------------------------------------- |
+| `strengthen`           | Upgrade `guidance()` → `enforce()` using linter-specific reference docs |
+| `edit-spec`            | Edit a spec file — guided workflow with compile step                    |
+| `migrate-to-spec`      | Convert a hand-written CLAUDE.md to a typed `.spec.ts`                  |
+| `generate-rule`        | Add a new `enforce()` / `guidance()` rule to a spec                     |
+| `pr-to-lint-rule`      | Turn a recurring PR review comment into a lint rule + spec entry        |
+| `enforce-rules-format` | Validate all rules have enforcement classification                      |
+| `audit-feedback-loop`  | Score your repo's feedback loop maturity                                |
+
+</details>
 
 ## Maturity Levels
 
-From [Feedback Loop Is All You Need](https://zernie.com/blog/feedback-loop-is-all-you-need):
+From [Feedback Loop Is All You Need](https://zernie.com/blog/feedback-loop-is-all-you-need): **Vibes → Guardrails → Architecture as Code → The Organism**.
+
+<details>
+<summary>What each level means</summary>
 
 | Level | Name                 | What it means                                                       |
 | ----- | -------------------- | ------------------------------------------------------------------- |
@@ -456,20 +433,16 @@ From [Feedback Loop Is All You Need](https://zernie.com/blog/feedback-loop-is-al
 | 2     | Architecture as Code | Custom lint rules + enforced CLAUDE.md                              |
 | 3     | The Organism         | CI + custom rules + visual tests + observability + scheduled agents |
 
-## Output Targets
-
-Specs compile to `CLAUDE.md` by default. Set `target: "AGENTS.md"` or `target: ["CLAUDE.md", "AGENTS.md"]` for multiple outputs from one spec. For non-markdown formats (`.cursorrules`, Copilot), use [rule-porter](https://github.com/nichochar/rule-porter) or [rulesync](https://github.com/dyoshikawa/rulesync) to convert. [Spec format →](docs/spec-format.md)
+</details>
 
 ## Related Tools
 
-vigiles doesn't try to do everything. It owns one thing: compile-time verification of typed specs against real linter configs, filesystems, and package.json. Everything else, compose:
+vigiles owns one thing: compile-time verification of typed specs against real linter configs, filesystems, and package.json, plus testing the harness those specs describe. Everything else it composes with rather than replaces — architectural linters ([ast-grep](https://ast-grep.github.io/), [Dependency Cruiser](https://github.com/sverweij/dependency-cruiser)) referenced via `enforce()`, file-sync tools ([Ruler](https://github.com/intellectronica/ruler), [rulesync](https://github.com/dyoshikawa/rulesync)) that distribute the compiled output, and markdown/prose linters that check a different layer. [How vigiles composes with each, and why runtime-LLM rule checkers are the opposite paradigm →](docs/related-tools.md)
+
+## Documentation
 
-- **Architectural linting** — [ast-grep](https://ast-grep.github.io/), [Dependency Cruiser](https://github.com/sverweij/dependency-cruiser), [Steiger](https://github.com/feature-sliced/steiger). Reference their rules via `enforce()`.
-- **File sync** across agents — [Ruler](https://github.com/intellectronica/ruler), [rulesync](https://github.com/dyoshikawa/rulesync), [block/ai-rules](https://github.com/block/ai-rules). vigiles compiles the source; sync tools distribute.
-- **Markdown linting** — [markdownlint](https://github.com/DavidAnson/markdownlint). vigiles generates markdown; structure is correct by construction.
-- **Code-block linting in docs** — [eslint-plugin-markdown](https://github.com/eslint/eslint-plugin-markdown) for syntax, [twoslash](https://shikijs.github.io/twoslash/) for TS type-checking.
-- **Prose quality** — [Vale](https://vale.sh). Different concern.
-- **Runtime LLM rule checking** (e.g. ai-rulez `"AI-Powered Rule Enforcement"`) — opposite paradigm. Those tools send your code to a model on every check, costing tokens and giving non-reproducible verdicts. vigiles compiles once and checks deterministically forever after with `eslint`, `ruff`, `tsc`, Cedar evaluation — tools as deterministic as their inputs.
+- **[docs/](docs/README.md)** — how-to & reference: the adoption ladder, CLI, linter support, the harness-testing guide, skills/agents.
+- **[research/](research/README.md)** — the thinking behind it: design docs, the [harness-testing coverage roadmap](research/harness-testing-coverage-matrix.md), benchmark findings, landscape, and parked ideas.
 
 ## License
 

package/dist/cli.js

@@ -24,6 +24,7 @@ const generate_schema_js_1 = require("./generate-schema.js");
 const compile_generator_js_1 = require("./compile-generator.js");
 const action_gate_js_1 = require("./action-gate.js");
 const refs_js_1 = require("./refs.js");
+const mcp_js_1 = require("./mcp.js");
 const skill_runtime_js_1 = require("./skill-runtime.js");
 const linters_js_1 = require("./linters.js");
 const harness_test_js_1 = require("./harness-test.js");
@@ -433,6 +434,46 @@ function verifyMarkdownSymbols(files, silent) {
     }
     return errors;
 }
+/**
+ * Verify `vigiles:mcp server#tool` marks in instruction files against the live
+ * MCP servers declared in `.mcp.json` — the referenced tool must exist on the
+ * server (it gets started for the check). No `.mcp.json` ⇒ skipped; a server is
+ * only started if a mark actually references it. Returns the count of broken
+ * references. Async because it speaks to real servers.
+ */
+async function verifyMarkdownMcpRefs(files, silent) {
+    const cwd = process.cwd();
+    const servers = (0, mcp_js_1.loadMcpServers)(cwd);
+    if (files.length === 0 || Object.keys(servers).length === 0)
+        return 0;
+    let printedHeader = false;
+    let errors = 0;
+    for (const f of files) {
+        let markdown;
+        try {
+            markdown = (0, node_fs_1.readFileSync)((0, node_path_1.resolve)(cwd, f), "utf-8");
+        }
+        catch {
+            continue;
+        }
+        const broken = await (0, mcp_js_1.verifyMcpRefs)(markdown, servers);
+        if (broken.length === 0)
+            continue;
+        if (!silent) {
+            if (!printedHeader) {
+                console.log("\nMCP reference check:\n");
+                printedHeader = true;
+            }
+            for (const b of broken) {
+                const msg = (0, mcp_js_1.mcpRefMessage)(b);
+                console.log(`  ✗ ${f}:${String(b.line)} ${msg}`);
+                ghAnnotate("error", msg, f, b.line);
+            }
+        }
+        errors += broken.length;
+    }
+    return errors;
+}
 /** Exit codes: 0 clean, 1 warnings only, 2 hard errors. */
 function auditExitCode(report) {
     if (report.hashErrors > 0 ||
@@ -441,7 +482,8 @@ function auditExitCode(report) {
         report.frontmatterErrors > 0 ||
         report.integrityErrors > 0 ||
         report.coverageErrors > 0 ||
-        report.symbolRefErrors > 0)
+        report.symbolRefErrors > 0 ||
+        report.mcpRefErrors > 0)
         return 2;
     if (report.duplicatePairs > 0 ||
         report.orphanCount > 0 ||
@@ -730,6 +772,9 @@ async function audit(restArgs, flags, config) {
     }
     // 9. Verify code-shaped symbol references live (see src/refs.ts).
     const symbolRefErrors = verifyMarkdownSymbols(files, silent);
+    // 10. Verify `vigiles:mcp server#tool` marks against live MCP servers
+    // (only when a .mcp.json declares them). See src/mcp.ts.
+    const mcpRefErrors = await verifyMarkdownMcpRefs(files, silent);
     const report = {
         hashErrors: hashResult.hashErrors,
         validationErrors: hashResult.validationErrors,
@@ -746,6 +791,7 @@ async function audit(restArgs, flags, config) {
         orphanCount: orphanReport.orphans.length,
         docRefErrors: docRefReport.errors.length,
         symbolRefErrors,
+        mcpRefErrors,
         files,
     };
     if (summary) {
@@ -775,6 +821,8 @@ function printAuditSummary(report) {
         parts.push(`${String(report.docRefErrors)} broken doc refs`);
     if (report.symbolRefErrors > 0)
         parts.push(`${String(report.symbolRefErrors)} broken symbol refs`);
+    if (report.mcpRefErrors > 0)
+        parts.push(`${String(report.mcpRefErrors)} broken MCP refs`);
     const undocumented = report.coverageEnabled - report.coverageDocumented;
     if (undocumented > 0)
         parts.push(`${String(undocumented)} undocumented rules`);

package/dist/eval.d.ts

@@ -10,6 +10,14 @@ export interface EvalArm {
      * src/plugin-loader.ts.
      */
     readonly plugin?: string;
+    /**
+     * Path to a plugin dir to install NATIVELY (`claude --plugin-dir`) for this
+     * arm, so its skills/commands/agents activate the real way — the real model
+     * can trigger a skill by its description (vs. `plugin`, which materializes a
+     * file subset that does not register skills). Point at a COMPLETE plugin. Lets
+     * an arm be "skill installed" vs "off" to measure real activation.
+     */
+    readonly pluginDir?: string;
 }
 /** Context handed to `measure` after a run, to compute that run's metrics. */
 export interface RunContext {

package/dist/eval.js

@@ -39,7 +39,7 @@ function writeFiles(cwd, files) {
         (0, node_fs_1.writeFileSync)(full, content);
     }
 }
-function spawnAgent(task, cwd, model, tools, hasSettings, timeoutMs) {
+function spawnAgent(task, cwd, model, tools, hasSettings, pluginDir, timeoutMs) {
     return new Promise((resolvePromise) => {
         const args = [
             "-p",
@@ -50,6 +50,7 @@ function spawnAgent(task, cwd, model, tools, hasSettings, timeoutMs) {
             model,
             "--permission-mode",
             "acceptEdits",
+            ...(pluginDir !== undefined ? ["--plugin-dir", (0, node_path_1.resolve)(pluginDir)] : []),
             ...(hasSettings ? ["--settings", "settings.json"] : []),
             "--allowedTools",
             ...tools,
@@ -173,7 +174,7 @@ async function runEval(spec) {
                 if (hasSettings) {
                     (0, node_fs_1.writeFileSync)((0, node_path_1.join)(cwd, "settings.json"), JSON.stringify(settings, null, 2).replaceAll("{cwd}", cwd));
                 }
-                const out = await spawnAgent(spec.task, cwd, model, tools, hasSettings, timeoutMs);
+                const out = await spawnAgent(spec.task, cwd, model, tools, hasSettings, arm.pluginDir, timeoutMs);
                 rows.push(spec.measure(makeContext(cwd, out)));
             }
             finally {

package/dist/harness-assert.d.ts

@@ -13,7 +13,7 @@
  *     `expect(...).toHaveCreated(...)` sugar. The signature is identical for
  *     vitest and jest, so the same object supports both.
  */
-import { type HarnessTestSpec, type HarnessTestResult } from "./harness-test.js";
+import { type HarnessTestSpec, type HarnessTestResult, type ToolCall } from "./harness-test.js";
 import type { EvalReport } from "./eval.js";
 import type { HookRunResult } from "./run-hook.js";
 /**
@@ -32,6 +32,47 @@ export declare function assertServedTurns(r: HarnessTestResult, n: number): void
 export declare function assertHookBlocked(r: HookRunResult): void;
 /** Assert a `runHook` result allowed (did not block). */
 export declare function assertHookAllowed(r: HookRunResult): void;
+/**
+ * Assert the agent invoked a tool whose name matches `name` (string = exact,
+ * RegExp = test) — e.g. a skill (`"Skill"`), an MCP tool (`/^mcp__github__/`), or
+ * a subagent (`"Task"`). Needs `transcript: true`. The action invariant the
+ * skill/MCP/command surfaces are really about.
+ */
+export declare function assertToolUsed(r: HarnessTestResult, name: string | RegExp): void;
+/**
+ * Assert the agent did NOT invoke any tool matching `name` — the safety negative
+ * (e.g. a destructive MCP tool was never called). "File unchanged" can pass by
+ * accident; "the tool was never used" is the real invariant. Needs `transcript`.
+ */
+export declare function assertToolNotUsed(r: HarnessTestResult, name: string | RegExp): void;
+/**
+ * Assert the `Skill` tool resolved `skill` (e.g. `"superpowers:test-driven-development"`)
+ * without error — the correct skill-activation invariant, vs. grepping the body.
+ */
+export declare function assertSkillResolved(r: HarnessTestResult, skill: string): void;
+/**
+ * Assert how many tools matching `name` the agent invoked is within bounds — a
+ * budget invariant (e.g. `{ max: 1 }` = "at most one Write", `{ exactly: 0 }` =
+ * "never touched it"). Catches runaway loops and wasted work. Needs `transcript`.
+ */
+export declare function assertToolCount(r: HarnessTestResult, name: string | RegExp, bounds: {
+    min?: number;
+    max?: number;
+    exactly?: number;
+}): void;
+/**
+ * Assert the named tools occurred in this order (as a subsequence — gaps allowed)
+ * — an ordering invariant. e.g. `["Read", "Edit"]` checks a Read came before an
+ * Edit. For a stricter rule (every Edit preceded by a Read), use `assertToolCalls`.
+ * Needs `transcript`.
+ */
+export declare function assertToolSequence(r: HarnessTestResult, names: ReadonlyArray<string | RegExp>): void;
+/**
+ * The escape hatch: assert any custom invariant over the full list of tool calls
+ * the agent made — for rules the helpers above don't express, e.g. "every Edit
+ * was preceded by a Read of that file". Needs `transcript`.
+ */
+export declare function assertToolCalls(r: HarnessTestResult, predicate: (calls: readonly ToolCall[]) => boolean, message?: string): void;
 /** The gap on `metric` between two arms (arm − baseline). */
 export declare function improvement(report: EvalReport, baseline: string, arm: string, metric: string): number;
 /**

package/dist/harness-assert.js

@@ -7,6 +7,12 @@ exports.assertNotCreated = assertNotCreated;
 exports.assertServedTurns = assertServedTurns;
 exports.assertHookBlocked = assertHookBlocked;
 exports.assertHookAllowed = assertHookAllowed;
+exports.assertToolUsed = assertToolUsed;
+exports.assertToolNotUsed = assertToolNotUsed;
+exports.assertSkillResolved = assertSkillResolved;
+exports.assertToolCount = assertToolCount;
+exports.assertToolSequence = assertToolSequence;
+exports.assertToolCalls = assertToolCalls;
 exports.improvement = improvement;
 exports.assertImproves = assertImproves;
 /**
@@ -71,6 +77,93 @@ function assertHookAllowed(r) {
         fail(`expected the hook to allow, but it blocked (exit ${String(r.exitCode)}, decision ${String(r.decision)})`);
     }
 }
+function nameMatches(name, pat) {
+    return typeof pat === "string" ? name === pat : pat.test(name);
+}
+function toolNames(r) {
+    return r.toolCalls.map((c) => c.name).join(", ") || "none";
+}
+/**
+ * Assert the agent invoked a tool whose name matches `name` (string = exact,
+ * RegExp = test) — e.g. a skill (`"Skill"`), an MCP tool (`/^mcp__github__/`), or
+ * a subagent (`"Task"`). Needs `transcript: true`. The action invariant the
+ * skill/MCP/command surfaces are really about.
+ */
+function assertToolUsed(r, name) {
+    if (!r.toolCalls.some((c) => nameMatches(c.name, name))) {
+        fail(`expected a tool matching ${String(name)} to be used; tools used: [${toolNames(r)}] (did you set transcript:true?)`);
+    }
+}
+/**
+ * Assert the agent did NOT invoke any tool matching `name` — the safety negative
+ * (e.g. a destructive MCP tool was never called). "File unchanged" can pass by
+ * accident; "the tool was never used" is the real invariant. Needs `transcript`.
+ */
+function assertToolNotUsed(r, name) {
+    const hit = r.toolCalls.find((c) => nameMatches(c.name, name));
+    if (hit) {
+        fail(`expected no tool matching ${String(name)} to be used, but ${hit.name} was`);
+    }
+}
+/**
+ * Assert the `Skill` tool resolved `skill` (e.g. `"superpowers:test-driven-development"`)
+ * without error — the correct skill-activation invariant, vs. grepping the body.
+ */
+function assertSkillResolved(r, skill) {
+    const call = r.toolCalls.find((c) => c.name === "Skill" && c.input?.skill === skill);
+    if (!call) {
+        const seen = r.toolCalls
+            .filter((c) => c.name === "Skill")
+            .map((c) => c.input?.skill ?? "?")
+            .join(", ");
+        fail(`expected the Skill tool to resolve "${skill}"; Skill calls: [${seen || "none"}]`);
+    }
+    if (call.isError) {
+        fail(`the Skill "${skill}" was invoked but errored: ${call.resultText.slice(0, 200)}`);
+    }
+}
+// --- sequence / budget invariants over the agent's actions -----------------
+/**
+ * Assert how many tools matching `name` the agent invoked is within bounds — a
+ * budget invariant (e.g. `{ max: 1 }` = "at most one Write", `{ exactly: 0 }` =
+ * "never touched it"). Catches runaway loops and wasted work. Needs `transcript`.
+ */
+function assertToolCount(r, name, bounds) {
+    const n = r.toolCalls.filter((c) => nameMatches(c.name, name)).length;
+    const ok = (bounds.exactly === undefined || n === bounds.exactly) &&
+        (bounds.min === undefined || n >= bounds.min) &&
+        (bounds.max === undefined || n <= bounds.max);
+    if (!ok) {
+        fail(`expected count of ${String(name)} to satisfy ${JSON.stringify(bounds)}, got ${String(n)} (tools: [${toolNames(r)}])`);
+    }
+}
+/**
+ * Assert the named tools occurred in this order (as a subsequence — gaps allowed)
+ * — an ordering invariant. e.g. `["Read", "Edit"]` checks a Read came before an
+ * Edit. For a stricter rule (every Edit preceded by a Read), use `assertToolCalls`.
+ * Needs `transcript`.
+ */
+function assertToolSequence(r, names) {
+    let i = 0;
+    for (const c of r.toolCalls) {
+        const want = names[i];
+        if (want !== undefined && nameMatches(c.name, want))
+            i++;
+    }
+    if (i < names.length) {
+        fail(`expected tools in order [${names.map((n) => String(n)).join(" → ")}]; got [${toolNames(r)}]`);
+    }
+}
+/**
+ * The escape hatch: assert any custom invariant over the full list of tool calls
+ * the agent made — for rules the helpers above don't express, e.g. "every Edit
+ * was preceded by a Read of that file". Needs `transcript`.
+ */
+function assertToolCalls(r, predicate, message = "tool-call invariant failed") {
+    if (!predicate(r.toolCalls)) {
+        fail(`${message}; tools used: [${toolNames(r)}]`);
+    }
+}
 /** The gap on `metric` between two arms (arm − baseline). */
 function improvement(report, baseline, arm, metric) {
     const a = report.arms[arm]?.metrics[metric] ?? 0;

package/dist/harness-test.d.ts

@@ -12,12 +12,29 @@ export interface HarnessTestSpec {
      * subset. Inline `settings`/`files` layer on top. See src/plugin-loader.ts.
      */
     readonly plugin?: string;
+    /**
+     * Path to a plugin dir to install NATIVELY via `claude --plugin-dir`, so its
+     * skills / commands / agents / hooks register and ACTIVATE the real way — a
+     * scripted `Skill` tool_use resolves, and the real model can trigger them.
+     * Unlike `plugin` (which materializes a file subset that does NOT register
+     * skills for the `Skill` tool), this is the real install path, so point it at a
+     * COMPLETE plugin (internal references resolve). Inline `settings`/`files` and
+     * `plugin` still layer on top. Resolved to an absolute path.
+     */
+    readonly pluginDir?: string;
     /** The scripted model turns the agent will take. */
     readonly model: readonly ModelTurn[];
     /** The user prompt. Default: "go". */
     readonly prompt?: string;
     /** Tools the agent may use. Default: Read Edit Write Bash. */
     readonly allowedTools?: readonly string[];
+    /**
+     * Capture the full event transcript (`--output-format stream-json`) into
+     * `stdout`, instead of just the final result object, so you can assert on what
+     * the agent's tools returned — e.g. the body a `Skill` tool_use resolved. With
+     * this on, `stdout` is newline-delimited JSON events, not a single object.
+     */
+    readonly transcript?: boolean;
     /** Per-run wall-clock timeout in ms. Default 60000. */
     readonly timeoutMs?: number;
 }
@@ -30,11 +47,33 @@ export interface HarnessTestResult {
     readonly cwd: string;
     /** Number of model turns the agent took (mock turns served). */
     readonly turns: number;
+    /**
+     * The tools the agent invoked, each paired with its result — parsed from the
+     * transcript. Empty unless `transcript: true`. Lets a test assert on the
+     * agent's *actions* (skills, MCP tools, subagents) instead of grepping stdout.
+     */
+    readonly toolCalls: readonly ToolCall[];
     /** Final contents of a file under the working dir, or null if absent. */
     file(path: string): string | null;
     /** Remove the temp working dir. */
     cleanup(): void;
 }
+/** A tool the agent invoked, paired with its result (transcript mode only). */
+export interface ToolCall {
+    readonly name: string;
+    readonly input: unknown;
+    /** The tool_result text ("" if none / not captured). */
+    readonly resultText: string;
+    /** Whether the tool_result came back flagged as an error. */
+    readonly isError: boolean;
+}
+/**
+ * Parse `--output-format stream-json` (the `transcript: true` output) into the
+ * tools the agent invoked, each joined to its result by id. Returns [] for the
+ * non-stream `json` output. The seam that lets a test assert on the agent's
+ * actions, not a brittle stdout substring.
+ */
+export declare function parseToolCalls(streamJson: string): ToolCall[];
 /** Whether the `claude` CLI is available — harness tests need it. */
 export declare function claudeAvailable(): boolean;
 /**

package/dist/harness-test.js

@@ -1,6 +1,7 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.resolveHarness = exports.loadPlugin = exports.scriptModel = void 0;
+exports.parseToolCalls = parseToolCalls;
 exports.claudeAvailable = claudeAvailable;
 exports.runHarnessTest = runHarnessTest;
 /**
@@ -43,6 +44,63 @@ Object.defineProperty(exports, "scriptModel", { enumerable: true, get: function
 var plugin_loader_js_2 = require("./plugin-loader.js");
 Object.defineProperty(exports, "loadPlugin", { enumerable: true, get: function () { return plugin_loader_js_2.loadPlugin; } });
 Object.defineProperty(exports, "resolveHarness", { enumerable: true, get: function () { return plugin_loader_js_2.resolveHarness; } });
+function contentText(content) {
+    if (typeof content === "string")
+        return content;
+    if (!Array.isArray(content))
+        return "";
+    return content
+        .map((b) => {
+        if (typeof b === "string")
+            return b;
+        const t = b.text;
+        return typeof t === "string" ? t : "";
+    })
+        .join("");
+}
+/**
+ * Parse `--output-format stream-json` (the `transcript: true` output) into the
+ * tools the agent invoked, each joined to its result by id. Returns [] for the
+ * non-stream `json` output. The seam that lets a test assert on the agent's
+ * actions, not a brittle stdout substring.
+ */
+function parseToolCalls(streamJson) {
+    const uses = [];
+    const results = new Map();
+    for (const line of streamJson.split("\n")) {
+        if (!line.trim())
+            continue;
+        let evt;
+        try {
+            evt = JSON.parse(line);
+        }
+        catch {
+            continue;
+        }
+        const content = evt.message?.content;
+        if (!Array.isArray(content))
+            continue;
+        for (const b of content) {
+            if (b.type === "tool_use" && typeof b.name === "string") {
+                const id = typeof b.id === "string" ? b.id : "";
+                uses.push({ id, name: b.name, input: b.input });
+            }
+            else if (b.type === "tool_result") {
+                const id = typeof b.tool_use_id === "string" ? b.tool_use_id : "";
+                results.set(id, {
+                    text: contentText(b.content),
+                    isError: b.is_error === true,
+                });
+            }
+        }
+    }
+    return uses.map((u) => ({
+        name: u.name,
+        input: u.input,
+        resultText: results.get(u.id)?.text ?? "",
+        isError: results.get(u.id)?.isError ?? false,
+    }));
+}
 /** Whether the `claude` CLI is available — harness tests need it. */
 function claudeAvailable() {
     try {
@@ -107,10 +165,14 @@ async function runHarnessTest(spec) {
         const args = [
             "-p",
             spec.prompt ?? "go",
-            "--output-format",
-            "json",
+            ...(spec.transcript
+                ? ["--output-format", "stream-json", "--verbose"]
+                : ["--output-format", "json"]),
             "--model",
             "claude-sonnet-4-5",
+            ...(spec.pluginDir !== undefined
+                ? ["--plugin-dir", (0, node_path_1.resolve)(spec.pluginDir)]
+                : []),
             ...(settings !== undefined ? ["--settings", "settings.json"] : []),
             "--allowedTools",
             ...tools,
@@ -122,6 +184,7 @@ async function runHarnessTest(spec) {
             stderr: out.stderr,
             cwd,
             turns: mock.count,
+            toolCalls: parseToolCalls(out.stdout),
             file: (p) => {
                 const f = (0, node_path_1.resolve)(cwd, p);
                 return (0, node_fs_1.existsSync)(f) ? (0, node_fs_1.readFileSync)(f, "utf-8") : null;

package/dist/mcp.d.ts

@@ -0,0 +1,48 @@
+export interface McpServerConfig {
+    readonly command: string;
+    readonly args?: readonly string[];
+    readonly env?: Record<string, string>;
+    readonly cwd?: string;
+}
+export interface McpToolInfo {
+    readonly name: string;
+    readonly description?: string;
+}
+/**
+ * Start an MCP server over stdio, complete the handshake, and return its tools.
+ * Kills the server when done. Throws on spawn/timeout/exit/protocol error.
+ */
+export declare function listMcpTools(server: McpServerConfig, timeoutMs?: number): Promise<McpToolInfo[]>;
+export interface McpRefResult {
+    readonly exists: boolean;
+    readonly available: string[];
+    readonly suggestions: string[];
+}
+/**
+ * Verify `toolName` exists on `server`; on a miss, suggest the closest tool names
+ * (edit distance) — "did you mean issue_write?".
+ */
+export declare function verifyMcpTool(server: McpServerConfig, toolName: string, timeoutMs?: number): Promise<McpRefResult>;
+export interface McpRef {
+    readonly server: string;
+    readonly tool: string;
+    readonly line: number;
+}
+export type McpRefReason = "server-undeclared" | "server-unreachable" | "tool-missing";
+export interface McpRefError extends McpRef {
+    readonly reason: McpRefReason;
+    readonly suggestions: string[];
+}
+/** Parse `vigiles:mcp server#tool` marks from a markdown file's inline spans. */
+export declare function parseMcpRefs(markdown: string): McpRef[];
+/** Read `mcpServers` from `.mcp.json` (the stdio-server config map), or `{}`. */
+export declare function loadMcpServers(cwd: string): Record<string, McpServerConfig>;
+/**
+ * Verify every `vigiles:mcp server#tool` mark in `markdown` against the live
+ * servers in `mcpServers` (each referenced server is started once). A reference
+ * to an undeclared server, an unreachable server, or a missing tool is an error.
+ */
+export declare function verifyMcpRefs(markdown: string, mcpServers: Record<string, McpServerConfig>, timeoutMs?: number): Promise<McpRefError[]>;
+/** Human-readable message for an MCP reference error (with "did you mean"). */
+export declare function mcpRefMessage(e: McpRefError): string;
+//# sourceMappingURL=mcp.d.ts.map

package/dist/mcp.js

@@ -0,0 +1,247 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.listMcpTools = listMcpTools;
+exports.verifyMcpTool = verifyMcpTool;
+exports.parseMcpRefs = parseMcpRefs;
+exports.loadMcpServers = loadMcpServers;
+exports.verifyMcpRefs = verifyMcpRefs;
+exports.mcpRefMessage = mcpRefMessage;
+/**
+ * Minimal MCP client over stdio — start a server, do the JSON-RPC handshake, and
+ * list its tools. This lets vigiles VERIFY a referenced `mcp__server__tool`
+ * resolves against the real server (the way `enforce()` resolves a linter rule
+ * against its catalog), catching a skill/CLAUDE.md that cites an MCP tool that was
+ * renamed or removed — e.g. the GitHub MCP server renaming `create_issue` →
+ * `issue_write`, which otherwise fails silently at runtime.
+ *
+ * MCP stdio transport = newline-delimited JSON-RPC 2.0.
+ */
+const node_child_process_1 = require("node:child_process");
+const node_fs_1 = require("node:fs");
+const node_path_1 = require("node:path");
+const refs_js_1 = require("./refs.js");
+const hash_js_1 = require("./hash.js");
+function dispatch(line, pending) {
+    let msg;
+    try {
+        msg = JSON.parse(line);
+    }
+    catch {
+        return;
+    }
+    if (typeof msg.id !== "number")
+        return;
+    const p = pending.get(msg.id);
+    if (p) {
+        pending.delete(msg.id);
+        p.resolve(msg);
+    }
+}
+/**
+ * Start an MCP server over stdio, complete the handshake, and return its tools.
+ * Kills the server when done. Throws on spawn/timeout/exit/protocol error.
+ */
+async function listMcpTools(server, timeoutMs = 10000) {
+    const child = (0, node_child_process_1.spawn)(server.command, [...(server.args ?? [])], {
+        env: server.env ? { ...process.env, ...server.env } : process.env,
+        cwd: server.cwd,
+        stdio: ["pipe", "pipe", "ignore"],
+    });
+    const { stdin, stdout } = child;
+    if (!stdin || !stdout) {
+        child.kill("SIGKILL");
+        throw new Error("failed to open MCP server stdio");
+    }
+    const pending = new Map();
+    const failAll = (err) => {
+        for (const [, p] of pending)
+            p.reject(err);
+        pending.clear();
+    };
+    let buffer = "";
+    stdout.setEncoding("utf-8");
+    stdout.on("data", (chunk) => {
+        buffer += chunk;
+        let nl = buffer.indexOf("\n");
+        while (nl >= 0) {
+            const line = buffer.slice(0, nl).trim();
+            buffer = buffer.slice(nl + 1);
+            if (line)
+                dispatch(line, pending);
+            nl = buffer.indexOf("\n");
+        }
+    });
+    child.on("error", (e) => {
+        failAll(e);
+    });
+    child.on("close", () => {
+        failAll(new Error("MCP server exited before responding"));
+    });
+    const send = (obj) => {
+        stdin.write(`${JSON.stringify(obj)}\n`);
+    };
+    const request = (id, method, params) => new Promise((resolve, reject) => {
+        pending.set(id, { resolve, reject });
+        send({ jsonrpc: "2.0", id, method, params });
+    });
+    const timer = setTimeout(() => {
+        failAll(new Error(`MCP server timed out after ${String(timeoutMs)}ms`));
+        child.kill("SIGKILL");
+    }, timeoutMs);
+    try {
+        await request(1, "initialize", {
+            protocolVersion: "2024-11-05",
+            capabilities: {},
+            clientInfo: { name: "vigiles", version: "0" },
+        });
+        send({ jsonrpc: "2.0", method: "notifications/initialized" });
+        const res = await request(2, "tools/list", {});
+        if (res.error) {
+            throw new Error(`tools/list failed: ${res.error.message ?? "unknown"}`);
+        }
+        return (res.result?.tools ?? []).map((t) => ({
+            name: t.name,
+            description: t.description,
+        }));
+    }
+    finally {
+        clearTimeout(timer);
+        child.kill("SIGKILL");
+    }
+}
+/**
+ * Verify `toolName` exists on `server`; on a miss, suggest the closest tool names
+ * (edit distance) — "did you mean issue_write?".
+ */
+async function verifyMcpTool(server, toolName, timeoutMs = 10000) {
+    const available = (await listMcpTools(server, timeoutMs)).map((t) => t.name);
+    const exists = available.includes(toolName);
+    return {
+        exists,
+        available,
+        suggestions: exists ? [] : closest(toolName, available),
+    };
+}
+function editDistance(a, b) {
+    if (a === b)
+        return 0;
+    const m = a.length;
+    const n = b.length;
+    if (m === 0)
+        return n;
+    if (n === 0)
+        return m;
+    const dp = Array.from({ length: n + 1 }, (_, i) => i);
+    for (let i = 1; i <= m; i++) {
+        let prev = dp[0];
+        dp[0] = i;
+        for (let j = 1; j <= n; j++) {
+            const tmp = dp[j];
+            dp[j] =
+                a[i - 1] === b[j - 1] ? prev : 1 + Math.min(prev, dp[j], dp[j - 1]);
+            prev = tmp;
+        }
+    }
+    return dp[n];
+}
+function closest(target, candidates, max = 4) {
+    return candidates
+        .map((c) => ({ c, d: editDistance(target, c) }))
+        .filter((x) => x.d <= max)
+        .sort((a, b) => a.d - b.d)
+        .slice(0, 3)
+        .map((x) => x.c);
+}
+// --- MCP tool references in instruction files ------------------------------
+// `vigiles:mcp <server>#<tool>` inside an inline code span — the MCP analogue of
+// the `vigiles:symbol path#name` mark. Self-contained (server + tool in one
+// token) so it binds unambiguously.
+const MCP_MARK = /^vigiles:mcp\s+([\w-]+)#([\w.-]+)$/;
+/** Parse `vigiles:mcp server#tool` marks from a markdown file's inline spans. */
+function parseMcpRefs(markdown) {
+    const refs = [];
+    for (const span of (0, refs_js_1.inlineSpans)(markdown)) {
+        const m = MCP_MARK.exec(span.text);
+        if (m)
+            refs.push({ server: m[1], tool: m[2], line: span.line });
+    }
+    return refs;
+}
+/** Read `mcpServers` from `.mcp.json` (the stdio-server config map), or `{}`. */
+function loadMcpServers(cwd) {
+    const p = (0, node_path_1.join)(cwd, ".mcp.json");
+    if (!(0, node_fs_1.existsSync)(p))
+        return {};
+    try {
+        const json = JSON.parse((0, node_fs_1.readFileSync)(p, "utf-8"));
+        return json.mcpServers ?? {};
+    }
+    catch {
+        return {};
+    }
+}
+async function verifyOneServer(group, cfg, timeoutMs) {
+    if (!cfg) {
+        return group.map((r) => ({
+            ...r,
+            reason: "server-undeclared",
+            suggestions: [],
+        }));
+    }
+    let available;
+    try {
+        available = (await listMcpTools(cfg, timeoutMs)).map((t) => t.name);
+    }
+    catch {
+        return group.map((r) => ({
+            ...r,
+            reason: "server-unreachable",
+            suggestions: [],
+        }));
+    }
+    const errs = [];
+    for (const r of group) {
+        if (!available.includes(r.tool)) {
+            errs.push({
+                ...r,
+                reason: "tool-missing",
+                suggestions: closest(r.tool, available),
+            });
+        }
+    }
+    return errs;
+}
+/**
+ * Verify every `vigiles:mcp server#tool` mark in `markdown` against the live
+ * servers in `mcpServers` (each referenced server is started once). A reference
+ * to an undeclared server, an unreachable server, or a missing tool is an error.
+ */
+async function verifyMcpRefs(markdown, mcpServers, timeoutMs = 10000) {
+    const byServer = new Map();
+    for (const r of parseMcpRefs(markdown)) {
+        const arr = byServer.get(r.server) ?? [];
+        arr.push(r);
+        byServer.set(r.server, arr);
+    }
+    const all = [];
+    for (const [server, group] of byServer) {
+        all.push(...(await verifyOneServer(group, mcpServers[server], timeoutMs)));
+    }
+    return all;
+}
+/** Human-readable message for an MCP reference error (with "did you mean"). */
+function mcpRefMessage(e) {
+    switch (e.reason) {
+        case "server-undeclared":
+            return `MCP server "${e.server}" is not declared in .mcp.json`;
+        case "server-unreachable":
+            return `MCP server "${e.server}" failed to start`;
+        case "tool-missing":
+            return `MCP tool "${e.server}#${e.tool}" not found${e.suggestions.length > 0
+                ? ` — did you mean ${e.suggestions.map((s) => `"${s}"`).join(", ")}?`
+                : ""}`;
+        default:
+            return (0, hash_js_1.assertNever)(e.reason);
+    }
+}
+//# sourceMappingURL=mcp.js.map

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vigiles",
-  "version": "2.3.0",
+  "version": "2.4.0",
   "description": "Compile .spec.ts files to instruction files (CLAUDE.md, AGENTS.md) with linter cross-referencing",
   "bin": {
     "vigiles": "dist/cli.js"
@@ -17,6 +17,7 @@
     "./harness-assert": "./dist/harness-assert.js",
     "./run-hook": "./dist/run-hook.js",
     "./plugin-loader": "./dist/plugin-loader.js",
+    "./mcp": "./dist/mcp.js",
     "./judge": "./dist/judge.js",
     "./mock-model": "./dist/mock-model.js",
     "./vitest": {
@@ -39,10 +40,11 @@
   ],
   "scripts": {
     "build": "tsc",
-    "test": "npm run build && node --test dist/spec.test.js dist/validate.test.js dist/cli.test.js dist/proofs.test.js dist/inline.test.js dist/sidecar.test.js dist/coverage.test.js dist/session.test.js dist/orphans.test.js dist/cedar.test.js dist/doc-refs.test.js dist/frontmatter.test.js dist/skill-pipeline.test.js dist/skill-runtime.test.js dist/skill-driver.test.js dist/skill-test.test.js dist/compile-generator.test.js dist/community-skills.test.js dist/action-gate.test.js dist/symbols.test.js dist/refs.test.js dist/harness-test.test.js dist/eval.test.js dist/run-scripts.test.js dist/plugin-loader.test.js dist/harness-assert.test.js dist/judge.test.js dist/run-hook.test.js",
+    "test": "npm run build && node --test dist/spec.test.js dist/validate.test.js dist/cli.test.js dist/proofs.test.js dist/inline.test.js dist/sidecar.test.js dist/coverage.test.js dist/session.test.js dist/orphans.test.js dist/cedar.test.js dist/doc-refs.test.js dist/frontmatter.test.js dist/skill-pipeline.test.js dist/skill-runtime.test.js dist/skill-driver.test.js dist/skill-test.test.js dist/compile-generator.test.js dist/community-skills.test.js dist/action-gate.test.js dist/symbols.test.js dist/refs.test.js dist/harness-test.test.js dist/eval.test.js dist/run-scripts.test.js dist/plugin-loader.test.js dist/harness-assert.test.js dist/judge.test.js dist/run-hook.test.js dist/mcp.test.js",
     "lint": "eslint src/",
     "fmt": "prettier --write .",
     "fmt:check": "prettier --check .",
+    "demo": "npm run build && bash examples/demo/run.sh",
     "test:e2e": "bash test/e2e/run.sh",
     "test:harness": "npm run build && node dist/cli.js test",
     "test:eval": "npm run build && node dist/cli.js eval",