npm - vigiles - Versions diffs - 2.5.0 → 2.6.0 - Mend

vigiles 2.5.0 → 2.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (18) hide show

package/README.md CHANGED Viewed

@@ -9,8 +9,8 @@
 </p>
 <p align="center">
-  <strong>Test &amp; verify your Claude Code harness.</strong><br />
-  vigiles <strong>verifies the references</strong> your instruction files make — linter rules, file paths, scripts, code symbols — and <strong>evals</strong> whether your hooks, skills, and CLAUDE.md actually change what the agent does.
+  <strong>The missing linting + testing layer for agentic coding.</strong><br />
+  vigiles <strong>lints</strong> the references your instruction files make — linter rules, file paths, scripts, code symbols — and <strong>tests</strong> whether your hooks, skills, and CLAUDE.md actually change what the agent does.
 </p>
 <p align="center">
@@ -19,30 +19,39 @@
   <a href="https://github.com/zernie/vigiles/blob/main/LICENSE"><img src="https://img.shields.io/github/license/zernie/vigiles" alt="License" /></a>
 </p>
+<p align="center">
+  <b>Two pillars →</b>
+  &nbsp;<a href="#verify-your-instruction-files">① Verify your instruction files</a>
+  &nbsp;·&nbsp;
+  <a href="#test-your-claude-code-harness">② Test your Claude Code harness</a>
+</p>
 ---
 <details>
 <summary><b>Contents</b></summary>
-**Pillar 1 — verify your instruction files** · references your CLAUDE.md makes that a linter, the filesystem, and package.json can prove
+- [**Two pillars — pick one or both**](#two-pillars--pick-one-or-both)
+- **Pillar 1** — [verify your instruction files](#verify-your-instruction-files) · full guide: [docs/verifying-instruction-files.md](docs/verifying-instruction-files.md)
+- **Pillar 2** — [test your Claude Code harness](#test-your-claude-code-harness) · full guide: [docs/harness-testing.md](docs/harness-testing.md)
+- [Quick start](#quick-start) · [CLI & CI](#cli--ci) · [Skills](#skills) · [Related tools](#related-tools)
+</details>
-- Three adoption levels: [inline comments](#level-0--inline-comments-30-seconds-no-new-files) → [YAML frontmatter](#level-1--yaml-frontmatter-editor-autocomplete-still-no-typescript) → [typed spec](#level-2--typed-spec-compiler-grade-guarantees)
-- [What changes with vigiles](#what-changes-with-vigiles)
-- [Quick start](#quick-start)
-- [Three rule types](#three-rule-types) — `enforce` / `guidance` / `guard`
-- [Verified references](#verified-references) — `file` / `cmd` / `symbol` / `ref`
+## Two pillars — pick one or both
-**Pillar 2 — [test your Claude Code harness](#test-your-claude-code-harness)** · eval whether your hooks, skills, and CLAUDE.md actually change what the agent does
+An agent runs real commands in your repo — it can delete the wrong files, leak a secret, or burn tokens looping on a stale instruction nobody checked. You'd never ship an app without a linter and a test suite. An AI agent steering your codebase is no different — so why is its harness trusted on vibes?
-- [Level 1 — unit-test a hook (no AI)](#level-1--test-a-hook-by-itself-no-ai-milliseconds)
-- [Level 2 — does it fire in a real session?](#level-2--does-it-fire-in-a-real-session-free-scripted-ai)
-- [Level 3 — does it change behaviour?](#level-3--does-it-change-what-claude-does-real-ai-occasional)
-- [Test skills for real + assert on actions](#test-your-skills-for-real--and-assert-on-what-claude-did)
-- [Run them in CI](#run-them-in-ci)
+`Agent = Model + Harness`. Your harness is everything that steers a run — the **instructions** you write _and_ the **hooks, skills, and settings** that enforce them. vigiles is the **missing linting + testing layer for agentic coding**: it **lints** your instruction files and **tests** your harness. Two pillars of equal weight — adopt either on its own, or both:
-**More** — [CLI & CI](#cli--ci) · [Skills](#skills) · [Maturity levels](#maturity-levels) · [Related tools](#related-tools)
+|       | Pillar                                                              | What it does                                                                                                                                                                 |
+| ----- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **①** | [**Verify your instruction files**](#verify-your-instruction-files) | Every linter rule, file path, script, and code symbol your CLAUDE.md cites is checked against reality, so stale references can't silently mislead the agent.                 |
+| **②** | [**Test your harness**](#test-your-claude-code-harness)             | Your hooks and skills are code — vigiles tests that they actually fire, **deterministically and for free** (no model, no API key) before you ever pay for a real-model eval. |
-</details>
+They share the thesis but not a dependency: verify your instructions without ever writing a harness test, or test your harness without a single `.spec.ts`. Pick the pillar that hurts today.
+## Verify your instruction files
 Your CLAUDE.md lies to your agent. Here's the fix.
@@ -70,13 +79,11 @@ Reads fine. Four things are wrong:
 The agent reads this, trusts it, and writes code based on stale claims nobody verified. vigiles **verifies the references in your instruction files** — that each linter rule exists and is enabled, that every file path and script is real, and that referenced **code symbols** (functions, classes, constants) actually exist in the files that define them — and meets you at whatever commitment level you want.
-> **See it in 60 seconds:** `npm run demo` runs `vigiles audit` against a deliberately-broken instruction file and catches a renamed symbol and a missing MCP tool (_"did you mean `purge`?"_), while the truthful references pass silently. [examples/demo →](examples/demo)
-Three levels. Each is independently useful; adopt as far up as you like.
+> **Set it up — paste this into Claude Code:**
+>
+> > Install vigiles and verify my instruction files. Scan my CLAUDE.md / AGENTS.md, turn its real claims into checked references (linter rules, file paths, scripts, symbols), run `vigiles audit`, and show me what's already stale. Use sensible defaults, but **ask me first** whether to stay in markdown mode or generate a typed `.spec.ts`, and whether to wire the audit into CI and install the edit-blocking hooks.
-### Level 0 — inline comments (30 seconds, no new files)
-Add a comment to your existing CLAUDE.md and audit it:
+Or do it by hand — add a marker to your existing CLAUDE.md and audit it, no install, no new files:
 ```md
 <!-- vigiles:enforce eslint/no-console "Route output through logger.ts" -->
@@ -86,181 +93,46 @@ Add a comment to your existing CLAUDE.md and audit it:
 npx vigiles audit CLAUDE.md
 ```
-Each rule is checked against your real linter config — typos get closest-match suggestions, disabled rules are flagged. Zero install commitment, zero new files.
-### Level 1 — YAML frontmatter (editor autocomplete, still no TypeScript)
-Promote your rules into a `vigiles:` block at the top of the file:
-```yaml
----
-# yaml-language-server: $schema=./.vigiles/schema.json
-vigiles:
-  enforce:
-    - rule: "@typescript-eslint/no-explicit-any"
-      why: "Use unknown and narrow with type guards."
-    - rule: eslint/no-console
-      why: "Route output through logger.ts"
----
-```
-`npx vigiles generate-schema` emits a JSON Schema from your project's _actual_ enabled rules, so your editor's built-in YAML language server (VS Code, JetBrains, neovim) autocompletes rule names and red-squiggles typos — at edit time, with no TypeScript in the project. `vigiles audit` enforces the same rules in CI. [Markdown mode →](docs/markdown-mode.md)
-### Level 2 — typed spec (compiler-grade guarantees)
-When you want the strongest guarantees, compile a typed spec. Every linter rule reference is verified against your real config, every file path against the filesystem, every npm script against package.json. Stale references become compile errors — caught at edit time, not when the agent silently ignores you.
-```typescript
-// CLAUDE.md.spec.ts
-import { claude, enforce, guidance } from "vigiles/spec";
-export default claude({
-  commands: {
-    "npm run build": "Compile TypeScript to dist/",
-    "npm test": "Build and run all tests",
-    // ✗ "npm run typecheck" → compile error: script not in package.json
-  },
-  keyFiles: {
-    "src/utils/narrowing.ts": "Type guard utilities",
-    // ✗ "src/utils/type-helpers.ts" → compile error: file not found
-  },
-  rules: {
-    "no-explicit-any": enforce(
-      "@typescript-eslint/no-explicit-any",
-      "Use unknown and narrow with type guards.",
-    ),
-    // ✗ if rule is disabled in config → compile error
-    "research-first": guidance("Google unfamiliar APIs first."),
-  },
-});
-```
-```bash
-$ npx vigiles compile
-✓ CLAUDE.md.spec.ts → CLAUDE.md
-  2 rules (1 linter-verified, 1 guidance)
-  ~180 tokens
-```
-At this level the spec is the source of truth and CLAUDE.md is a build artifact. The agent edits the spec — hooks auto-compile, types catch typos in the editor, CI catches drift.
-Companion repo for [Feedback Loop Is All You Need](https://zernie.com/blog/feedback-loop-is-all-you-need).
-## What Changes With vigiles
-### Claude Code
-|                                     | Without vigiles              | With vigiles                                                   |
-| ----------------------------------- | ---------------------------- | -------------------------------------------------------------- |
-| **Instructions**                    | Hand-written CLAUDE.md       | Compiled from `.spec.ts` (build artifact)                      |
-| **Linter rule references**          | Trust-based (nobody checks)  | Verified at compile time against real config                   |
-| **File paths**                      | Rot silently when renamed    | `file()` references checked against filesystem                 |
-| **Commands**                        | Stale scripts go unnoticed   | `cmd()` references checked against package.json                |
-| **Direct edits to CLAUDE.md**       | Anyone can, nobody knows     | PreToolUse hook blocks edits, redirects to spec                |
-| **Spec edits**                      | N/A                          | PostToolUse hook auto-compiles to markdown                     |
-| **Linter config changes**           | CLAUDE.md drifts out of sync | PostToolUse hook auto-regenerates types                        |
-| **guidance → enforce upgrades**     | Manual guesswork             | `/strengthen` reads per-linter docs, suggests upgrades         |
-| **New lint rules from PR feedback** | Copy-paste from review       | `/pr-to-lint-rule` generates rule + tests + spec entry         |
-| **CI**                              | Nothing to verify            | `vigiles audit` catches hand-edits, disabled rules, stale refs |
-<details>
-<summary><b>Codex</b> (same compile-time checks, no hooks)</summary>
-|                               | Without vigiles                  | With vigiles                                            |
-| ----------------------------- | -------------------------------- | ------------------------------------------------------- |
-| **Instructions**              | Hand-written AGENTS.md           | Compiled from `.spec.ts`                                |
-| **Linter rule references**    | Trust-based                      | Verified at compile time                                |
-| **File paths / commands**     | Rot silently                     | Checked at compile time                                 |
-| **Direct edits to AGENTS.md** | Undetected                       | CI catches hash mismatch                                |
-| **Hooks / auto-compile**      | Not available (no plugin system) | Not available — run `vigiles compile` manually or in CI |
-| **CI**                        | Nothing to verify                | Same `vigiles audit` pipeline as Claude                 |
-</details>
+Each reference is checked against reality — a typo gets a closest-match suggestion, a disabled rule is flagged. That's **markdown mode**; step up to a **typed spec** (`.spec.ts` → compiled CLAUDE.md, compiler-grade guarantees) when you want it.
-Everything vigiles compiles and audits is **deterministic** — same input, same output, no LLM in the loop. The non-deterministic parts (authoring specs, suggesting upgrades, writing custom rules) are agent skills that run outside the compilation pipeline. [Determinism breakdown and flow diagram →](docs/comparison.md)
+→ **Full guide: [docs/verifying-instruction-files.md](docs/verifying-instruction-files.md)** — the adoption ladder, the three rule types (`enforce` / `guidance` / `guard`), verified references (`file` / `cmd` / `symbol` / `ref`), and the before/after tables.
 ## Quick Start
-The fastest path is markdown mode — add a marker to your existing CLAUDE.md and audit it, no install or new files (see [Level 0 / Level 1](#level-0--inline-comments-30-seconds-no-new-files) above and [docs/markdown-mode.md](docs/markdown-mode.md)). When you want compiler-grade guarantees, scaffold a typed spec:
 ```bash
 npx vigiles init
 ```
-The wizard auto-detects your project, creates a spec, scans your linters, compiles to markdown, adds a CI step, and installs Claude Code hooks. After install: the agent edits the spec (hooks block direct CLAUDE.md edits), the spec auto-compiles on save, and `vigiles audit` catches drift in CI.
-Start with `guidance()` rules (zero config). When you're ready, run `/strengthen` to find rules that can be upgraded to compile-verified `enforce()`. Already have a hand-written CLAUDE.md? The wizard detects it and offers migration. Flags (`--strict`, `--target=AGENTS.md`, `--no-gha`) and non-interactive agent usage are in the [CLI reference](docs/cli.md) and [agent setup guide](docs/agent-setup.md).
-## Three Rule Types
-**`enforce()`** — delegated to a linter. vigiles verifies the rule exists in the catalog AND is enabled in your project config. A disabled rule is a compile error.
-<!-- vigiles:ignore -->
-```typescript
-"no-any":    enforce("@typescript-eslint/no-explicit-any", "Use unknown and narrow."),
-"no-print":  enforce("ruff/T201", "Use logging module."),
-"no-unwrap": enforce("clippy/unwrap_used", "Use expect() with context."),
-```
-Supports ESLint, Stylelint, Ruff, Clippy, Pylint, RuboCop, and Cedar policies. [Full linter support details →](docs/linter-support.md)
-**`guidance()`** — prose advice. No mechanical enforcement, but not untracked: guidance rules participate in the monotonicity proof system. Once a rule exists, it can be strengthened ( `guidance` → `enforce` ) but never weakened or removed without an explicit allowlist. This prevents silent erosion of conventions over time.
-```typescript
-"research-first": guidance("Google unfamiliar APIs first."),
-```
-**`guard()`** — reactive: runs a command when watched files change (e.g. `*.spec.ts` → `npx vigiles compile`). One declaration emits hooks for every supported system (Claude Code PostToolUse, husky pre-commit, etc.) — no copy-pasting the same trigger across `.claude/settings.json`, `.husky/`, and CI. Same monotonicity guarantees as `enforce()`. [Full spec format →](docs/spec-format.md)
-## Verified References
-`file()`, `cmd()`, `symbol()`, and `ref()` catch stale references at compile time:
-```typescript
-import { claude, file, cmd, symbol, ref, instructions } from "vigiles/spec";
-export default claude({
-  sections: {
-    architecture: instructions`
-      Core engine in ${file("src/compile.ts")}.
-      Compile specs with ${symbol("src/compile.ts", "compileClaude")}.
-      Run ${cmd("npm test")} to verify.
-      See ${ref("skills/strengthen/SKILL.md")} for the strengthen skill.
-    `,
-    // If any path / script / symbol is stale → compile error
-  },
-  // ...
-});
-```
-There's a small family of inline **marks** that `audit` checks, each binding a reference to its real source:
+The wizard auto-detects your project, creates a spec, scans your linters, compiles to markdown, adds a CI step, and installs Claude Code hooks. After install: the agent edits the spec (hooks block direct CLAUDE.md edits), the spec auto-compiles on save, and `vigiles audit` catches drift in CI. Prefer no new files? Stay in [markdown mode](docs/markdown-mode.md). Start with `guidance()` rules and `/strengthen` them to `enforce()` later; flags and agent usage are in the [CLI reference](docs/cli.md).
-- `` `vigiles:symbol file#name` `` — the named file actually **defines** that symbol (function, class, method, constant), parsed with [ast-grep](https://ast-grep.github.io) across **JS/TS, Python, Ruby, Rust, and CSS**. Rename it and `audit` fails; in markdown mode the `refs-hook` **forces the mark**, blocking edits that leave a code reference bare. [Details →](research/symbol-verification.md)
-- `` `vigiles:mcp server#tool` `` — the referenced **MCP tool exists** on its server. `audit` reads `.mcp.json`, starts the server, lists its tools, and flags a renamed/removed one with a "did you mean" — catching e.g. the GitHub MCP server renaming `create_issue` → `issue_write`, which otherwise fails silently.
-**Typo-safe at authoring time, too.** `vigiles generate-types` emits a `.vigiles/generated.d.ts` so `enforce("eslint/no-consolee")` red-squiggles in your editor; `generate-schema` gives Level 1 frontmatter the same via your YAML language server. Both have `--check` CI freshness modes. [How it works →](docs/linter-support.md#generate-types)
+Companion repo for [Feedback Loop Is All You Need](https://zernie.com/blog/feedback-loop-is-all-you-need).
 ## Test your Claude Code harness
-You wrote hooks, a skill, a CLAUDE.md rule — how do you know they work, beyond
-running Claude and eyeballing it? vigiles ships a library to **test the harness
-itself**, at three levels, cheapest first. It's plain async functions, so it
-drops into **node:test / vitest / jest**, or a zero-setup `vigiles test`.
+Verifying references proves your instructions are _true_ — but the hooks and
+skills that enforce them still have to **actually fire**. A hook can be wired
+wrong, a skill's description can fail to trigger, injected context can never reach
+the model — all silently, all passing a naive "did it run?" check. So vigiles's
+second pillar **tests the harness itself**, as the assembled machine it ships as.
-### Level 1 — test a hook by itself (no AI, milliseconds)
+> **Set it up — paste this into Claude Code:**
+>
+> > Install vigiles and use its `test-harness` skill to write and run a harness test for this project. If I didn't say what to test, pick something real from my hooks / skills / settings, choose the cheapest tier (unit / deterministic / eval), write the test, and run it. Use good defaults, but **ask me** whether to gate it in CI and whether to add a real-model eval.
-A hook is just a process that's handed a "Claude is about to do X" event and
-answers block/allow. Hand it a fake event and check the answer — no `claude`, no
-model, and **every** event type is reachable (incl. Edit/Write, PreCompact,
-SessionEnd):
+It's a small library of plain async functions (drops into node:test / vitest /
+jest, or a zero-setup `vigiles test`), with three tiers, cheapest first. The
+design bet is **deterministic and cheap**: the first two tiers never call a model
+or need an API key, so they run on every commit for free — the opposite of
+eval-only frameworks like promptfoo, where every run hits a real model **by
+design**. You only reach for the paid real-model tier when the question genuinely
+needs it.
+- **Unit-test a hook** — `runHook` hands a hook a fake event and checks block/allow. No `claude`, no model, milliseconds, reaches **every** event type.
+- **Deterministic harness test** — `runHarnessTest` runs the **real** `claude` against a **scripted mock model**, so your hooks fire for real with no API key and the same result every time.
+- **Eval** — `runEval` runs the real model A/B (change on vs off) and reports the gap as **mean ± se**, with a Welch-t-test [significance gate](docs/harness-testing.md#significance--is-the-gap-real), regression baselines, and cost/latency/token tracking.
 ```typescript
-import { runHook } from "vigiles/run-hook";
+import { runHook } from "vigiles/testing";
 const r = runHook(guardCommand, {
   hook_event_name: "PreToolUse",
@@ -270,174 +142,10 @@ const r = runHook(guardCommand, {
 assert(r.blocked); // exit 2 / decision:"block" / permissionDecision:"deny"
 ```
-The same shape governs **MCP tools** — the dominant real MCP test — with no
-server running, because the hook only sees the tool _name_:
-```typescript
-// block the destructive github-MCP tool; read-only ones pass
-runHook(guard, {
-  hook_event_name: "PreToolUse",
-  tool_name: "mcp__github__merge_pull_request",
-  tool_input: { pull_number: 42 },
-}).blocked; // true
-```
-### Level 2 — does it fire in a real session? (free, scripted "AI")
-Right logic ≠ wired in correctly. `runHarnessTest` runs the **real** `claude`
-against a **scripted mock model** you control — your hooks fire for real, the
-agent's turns are fixed, no API key, same result every time. Covers the
-governance shapes: SessionStart, Stop, UserPromptSubmit, and Bash **and
-Edit/Write** Pre/PostToolUse.
-```typescript
-import { runHarnessTest, scriptModel } from "vigiles/harness-test";
-const r = await runHarnessTest({
-  settings: {
-    hooks: {
-      Stop: [
-        { hooks: [{ type: "command", command: "test -f DONE || exit 2" }] },
-      ],
-    },
-  },
-  model: scriptModel([
-    { text: "I'm done" }, // tries to stop → blocked (no DONE)
-    { tool: "Bash", input: { command: "touch DONE" } },
-    { text: "now done" },
-  ]),
-});
-assert(JSON.parse(r.stdout).num_turns > 1); // the Stop hook forced more work
-```
-### Level 3 — does it change what Claude does? (real AI, occasional)
-`runEval` runs the **real** model N times with your change **on vs off** and
-reports the gap as **mean ± se** — so you can tell signal from noise instead of
-eyeballing two averages:
-```typescript
-import { runEval, formatEvalReport } from "vigiles/eval";
-const report = await runEval({
-  arms: { off: {}, on: { settings: { hooks: { PostToolUse: [refsHook] } } } },
-  task: "Document chargeCard in SKILL.md, referencing it by name.",
-  measure: (ctx) => ({
-    marked: ctx.sh("grep -c vigiles:symbol SKILL.md") !== "0",
-  }),
-  trials: 6,
-  cache: "readwrite", // replay past runs — editing `measure` re-scores for free
-});
-console.log(formatEvalReport(report));
-// off  marked=0.00   on  marked=0.50±0.20 pass^k=0   ($0.07 · 1.2s/run · 4.1k tok)
-```
-`assertSignificant(report, { baseline: "off", arm: "on", metric: "marked" })`
-turns the gap into a CI gate — a Welch t-test decides whether it cleared the
-noise floor, **computed** from the arms' spread, not hand-fed. Runs go
-**concurrently**, track **cost / latency / tokens** (cap them with `maxCostUsd`),
-and the **record/replay cache** makes re-scoring after a `measure` edit free.
-Same tier, different question: **`measureTriggerRate`** measures how reliably a
-skill's _description fires_ across varied prompts — the #1 skill-authoring pain.
-### Test your skills for real — and assert on what Claude _did_
-Install a plugin the way Claude actually does (`pluginDir` → `--plugin-dir`) so
-its **skills genuinely activate**, then assert on the agent's _actions_, not a
-stdout grep:
-```typescript
-import { assertSkillResolved, assertToolNotUsed } from "vigiles/harness-assert";
-const r = await runHarnessTest({
-  pluginDir: "./my-plugin",
-  transcript: true, // populate r.toolCalls
-  allowedTools: ["Read", "Write", "Bash", "Skill"],
-  model: scriptModel([
-    { tool: "Skill", input: { skill: "my-plugin:greet" } },
-    { text: "ok" },
-  ]),
-});
-assertSkillResolved(r, "my-plugin:greet"); // the skill fired, no error
-assertToolNotUsed(r, /^mcp__github__merge/); // the safety negative: the scary tool was never called
-```
-`assertToolNotUsed` is how you test a safety rule **honestly** — _proving_ the
-dangerous tool was never used, which "the file looks unchanged" can't. It works
-on **real third-party plugins** too: the suite confirms real `obra/superpowers`
-and `wshobson/agents` skills resolve this way, with no markers injected.
-### Did the injected context actually reach the model?
-A SessionStart hook or a slash command can _fire_ and still inject **nothing** —
-wrong output shape, wrong platform. `trace.modelRequests` records what the model
-actually received (system + messages), so you assert it landed, not just that the
-hook ran — **"fired ≠ landed"**:
-```typescript
-import { assertRequestContains } from "vigiles/harness-assert";
-assertRequestContains(r, "You have superpowers"); // the additionalContext reached the model
-```
-(Dogfood: this is exactly how vigiles found that real `obra/superpowers` emits a
-_top-level_ `additionalContext`, which Claude Code — reading the _nested_ form —
-never injects. The hook fired; the context never landed.)
-### Running an untrusted plugin? It's confined by default
-Testing a third-party plugin means executing **its** hooks. `runHarnessTest` is
-safe by default: code you wrote (inline `settings`/`files`) runs directly, but an
-external `plugin` / `pluginDir` is **confined under bubblewrap** — a network
-namespace with **no egress** (a malicious hook can't phone home), a read-only
-filesystem, and a **cleared environment** (your `ANTHROPIC_API_KEY` and other
-secrets aren't even visible). If no sandbox is available the run **refuses**
-rather than executing unconfined:
-```typescript
-runHarnessTest({ pluginDir: "./vendor/some-plugin", model }); // confined, or refuses
-runHarnessTest({ pluginDir: "./audited", model, sandbox: false }); // you vouch for it → direct
-```
-Confinement is **Linux-only** (bubblewrap); on macOS / Windows an untrusted run
-refuses unless you pass `sandbox: false`. The suite dogfoods it on real
-`obra/superpowers` — its `SessionStart` hook runs in a no-egress sandbox, and the
-test proves egress is blocked while the scripted mock stays reachable.
-### Run them in CI
-`vigiles test` runs `*.harness.mjs` files (free, no key); `vigiles eval` runs
-`*.eval.mjs` files (real model). Point a test at a whole plugin (or `"./"` for
-your repo) to load **what ships** — hooks (with `${CLAUDE_PLUGIN_ROOT}`
-resolved), CLAUDE.md, skills, subagents, commands — and `loadPlugin().warnings`
-flags anything only a real model can drive, so you never silently test an empty
-machine.
-```bash
-npx vigiles test examples/harness/policy-gate.harness.mjs
-npx vigiles eval --trials=6 examples/harness/skill-outcome.eval.mjs
-```
-### What's covered today — surface × tier
-| Surface                                                       | Unit / static                | Integration (no API key)    | Eval (real model) |
-| ------------------------------------------------------------- | ---------------------------- | --------------------------- | ----------------- |
-| Hooks — Bash / SessionStart / Stop / UserPromptSubmit         | ✅ logic                     | ✅ fires                    | ✅                |
-| Hooks — Edit / Write                                          | ✅ logic                     | ✅ fires                    | ✅                |
-| Hooks — PreCompact / Notification / SessionEnd / SubagentStop | ✅ logic                     | — (mock can't trigger)      | 🟡                |
-| CLAUDE.md / instructions                                      | ✅ refs                      | 🟡 present, not behaviour   | ✅ behaviour      |
-| Skills                                                        | 🟡 refs                      | ✅ resolves via `pluginDir` | ✅ activation     |
-| Subagents (`agents/`)                                         | ✅ tool rail · 🟡 refs       | 🟡 rail not live-armed      | ✅ via Task       |
-| Slash commands (`commands/`)                                  | 🟡 refs                      | 🟡 needs prompt capture     | ✅ via `/cmd`     |
-| MCP servers                                                   | ✅ tool refs (`vigiles:mcp`) | 🔴                          | 🔴                |
-| settings.json                                                 | 🟡 assert merged             | ✅ applied                  | ✅                |
-| Hook context injection (does it _land_?)                      | — n/a                        | ✅ `trace.modelRequests`    | ✅                |
-| Untrusted plugin execution                                    | — n/a                        | ✅ confined (bwrap, Linux)  | 🟡 outer sandbox  |
-✅ shipped · 🟡 partial · 🔴 gap · — n/a. Full detail + roadmap: [`research/harness-testing-coverage-matrix.md`](research/harness-testing-coverage-matrix.md).
-[Full guide → `docs/harness-testing.md`](docs/harness-testing.md) · [benchmarks](research/benchmarks-runtime-gates.md).
+**[Full guide → `docs/harness-testing.md`](docs/harness-testing.md)** — the tier
+walkthrough, testing skills for real, "fired ≠ landed" (`trace.modelRequests`),
+the safe-by-default sandbox for untrusted plugins, the surface × tier coverage
+matrix, and how it compares to promptfoo. Also: [benchmarks](research/benchmarks-runtime-gates.md).
 ## CLI & CI
@@ -456,43 +164,29 @@ npx vigiles eval        # Run *.eval.mjs real-model harness evals (--trials=N)
 Install with [Vercel Skills](https://github.com/vercel-labs/skills): `npx skills add zernie/vigiles`
 <details>
-<summary><b>The 7 skills</b></summary>
-| Skill                  | What it does                                                            |
-| ---------------------- | ----------------------------------------------------------------------- |
-| `strengthen`           | Upgrade `guidance()` → `enforce()` using linter-specific reference docs |
-| `edit-spec`            | Edit a spec file — guided workflow with compile step                    |
-| `migrate-to-spec`      | Convert a hand-written CLAUDE.md to a typed `.spec.ts`                  |
-| `generate-rule`        | Add a new `enforce()` / `guidance()` rule to a spec                     |
-| `pr-to-lint-rule`      | Turn a recurring PR review comment into a lint rule + spec entry        |
-| `enforce-rules-format` | Validate all rules have enforcement classification                      |
-| `audit-feedback-loop`  | Score your repo's feedback loop maturity                                |
-</details>
-## Maturity Levels
-From [Feedback Loop Is All You Need](https://zernie.com/blog/feedback-loop-is-all-you-need): **Vibes → Guardrails → Architecture as Code → The Organism**.
-<details>
-<summary>What each level means</summary>
-| Level | Name                 | What it means                                                       |
-| ----- | -------------------- | ------------------------------------------------------------------- |
-| 0     | Vibes                | No CI, no linters, no CLAUDE.md                                     |
-| 1     | Guardrails           | CI + standard linters, no custom rules                              |
-| 2     | Architecture as Code | Custom lint rules + enforced CLAUDE.md                              |
-| 3     | The Organism         | CI + custom rules + visual tests + observability + scheduled agents |
+<summary><b>The 8 skills</b></summary>
+| Skill                  | What it does                                                                                      |
+| ---------------------- | ------------------------------------------------------------------------------------------------- |
+| `strengthen`           | Upgrade `guidance()` → `enforce()` using linter-specific reference docs                           |
+| `edit-spec`            | Edit a spec file — guided workflow with compile step                                              |
+| `migrate-to-spec`      | Convert a hand-written CLAUDE.md to a typed `.spec.ts`                                            |
+| `generate-rule`        | Add a new `enforce()` / `guidance()` rule to a spec                                               |
+| `pr-to-lint-rule`      | Turn a recurring PR review comment into a lint rule + spec entry                                  |
+| `enforce-rules-format` | Validate all rules have enforcement classification                                                |
+| `audit-feedback-loop`  | Score your repo's feedback loop maturity                                                          |
+| `test-harness`         | Test a Claude Code harness — pick the tier (unit / deterministic / eval) and write a passing test |
 </details>
 ## Related Tools
-vigiles owns one thing: compile-time verification of typed specs against real linter configs, filesystems, and package.json, plus testing the harness those specs describe. Everything else it composes with rather than replaces — architectural linters ([ast-grep](https://ast-grep.github.io/), [Dependency Cruiser](https://github.com/sverweij/dependency-cruiser)) referenced via `enforce()`, file-sync tools ([Ruler](https://github.com/intellectronica/ruler), [rulesync](https://github.com/dyoshikawa/rulesync)) that distribute the compiled output, and markdown/prose linters that check a different layer. [How vigiles composes with each, and why runtime-LLM rule checkers are the opposite paradigm →](docs/related-tools.md)
+vigiles composes with other tools rather than replacing them: architectural linters ([ast-grep](https://ast-grep.github.io/), [Dependency Cruiser](https://github.com/sverweij/dependency-cruiser)) referenced via `enforce()`, and file-sync tools ([Ruler](https://github.com/intellectronica/ruler), [rulesync](https://github.com/dyoshikawa/rulesync)) that distribute the compiled output. [How it fits with each, and why runtime-LLM rule checkers are the opposite paradigm →](docs/related-tools.md)
 ## Documentation
-- **[docs/](docs/README.md)** — how-to & reference: the adoption ladder, CLI, linter support, the harness-testing guide, skills/agents.
+- **The two pillar guides:** [verifying instruction files](docs/verifying-instruction-files.md) (Pillar 1) · [testing your harness](docs/harness-testing.md) (Pillar 2).
+- **[docs/](docs/README.md)** — the full how-to & reference index: adoption ladder, CLI, linter support, skills/agents.
 - **[research/](research/README.md)** — the thinking behind it: design docs, the [harness-testing coverage roadmap](research/harness-testing-coverage-matrix.md), benchmark findings, landscape, and parked ideas.
 ## License

package/dist/claude-code.d.ts ADDED Viewed

@@ -0,0 +1,10 @@
+/**
+ * `vigiles/claude-code` — the Claude Code-specific harness pieces a *different*
+ * harness would swap out: the plugin/repo loader (reads real Claude Code plugin
+ * layouts) and the scriptable Anthropic Messages mock. Split from
+ * `vigiles/testing` on purpose — the test API above is the stable surface; this
+ * is the adapter, so a future `vigiles/<other-harness>` can sit beside it.
+ */
+export * from "./plugin-loader.js";
+export * from "./mock-model.js";
+//# sourceMappingURL=claude-code.d.ts.map

package/dist/claude-code.js ADDED Viewed

@@ -0,0 +1,26 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __exportStar = (this && this.__exportStar) || function(m, exports) {
+    for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+/**
+ * `vigiles/claude-code` — the Claude Code-specific harness pieces a *different*
+ * harness would swap out: the plugin/repo loader (reads real Claude Code plugin
+ * layouts) and the scriptable Anthropic Messages mock. Split from
+ * `vigiles/testing` on purpose — the test API above is the stable surface; this
+ * is the adapter, so a future `vigiles/<other-harness>` can sit beside it.
+ */
+__exportStar(require("./plugin-loader.js"), exports);
+__exportStar(require("./mock-model.js"), exports);
+//# sourceMappingURL=claude-code.js.map

package/dist/egress-proxy.d.ts ADDED Viewed

	@@ -0,0 +1,2 @@
1	+ export {};
2	+ //# sourceMappingURL=egress-proxy.d.ts.map

package/dist/egress-proxy.js ADDED Viewed

@@ -0,0 +1,60 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+/**
+ * Recording egress proxy — runs INSIDE the sandbox netns (on loopback), so a
+ * confined hook configured with `HTTP(S)_PROXY` routes its network attempts here.
+ * It RECORDS each target (`host:port`) to an ndjson log and BLOCKS it (responds
+ * 502 / closes) — the netns already has no external route, so nothing actually
+ * leaves; this just turns "silently blocked" into "blocked AND recorded", so a
+ * test can assert what a hook/skill tried to reach (phone-home / which registry
+ * an install would hit).
+ *
+ * Honest limit: this records what PROXY-honoring tools (npm, pip, curl, fetch)
+ * attempt. Raw-socket egress bypasses the proxy — but the netns still blocks it
+ * hard, so it can't get out; it just won't appear in the record. The block is the
+ * boundary; the record is best-effort observability over it.
+ *
+ * Run as: `node dist/egress-proxy.js <egress-log-path> <port-file-path>`.
+ */
+/* v8 ignore start -- a standalone subprocess run only inside the sandbox netns;
+   exercised by the bwrap-gated end-to-end test, not the unit gate. The pure
+   parser (parseEgressLog) carries the testable logic. */
+const node_http_1 = require("node:http");
+const node_fs_1 = require("node:fs");
+const [, , logPath, portPath] = process.argv;
+function record(host, port) {
+    try {
+        (0, node_fs_1.appendFileSync)(logPath, JSON.stringify({ host, port, ts: Date.now() }) + "\n");
+    }
+    catch {
+        /* best-effort: a recording failure must not crash the hook under test */
+    }
+}
+const server = (0, node_http_1.createServer)((req, res) => {
+    // Plain HTTP via a proxy: req.url is absolute, e.g. http://host:port/path.
+    try {
+        const u = new URL(req.url ?? "");
+        record(u.hostname, Number(u.port) || 80);
+    }
+    catch {
+        /* unparseable target — skip */
+    }
+    res.writeHead(502, { "content-type": "text/plain" });
+    res.end("blocked by vigiles egress recorder\n");
+});
+// HTTPS via a proxy: the client sends `CONNECT host:port`. Record + refuse.
+server.on("connect", (req, socket) => {
+    const [host, port] = (req.url ?? "").split(":");
+    record(host, Number(port) || 443);
+    socket.write("HTTP/1.1 502 Blocked\r\n\r\n");
+    socket.end();
+});
+server.on("clientError", (_e, socket) => socket.destroy());
+server.listen(0, "127.0.0.1", () => {
+    const addr = server.address();
+    const port = typeof addr === "object" && addr ? addr.port : 0;
+    // Hand the chosen port back to the wrapper, which exports HTTP(S)_PROXY.
+    (0, node_fs_1.writeFileSync)(portPath, String(port));
+});
+/* v8 ignore stop */
+//# sourceMappingURL=egress-proxy.js.map