vigiles 6.0.0 → 8.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (92) hide show
  1. package/README.md +189 -88
  2. package/dist/action-gate.js +1 -1
  3. package/dist/adapters/claude-code/agent-runtime.d.ts +46 -11
  4. package/dist/adapters/claude-code/agent-runtime.js +95 -24
  5. package/dist/adapters/claude-code/effect-region.js +1 -1
  6. package/dist/adapters/claude-code/skill-runtime.d.ts +1 -1
  7. package/dist/adapters/claude-code/skill-runtime.js +1 -1
  8. package/dist/adapters/codex/hook-protocol.js +3 -0
  9. package/dist/adapters/codex/mock-model.js +1 -1
  10. package/dist/cli-commands.d.ts +19 -0
  11. package/dist/cli-commands.js +47 -0
  12. package/dist/cli.d.ts +1 -1
  13. package/dist/cli.js +1054 -201
  14. package/dist/core/adopt.d.ts +65 -0
  15. package/dist/core/adopt.js +199 -0
  16. package/dist/core/bash-effects.d.ts +12 -0
  17. package/dist/core/bash-effects.js +31 -0
  18. package/dist/core/capability-diff.d.ts +46 -0
  19. package/dist/core/capability-diff.js +97 -0
  20. package/dist/core/compose.d.ts +1 -1
  21. package/dist/core/compose.js +1 -1
  22. package/dist/core/evolve.d.ts +4 -0
  23. package/dist/core/evolve.js +4 -0
  24. package/dist/core/frontmatter.d.ts +8 -7
  25. package/dist/core/frontmatter.js +8 -7
  26. package/dist/core/generate-harness.d.ts +1 -1
  27. package/dist/core/generate-harness.js +3 -3
  28. package/dist/core/generate-schema.js +1 -1
  29. package/dist/core/guards.d.ts +126 -0
  30. package/dist/core/guards.js +309 -0
  31. package/dist/core/harness-driver.d.ts +1 -1
  32. package/dist/core/hook-program.d.ts +459 -0
  33. package/dist/core/hook-program.js +468 -0
  34. package/dist/core/hook-protocol.d.ts +7 -0
  35. package/dist/core/hook-providers.d.ts +138 -0
  36. package/dist/core/hook-providers.js +155 -0
  37. package/dist/core/hook-spec.d.ts +74 -0
  38. package/dist/core/hook-spec.js +130 -0
  39. package/dist/core/inline.d.ts +6 -6
  40. package/dist/core/inline.js +7 -7
  41. package/dist/core/integrity.d.ts +31 -0
  42. package/dist/core/integrity.js +45 -0
  43. package/dist/core/mcp-tool.d.ts +12 -0
  44. package/dist/core/mcp-tool.js +20 -0
  45. package/dist/core/mcp.d.ts +13 -0
  46. package/dist/core/mcp.js +67 -0
  47. package/dist/core/orphans.js +1 -1
  48. package/dist/core/spec.d.ts +40 -2
  49. package/dist/core/spec.js +16 -1
  50. package/dist/core/types.d.ts +37 -5
  51. package/dist/core/validate.js +26 -26
  52. package/dist/dialect-drift.d.ts +65 -0
  53. package/dist/dialect-drift.js +216 -0
  54. package/dist/eval.d.ts +40 -5
  55. package/dist/eval.js +59 -5
  56. package/dist/guardrail-check.d.ts +85 -0
  57. package/dist/guardrail-check.js +152 -0
  58. package/dist/harness-assert.d.ts +10 -0
  59. package/dist/harness-assert.js +30 -0
  60. package/dist/hook-install.d.ts +43 -0
  61. package/dist/hook-install.js +91 -0
  62. package/dist/hook.d.ts +52 -0
  63. package/dist/hook.js +98 -0
  64. package/dist/leaderboard.d.ts +6 -0
  65. package/dist/leaderboard.js +43 -1
  66. package/dist/linting.d.ts +9 -5
  67. package/dist/linting.js +17 -5
  68. package/dist/optimize.js +1 -1
  69. package/dist/scaffold-test.js +21 -7
  70. package/dist/scan-behavioral.d.ts +60 -0
  71. package/dist/scan-behavioral.js +239 -1
  72. package/dist/scan-trigger-suggest.d.ts +54 -0
  73. package/dist/scan-trigger-suggest.js +70 -0
  74. package/dist/scan.d.ts +31 -1
  75. package/dist/scan.js +65 -3
  76. package/dist/score-explainer.js +1 -1
  77. package/dist/self-command-refs.d.ts +21 -0
  78. package/dist/self-command-refs.js +125 -0
  79. package/dist/setup-plan.d.ts +59 -1
  80. package/dist/setup-plan.js +103 -5
  81. package/dist/testing.d.ts +5 -3
  82. package/dist/testing.js +37 -23
  83. package/dist/tool-intercept.d.ts +4 -4
  84. package/dist/tool-intercept.js +5 -5
  85. package/dist/unit.d.ts +2 -0
  86. package/dist/unit.js +8 -1
  87. package/hooks/post-edit.sh +1 -1
  88. package/hooks/refs-nudge.sh +1 -1
  89. package/package.json +5 -3
  90. package/skills/adopt-spec/SKILL.md +7 -7
  91. package/skills/linter-docs/eslint.md +1 -1
  92. package/skills/strengthen/SKILL.md +1 -1
package/README.md CHANGED
@@ -1,3 +1,44 @@
1
+ <!--
2
+ README DIRECTION — read before editing; keep changes aligned.
3
+ This file is the FRONT DOOR + a marketing asset for someone who already lives
4
+ in Claude Code / Codex. Optimize for a phone-skimmer.
5
+
6
+ 1. LEAD WITH BENEFITS, not mechanics or vocabulary. Say what the user GETS
7
+ (a guard that can't silently fail; a CLAUDE.md that stops lying) before how.
8
+ 1b. NEVER OPEN WITH A NEGATIVE, APOLOGY, OR CAVEAT. A bolded lead-in is the
9
+ FIRST thing a skimmer reads, so it must be the benefit/on-ramp, never a
10
+ deficiency or competitor: write "Start in plain markdown", NOT "No
11
+ TypeScript?". Put the STRONGEST proof (e.g. 2/7→7/7) on its OWN line, never
12
+ buried mid-paragraph, and END a section on the win, not the caveat (demote
13
+ trade-offs to a trailing aside). Break run-on em-dash/semicolon chains — a
14
+ paragraph is ≤ ~3 lines, one idea.
15
+ 1c. LEAD WITH THE CONCRETE PAIN the reader already feels — named in THEIR
16
+ situation, with the SPECIFIC silent failure, not an abstraction. "You
17
+ installed plugins and wrote skills — but do they actually work? A skill that
18
+ never fires, a hook that blocks nothing, a CLAUDE.md full of dead refs" beats
19
+ "reliability for your harness"; "a library with no tests" is the anchoring
20
+ analogy. This is NOT a 1b violation: a pain about the READER's situation is a
21
+ hook, not an apology — 1b bans opening with vigiles's OWN deficiency, a
22
+ caveat, or a competitor, never the user's pain. SAME FOR THE SUBDOCS: open
23
+ every guide with the concrete pain, THEN the "what this doc is" line + the
24
+ README uplink (per docs-quality in CLAUDE.md).
25
+ 2. COMPILED SPECS ARE EASY — reassure, never intimidate. You rarely hand-write
26
+ a .spec.ts: model-invocable skills (edit-spec / strengthen / test-harness)
27
+ author + edit it, and hooks auto-compile on save. Present the typed spec as
28
+ the SMOOTH, AUTO-ENFORCED next step you don't think about — the natural
29
+ evolution (JS→TS: gradual, strict opt-in), NEVER a wall. Markdown + inline
30
+ mode work with zero TS for anyone not there yet (progressive adoption).
31
+ 3. THE INSTRUMENTS stay first-class — including Eval (measuring whether a skill
32
+ actually helps is core, not optional). NOTE: Guard / compiled hooks is PARKED
33
+ FOR LAUNCH (commented out below; see research/roadmap.md "Launch readiness") —
34
+ so the live set is Lint/Test/Eval ("three instruments"); re-add Guard post-HN.
35
+ 4. SCANNABLE + SHORT — ~200-line cap; punchy table cells, bullets, runnable
36
+ blocks; benefits over jargon. Push depth into docs/ and LINK it.
37
+ 5. NO INTERNAL VOCABULARY (moat / measurement-authority / flywheel) and NO
38
+ research/ links — name the user benefit (see public-vs-internal-docs +
39
+ readme-brevity in CLAUDE.md).
40
+ -->
41
+
1
42
  <p align="center">
2
43
  <img src="logo.png" width="140" alt="vigiles logo" />
3
44
  </p>
@@ -5,11 +46,7 @@
5
46
  <h1 align="center">vigiles</h1>
6
47
 
7
48
  <p align="center">
8
- <strong>Lint, test &amp; measure the harness your AI agent runs on.</strong>
9
- </p>
10
-
11
- <p align="center">
12
- Your CLAUDE.md, hooks, and skills steer the agent — but nothing checks they're <em>true</em>, nothing tests they <em>work</em>, and nothing measures whether they actually <em>help</em>. vigiles does all three.
49
+ <strong>Make the harness your AI agent runs on reliable.</strong>
13
50
  </p>
14
51
 
15
52
  <p align="center">
@@ -20,57 +57,85 @@
20
57
 
21
58
  ---
22
59
 
23
- `Agent = Model + Harness`. You'd never ship an app without a linter, a test suite,
24
- and a benchmark yet the harness steering your agent runs on vibes. vigiles[^name]
25
- is the deterministic layer for it and the only one that can **A/B-test what
26
- actually moves the needle on your Claude subscription**, not metered API. Three
27
- things, adopt any:
60
+ **You installed a bunch of plugins and wrote a few skills but do they actually work?**
61
+ A skill that never fires, a safety hook that blocks nothing, a CLAUDE.md full of dead
62
+ references your harness fails **silently**, and you find out mid-task.
63
+
64
+ **It's a library with no tests.**
28
65
 
29
- | | |
30
- | -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
31
- | **🔎 Lint** | Every file path, script, code symbol, and linter rule your CLAUDE.md cites is checked against reality so a renamed file or a disabled rule can't silently mislead the agent. **[→](docs/verifying-instruction-files.md)** |
32
- | **🧪 Test** | Hooks, skills, and subagents are code. vigiles tests they _do their job_ — and almost all of it is **deterministic, no API key**; the real-model evals run on your **Claude subscription**, not metered tokens. **[→](docs/harness-testing.md)** |
33
- | **📊 Measure** | Does that skill or plugin actually help — or just add cost? **A/B it on real tasks** and read the bill + correctness, not the vendor's headline. The same engine ranks the hyped ecosystem: **what works vs hype**. **[→](docs/measuring-skills.md)** |
66
+ `Agent = Model + Harness` — the model gets the headlines, the harness is the half you
67
+ own. vigiles[^name] makes it prove itself **before the agent ever runs** — verify, test,
68
+ measure. Mostly free, no API keyreal-model checks run on your **Claude Pro/Max
69
+ subscription, not metered tokens**.
34
70
 
35
- Pick the one that hurts today. **Works with Claude Code and Codex**
36
- ([`vigiles/codex`](docs/harnesses.md)), and you can
37
- [teach it your own harness](docs/authoring-an-adapter.md).
71
+ Three instruments, adopt any:
72
+
73
+ | | |
74
+ | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
75
+ | **🔎 Lint** | Your CLAUDE.md stops lying — every path, script, symbol & linter rule checked against **reality**. **[→](docs/verifying-instruction-files.md)** |
76
+ | **🧪 Test** | Prove your hooks, skills & subagents do their job — **free, no API key**. **[→](docs/harness-testing.md)** |
77
+ | **📊 Eval** | Know if a skill helps or just costs — **A/B on real tasks**, on your own subscription. **[→](docs/measuring-skills.md)** |
78
+
79
+ <!-- PARKED FOR LAUNCH — Guard / compiled hooks. Re-add this row + the ④ section below post-HN. See research/roadmap.md "Launch readiness".
80
+ | **🛡 Guard** | A safety hook that **can't silently fail open** — write a typed function, get a guard that blocks. **[→](docs/compiled-hooks.md)** |
81
+ -->
82
+
83
+ <p align="center">
84
+ <img src="vigiles-demo.gif" width="720" alt="vigiles lint catching a file that moved and a script that was renamed" />
85
+ </p>
86
+ <!-- Regenerate the GIF: `python3 scripts/make-demo-gif.py` (output is verbatim CLI; see scripts/demo.sh for a live asciinema recording). -->
87
+
88
+ **Two ways in** — pick the pain that's yours:
89
+
90
+ - **Run agents on your own repo?** Start with **Lint** below, or `npx vigiles init`.
91
+ - **Ship plugins to a marketplace?** **[`vigiles scan`](docs/for-plugin-authors.md)**
92
+ ranks structural health with no key — see the **[plugin-author guide →](docs/for-plugin-authors.md)**.
93
+
94
+ **Your agent writes the spec — and you can always eject.** You rarely hand-write a
95
+ `.spec.ts`: skills author it for you, **`init` adopts an existing CLAUDE.md
96
+ non-destructively** (untouched until you compile), and plain markdown + inline
97
+ `<!-- vigiles:enforce -->` comments work with zero TypeScript. **Claude Code and
98
+ Codex** ([`vigiles/codex`](docs/harnesses.md)), or [your own
99
+ harness](docs/authoring-an-adapter.md). _(New here? [the FAQ](docs/faq.md) covers the
100
+ "do I write TypeScript / will it touch my files" questions.)_
38
101
 
39
102
  ## Quick start
40
103
 
41
104
  **Paste into Claude Code or Codex:**
42
105
 
43
106
  ```text
44
- Set up vigiles in this repo with good defaults (lint + test, non-interactive).
45
- Verify my CLAUDE.md / AGENTS.md references and show me what's stale, then write
46
- and run a harness test for one of my hooks or skills. Ask me first before gating
47
- it in CI, adding a real-model eval, or enforcing strictly (--strict).
107
+ Set up vigiles in this repo: run `npx vigiles init` and accept the defaults. If I
108
+ already have a CLAUDE.md or AGENTS.md, adopt it into a spec and show me which
109
+ references are stale. Then install the dep, compile, and write + run one harness
110
+ test for a hook or skill of mine. Don't enforce a spec-per-file or add a real-model
111
+ eval without asking me first.
48
112
  ```
49
113
 
114
+ The same prompt works in Codex.
115
+
50
116
  Or do it yourself:
51
117
 
52
118
  ```bash
53
119
  npx vigiles init # sets up lint + test: spec + harness test + CI + plugin
54
120
  ```
55
121
 
56
- It's interactive in a terminal and non-interactive for agents/CI (or with
57
- `--yes`), so "set up vigiles" from a Claude Code / Codex prompt Just Works — and
58
- it installs a model-invocable **`test-harness` skill**, so afterward you can just
59
- tell your agent _"test my skills"_ and it picks the tier and writes the test.
122
+ Interactive in a terminal, non-interactive for agents/CI (or `--yes`).
123
+
124
+ **You don't hand-write any of this your agent does.** `init` installs
125
+ model-invocable skills, so a plain-English ask does the work (it edits the source
126
+ and recompiles on save; you never touch it by hand):
127
+
128
+ - _"test my skills"_ → scaffolds **and runs** a trigger/behaviour test (`test-harness`)
129
+ - _"harden my rules"_ → upgrades prose guidance into enforced linter rules (`strengthen`)
130
+ - _"add a rule to my CLAUDE.md"_ → edits the source and recompiles (`edit-spec`)
60
131
 
61
132
  <details>
62
133
  <summary>What <code>init</code> sets up</summary>
63
134
 
64
- - **Both lint and test** by default; scope with `--lint` / `--test` (one or both).
65
- - Adds `vigiles` to your `devDependencies`.
66
- - Installs the Claude Code plugin (skills + hooks) via the marketplace —
67
- globally, never vendored into your repo.
68
- - Wires CI as a `zernie/vigiles@v1` workflow (a composite over the same CLI):
69
-
70
- ```yaml
71
- - uses: actions/checkout@v4
72
- - uses: zernie/vigiles@v1 # lints by default; posts a sticky PR comment + a `valid` output
73
- ```
135
+ - **Both lint and test** by default; scope with `--lint` / `--test`.
136
+ - **Already have a CLAUDE.md / AGENTS.md? `init` adopts it** into a spec faithfully and **non-destructively** — your file is left untouched until you choose to `compile` (and `eject` undoes it).
137
+ - Adds `vigiles` to `devDependencies`; installs the Claude Code plugin (skills + hooks) via the marketplace — globally, never vendored.
138
+ - Wires CI as a `zernie/vigiles@v1` workflow (a composite over the same CLI) that posts a sticky PR comment + a `valid` output.
74
139
 
75
140
  Prefer to write tests yourself? They can be JS **or** TS
76
141
  (`*.harness.{mjs,ts}`) — run them with `npx vigiles test`.
@@ -79,11 +144,10 @@ Prefer to write tests yourself? They can be JS **or** TS
79
144
 
80
145
  ## ① Lint — your CLAUDE.md lies to your agent
81
146
 
82
- Your CLAUDE.md points the agent at `src/auth/login.ts` and tells it to run
83
- `npm run check`. But the file moved to `src/auth/session.ts` six commits ago, and
84
- the script was renamed. The agent trusts the stale claim and acts on fiction.
85
-
86
- `npx vigiles lint` resolves every reference against reality:
147
+ **Your CLAUDE.md drifts the moment you refactor.** It points the agent at
148
+ `src/auth/login.ts` and says run `npm run check` but the file moved six commits
149
+ ago and the script was renamed. The agent trusts the stale claim and acts on
150
+ fiction. `npx vigiles lint` resolves every reference against reality:
87
151
 
88
152
  ```text
89
153
  CLAUDE.md:
@@ -92,31 +156,25 @@ CLAUDE.md:
92
156
  ✓ @typescript-eslint/no-floating-promises — exists and enabled in eslint config
93
157
  ```
94
158
 
95
- File paths, scripts, and code symbols — plus linter rules across **7 catalogs**
96
- (the rule exists **and is enabled**). Start with one inline comment, no new files;
97
- step up to a typed `.spec.ts` (compiled to CLAUDE.md, compiler-grade) when you want
98
- it and that same spec is what makes your skills & subagents deterministically
99
- **testable** (a `result()` contract an assert, no LLM judge see ② below), not
100
- just lint-able. **[Full guide →](docs/verifying-instruction-files.md)**
101
-
102
- > **Markdown is prose; a typed spec is a _program_.** Opt in and an agent that
103
- > leaks, exceeds its effect floor, or hands off mismatched data is a **type
104
- > error** your **multi-agent pipeline won't compile if the handoffs don't line
105
- > up**. Others lint prose after the fact; vigiles is a **compiler for harnesses**.
106
- > Graduated like TypeScript's `strict` — markdown still works, type-safety is
107
- > opt-in. **[The moat →](research/typed-spec-moat.md)**
108
-
109
- **Same cross-reference, any plugin.** `npx vigiles scan` checks a plugin's
110
- _contracts_ — every subagent tool, `mcp__server__tool`, `mcp_tool` hook, hook
111
- event, and script path actually **exists and resolves**, not just parses (valid
112
- YAML ≠ a tool that's real). A **superset of Anthropic's `claude plugin validate`**,
113
- no key. **[Audit any plugin →](docs/cli.md#scan-dir)**
159
+ File paths, scripts, code symbols — plus linter rules across **7 linters**
160
+ (ESLint, Ruff, Clippy + four more): each rule exists **and is enabled**.
161
+
162
+ **Start with one inline comment no new files.** Step up to a typed `.spec.ts`
163
+ (your agent writes it; it compiles to CLAUDE.md) when you want compiler-grade
164
+ guarantees and `vigiles eject` hands it back to plain markdown anytime.
165
+ **[Full guide →](docs/verifying-instruction-files.md)**
166
+
167
+ > **Want more? Bad states can stop compiling.** Opt in deeper and a broken
168
+ > hand-off between agents becomes a build error instead of a runtime surprise —
169
+ > graduated like TypeScript's `strict`, on only when you want it.
170
+ > **[How →](docs/compiled-hooks.md)**
114
171
 
115
172
  ## ② Test — does your harness do its job?
116
173
 
117
- A hook can be wired wrong. A skill's description can fail to trigger — or hijack
118
- unrelated prompts. Injected context can never reach the model. All of it passes a
119
- naive "did it run?" check. vigiles tests the assembled harness for real:
174
+ **You wired the hook but does it actually block?** A skill's description can fail
175
+ to trigger, or hijack unrelated prompts; injected context can silently never reach
176
+ the model. All of it passes a naive "did it run?" check. vigiles tests the
177
+ assembled harness for real:
120
178
 
121
179
  ```typescript
122
180
  import { runHook } from "vigiles/testing";
@@ -133,18 +191,18 @@ It goes well past _"did it fire?"_:
133
191
 
134
192
  - **Hooks block** what they must — `runHook`, or the real agent CLI via `runHarnessTest`.
135
193
  - **Skills trigger** on the right prompts and stay quiet on the wrong ones — recall _and_ precision (`measureTriggerRate`).
136
- - **Behaviour is good** — score a skill's output directly, or A/B it on-vs-off for the real lift (`measure` / `runEval`, with significance testing).
194
+ - **Behaviour is good** — score a skill's output, or A/B it on-vs-off for the real lift (`measure` / `runEval`, with significance testing).
137
195
  - **Safety holds** — the agent _didn't_ push to the wrong branch or hit a paid API; `interceptTools` catches the attempt so the side effect never happens.
138
196
 
139
197
  Almost every tier runs with **no model and no API key** — milliseconds, on every
140
- commit. Only the real-model evals need a model, and they run on your own `claude`
141
- CLI. **[How it works →](docs/harness-testing.md)**
198
+ commit; only the real-model evals need a model, on your own `claude` CLI.
199
+ **[How it works →](docs/harness-testing.md)**
142
200
 
143
- ## ③ Measure — does it actually help, or just cost more?
201
+ ## ③ Eval — does it actually help, or just cost more?
144
202
 
145
- A skill claims "65% fewer tokens." A plugin promises "3× faster." Stars and
146
- vibes — **zero measurement**. vigiles A/Bs the claim on real coding tasks, the
147
- harness loaded exactly as it ships, and reports the **metric triple**:
203
+ **"65% fewer tokens." "3× faster." Says who?** A skill claims it, a plugin promises
204
+ itstars and vibes, **zero measurement**. vigiles A/Bs the claim on real coding
205
+ tasks, the harness loaded exactly as it ships, and reports the **metric triple**:
148
206
 
149
207
  ```typescript
150
208
  import { measureArms } from "vigiles/testing";
@@ -155,33 +213,76 @@ const r = await measureArms({
155
213
  arms: { baseline: {}, skill: { files: { "SKILL.md": THE_SKILL } } },
156
214
  measure: (ctx) => ({ cost: ctx.usage.costUsd, correct: check(ctx) }),
157
215
  });
158
- // → is the bill lower? did correctness survive? (not just "fewer output tokens")
159
216
  ```
160
217
 
161
- - **The bill (`costUsd`)** — the honest number; it weights cache ~0.1× and output 1×, so a "saved tokens" headline can't hide behind cheap cache.
162
- - **The optimization target** — whatever the skill claims to move (output tokens, latency, tool calls). Verified on its own terms.
218
+ - **The bill (`costUsd`)** — weights cache ~0.1× / output 1×, so a "saved tokens" headline can't hide behind cheap cache.
219
+ - **The target** — whatever the skill claims to move (output tokens, latency, tool calls), verified on its own terms.
163
220
  - **The blast radius** — correctness, a deterministic 1/0. A token win that breaks the code is **not a win**.
164
221
 
165
- The kicker: every run is **your own `claude` CLI on your Pro/Max subscription** —
166
- so you can measure on every change, and the same engine powers the **ecosystem
167
- benchmark** ("we A/B-tested the most-hyped skills what works vs hype"). That's
168
- the moat: continuous, sub-affordable measurement nobody metered-by-the-token can
169
- match. **[Measure a skill →](docs/measuring-skills.md)** · **[Why it's affordable →](docs/eval-architecture.md)**
222
+ **Safe to repeat.** Each real-model run is sandboxed (ephemeral dir, egress blocked
223
+ or allow-listed), and `interceptTools` catches an irreversible external a push, a
224
+ paid API as an _attempt_, never running it. **[Safety, sandboxing & FAQ →](docs/safety.md)**
225
+
226
+ **The eval you can actually afford.** promptfoo / DeepEval hit a metered API and
227
+ bill **per token, every run**. vigiles answers most questions with **no model at
228
+ all**, and runs the rest on your own **Claude Pro/Max subscription — $0 extra**. So
229
+ you can measure on every change. **[Eval a skill →](docs/measuring-skills.md)** · **[Why it's affordable →](docs/eval-architecture.md)**
230
+
231
+ <!-- PARKED FOR LAUNCH — Guard / compiled hooks. Re-add this whole section (and the table row above) post-HN. See research/roadmap.md "Launch readiness".
232
+
233
+ ## ④ Guard — a safety hook that can't silently fail open
234
+
235
+ **Your safety hook looks like it blocks — and doesn't.** A guard is your last stop
236
+ before something irreversible, but a hand-written one **fails open** without telling
237
+ you. _(Already write safety hooks? This is the power tool.)_ Write a pure typed
238
+ function instead; vigiles emits the exit code, the JSON, and an AST-backed matcher:
239
+
240
+ ```typescript
241
+ import { defineHook, tool, deny, allow } from "vigiles/hook";
242
+
243
+ export default defineHook({
244
+ on: "PreToolUse",
245
+ match: tool("Bash"),
246
+ decide: (e) =>
247
+ e.command.runs("git push", { force: true })
248
+ ? deny("no force-push to a protected branch")
249
+ : allow(),
250
+ });
251
+ ```
252
+
253
+ **The proof:** a widely-copied OSS safety hook blocks **2/7** of the disaster
254
+ battery. The compiled rewrite blocks **7/7** — measured, not asserted.
255
+
256
+ You never hand-write the exit code or JSON field (the usual false confidence), the
257
+ matcher is **AST-backed** (it catches the `cd x && git push -f` a glob misses), and
258
+ the artifact is **stamped** so a later hand-edit is refused.
259
+
260
+ _Scope: this fixes a hook's logic, not the harness's delivery — a subagent's tool
261
+ calls still bypass any PreToolUse hook
262
+ ([#34692](https://github.com/anthropics/claude-code/issues/34692)), so it's a strong
263
+ default, not an unbypassable wall._
264
+ **[Compiled hooks — bug classes + trade-offs →](docs/compiled-hooks.md)**
265
+
266
+ -->
267
+
268
+ ## FAQ
170
269
 
171
- **The eval you can actually afford.** Most of vigiles needs no model at all; the
172
- rest drives your subscription:
270
+ - **Isn't this just a markdown linter?** No it checks whether your instruction file is _true_ (every path/script/symbol/rule exists and is enabled), then tests and measures your harness. A style linter can't do any of that.
271
+ - **Do I have to write TypeScript?** No. Lint your markdown with zero new files; when you want a spec, your agent writes it. It's gradual, like TS's `strict`.
272
+ - **Does it overwrite my files?** No. `init` adopts an existing CLAUDE.md _non-destructively_ — untouched until you `compile`, and `eject` reverses it.
273
+ - **Need an API key?** No for almost everything (free, every commit). Real-model evals run on your Claude Pro/Max subscription — $0 metered tokens.
274
+ - **Non-JS repo?** `npx vigiles lint` verifies your CLAUDE.md with no install (Ruff/Clippy/Pylint/… too).
173
275
 
174
- | | Runs on | Cost |
175
- | ---------------------- | ----------------------- | ------------------------------------------- |
176
- | promptfoo, DeepEval, … | metered API SDK | billed **per token, every run** |
177
- | **vigiles** | your Claude Pro/Max sub | **$0 extra** — and most tiers need no model |
276
+ **[Full FAQ →](docs/faq.md)**
178
277
 
179
278
  ## More
180
279
 
181
- - **[Plugin health leaderboard →](docs/cli.md#scan-dir)** — point `scan` at a marketplace (e.g. `wshobson/agents`) and it ranks every plugin by structural health (0–100, A–F), worst issues first still **no key**. Add `--trigger` for the model-gated column: do the skills actually fire?
182
- - **[CLI & GitHub Action →](docs/cli.md)** — every command, the Action (inputs / output / versioning), and the Claude Code plugin. The full **[lint rules matrix →](docs/verifying-instruction-files.md#the-validation-rules--the-full-matrix)** lives with the linting guide.
183
- - **[Skills →](docs/skills.md)** — consumer skills installed as a Claude Code plugin: `/plugin marketplace add zernie/vigiles` then `/plugin install vigiles@vigiles` (or let `vigiles init` do it). The model-invocable ones (`test-harness`, `strengthen`, `edit-spec`) fire on their own — ask _"test my skills"_, _"strengthen my rules"_, or _"add a rule to CLAUDE.md"_ and the agent reaches for them; `adopt-spec` and `linter-docs` are user-invoked.
184
- - **[Docs index →](docs/README.md)** · **[Research →](research/README.md)** · **[Related tools →](docs/related-tools.md)** (ast-grep, Dependency Cruiser, Ruler, rulesync).
280
+ - **[CLI →](docs/cli.md)** — every command and the plugin · **[GitHub Action →](docs/github-action.md)** run it in CI. The full **[lint rules matrix →](docs/verifying-instruction-files.md#the-validation-rules--the-full-matrix)** lives with the linting guide.
281
+ - **[Skills →](docs/skills.md)** — the skills `init` installs, and how the model-invocable ones trigger.
282
+ - **[Ship plugins? The plugin-author guide →](docs/for-plugin-authors.md)** — scan a draft for structural health, make your skills fire for users, then rank a whole marketplace (0–100, A–F, worst issues first) **no key**.
283
+ - **[Docs index →](docs/README.md)** · **[API reference →](https://zernie.github.io/vigiles/)** · **[Related tools →](docs/related-tools.md)** (ast-grep, Dependency Cruiser, Ruler, rulesync).
284
+ - **[Stability →](STABILITY.md)** — 0.x: the CLI is stable; the library API is still evolving; experimental surfaces are marked.
285
+ - **Not for you if** you want a model/capability benchmark or runtime guardrails in the request path — vigiles is build-/CI-time.
185
286
  - Companion to [Feedback Loop Is All You Need](https://zernie.com/blog/feedback-loop-is-all-you-need).
186
287
 
187
288
  ## License
@@ -13,7 +13,7 @@ exports.loadActionGates = loadActionGates;
13
13
  *
14
14
  * It is the same deterministic gate primitive (reuses `runGate` + the
15
15
  * author-time reference resolution), re-anchored from step → action. Delivered
16
- * as a PostToolUse hook (`vigiles action-hook`): exit 2 blocks the action and
16
+ * as a PostToolUse hook (`vigiles hook-runtime action`): exit 2 blocks the action and
17
17
  * feeds the reason back, exit 0 allows it.
18
18
  *
19
19
  * Config: `.vigiles/action-gates.json` → `{ "gates": [ { on, gate, when? } ] }`.
@@ -10,7 +10,7 @@
10
10
  * doesn't list.
11
11
  *
12
12
  * This is the same emit-a-hook pattern the skill runtime already ships
13
- * (`src/skill-runtime.ts`): there a `Stop` hook reads the active skill's
13
+ * (`src/adapters/claude-code/skill-runtime.ts`): there a `Stop` hook reads the active skill's
14
14
  * compiled SKILL.md and runs its result gate; here a `PreToolUse` hook reads
15
15
  * the active agent's compiled `.md`, parses its `tools:` allowlist, and
16
16
  * allows/denies the tool call. The compiled markdown's frontmatter is the
@@ -66,21 +66,56 @@ export interface PreToolDecision {
66
66
  * feeding the contract back to the model so it self-corrects.
67
67
  */
68
68
  export declare function decidePreToolUse(allowed: readonly string[] | null, tool: string): PreToolDecision;
69
- /** Record the subagent currently dispatched, so PreToolUse enforces its contract. */
69
+ /**
70
+ * Read the active-agent stack (oldest → newest; the dispatched subagent chain).
71
+ * Back-compat: a legacy single-slot `{ agent: string }` marker reads as a one-frame
72
+ * stack; a malformed file or non-string entries → an empty stack (fail-open).
73
+ */
74
+ export declare function readActiveStack(cwd: string): string[];
75
+ /**
76
+ * Push a dispatched subagent onto the active stack (a `PreToolUse` spawn). The
77
+ * PreToolUse gate then enforces this subagent's contract until it returns. Under
78
+ * nesting each dispatch pushes a frame, so the chain is tracked, not overwritten.
79
+ */
80
+ export declare function pushActiveAgent(cwd: string, agentPath: string): void;
81
+ /**
82
+ * Pop the top frame — the subagent returned (`SubagentStop`), so control returns
83
+ * to its PARENT (the next frame down), whose contract the gate enforces again.
84
+ * This is the nesting-safe close, distinct from {@link clearActiveAgent} (which
85
+ * drops the whole stack). Popping an empty stack is a no-op.
86
+ */
87
+ export declare function popActiveAgent(cwd: string): void;
88
+ /**
89
+ * Push a subagent frame (the manual `agent-start` fallback + the deterministic
90
+ * spawn open-signal both use this). A single call is equivalent to a one-frame
91
+ * stack, so the top — what the gate reads — is this agent. Alias of
92
+ * {@link pushActiveAgent} kept under the historical name.
93
+ */
70
94
  export declare function setActiveAgent(cwd: string, agentPath: string): void;
71
- /** Clear the active-agent marker (the subagent finished). */
95
+ /**
96
+ * Clear the WHOLE stack (a hard reset / session end). Distinct from
97
+ * {@link popActiveAgent}, which returns to the parent frame. Idempotent.
98
+ */
72
99
  export declare function clearActiveAgent(cwd: string): void;
73
- /** The path of the active agent's compiled `.md`, or null when none is active. */
100
+ /**
101
+ * The active agent's compiled `.md` — the STACK TOP — or null when none is active.
102
+ * The gate reads the top, so a returned nested subagent reveals its parent's
103
+ * contract again (the contract-escape fix).
104
+ */
74
105
  export declare function readActiveAgent(cwd: string): string | null;
75
106
  /**
76
- * EXPERIMENTAL — parked (P3), flat-only, do NOT auto-wire. This deterministic
77
- * subagent-window tracking assumes FLAT dispatch (one active subagent at a time);
78
- * it is NOT nesting-safe Claude Code v2.1.172 added depth-5 nested subagents, so
79
- * correct tracking needs a depth-aware STACK (push on dispatch, pop on SubagentStop)
80
- * + verifying the spawn tool name (`Agent` vs `Task`). See
81
- * research/effect-boundary-design.md ("Why dropped").
107
+ * EXPERIMENTAL — parked (P3), do NOT auto-wire. The subagent-window tracking is
108
+ * now nesting-safe: a depth-aware STACK (push on dispatch, pop on SubagentStop —
109
+ * see {@link pushActiveAgent}/{@link popActiveAgent}) closes the contract-escape
110
+ * the flat single-slot model allowed under Claude Code v2.1.172 depth-5 nesting
111
+ * (certified in research/prototypes/.../AgentWindowStack.tla). The open signal
112
+ * recognizes BOTH spawn tool names (`Task` and the nested-spawn `Agent`), gated on
113
+ * a resolvable `subagent_type` so a non-spawn call never opens a frame. Still
114
+ * parked because the `effect()` sub-region goal it served was dropped (see
115
+ * research/effect-boundary-design.md, "Why dropped") — the stack is shipped for
116
+ * when active-agent contract enforcement under nesting is wanted on its own.
82
117
  *
83
- * Resolve a `Task` tool's `subagent_type` to the compiled agent `.md` to
118
+ * Resolve a spawn tool's `subagent_type` to the compiled agent `.md` to
84
119
  * activate, or null when none is found. The DETERMINISTIC open signal that
85
120
  * replaces the model-invoked `agent-start`: Claude Code fires `PreToolUse` for
86
121
  * the parent's `Task` dispatch (and `SubagentStop` when it returns), so the
@@ -11,7 +11,7 @@
11
11
  * doesn't list.
12
12
  *
13
13
  * This is the same emit-a-hook pattern the skill runtime already ships
14
- * (`src/skill-runtime.ts`): there a `Stop` hook reads the active skill's
14
+ * (`src/adapters/claude-code/skill-runtime.ts`): there a `Stop` hook reads the active skill's
15
15
  * compiled SKILL.md and runs its result gate; here a `PreToolUse` hook reads
16
16
  * the active agent's compiled `.md`, parses its `tools:` allowlist, and
17
17
  * allows/denies the tool call. The compiled markdown's frontmatter is the
@@ -28,6 +28,9 @@ exports.parseAgentTools = parseAgentTools;
28
28
  exports.parseAgentToolList = parseAgentToolList;
29
29
  exports.parseAgentPurity = parseAgentPurity;
30
30
  exports.decidePreToolUse = decidePreToolUse;
31
+ exports.readActiveStack = readActiveStack;
32
+ exports.pushActiveAgent = pushActiveAgent;
33
+ exports.popActiveAgent = popActiveAgent;
31
34
  exports.setActiveAgent = setActiveAgent;
32
35
  exports.clearActiveAgent = clearActiveAgent;
33
36
  exports.readActiveAgent = readActiveAgent;
@@ -100,46 +103,114 @@ function decidePreToolUse(allowed, tool) {
100
103
  };
101
104
  }
102
105
  // ---------------------------------------------------------------------------
103
- // Active-agent tracking (mirrors .vigiles/active-skill.json)
106
+ // Active-agent tracking — a depth-aware STACK (mirrors .vigiles/active-skill.json)
104
107
  // ---------------------------------------------------------------------------
108
+ //
109
+ // Claude Code v2.1.172 added nested subagents (a subagent with the spawn tool can
110
+ // dispatch its own, up to depth 5). A single active-agent slot is NOT nesting-safe:
111
+ // when an inner subagent returns, clearing the whole slot drops the OUTER agent's
112
+ // contract while it is still running, so the PreToolUse gate then allows a tool the
113
+ // outer subagent forbids — a CONTRACT ESCAPE. The fix (certified in TLC, see
114
+ // research/prototypes/typed-spec-formal-verification/AgentWindowStack.tla) is a
115
+ // STACK: push on dispatch, pop on SubagentStop (back to the parent), gate on the
116
+ // stack TOP. Counterexample the flat model fails and the stack model passes:
117
+ // Open(writer); Open(writer); Stop; Call(Bash).
105
118
  const ACTIVE_PATH = ".vigiles/active-agent.json";
106
- /** Record the subagent currently dispatched, so PreToolUse enforces its contract. */
107
- function setActiveAgent(cwd, agentPath) {
119
+ /**
120
+ * Read the active-agent stack (oldest → newest; the dispatched subagent chain).
121
+ * Back-compat: a legacy single-slot `{ agent: string }` marker reads as a one-frame
122
+ * stack; a malformed file or non-string entries → an empty stack (fail-open).
123
+ */
124
+ function readActiveStack(cwd) {
125
+ const p = (0, node_path_1.resolve)(cwd, ACTIVE_PATH);
126
+ if (!(0, node_fs_1.existsSync)(p))
127
+ return [];
128
+ try {
129
+ const parsed = JSON.parse((0, node_fs_1.readFileSync)(p, "utf-8"));
130
+ if (Array.isArray(parsed.stack)) {
131
+ return parsed.stack.filter((x) => typeof x === "string");
132
+ }
133
+ // legacy single-slot format
134
+ if (typeof parsed.agent === "string")
135
+ return [parsed.agent];
136
+ return [];
137
+ }
138
+ catch {
139
+ return [];
140
+ }
141
+ }
142
+ function writeActiveStack(cwd, stack) {
108
143
  const p = (0, node_path_1.resolve)(cwd, ACTIVE_PATH);
144
+ if (stack.length === 0) {
145
+ if ((0, node_fs_1.existsSync)(p))
146
+ (0, node_fs_1.rmSync)(p);
147
+ return;
148
+ }
109
149
  (0, node_fs_1.mkdirSync)((0, node_path_1.dirname)(p), { recursive: true });
110
- (0, node_fs_1.writeFileSync)(p, JSON.stringify({ agent: agentPath }) + "\n");
150
+ (0, node_fs_1.writeFileSync)(p, JSON.stringify({ stack }) + "\n");
151
+ }
152
+ /**
153
+ * Push a dispatched subagent onto the active stack (a `PreToolUse` spawn). The
154
+ * PreToolUse gate then enforces this subagent's contract until it returns. Under
155
+ * nesting each dispatch pushes a frame, so the chain is tracked, not overwritten.
156
+ */
157
+ function pushActiveAgent(cwd, agentPath) {
158
+ writeActiveStack(cwd, [...readActiveStack(cwd), agentPath]);
159
+ }
160
+ /**
161
+ * Pop the top frame — the subagent returned (`SubagentStop`), so control returns
162
+ * to its PARENT (the next frame down), whose contract the gate enforces again.
163
+ * This is the nesting-safe close, distinct from {@link clearActiveAgent} (which
164
+ * drops the whole stack). Popping an empty stack is a no-op.
165
+ */
166
+ function popActiveAgent(cwd) {
167
+ const stack = readActiveStack(cwd);
168
+ stack.pop();
169
+ writeActiveStack(cwd, stack);
111
170
  }
112
- /** Clear the active-agent marker (the subagent finished). */
171
+ /**
172
+ * Push a subagent frame (the manual `agent-start` fallback + the deterministic
173
+ * spawn open-signal both use this). A single call is equivalent to a one-frame
174
+ * stack, so the top — what the gate reads — is this agent. Alias of
175
+ * {@link pushActiveAgent} kept under the historical name.
176
+ */
177
+ function setActiveAgent(cwd, agentPath) {
178
+ pushActiveAgent(cwd, agentPath);
179
+ }
180
+ /**
181
+ * Clear the WHOLE stack (a hard reset / session end). Distinct from
182
+ * {@link popActiveAgent}, which returns to the parent frame. Idempotent.
183
+ */
113
184
  function clearActiveAgent(cwd) {
114
185
  const p = (0, node_path_1.resolve)(cwd, ACTIVE_PATH);
115
186
  if ((0, node_fs_1.existsSync)(p))
116
187
  (0, node_fs_1.rmSync)(p);
117
188
  }
118
- /** The path of the active agent's compiled `.md`, or null when none is active. */
189
+ /**
190
+ * The active agent's compiled `.md` — the STACK TOP — or null when none is active.
191
+ * The gate reads the top, so a returned nested subagent reveals its parent's
192
+ * contract again (the contract-escape fix).
193
+ */
119
194
  function readActiveAgent(cwd) {
120
- const p = (0, node_path_1.resolve)(cwd, ACTIVE_PATH);
121
- if (!(0, node_fs_1.existsSync)(p))
122
- return null;
123
- try {
124
- const parsed = JSON.parse((0, node_fs_1.readFileSync)(p, "utf-8"));
125
- return typeof parsed.agent === "string" ? parsed.agent : null;
126
- }
127
- catch {
128
- return null;
129
- }
195
+ const stack = readActiveStack(cwd);
196
+ return stack.length > 0 ? stack[stack.length - 1] : null;
130
197
  }
131
198
  // ---------------------------------------------------------------------------
132
199
  // PreToolUse-hook decision
133
200
  // ---------------------------------------------------------------------------
134
201
  /**
135
- * EXPERIMENTAL — parked (P3), flat-only, do NOT auto-wire. This deterministic
136
- * subagent-window tracking assumes FLAT dispatch (one active subagent at a time);
137
- * it is NOT nesting-safe Claude Code v2.1.172 added depth-5 nested subagents, so
138
- * correct tracking needs a depth-aware STACK (push on dispatch, pop on SubagentStop)
139
- * + verifying the spawn tool name (`Agent` vs `Task`). See
140
- * research/effect-boundary-design.md ("Why dropped").
202
+ * EXPERIMENTAL — parked (P3), do NOT auto-wire. The subagent-window tracking is
203
+ * now nesting-safe: a depth-aware STACK (push on dispatch, pop on SubagentStop —
204
+ * see {@link pushActiveAgent}/{@link popActiveAgent}) closes the contract-escape
205
+ * the flat single-slot model allowed under Claude Code v2.1.172 depth-5 nesting
206
+ * (certified in research/prototypes/.../AgentWindowStack.tla). The open signal
207
+ * recognizes BOTH spawn tool names (`Task` and the nested-spawn `Agent`), gated on
208
+ * a resolvable `subagent_type` so a non-spawn call never opens a frame. Still
209
+ * parked because the `effect()` sub-region goal it served was dropped (see
210
+ * research/effect-boundary-design.md, "Why dropped") — the stack is shipped for
211
+ * when active-agent contract enforcement under nesting is wanted on its own.
141
212
  *
142
- * Resolve a `Task` tool's `subagent_type` to the compiled agent `.md` to
213
+ * Resolve a spawn tool's `subagent_type` to the compiled agent `.md` to
143
214
  * activate, or null when none is found. The DETERMINISTIC open signal that
144
215
  * replaces the model-invoked `agent-start`: Claude Code fires `PreToolUse` for
145
216
  * the parent's `Task` dispatch (and `SubagentStop` when it returns), so the
@@ -8,7 +8,7 @@ exports.hasEffectBoundary = hasEffectBoundary;
8
8
  * Effect-boundary position-aware state tracking.
9
9
  * Mirrors active-unit tracking (.vigiles/active-agent.json / .vigiles/active-skill.json).
10
10
  * The PreToolUse hook reads this file to decide whether the agent is inside an
11
- * effect boundary (set by `vigiles effect-enter`) or outside it.
11
+ * effect boundary (set by `vigiles hook-runtime effect-enter`) or outside it.
12
12
  */
13
13
  const node_fs_1 = require("node:fs");
14
14
  const node_path_1 = require("node:path");