vigiles 6.0.0 → 8.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +189 -88
- package/dist/action-gate.js +1 -1
- package/dist/adapters/claude-code/agent-runtime.d.ts +46 -11
- package/dist/adapters/claude-code/agent-runtime.js +95 -24
- package/dist/adapters/claude-code/effect-region.js +1 -1
- package/dist/adapters/claude-code/skill-runtime.d.ts +1 -1
- package/dist/adapters/claude-code/skill-runtime.js +1 -1
- package/dist/adapters/codex/hook-protocol.js +3 -0
- package/dist/adapters/codex/mock-model.js +1 -1
- package/dist/cli-commands.d.ts +19 -0
- package/dist/cli-commands.js +47 -0
- package/dist/cli.d.ts +1 -1
- package/dist/cli.js +1054 -201
- package/dist/core/adopt.d.ts +65 -0
- package/dist/core/adopt.js +199 -0
- package/dist/core/bash-effects.d.ts +12 -0
- package/dist/core/bash-effects.js +31 -0
- package/dist/core/capability-diff.d.ts +46 -0
- package/dist/core/capability-diff.js +97 -0
- package/dist/core/compose.d.ts +1 -1
- package/dist/core/compose.js +1 -1
- package/dist/core/evolve.d.ts +4 -0
- package/dist/core/evolve.js +4 -0
- package/dist/core/frontmatter.d.ts +8 -7
- package/dist/core/frontmatter.js +8 -7
- package/dist/core/generate-harness.d.ts +1 -1
- package/dist/core/generate-harness.js +3 -3
- package/dist/core/generate-schema.js +1 -1
- package/dist/core/guards.d.ts +126 -0
- package/dist/core/guards.js +309 -0
- package/dist/core/harness-driver.d.ts +1 -1
- package/dist/core/hook-program.d.ts +459 -0
- package/dist/core/hook-program.js +468 -0
- package/dist/core/hook-protocol.d.ts +7 -0
- package/dist/core/hook-providers.d.ts +138 -0
- package/dist/core/hook-providers.js +155 -0
- package/dist/core/hook-spec.d.ts +74 -0
- package/dist/core/hook-spec.js +130 -0
- package/dist/core/inline.d.ts +6 -6
- package/dist/core/inline.js +7 -7
- package/dist/core/integrity.d.ts +31 -0
- package/dist/core/integrity.js +45 -0
- package/dist/core/mcp-tool.d.ts +12 -0
- package/dist/core/mcp-tool.js +20 -0
- package/dist/core/mcp.d.ts +13 -0
- package/dist/core/mcp.js +67 -0
- package/dist/core/orphans.js +1 -1
- package/dist/core/spec.d.ts +40 -2
- package/dist/core/spec.js +16 -1
- package/dist/core/types.d.ts +37 -5
- package/dist/core/validate.js +26 -26
- package/dist/dialect-drift.d.ts +65 -0
- package/dist/dialect-drift.js +216 -0
- package/dist/eval.d.ts +40 -5
- package/dist/eval.js +59 -5
- package/dist/guardrail-check.d.ts +85 -0
- package/dist/guardrail-check.js +152 -0
- package/dist/harness-assert.d.ts +10 -0
- package/dist/harness-assert.js +30 -0
- package/dist/hook-install.d.ts +43 -0
- package/dist/hook-install.js +91 -0
- package/dist/hook.d.ts +52 -0
- package/dist/hook.js +98 -0
- package/dist/leaderboard.d.ts +6 -0
- package/dist/leaderboard.js +43 -1
- package/dist/linting.d.ts +9 -5
- package/dist/linting.js +17 -5
- package/dist/optimize.js +1 -1
- package/dist/scaffold-test.js +21 -7
- package/dist/scan-behavioral.d.ts +60 -0
- package/dist/scan-behavioral.js +239 -1
- package/dist/scan-trigger-suggest.d.ts +54 -0
- package/dist/scan-trigger-suggest.js +70 -0
- package/dist/scan.d.ts +31 -1
- package/dist/scan.js +65 -3
- package/dist/score-explainer.js +1 -1
- package/dist/self-command-refs.d.ts +21 -0
- package/dist/self-command-refs.js +125 -0
- package/dist/setup-plan.d.ts +59 -1
- package/dist/setup-plan.js +103 -5
- package/dist/testing.d.ts +5 -3
- package/dist/testing.js +37 -23
- package/dist/tool-intercept.d.ts +4 -4
- package/dist/tool-intercept.js +5 -5
- package/dist/unit.d.ts +2 -0
- package/dist/unit.js +8 -1
- package/hooks/post-edit.sh +1 -1
- package/hooks/refs-nudge.sh +1 -1
- package/package.json +5 -3
- package/skills/adopt-spec/SKILL.md +7 -7
- package/skills/linter-docs/eslint.md +1 -1
- package/skills/strengthen/SKILL.md +1 -1
package/README.md
CHANGED
|
@@ -1,3 +1,44 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
README DIRECTION — read before editing; keep changes aligned.
|
|
3
|
+
This file is the FRONT DOOR + a marketing asset for someone who already lives
|
|
4
|
+
in Claude Code / Codex. Optimize for a phone-skimmer.
|
|
5
|
+
|
|
6
|
+
1. LEAD WITH BENEFITS, not mechanics or vocabulary. Say what the user GETS
|
|
7
|
+
(a guard that can't silently fail; a CLAUDE.md that stops lying) before how.
|
|
8
|
+
1b. NEVER OPEN WITH A NEGATIVE, APOLOGY, OR CAVEAT. A bolded lead-in is the
|
|
9
|
+
FIRST thing a skimmer reads, so it must be the benefit/on-ramp, never a
|
|
10
|
+
deficiency or competitor: write "Start in plain markdown", NOT "No
|
|
11
|
+
TypeScript?". Put the STRONGEST proof (e.g. 2/7→7/7) on its OWN line, never
|
|
12
|
+
buried mid-paragraph, and END a section on the win, not the caveat (demote
|
|
13
|
+
trade-offs to a trailing aside). Break run-on em-dash/semicolon chains — a
|
|
14
|
+
paragraph is ≤ ~3 lines, one idea.
|
|
15
|
+
1c. LEAD WITH THE CONCRETE PAIN the reader already feels — named in THEIR
|
|
16
|
+
situation, with the SPECIFIC silent failure, not an abstraction. "You
|
|
17
|
+
installed plugins and wrote skills — but do they actually work? A skill that
|
|
18
|
+
never fires, a hook that blocks nothing, a CLAUDE.md full of dead refs" beats
|
|
19
|
+
"reliability for your harness"; "a library with no tests" is the anchoring
|
|
20
|
+
analogy. This is NOT a 1b violation: a pain about the READER's situation is a
|
|
21
|
+
hook, not an apology — 1b bans opening with vigiles's OWN deficiency, a
|
|
22
|
+
caveat, or a competitor, never the user's pain. SAME FOR THE SUBDOCS: open
|
|
23
|
+
every guide with the concrete pain, THEN the "what this doc is" line + the
|
|
24
|
+
README uplink (per docs-quality in CLAUDE.md).
|
|
25
|
+
2. COMPILED SPECS ARE EASY — reassure, never intimidate. You rarely hand-write
|
|
26
|
+
a .spec.ts: model-invocable skills (edit-spec / strengthen / test-harness)
|
|
27
|
+
author + edit it, and hooks auto-compile on save. Present the typed spec as
|
|
28
|
+
the SMOOTH, AUTO-ENFORCED next step you don't think about — the natural
|
|
29
|
+
evolution (JS→TS: gradual, strict opt-in), NEVER a wall. Markdown + inline
|
|
30
|
+
mode work with zero TS for anyone not there yet (progressive adoption).
|
|
31
|
+
3. THE INSTRUMENTS stay first-class — including Eval (measuring whether a skill
|
|
32
|
+
actually helps is core, not optional). NOTE: Guard / compiled hooks is PARKED
|
|
33
|
+
FOR LAUNCH (commented out below; see research/roadmap.md "Launch readiness") —
|
|
34
|
+
so the live set is Lint/Test/Eval ("three instruments"); re-add Guard post-HN.
|
|
35
|
+
4. SCANNABLE + SHORT — ~200-line cap; punchy table cells, bullets, runnable
|
|
36
|
+
blocks; benefits over jargon. Push depth into docs/ and LINK it.
|
|
37
|
+
5. NO INTERNAL VOCABULARY (moat / measurement-authority / flywheel) and NO
|
|
38
|
+
research/ links — name the user benefit (see public-vs-internal-docs +
|
|
39
|
+
readme-brevity in CLAUDE.md).
|
|
40
|
+
-->
|
|
41
|
+
|
|
1
42
|
<p align="center">
|
|
2
43
|
<img src="logo.png" width="140" alt="vigiles logo" />
|
|
3
44
|
</p>
|
|
@@ -5,11 +46,7 @@
|
|
|
5
46
|
<h1 align="center">vigiles</h1>
|
|
6
47
|
|
|
7
48
|
<p align="center">
|
|
8
|
-
<strong>
|
|
9
|
-
</p>
|
|
10
|
-
|
|
11
|
-
<p align="center">
|
|
12
|
-
Your CLAUDE.md, hooks, and skills steer the agent — but nothing checks they're <em>true</em>, nothing tests they <em>work</em>, and nothing measures whether they actually <em>help</em>. vigiles does all three.
|
|
49
|
+
<strong>Make the harness your AI agent runs on reliable.</strong>
|
|
13
50
|
</p>
|
|
14
51
|
|
|
15
52
|
<p align="center">
|
|
@@ -20,57 +57,85 @@
|
|
|
20
57
|
|
|
21
58
|
---
|
|
22
59
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
60
|
+
**You installed a bunch of plugins and wrote a few skills — but do they actually work?**
|
|
61
|
+
A skill that never fires, a safety hook that blocks nothing, a CLAUDE.md full of dead
|
|
62
|
+
references — your harness fails **silently**, and you find out mid-task.
|
|
63
|
+
|
|
64
|
+
**It's a library with no tests.**
|
|
28
65
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
| **📊 Measure** | Does that skill or plugin actually help — or just add cost? **A/B it on real tasks** and read the bill + correctness, not the vendor's headline. The same engine ranks the hyped ecosystem: **what works vs hype**. **[→](docs/measuring-skills.md)** |
|
|
66
|
+
`Agent = Model + Harness` — the model gets the headlines, the harness is the half you
|
|
67
|
+
own. vigiles[^name] makes it prove itself **before the agent ever runs** — verify, test,
|
|
68
|
+
measure. Mostly free, no API key — real-model checks run on your **Claude Pro/Max
|
|
69
|
+
subscription, not metered tokens**.
|
|
34
70
|
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
71
|
+
Three instruments, adopt any:
|
|
72
|
+
|
|
73
|
+
| | |
|
|
74
|
+
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
75
|
+
| **🔎 Lint** | Your CLAUDE.md stops lying — every path, script, symbol & linter rule checked against **reality**. **[→](docs/verifying-instruction-files.md)** |
|
|
76
|
+
| **🧪 Test** | Prove your hooks, skills & subagents do their job — **free, no API key**. **[→](docs/harness-testing.md)** |
|
|
77
|
+
| **📊 Eval** | Know if a skill helps or just costs — **A/B on real tasks**, on your own subscription. **[→](docs/measuring-skills.md)** |
|
|
78
|
+
|
|
79
|
+
<!-- PARKED FOR LAUNCH — Guard / compiled hooks. Re-add this row + the ④ section below post-HN. See research/roadmap.md "Launch readiness".
|
|
80
|
+
| **🛡 Guard** | A safety hook that **can't silently fail open** — write a typed function, get a guard that blocks. **[→](docs/compiled-hooks.md)** |
|
|
81
|
+
-->
|
|
82
|
+
|
|
83
|
+
<p align="center">
|
|
84
|
+
<img src="vigiles-demo.gif" width="720" alt="vigiles lint catching a file that moved and a script that was renamed" />
|
|
85
|
+
</p>
|
|
86
|
+
<!-- Regenerate the GIF: `python3 scripts/make-demo-gif.py` (output is verbatim CLI; see scripts/demo.sh for a live asciinema recording). -->
|
|
87
|
+
|
|
88
|
+
**Two ways in** — pick the pain that's yours:
|
|
89
|
+
|
|
90
|
+
- **Run agents on your own repo?** Start with **Lint** below, or `npx vigiles init`.
|
|
91
|
+
- **Ship plugins to a marketplace?** **[`vigiles scan`](docs/for-plugin-authors.md)**
|
|
92
|
+
ranks structural health with no key — see the **[plugin-author guide →](docs/for-plugin-authors.md)**.
|
|
93
|
+
|
|
94
|
+
**Your agent writes the spec — and you can always eject.** You rarely hand-write a
|
|
95
|
+
`.spec.ts`: skills author it for you, **`init` adopts an existing CLAUDE.md
|
|
96
|
+
non-destructively** (untouched until you compile), and plain markdown + inline
|
|
97
|
+
`<!-- vigiles:enforce -->` comments work with zero TypeScript. **Claude Code and
|
|
98
|
+
Codex** ([`vigiles/codex`](docs/harnesses.md)), or [your own
|
|
99
|
+
harness](docs/authoring-an-adapter.md). _(New here? [the FAQ](docs/faq.md) covers the
|
|
100
|
+
"do I write TypeScript / will it touch my files" questions.)_
|
|
38
101
|
|
|
39
102
|
## Quick start
|
|
40
103
|
|
|
41
104
|
**Paste into Claude Code or Codex:**
|
|
42
105
|
|
|
43
106
|
```text
|
|
44
|
-
Set up vigiles in this repo
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
107
|
+
Set up vigiles in this repo: run `npx vigiles init` and accept the defaults. If I
|
|
108
|
+
already have a CLAUDE.md or AGENTS.md, adopt it into a spec and show me which
|
|
109
|
+
references are stale. Then install the dep, compile, and write + run one harness
|
|
110
|
+
test for a hook or skill of mine. Don't enforce a spec-per-file or add a real-model
|
|
111
|
+
eval without asking me first.
|
|
48
112
|
```
|
|
49
113
|
|
|
114
|
+
The same prompt works in Codex.
|
|
115
|
+
|
|
50
116
|
Or do it yourself:
|
|
51
117
|
|
|
52
118
|
```bash
|
|
53
119
|
npx vigiles init # sets up lint + test: spec + harness test + CI + plugin
|
|
54
120
|
```
|
|
55
121
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
122
|
+
Interactive in a terminal, non-interactive for agents/CI (or `--yes`).
|
|
123
|
+
|
|
124
|
+
**You don't hand-write any of this — your agent does.** `init` installs
|
|
125
|
+
model-invocable skills, so a plain-English ask does the work (it edits the source
|
|
126
|
+
and recompiles on save; you never touch it by hand):
|
|
127
|
+
|
|
128
|
+
- _"test my skills"_ → scaffolds **and runs** a trigger/behaviour test (`test-harness`)
|
|
129
|
+
- _"harden my rules"_ → upgrades prose guidance into enforced linter rules (`strengthen`)
|
|
130
|
+
- _"add a rule to my CLAUDE.md"_ → edits the source and recompiles (`edit-spec`)
|
|
60
131
|
|
|
61
132
|
<details>
|
|
62
133
|
<summary>What <code>init</code> sets up</summary>
|
|
63
134
|
|
|
64
|
-
- **Both lint and test** by default; scope with `--lint` / `--test
|
|
65
|
-
-
|
|
66
|
-
-
|
|
67
|
-
|
|
68
|
-
- Wires CI as a `zernie/vigiles@v1` workflow (a composite over the same CLI):
|
|
69
|
-
|
|
70
|
-
```yaml
|
|
71
|
-
- uses: actions/checkout@v4
|
|
72
|
-
- uses: zernie/vigiles@v1 # lints by default; posts a sticky PR comment + a `valid` output
|
|
73
|
-
```
|
|
135
|
+
- **Both lint and test** by default; scope with `--lint` / `--test`.
|
|
136
|
+
- **Already have a CLAUDE.md / AGENTS.md? `init` adopts it** into a spec faithfully and **non-destructively** — your file is left untouched until you choose to `compile` (and `eject` undoes it).
|
|
137
|
+
- Adds `vigiles` to `devDependencies`; installs the Claude Code plugin (skills + hooks) via the marketplace — globally, never vendored.
|
|
138
|
+
- Wires CI as a `zernie/vigiles@v1` workflow (a composite over the same CLI) that posts a sticky PR comment + a `valid` output.
|
|
74
139
|
|
|
75
140
|
Prefer to write tests yourself? They can be JS **or** TS
|
|
76
141
|
(`*.harness.{mjs,ts}`) — run them with `npx vigiles test`.
|
|
@@ -79,11 +144,10 @@ Prefer to write tests yourself? They can be JS **or** TS
|
|
|
79
144
|
|
|
80
145
|
## ① Lint — your CLAUDE.md lies to your agent
|
|
81
146
|
|
|
82
|
-
Your CLAUDE.md
|
|
83
|
-
`npm run check
|
|
84
|
-
the script was renamed. The agent trusts the stale claim and acts on
|
|
85
|
-
|
|
86
|
-
`npx vigiles lint` resolves every reference against reality:
|
|
147
|
+
**Your CLAUDE.md drifts the moment you refactor.** It points the agent at
|
|
148
|
+
`src/auth/login.ts` and says run `npm run check` — but the file moved six commits
|
|
149
|
+
ago and the script was renamed. The agent trusts the stale claim and acts on
|
|
150
|
+
fiction. `npx vigiles lint` resolves every reference against reality:
|
|
87
151
|
|
|
88
152
|
```text
|
|
89
153
|
CLAUDE.md:
|
|
@@ -92,31 +156,25 @@ CLAUDE.md:
|
|
|
92
156
|
✓ @typescript-eslint/no-floating-promises — exists and enabled in eslint config
|
|
93
157
|
```
|
|
94
158
|
|
|
95
|
-
File paths, scripts,
|
|
96
|
-
(
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
>
|
|
104
|
-
>
|
|
105
|
-
>
|
|
106
|
-
>
|
|
107
|
-
> opt-in. **[The moat →](research/typed-spec-moat.md)**
|
|
108
|
-
|
|
109
|
-
**Same cross-reference, any plugin.** `npx vigiles scan` checks a plugin's
|
|
110
|
-
_contracts_ — every subagent tool, `mcp__server__tool`, `mcp_tool` hook, hook
|
|
111
|
-
event, and script path actually **exists and resolves**, not just parses (valid
|
|
112
|
-
YAML ≠ a tool that's real). A **superset of Anthropic's `claude plugin validate`**,
|
|
113
|
-
no key. **[Audit any plugin →](docs/cli.md#scan-dir)**
|
|
159
|
+
File paths, scripts, code symbols — plus linter rules across **7 linters**
|
|
160
|
+
(ESLint, Ruff, Clippy + four more): each rule exists **and is enabled**.
|
|
161
|
+
|
|
162
|
+
**Start with one inline comment — no new files.** Step up to a typed `.spec.ts`
|
|
163
|
+
(your agent writes it; it compiles to CLAUDE.md) when you want compiler-grade
|
|
164
|
+
guarantees — and `vigiles eject` hands it back to plain markdown anytime.
|
|
165
|
+
**[Full guide →](docs/verifying-instruction-files.md)**
|
|
166
|
+
|
|
167
|
+
> **Want more? Bad states can stop compiling.** Opt in deeper and a broken
|
|
168
|
+
> hand-off between agents becomes a build error instead of a runtime surprise —
|
|
169
|
+
> graduated like TypeScript's `strict`, on only when you want it.
|
|
170
|
+
> **[How →](docs/compiled-hooks.md)**
|
|
114
171
|
|
|
115
172
|
## ② Test — does your harness do its job?
|
|
116
173
|
|
|
117
|
-
|
|
118
|
-
unrelated prompts
|
|
119
|
-
naive "did it run?" check. vigiles tests the
|
|
174
|
+
**You wired the hook — but does it actually block?** A skill's description can fail
|
|
175
|
+
to trigger, or hijack unrelated prompts; injected context can silently never reach
|
|
176
|
+
the model. All of it passes a naive "did it run?" check. vigiles tests the
|
|
177
|
+
assembled harness for real:
|
|
120
178
|
|
|
121
179
|
```typescript
|
|
122
180
|
import { runHook } from "vigiles/testing";
|
|
@@ -133,18 +191,18 @@ It goes well past _"did it fire?"_:
|
|
|
133
191
|
|
|
134
192
|
- **Hooks block** what they must — `runHook`, or the real agent CLI via `runHarnessTest`.
|
|
135
193
|
- **Skills trigger** on the right prompts and stay quiet on the wrong ones — recall _and_ precision (`measureTriggerRate`).
|
|
136
|
-
- **Behaviour is good** — score a skill's output
|
|
194
|
+
- **Behaviour is good** — score a skill's output, or A/B it on-vs-off for the real lift (`measure` / `runEval`, with significance testing).
|
|
137
195
|
- **Safety holds** — the agent _didn't_ push to the wrong branch or hit a paid API; `interceptTools` catches the attempt so the side effect never happens.
|
|
138
196
|
|
|
139
197
|
Almost every tier runs with **no model and no API key** — milliseconds, on every
|
|
140
|
-
commit
|
|
141
|
-
|
|
198
|
+
commit; only the real-model evals need a model, on your own `claude` CLI.
|
|
199
|
+
**[How it works →](docs/harness-testing.md)**
|
|
142
200
|
|
|
143
|
-
## ③
|
|
201
|
+
## ③ Eval — does it actually help, or just cost more?
|
|
144
202
|
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
harness loaded exactly as it ships, and reports the **metric triple**:
|
|
203
|
+
**"65% fewer tokens." "3× faster." Says who?** A skill claims it, a plugin promises
|
|
204
|
+
it — stars and vibes, **zero measurement**. vigiles A/Bs the claim on real coding
|
|
205
|
+
tasks, the harness loaded exactly as it ships, and reports the **metric triple**:
|
|
148
206
|
|
|
149
207
|
```typescript
|
|
150
208
|
import { measureArms } from "vigiles/testing";
|
|
@@ -155,33 +213,76 @@ const r = await measureArms({
|
|
|
155
213
|
arms: { baseline: {}, skill: { files: { "SKILL.md": THE_SKILL } } },
|
|
156
214
|
measure: (ctx) => ({ cost: ctx.usage.costUsd, correct: check(ctx) }),
|
|
157
215
|
});
|
|
158
|
-
// → is the bill lower? did correctness survive? (not just "fewer output tokens")
|
|
159
216
|
```
|
|
160
217
|
|
|
161
|
-
- **The bill (`costUsd`)** —
|
|
162
|
-
- **The
|
|
218
|
+
- **The bill (`costUsd`)** — weights cache ~0.1× / output 1×, so a "saved tokens" headline can't hide behind cheap cache.
|
|
219
|
+
- **The target** — whatever the skill claims to move (output tokens, latency, tool calls), verified on its own terms.
|
|
163
220
|
- **The blast radius** — correctness, a deterministic 1/0. A token win that breaks the code is **not a win**.
|
|
164
221
|
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
222
|
+
**Safe to repeat.** Each real-model run is sandboxed (ephemeral dir, egress blocked
|
|
223
|
+
or allow-listed), and `interceptTools` catches an irreversible external — a push, a
|
|
224
|
+
paid API — as an _attempt_, never running it. **[Safety, sandboxing & FAQ →](docs/safety.md)**
|
|
225
|
+
|
|
226
|
+
**The eval you can actually afford.** promptfoo / DeepEval hit a metered API and
|
|
227
|
+
bill **per token, every run**. vigiles answers most questions with **no model at
|
|
228
|
+
all**, and runs the rest on your own **Claude Pro/Max subscription — $0 extra**. So
|
|
229
|
+
you can measure on every change. **[Eval a skill →](docs/measuring-skills.md)** · **[Why it's affordable →](docs/eval-architecture.md)**
|
|
230
|
+
|
|
231
|
+
<!-- PARKED FOR LAUNCH — Guard / compiled hooks. Re-add this whole section (and the table row above) post-HN. See research/roadmap.md "Launch readiness".
|
|
232
|
+
|
|
233
|
+
## ④ Guard — a safety hook that can't silently fail open
|
|
234
|
+
|
|
235
|
+
**Your safety hook looks like it blocks — and doesn't.** A guard is your last stop
|
|
236
|
+
before something irreversible, but a hand-written one **fails open** without telling
|
|
237
|
+
you. _(Already write safety hooks? This is the power tool.)_ Write a pure typed
|
|
238
|
+
function instead; vigiles emits the exit code, the JSON, and an AST-backed matcher:
|
|
239
|
+
|
|
240
|
+
```typescript
|
|
241
|
+
import { defineHook, tool, deny, allow } from "vigiles/hook";
|
|
242
|
+
|
|
243
|
+
export default defineHook({
|
|
244
|
+
on: "PreToolUse",
|
|
245
|
+
match: tool("Bash"),
|
|
246
|
+
decide: (e) =>
|
|
247
|
+
e.command.runs("git push", { force: true })
|
|
248
|
+
? deny("no force-push to a protected branch")
|
|
249
|
+
: allow(),
|
|
250
|
+
});
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
**The proof:** a widely-copied OSS safety hook blocks **2/7** of the disaster
|
|
254
|
+
battery. The compiled rewrite blocks **7/7** — measured, not asserted.
|
|
255
|
+
|
|
256
|
+
You never hand-write the exit code or JSON field (the usual false confidence), the
|
|
257
|
+
matcher is **AST-backed** (it catches the `cd x && git push -f` a glob misses), and
|
|
258
|
+
the artifact is **stamped** so a later hand-edit is refused.
|
|
259
|
+
|
|
260
|
+
_Scope: this fixes a hook's logic, not the harness's delivery — a subagent's tool
|
|
261
|
+
calls still bypass any PreToolUse hook
|
|
262
|
+
([#34692](https://github.com/anthropics/claude-code/issues/34692)), so it's a strong
|
|
263
|
+
default, not an unbypassable wall._
|
|
264
|
+
**[Compiled hooks — bug classes + trade-offs →](docs/compiled-hooks.md)**
|
|
265
|
+
|
|
266
|
+
-->
|
|
267
|
+
|
|
268
|
+
## FAQ
|
|
170
269
|
|
|
171
|
-
**
|
|
172
|
-
|
|
270
|
+
- **Isn't this just a markdown linter?** No — it checks whether your instruction file is _true_ (every path/script/symbol/rule exists and is enabled), then tests and measures your harness. A style linter can't do any of that.
|
|
271
|
+
- **Do I have to write TypeScript?** No. Lint your markdown with zero new files; when you want a spec, your agent writes it. It's gradual, like TS's `strict`.
|
|
272
|
+
- **Does it overwrite my files?** No. `init` adopts an existing CLAUDE.md _non-destructively_ — untouched until you `compile`, and `eject` reverses it.
|
|
273
|
+
- **Need an API key?** No for almost everything (free, every commit). Real-model evals run on your Claude Pro/Max subscription — $0 metered tokens.
|
|
274
|
+
- **Non-JS repo?** `npx vigiles lint` verifies your CLAUDE.md with no install (Ruff/Clippy/Pylint/… too).
|
|
173
275
|
|
|
174
|
-
|
|
175
|
-
| ---------------------- | ----------------------- | ------------------------------------------- |
|
|
176
|
-
| promptfoo, DeepEval, … | metered API SDK | billed **per token, every run** |
|
|
177
|
-
| **vigiles** | your Claude Pro/Max sub | **$0 extra** — and most tiers need no model |
|
|
276
|
+
**[Full FAQ →](docs/faq.md)**
|
|
178
277
|
|
|
179
278
|
## More
|
|
180
279
|
|
|
181
|
-
- **[
|
|
182
|
-
- **[
|
|
183
|
-
- **[
|
|
184
|
-
- **[Docs index →](docs/README.md)** · **[
|
|
280
|
+
- **[CLI →](docs/cli.md)** — every command and the plugin · **[GitHub Action →](docs/github-action.md)** — run it in CI. The full **[lint rules matrix →](docs/verifying-instruction-files.md#the-validation-rules--the-full-matrix)** lives with the linting guide.
|
|
281
|
+
- **[Skills →](docs/skills.md)** — the skills `init` installs, and how the model-invocable ones trigger.
|
|
282
|
+
- **[Ship plugins? The plugin-author guide →](docs/for-plugin-authors.md)** — scan a draft for structural health, make your skills fire for users, then rank a whole marketplace (0–100, A–F, worst issues first) — **no key**.
|
|
283
|
+
- **[Docs index →](docs/README.md)** · **[API reference →](https://zernie.github.io/vigiles/)** · **[Related tools →](docs/related-tools.md)** (ast-grep, Dependency Cruiser, Ruler, rulesync).
|
|
284
|
+
- **[Stability →](STABILITY.md)** — 0.x: the CLI is stable; the library API is still evolving; experimental surfaces are marked.
|
|
285
|
+
- **Not for you if** you want a model/capability benchmark or runtime guardrails in the request path — vigiles is build-/CI-time.
|
|
185
286
|
- Companion to [Feedback Loop Is All You Need](https://zernie.com/blog/feedback-loop-is-all-you-need).
|
|
186
287
|
|
|
187
288
|
## License
|
package/dist/action-gate.js
CHANGED
|
@@ -13,7 +13,7 @@ exports.loadActionGates = loadActionGates;
|
|
|
13
13
|
*
|
|
14
14
|
* It is the same deterministic gate primitive (reuses `runGate` + the
|
|
15
15
|
* author-time reference resolution), re-anchored from step → action. Delivered
|
|
16
|
-
* as a PostToolUse hook (`vigiles action
|
|
16
|
+
* as a PostToolUse hook (`vigiles hook-runtime action`): exit 2 blocks the action and
|
|
17
17
|
* feeds the reason back, exit 0 allows it.
|
|
18
18
|
*
|
|
19
19
|
* Config: `.vigiles/action-gates.json` → `{ "gates": [ { on, gate, when? } ] }`.
|
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
* doesn't list.
|
|
11
11
|
*
|
|
12
12
|
* This is the same emit-a-hook pattern the skill runtime already ships
|
|
13
|
-
* (`src/skill-runtime.ts`): there a `Stop` hook reads the active skill's
|
|
13
|
+
* (`src/adapters/claude-code/skill-runtime.ts`): there a `Stop` hook reads the active skill's
|
|
14
14
|
* compiled SKILL.md and runs its result gate; here a `PreToolUse` hook reads
|
|
15
15
|
* the active agent's compiled `.md`, parses its `tools:` allowlist, and
|
|
16
16
|
* allows/denies the tool call. The compiled markdown's frontmatter is the
|
|
@@ -66,21 +66,56 @@ export interface PreToolDecision {
|
|
|
66
66
|
* feeding the contract back to the model so it self-corrects.
|
|
67
67
|
*/
|
|
68
68
|
export declare function decidePreToolUse(allowed: readonly string[] | null, tool: string): PreToolDecision;
|
|
69
|
-
/**
|
|
69
|
+
/**
|
|
70
|
+
* Read the active-agent stack (oldest → newest; the dispatched subagent chain).
|
|
71
|
+
* Back-compat: a legacy single-slot `{ agent: string }` marker reads as a one-frame
|
|
72
|
+
* stack; a malformed file or non-string entries → an empty stack (fail-open).
|
|
73
|
+
*/
|
|
74
|
+
export declare function readActiveStack(cwd: string): string[];
|
|
75
|
+
/**
|
|
76
|
+
* Push a dispatched subagent onto the active stack (a `PreToolUse` spawn). The
|
|
77
|
+
* PreToolUse gate then enforces this subagent's contract until it returns. Under
|
|
78
|
+
* nesting each dispatch pushes a frame, so the chain is tracked, not overwritten.
|
|
79
|
+
*/
|
|
80
|
+
export declare function pushActiveAgent(cwd: string, agentPath: string): void;
|
|
81
|
+
/**
|
|
82
|
+
* Pop the top frame — the subagent returned (`SubagentStop`), so control returns
|
|
83
|
+
* to its PARENT (the next frame down), whose contract the gate enforces again.
|
|
84
|
+
* This is the nesting-safe close, distinct from {@link clearActiveAgent} (which
|
|
85
|
+
* drops the whole stack). Popping an empty stack is a no-op.
|
|
86
|
+
*/
|
|
87
|
+
export declare function popActiveAgent(cwd: string): void;
|
|
88
|
+
/**
|
|
89
|
+
* Push a subagent frame (the manual `agent-start` fallback + the deterministic
|
|
90
|
+
* spawn open-signal both use this). A single call is equivalent to a one-frame
|
|
91
|
+
* stack, so the top — what the gate reads — is this agent. Alias of
|
|
92
|
+
* {@link pushActiveAgent} kept under the historical name.
|
|
93
|
+
*/
|
|
70
94
|
export declare function setActiveAgent(cwd: string, agentPath: string): void;
|
|
71
|
-
/**
|
|
95
|
+
/**
|
|
96
|
+
* Clear the WHOLE stack (a hard reset / session end). Distinct from
|
|
97
|
+
* {@link popActiveAgent}, which returns to the parent frame. Idempotent.
|
|
98
|
+
*/
|
|
72
99
|
export declare function clearActiveAgent(cwd: string): void;
|
|
73
|
-
/**
|
|
100
|
+
/**
|
|
101
|
+
* The active agent's compiled `.md` — the STACK TOP — or null when none is active.
|
|
102
|
+
* The gate reads the top, so a returned nested subagent reveals its parent's
|
|
103
|
+
* contract again (the contract-escape fix).
|
|
104
|
+
*/
|
|
74
105
|
export declare function readActiveAgent(cwd: string): string | null;
|
|
75
106
|
/**
|
|
76
|
-
* EXPERIMENTAL — parked (P3),
|
|
77
|
-
*
|
|
78
|
-
*
|
|
79
|
-
*
|
|
80
|
-
*
|
|
81
|
-
*
|
|
107
|
+
* EXPERIMENTAL — parked (P3), do NOT auto-wire. The subagent-window tracking is
|
|
108
|
+
* now nesting-safe: a depth-aware STACK (push on dispatch, pop on SubagentStop —
|
|
109
|
+
* see {@link pushActiveAgent}/{@link popActiveAgent}) closes the contract-escape
|
|
110
|
+
* the flat single-slot model allowed under Claude Code v2.1.172 depth-5 nesting
|
|
111
|
+
* (certified in research/prototypes/.../AgentWindowStack.tla). The open signal
|
|
112
|
+
* recognizes BOTH spawn tool names (`Task` and the nested-spawn `Agent`), gated on
|
|
113
|
+
* a resolvable `subagent_type` so a non-spawn call never opens a frame. Still
|
|
114
|
+
* parked because the `effect()` sub-region goal it served was dropped (see
|
|
115
|
+
* research/effect-boundary-design.md, "Why dropped") — the stack is shipped for
|
|
116
|
+
* when active-agent contract enforcement under nesting is wanted on its own.
|
|
82
117
|
*
|
|
83
|
-
* Resolve a
|
|
118
|
+
* Resolve a spawn tool's `subagent_type` to the compiled agent `.md` to
|
|
84
119
|
* activate, or null when none is found. The DETERMINISTIC open signal that
|
|
85
120
|
* replaces the model-invoked `agent-start`: Claude Code fires `PreToolUse` for
|
|
86
121
|
* the parent's `Task` dispatch (and `SubagentStop` when it returns), so the
|
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
* doesn't list.
|
|
12
12
|
*
|
|
13
13
|
* This is the same emit-a-hook pattern the skill runtime already ships
|
|
14
|
-
* (`src/skill-runtime.ts`): there a `Stop` hook reads the active skill's
|
|
14
|
+
* (`src/adapters/claude-code/skill-runtime.ts`): there a `Stop` hook reads the active skill's
|
|
15
15
|
* compiled SKILL.md and runs its result gate; here a `PreToolUse` hook reads
|
|
16
16
|
* the active agent's compiled `.md`, parses its `tools:` allowlist, and
|
|
17
17
|
* allows/denies the tool call. The compiled markdown's frontmatter is the
|
|
@@ -28,6 +28,9 @@ exports.parseAgentTools = parseAgentTools;
|
|
|
28
28
|
exports.parseAgentToolList = parseAgentToolList;
|
|
29
29
|
exports.parseAgentPurity = parseAgentPurity;
|
|
30
30
|
exports.decidePreToolUse = decidePreToolUse;
|
|
31
|
+
exports.readActiveStack = readActiveStack;
|
|
32
|
+
exports.pushActiveAgent = pushActiveAgent;
|
|
33
|
+
exports.popActiveAgent = popActiveAgent;
|
|
31
34
|
exports.setActiveAgent = setActiveAgent;
|
|
32
35
|
exports.clearActiveAgent = clearActiveAgent;
|
|
33
36
|
exports.readActiveAgent = readActiveAgent;
|
|
@@ -100,46 +103,114 @@ function decidePreToolUse(allowed, tool) {
|
|
|
100
103
|
};
|
|
101
104
|
}
|
|
102
105
|
// ---------------------------------------------------------------------------
|
|
103
|
-
// Active-agent tracking (mirrors .vigiles/active-skill.json)
|
|
106
|
+
// Active-agent tracking — a depth-aware STACK (mirrors .vigiles/active-skill.json)
|
|
104
107
|
// ---------------------------------------------------------------------------
|
|
108
|
+
//
|
|
109
|
+
// Claude Code v2.1.172 added nested subagents (a subagent with the spawn tool can
|
|
110
|
+
// dispatch its own, up to depth 5). A single active-agent slot is NOT nesting-safe:
|
|
111
|
+
// when an inner subagent returns, clearing the whole slot drops the OUTER agent's
|
|
112
|
+
// contract while it is still running, so the PreToolUse gate then allows a tool the
|
|
113
|
+
// outer subagent forbids — a CONTRACT ESCAPE. The fix (certified in TLC, see
|
|
114
|
+
// research/prototypes/typed-spec-formal-verification/AgentWindowStack.tla) is a
|
|
115
|
+
// STACK: push on dispatch, pop on SubagentStop (back to the parent), gate on the
|
|
116
|
+
// stack TOP. Counterexample the flat model fails and the stack model passes:
|
|
117
|
+
// Open(writer); Open(writer); Stop; Call(Bash).
|
|
105
118
|
const ACTIVE_PATH = ".vigiles/active-agent.json";
|
|
106
|
-
/**
|
|
107
|
-
|
|
119
|
+
/**
|
|
120
|
+
* Read the active-agent stack (oldest → newest; the dispatched subagent chain).
|
|
121
|
+
* Back-compat: a legacy single-slot `{ agent: string }` marker reads as a one-frame
|
|
122
|
+
* stack; a malformed file or non-string entries → an empty stack (fail-open).
|
|
123
|
+
*/
|
|
124
|
+
function readActiveStack(cwd) {
|
|
125
|
+
const p = (0, node_path_1.resolve)(cwd, ACTIVE_PATH);
|
|
126
|
+
if (!(0, node_fs_1.existsSync)(p))
|
|
127
|
+
return [];
|
|
128
|
+
try {
|
|
129
|
+
const parsed = JSON.parse((0, node_fs_1.readFileSync)(p, "utf-8"));
|
|
130
|
+
if (Array.isArray(parsed.stack)) {
|
|
131
|
+
return parsed.stack.filter((x) => typeof x === "string");
|
|
132
|
+
}
|
|
133
|
+
// legacy single-slot format
|
|
134
|
+
if (typeof parsed.agent === "string")
|
|
135
|
+
return [parsed.agent];
|
|
136
|
+
return [];
|
|
137
|
+
}
|
|
138
|
+
catch {
|
|
139
|
+
return [];
|
|
140
|
+
}
|
|
141
|
+
}
|
|
142
|
+
function writeActiveStack(cwd, stack) {
|
|
108
143
|
const p = (0, node_path_1.resolve)(cwd, ACTIVE_PATH);
|
|
144
|
+
if (stack.length === 0) {
|
|
145
|
+
if ((0, node_fs_1.existsSync)(p))
|
|
146
|
+
(0, node_fs_1.rmSync)(p);
|
|
147
|
+
return;
|
|
148
|
+
}
|
|
109
149
|
(0, node_fs_1.mkdirSync)((0, node_path_1.dirname)(p), { recursive: true });
|
|
110
|
-
(0, node_fs_1.writeFileSync)(p, JSON.stringify({
|
|
150
|
+
(0, node_fs_1.writeFileSync)(p, JSON.stringify({ stack }) + "\n");
|
|
151
|
+
}
|
|
152
|
+
/**
|
|
153
|
+
* Push a dispatched subagent onto the active stack (a `PreToolUse` spawn). The
|
|
154
|
+
* PreToolUse gate then enforces this subagent's contract until it returns. Under
|
|
155
|
+
* nesting each dispatch pushes a frame, so the chain is tracked, not overwritten.
|
|
156
|
+
*/
|
|
157
|
+
function pushActiveAgent(cwd, agentPath) {
|
|
158
|
+
writeActiveStack(cwd, [...readActiveStack(cwd), agentPath]);
|
|
159
|
+
}
|
|
160
|
+
/**
|
|
161
|
+
* Pop the top frame — the subagent returned (`SubagentStop`), so control returns
|
|
162
|
+
* to its PARENT (the next frame down), whose contract the gate enforces again.
|
|
163
|
+
* This is the nesting-safe close, distinct from {@link clearActiveAgent} (which
|
|
164
|
+
* drops the whole stack). Popping an empty stack is a no-op.
|
|
165
|
+
*/
|
|
166
|
+
function popActiveAgent(cwd) {
|
|
167
|
+
const stack = readActiveStack(cwd);
|
|
168
|
+
stack.pop();
|
|
169
|
+
writeActiveStack(cwd, stack);
|
|
111
170
|
}
|
|
112
|
-
/**
|
|
171
|
+
/**
|
|
172
|
+
* Push a subagent frame (the manual `agent-start` fallback + the deterministic
|
|
173
|
+
* spawn open-signal both use this). A single call is equivalent to a one-frame
|
|
174
|
+
* stack, so the top — what the gate reads — is this agent. Alias of
|
|
175
|
+
* {@link pushActiveAgent} kept under the historical name.
|
|
176
|
+
*/
|
|
177
|
+
function setActiveAgent(cwd, agentPath) {
|
|
178
|
+
pushActiveAgent(cwd, agentPath);
|
|
179
|
+
}
|
|
180
|
+
/**
|
|
181
|
+
* Clear the WHOLE stack (a hard reset / session end). Distinct from
|
|
182
|
+
* {@link popActiveAgent}, which returns to the parent frame. Idempotent.
|
|
183
|
+
*/
|
|
113
184
|
function clearActiveAgent(cwd) {
|
|
114
185
|
const p = (0, node_path_1.resolve)(cwd, ACTIVE_PATH);
|
|
115
186
|
if ((0, node_fs_1.existsSync)(p))
|
|
116
187
|
(0, node_fs_1.rmSync)(p);
|
|
117
188
|
}
|
|
118
|
-
/**
|
|
189
|
+
/**
|
|
190
|
+
* The active agent's compiled `.md` — the STACK TOP — or null when none is active.
|
|
191
|
+
* The gate reads the top, so a returned nested subagent reveals its parent's
|
|
192
|
+
* contract again (the contract-escape fix).
|
|
193
|
+
*/
|
|
119
194
|
function readActiveAgent(cwd) {
|
|
120
|
-
const
|
|
121
|
-
|
|
122
|
-
return null;
|
|
123
|
-
try {
|
|
124
|
-
const parsed = JSON.parse((0, node_fs_1.readFileSync)(p, "utf-8"));
|
|
125
|
-
return typeof parsed.agent === "string" ? parsed.agent : null;
|
|
126
|
-
}
|
|
127
|
-
catch {
|
|
128
|
-
return null;
|
|
129
|
-
}
|
|
195
|
+
const stack = readActiveStack(cwd);
|
|
196
|
+
return stack.length > 0 ? stack[stack.length - 1] : null;
|
|
130
197
|
}
|
|
131
198
|
// ---------------------------------------------------------------------------
|
|
132
199
|
// PreToolUse-hook decision
|
|
133
200
|
// ---------------------------------------------------------------------------
|
|
134
201
|
/**
|
|
135
|
-
* EXPERIMENTAL — parked (P3),
|
|
136
|
-
*
|
|
137
|
-
*
|
|
138
|
-
*
|
|
139
|
-
*
|
|
140
|
-
*
|
|
202
|
+
* EXPERIMENTAL — parked (P3), do NOT auto-wire. The subagent-window tracking is
|
|
203
|
+
* now nesting-safe: a depth-aware STACK (push on dispatch, pop on SubagentStop —
|
|
204
|
+
* see {@link pushActiveAgent}/{@link popActiveAgent}) closes the contract-escape
|
|
205
|
+
* the flat single-slot model allowed under Claude Code v2.1.172 depth-5 nesting
|
|
206
|
+
* (certified in research/prototypes/.../AgentWindowStack.tla). The open signal
|
|
207
|
+
* recognizes BOTH spawn tool names (`Task` and the nested-spawn `Agent`), gated on
|
|
208
|
+
* a resolvable `subagent_type` so a non-spawn call never opens a frame. Still
|
|
209
|
+
* parked because the `effect()` sub-region goal it served was dropped (see
|
|
210
|
+
* research/effect-boundary-design.md, "Why dropped") — the stack is shipped for
|
|
211
|
+
* when active-agent contract enforcement under nesting is wanted on its own.
|
|
141
212
|
*
|
|
142
|
-
* Resolve a
|
|
213
|
+
* Resolve a spawn tool's `subagent_type` to the compiled agent `.md` to
|
|
143
214
|
* activate, or null when none is found. The DETERMINISTIC open signal that
|
|
144
215
|
* replaces the model-invoked `agent-start`: Claude Code fires `PreToolUse` for
|
|
145
216
|
* the parent's `Task` dispatch (and `SubagentStop` when it returns), so the
|
|
@@ -8,7 +8,7 @@ exports.hasEffectBoundary = hasEffectBoundary;
|
|
|
8
8
|
* Effect-boundary position-aware state tracking.
|
|
9
9
|
* Mirrors active-unit tracking (.vigiles/active-agent.json / .vigiles/active-skill.json).
|
|
10
10
|
* The PreToolUse hook reads this file to decide whether the agent is inside an
|
|
11
|
-
* effect boundary (set by `vigiles effect-enter`) or outside it.
|
|
11
|
+
* effect boundary (set by `vigiles hook-runtime effect-enter`) or outside it.
|
|
12
12
|
*/
|
|
13
13
|
const node_fs_1 = require("node:fs");
|
|
14
14
|
const node_path_1 = require("node:path");
|