vigiles 9.1.0 → 11.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +126 -112
- package/dist/adapters/claude-code/dialect.js +15 -0
- package/dist/audit-html.d.ts +15 -4
- package/dist/audit-html.js +15 -6
- package/dist/audit-report.d.ts +58 -2
- package/dist/audit-report.js +29 -0
- package/dist/audit-report.template.html +34 -24
- package/dist/audit-score.d.ts +19 -12
- package/dist/audit-score.js +79 -15
- package/dist/audit-serve.d.ts +109 -0
- package/dist/audit-serve.js +257 -0
- package/dist/cli.js +435 -20
- package/dist/core/CLAUDE.md.spec.d.ts +3 -0
- package/dist/core/CLAUDE.md.spec.js +26 -0
- package/dist/core/compile.d.ts +5 -1
- package/dist/core/compile.js +19 -10
- package/dist/core/delegation-trifecta.d.ts +64 -0
- package/dist/core/delegation-trifecta.js +124 -0
- package/dist/core/dialect.d.ts +18 -0
- package/dist/core/hook-block-ineffective.d.ts +62 -0
- package/dist/core/hook-block-ineffective.js +153 -0
- package/dist/core/hook-matcher.d.ts +66 -0
- package/dist/core/hook-matcher.js +182 -0
- package/dist/core/hook-normalize.d.ts +43 -0
- package/dist/core/hook-normalize.js +78 -0
- package/dist/core/lethal-trifecta.d.ts +100 -0
- package/dist/core/lethal-trifecta.js +197 -0
- package/dist/core/plugin-dir-layout.d.ts +30 -0
- package/dist/core/plugin-dir-layout.js +73 -0
- package/dist/core/rule-meta.d.ts +82 -0
- package/dist/core/rule-meta.js +266 -0
- package/dist/core/skill-missing-fence.d.ts +47 -0
- package/dist/core/skill-missing-fence.js +119 -0
- package/dist/core/skill-resources.d.ts +27 -0
- package/dist/core/skill-resources.js +167 -0
- package/dist/core/types.d.ts +71 -0
- package/dist/core/validate.d.ts +1 -0
- package/dist/core/validate.js +26 -4
- package/dist/leaderboard.d.ts +1 -0
- package/dist/leaderboard.js +64 -15
- package/dist/scan-behavioral.d.ts +85 -0
- package/dist/scan-behavioral.js +225 -0
- package/dist/scan.d.ts +106 -0
- package/dist/scan.js +269 -53
- package/dist/setup-plan.d.ts +6 -3
- package/dist/setup-plan.js +12 -2
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -6,18 +6,31 @@
|
|
|
6
6
|
SPINE = CONCEPT 5 (proof/demo-led). Lead with REAL, screenshotable catches on
|
|
7
7
|
plugins people actually ship, THEN explain the mechanism. The proofs are not
|
|
8
8
|
illustrative — every block traces to a real dogfood run captured in
|
|
9
|
-
research/dogfood/.
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
NEVER replace a real catch with a fabricated one.
|
|
9
|
+
research/dogfood/. THREE COMMUNITY catches, anonymized (2026-06-29: Proof 1 is now a
|
|
10
|
+
lethal-trifecta exfil path → Safety 80, from madappgang's `tester` shown as
|
|
11
|
+
"my-plugin" — added to pay off the new Safety-ring hero; Proof 2 a skill-description
|
|
12
|
+
collision → wrong-skill-fires (claude-flow, Triggering F); Proof 3 an
|
|
13
|
+
AskUserQuestion-never-available tool) — all real GRADED/structural defects that
|
|
14
|
+
REPRODUCE on current main. NEVER replace a real catch with a fabricated one. (The
|
|
15
|
+
earlier Proof 1 was a missing-SKILL.md/Truthfulness catch, swapped 2026-06-28: its
|
|
16
|
+
source (superpowers) is clean on current main and NO reproducible dead-file-ref
|
|
17
|
+
exists in popular OSS — those are an adopt+strengthen payoff, see
|
|
18
|
+
research/oss-audit-render-findings.md.)
|
|
19
|
+
|
|
20
|
+
WHY ONLY TWO (decided 2026-06-28): the earlier Proofs 3-4 leaned on
|
|
21
|
+
pr-review-toolkit's "review agents inherit all tools" as an official-plugin
|
|
22
|
+
defect. But inherit-all (a subagent with no `tools:` line) is now ADVISORY, not a
|
|
23
|
+
graded penalty — omitting the tool contract is a near-universal, legitimate
|
|
24
|
+
authoring style (an OSS sweep of 122 plugins found 109 whose only finding was
|
|
25
|
+
this), so penalizing it cried wolf. With that change the official plugins are all
|
|
26
|
+
a clean A, so a "even Anthropic has bugs" proof would be dishonest — Proofs 3-4
|
|
27
|
+
were DROPPED rather than reframed. The leaderboard feature still exists; it just
|
|
28
|
+
isn't a headline proof.
|
|
15
29
|
|
|
16
30
|
DON'T SHAME OSS: community catches are real but ANONYMIZED in public copy (no
|
|
17
31
|
obra/superpowers, madappgang by name) — real names live only in research/dogfood/.
|
|
18
|
-
|
|
19
|
-
volunteer's repo to show its bug.
|
|
20
|
-
RING score (88) — only the leaderboard score (52 F) — so the two never clash.
|
|
32
|
+
If an official/vendor proof returns, punch UP (name Anthropic's own); never name a
|
|
33
|
+
volunteer's repo to show its bug.
|
|
21
34
|
|
|
22
35
|
1. LEAD WITH BENEFITS / the reader's CONCRETE PAIN, never an apology, caveat, or
|
|
23
36
|
competitor. A bolded lead-in is the first thing read — make it the hook/win.
|
|
@@ -34,11 +47,33 @@
|
|
|
34
47
|
Push depth into docs/ and LINK it.
|
|
35
48
|
6. NO INTERNAL VOCABULARY (moat / measurement-authority / flywheel) and NO
|
|
36
49
|
research/ links — name the user benefit.
|
|
37
|
-
7. ASSETS: the hero vigiles-audit.png
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
50
|
+
7. ASSETS: the hero vigiles-audit.png is a REAL current report (a community
|
|
51
|
+
plugin rendered as "my-plugin" to anonymize) — C 72 with five rings, the
|
|
52
|
+
SAFETY ring (80) flagging a subagent holding all three lethal-trifecta legs
|
|
53
|
+
(a prompt-injection exfil path) + an inline subagent-tool-contract fix; the
|
|
54
|
+
dramatic Safety catch is the whole point of leading with this report (chose
|
|
55
|
+
the "bite" over a clean A 92 on 2026-06-29). No dialect-drift banner (HTML
|
|
56
|
+
report is terminal-banner-free by design). Re-render via headless Chromium on
|
|
57
|
+
the React report if the UI changes (recipe: copy a trifecta-bearing plugin to
|
|
58
|
+
my-plugin/, `node dist/cli.js audit my-plugin --no-json --no-serve`,
|
|
59
|
+
headless_shell `--window-size=820,1180 --force-device-scale-factor=2
|
|
60
|
+
--screenshot` on vigiles-report.html, then `rm -rf my-plugin
|
|
61
|
+
vigiles-report.html`). (vigiles-demo.gif was removed
|
|
62
|
+
from Proof 1 — it rendered as a frozen half-typed terminal and was redundant
|
|
63
|
+
with the code block; if a lint demo returns, it belongs in the Lint section
|
|
64
|
+
with a non-frozen asset.)
|
|
65
|
+
|
|
66
|
+
READABILITY (the 2026-06-29 pass — why this reads the way it does):
|
|
67
|
+
A. ONE bold per block, on the single phrase the eye should catch. Bold
|
|
68
|
+
everywhere = bold nowhere. Link CTAs may stay bold (they're navigation).
|
|
69
|
+
B. ONE idea per sentence. No em-dash clause-chains, no stacked parentheticals.
|
|
70
|
+
If a clause needs a paren, cut it or give it its own line.
|
|
71
|
+
C. PLAIN words in every LEAD; push jargon (rings, recall/precision,
|
|
72
|
+
interceptTools, selector, deterministic) into the linked docs. A skimmer who
|
|
73
|
+
lives in Claude Code still may not know the vocabulary.
|
|
74
|
+
D. SHOW via the proofs/code blocks; don't stack adjectives ("real, popular,
|
|
75
|
+
free, model-less") on top of what the block already proves.
|
|
76
|
+
E. SELL the outcome before the mechanism; the instruments come AFTER the proofs.
|
|
42
77
|
-->
|
|
43
78
|
|
|
44
79
|
<p align="center">
|
|
@@ -48,7 +83,7 @@
|
|
|
48
83
|
<h1 align="center">vigiles</h1>
|
|
49
84
|
|
|
50
85
|
<p align="center">
|
|
51
|
-
<strong>The tests your
|
|
86
|
+
<strong>The tests your agent's skills and hooks never had.</strong>
|
|
52
87
|
</p>
|
|
53
88
|
|
|
54
89
|
<p align="center">
|
|
@@ -59,153 +94,133 @@
|
|
|
59
94
|
|
|
60
95
|
---
|
|
61
96
|
|
|
62
|
-
**You installed
|
|
63
|
-
A skill that never fires, a subagent wired to a tool that doesn't exist, a CLAUDE.md
|
|
64
|
-
full of dead references — your harness fails **silently**, and you find out mid-task.
|
|
97
|
+
**You installed some plugins and wrote a few skills. Do they actually work?**
|
|
65
98
|
|
|
66
|
-
|
|
67
|
-
|
|
99
|
+
Your skills, hooks, and instructions are your agent's **harness** — the half you wrote
|
|
100
|
+
and own, and the half nothing checks. A skill that never fires. Two skills the agent
|
|
101
|
+
confuses. A subagent wired to a tool that doesn't exist. It breaks silently, and you
|
|
102
|
+
find out mid-task.
|
|
103
|
+
|
|
104
|
+
It's a library with no tests. This runs them:
|
|
68
105
|
|
|
69
106
|
```bash
|
|
70
107
|
npx vigiles audit
|
|
71
108
|
```
|
|
72
109
|
|
|
73
|
-
Here's what it
|
|
110
|
+
No key, no config, safe on any repo. Here's what it caught on plugins people actually
|
|
111
|
+
ship. ↓
|
|
74
112
|
|
|
75
|
-
##
|
|
113
|
+
## What it caught
|
|
76
114
|
|
|
77
115
|
<p align="center">
|
|
78
|
-
<img src="vigiles-audit.png" width="760" alt="vigiles audit report
|
|
116
|
+
<img src="vigiles-audit.png" width="760" alt="vigiles audit report scoring my-plugin C (72/100): five categories scored A–F — Truthfulness, Triggering, Structure, Safety, Tested — with the Safety category flagging a subagent that holds all three lethal-trifecta legs (a prompt-injection exfil path), plus an inline fix card for a subagent declaring a tool that doesn't exist" />
|
|
79
117
|
</p>
|
|
80
|
-
<!-- REFRESH before launch from research/dogfood/audit-superpowers.html (a real catch, no drift banner). -->
|
|
81
|
-
|
|
82
|
-
Four deterministic rings, **each finding's fix inline**, and a shareable HTML report.
|
|
83
|
-
Like Lighthouse, `audit` is a **local report you run on your machine** — safe on any
|
|
84
|
-
repo (even one wired to prod), identical on every OS. **Not a CI step** (CI uses
|
|
85
|
-
`lint`). **[Audit a harness →](docs/for-plugin-authors.md)**
|
|
86
|
-
|
|
87
|
-
## Proof 1 — your CLAUDE.md is lying to your agent
|
|
88
|
-
|
|
89
|
-
```text
|
|
90
|
-
● Truthfulness 92
|
|
91
|
-
└ ✗ skills/using-debugging/SKILL.md (referenced but MISSING)
|
|
92
|
-
```
|
|
93
118
|
|
|
94
|
-
|
|
95
|
-
|
|
119
|
+
**Like Google's Lighthouse, but for your agent harness.** Five categories, each scored
|
|
120
|
+
A–F — Truthfulness, Triggering, Structure, Safety, Tested — with every fix shown inline.
|
|
96
121
|
|
|
97
|
-
|
|
98
|
-
|
|
122
|
+
It runs locally and only reads, so it's safe on any repo and the same on every OS.
|
|
123
|
+
For CI gating, use `vigiles lint` instead. **[Audit a harness →](docs/for-plugin-authors.md)**
|
|
99
124
|
|
|
100
|
-
## Proof
|
|
125
|
+
## Proof 1 — your agent can read your secrets and ship them out
|
|
101
126
|
|
|
102
127
|
```text
|
|
103
|
-
|
|
104
|
-
|
|
128
|
+
◑ Safety 80 (80/100)
|
|
129
|
+
└ subagent "tester" holds all three lethal-trifecta legs:
|
|
130
|
+
reads private data (Bash, Read) · takes in untrusted web content (WebFetch)
|
|
131
|
+
· can send data out (Bash, WebFetch)
|
|
105
132
|
```
|
|
106
133
|
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
## Proof 3 — even the official plugins
|
|
134
|
+
Give one subagent all three powers and it's a **prompt-injection exfil path**: a poisoned
|
|
135
|
+
web page can tell it to read your `.env` and POST it anywhere — no exploit code, just the
|
|
136
|
+
tools it was handed. vigiles flags it from the tool list alone, free, no model.
|
|
137
|
+
**[How the Safety check works →](docs/for-plugin-authors.md)**
|
|
113
138
|
|
|
114
|
-
|
|
139
|
+
## Proof 2 — two skills your agent can't tell apart
|
|
115
140
|
|
|
116
141
|
```text
|
|
117
|
-
✗
|
|
118
|
-
|
|
142
|
+
✗ Triggering 0 (0/100)
|
|
143
|
+
└ 45 pairs of near-identical skill descriptions — the agent can't tell them
|
|
144
|
+
apart, so the wrong one fires (e.g. "agent-coder" ↔ "agent-tester", 83% alike)
|
|
119
145
|
```
|
|
120
146
|
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
the
|
|
147
|
+
One popular plugin ships **45 pairs of skills** with near-identical descriptions. The
|
|
148
|
+
agent picks which skill to run by reading those descriptions, so when two match it
|
|
149
|
+
fires the wrong one. The markdown is perfectly valid.
|
|
150
|
+
**[How triggering works →](docs/measuring-skills.md)**
|
|
124
151
|
|
|
125
|
-
## Proof
|
|
126
|
-
|
|
127
|
-
`audit` ranks a folder of plugins by what's actually **broken** — no key. Run across
|
|
128
|
-
**all of Anthropic's official plugins**, it flags exactly **one** real outlier — no
|
|
129
|
-
false-positive noise (untested surfaces are advisory):
|
|
152
|
+
## Proof 3 — a tool your subagent silently can't call
|
|
130
153
|
|
|
131
154
|
```text
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
25 70 C pr-review-toolkit — 6 agents inherit all tools (a reviewer with Write + Bash)
|
|
155
|
+
✗ tester — Tool "AskUserQuestion" is never available to a subagent.
|
|
156
|
+
→ remove or correct it — it's silently dropped from the contract.
|
|
135
157
|
```
|
|
136
158
|
|
|
137
|
-
|
|
159
|
+
This subagent — a helper your main agent hands a task to — declares a tool that
|
|
160
|
+
doesn't exist. The harness drops it silently, so the agent loses a capability it
|
|
161
|
+
thinks it has. vigiles catches it and gives you the **one-line fix**.
|
|
138
162
|
|
|
139
|
-
|
|
163
|
+
That's the whole idea — it checks your harness against reality, not style. Every path,
|
|
164
|
+
script, code symbol, and linter rule, verified to exist _and_ be enabled across 7
|
|
165
|
+
catalogs (ESLint, Ruff, Clippy + four more).
|
|
166
|
+
**[Full guide →](docs/verifying-instruction-files.md)**
|
|
140
167
|
|
|
141
|
-
|
|
142
|
-
|
|
168
|
+
All three catches are free and need no model — and vigiles **prevents** other whole
|
|
169
|
+
classes of bug by construction (a typed spec or compiled hook just won't compile).
|
|
170
|
+
**[Everything it catches and prevents →](docs/what-vigiles-catches.md)** · point `audit`
|
|
171
|
+
at a whole marketplace and it ranks every plugin the same way.
|
|
172
|
+
**[Audit a marketplace →](docs/for-plugin-authors.md)**
|
|
143
173
|
|
|
144
|
-
## How it
|
|
174
|
+
## How it works
|
|
145
175
|
|
|
146
|
-
|
|
147
|
-
|
|
176
|
+
The model isn't yours to fix. Your harness is. `audit` shows you the problems — here's
|
|
177
|
+
what fixes and proves each one, almost all of it with no model and no key.
|
|
148
178
|
|
|
149
179
|
### 🔎 Lint — your CLAUDE.md stops lying
|
|
150
180
|
|
|
151
|
-
Every path, script, symbol
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
edit them for you. Prefer zero new files? Plain markdown + one inline
|
|
156
|
-
`<!-- vigiles:enforce -->` comment lints too — no TypeScript.
|
|
181
|
+
Every path, script, symbol, and rule verified against reality — the catches above.
|
|
182
|
+
You don't write the checks: `npx vigiles init` turns your CLAUDE.md, skills, and
|
|
183
|
+
subagents into _specs_ (same content, plus a layer vigiles can verify). Non-destructive,
|
|
184
|
+
edited by your agent in plain English, undone by `eject`.
|
|
157
185
|
**[How →](docs/verifying-instruction-files.md)**
|
|
158
186
|
|
|
159
187
|
### 🧪 Test — does the harness actually do its job?
|
|
160
188
|
|
|
161
|
-
A hook that blocks nothing, a skill that hijacks unrelated prompts,
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
```typescript
|
|
166
|
-
import { runHook } from "vigiles/testing";
|
|
167
|
-
|
|
168
|
-
const r = runHook(guard, {
|
|
169
|
-
hook_event_name: "PreToolUse",
|
|
170
|
-
tool_name: "Bash",
|
|
171
|
-
tool_input: { command: "git commit --no-verify" },
|
|
172
|
-
});
|
|
173
|
-
assert(r.blocked); // a red ✗ means your guard silently lets it through
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
Hooks **block** · skills **trigger** (recall _and_ precision) · subagents **finish**
|
|
177
|
-
in the outcome they promised (no LLM judge) · **safety holds** — `interceptTools`
|
|
178
|
-
catches a push or paid-API call as an _attempt_, so the side effect never happens.
|
|
179
|
-
Almost every tier runs with **no model and no key**, on every commit.
|
|
189
|
+
A hook that blocks nothing, a skill that hijacks unrelated prompts, context that never
|
|
190
|
+
reaches the model — each passes a naive "did it run?" check. vigiles tests the real
|
|
191
|
+
thing: hooks **block**, skills **fire**, subagents **finish what they promised**, and a
|
|
192
|
+
stray `git push` is caught before it happens. No model, no key, on every commit.
|
|
180
193
|
**[How testing works →](docs/harness-testing.md)**
|
|
181
194
|
|
|
182
195
|
### 📊 Eval — does a skill help, or just cost more?
|
|
183
196
|
|
|
184
|
-
_"65% fewer tokens." Says who?_ vigiles A/Bs the claim on real coding tasks and reports
|
|
185
|
-
the
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
**The eval you can actually afford:** promptfoo / DeepEval hit a metered API and bill
|
|
189
|
-
**per token, every run**. vigiles answers most questions with **no model at all**, and
|
|
190
|
-
runs the rest on your own **Claude Pro/Max subscription — $0 extra.**
|
|
191
|
-
**[Measure a skill →](docs/measuring-skills.md)**
|
|
197
|
+
_"65% fewer tokens." Says who?_ vigiles[^name] A/Bs the claim on real coding tasks and reports
|
|
198
|
+
the token bill, whether it hit its target, and whether the code still works. promptfoo
|
|
199
|
+
and DeepEval bill **per token, every run**; vigiles runs on your own Claude Pro/Max
|
|
200
|
+
subscription. **[Measure a skill →](docs/measuring-skills.md)**
|
|
192
201
|
|
|
193
202
|
## Quick start
|
|
194
203
|
|
|
195
|
-
**
|
|
204
|
+
**1. See what's broken** — read-only, no setup:
|
|
205
|
+
|
|
206
|
+
```bash
|
|
207
|
+
npx vigiles audit
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
**2. Set it up** when you like what you see. Paste into Claude Code or Codex:
|
|
196
211
|
|
|
197
212
|
```text
|
|
198
213
|
Set up vigiles in this repo: run `npx vigiles init` and accept the defaults. If I
|
|
199
214
|
already have a CLAUDE.md or AGENTS.md, adopt it into a spec and show me which
|
|
200
|
-
references are stale. Then
|
|
201
|
-
|
|
202
|
-
eval without asking me first.
|
|
215
|
+
references are stale. Then compile and write + run one harness test for a hook or
|
|
216
|
+
skill of mine. Don't run a real-model eval without asking me first.
|
|
203
217
|
```
|
|
204
218
|
|
|
205
|
-
Or
|
|
219
|
+
Or run it yourself:
|
|
206
220
|
|
|
207
221
|
```bash
|
|
208
|
-
npx vigiles init #
|
|
222
|
+
npx vigiles init # adopts your files (non-destructive — eject reverses), adds CI,
|
|
223
|
+
# installs the Claude Code plugin globally
|
|
209
224
|
```
|
|
210
225
|
|
|
211
226
|
Interactive in a terminal, non-interactive for agents/CI (or `--yes`).
|
|
@@ -234,15 +249,14 @@ JS **or** TS (`*.harness.{mjs,ts}`) — run with `npx vigiles test`.
|
|
|
234
249
|
## FAQ
|
|
235
250
|
|
|
236
251
|
- **Isn't this just a markdown linter?** No — it checks whether your instruction file is _true_ (every path/script/symbol/rule exists and is enabled), then tests and measures your harness. A style linter can't do any of that.
|
|
237
|
-
- **Do I have to write TypeScript?** No — your agent writes the spec (`init` adopts your CLAUDE.md into one)
|
|
238
|
-
- **Does it overwrite my files?** No. `init` adopts an existing CLAUDE.md _non-destructively_ — untouched until you `compile`, and `eject` reverses it.
|
|
239
|
-
- **Need an API key?** No for almost everything (free, every commit). Real-model evals run on your Claude Pro/Max subscription — $0 metered tokens.
|
|
252
|
+
- **Do I have to write TypeScript?** No — your agent writes the spec (`init` adopts your CLAUDE.md into one), or plain markdown lints with zero new files. Compiler-grade guarantees are opt-in, like TS's `strict`.
|
|
240
253
|
- **Non-JS repo?** `npx vigiles lint` verifies your CLAUDE.md with no install (Ruff/Clippy/Pylint/… too).
|
|
241
254
|
|
|
242
255
|
**[Full FAQ →](docs/faq.md)**
|
|
243
256
|
|
|
244
257
|
## More
|
|
245
258
|
|
|
259
|
+
- **[What vigiles catches and prevents →](docs/what-vigiles-catches.md)** — the full matrix of harness problems it handles, biggest first, marked prevent / catch / measure.
|
|
246
260
|
- **[CLI →](docs/cli.md)** · **[GitHub Action →](docs/github-action.md)** · the full **[lint rules matrix →](docs/verifying-instruction-files.md#the-validation-rules--the-full-matrix)** lives with the linting guide.
|
|
247
261
|
- **[Skills →](docs/skills.md)** — the skills `init` installs, and how the model-invocable ones trigger.
|
|
248
262
|
- **[Ship plugins? The plugin-author guide →](docs/for-plugin-authors.md)** — scan a draft, make your skills fire, rank a whole marketplace — no key.
|
|
@@ -79,6 +79,21 @@ exports.claudeCodeDialect = {
|
|
|
79
79
|
"SessionStart",
|
|
80
80
|
"SessionEnd",
|
|
81
81
|
],
|
|
82
|
+
// Events where a block decision is silently ignored ENTIRELY — no veto AND no
|
|
83
|
+
// model feedback (exit 2 there writes stderr only to the user). These are the
|
|
84
|
+
// ONLY events hook-block-ineffective flags as wrong-event. PostToolUse is NOT
|
|
85
|
+
// here: its exit 2 feeds stderr back to the model (a legitimate nudge/feedback
|
|
86
|
+
// channel), so flagging it would cry wolf (e.g. vigiles's own refs-nudge.sh).
|
|
87
|
+
noEffectHookEvents: [
|
|
88
|
+
"SessionStart",
|
|
89
|
+
"SessionEnd",
|
|
90
|
+
"Notification",
|
|
91
|
+
"PreCompact",
|
|
92
|
+
],
|
|
93
|
+
// PreToolUse is the one event whose deny needs the structured
|
|
94
|
+
// `hookSpecificOutput.permissionDecision:"deny"`; the legacy top-level
|
|
95
|
+
// `decision` field is ignored there.
|
|
96
|
+
permissionDecisionHookEvents: ["PreToolUse"],
|
|
82
97
|
// Claude Code natively reads CLAUDE.md only — it does NOT auto-load AGENTS.md
|
|
83
98
|
// (anthropics/claude-code#34235 is open; AGENTS.md works solely via an
|
|
84
99
|
// `@AGENTS.md` import inside CLAUDE.md or a symlink). AGENTS.md is the
|
package/dist/audit-html.d.ts
CHANGED
|
@@ -5,16 +5,27 @@ import type { AuditReport } from "./audit-report.js";
|
|
|
5
5
|
* we use `__dirname`, not `import.meta`.
|
|
6
6
|
*/
|
|
7
7
|
export declare function templatePath(): string | null;
|
|
8
|
+
/**
|
|
9
|
+
* Live-server info injected into the report when it's served by `audit --serve`
|
|
10
|
+
* (not a static file): the per-run token the report must send on every adopt
|
|
11
|
+
* POST. Absent for a plain static report — the buttons fall back to copy-command.
|
|
12
|
+
*/
|
|
13
|
+
export interface ServeInfo {
|
|
14
|
+
readonly token: string;
|
|
15
|
+
}
|
|
8
16
|
/**
|
|
9
17
|
* Inject the report JSON into a template by replacing the quoted placeholder
|
|
10
18
|
* string with the JSON object literal. Pure — the testable core. Throws if the
|
|
11
|
-
* template is missing the placeholder.
|
|
19
|
+
* template is missing the placeholder. When `serve` is given (the `--serve`
|
|
20
|
+
* path), also sets `window.__VIGILES_SERVE__` so the report's adopt buttons POST
|
|
21
|
+
* to the local server instead of copying the command.
|
|
12
22
|
*/
|
|
13
|
-
export declare function injectReportData(template: string, report: AuditReport): string;
|
|
23
|
+
export declare function injectReportData(template: string, report: AuditReport, serve?: ServeInfo): string;
|
|
14
24
|
/**
|
|
15
25
|
* Render the self-contained HTML report (React template + injected data). Throws
|
|
16
26
|
* if the template hasn't been built — the caller (writeAuditHtml) catches that and
|
|
17
|
-
* skips the HTML, since the JSON + terminal report don't depend on it.
|
|
27
|
+
* skips the HTML, since the JSON + terminal report don't depend on it. Pass
|
|
28
|
+
* `serve` to render the live (`--serve`) variant whose buttons POST.
|
|
18
29
|
*/
|
|
19
|
-
export declare function renderAuditHtml(report: AuditReport): string;
|
|
30
|
+
export declare function renderAuditHtml(report: AuditReport, serve?: ServeInfo): string;
|
|
20
31
|
//# sourceMappingURL=audit-html.d.ts.map
|
package/dist/audit-html.js
CHANGED
|
@@ -37,25 +37,34 @@ function escapeForScript(json) {
|
|
|
37
37
|
/**
|
|
38
38
|
* Inject the report JSON into a template by replacing the quoted placeholder
|
|
39
39
|
* string with the JSON object literal. Pure — the testable core. Throws if the
|
|
40
|
-
* template is missing the placeholder.
|
|
40
|
+
* template is missing the placeholder. When `serve` is given (the `--serve`
|
|
41
|
+
* path), also sets `window.__VIGILES_SERVE__` so the report's adopt buttons POST
|
|
42
|
+
* to the local server instead of copying the command.
|
|
41
43
|
*/
|
|
42
|
-
function injectReportData(template, report) {
|
|
44
|
+
function injectReportData(template, report, serve) {
|
|
43
45
|
const re = new RegExp(`(["'])${PLACEHOLDER}\\1`);
|
|
44
46
|
if (!re.test(template)) {
|
|
45
47
|
throw new Error("audit report template is missing the data placeholder");
|
|
46
48
|
}
|
|
47
|
-
|
|
49
|
+
let out = template.replace(re, escapeForScript(JSON.stringify(report)));
|
|
50
|
+
if (serve) {
|
|
51
|
+
// Prepend the serve global onto the same statement that carries the data, so
|
|
52
|
+
// it's set before the app bundle reads it. The token is hex, but escape anyway.
|
|
53
|
+
out = out.replace("window.__VIGILES_DATA__", `window.__VIGILES_SERVE__=${escapeForScript(JSON.stringify(serve))};window.__VIGILES_DATA__`);
|
|
54
|
+
}
|
|
55
|
+
return out;
|
|
48
56
|
}
|
|
49
57
|
/**
|
|
50
58
|
* Render the self-contained HTML report (React template + injected data). Throws
|
|
51
59
|
* if the template hasn't been built — the caller (writeAuditHtml) catches that and
|
|
52
|
-
* skips the HTML, since the JSON + terminal report don't depend on it.
|
|
60
|
+
* skips the HTML, since the JSON + terminal report don't depend on it. Pass
|
|
61
|
+
* `serve` to render the live (`--serve`) variant whose buttons POST.
|
|
53
62
|
*/
|
|
54
|
-
function renderAuditHtml(report) {
|
|
63
|
+
function renderAuditHtml(report, serve) {
|
|
55
64
|
const p = templatePath();
|
|
56
65
|
if (!p) {
|
|
57
66
|
throw new Error("audit report template not built — run `npm run build` (builds report/), or use --json / --no-html");
|
|
58
67
|
}
|
|
59
|
-
return injectReportData((0, node_fs_1.readFileSync)(p, "utf-8"), report);
|
|
68
|
+
return injectReportData((0, node_fs_1.readFileSync)(p, "utf-8"), report, serve);
|
|
60
69
|
}
|
|
61
70
|
//# sourceMappingURL=audit-html.js.map
|
package/dist/audit-report.d.ts
CHANGED
|
@@ -14,13 +14,22 @@
|
|
|
14
14
|
import { type AuditScore } from "./audit-score.js";
|
|
15
15
|
import { type Recommendation } from "./optimize.js";
|
|
16
16
|
import type { AdoptabilityResult } from "./adoptability.js";
|
|
17
|
-
import type { ScanReport } from "./scan.js";
|
|
17
|
+
import type { ScanReport, MarketplaceInfo } from "./scan.js";
|
|
18
|
+
import type { PluginScore } from "./leaderboard.js";
|
|
18
19
|
/** The current schema version. Bump only on a BREAKING change to the shape. */
|
|
19
20
|
export declare const AUDIT_SCHEMA_VERSION = 1;
|
|
20
21
|
export interface AuditReportMeta {
|
|
21
22
|
/** Wire-format version — consumers gate on this. */
|
|
22
23
|
readonly schemaVersion: typeof AUDIT_SCHEMA_VERSION;
|
|
23
24
|
readonly tool: "vigiles";
|
|
25
|
+
/**
|
|
26
|
+
* Discriminates the three `audit --json` shapes a consumer may receive:
|
|
27
|
+
* `audit` (one plugin → {@link AuditReport}), `leaderboard` (a marketplace /
|
|
28
|
+
* multiple dirs → {@link LeaderboardReport}), `marketplace` (a curated,
|
|
29
|
+
* all-external marketplace → {@link MarketplaceReport}). Always present so the
|
|
30
|
+
* JSON is self-describing.
|
|
31
|
+
*/
|
|
32
|
+
readonly kind: "audit";
|
|
24
33
|
/** The vigiles version that produced the report. */
|
|
25
34
|
readonly vigilesVersion: string;
|
|
26
35
|
/** The detected/selected harness (`claude-code`, `codex`, …). */
|
|
@@ -68,7 +77,7 @@ export interface Adoptable {
|
|
|
68
77
|
*/
|
|
69
78
|
export interface AuditReport {
|
|
70
79
|
readonly meta: AuditReportMeta;
|
|
71
|
-
/** The
|
|
80
|
+
/** The five deterministic category rings + the weighted overall + grade. */
|
|
72
81
|
readonly score: AuditScore;
|
|
73
82
|
/** The deterministic, ranked fixes (the inline recommendations). */
|
|
74
83
|
readonly recommendations: readonly Recommendation[];
|
|
@@ -104,4 +113,51 @@ export interface BuildAuditReportOptions {
|
|
|
104
113
|
* HTML-embedded form omits it so the rendered file stays deterministic.
|
|
105
114
|
*/
|
|
106
115
|
export declare function buildAuditReport(report: ScanReport, opts: BuildAuditReportOptions): AuditReport;
|
|
116
|
+
/**
|
|
117
|
+
* The versioned envelope for a `audit --json` run over MULTIPLE plugins (a
|
|
118
|
+
* marketplace expanded into its members, or several dirs) — the leaderboard.
|
|
119
|
+
* Shares the same `meta.schemaVersion`/`tool`/`kind` self-description as
|
|
120
|
+
* {@link AuditReport} so every `audit --json` shape is a versioned object, never
|
|
121
|
+
* a bare array. `kind:"leaderboard"` is the discriminant; `plugins` carries the
|
|
122
|
+
* ranked per-plugin scores.
|
|
123
|
+
*/
|
|
124
|
+
export interface LeaderboardReport {
|
|
125
|
+
readonly meta: {
|
|
126
|
+
readonly schemaVersion: typeof AUDIT_SCHEMA_VERSION;
|
|
127
|
+
readonly tool: "vigiles";
|
|
128
|
+
readonly kind: "leaderboard";
|
|
129
|
+
readonly vigilesVersion: string;
|
|
130
|
+
/** The marketplace / parent dir that was expanded and ranked. */
|
|
131
|
+
readonly dir: string;
|
|
132
|
+
readonly generatedAt?: string;
|
|
133
|
+
};
|
|
134
|
+
readonly plugins: readonly PluginScore[];
|
|
135
|
+
}
|
|
136
|
+
/** Assemble the versioned {@link LeaderboardReport} — pure, no clock. */
|
|
137
|
+
export declare function buildLeaderboardReport(plugins: readonly PluginScore[], opts: {
|
|
138
|
+
vigilesVersion: string;
|
|
139
|
+
dir: string;
|
|
140
|
+
}): LeaderboardReport;
|
|
141
|
+
/**
|
|
142
|
+
* The versioned envelope for a `audit --json` run on a CURATED marketplace whose
|
|
143
|
+
* members are all external (git/url, nothing on disk to scan). Wraps the
|
|
144
|
+
* {@link MarketplaceInfo} inventory so this path, too, emits a versioned object
|
|
145
|
+
* rather than a raw, unversioned struct. `kind:"marketplace"` is the discriminant.
|
|
146
|
+
*/
|
|
147
|
+
export interface MarketplaceReport {
|
|
148
|
+
readonly meta: {
|
|
149
|
+
readonly schemaVersion: typeof AUDIT_SCHEMA_VERSION;
|
|
150
|
+
readonly tool: "vigiles";
|
|
151
|
+
readonly kind: "marketplace";
|
|
152
|
+
readonly vigilesVersion: string;
|
|
153
|
+
readonly dir: string;
|
|
154
|
+
readonly generatedAt?: string;
|
|
155
|
+
};
|
|
156
|
+
readonly marketplace: MarketplaceInfo;
|
|
157
|
+
}
|
|
158
|
+
/** Assemble the versioned {@link MarketplaceReport} — pure, no clock. */
|
|
159
|
+
export declare function buildMarketplaceReport(marketplace: MarketplaceInfo, opts: {
|
|
160
|
+
vigilesVersion: string;
|
|
161
|
+
dir: string;
|
|
162
|
+
}): MarketplaceReport;
|
|
107
163
|
//# sourceMappingURL=audit-report.d.ts.map
|
package/dist/audit-report.js
CHANGED
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
3
|
exports.AUDIT_SCHEMA_VERSION = void 0;
|
|
4
4
|
exports.buildAuditReport = buildAuditReport;
|
|
5
|
+
exports.buildLeaderboardReport = buildLeaderboardReport;
|
|
6
|
+
exports.buildMarketplaceReport = buildMarketplaceReport;
|
|
5
7
|
/**
|
|
6
8
|
* The `AuditReport` — the versioned JSON contract that IS the audit's product
|
|
7
9
|
* boundary. Everything renders FROM it: the local self-contained HTML report,
|
|
@@ -49,6 +51,7 @@ function buildAuditReport(report, opts) {
|
|
|
49
51
|
meta: {
|
|
50
52
|
schemaVersion: exports.AUDIT_SCHEMA_VERSION,
|
|
51
53
|
tool: "vigiles",
|
|
54
|
+
kind: "audit",
|
|
52
55
|
vigilesVersion: opts.vigilesVersion,
|
|
53
56
|
harness: opts.harness,
|
|
54
57
|
dir: report.dir,
|
|
@@ -69,4 +72,30 @@ function buildAuditReport(report, opts) {
|
|
|
69
72
|
...(adoptable ? { adoptable } : {}),
|
|
70
73
|
};
|
|
71
74
|
}
|
|
75
|
+
/** Assemble the versioned {@link LeaderboardReport} — pure, no clock. */
|
|
76
|
+
function buildLeaderboardReport(plugins, opts) {
|
|
77
|
+
return {
|
|
78
|
+
meta: {
|
|
79
|
+
schemaVersion: exports.AUDIT_SCHEMA_VERSION,
|
|
80
|
+
tool: "vigiles",
|
|
81
|
+
kind: "leaderboard",
|
|
82
|
+
vigilesVersion: opts.vigilesVersion,
|
|
83
|
+
dir: opts.dir,
|
|
84
|
+
},
|
|
85
|
+
plugins,
|
|
86
|
+
};
|
|
87
|
+
}
|
|
88
|
+
/** Assemble the versioned {@link MarketplaceReport} — pure, no clock. */
|
|
89
|
+
function buildMarketplaceReport(marketplace, opts) {
|
|
90
|
+
return {
|
|
91
|
+
meta: {
|
|
92
|
+
schemaVersion: exports.AUDIT_SCHEMA_VERSION,
|
|
93
|
+
tool: "vigiles",
|
|
94
|
+
kind: "marketplace",
|
|
95
|
+
vigilesVersion: opts.vigilesVersion,
|
|
96
|
+
dir: opts.dir,
|
|
97
|
+
},
|
|
98
|
+
marketplace,
|
|
99
|
+
};
|
|
100
|
+
}
|
|
72
101
|
//# sourceMappingURL=audit-report.js.map
|