vigiles 9.0.0 → 9.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +147 -209
- package/dist/audit-report.d.ts +37 -0
- package/dist/audit-report.js +21 -0
- package/dist/audit-report.template.html +37 -22
- package/dist/audit-score.d.ts +27 -7
- package/dist/audit-score.js +48 -45
- package/dist/cli.js +136 -5
- package/dist/core/adopt.d.ts +28 -0
- package/dist/core/adopt.js +203 -0
- package/dist/leaderboard.d.ts +32 -0
- package/dist/leaderboard.js +92 -44
- package/dist/scan.js +13 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -3,42 +3,42 @@
|
|
|
3
3
|
This file is the FRONT DOOR + a marketing asset for someone who already lives
|
|
4
4
|
in Claude Code / Codex. Optimize for a phone-skimmer.
|
|
5
5
|
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
2.
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
6
|
+
SPINE = CONCEPT 5 (proof/demo-led). Lead with REAL, screenshotable catches on
|
|
7
|
+
plugins people actually ship, THEN explain the mechanism. The proofs are not
|
|
8
|
+
illustrative — every block traces to a real dogfood run captured in
|
|
9
|
+
research/dogfood/. HYBRID proof source (decided 2026-06-28): Proofs 1-2 are
|
|
10
|
+
COMMUNITY catches, anonymized (a missing SKILL.md, an AskUserQuestion-never-
|
|
11
|
+
available tool) — official plugins don't have those silent-failure bugs. Proofs
|
|
12
|
+
3-4 are OFFICIAL + NAMED: pr-review-toolkit (malformed YAML + unrestricted review
|
|
13
|
+
agents) and the all-37 Anthropic-official leaderboard (A→F, LSP stubs excluded).
|
|
14
|
+
NEVER replace a real catch with a fabricated one.
|
|
15
|
+
|
|
16
|
+
DON'T SHAME OSS: community catches are real but ANONYMIZED in public copy (no
|
|
17
|
+
obra/superpowers, madappgang by name) — real names live only in research/dogfood/.
|
|
18
|
+
Punch UP at official/vendor plugins (Anthropic's own are NAMED); never name a
|
|
19
|
+
volunteer's repo to show its bug. Proof 3/4 don't show pr-review-toolkit's audit-
|
|
20
|
+
RING score (88) — only the leaderboard score (52 F) — so the two never clash.
|
|
21
|
+
|
|
22
|
+
1. LEAD WITH BENEFITS / the reader's CONCRETE PAIN, never an apology, caveat, or
|
|
23
|
+
competitor. A bolded lead-in is the first thing read — make it the hook/win.
|
|
24
|
+
End a section on the win, not the trade-off. A paragraph is ≤ ~3 lines.
|
|
25
|
+
2. PROOF FIRST, mechanism second. The three instruments (Lint/Test/Eval) come
|
|
26
|
+
AFTER the proof stack as "how it does it", not as a competing front door.
|
|
27
|
+
3. SPEC-FIRST IS THE DEFAULT but easy — `init` ADOPTS your CLAUDE.md into a spec,
|
|
28
|
+
skills edit it, you rarely hand-write .spec.ts. Give it ONE home (Quick start),
|
|
29
|
+
not five scattered mentions. `eject` always reverses. Inline markdown is the
|
|
30
|
+
zero-TS floor.
|
|
31
|
+
4. Guard / compiled hooks is PARKED FOR LAUNCH (see research/roadmap.md). Live set
|
|
32
|
+
is Lint/Test/Eval. Do NOT make the 2/7→7/7 battery the hero — re-add post-HN.
|
|
33
|
+
5. SCANNABLE + SHORT — ~200-line cap; punchy cells, bullets, runnable blocks.
|
|
34
|
+
Push depth into docs/ and LINK it.
|
|
35
|
+
6. NO INTERNAL VOCABULARY (moat / measurement-authority / flywheel) and NO
|
|
36
|
+
research/ links — name the user benefit.
|
|
37
|
+
7. ASSETS: the hero vigiles-audit.png must be REFRESHED from a real run
|
|
38
|
+
(research/dogfood/audit-superpowers.html) without the dialect drift banner
|
|
39
|
+
before launch. (vigiles-demo.gif was removed from Proof 1 — it rendered as a
|
|
40
|
+
frozen half-typed terminal and was redundant with the code block; if a lint
|
|
41
|
+
demo returns, it belongs in the Lint section with a non-frozen asset.)
|
|
42
42
|
-->
|
|
43
43
|
|
|
44
44
|
<p align="center">
|
|
@@ -48,7 +48,7 @@
|
|
|
48
48
|
<h1 align="center">vigiles</h1>
|
|
49
49
|
|
|
50
50
|
<p align="center">
|
|
51
|
-
<strong>
|
|
51
|
+
<strong>The tests your AI agent harness never had.</strong>
|
|
52
52
|
</p>
|
|
53
53
|
|
|
54
54
|
<p align="center">
|
|
@@ -63,145 +63,104 @@
|
|
|
63
63
|
A skill that never fires, a subagent wired to a tool that doesn't exist, a CLAUDE.md
|
|
64
64
|
full of dead references — your harness fails **silently**, and you find out mid-task.
|
|
65
65
|
|
|
66
|
-
**It's a library with no tests.**
|
|
67
|
-
|
|
68
|
-
**One command shows you — like a Lighthouse report for your harness:**
|
|
66
|
+
**It's a library with no tests.** One command runs them — no key, no config, safe on
|
|
67
|
+
any repo:
|
|
69
68
|
|
|
70
69
|
```bash
|
|
71
|
-
npx vigiles audit
|
|
70
|
+
npx vigiles audit
|
|
72
71
|
```
|
|
73
72
|
|
|
74
|
-
|
|
75
|
-
|
|
73
|
+
Here's what it found on real, popular plugins. ↓
|
|
74
|
+
|
|
75
|
+
## The report
|
|
76
76
|
|
|
77
77
|
<p align="center">
|
|
78
78
|
<img src="vigiles-audit.png" width="760" alt="vigiles audit report: an overall score with four category rings — Truthfulness, Triggering, Structure, Tested — and fix cards" />
|
|
79
79
|
</p>
|
|
80
|
+
<!-- REFRESH before launch from research/dogfood/audit-superpowers.html (a real catch, no drift banner). -->
|
|
80
81
|
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
| **🔧 Structure** | Tool contracts, MCP servers & frontmatter are sound |
|
|
86
|
-
| **🧪 Tested** | Every surface ships a test |
|
|
87
|
-
|
|
88
|
-
Like Lighthouse, `audit` is a **local report you run on your machine** — a
|
|
89
|
-
deterministic read, safe on any repo (even one wired to prod), identical on every
|
|
90
|
-
OS. **Not a CI step** (CI uses `lint`, the deterministic gate). The **executing
|
|
91
|
-
checks** (resolve MCP live · **do your skills fire?** · **what would vigiles
|
|
92
|
-
catch in your repo?**) run only when there's a human to consent: `audit` **asks
|
|
93
|
-
once** at a terminal and remembers. **[Audit a harness →](docs/for-plugin-authors.md)**
|
|
94
|
-
|
|
95
|
-
`Agent = Model + Harness` — the model gets the headlines, the harness is the half you
|
|
96
|
-
own. vigiles[^name] is how you make it prove itself: `audit` is the dashboard, and
|
|
97
|
-
**three instruments** fix and prove what it finds —
|
|
98
|
-
|
|
99
|
-
| | |
|
|
100
|
-
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
101
|
-
| **🔎 Lint** | Your CLAUDE.md stops lying — every path, script, symbol & linter rule checked against **reality**. **[→](docs/verifying-instruction-files.md)** |
|
|
102
|
-
| **🧪 Test** | Prove your hooks, skills & subagents do their job — **free, no API key**. **[→](docs/harness-testing.md)** |
|
|
103
|
-
| **📊 Eval** | Know if a skill helps or just costs — **A/B on real tasks**, on your own subscription. **[→](docs/measuring-skills.md)** |
|
|
104
|
-
|
|
105
|
-
<!-- PARKED FOR LAUNCH — Guard / compiled hooks. Re-add this row + the ④ section below post-HN. See research/roadmap.md "Launch readiness".
|
|
106
|
-
| **🛡 Guard** | A safety hook that **can't silently fail open** — write a typed function, get a guard that blocks. **[→](docs/compiled-hooks.md)** |
|
|
107
|
-
-->
|
|
108
|
-
|
|
109
|
-
**Two ways in** — pick the pain that's yours:
|
|
110
|
-
|
|
111
|
-
- **Run agents on your own repo?** `npx vigiles audit`, then `npx vigiles init`.
|
|
112
|
-
- **Ship plugins to a marketplace?** `npx vigiles audit ./plugins/*/` ranks a whole
|
|
113
|
-
marketplace (0–100, A–F) — see the **[plugin-author guide →](docs/for-plugin-authors.md)**.
|
|
82
|
+
Four deterministic rings, **each finding's fix inline**, and a shareable HTML report.
|
|
83
|
+
Like Lighthouse, `audit` is a **local report you run on your machine** — safe on any
|
|
84
|
+
repo (even one wired to prod), identical on every OS. **Not a CI step** (CI uses
|
|
85
|
+
`lint`). **[Audit a harness →](docs/for-plugin-authors.md)**
|
|
114
86
|
|
|
115
|
-
|
|
116
|
-
`.spec.ts` for you, **`init` adopts an existing CLAUDE.md non-destructively**
|
|
117
|
-
(untouched until you compile), and plain markdown + inline `<!-- vigiles:enforce -->`
|
|
118
|
-
comments work with zero TypeScript. Works with **Claude Code and Codex**
|
|
119
|
-
([`vigiles/codex`](docs/harnesses.md)) or [your own harness](docs/authoring-an-adapter.md).
|
|
120
|
-
|
|
121
|
-
## Quick start
|
|
122
|
-
|
|
123
|
-
**Paste into Claude Code or Codex:**
|
|
87
|
+
## Proof 1 — your CLAUDE.md is lying to your agent
|
|
124
88
|
|
|
125
89
|
```text
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
references are stale. Then install the dep, compile, and write + run one harness
|
|
129
|
-
test for a hook or skill of mine. Don't enforce a spec-per-file or add a real-model
|
|
130
|
-
eval without asking me first.
|
|
90
|
+
● Truthfulness 92
|
|
91
|
+
└ ✗ skills/using-debugging/SKILL.md (referenced but MISSING)
|
|
131
92
|
```
|
|
132
93
|
|
|
133
|
-
|
|
94
|
+
A real, widely-installed plugin — its instructions send the agent to a skill file
|
|
95
|
+
that **isn't there**. Valid markdown, but not _true_, and your agent trusts it anyway.
|
|
134
96
|
|
|
135
|
-
|
|
97
|
+
File paths, scripts, code symbols — plus linter rules across **7 linters** (ESLint,
|
|
98
|
+
Ruff, Clippy + four more): each one **exists _and_ is enabled**. **[Full guide →](docs/verifying-instruction-files.md)**
|
|
136
99
|
|
|
137
|
-
|
|
138
|
-
npx vigiles init # sets up lint + test: spec + harness test + CI + plugin
|
|
139
|
-
```
|
|
100
|
+
## Proof 2 — a tool your subagent silently can't call
|
|
140
101
|
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
and recompiles on save; you never touch it by hand):
|
|
102
|
+
```text
|
|
103
|
+
✗ tester — Tool "AskUserQuestion" is never available to a subagent.
|
|
104
|
+
→ remove or correct it — it's silently dropped from the contract.
|
|
105
|
+
```
|
|
146
106
|
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
107
|
+
A real upstream subagent declares a tool the harness **silently drops**, so it loses
|
|
108
|
+
a capability it thinks it has. vigiles flags it _and_ hands you the one-line fix —
|
|
109
|
+
**free, no model.** That's the difference from a markdown linter: it checks your
|
|
110
|
+
harness against **reality**, not style.
|
|
150
111
|
|
|
151
|
-
|
|
152
|
-
<summary>What <code>init</code> sets up</summary>
|
|
112
|
+
## Proof 3 — even the official plugins
|
|
153
113
|
|
|
154
|
-
-
|
|
155
|
-
- **Already have a CLAUDE.md / AGENTS.md? `init` adopts it** into a spec faithfully and **non-destructively** — your file is left untouched until you choose to `compile` (and `eject` undoes it).
|
|
156
|
-
- Adds `vigiles` to `devDependencies`; installs the Claude Code plugin (skills + hooks) via the marketplace — globally, never vendored.
|
|
157
|
-
- Wires CI as a `zernie/vigiles@v1` workflow (a composite over the same CLI) that posts a sticky PR comment + a `valid` output.
|
|
114
|
+
Not cherry-picked community repos — **Anthropic's own official `pr-review-toolkit`**:
|
|
158
115
|
|
|
159
|
-
|
|
160
|
-
|
|
116
|
+
```text
|
|
117
|
+
✗ silent-failure-hunter.md — frontmatter isn't valid YAML (won't parse)
|
|
118
|
+
⚠ 6 review agents inherit ALL tools — a code-reviewer with Write + Bash
|
|
119
|
+
```
|
|
161
120
|
|
|
162
|
-
|
|
121
|
+
A subagent whose YAML doesn't parse, and six review agents holding far more power than
|
|
122
|
+
a reviewer needs — real findings on a first-party plugin. _(We name Anthropic's own;
|
|
123
|
+
the community plugins above stay anonymous — punch up, don't shame volunteers.)_
|
|
163
124
|
|
|
164
|
-
##
|
|
125
|
+
## Proof 4 — rank a whole marketplace
|
|
165
126
|
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
fiction. `npx vigiles lint` resolves every reference against reality:
|
|
127
|
+
`audit` ranks a folder of plugins by what's actually **broken** — no key. Run across
|
|
128
|
+
**all of Anthropic's official plugins**, it flags exactly **one** real outlier — no
|
|
129
|
+
false-positive noise (untested surfaces are advisory):
|
|
170
130
|
|
|
171
131
|
```text
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
✓ @typescript-eslint/no-floating-promises — exists and enabled in eslint config
|
|
132
|
+
# score grade plugin
|
|
133
|
+
1 100 A code-review ← 24 of 25 come back a clean A
|
|
134
|
+
25 70 C pr-review-toolkit — 6 agents inherit all tools (a reviewer with Write + Bash)
|
|
176
135
|
```
|
|
177
136
|
|
|
178
|
-
<
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
<!-- Regenerate the GIF: `python3 scripts/make-demo-gif.py` (output is verbatim CLI; see scripts/demo.sh for a live asciinema recording). -->
|
|
137
|
+
<sub>Real scores on Anthropic's own marketplace — a fair tool flags the one real issue, not noise (empty LSP stubs excluded).</sub>
|
|
138
|
+
|
|
139
|
+
**[Plugin-author guide →](docs/for-plugin-authors.md)**
|
|
182
140
|
|
|
183
|
-
|
|
184
|
-
|
|
141
|
+
> **And it grades itself: 100/100, A, all four rings green.** vigiles runs `audit` on
|
|
142
|
+
> its own harness in CI. We eat what we cook.
|
|
185
143
|
|
|
186
|
-
|
|
187
|
-
faithfully and non-destructively.** Prefer zero new files? Plain markdown + one
|
|
188
|
-
inline `<!-- vigiles:enforce -->` comment lints too — no spec, no TypeScript. And
|
|
189
|
-
`vigiles eject` hands a spec back to markdown anytime.
|
|
190
|
-
**[Full guide →](docs/verifying-instruction-files.md)**
|
|
144
|
+
## How it does it — three instruments
|
|
191
145
|
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
> graduated like TypeScript's `strict`, on only when you want it.
|
|
195
|
-
> **[How →](docs/compiled-hooks.md)**
|
|
146
|
+
`Agent = Model + Harness`[^name]. The model gets the headlines; the **harness** is the
|
|
147
|
+
half you own. `audit` is the dashboard — these fix and prove what it finds.
|
|
196
148
|
|
|
197
|
-
|
|
149
|
+
### 🔎 Lint — your CLAUDE.md stops lying
|
|
198
150
|
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
151
|
+
Every path, script, symbol & linter rule resolved against reality (the catches
|
|
152
|
+
above). **You don't write any of it** — `npx vigiles init` **adopts your existing
|
|
153
|
+
CLAUDE.md _and every skill and subagent_ into verified specs**, non-destructively
|
|
154
|
+
(untouched until you `compile`; `eject` reverses). After that, plain-English asks
|
|
155
|
+
edit them for you. Prefer zero new files? Plain markdown + one inline
|
|
156
|
+
`<!-- vigiles:enforce -->` comment lints too — no TypeScript.
|
|
157
|
+
**[How →](docs/verifying-instruction-files.md)**
|
|
203
158
|
|
|
204
|
-
|
|
159
|
+
### 🧪 Test — does the harness actually do its job?
|
|
160
|
+
|
|
161
|
+
A hook that blocks nothing, a skill that hijacks unrelated prompts, injected context
|
|
162
|
+
that never reaches the model — all pass a naive "did it run?" check. Start at the
|
|
163
|
+
cheapest tier: a hook, called directly. **No model, no key:**
|
|
205
164
|
|
|
206
165
|
```typescript
|
|
207
166
|
import { runHook } from "vigiles/testing";
|
|
@@ -214,89 +173,68 @@ const r = runHook(guard, {
|
|
|
214
173
|
assert(r.blocked); // a red ✗ means your guard silently lets it through
|
|
215
174
|
```
|
|
216
175
|
|
|
217
|
-
|
|
176
|
+
Hooks **block** · skills **trigger** (recall _and_ precision) · subagents **finish**
|
|
177
|
+
in the outcome they promised (no LLM judge) · **safety holds** — `interceptTools`
|
|
178
|
+
catches a push or paid-API call as an _attempt_, so the side effect never happens.
|
|
179
|
+
Almost every tier runs with **no model and no key**, on every commit.
|
|
180
|
+
**[How testing works →](docs/harness-testing.md)**
|
|
218
181
|
|
|
219
|
-
|
|
220
|
-
- **Skills trigger** on the right prompts and stay quiet on the wrong ones — recall _and_ precision (`measureTriggerRate`).
|
|
221
|
-
- **Subagents finish right** — assert a subagent ended in the success (or error) outcome it promised, with a plain check, no LLM judge (`assertAgentOk` / `assertAgentErr`).
|
|
222
|
-
- **Behaviour is good** — score a skill's output, or A/B it on-vs-off for the real lift (`measure` / `runEval`, with significance testing).
|
|
223
|
-
- **Safety holds** — the agent _didn't_ push to the wrong branch or hit a paid API; `interceptTools` catches the attempt so the side effect never happens.
|
|
182
|
+
### 📊 Eval — does a skill help, or just cost more?
|
|
224
183
|
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
184
|
+
_"65% fewer tokens." Says who?_ vigiles A/Bs the claim on real coding tasks and reports
|
|
185
|
+
the **bill**, the **target it claims to move**, and the **blast radius** (did the code
|
|
186
|
+
still work?).
|
|
228
187
|
|
|
229
|
-
|
|
188
|
+
**The eval you can actually afford:** promptfoo / DeepEval hit a metered API and bill
|
|
189
|
+
**per token, every run**. vigiles answers most questions with **no model at all**, and
|
|
190
|
+
runs the rest on your own **Claude Pro/Max subscription — $0 extra.**
|
|
191
|
+
**[Measure a skill →](docs/measuring-skills.md)**
|
|
230
192
|
|
|
231
|
-
|
|
232
|
-
it — stars and vibes, **zero measurement**. vigiles A/Bs the claim on real coding
|
|
233
|
-
tasks, the harness loaded exactly as it ships, and reports **three numbers**:
|
|
193
|
+
## Quick start
|
|
234
194
|
|
|
235
|
-
|
|
236
|
-
import { measureArms } from "vigiles/testing";
|
|
195
|
+
**Paste into Claude Code or Codex:**
|
|
237
196
|
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
197
|
+
```text
|
|
198
|
+
Set up vigiles in this repo: run `npx vigiles init` and accept the defaults. If I
|
|
199
|
+
already have a CLAUDE.md or AGENTS.md, adopt it into a spec and show me which
|
|
200
|
+
references are stale. Then install the dep, compile, and write + run one harness
|
|
201
|
+
test for a hook or skill of mine. Don't enforce a spec-per-file or add a real-model
|
|
202
|
+
eval without asking me first.
|
|
244
203
|
```
|
|
245
204
|
|
|
246
|
-
|
|
247
|
-
- **The target** — whatever the skill claims to move (output tokens, latency, tool calls), verified on its own terms.
|
|
248
|
-
- **The blast radius** — correctness, a deterministic 1/0. A token win that breaks the code is **not a win**.
|
|
249
|
-
|
|
250
|
-
**Safe to repeat.** Each real-model run is sandboxed (ephemeral dir, egress blocked
|
|
251
|
-
or allow-listed), and `interceptTools` catches an irreversible external — a push, a
|
|
252
|
-
paid API — as an _attempt_, never running it. **[Safety, sandboxing & FAQ →](docs/safety.md)**
|
|
253
|
-
|
|
254
|
-
**The eval you can actually afford.** promptfoo / DeepEval hit a metered API and
|
|
255
|
-
bill **per token, every run**. vigiles answers most questions with **no model at
|
|
256
|
-
all**, and runs the rest on your own **Claude Pro/Max subscription — $0 extra**. So
|
|
257
|
-
you can measure on every change. **[Eval a skill →](docs/measuring-skills.md)** · **[Why it's affordable →](docs/eval-architecture.md)**
|
|
205
|
+
Or do it yourself:
|
|
258
206
|
|
|
259
|
-
|
|
207
|
+
```bash
|
|
208
|
+
npx vigiles init # lint + test: spec + harness test + CI + plugin
|
|
209
|
+
```
|
|
260
210
|
|
|
261
|
-
|
|
211
|
+
Interactive in a terminal, non-interactive for agents/CI (or `--yes`).
|
|
262
212
|
|
|
263
|
-
**
|
|
264
|
-
|
|
265
|
-
you. _(Already write safety hooks? This is the power tool.)_ Write a pure typed
|
|
266
|
-
function instead; vigiles emits the exit code, the JSON, and an AST-backed matcher:
|
|
213
|
+
**You don't hand-write any of this — your agent does.** `init` installs
|
|
214
|
+
model-invocable skills, so a plain-English ask does the work:
|
|
267
215
|
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
export default defineHook({
|
|
272
|
-
on: "PreToolUse",
|
|
273
|
-
match: tool("Bash"),
|
|
274
|
-
decide: (e) =>
|
|
275
|
-
e.command.runs("git push", { force: true })
|
|
276
|
-
? deny("no force-push to a protected branch")
|
|
277
|
-
: allow(),
|
|
278
|
-
});
|
|
279
|
-
```
|
|
216
|
+
- _"test my skills"_ → scaffolds **and runs** a trigger/behaviour test (`test-harness`)
|
|
217
|
+
- _"harden my rules"_ → upgrades prose guidance into enforced linter rules (`strengthen`)
|
|
218
|
+
- _"add a rule to my CLAUDE.md"_ → edits the source and recompiles (`edit-spec`)
|
|
280
219
|
|
|
281
|
-
|
|
282
|
-
|
|
220
|
+
<details>
|
|
221
|
+
<summary>What <code>init</code> sets up</summary>
|
|
283
222
|
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
the
|
|
223
|
+
- **Both lint and test** by default; scope with `--lint` / `--test`.
|
|
224
|
+
- **Already have a CLAUDE.md / AGENTS.md, skills, or subagents? `init` adopts them all** into specs faithfully and **non-destructively** — untouched until you `compile` (and `eject` undoes it).
|
|
225
|
+
- Adds `vigiles` to `devDependencies`; installs the Claude Code plugin (skills + hooks) via the marketplace — globally, never vendored.
|
|
226
|
+
- Wires CI as a `zernie/vigiles@v1` workflow that posts a sticky PR comment + a `valid` output.
|
|
287
227
|
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
(
|
|
291
|
-
default, not an unbypassable wall._
|
|
292
|
-
**[Compiled hooks — bug classes + trade-offs →](docs/compiled-hooks.md)**
|
|
228
|
+
Works with **Claude Code and Codex** ([`vigiles/codex`](docs/harnesses.md)) or
|
|
229
|
+
[your own harness](docs/authoring-an-adapter.md). Prefer to write tests yourself?
|
|
230
|
+
JS **or** TS (`*.harness.{mjs,ts}`) — run with `npx vigiles test`.
|
|
293
231
|
|
|
294
|
-
|
|
232
|
+
</details>
|
|
295
233
|
|
|
296
234
|
## FAQ
|
|
297
235
|
|
|
298
236
|
- **Isn't this just a markdown linter?** No — it checks whether your instruction file is _true_ (every path/script/symbol/rule exists and is enabled), then tests and measures your harness. A style linter can't do any of that.
|
|
299
|
-
- **Do I have to write TypeScript?** No — your agent writes the spec (`init` adopts your CLAUDE.md into one). Prefer zero new files? Plain markdown lints too.
|
|
237
|
+
- **Do I have to write TypeScript?** No — your agent writes the spec (`init` adopts your CLAUDE.md into one). Prefer zero new files? Plain markdown lints too. Deeper compiler-grade guarantees are gradual and opt-in, like TS's `strict`.
|
|
300
238
|
- **Does it overwrite my files?** No. `init` adopts an existing CLAUDE.md _non-destructively_ — untouched until you `compile`, and `eject` reverses it.
|
|
301
239
|
- **Need an API key?** No for almost everything (free, every commit). Real-model evals run on your Claude Pro/Max subscription — $0 metered tokens.
|
|
302
240
|
- **Non-JS repo?** `npx vigiles lint` verifies your CLAUDE.md with no install (Ruff/Clippy/Pylint/… too).
|
|
@@ -305,11 +243,11 @@ default, not an unbypassable wall._
|
|
|
305
243
|
|
|
306
244
|
## More
|
|
307
245
|
|
|
308
|
-
- **[CLI →](docs/cli.md)**
|
|
246
|
+
- **[CLI →](docs/cli.md)** · **[GitHub Action →](docs/github-action.md)** · the full **[lint rules matrix →](docs/verifying-instruction-files.md#the-validation-rules--the-full-matrix)** lives with the linting guide.
|
|
309
247
|
- **[Skills →](docs/skills.md)** — the skills `init` installs, and how the model-invocable ones trigger.
|
|
310
|
-
- **[Ship plugins? The plugin-author guide →](docs/for-plugin-authors.md)** — scan a draft
|
|
311
|
-
- **[Docs index →](docs/README.md)** · **[API reference →](https://zernie.github.io/vigiles/)** · **[Related tools →](docs/related-tools.md)
|
|
312
|
-
- **[Stability →](STABILITY.md)** — 0.x: the CLI is stable; the library API is still evolving
|
|
248
|
+
- **[Ship plugins? The plugin-author guide →](docs/for-plugin-authors.md)** — scan a draft, make your skills fire, rank a whole marketplace — no key.
|
|
249
|
+
- **[Docs index →](docs/README.md)** · **[API reference →](https://zernie.github.io/vigiles/)** · **[Related tools →](docs/related-tools.md)**.
|
|
250
|
+
- **[Stability →](STABILITY.md)** — 0.x: the CLI is stable; the library API is still evolving.
|
|
313
251
|
- **Not for you if** you want a model/capability benchmark or runtime guardrails in the request path — vigiles is build-/CI-time.
|
|
314
252
|
- Companion to [Feedback Loop Is All You Need](https://zernie.com/blog/feedback-loop-is-all-you-need).
|
|
315
253
|
|
package/dist/audit-report.d.ts
CHANGED
|
@@ -39,6 +39,29 @@ export interface AuditInventory {
|
|
|
39
39
|
readonly mcp: boolean;
|
|
40
40
|
readonly untested: number;
|
|
41
41
|
}
|
|
42
|
+
/**
|
|
43
|
+
* A surface (skill / subagent / instruction file) that EXISTS but doesn't yet
|
|
44
|
+
* have a `.spec.ts` — so it can be adopted into a typed spec. The report can't
|
|
45
|
+
* write files (it's a browser app), so it EMITS the exact CLI command instead.
|
|
46
|
+
*/
|
|
47
|
+
export interface AdoptableSurface {
|
|
48
|
+
/** The repo-relative path of the surface (e.g. `skills/foo/SKILL.md`). */
|
|
49
|
+
readonly path: string;
|
|
50
|
+
/** The exact command that adopts this one surface. */
|
|
51
|
+
readonly command: string;
|
|
52
|
+
}
|
|
53
|
+
/**
|
|
54
|
+
* The adoptable-surfaces list + the "create all" command — the data the report's
|
|
55
|
+
* "Create spec" / "Create all specs" affordances copy to the clipboard. Present
|
|
56
|
+
* only when there's at least one un-spec'd surface; the CLI computes the surface
|
|
57
|
+
* paths (the layout-aware `discoverAdoptableSurfaces`) and passes them in, so the
|
|
58
|
+
* pure builder stays adapter-agnostic.
|
|
59
|
+
*/
|
|
60
|
+
export interface Adoptable {
|
|
61
|
+
readonly surfaces: readonly AdoptableSurface[];
|
|
62
|
+
/** The one command that adopts every surface at once. */
|
|
63
|
+
readonly createAllCommand: string;
|
|
64
|
+
}
|
|
42
65
|
/**
|
|
43
66
|
* The full audit, as the dashboard / CI / HTML all consume it. Self-describing
|
|
44
67
|
* and versioned; additive-only within a `schemaVersion`.
|
|
@@ -56,10 +79,24 @@ export interface AuditReport {
|
|
|
56
79
|
* Additive/optional, so the schema version is unchanged.
|
|
57
80
|
*/
|
|
58
81
|
readonly adoptability?: AdoptabilityResult;
|
|
82
|
+
/**
|
|
83
|
+
* The surfaces that exist but aren't spec-managed yet, each with the command
|
|
84
|
+
* that adopts it, plus a "create all" command. Drives the report's "Create
|
|
85
|
+
* spec" / "Create all specs" command-emit buttons. Present only when there's
|
|
86
|
+
* at least one adoptable surface. Additive/optional — schema version unchanged.
|
|
87
|
+
*/
|
|
88
|
+
readonly adoptable?: Adoptable;
|
|
59
89
|
}
|
|
60
90
|
export interface BuildAuditReportOptions {
|
|
61
91
|
readonly harness: string;
|
|
62
92
|
readonly vigilesVersion: string;
|
|
93
|
+
/**
|
|
94
|
+
* The repo-relative paths of surfaces that exist but have no `.spec.ts` yet,
|
|
95
|
+
* computed by the CLI's layout-aware `discoverAdoptableSurfaces` (so the pure
|
|
96
|
+
* builder stays adapter-agnostic — it only formats the commands). Omit/empty
|
|
97
|
+
* when there's nothing to adopt.
|
|
98
|
+
*/
|
|
99
|
+
readonly adoptableSurfaces?: readonly string[];
|
|
63
100
|
}
|
|
64
101
|
/**
|
|
65
102
|
* Assemble the versioned {@link AuditReport} from a scan report — pure, no clock.
|
package/dist/audit-report.js
CHANGED
|
@@ -19,12 +19,32 @@ const audit_score_js_1 = require("./audit-score.js");
|
|
|
19
19
|
const optimize_js_1 = require("./optimize.js");
|
|
20
20
|
/** The current schema version. Bump only on a BREAKING change to the shape. */
|
|
21
21
|
exports.AUDIT_SCHEMA_VERSION = 1;
|
|
22
|
+
/** The one command that adopts every un-spec'd surface (bare `init`). */
|
|
23
|
+
const CREATE_ALL_COMMAND = "npx vigiles init";
|
|
24
|
+
/** The command that adopts ONE surface at a given repo-relative path. */
|
|
25
|
+
function adoptCommand(path) {
|
|
26
|
+
return `npx vigiles init --target=${path}`;
|
|
27
|
+
}
|
|
28
|
+
/**
|
|
29
|
+
* Build the {@link Adoptable} payload from the layout-aware surface paths — pure,
|
|
30
|
+
* just formats the per-surface + create-all commands. Returns `undefined` when
|
|
31
|
+
* there's nothing to adopt (so the field stays absent).
|
|
32
|
+
*/
|
|
33
|
+
function buildAdoptable(surfaces) {
|
|
34
|
+
if (!surfaces || surfaces.length === 0)
|
|
35
|
+
return undefined;
|
|
36
|
+
return {
|
|
37
|
+
surfaces: surfaces.map((path) => ({ path, command: adoptCommand(path) })),
|
|
38
|
+
createAllCommand: CREATE_ALL_COMMAND,
|
|
39
|
+
};
|
|
40
|
+
}
|
|
22
41
|
/**
|
|
23
42
|
* Assemble the versioned {@link AuditReport} from a scan report — pure, no clock.
|
|
24
43
|
* The CLI attaches `meta.generatedAt` when it writes the JSON artifact; the
|
|
25
44
|
* HTML-embedded form omits it so the rendered file stays deterministic.
|
|
26
45
|
*/
|
|
27
46
|
function buildAuditReport(report, opts) {
|
|
47
|
+
const adoptable = buildAdoptable(opts.adoptableSurfaces);
|
|
28
48
|
return {
|
|
29
49
|
meta: {
|
|
30
50
|
schemaVersion: exports.AUDIT_SCHEMA_VERSION,
|
|
@@ -46,6 +66,7 @@ function buildAuditReport(report, opts) {
|
|
|
46
66
|
mcp: report.mcp,
|
|
47
67
|
untested: report.untested,
|
|
48
68
|
},
|
|
69
|
+
...(adoptable ? { adoptable } : {}),
|
|
49
70
|
};
|
|
50
71
|
}
|
|
51
72
|
//# sourceMappingURL=audit-report.js.map
|