@descent-vtt/spec-guard 0.5.1 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +444 -283
- package/README.md +1094 -868
- package/dist/cli.d.ts +18 -2
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +136 -7
- package/dist/cli.js.map +1 -1
- package/dist/glob.d.ts +17 -2
- package/dist/glob.d.ts.map +1 -1
- package/dist/glob.js +25 -9
- package/dist/glob.js.map +1 -1
- package/dist/graph.d.ts +124 -0
- package/dist/graph.d.ts.map +1 -0
- package/dist/graph.js +241 -0
- package/dist/graph.js.map +1 -0
- package/dist/index.d.ts +19 -7
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +9 -3
- package/dist/index.js.map +1 -1
- package/dist/layers.d.ts +72 -0
- package/dist/layers.d.ts.map +1 -0
- package/dist/layers.js +102 -0
- package/dist/layers.js.map +1 -0
- package/dist/mcp.d.ts +117 -0
- package/dist/mcp.d.ts.map +1 -0
- package/dist/mcp.js +544 -0
- package/dist/mcp.js.map +1 -0
- package/dist/parser.d.ts +50 -1
- package/dist/parser.d.ts.map +1 -1
- package/dist/parser.js +232 -16
- package/dist/parser.js.map +1 -1
- package/dist/query.d.ts +108 -0
- package/dist/query.d.ts.map +1 -0
- package/dist/query.js +206 -0
- package/dist/query.js.map +1 -0
- package/dist/reporter.d.ts.map +1 -1
- package/dist/reporter.js +62 -5
- package/dist/reporter.js.map +1 -1
- package/dist/rules.d.ts +108 -0
- package/dist/rules.d.ts.map +1 -0
- package/dist/rules.js +153 -0
- package/dist/rules.js.map +1 -0
- package/dist/runner.d.ts +24 -0
- package/dist/runner.d.ts.map +1 -1
- package/dist/runner.js +335 -117
- package/dist/runner.js.map +1 -1
- package/dist/specs.d.ts +44 -0
- package/dist/specs.d.ts.map +1 -0
- package/dist/specs.js +62 -0
- package/dist/specs.js.map +1 -0
- package/dist/types.d.ts +60 -2
- package/dist/types.d.ts.map +1 -1
- package/package.json +73 -73
package/README.md
CHANGED
|
@@ -1,868 +1,1094 @@
|
|
|
1
|
-
# spec-guard
|
|
2
|
-
|
|
3
|
-
**Executable architecture assertions for Markdown specs & ADRs.**
|
|
4
|
-
|
|
5
|
-
Turn the claims your design documents make about your codebase into assertions
|
|
6
|
-
that run in CI, in milliseconds, with no runtime dependencies.
|
|
7
|
-
|
|
8
|
-
```md
|
|
9
|
-
The retired payment gateway is gone from the service layer.
|
|
10
|
-
|
|
11
|
-
<!-- @assert-absence target="src/services" symbol="LegacyPaymentGateway" -->
|
|
12
|
-
```
|
|
13
|
-
|
|
14
|
-
If someone reintroduces `LegacyPaymentGateway`, CI fails and points at the line
|
|
15
|
-
of the ADR that promised it was gone.
|
|
16
|
-
|
|
17
|
-
---
|
|
18
|
-
|
|
19
|
-
## Why this exists
|
|
20
|
-
|
|
21
|
-
Architecture Decision Records, RFCs and technical briefs are full of hard claims
|
|
22
|
-
about a repository:
|
|
23
|
-
|
|
24
|
-
- *"`LegacyPaymentGateway` no longer exists anywhere in the service layer."*
|
|
25
|
-
- *"There is exactly one `UserSessionManager`."*
|
|
26
|
-
- *"No secret is ever read from `process.env` outside `src/config`."*
|
|
27
|
-
|
|
28
|
-
Every one of those claims is true on the day it is written and unverified from
|
|
29
|
-
the day after. Prose does not fail a build. Six months later the document is
|
|
30
|
-
still confidently asserting something the code stopped doing in March.
|
|
31
|
-
|
|
32
|
-
That has always been an annoyance. It became a defect multiplier the moment
|
|
33
|
-
coding agents started reading these documents as ground truth. An agent that
|
|
34
|
-
reads *"module X does not exist yet"* will happily build a second X. A reviewer
|
|
35
|
-
who trusts *"this symbol occurs zero times"* skips the grep. Stale documentation
|
|
36
|
-
is now executable in the worst sense: it executes inside someone's head, or
|
|
37
|
-
inside a model's context window, and produces work that has to be thrown away.
|
|
38
|
-
|
|
39
|
-
spec-guard makes the claims executable in the good sense. It reads directives
|
|
40
|
-
written as ordinary HTML comments - invisible in every Markdown renderer - and
|
|
41
|
-
checks them against the real tree with ripgrep. A stale claim becomes a failing
|
|
42
|
-
build with a line number, not a landmine.
|
|
43
|
-
|
|
44
|
-
## Quickstart
|
|
45
|
-
|
|
46
|
-
```bash
|
|
47
|
-
npm install --save-dev @descent-vtt/spec-guard
|
|
48
|
-
npx spec-guard "docs/**/*.md"
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
**The package is scoped; the command is not.** Once installed, the binary is
|
|
52
|
-
plain `spec-guard`, so `npx spec-guard`, `npm scripts` and a global install all
|
|
53
|
-
use that name:
|
|
54
|
-
|
|
55
|
-
```bash
|
|
56
|
-
npm install -g @descent-vtt/spec-guard # then: spec-guard "docs/**/*.md"
|
|
57
|
-
```
|
|
58
|
-
|
|
59
|
-
Without a local install, `npx` needs the full package name - `npx spec-guard`
|
|
60
|
-
on its own would resolve to a different package on the registry:
|
|
61
|
-
|
|
62
|
-
```bash
|
|
63
|
-
npx @descent-vtt/spec-guard "docs/**/*.md"
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
Add a directive to any Markdown file, directly under the sentence it makes
|
|
67
|
-
executable:
|
|
68
|
-
|
|
69
|
-
```md
|
|
70
|
-
## Decision
|
|
71
|
-
|
|
72
|
-
Session state has exactly one owner.
|
|
73
|
-
|
|
74
|
-
<!-- @assert-count target="src/" symbol="UserSessionManager" expected="1" -->
|
|
75
|
-
```
|
|
76
|
-
|
|
77
|
-
Run it:
|
|
78
|
-
|
|
79
|
-
```bash
|
|
80
|
-
npx spec-guard "docs/**/*.md" --verbose
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
```text
|
|
84
|
-
spec-guard 1 spec · 1 assertion · ripgrep
|
|
85
|
-
|
|
86
|
-
✔ docs/adr/0007-sessions.md:5 @assert-count "UserSessionManager" (1 match) in src
|
|
87
|
-
|
|
88
|
-
1 passed · 7ms
|
|
89
|
-
✔ every spec assertion holds
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
And when it drifts:
|
|
93
|
-
|
|
94
|
-
```text
|
|
95
|
-
✖ docs/adr/0007-sessions.md:5 @assert-count
|
|
96
|
-
"UserSessionManager" must appear exactly 1 time in src
|
|
97
|
-
expected exactly 1 match, found 3
|
|
98
|
-
src/auth/UserSessionManager.ts:12:14 export class UserSessionManager {
|
|
99
|
-
src/legacy/sessions.ts:4:22 import { UserSessionManager } from '../auth';
|
|
100
|
-
src/workers/refresh.ts:9:10 const manager = new UserSessionManager();
|
|
101
|
-
|
|
102
|
-
0 passed · 1 failed · 9ms
|
|
103
|
-
```
|
|
104
|
-
|
|
105
|
-
Exit code 1. The ADR is now a test.
|
|
106
|
-
|
|
107
|
-
## Directives
|
|
108
|
-
|
|
109
|
-
Every directive is an HTML comment. It may span multiple lines. Attribute values
|
|
110
|
-
may use double or single quotes, and a bare attribute means `="true"`.
|
|
111
|
-
|
|
112
|
-
### `@assert-absence` - this symbol is gone
|
|
113
|
-
|
|
114
|
-
```md
|
|
115
|
-
<!-- @assert-absence target="src/controllers,src/services" symbol="LegacyPaymentGateway" -->
|
|
116
|
-
<!-- @assert-absence target="src/" symbol="STRIPE_SECRET_KEY" expected="0" -->
|
|
117
|
-
<!-- @assert-absence target="src/" symbol="TODO" expected="5" reason="burn down the backlog" -->
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
Fails when the symbol occurs more than `expected` times (default `0`).
|
|
121
|
-
`max="..."` is accepted as a synonym for `expected`.
|
|
122
|
-
|
|
123
|
-
### `exclude` - everywhere except
|
|
124
|
-
|
|
125
|
-
Most real rules are "nowhere except one place", not "not here":
|
|
126
|
-
|
|
127
|
-
```md
|
|
128
|
-
<!-- @assert-absence target="src" symbol="process.env" exclude="src/config/**" -->
|
|
129
|
-
```
|
|
130
|
-
|
|
131
|
-
That is the rule this README opens with - *no secret is ever read from
|
|
132
|
-
`process.env` outside `src/config`* - written as one assertion instead of a
|
|
133
|
-
hand-maintained list of every directory that is not `src/config`.
|
|
134
|
-
|
|
135
|
-
`exclude` follows gitignore rules, which are **not** the same as `glob`'s:
|
|
136
|
-
|
|
137
|
-
| Pattern | Excludes |
|
|
138
|
-
| --- | --- |
|
|
139
|
-
| `tests` | any directory or file named `tests`, at any depth, and everything under it |
|
|
140
|
-
| `src/config` | that directory and everything under it |
|
|
141
|
-
| `src/config/**` | the same, written explicitly |
|
|
142
|
-
| `*.test.ts` | any file with that name shape, at any depth |
|
|
143
|
-
|
|
144
|
-
The difference is deliberate. `glob="*.ts"` filters files, so basename matching
|
|
145
|
-
is what you want; `exclude="tests"` means the directory, because that is what
|
|
146
|
-
people mean when they write it - and it is what `rg -g '!tests'` does. Both
|
|
147
|
-
engines implement the same rule, and a parity matrix asserts they agree on it.
|
|
148
|
-
|
|
149
|
-
List attributes accept commas or whitespace, so both of these work:
|
|
150
|
-
|
|
151
|
-
```md
|
|
152
|
-
<!-- @assert-absence target="src" symbol="TODO" exclude="src/legacy/** tests" -->
|
|
153
|
-
<!-- @assert-absence target="src" symbol="TODO" exclude="src/legacy/**,tests" -->
|
|
154
|
-
```
|
|
155
|
-
|
|
156
|
-
(A path containing a space therefore cannot be written; there is no quoting
|
|
157
|
-
inside an attribute value.)
|
|
158
|
-
|
|
159
|
-
### `comments` - the note about a deletion is not the deletion
|
|
160
|
-
|
|
161
|
-
You delete a symbol and leave the explanation where the next person will look:
|
|
162
|
-
|
|
163
|
-
```ts
|
|
164
|
-
// LegacyThing was removed in ADR-398; do not reintroduce it.
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
A plain text search reads that comment as an occurrence, so the assertion that
|
|
168
|
-
keeps the symbol deleted fails on the sentence proving it was deleted. Both
|
|
169
|
-
fixes are bad: delete the note and lose the reason, or delete the rule.
|
|
170
|
-
|
|
171
|
-
So matches inside comments do not count:
|
|
172
|
-
|
|
173
|
-
```md
|
|
174
|
-
<!-- comments are ignored by default -->
|
|
175
|
-
<!-- @assert-absence target="src" symbol="LegacyThing" -->
|
|
176
|
-
|
|
177
|
-
<!-- ...unless you ask for them -->
|
|
178
|
-
<!-- @assert-absence target="src" symbol="Copyright" comments="include" -->
|
|
179
|
-
```
|
|
180
|
-
|
|
181
|
-
`comments="include"` is right for assertions that really are about text — a
|
|
182
|
-
licence header, or a name that must appear nowhere in the repository at all.
|
|
183
|
-
|
|
184
|
-
Because this is the one thing that can turn a failing run green without anyone
|
|
185
|
-
touching code, a run that passed this way says so:
|
|
186
|
-
|
|
187
|
-
```
|
|
188
|
-
⚠ 1 match inside comments was not counted; add comments="include" to count it
|
|
189
|
-
|
|
190
|
-
1 passed · 7ms
|
|
191
|
-
✔ every spec assertion holds
|
|
192
|
-
```
|
|
193
|
-
|
|
194
|
-
Comment syntax is known for around 59 extensions across 9 families (JS/TS, C,
|
|
195
|
-
C#, Rust, Go, Python-style `#`, SQL-style `--`, markup, and formats with no
|
|
196
|
-
comments at all). Strings are tracked too, because `//` inside a URL is not a
|
|
197
|
-
comment and reading it as one would hide real code. Where spec-guard is unsure —
|
|
198
|
-
an unknown extension, an unterminated literal — the text counts as code, and the
|
|
199
|
-
report says which files it could not classify. A match wrongly kept is a visible
|
|
200
|
-
failure you can argue with; a match wrongly dropped is a lie.
|
|
201
|
-
|
|
202
|
-
## What gets searched
|
|
203
|
-
|
|
204
|
-
An assertion is worth exactly as much as the set of files behind it, so
|
|
205
|
-
spec-guard is explicit about that set and never quietly narrows it.
|
|
206
|
-
|
|
207
|
-
**Four directory names are skipped**, and nothing else:
|
|
208
|
-
|
|
209
|
-
| Skipped | Why |
|
|
210
|
-
| --- | --- |
|
|
211
|
-
| `.git`, `.hg`, `.svn` | version-control stores hold compressed copies of code you deleted on purpose |
|
|
212
|
-
| `node_modules` | code you did not write, which your architecture rules are not about |
|
|
213
|
-
|
|
214
|
-
Everything else is searched. That includes **hidden directories** - `.github`,
|
|
215
|
-
`.husky`, `.claude-rules`, `.agents` - because that is where CI, hooks and agent
|
|
216
|
-
rules live, and a rule that cannot see your workflow files is not enforcing much.
|
|
217
|
-
It also includes `dist`, `build`, `out` and `coverage`, because spec-guard
|
|
218
|
-
cannot tell build output from a directory of build scripts, and guessing wrong
|
|
219
|
-
means a rule silently stops covering anything.
|
|
220
|
-
|
|
221
|
-
`.gitignore` is not consulted. It describes what git should carry, not what a
|
|
222
|
-
rule covers - and ripgrep applies it only inside a git repository, so honouring
|
|
223
|
-
it made the same tree answer differently depending on whether a `.git` directory
|
|
224
|
-
happened to exist above it.
|
|
225
|
-
|
|
226
|
-
To narrow scope, say so in the assertion:
|
|
227
|
-
|
|
228
|
-
```md
|
|
229
|
-
<!-- @assert-absence target="src" symbol="TODO" exclude="dist coverage" -->
|
|
230
|
-
```
|
|
231
|
-
|
|
232
|
-
`--no-default-skips` removes even those four, for a run that has to be certain.
|
|
233
|
-
|
|
234
|
-
### Nothing is skipped quietly
|
|
235
|
-
|
|
236
|
-
A file spec-guard could not read, or one whose bytes are not text but which
|
|
237
|
-
contained the symbol anyway, is a gap in the answer rather than a detail of it.
|
|
238
|
-
Those are reported, and `--strict` fails on them:
|
|
239
|
-
|
|
240
|
-
```text
|
|
241
|
-
✖ docs/adr.md:3 @assert-absence
|
|
242
|
-
"ApiKey" must not appear in .
|
|
243
|
-
expected no matches, found 0, and 1 file could not be inspected
|
|
244
|
-
⚠ 1 match in 1 binary file not counted: build/app.bin
|
|
245
|
-
```
|
|
246
|
-
|
|
247
|
-
The counts are in `--json` too, as `skipped` on each result.
|
|
248
|
-
|
|
249
|
-
### `@assert-count` - this symbol occurs exactly / at least / at most N times
|
|
250
|
-
|
|
251
|
-
```md
|
|
252
|
-
<!-- @assert-count target="src/" symbol="UserSessionManager" expected="1" -->
|
|
253
|
-
<!-- @assert-count target="src/ui/" symbol="PrimaryButton" min="1" -->
|
|
254
|
-
<!-- @assert-count target="src/core/" symbol="DeprecatedHelper" max="3" -->
|
|
255
|
-
<!-- @assert-count target="src/" symbol="Repository" min="2" max="10" -->
|
|
256
|
-
```
|
|
257
|
-
|
|
258
|
-
Requires `expected`, or `min` and/or `max`. `expected` cannot be combined with
|
|
259
|
-
`min`/`max`.
|
|
260
|
-
|
|
261
|
-
An exact count is the most brittle assertion in the set: a test that merely
|
|
262
|
-
mentions the symbol will break it. spec-guard does not silently skip test files
|
|
263
|
-
for you - an assertion that quietly ignores part of the tree is the failure mode
|
|
264
|
-
this tool exists to prevent - so say what you mean:
|
|
265
|
-
|
|
266
|
-
```md
|
|
267
|
-
<!-- @assert-count target="src" symbol="UserSessionManager" expected="1" exclude="*.test.ts *.spec.ts" -->
|
|
268
|
-
```
|
|
269
|
-
|
|
270
|
-
`min="1"` is often the better rule anyway: it says "this exists" without
|
|
271
|
-
breaking every time someone writes a second test.
|
|
272
|
-
|
|
273
|
-
### `@assert-import-absence` / `@assert-import-count` - dependencies, not text
|
|
274
|
-
|
|
275
|
-
The rules architecture documents actually contain are usually about
|
|
276
|
-
dependencies, and a text search answers those badly in both directions - it
|
|
277
|
-
matches a mention in a comment, and it misses `export * from '../db'`.
|
|
278
|
-
|
|
279
|
-
```md
|
|
280
|
-
<!-- @assert-import-absence target="src/ui" module="src/db" reason="the UI talks to services, not storage" -->
|
|
281
|
-
<!-- @assert-import-count target="src" module="axios" max="1" -->
|
|
282
|
-
```
|
|
283
|
-
|
|
284
|
-
These read the dependency rather than the text, in **five languages**:
|
|
285
|
-
|
|
286
|
-
| Language | Extensions | Reads |
|
|
287
|
-
| --- | --- | --- |
|
|
288
|
-
| JavaScript / TypeScript | `.js .mjs .cjs .jsx .ts .mts .cts .tsx` | `import`, `export ... from`, `require()`, `import()` |
|
|
289
|
-
| Python | `.py .pyi` | `import a.b`, `from .rel import x`, `importlib.import_module("x")` |
|
|
290
|
-
| Go | `.go` | `import "x"` and grouped `import ( ... )`, including aliases and `_` |
|
|
291
|
-
| Rust | `.rs` | `use a::{b, c}` with nested groups, `pub use`, `extern crate` |
|
|
292
|
-
| C# | `.cs .csx` | `using`, `using static`, `global using`, `using X = A.B` |
|
|
293
|
-
|
|
294
|
-
No parser and no new dependency. JavaScript gets a full tokenizer because a
|
|
295
|
-
module reference can appear anywhere in an expression; the other four are read
|
|
296
|
-
by masking comments and string literals with the classifier from
|
|
297
|
-
[ADR-0006](docs/adr/0006-comment-classification.md) and then reading statements
|
|
298
|
-
off what is left. Tree-sitter would have cost 94 MB unpacked against this
|
|
299
|
-
package's 0.33 MB - [ADR-0008](docs/adr/0008-polyglot-imports.md) has the
|
|
300
|
-
measurements, and the list of things a real parser would genuinely see that
|
|
301
|
-
this does not.
|
|
302
|
-
|
|
303
|
-
Module patterns are matched against the resolved `/`-separated path *and*
|
|
304
|
-
against the specifier as written, so `module="System.Text.Json"` and
|
|
305
|
-
`module="System/Text/Json"` both work. Matching is case-sensitive, so a rule
|
|
306
|
-
spanning C# and Go needs both conventions:
|
|
307
|
-
`module="app/db/** App/Db/**"`.
|
|
308
|
-
|
|
309
|
-
The unit counted is **files** that depend on the module. For JavaScript and
|
|
310
|
-
TypeScript it understands:
|
|
311
|
-
|
|
312
|
-
| Form | Counted |
|
|
313
|
-
| --- | --- |
|
|
314
|
-
| `import { A } from 'x'`, `import 'x'`, `import * as ns from 'x'` | yes |
|
|
315
|
-
| `import type { A } from 'x'` | yes, unless `types="ignore"` |
|
|
316
|
-
| `export * from 'x'`, `export { A } from 'x'` | yes - a re-export is a dependency, and a barrel file is how layering rules usually get broken |
|
|
317
|
-
| `require('x')` | yes, with a literal argument |
|
|
318
|
-
| `import('x')` | yes, with a literal argument |
|
|
319
|
-
| `import(name)`, `require(expr)` | **no - reported, see below** |
|
|
320
|
-
|
|
321
|
-
Relative specifiers are resolved by path arithmetic, so `module="src/db"`
|
|
322
|
-
matches `../db/client.js` seen from `src/ui/`, and Python's `from ..core import
|
|
323
|
-
x` resolves against the importing file the same way. There is deliberately no
|
|
324
|
-
filesystem resolution: tsconfig `paths` aliases and package `exports` maps are
|
|
325
|
-
not followed, so an alias is written out as itself (`module="@app/db"`).
|
|
326
|
-
|
|
327
|
-
**It tells you what it could not see.** Roughly 1% of module references in real
|
|
328
|
-
code are dynamic, and about 0.09% of files defeat the tokenizer outright. Both
|
|
329
|
-
are reported rather than counted as clean:
|
|
330
|
-
|
|
331
|
-
```text
|
|
332
|
-
✔ docs/adr.md:1 @assert-import-absence "src/db" (0 matches) in src/core
|
|
333
|
-
⚠ 1 module reference could not be resolved statically
|
|
334
|
-
⚠ src/core/plugin.ts:1 import(name)
|
|
335
|
-
```
|
|
336
|
-
|
|
337
|
-
The count is still true of everything that could be seen; the warning is what
|
|
338
|
-
stops it being mistaken for a complete answer. `--strict` turns those warnings
|
|
339
|
-
into failures. Files in scope that are not JavaScript or TypeScript are counted
|
|
340
|
-
and reported too, so a rule pointed at the wrong tree says "analysed 2 of 3
|
|
341
|
-
files" rather than quietly passing - and if *none* of them can be read, the
|
|
342
|
-
assertion fails rather than passing on an empty analysis.
|
|
343
|
-
|
|
344
|
-
[ADR-0005](docs/adr/0005-import-assertions.md) has the measurements and the
|
|
345
|
-
reasoning behind each boundary.
|
|
346
|
-
|
|
347
|
-
### `@assert-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
a
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
```
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
|
451
|
-
|
|
|
452
|
-
|
|
|
453
|
-
|
|
|
454
|
-
|
|
|
455
|
-
|
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
on
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
```
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
|
|
536
|
-
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
##
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
```
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
|
|
559
|
-
|
|
560
|
-
|
|
561
|
-
|
|
562
|
-
|
|
563
|
-
|
|
564
|
-
|
|
565
|
-
|
|
566
|
-
|
|
567
|
-
|
|
568
|
-
|
|
569
|
-
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
|
|
593
|
-
|
|
594
|
-
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
|
|
625
|
-
|
|
626
|
-
|
|
627
|
-
|
|
628
|
-
|
|
629
|
-
|
|
630
|
-
|
|
631
|
-
|
|
632
|
-
|
|
633
|
-
|
|
634
|
-
|
|
635
|
-
|
|
636
|
-
|
|
637
|
-
|
|
638
|
-
|
|
639
|
-
|
|
640
|
-
|
|
641
|
-
|
|
642
|
-
|
|
643
|
-
|
|
644
|
-
|
|
645
|
-
|
|
646
|
-
|
|
647
|
-
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
|
|
653
|
-
|
|
654
|
-
|
|
655
|
-
|
|
656
|
-
|
|
657
|
-
|
|
658
|
-
|
|
659
|
-
|
|
660
|
-
|
|
661
|
-
|
|
662
|
-
|
|
663
|
-
|
|
664
|
-
|
|
665
|
-
|
|
666
|
-
|
|
667
|
-
|
|
668
|
-
|
|
669
|
-
|
|
670
|
-
|
|
671
|
-
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
695
|
-
|
|
696
|
-
|
|
697
|
-
|
|
698
|
-
|
|
699
|
-
|
|
700
|
-
|
|
701
|
-
|
|
702
|
-
|
|
703
|
-
|
|
704
|
-
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
|
|
709
|
-
|
|
710
|
-
|
|
711
|
-
|
|
712
|
-
|
|
713
|
-
|
|
714
|
-
|
|
715
|
-
|
|
716
|
-
|
|
717
|
-
|
|
718
|
-
|
|
719
|
-
|
|
720
|
-
|
|
721
|
-
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
|
|
725
|
-
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
|
|
729
|
-
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
|
|
737
|
-
|
|
738
|
-
|
|
739
|
-
|
|
740
|
-
|
|
741
|
-
|
|
742
|
-
|
|
743
|
-
|
|
744
|
-
|
|
745
|
-
|
|
746
|
-
|
|
747
|
-
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
|
|
760
|
-
|
|
761
|
-
|
|
762
|
-
|
|
763
|
-
|
|
764
|
-
|
|
765
|
-
|
|
766
|
-
|
|
767
|
-
|
|
768
|
-
|
|
769
|
-
|
|
770
|
-
|
|
771
|
-
|
|
772
|
-
|
|
773
|
-
|
|
774
|
-
|
|
775
|
-
|
|
776
|
-
|
|
777
|
-
|
|
778
|
-
|
|
779
|
-
|
|
780
|
-
|
|
781
|
-
|
|
782
|
-
|
|
783
|
-
|
|
784
|
-
|
|
785
|
-
|
|
786
|
-
|
|
787
|
-
|
|
788
|
-
|
|
789
|
-
|
|
790
|
-
|
|
791
|
-
|
|
792
|
-
|
|
793
|
-
|
|
794
|
-
|
|
795
|
-
|
|
796
|
-
|
|
797
|
-
|
|
798
|
-
|
|
799
|
-
|
|
800
|
-
|
|
801
|
-
|
|
802
|
-
|
|
803
|
-
|
|
804
|
-
|
|
805
|
-
|
|
806
|
-
|
|
807
|
-
|
|
808
|
-
|
|
809
|
-
|
|
810
|
-
|
|
811
|
-
|
|
812
|
-
|
|
813
|
-
|
|
814
|
-
|
|
815
|
-
|
|
816
|
-
|
|
817
|
-
|
|
818
|
-
|
|
819
|
-
|
|
820
|
-
|
|
821
|
-
|
|
822
|
-
|
|
823
|
-
|
|
824
|
-
|
|
825
|
-
|
|
826
|
-
|
|
827
|
-
|
|
828
|
-
|
|
829
|
-
|
|
830
|
-
|
|
831
|
-
|
|
832
|
-
|
|
833
|
-
|
|
834
|
-
|
|
835
|
-
|
|
836
|
-
|
|
837
|
-
|
|
838
|
-
|
|
839
|
-
|
|
840
|
-
|
|
841
|
-
|
|
842
|
-
|
|
843
|
-
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
|
|
847
|
-
|
|
848
|
-
|
|
849
|
-
|
|
850
|
-
|
|
851
|
-
|
|
852
|
-
|
|
853
|
-
|
|
854
|
-
|
|
855
|
-
|
|
856
|
-
|
|
857
|
-
|
|
858
|
-
the
|
|
859
|
-
|
|
860
|
-
|
|
861
|
-
|
|
862
|
-
|
|
863
|
-
|
|
864
|
-
|
|
865
|
-
|
|
866
|
-
|
|
867
|
-
|
|
868
|
-
|
|
1
|
+
# spec-guard
|
|
2
|
+
|
|
3
|
+
**Executable architecture assertions for Markdown specs & ADRs.**
|
|
4
|
+
|
|
5
|
+
Turn the claims your design documents make about your codebase into assertions
|
|
6
|
+
that run in CI, in milliseconds, with no runtime dependencies.
|
|
7
|
+
|
|
8
|
+
```md
|
|
9
|
+
The retired payment gateway is gone from the service layer.
|
|
10
|
+
|
|
11
|
+
<!-- @assert-absence target="src/services" symbol="LegacyPaymentGateway" -->
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
If someone reintroduces `LegacyPaymentGateway`, CI fails and points at the line
|
|
15
|
+
of the ADR that promised it was gone.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## Why this exists
|
|
20
|
+
|
|
21
|
+
Architecture Decision Records, RFCs and technical briefs are full of hard claims
|
|
22
|
+
about a repository:
|
|
23
|
+
|
|
24
|
+
- *"`LegacyPaymentGateway` no longer exists anywhere in the service layer."*
|
|
25
|
+
- *"There is exactly one `UserSessionManager`."*
|
|
26
|
+
- *"No secret is ever read from `process.env` outside `src/config`."*
|
|
27
|
+
|
|
28
|
+
Every one of those claims is true on the day it is written and unverified from
|
|
29
|
+
the day after. Prose does not fail a build. Six months later the document is
|
|
30
|
+
still confidently asserting something the code stopped doing in March.
|
|
31
|
+
|
|
32
|
+
That has always been an annoyance. It became a defect multiplier the moment
|
|
33
|
+
coding agents started reading these documents as ground truth. An agent that
|
|
34
|
+
reads *"module X does not exist yet"* will happily build a second X. A reviewer
|
|
35
|
+
who trusts *"this symbol occurs zero times"* skips the grep. Stale documentation
|
|
36
|
+
is now executable in the worst sense: it executes inside someone's head, or
|
|
37
|
+
inside a model's context window, and produces work that has to be thrown away.
|
|
38
|
+
|
|
39
|
+
spec-guard makes the claims executable in the good sense. It reads directives
|
|
40
|
+
written as ordinary HTML comments - invisible in every Markdown renderer - and
|
|
41
|
+
checks them against the real tree with ripgrep. A stale claim becomes a failing
|
|
42
|
+
build with a line number, not a landmine.
|
|
43
|
+
|
|
44
|
+
## Quickstart
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
npm install --save-dev @descent-vtt/spec-guard
|
|
48
|
+
npx spec-guard "docs/**/*.md"
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
**The package is scoped; the command is not.** Once installed, the binary is
|
|
52
|
+
plain `spec-guard`, so `npx spec-guard`, `npm scripts` and a global install all
|
|
53
|
+
use that name:
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
npm install -g @descent-vtt/spec-guard # then: spec-guard "docs/**/*.md"
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Without a local install, `npx` needs the full package name - `npx spec-guard`
|
|
60
|
+
on its own would resolve to a different package on the registry:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
npx @descent-vtt/spec-guard "docs/**/*.md"
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Add a directive to any Markdown file, directly under the sentence it makes
|
|
67
|
+
executable:
|
|
68
|
+
|
|
69
|
+
```md
|
|
70
|
+
## Decision
|
|
71
|
+
|
|
72
|
+
Session state has exactly one owner.
|
|
73
|
+
|
|
74
|
+
<!-- @assert-count target="src/" symbol="UserSessionManager" expected="1" -->
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Run it:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
npx spec-guard "docs/**/*.md" --verbose
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
```text
|
|
84
|
+
spec-guard 1 spec · 1 assertion · ripgrep
|
|
85
|
+
|
|
86
|
+
✔ docs/adr/0007-sessions.md:5 @assert-count "UserSessionManager" (1 match) in src
|
|
87
|
+
|
|
88
|
+
1 passed · 7ms
|
|
89
|
+
✔ every spec assertion holds
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
And when it drifts:
|
|
93
|
+
|
|
94
|
+
```text
|
|
95
|
+
✖ docs/adr/0007-sessions.md:5 @assert-count
|
|
96
|
+
"UserSessionManager" must appear exactly 1 time in src
|
|
97
|
+
expected exactly 1 match, found 3
|
|
98
|
+
src/auth/UserSessionManager.ts:12:14 export class UserSessionManager {
|
|
99
|
+
src/legacy/sessions.ts:4:22 import { UserSessionManager } from '../auth';
|
|
100
|
+
src/workers/refresh.ts:9:10 const manager = new UserSessionManager();
|
|
101
|
+
|
|
102
|
+
0 passed · 1 failed · 9ms
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
Exit code 1. The ADR is now a test.
|
|
106
|
+
|
|
107
|
+
## Directives
|
|
108
|
+
|
|
109
|
+
Every directive is an HTML comment. It may span multiple lines. Attribute values
|
|
110
|
+
may use double or single quotes, and a bare attribute means `="true"`.
|
|
111
|
+
|
|
112
|
+
### `@assert-absence` - this symbol is gone
|
|
113
|
+
|
|
114
|
+
```md
|
|
115
|
+
<!-- @assert-absence target="src/controllers,src/services" symbol="LegacyPaymentGateway" -->
|
|
116
|
+
<!-- @assert-absence target="src/" symbol="STRIPE_SECRET_KEY" expected="0" -->
|
|
117
|
+
<!-- @assert-absence target="src/" symbol="TODO" expected="5" reason="burn down the backlog" -->
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Fails when the symbol occurs more than `expected` times (default `0`).
|
|
121
|
+
`max="..."` is accepted as a synonym for `expected`.
|
|
122
|
+
|
|
123
|
+
### `exclude` - everywhere except
|
|
124
|
+
|
|
125
|
+
Most real rules are "nowhere except one place", not "not here":
|
|
126
|
+
|
|
127
|
+
```md
|
|
128
|
+
<!-- @assert-absence target="src" symbol="process.env" exclude="src/config/**" -->
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
That is the rule this README opens with - *no secret is ever read from
|
|
132
|
+
`process.env` outside `src/config`* - written as one assertion instead of a
|
|
133
|
+
hand-maintained list of every directory that is not `src/config`.
|
|
134
|
+
|
|
135
|
+
`exclude` follows gitignore rules, which are **not** the same as `glob`'s:
|
|
136
|
+
|
|
137
|
+
| Pattern | Excludes |
|
|
138
|
+
| --- | --- |
|
|
139
|
+
| `tests` | any directory or file named `tests`, at any depth, and everything under it |
|
|
140
|
+
| `src/config` | that directory and everything under it |
|
|
141
|
+
| `src/config/**` | the same, written explicitly |
|
|
142
|
+
| `*.test.ts` | any file with that name shape, at any depth |
|
|
143
|
+
|
|
144
|
+
The difference is deliberate. `glob="*.ts"` filters files, so basename matching
|
|
145
|
+
is what you want; `exclude="tests"` means the directory, because that is what
|
|
146
|
+
people mean when they write it - and it is what `rg -g '!tests'` does. Both
|
|
147
|
+
engines implement the same rule, and a parity matrix asserts they agree on it.
|
|
148
|
+
|
|
149
|
+
List attributes accept commas or whitespace, so both of these work:
|
|
150
|
+
|
|
151
|
+
```md
|
|
152
|
+
<!-- @assert-absence target="src" symbol="TODO" exclude="src/legacy/** tests" -->
|
|
153
|
+
<!-- @assert-absence target="src" symbol="TODO" exclude="src/legacy/**,tests" -->
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
(A path containing a space therefore cannot be written; there is no quoting
|
|
157
|
+
inside an attribute value.)
|
|
158
|
+
|
|
159
|
+
### `comments` - the note about a deletion is not the deletion
|
|
160
|
+
|
|
161
|
+
You delete a symbol and leave the explanation where the next person will look:
|
|
162
|
+
|
|
163
|
+
```ts
|
|
164
|
+
// LegacyThing was removed in ADR-398; do not reintroduce it.
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
A plain text search reads that comment as an occurrence, so the assertion that
|
|
168
|
+
keeps the symbol deleted fails on the sentence proving it was deleted. Both
|
|
169
|
+
fixes are bad: delete the note and lose the reason, or delete the rule.
|
|
170
|
+
|
|
171
|
+
So matches inside comments do not count:
|
|
172
|
+
|
|
173
|
+
```md
|
|
174
|
+
<!-- comments are ignored by default -->
|
|
175
|
+
<!-- @assert-absence target="src" symbol="LegacyThing" -->
|
|
176
|
+
|
|
177
|
+
<!-- ...unless you ask for them -->
|
|
178
|
+
<!-- @assert-absence target="src" symbol="Copyright" comments="include" -->
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
`comments="include"` is right for assertions that really are about text — a
|
|
182
|
+
licence header, or a name that must appear nowhere in the repository at all.
|
|
183
|
+
|
|
184
|
+
Because this is the one thing that can turn a failing run green without anyone
|
|
185
|
+
touching code, a run that passed this way says so:
|
|
186
|
+
|
|
187
|
+
```
|
|
188
|
+
⚠ 1 match inside comments was not counted; add comments="include" to count it
|
|
189
|
+
|
|
190
|
+
1 passed · 7ms
|
|
191
|
+
✔ every spec assertion holds
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
Comment syntax is known for around 59 extensions across 9 families (JS/TS, C,
|
|
195
|
+
C#, Rust, Go, Python-style `#`, SQL-style `--`, markup, and formats with no
|
|
196
|
+
comments at all). Strings are tracked too, because `//` inside a URL is not a
|
|
197
|
+
comment and reading it as one would hide real code. Where spec-guard is unsure —
|
|
198
|
+
an unknown extension, an unterminated literal — the text counts as code, and the
|
|
199
|
+
report says which files it could not classify. A match wrongly kept is a visible
|
|
200
|
+
failure you can argue with; a match wrongly dropped is a lie.
|
|
201
|
+
|
|
202
|
+
## What gets searched
|
|
203
|
+
|
|
204
|
+
An assertion is worth exactly as much as the set of files behind it, so
|
|
205
|
+
spec-guard is explicit about that set and never quietly narrows it.
|
|
206
|
+
|
|
207
|
+
**Four directory names are skipped**, and nothing else:
|
|
208
|
+
|
|
209
|
+
| Skipped | Why |
|
|
210
|
+
| --- | --- |
|
|
211
|
+
| `.git`, `.hg`, `.svn` | version-control stores hold compressed copies of code you deleted on purpose |
|
|
212
|
+
| `node_modules` | code you did not write, which your architecture rules are not about |
|
|
213
|
+
|
|
214
|
+
Everything else is searched. That includes **hidden directories** - `.github`,
|
|
215
|
+
`.husky`, `.claude-rules`, `.agents` - because that is where CI, hooks and agent
|
|
216
|
+
rules live, and a rule that cannot see your workflow files is not enforcing much.
|
|
217
|
+
It also includes `dist`, `build`, `out` and `coverage`, because spec-guard
|
|
218
|
+
cannot tell build output from a directory of build scripts, and guessing wrong
|
|
219
|
+
means a rule silently stops covering anything.
|
|
220
|
+
|
|
221
|
+
`.gitignore` is not consulted. It describes what git should carry, not what a
|
|
222
|
+
rule covers - and ripgrep applies it only inside a git repository, so honouring
|
|
223
|
+
it made the same tree answer differently depending on whether a `.git` directory
|
|
224
|
+
happened to exist above it.
|
|
225
|
+
|
|
226
|
+
To narrow scope, say so in the assertion:
|
|
227
|
+
|
|
228
|
+
```md
|
|
229
|
+
<!-- @assert-absence target="src" symbol="TODO" exclude="dist coverage" -->
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
`--no-default-skips` removes even those four, for a run that has to be certain.
|
|
233
|
+
|
|
234
|
+
### Nothing is skipped quietly
|
|
235
|
+
|
|
236
|
+
A file spec-guard could not read, or one whose bytes are not text but which
|
|
237
|
+
contained the symbol anyway, is a gap in the answer rather than a detail of it.
|
|
238
|
+
Those are reported, and `--strict` fails on them:
|
|
239
|
+
|
|
240
|
+
```text
|
|
241
|
+
✖ docs/adr.md:3 @assert-absence
|
|
242
|
+
"ApiKey" must not appear in .
|
|
243
|
+
expected no matches, found 0, and 1 file could not be inspected
|
|
244
|
+
⚠ 1 match in 1 binary file not counted: build/app.bin
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
The counts are in `--json` too, as `skipped` on each result.
|
|
248
|
+
|
|
249
|
+
### `@assert-count` - this symbol occurs exactly / at least / at most N times
|
|
250
|
+
|
|
251
|
+
```md
|
|
252
|
+
<!-- @assert-count target="src/" symbol="UserSessionManager" expected="1" -->
|
|
253
|
+
<!-- @assert-count target="src/ui/" symbol="PrimaryButton" min="1" -->
|
|
254
|
+
<!-- @assert-count target="src/core/" symbol="DeprecatedHelper" max="3" -->
|
|
255
|
+
<!-- @assert-count target="src/" symbol="Repository" min="2" max="10" -->
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
Requires `expected`, or `min` and/or `max`. `expected` cannot be combined with
|
|
259
|
+
`min`/`max`.
|
|
260
|
+
|
|
261
|
+
An exact count is the most brittle assertion in the set: a test that merely
|
|
262
|
+
mentions the symbol will break it. spec-guard does not silently skip test files
|
|
263
|
+
for you - an assertion that quietly ignores part of the tree is the failure mode
|
|
264
|
+
this tool exists to prevent - so say what you mean:
|
|
265
|
+
|
|
266
|
+
```md
|
|
267
|
+
<!-- @assert-count target="src" symbol="UserSessionManager" expected="1" exclude="*.test.ts *.spec.ts" -->
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
`min="1"` is often the better rule anyway: it says "this exists" without
|
|
271
|
+
breaking every time someone writes a second test.
|
|
272
|
+
|
|
273
|
+
### `@assert-import-absence` / `@assert-import-count` - dependencies, not text
|
|
274
|
+
|
|
275
|
+
The rules architecture documents actually contain are usually about
|
|
276
|
+
dependencies, and a text search answers those badly in both directions - it
|
|
277
|
+
matches a mention in a comment, and it misses `export * from '../db'`.
|
|
278
|
+
|
|
279
|
+
```md
|
|
280
|
+
<!-- @assert-import-absence target="src/ui" module="src/db" reason="the UI talks to services, not storage" -->
|
|
281
|
+
<!-- @assert-import-count target="src" module="axios" max="1" -->
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
These read the dependency rather than the text, in **five languages**:
|
|
285
|
+
|
|
286
|
+
| Language | Extensions | Reads |
|
|
287
|
+
| --- | --- | --- |
|
|
288
|
+
| JavaScript / TypeScript | `.js .mjs .cjs .jsx .ts .mts .cts .tsx` | `import`, `export ... from`, `require()`, `import()` |
|
|
289
|
+
| Python | `.py .pyi` | `import a.b`, `from .rel import x`, `importlib.import_module("x")` |
|
|
290
|
+
| Go | `.go` | `import "x"` and grouped `import ( ... )`, including aliases and `_` |
|
|
291
|
+
| Rust | `.rs` | `use a::{b, c}` with nested groups, `pub use`, `extern crate` |
|
|
292
|
+
| C# | `.cs .csx` | `using`, `using static`, `global using`, `using X = A.B` |
|
|
293
|
+
|
|
294
|
+
No parser and no new dependency. JavaScript gets a full tokenizer because a
|
|
295
|
+
module reference can appear anywhere in an expression; the other four are read
|
|
296
|
+
by masking comments and string literals with the classifier from
|
|
297
|
+
[ADR-0006](docs/adr/0006-comment-classification.md) and then reading statements
|
|
298
|
+
off what is left. Tree-sitter would have cost 94 MB unpacked against this
|
|
299
|
+
package's 0.33 MB - [ADR-0008](docs/adr/0008-polyglot-imports.md) has the
|
|
300
|
+
measurements, and the list of things a real parser would genuinely see that
|
|
301
|
+
this does not.
|
|
302
|
+
|
|
303
|
+
Module patterns are matched against the resolved `/`-separated path *and*
|
|
304
|
+
against the specifier as written, so `module="System.Text.Json"` and
|
|
305
|
+
`module="System/Text/Json"` both work. Matching is case-sensitive, so a rule
|
|
306
|
+
spanning C# and Go needs both conventions:
|
|
307
|
+
`module="app/db/** App/Db/**"`.
|
|
308
|
+
|
|
309
|
+
The unit counted is **files** that depend on the module. For JavaScript and
|
|
310
|
+
TypeScript it understands:
|
|
311
|
+
|
|
312
|
+
| Form | Counted |
|
|
313
|
+
| --- | --- |
|
|
314
|
+
| `import { A } from 'x'`, `import 'x'`, `import * as ns from 'x'` | yes |
|
|
315
|
+
| `import type { A } from 'x'` | yes, unless `types="ignore"` |
|
|
316
|
+
| `export * from 'x'`, `export { A } from 'x'` | yes - a re-export is a dependency, and a barrel file is how layering rules usually get broken |
|
|
317
|
+
| `require('x')` | yes, with a literal argument |
|
|
318
|
+
| `import('x')` | yes, with a literal argument |
|
|
319
|
+
| `import(name)`, `require(expr)` | **no - reported, see below** |
|
|
320
|
+
|
|
321
|
+
Relative specifiers are resolved by path arithmetic, so `module="src/db"`
|
|
322
|
+
matches `../db/client.js` seen from `src/ui/`, and Python's `from ..core import
|
|
323
|
+
x` resolves against the importing file the same way. There is deliberately no
|
|
324
|
+
filesystem resolution: tsconfig `paths` aliases and package `exports` maps are
|
|
325
|
+
not followed, so an alias is written out as itself (`module="@app/db"`).
|
|
326
|
+
|
|
327
|
+
**It tells you what it could not see.** Roughly 1% of module references in real
|
|
328
|
+
code are dynamic, and about 0.09% of files defeat the tokenizer outright. Both
|
|
329
|
+
are reported rather than counted as clean:
|
|
330
|
+
|
|
331
|
+
```text
|
|
332
|
+
✔ docs/adr.md:1 @assert-import-absence "src/db" (0 matches) in src/core
|
|
333
|
+
⚠ 1 module reference could not be resolved statically
|
|
334
|
+
⚠ src/core/plugin.ts:1 import(name)
|
|
335
|
+
```
|
|
336
|
+
|
|
337
|
+
The count is still true of everything that could be seen; the warning is what
|
|
338
|
+
stops it being mistaken for a complete answer. `--strict` turns those warnings
|
|
339
|
+
into failures. Files in scope that are not JavaScript or TypeScript are counted
|
|
340
|
+
and reported too, so a rule pointed at the wrong tree says "analysed 2 of 3
|
|
341
|
+
files" rather than quietly passing - and if *none* of them can be read, the
|
|
342
|
+
assertion fails rather than passing on an empty analysis.
|
|
343
|
+
|
|
344
|
+
[ADR-0005](docs/adr/0005-import-assertions.md) has the measurements and the
|
|
345
|
+
reasoning behind each boundary.
|
|
346
|
+
|
|
347
|
+
### `@assert-layers` - dependencies point one way
|
|
348
|
+
|
|
349
|
+
"Domain must not depend on infrastructure" across three layers is three
|
|
350
|
+
import rules, and across five it is ten - a triangle nobody writes down, and
|
|
351
|
+
the row somebody forgets when a layer is added. One directive states the order:
|
|
352
|
+
|
|
353
|
+
```md
|
|
354
|
+
<!-- @assert-layers target="src" order="src/domain, src/application, src/infrastructure" -->
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
**Order runs from the layer everything may depend on to the layer that may
|
|
358
|
+
depend on everything.** A file may import from its own layer and from any layer
|
|
359
|
+
listed before it; importing from a layer listed after it is a violation, and
|
|
360
|
+
the report says which way it went:
|
|
361
|
+
|
|
362
|
+
```text
|
|
363
|
+
✖ docs/architecture.md:12 @assert-layers
|
|
364
|
+
src must keep its layers in order, src/domain < src/application < src/infrastructure
|
|
365
|
+
expected no violating files, found 1
|
|
366
|
+
src/domain/user.ts:3:1 src/domain -> src/infrastructure: import ../infrastructure/db.js
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
A layer is a pattern in the same language as `module=` and `exclude`, which is
|
|
370
|
+
why this works in all five languages above: `src/domain` is anchored at the
|
|
371
|
+
root, and a bare `domain` matches that segment anywhere - in `src/domain/user.ts`,
|
|
372
|
+
in Python's `app.domain.user`, in Rust's `crate::domain::user`. A layer may also
|
|
373
|
+
be a single file; `order="src/parser.ts, src/cli.ts"` understands that
|
|
374
|
+
`./parser.js` is `src/parser.ts`.
|
|
375
|
+
|
|
376
|
+
The unit is files, and `max`, `types`, `exclude` and `baseline` mean what they
|
|
377
|
+
mean on `@assert-import-absence`. Three things fail rather than pass, because
|
|
378
|
+
each is a rule that would otherwise check less than it says: a layer that
|
|
379
|
+
matches no file (`order="domain, aplication"`), a file two layers both claim,
|
|
380
|
+
and a target that does not exist. Files that no layer claims are left alone and
|
|
381
|
+
counted, so a directory nobody assigned shows up as a number rather than as
|
|
382
|
+
silence.
|
|
383
|
+
|
|
384
|
+
### `@assert-import-cycle` - no file depends on itself
|
|
385
|
+
|
|
386
|
+
```md
|
|
387
|
+
<!-- @assert-import-cycle target="src" types="ignore" -->
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
A cycle is reported as one concrete loop, with the line of every import on it:
|
|
391
|
+
|
|
392
|
+
```text
|
|
393
|
+
✖ docs/architecture.md:20 @assert-import-cycle
|
|
394
|
+
src must have no import cycles (type-only imports ignored)
|
|
395
|
+
expected no import cycles, found 1
|
|
396
|
+
src/orders/cart.ts:4:1 src/orders/cart.ts:4 -> src/billing/invoice.ts:2 -> src/orders/cart.ts
|
|
397
|
+
```
|
|
398
|
+
|
|
399
|
+
The count is **knots, not loops**: a set of files each reachable from the others
|
|
400
|
+
counts once however many routes run around it, so adding an import inside a
|
|
401
|
+
tangle that already exists does not move the number. `max="2"` adopts the rule
|
|
402
|
+
on a codebase that already has two, and fails on the third.
|
|
403
|
+
|
|
404
|
+
`types="ignore"` asks the runtime question - erased imports create no load-order
|
|
405
|
+
cycle - and the default asks the coupling one. This repository needs both: its
|
|
406
|
+
`src/` has exactly one cycle, and it is type-only.
|
|
407
|
+
|
|
408
|
+
**Cycles are JavaScript and TypeScript only**, because a cycle needs to know that
|
|
409
|
+
`./b.js` *is* `src/b.ts`, and in the other four languages an import names a
|
|
410
|
+
module, package or namespace rather than a file. A scope with no JavaScript or
|
|
411
|
+
TypeScript in it fails instead of reporting no cycles. The resolver is a short,
|
|
412
|
+
fixed table - the TypeScript source of an emitted `.js`, the path as written,
|
|
413
|
+
the path with each extension, the directory's `index` - checked against files
|
|
414
|
+
the walk already found, never against the disk.
|
|
415
|
+
|
|
416
|
+
It says what it could not follow. An import that should have become an edge and
|
|
417
|
+
did not - a relative path matching no file, or an alias like `@/db` or
|
|
418
|
+
`#internal/db` that cannot be a package - is reported, and fails the run under
|
|
419
|
+
`--strict`. An alias spelled like a real package, `@app/db` through `tsconfig`
|
|
420
|
+
paths, is the one thing it cannot see and cannot report: that would mean
|
|
421
|
+
reading `tsconfig.json`, which is the resolver
|
|
422
|
+
[ADR-0011](docs/adr/0011-layers-and-cycles.md) declines to become, for the
|
|
423
|
+
reasons it gives.
|
|
424
|
+
|
|
425
|
+
### `@assert-present` - this file exists
|
|
426
|
+
|
|
427
|
+
```md
|
|
428
|
+
<!-- @assert-present file="SECURITY.md" -->
|
|
429
|
+
<!-- @assert-present file="config/production.json,config/staging.json" -->
|
|
430
|
+
```
|
|
431
|
+
|
|
432
|
+
Passes when every listed path exists relative to `--root`. Directories count.
|
|
433
|
+
|
|
434
|
+
### Attributes
|
|
435
|
+
|
|
436
|
+
| Attribute | Applies to | Meaning |
|
|
437
|
+
| --- | --- | --- |
|
|
438
|
+
| `target` | absence, count | Comma-separated paths to search, relative to `--root`. Default `.` |
|
|
439
|
+
| `symbol` | absence, count | The literal string to search for (or a regex with `regex="true"`) |
|
|
440
|
+
| `file` | present | Comma-separated paths that must exist |
|
|
441
|
+
| `expected` | absence, count | Upper bound for absence; exact count for count |
|
|
442
|
+
| `min` / `max` | count (`max` also on absence) | Inclusive bounds |
|
|
443
|
+
| `glob` | absence, count | Include-only file filters, e.g. `*.ts,*.tsx` (ripgrep `-g` semantics) |
|
|
444
|
+
| `exclude` | absence, count | Paths to leave out, gitignore-style: `src/config/**`, `tests`, `*.test.ts` |
|
|
445
|
+
| `regex` | absence, count | Treat `symbol` as a regular expression |
|
|
446
|
+
| `word` | absence, count | Require word boundaries, so `Primary` does not match `PrimaryButton` |
|
|
447
|
+
| `ignore-case` | absence, count | Case-insensitive matching |
|
|
448
|
+
| `comments` | absence, count | `ignore` (default) or `include` for matches inside comments |
|
|
449
|
+
| `module` | import-absence, import-count | Which dependency, matched like `exclude` |
|
|
450
|
+
| `order` | layers | The layers, from the one everything may depend on to the one that may depend on everything |
|
|
451
|
+
| `types` | import assertions, layers, cycles | `include` (default) or `ignore` for `import type` |
|
|
452
|
+
| `allow-empty` | absence, count, import assertions, layers, cycles | Tolerate a scope that holds no files, or a layer that matches none. Off by default - see below |
|
|
453
|
+
| `baseline` | absence, import-absence, layers | Known violations that do not count: `path` or `path:count` |
|
|
454
|
+
| `ratchet` | absence, import-absence, layers | `two-sided` (default) or `one-way` - see below |
|
|
455
|
+
| `reason` | all | Human-readable justification, printed on failure |
|
|
456
|
+
|
|
457
|
+
Unknown attributes are an error, not a shrug: `expct="1"` fails the run instead
|
|
458
|
+
of silently asserting nothing.
|
|
459
|
+
|
|
460
|
+
### `allow-empty` - an assertion that covers nothing is a failure
|
|
461
|
+
|
|
462
|
+
```md
|
|
463
|
+
<!-- @assert-absence target="services" symbol="Legacy" glob="*.ts" -->
|
|
464
|
+
```
|
|
465
|
+
|
|
466
|
+
If `services/` holds no `.ts` file, that rule passes every time, forever,
|
|
467
|
+
without inspecting anything - and in the report it is indistinguishable from a
|
|
468
|
+
rule that inspected a thousand files and found nothing. So it fails instead:
|
|
469
|
+
|
|
470
|
+
```text
|
|
471
|
+
✖ docs/adr.md:3 @assert-absence
|
|
472
|
+
"Legacy" must not appear in services
|
|
473
|
+
no files were inspected, so this assertion verified nothing
|
|
474
|
+
(add allow-empty="true" if that is expected)
|
|
475
|
+
```
|
|
476
|
+
|
|
477
|
+
The usual causes are a `glob` matching no extension in the tree, an `exclude`
|
|
478
|
+
that swallowed the target, or a directory somebody emptied. Where covering
|
|
479
|
+
nothing yet is the honest state of the world - a rule written before the code
|
|
480
|
+
it guards - `allow-empty="true"` says so, and `--allow-empty-scope` says it for
|
|
481
|
+
a whole run.
|
|
482
|
+
|
|
483
|
+
### `baseline` - adopting a rule the codebase already breaks
|
|
484
|
+
|
|
485
|
+
A strict rule introduced into a mature codebase lands on violations that already
|
|
486
|
+
exist. `exclude` turns those into a permanent blind spot, and `expected="5"`
|
|
487
|
+
cannot tell "one fixed, one added" from "nothing happened". A baseline names
|
|
488
|
+
them:
|
|
489
|
+
|
|
490
|
+
```md
|
|
491
|
+
<!-- @assert-absence target="src" symbol="LegacyGateway"
|
|
492
|
+
baseline="src/legacy/gateway.ts:2
|
|
493
|
+
src/legacy/adapter.ts" -->
|
|
494
|
+
```
|
|
495
|
+
|
|
496
|
+
The rule now holds when no file outside the list matches and no file inside it
|
|
497
|
+
matches more than it declares. A bare path means one match.
|
|
498
|
+
|
|
499
|
+
**It ratchets in both directions.** New debt fails, obviously. So does debt that
|
|
500
|
+
has been *paid* and left on the list, because an entry the code no longer
|
|
501
|
+
supports is a spec asserting something untrue:
|
|
502
|
+
|
|
503
|
+
```text
|
|
504
|
+
expected no matches, found 0; the baseline is out of date and must be
|
|
505
|
+
pruned: src/legacy/adapter.ts (no longer matches)
|
|
506
|
+
```
|
|
507
|
+
|
|
508
|
+
The fix is deleting the line the message names. `ratchet="one-way"` relaxes that
|
|
509
|
+
half to a report-only note. Every run says how many matches the baseline
|
|
510
|
+
excluded, because a pass bought by an exemption is never silent.
|
|
511
|
+
|
|
512
|
+
To adopt a rule on a codebase that already violates it:
|
|
513
|
+
|
|
514
|
+
```bash
|
|
515
|
+
spec-guard docs/adr.md --print-baseline
|
|
516
|
+
```
|
|
517
|
+
|
|
518
|
+
prints the attribute that would exempt exactly today's violations, for you to
|
|
519
|
+
paste in. It prints; it does not edit. [ADR-0009](docs/adr/0009-debt-baselines.md)
|
|
520
|
+
explains why that distinction is the whole design, and why there is no `--fix`
|
|
521
|
+
for architecture rules.
|
|
522
|
+
|
|
523
|
+
## Document status - drafts and superseded ADRs
|
|
524
|
+
|
|
525
|
+
An ADR has a life. It is `Proposed` before anyone agrees to it, and `Superseded`
|
|
526
|
+
long before anyone deletes it - because deleting it deletes the reason a
|
|
527
|
+
decision was made.
|
|
528
|
+
|
|
529
|
+
spec-guard reads that status from the document and honours it. A document whose
|
|
530
|
+
status is `draft`, `proposed`, `rejected`, `deprecated` or `superseded` is
|
|
531
|
+
parsed, validated, reported - and not executed.
|
|
532
|
+
|
|
533
|
+
Three spellings are recognised, because three are in use:
|
|
534
|
+
|
|
535
|
+
<!-- Shown fenced on purpose: spec-guard masks code before reading a status. -->
|
|
536
|
+
|
|
537
|
+
```md
|
|
538
|
+
---
|
|
539
|
+
status: proposed
|
|
540
|
+
---
|
|
541
|
+
```
|
|
542
|
+
|
|
543
|
+
```md
|
|
544
|
+
## Status
|
|
545
|
+
|
|
546
|
+
Accepted (0.3.0).
|
|
547
|
+
```
|
|
548
|
+
|
|
549
|
+
```md
|
|
550
|
+
**Status:** accepted
|
|
551
|
+
```
|
|
552
|
+
|
|
553
|
+
Front-matter wins, then the `## Status` section, then the bold label (which is
|
|
554
|
+
only read above the first `##`). Anything else - `Provisional`, `In review`, a
|
|
555
|
+
misspelled `Supersedded`, or no status at all - keeps enforcing. That asymmetry
|
|
556
|
+
is deliberate: an unanticipated word that keeps enforcing is a visible failure
|
|
557
|
+
with an obvious fix, while one that stops enforcing is a green build over a rule
|
|
558
|
+
nobody is checking.
|
|
559
|
+
|
|
560
|
+
Withholding is never quiet:
|
|
561
|
+
|
|
562
|
+
```text
|
|
563
|
+
○ docs/adr/0011-queues.md is Proposed. - 2 assertions not executed
|
|
564
|
+
|
|
565
|
+
12 passed · 2 not in force · 48ms
|
|
566
|
+
```
|
|
567
|
+
|
|
568
|
+
By name, not by count - the same fact reaches `--format json` as `inactiveSpecs`
|
|
569
|
+
and `--format sarif` as a note-level execution notification. A withheld
|
|
570
|
+
directive is still checked for typos and bad attributes, so a draft's mistake is
|
|
571
|
+
found on the day it is written rather than on the day it is accepted.
|
|
572
|
+
`--ignore-status` runs everything, which is how you ask whether a draft would
|
|
573
|
+
pass if you accepted it today.
|
|
574
|
+
|
|
575
|
+
[ADR-0010](docs/adr/0010-spec-status.md) has the full reasoning, including why
|
|
576
|
+
there is no per-directive `if-status` attribute.
|
|
577
|
+
|
|
578
|
+
## Asking before writing - `query` and the MCP server
|
|
579
|
+
|
|
580
|
+
A run tells you, after the fact, that code broke a rule. `spec-guard query`
|
|
581
|
+
tells you which rules a file is under before you touch it:
|
|
582
|
+
|
|
583
|
+
```bash
|
|
584
|
+
spec-guard query src/domain/user.ts --spec "docs/**/*.md"
|
|
585
|
+
```
|
|
586
|
+
|
|
587
|
+
```text
|
|
588
|
+
src/domain/user.ts
|
|
589
|
+
3 rules from 2 documents
|
|
590
|
+
|
|
591
|
+
ADR-0004: No legacy client (docs/adr/0004-legacy.md, accepted)
|
|
592
|
+
:12 @assert-absence "LegacyClient" must not appear in src
|
|
593
|
+
|
|
594
|
+
ADR-0011: Layering constraints (docs/adr/0011-layers.md, accepted)
|
|
595
|
+
:40 @assert-layers src must keep its layers in order, src/domain < src/application < src/infrastructure
|
|
596
|
+
layer: src/domain (1 of 3)
|
|
597
|
+
may import: src/domain
|
|
598
|
+
must not import: src/application, src/infrastructure
|
|
599
|
+
reason: the domain depends on nothing
|
|
600
|
+
:41 @assert-import-absence src/domain must not import "pg"
|
|
601
|
+
|
|
602
|
+
1 more rule would govern this path if docs/adr/0014-clocks.md (proposed) were in force; --ignore-status lists it
|
|
603
|
+
|
|
604
|
+
14 spec files read in 6.1ms
|
|
605
|
+
```
|
|
606
|
+
|
|
607
|
+
It answers from the specs alone, without reading the codebase, so it works for
|
|
608
|
+
a file that does not exist yet. A directory lists every rule that could reach a
|
|
609
|
+
file under it. `--json` is the same answer for a script. Rules in documents that
|
|
610
|
+
are not in force are counted and named, never silently left out.
|
|
611
|
+
|
|
612
|
+
The arithmetic that decides whether a rule governs a path is tested against the
|
|
613
|
+
walk a real run makes, file for file, under both engines and on randomly
|
|
614
|
+
generated trees. [ADR-0012](docs/adr/0012-query-and-mcp.md) lists the three
|
|
615
|
+
things it cannot see: file content (binary, oversized, unreadable), symbolic
|
|
616
|
+
links, and letter case on case-insensitive filesystems.
|
|
617
|
+
|
|
618
|
+
### The MCP server
|
|
619
|
+
|
|
620
|
+
`spec-guard mcp` serves the same answer to an AI agent over the
|
|
621
|
+
[Model Context Protocol](https://modelcontextprotocol.io) on stdio, with no
|
|
622
|
+
dependency on the MCP SDK. Register it with your client as a stdio server; the
|
|
623
|
+
usual `mcpServers` entry is:
|
|
624
|
+
|
|
625
|
+
```json
|
|
626
|
+
{
|
|
627
|
+
"mcpServers": {
|
|
628
|
+
"spec-guard": {
|
|
629
|
+
"command": "npx",
|
|
630
|
+
"args": ["spec-guard", "mcp", "--spec", "docs/**/*.md"]
|
|
631
|
+
}
|
|
632
|
+
}
|
|
633
|
+
}
|
|
634
|
+
```
|
|
635
|
+
|
|
636
|
+
The root defaults to the directory the client starts the server in; pass
|
|
637
|
+
`--root` if yours starts servers somewhere else. On Windows, `npx` is a `.cmd`
|
|
638
|
+
script that a client spawning without a shell cannot launch; point `command` at
|
|
639
|
+
`node` and the first argument at
|
|
640
|
+
`node_modules/@descent-vtt/spec-guard/bin/spec-guard.js` instead. Run flags such
|
|
641
|
+
as `--engine`, `--strict`, `--ignore-status` and `--allow-missing-targets` apply
|
|
642
|
+
to its checks.
|
|
643
|
+
|
|
644
|
+
| Tool | What it does |
|
|
645
|
+
| --- | --- |
|
|
646
|
+
| `get_architectural_rules(path, include_inactive?)` | The query above, as text and as structured content |
|
|
647
|
+
| `check_architecture(paths?)` | A run. Given paths, only the rules that govern them, each over its whole scope, with every violation marked as in those paths or not |
|
|
648
|
+
|
|
649
|
+
| Resource | What it holds |
|
|
650
|
+
| --- | --- |
|
|
651
|
+
| `spec://rules` | Every rule in force, as JSON |
|
|
652
|
+
| `spec://doc/{+path}` | Any spec document, in force or not |
|
|
653
|
+
|
|
654
|
+
Both protocol eras are served: clients that open with `initialize` (revisions
|
|
655
|
+
2024-10-07 to 2025-11-25), and clients on 2026-07-28 that put the protocol
|
|
656
|
+
version on every request and probe with `server/discover`. Nothing is cached, so
|
|
657
|
+
an ADR edited mid-session is read as edited. ADR-0012 has the shapes, the
|
|
658
|
+
sources they follow, and what is deliberately not implemented.
|
|
659
|
+
|
|
660
|
+
## CLI
|
|
661
|
+
|
|
662
|
+
```bash
|
|
663
|
+
spec-guard [patterns...] [options] # execute the directives
|
|
664
|
+
spec-guard query <paths...> [options] # the rules in force for files or directories
|
|
665
|
+
spec-guard mcp [options] # serve the rules over MCP on stdio
|
|
666
|
+
```
|
|
667
|
+
|
|
668
|
+
| Option | Description |
|
|
669
|
+
| --- | --- |
|
|
670
|
+
| `-r, --root <path>` | Codebase root that assertions resolve against (default: cwd) |
|
|
671
|
+
| `--spec <pattern>` | A spec glob or path, repeatable. `query` and `mcp` take their specs only from here |
|
|
672
|
+
| `-v, --verbose` | Print passing assertions too |
|
|
673
|
+
| `--fail-fast` | Stop at the first failing assertion |
|
|
674
|
+
| `--json` | Machine-readable report on stdout (same as `--format json`) |
|
|
675
|
+
| `--format <human\|json\|sarif>` | Output format. `sarif` uploads to GitHub code scanning |
|
|
676
|
+
| `--engine <auto\|rg\|js>` | Search engine (default `auto`: scanner for small trees, ripgrep for big ones) |
|
|
677
|
+
| `--strict` | Treat analysis that could not be completed as a failure |
|
|
678
|
+
| `--allow-missing-targets` | Warn instead of failing when a `target` path does not exist |
|
|
679
|
+
| `--allow-empty-scope` | Warn instead of failing when an assertion inspects no files |
|
|
680
|
+
| `--print-baseline` | Print the `baseline="..."` that would exempt today's violations, and exit |
|
|
681
|
+
| `--no-default-skips` | Search `.git`, `.hg`, `.svn` and `node_modules` too |
|
|
682
|
+
| `--ignore-status` | Execute directives in draft, proposed and superseded documents too |
|
|
683
|
+
| `--include-specs` | Also count matches inside the spec files themselves |
|
|
684
|
+
| `--max-snippets <n>` | Failure snippets per assertion (default 5) |
|
|
685
|
+
| `--concurrency <n>` | Search passes in flight at once (default 8) |
|
|
686
|
+
| `--allow-empty` | Exit 0 when no spec file matched the patterns (about the run, not an assertion) |
|
|
687
|
+
| `--color` / `--no-color` | Force colour on or off (`NO_COLOR` honoured) |
|
|
688
|
+
|
|
689
|
+
Patterns are expanded by spec-guard itself, so quoted globs behave identically
|
|
690
|
+
on Windows, macOS and Linux. A directory expands to the Markdown files in it.
|
|
691
|
+
|
|
692
|
+
### Exit codes
|
|
693
|
+
|
|
694
|
+
| Code | Meaning |
|
|
695
|
+
| --- | --- |
|
|
696
|
+
| `0` | Every assertion held |
|
|
697
|
+
| `1` | An assertion failed, or a directive was malformed |
|
|
698
|
+
| `2` | spec-guard could not run: bad usage, no spec files matched, `--engine rg` with no ripgrep |
|
|
699
|
+
|
|
700
|
+
## CI integration
|
|
701
|
+
|
|
702
|
+
```yaml
|
|
703
|
+
# .github/workflows/specs.yml
|
|
704
|
+
name: Specs
|
|
705
|
+
on: [push, pull_request]
|
|
706
|
+
|
|
707
|
+
jobs:
|
|
708
|
+
spec-guard:
|
|
709
|
+
runs-on: ubuntu-latest
|
|
710
|
+
steps:
|
|
711
|
+
- uses: actions/checkout@v5
|
|
712
|
+
- uses: actions/setup-node@v5
|
|
713
|
+
with:
|
|
714
|
+
node-version: '22'
|
|
715
|
+
- run: npx @descent-vtt/spec-guard "docs/**/*.md" "README.md" --verbose
|
|
716
|
+
```
|
|
717
|
+
|
|
718
|
+
That job needs nothing else installed. GitHub-hosted runners do **not** ship
|
|
719
|
+
ripgrep on `PATH` - spec-guard's own CI reports `engine: javascript` there - so
|
|
720
|
+
the fallback is what actually runs, and it produces identical results. If your
|
|
721
|
+
repository is large enough that you want ripgrep's speed, install it first:
|
|
722
|
+
|
|
723
|
+
```yaml
|
|
724
|
+
- run: sudo apt-get update && sudo apt-get install -y ripgrep
|
|
725
|
+
```
|
|
726
|
+
|
|
727
|
+
Or point spec-guard at a binary you already have with `SPEC_GUARD_RG=/path/to/rg`.
|
|
728
|
+
|
|
729
|
+
As a pre-commit hook (assuming a local install, so the bare command resolves):
|
|
730
|
+
|
|
731
|
+
```bash
|
|
732
|
+
npx spec-guard "docs/**/*.md" --fail-fast
|
|
733
|
+
```
|
|
734
|
+
|
|
735
|
+
### Annotations on the pull request
|
|
736
|
+
|
|
737
|
+
`--format sarif` writes a SARIF 2.1.0 document, which GitHub turns into a
|
|
738
|
+
comment on the offending line:
|
|
739
|
+
|
|
740
|
+
```yaml
|
|
741
|
+
- run: npx @descent-vtt/spec-guard "docs/**/*.md" --format sarif > spec-guard.sarif || true
|
|
742
|
+
- uses: github/codeql-action/upload-sarif@v3
|
|
743
|
+
with:
|
|
744
|
+
sarif_file: spec-guard.sarif
|
|
745
|
+
category: spec-guard
|
|
746
|
+
```
|
|
747
|
+
|
|
748
|
+
That job needs `permissions: security-events: write`. One alert per broken rule
|
|
749
|
+
rather than per match - the thing that broke is the rule - anchored on the first
|
|
750
|
+
offending line, with the directive as a related location because that is often
|
|
751
|
+
where the fix goes. Alerts carry a fingerprint derived from what the assertion
|
|
752
|
+
is about rather than where its matches landed, so inserting a line above a
|
|
753
|
+
violation does not close the alert and open a new one.
|
|
754
|
+
|
|
755
|
+
**There is no language server, and that is a decision rather than a gap.**
|
|
756
|
+
spec-guard's claims are about a whole repository - "this symbol appears nowhere
|
|
757
|
+
in `src`" - and a language server is handed one buffer at a time. Answering a
|
|
758
|
+
repository-wide question on every keystroke means rescanning the tree on every
|
|
759
|
+
keystroke; the alternative is answering a smaller question and calling it the
|
|
760
|
+
same one. The CLI does a full run of this repository in about 130 ms, so a
|
|
761
|
+
pre-commit hook or a watch loop already closes the feedback gap without a
|
|
762
|
+
daemon, an extension per editor, or a protocol version matrix.
|
|
763
|
+
|
|
764
|
+
## How it works
|
|
765
|
+
|
|
766
|
+
```text
|
|
767
|
+
parser ──▶ Directive[] ──▶ runner ──▶ Assertion[] ──▶ engine ──▶ AssertionResult[] ──▶ reporter
|
|
768
|
+
(pure) (I/O) (rg | js) (pure)
|
|
769
|
+
```
|
|
770
|
+
|
|
771
|
+
1. **Parse.** Markdown is scanned for `<!-- @assert-* -->` comments. Fenced code
|
|
772
|
+
blocks and inline code spans are masked first, so documentation that shows
|
|
773
|
+
the syntax (like this README) never executes it.
|
|
774
|
+
2. **Resolve.** Each directive becomes a typed assertion. Bad numbers, unknown
|
|
775
|
+
booleans, unknown attributes and paths escaping `--root` are rejected here,
|
|
776
|
+
before any I/O.
|
|
777
|
+
3. **Search.** Assertions that share a target list and flags are answered by a
|
|
778
|
+
*single* pass over the tree.
|
|
779
|
+
4. **Report.** ANSI output with the spec location, the expectation, the observed
|
|
780
|
+
count, and up to five real snippets; or JSON for tooling.
|
|
781
|
+
|
|
782
|
+
### Performance
|
|
783
|
+
|
|
784
|
+
Scanning a tree costs about the same whether you look for one symbol or twenty,
|
|
785
|
+
so spec-guard groups assertions by target set and flags and answers each group
|
|
786
|
+
in one pass.
|
|
787
|
+
|
|
788
|
+
`auto` then picks between the two engines **per search group**, because they
|
|
789
|
+
have different shapes of cost: ripgrep is dominated by process startup and
|
|
790
|
+
barely notices tree size, while the built-in scanner has no startup cost and
|
|
791
|
+
grows linearly. Starting a process to search a handful of files is a bad trade.
|
|
792
|
+
Re-measured for 0.4.0 on Windows 11 / Node 24, median of three, one symbol over
|
|
793
|
+
a synthetic tree:
|
|
794
|
+
|
|
795
|
+
| files | scanner | ripgrep | |
|
|
796
|
+
| --- | --- | --- | --- |
|
|
797
|
+
| 100 | **17 ms** | 214 ms | scanner, 12.9x |
|
|
798
|
+
| 500 | **70 ms** | 263 ms | scanner, 3.8x |
|
|
799
|
+
| 1,000 | 216 ms | **181 ms** | ripgrep, 1.2x |
|
|
800
|
+
| 3,000 | 735 ms | **222 ms** | ripgrep, 3.3x |
|
|
801
|
+
| 5,000 | 1,046 ms | **241 ms** | ripgrep, 4.3x |
|
|
802
|
+
|
|
803
|
+
The decision uses a bounded enumeration as its probe: spec-guard walks the
|
|
804
|
+
target set until it either finishes - in which case the file list is already in
|
|
805
|
+
hand and the scanner runs against it, with no process and no second walk - or
|
|
806
|
+
exceeds a budget, in which case ripgrep takes over. The crossover above sits
|
|
807
|
+
between 500 and 1,000 files, which is what the Windows budget of 512 encodes;
|
|
808
|
+
on Linux it is far lower, because what is really being measured is process
|
|
809
|
+
spawn cost. `scripts/bench-engines.mjs` reproduces this, and
|
|
810
|
+
[ADR-0004](docs/adr/0004-adaptive-engine.md) has the full tables.
|
|
811
|
+
|
|
812
|
+
Treat single measurements from one machine with suspicion. On the development
|
|
813
|
+
machine used here, an absence assertion over 2,000 files measures 303 ms with
|
|
814
|
+
the scanner and 399 ms with ripgrep, but ripgrep's own spread across five runs
|
|
815
|
+
was 379-1039 ms - wide enough that the two are not really distinguishable at
|
|
816
|
+
that size. The order-of-magnitude differences at the ends of the table are the
|
|
817
|
+
part worth trusting.
|
|
818
|
+
|
|
819
|
+
0.5.0 added four language analysers, a scope probe on every assertion and a
|
|
820
|
+
baseline pass, so the obvious question is what that cost. Both versions were run
|
|
821
|
+
against the same synthetic 1,000-file tree, alternating rounds inside one
|
|
822
|
+
process so that this machine's load - which varies by a factor of two over a
|
|
823
|
+
session - falls on both equally:
|
|
824
|
+
|
|
825
|
+
| | median | min |
|
|
826
|
+
| --- | --- | --- |
|
|
827
|
+
| 0.4.0 | 908.8 ms | 493.7 ms |
|
|
828
|
+
| 0.5.0 | 913.4 ms | 496.6 ms |
|
|
829
|
+
|
|
830
|
+
0.5% apart, against a run-to-run spread far wider than that. The two versions
|
|
831
|
+
also returned identical counts for all five assertions, which is the more
|
|
832
|
+
useful half of the result.
|
|
833
|
+
|
|
834
|
+
0.5.1 replaced three hand-maintained "are these the same question" field lists
|
|
835
|
+
with one, and the surviving version serialises the exclude set where the old one
|
|
836
|
+
compared it by identity - a hot path made slower on purpose, so it was measured
|
|
837
|
+
rather than assumed. Same tree, same alternating rounds:
|
|
838
|
+
|
|
839
|
+
| | median | min |
|
|
840
|
+
| --- | --- | --- |
|
|
841
|
+
| 0.5.0 | 925.1 ms | 432.1 ms |
|
|
842
|
+
| 0.5.1 | 830.6 ms | 428.5 ms |
|
|
843
|
+
|
|
844
|
+
The medians are 10% apart and the minima are level, which is this machine's
|
|
845
|
+
noise band doing most of the talking - the honest reading is "no regression",
|
|
846
|
+
not "10% faster". The plausible mechanism for any real gain is that the runner
|
|
847
|
+
now groups by the engine's own definition of a shared pass, so a group of five
|
|
848
|
+
where two requests disagree is run as three and two rather than being refused
|
|
849
|
+
whole and run as five. Counts identical across all five assertions, again.
|
|
850
|
+
|
|
851
|
+
Two honest caveats:
|
|
852
|
+
|
|
853
|
+
- **The reported engine is the one that ran, not the one available.** On a small
|
|
854
|
+
repository `--json` reports `"engine": "javascript"` even with ripgrep
|
|
855
|
+
installed. That is the optimisation working, not a failure to find `rg`.
|
|
856
|
+
- **The engines are one implementation, not two that agree.** This used to say
|
|
857
|
+
that ripgrep was the reference and the scanner merely matched it "on
|
|
858
|
+
everything the test suite covers", with a note that the two treated
|
|
859
|
+
`.gitignore` and ignored directories differently and that you should pin an
|
|
860
|
+
engine if it mattered. It did matter: on a tree with eight copies of a symbol
|
|
861
|
+
the scanner found two and ripgrep found four. Since 0.4.0 ripgrep answers only
|
|
862
|
+
*which files contain this text*, and the scanner does all the counting,
|
|
863
|
+
classification and reporting for both. `--engine` now changes how long a run
|
|
864
|
+
takes and nothing about its verdict, and the suite asserts that on trees built
|
|
865
|
+
from every case that used to split them.
|
|
866
|
+
|
|
867
|
+
## Programmatic API
|
|
868
|
+
|
|
869
|
+
```ts
|
|
870
|
+
import { runSpecGuard, formatReport } from '@descent-vtt/spec-guard';
|
|
871
|
+
|
|
872
|
+
const report = await runSpecGuard({
|
|
873
|
+
patterns: ['docs/**/*.md'],
|
|
874
|
+
root: process.cwd(),
|
|
875
|
+
});
|
|
876
|
+
|
|
877
|
+
if (!report.ok) {
|
|
878
|
+
console.error(formatReport(report, { color: true, verbose: false }));
|
|
879
|
+
process.exitCode = 1;
|
|
880
|
+
}
|
|
881
|
+
```
|
|
882
|
+
|
|
883
|
+
`runSpecGuard` returns the full report: every assertion, its bounds, the
|
|
884
|
+
observed count, match locations, warnings and timings. The parser
|
|
885
|
+
(`parseDirectives`) and reporter (`formatReport`, `formatJson`) are pure
|
|
886
|
+
functions you can use on their own.
|
|
887
|
+
|
|
888
|
+
## Design decisions
|
|
889
|
+
|
|
890
|
+
This tool was specified loosely and built opinionatedly. Where the
|
|
891
|
+
implementation departs from the obvious reading of the brief, here is why.
|
|
892
|
+
|
|
893
|
+
**Spec files are excluded from their own searches.** An ADR that says
|
|
894
|
+
"`LegacyGateway` must not appear" contains the string `LegacyGateway`. Without
|
|
895
|
+
this rule, absence assertions would fail on the document asserting them - the
|
|
896
|
+
single most confusing possible first-run experience. `--include-specs` restores
|
|
897
|
+
the naive behaviour, and counts the prose of your specs; the directives
|
|
898
|
+
themselves stay uncounted, because a directive is an HTML comment and a rule
|
|
899
|
+
whose own text trips it can never be satisfied. `comments="include"` counts even
|
|
900
|
+
those.
|
|
901
|
+
|
|
902
|
+
**Fenced code and inline code are masked before parsing.** A README documenting
|
|
903
|
+
the syntax must not execute it. Masking preserves byte offsets, so reported line
|
|
904
|
+
numbers stay exact. (This is subtle: pairing backtick runs the naive way
|
|
905
|
+
desynchronises after a stray unmatched run and un-masks real prose. spec-guard
|
|
906
|
+
uses CommonMark's equal-length pairing rule, and there is a regression test.)
|
|
907
|
+
|
|
908
|
+
**A document's lifecycle status is read; a directive's is not.** Status is
|
|
909
|
+
document-level and visible in every rendered Markdown view. A per-directive
|
|
910
|
+
`if-status` attribute would be a switch disabling one assertion inside an
|
|
911
|
+
accepted ADR, invisible to anyone who does not read the raw source - and the
|
|
912
|
+
one thing a mechanism for not running an assertion must never be is invisible.
|
|
913
|
+
|
|
914
|
+
**Malformed directives fail the run.** A typo like `expct="1"` could be ignored
|
|
915
|
+
as "not a directive". It is instead an error, because a spec tool whose typos
|
|
916
|
+
silently assert nothing is worse than no spec tool.
|
|
917
|
+
|
|
918
|
+
**Four attributes were added beyond the original brief** - `word`, `regex`,
|
|
919
|
+
`glob` and `ignore-case` - because `symbol="Primary"` matching `PrimaryButton`
|
|
920
|
+
is the first thing every user hits, and `reason` because a failure message
|
|
921
|
+
should say *why* the rule exists.
|
|
922
|
+
|
|
923
|
+
**`min`/`max` are accepted on `@assert-absence` too.** "At most 5 TODOs" is an
|
|
924
|
+
absence claim with a budget, and burning a budget down is a real workflow.
|
|
925
|
+
|
|
926
|
+
**The engine is detected lazily.** Probing with `rg --version` up front costs a
|
|
927
|
+
process spawn (~27 ms on Windows) on the critical path of every run, including
|
|
928
|
+
runs where ripgrep is missing. Discovering its absence from the first real
|
|
929
|
+
search is free.
|
|
930
|
+
|
|
931
|
+
**Assertions that share a target list share one pass over the tree.** This used
|
|
932
|
+
to need a proof: merging literals into a ripgrep alternation can lose matches
|
|
933
|
+
through containment (`Primary` / `PrimaryButton`) and dovetailing (`abc` / `cd`
|
|
934
|
+
in `abcd`), so spec-guard checked for both and fell back to separate passes.
|
|
935
|
+
Since 0.4.0 it does not need the proof, because ripgrep no longer counts
|
|
936
|
+
anything - it answers only *which files contain this text*, and the scanner
|
|
937
|
+
counts each pattern separately over the shared file contents. The batching
|
|
938
|
+
checks were deleted along with the risk they guarded.
|
|
939
|
+
|
|
940
|
+
**Exit code 2 exists.** "Your specs failed" and "spec-guard could not run" are
|
|
941
|
+
different facts, and CI should be able to tell them apart.
|
|
942
|
+
|
|
943
|
+
**A missing `target` fails the run.** It used to warn and search what was left,
|
|
944
|
+
which meant an assertion pointed at a renamed directory searched nothing, found
|
|
945
|
+
nothing, and reported success — the exact shape of a green check that verified
|
|
946
|
+
nothing. `--allow-missing-targets` restores the old behaviour for repositories
|
|
947
|
+
where a path is legitimately optional.
|
|
948
|
+
|
|
949
|
+
**Hidden directories are searched, and `.gitignore` is not consulted.** An
|
|
950
|
+
audit of 0.3.0 found the tool reporting a clean pass on a repository whose
|
|
951
|
+
forbidden symbol sat in `.github/workflows/ci.yml`; the scanner and ripgrep also
|
|
952
|
+
disagreed with each other, finding two matches and four on the same tree. Scope
|
|
953
|
+
is now one policy that both engines are driven by, the skip list is four names
|
|
954
|
+
long, and anything spec-guard could not inspect is reported rather than assumed
|
|
955
|
+
clean ([ADR-0007](docs/adr/0007-search-scope.md)).
|
|
956
|
+
|
|
957
|
+
**Comments are excluded by default, and the exclusion is reported.** Counting
|
|
958
|
+
the note that records a deletion as an occurrence of the thing deleted punishes
|
|
959
|
+
the documentation this tool exists to keep honest ([ADR-0006](docs/adr/0006-comment-classification.md)).
|
|
960
|
+
The reverse risk — a rule that quietly stops checking anything because every
|
|
961
|
+
match now sits in a comment — is why every run says how many matches it dropped.
|
|
962
|
+
|
|
963
|
+
**An assertion that inspected no files fails.** It is the same defect as a
|
|
964
|
+
missing `target` seen from a different angle: a rule whose scope is empty passes
|
|
965
|
+
forever and reads exactly like a rule that found nothing. Turning this on found
|
|
966
|
+
a vacuous assertion inside this repository's own test suite on the first run.
|
|
967
|
+
`allow-empty="true"` covers the honest case of a rule written before the code it
|
|
968
|
+
guards.
|
|
969
|
+
|
|
970
|
+
**Four more languages, and no parser.** Import assertions read Python, Go, Rust
|
|
971
|
+
and C# as well as JavaScript. Not by adding four tokenizers - by reusing the
|
|
972
|
+
comment and string lexer that already existed, masking the source with it, and
|
|
973
|
+
reading statements off what is left. Tree-sitter would have been the modern
|
|
974
|
+
answer and costs 94 MB unpacked against this package's 0.33 MB; the four things
|
|
975
|
+
a real parser would genuinely see that this cannot are listed in
|
|
976
|
+
[ADR-0008](docs/adr/0008-polyglot-imports.md) rather than glossed over.
|
|
977
|
+
|
|
978
|
+
**There is no `--fix`.** Every edit a machine can make to a failing boundary
|
|
979
|
+
assertion is an edit that records the rule no longer holding: widen the bound,
|
|
980
|
+
add an exclusion, append to the baseline, insert an ignore comment. A one-flag
|
|
981
|
+
path from red to green is a bad button for a person and a much worse one for an
|
|
982
|
+
agent whose loop terminates on a green build. `--print-baseline` prints what you
|
|
983
|
+
could paste; it does not paste it ([ADR-0009](docs/adr/0009-debt-baselines.md)).
|
|
984
|
+
|
|
985
|
+
## spec-guard checks itself
|
|
986
|
+
|
|
987
|
+
The invariants in [`docs/adr/0001-invariants.md`](docs/adr/0001-invariants.md)
|
|
988
|
+
and [`docs/adr/0002-directive-format.md`](docs/adr/0002-directive-format.md) are
|
|
989
|
+
executed against this repository on every CI run. The CLI never calls
|
|
990
|
+
`console.log`, nothing outside the engine spawns a process, and the parser and
|
|
991
|
+
reporter never touch the filesystem - because those documents say so, and the
|
|
992
|
+
build fails if they stop being true.
|
|
993
|
+
|
|
994
|
+
This README is executable too:
|
|
995
|
+
|
|
996
|
+
<!-- @assert-present file="src/parser.ts,src/engine.ts,src/reporter.ts,src/runner.ts,src/cli.ts,src/comments.ts" -->
|
|
997
|
+
|
|
998
|
+
## Development
|
|
999
|
+
|
|
1000
|
+
```bash
|
|
1001
|
+
npm install
|
|
1002
|
+
npm run build # tsc -> dist/
|
|
1003
|
+
npm test # vitest
|
|
1004
|
+
npm run test:coverage
|
|
1005
|
+
npm run test:mutation # stryker (~16 min in CI; over two hours locally)
|
|
1006
|
+
npm run lint # tsc --noEmit
|
|
1007
|
+
npm run selfcheck # run spec-guard on its own docs
|
|
1008
|
+
```
|
|
1009
|
+
|
|
1010
|
+
The test suite runs every assertion case against **both** engines and asserts
|
|
1011
|
+
they agree, so the fallback cannot quietly drift from ripgrep. `@vscode/ripgrep`
|
|
1012
|
+
is a devDependency purely so that the ripgrep path is exercised on every
|
|
1013
|
+
platform in CI, including machines that have no `rg` on PATH; it ships a
|
|
1014
|
+
prebuilt binary and is never a runtime dependency.
|
|
1015
|
+
|
|
1016
|
+
### Mutation testing
|
|
1017
|
+
|
|
1018
|
+
Coverage says a line ran. It does not say an assertion would notice if the line
|
|
1019
|
+
behaved differently. This repository measures the difference: line coverage is
|
|
1020
|
+
**100%**, and **97.65%** of 4,299 mutants are killed in CI, with no file below
|
|
1021
|
+
95%. The second number is the one worth reading - and it is the ceiling, not a
|
|
1022
|
+
way-point: every surviving mutant has been checked individually and the 101 that
|
|
1023
|
+
are left produce byte-identical output.
|
|
1024
|
+
|
|
1025
|
+
That gap is the point. The first run scored 77.23%, and the weakest file was the
|
|
1026
|
+
reporter at 65.48% - not because it lacked tests, but because its tests were
|
|
1027
|
+
almost all `toContain` against colourless output. A mutant could prepend a junk
|
|
1028
|
+
line, drop a colour, or turn `remaining > 0` into `remaining >= 0` and every
|
|
1029
|
+
test still passed. Exact whole-output comparison took it to 87.30%. The engine
|
|
1030
|
+
and the walker were then rewritten for testability rather than papered over with
|
|
1031
|
+
more tests, which is what moved them from 75%/78% to 83%/93%.
|
|
1032
|
+
|
|
1033
|
+
The most recent pass took every module to its ceiling, and its value was not the
|
|
1034
|
+
number. Writing down four contracts that had only ever been checked through
|
|
1035
|
+
their effect on a match count turned up four wrong answers: two assertions on
|
|
1036
|
+
one symbol answered each other's comment handling, an unreadable directory went
|
|
1037
|
+
unreported on any repository small enough to scan in process, snippets from CRLF
|
|
1038
|
+
files carried a carriage return into the terminal, and an invalid pattern was
|
|
1039
|
+
reported with its error message twice. Seventeen branches turned out to be
|
|
1040
|
+
unable to decide anything and were deleted rather than pinned; thirty-five
|
|
1041
|
+
negative controls - each defect reintroduced one at a time - confirm the suite
|
|
1042
|
+
goes red for every one, and the changes meant to be invisible were checked
|
|
1043
|
+
against 6,976 files of real code, reference for reference, against the published
|
|
1044
|
+
0.5.0 build.
|
|
1045
|
+
|
|
1046
|
+
Then the survivors themselves were checked rather than excused. All 130 were
|
|
1047
|
+
applied one at a time and run through a fingerprint of 4,176 observations, and
|
|
1048
|
+
**29 of them turned out not to be equivalent at all** - a Rust brace counter
|
|
1049
|
+
that only matters when a second statement follows, a type-only test that marks
|
|
1050
|
+
`import A, { B } from 'x'` type-only when loosened, a comment check that opens a
|
|
1051
|
+
block comment on `2*3`. All 29 are now tested and all 29 die. All of it is in
|
|
1052
|
+
[ADR-0003](docs/adr/0003-mutation-testing.md), including a Stryker limitation
|
|
1053
|
+
found on the way: a mutant that stops a test file *loading* is reported as
|
|
1054
|
+
survived even though the suite is in fact killing it.
|
|
1055
|
+
|
|
1056
|
+
CI runs it in **two tiers**, both gated at 97%. Branches and pull requests run
|
|
1057
|
+
Stryker incrementally, reusing the verdict for any mutant whose source and
|
|
1058
|
+
covering tests are both unchanged. Pushes to `main`, the weekly schedule and
|
|
1059
|
+
manual runs do the full sweep, which is the authoritative number and the one
|
|
1060
|
+
quoted above; a full sweep is also what publishes the cache the branches start
|
|
1061
|
+
from, so an incremental verdict can never be built on another incremental
|
|
1062
|
+
verdict.
|
|
1063
|
+
|
|
1064
|
+
The full sweep was 6m54s at 2,118 mutants, 15m51s at 3,493, and is 17m at
|
|
1065
|
+
4,299. That growth is why the tiers exist: a check that gets quietly more
|
|
1066
|
+
expensive every release is a check somebody eventually proposes lowering.
|
|
1067
|
+
|
|
1068
|
+
Run it locally with `npm run test:mutation` if you like, but do not calibrate
|
|
1069
|
+
anything on the result: on the Windows machine this was developed on the same
|
|
1070
|
+
suite takes over two hours against 21 minutes hosted, and it scores *higher*,
|
|
1071
|
+
because far more mutants hang there and Stryker counts a hang as a kill. Linux
|
|
1072
|
+
CI is the measurement.
|
|
1073
|
+
|
|
1074
|
+
Eight cautionary tales are in [ADR-0003](docs/adr/0003-mutation-testing.md): a
|
|
1075
|
+
run whose score was pure fiction because the mutants were never activated, a
|
|
1076
|
+
tuning knob that lifted the score six points without adding a test, the platform
|
|
1077
|
+
gap above, the baseline being re-anchored when a tokenizer arrived, and a score
|
|
1078
|
+
that rose partly because code carrying hard-to-kill mutants was deleted rather
|
|
1079
|
+
than because tests improved, and the 0.5.0 sweep that failed the build at 83.77%
|
|
1080
|
+
because a suite can be thorough about the thing it was written to test and
|
|
1081
|
+
silent about the machinery underneath it, and the excuse that let the tokenizer
|
|
1082
|
+
sit at 73% for three releases because "that kind of code carries more equivalent
|
|
1083
|
+
mutants" sounded like judgement rather than an unchecked assumption - along with
|
|
1084
|
+
the real bugs that chasing the gate uncovered, and a class of mutant the runner
|
|
1085
|
+
reports as survived while the suite is in fact killing it.
|
|
1086
|
+
|
|
1087
|
+
## Requirements
|
|
1088
|
+
|
|
1089
|
+
- Node.js 22 or newer (native ESM)
|
|
1090
|
+
- ripgrep optional - used when present, replaced by a built-in scanner when not
|
|
1091
|
+
|
|
1092
|
+
## License
|
|
1093
|
+
|
|
1094
|
+
MIT
|