@descent-vtt/spec-guard 0.5.1 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,868 +1,1009 @@
1
- # spec-guard
2
-
3
- **Executable architecture assertions for Markdown specs & ADRs.**
4
-
5
- Turn the claims your design documents make about your codebase into assertions
6
- that run in CI, in milliseconds, with no runtime dependencies.
7
-
8
- ```md
9
- The retired payment gateway is gone from the service layer.
10
-
11
- <!-- @assert-absence target="src/services" symbol="LegacyPaymentGateway" -->
12
- ```
13
-
14
- If someone reintroduces `LegacyPaymentGateway`, CI fails and points at the line
15
- of the ADR that promised it was gone.
16
-
17
- ---
18
-
19
- ## Why this exists
20
-
21
- Architecture Decision Records, RFCs and technical briefs are full of hard claims
22
- about a repository:
23
-
24
- - *"`LegacyPaymentGateway` no longer exists anywhere in the service layer."*
25
- - *"There is exactly one `UserSessionManager`."*
26
- - *"No secret is ever read from `process.env` outside `src/config`."*
27
-
28
- Every one of those claims is true on the day it is written and unverified from
29
- the day after. Prose does not fail a build. Six months later the document is
30
- still confidently asserting something the code stopped doing in March.
31
-
32
- That has always been an annoyance. It became a defect multiplier the moment
33
- coding agents started reading these documents as ground truth. An agent that
34
- reads *"module X does not exist yet"* will happily build a second X. A reviewer
35
- who trusts *"this symbol occurs zero times"* skips the grep. Stale documentation
36
- is now executable in the worst sense: it executes inside someone's head, or
37
- inside a model's context window, and produces work that has to be thrown away.
38
-
39
- spec-guard makes the claims executable in the good sense. It reads directives
40
- written as ordinary HTML comments - invisible in every Markdown renderer - and
41
- checks them against the real tree with ripgrep. A stale claim becomes a failing
42
- build with a line number, not a landmine.
43
-
44
- ## Quickstart
45
-
46
- ```bash
47
- npm install --save-dev @descent-vtt/spec-guard
48
- npx spec-guard "docs/**/*.md"
49
- ```
50
-
51
- **The package is scoped; the command is not.** Once installed, the binary is
52
- plain `spec-guard`, so `npx spec-guard`, `npm scripts` and a global install all
53
- use that name:
54
-
55
- ```bash
56
- npm install -g @descent-vtt/spec-guard # then: spec-guard "docs/**/*.md"
57
- ```
58
-
59
- Without a local install, `npx` needs the full package name - `npx spec-guard`
60
- on its own would resolve to a different package on the registry:
61
-
62
- ```bash
63
- npx @descent-vtt/spec-guard "docs/**/*.md"
64
- ```
65
-
66
- Add a directive to any Markdown file, directly under the sentence it makes
67
- executable:
68
-
69
- ```md
70
- ## Decision
71
-
72
- Session state has exactly one owner.
73
-
74
- <!-- @assert-count target="src/" symbol="UserSessionManager" expected="1" -->
75
- ```
76
-
77
- Run it:
78
-
79
- ```bash
80
- npx spec-guard "docs/**/*.md" --verbose
81
- ```
82
-
83
- ```text
84
- spec-guard 1 spec · 1 assertion · ripgrep
85
-
86
- ✔ docs/adr/0007-sessions.md:5 @assert-count "UserSessionManager" (1 match) in src
87
-
88
- 1 passed · 7ms
89
- ✔ every spec assertion holds
90
- ```
91
-
92
- And when it drifts:
93
-
94
- ```text
95
- ✖ docs/adr/0007-sessions.md:5 @assert-count
96
- "UserSessionManager" must appear exactly 1 time in src
97
- expected exactly 1 match, found 3
98
- src/auth/UserSessionManager.ts:12:14 export class UserSessionManager {
99
- src/legacy/sessions.ts:4:22 import { UserSessionManager } from '../auth';
100
- src/workers/refresh.ts:9:10 const manager = new UserSessionManager();
101
-
102
- 0 passed · 1 failed · 9ms
103
- ```
104
-
105
- Exit code 1. The ADR is now a test.
106
-
107
- ## Directives
108
-
109
- Every directive is an HTML comment. It may span multiple lines. Attribute values
110
- may use double or single quotes, and a bare attribute means `="true"`.
111
-
112
- ### `@assert-absence` - this symbol is gone
113
-
114
- ```md
115
- <!-- @assert-absence target="src/controllers,src/services" symbol="LegacyPaymentGateway" -->
116
- <!-- @assert-absence target="src/" symbol="STRIPE_SECRET_KEY" expected="0" -->
117
- <!-- @assert-absence target="src/" symbol="TODO" expected="5" reason="burn down the backlog" -->
118
- ```
119
-
120
- Fails when the symbol occurs more than `expected` times (default `0`).
121
- `max="..."` is accepted as a synonym for `expected`.
122
-
123
- ### `exclude` - everywhere except
124
-
125
- Most real rules are "nowhere except one place", not "not here":
126
-
127
- ```md
128
- <!-- @assert-absence target="src" symbol="process.env" exclude="src/config/**" -->
129
- ```
130
-
131
- That is the rule this README opens with - *no secret is ever read from
132
- `process.env` outside `src/config`* - written as one assertion instead of a
133
- hand-maintained list of every directory that is not `src/config`.
134
-
135
- `exclude` follows gitignore rules, which are **not** the same as `glob`'s:
136
-
137
- | Pattern | Excludes |
138
- | --- | --- |
139
- | `tests` | any directory or file named `tests`, at any depth, and everything under it |
140
- | `src/config` | that directory and everything under it |
141
- | `src/config/**` | the same, written explicitly |
142
- | `*.test.ts` | any file with that name shape, at any depth |
143
-
144
- The difference is deliberate. `glob="*.ts"` filters files, so basename matching
145
- is what you want; `exclude="tests"` means the directory, because that is what
146
- people mean when they write it - and it is what `rg -g '!tests'` does. Both
147
- engines implement the same rule, and a parity matrix asserts they agree on it.
148
-
149
- List attributes accept commas or whitespace, so both of these work:
150
-
151
- ```md
152
- <!-- @assert-absence target="src" symbol="TODO" exclude="src/legacy/** tests" -->
153
- <!-- @assert-absence target="src" symbol="TODO" exclude="src/legacy/**,tests" -->
154
- ```
155
-
156
- (A path containing a space therefore cannot be written; there is no quoting
157
- inside an attribute value.)
158
-
159
- ### `comments` - the note about a deletion is not the deletion
160
-
161
- You delete a symbol and leave the explanation where the next person will look:
162
-
163
- ```ts
164
- // LegacyThing was removed in ADR-398; do not reintroduce it.
165
- ```
166
-
167
- A plain text search reads that comment as an occurrence, so the assertion that
168
- keeps the symbol deleted fails on the sentence proving it was deleted. Both
169
- fixes are bad: delete the note and lose the reason, or delete the rule.
170
-
171
- So matches inside comments do not count:
172
-
173
- ```md
174
- <!-- comments are ignored by default -->
175
- <!-- @assert-absence target="src" symbol="LegacyThing" -->
176
-
177
- <!-- ...unless you ask for them -->
178
- <!-- @assert-absence target="src" symbol="Copyright" comments="include" -->
179
- ```
180
-
181
- `comments="include"` is right for assertions that really are about text — a
182
- licence header, or a name that must appear nowhere in the repository at all.
183
-
184
- Because this is the one thing that can turn a failing run green without anyone
185
- touching code, a run that passed this way says so:
186
-
187
- ```
188
- ⚠ 1 match inside comments was not counted; add comments="include" to count it
189
-
190
- 1 passed · 7ms
191
- ✔ every spec assertion holds
192
- ```
193
-
194
- Comment syntax is known for around 59 extensions across 9 families (JS/TS, C,
195
- C#, Rust, Go, Python-style `#`, SQL-style `--`, markup, and formats with no
196
- comments at all). Strings are tracked too, because `//` inside a URL is not a
197
- comment and reading it as one would hide real code. Where spec-guard is unsure —
198
- an unknown extension, an unterminated literal — the text counts as code, and the
199
- report says which files it could not classify. A match wrongly kept is a visible
200
- failure you can argue with; a match wrongly dropped is a lie.
201
-
202
- ## What gets searched
203
-
204
- An assertion is worth exactly as much as the set of files behind it, so
205
- spec-guard is explicit about that set and never quietly narrows it.
206
-
207
- **Four directory names are skipped**, and nothing else:
208
-
209
- | Skipped | Why |
210
- | --- | --- |
211
- | `.git`, `.hg`, `.svn` | version-control stores hold compressed copies of code you deleted on purpose |
212
- | `node_modules` | code you did not write, which your architecture rules are not about |
213
-
214
- Everything else is searched. That includes **hidden directories** - `.github`,
215
- `.husky`, `.claude-rules`, `.agents` - because that is where CI, hooks and agent
216
- rules live, and a rule that cannot see your workflow files is not enforcing much.
217
- It also includes `dist`, `build`, `out` and `coverage`, because spec-guard
218
- cannot tell build output from a directory of build scripts, and guessing wrong
219
- means a rule silently stops covering anything.
220
-
221
- `.gitignore` is not consulted. It describes what git should carry, not what a
222
- rule covers - and ripgrep applies it only inside a git repository, so honouring
223
- it made the same tree answer differently depending on whether a `.git` directory
224
- happened to exist above it.
225
-
226
- To narrow scope, say so in the assertion:
227
-
228
- ```md
229
- <!-- @assert-absence target="src" symbol="TODO" exclude="dist coverage" -->
230
- ```
231
-
232
- `--no-default-skips` removes even those four, for a run that has to be certain.
233
-
234
- ### Nothing is skipped quietly
235
-
236
- A file spec-guard could not read, or one whose bytes are not text but which
237
- contained the symbol anyway, is a gap in the answer rather than a detail of it.
238
- Those are reported, and `--strict` fails on them:
239
-
240
- ```text
241
- ✖ docs/adr.md:3 @assert-absence
242
- "ApiKey" must not appear in .
243
- expected no matches, found 0, and 1 file could not be inspected
244
- ⚠ 1 match in 1 binary file not counted: build/app.bin
245
- ```
246
-
247
- The counts are in `--json` too, as `skipped` on each result.
248
-
249
- ### `@assert-count` - this symbol occurs exactly / at least / at most N times
250
-
251
- ```md
252
- <!-- @assert-count target="src/" symbol="UserSessionManager" expected="1" -->
253
- <!-- @assert-count target="src/ui/" symbol="PrimaryButton" min="1" -->
254
- <!-- @assert-count target="src/core/" symbol="DeprecatedHelper" max="3" -->
255
- <!-- @assert-count target="src/" symbol="Repository" min="2" max="10" -->
256
- ```
257
-
258
- Requires `expected`, or `min` and/or `max`. `expected` cannot be combined with
259
- `min`/`max`.
260
-
261
- An exact count is the most brittle assertion in the set: a test that merely
262
- mentions the symbol will break it. spec-guard does not silently skip test files
263
- for you - an assertion that quietly ignores part of the tree is the failure mode
264
- this tool exists to prevent - so say what you mean:
265
-
266
- ```md
267
- <!-- @assert-count target="src" symbol="UserSessionManager" expected="1" exclude="*.test.ts *.spec.ts" -->
268
- ```
269
-
270
- `min="1"` is often the better rule anyway: it says "this exists" without
271
- breaking every time someone writes a second test.
272
-
273
- ### `@assert-import-absence` / `@assert-import-count` - dependencies, not text
274
-
275
- The rules architecture documents actually contain are usually about
276
- dependencies, and a text search answers those badly in both directions - it
277
- matches a mention in a comment, and it misses `export * from '../db'`.
278
-
279
- ```md
280
- <!-- @assert-import-absence target="src/ui" module="src/db" reason="the UI talks to services, not storage" -->
281
- <!-- @assert-import-count target="src" module="axios" max="1" -->
282
- ```
283
-
284
- These read the dependency rather than the text, in **five languages**:
285
-
286
- | Language | Extensions | Reads |
287
- | --- | --- | --- |
288
- | JavaScript / TypeScript | `.js .mjs .cjs .jsx .ts .mts .cts .tsx` | `import`, `export ... from`, `require()`, `import()` |
289
- | Python | `.py .pyi` | `import a.b`, `from .rel import x`, `importlib.import_module("x")` |
290
- | Go | `.go` | `import "x"` and grouped `import ( ... )`, including aliases and `_` |
291
- | Rust | `.rs` | `use a::{b, c}` with nested groups, `pub use`, `extern crate` |
292
- | C# | `.cs .csx` | `using`, `using static`, `global using`, `using X = A.B` |
293
-
294
- No parser and no new dependency. JavaScript gets a full tokenizer because a
295
- module reference can appear anywhere in an expression; the other four are read
296
- by masking comments and string literals with the classifier from
297
- [ADR-0006](docs/adr/0006-comment-classification.md) and then reading statements
298
- off what is left. Tree-sitter would have cost 94 MB unpacked against this
299
- package's 0.33 MB - [ADR-0008](docs/adr/0008-polyglot-imports.md) has the
300
- measurements, and the list of things a real parser would genuinely see that
301
- this does not.
302
-
303
- Module patterns are matched against the resolved `/`-separated path *and*
304
- against the specifier as written, so `module="System.Text.Json"` and
305
- `module="System/Text/Json"` both work. Matching is case-sensitive, so a rule
306
- spanning C# and Go needs both conventions:
307
- `module="app/db/** App/Db/**"`.
308
-
309
- The unit counted is **files** that depend on the module. For JavaScript and
310
- TypeScript it understands:
311
-
312
- | Form | Counted |
313
- | --- | --- |
314
- | `import { A } from 'x'`, `import 'x'`, `import * as ns from 'x'` | yes |
315
- | `import type { A } from 'x'` | yes, unless `types="ignore"` |
316
- | `export * from 'x'`, `export { A } from 'x'` | yes - a re-export is a dependency, and a barrel file is how layering rules usually get broken |
317
- | `require('x')` | yes, with a literal argument |
318
- | `import('x')` | yes, with a literal argument |
319
- | `import(name)`, `require(expr)` | **no - reported, see below** |
320
-
321
- Relative specifiers are resolved by path arithmetic, so `module="src/db"`
322
- matches `../db/client.js` seen from `src/ui/`, and Python's `from ..core import
323
- x` resolves against the importing file the same way. There is deliberately no
324
- filesystem resolution: tsconfig `paths` aliases and package `exports` maps are
325
- not followed, so an alias is written out as itself (`module="@app/db"`).
326
-
327
- **It tells you what it could not see.** Roughly 1% of module references in real
328
- code are dynamic, and about 0.09% of files defeat the tokenizer outright. Both
329
- are reported rather than counted as clean:
330
-
331
- ```text
332
- ✔ docs/adr.md:1 @assert-import-absence "src/db" (0 matches) in src/core
333
- ⚠ 1 module reference could not be resolved statically
334
- ⚠ src/core/plugin.ts:1 import(name)
335
- ```
336
-
337
- The count is still true of everything that could be seen; the warning is what
338
- stops it being mistaken for a complete answer. `--strict` turns those warnings
339
- into failures. Files in scope that are not JavaScript or TypeScript are counted
340
- and reported too, so a rule pointed at the wrong tree says "analysed 2 of 3
341
- files" rather than quietly passing - and if *none* of them can be read, the
342
- assertion fails rather than passing on an empty analysis.
343
-
344
- [ADR-0005](docs/adr/0005-import-assertions.md) has the measurements and the
345
- reasoning behind each boundary.
346
-
347
- ### `@assert-present` - this file exists
348
-
349
- ```md
350
- <!-- @assert-present file="SECURITY.md" -->
351
- <!-- @assert-present file="config/production.json,config/staging.json" -->
352
- ```
353
-
354
- Passes when every listed path exists relative to `--root`. Directories count.
355
-
356
- ### Attributes
357
-
358
- | Attribute | Applies to | Meaning |
359
- | --- | --- | --- |
360
- | `target` | absence, count | Comma-separated paths to search, relative to `--root`. Default `.` |
361
- | `symbol` | absence, count | The literal string to search for (or a regex with `regex="true"`) |
362
- | `file` | present | Comma-separated paths that must exist |
363
- | `expected` | absence, count | Upper bound for absence; exact count for count |
364
- | `min` / `max` | count (`max` also on absence) | Inclusive bounds |
365
- | `glob` | absence, count | Include-only file filters, e.g. `*.ts,*.tsx` (ripgrep `-g` semantics) |
366
- | `exclude` | absence, count | Paths to leave out, gitignore-style: `src/config/**`, `tests`, `*.test.ts` |
367
- | `regex` | absence, count | Treat `symbol` as a regular expression |
368
- | `word` | absence, count | Require word boundaries, so `Primary` does not match `PrimaryButton` |
369
- | `ignore-case` | absence, count | Case-insensitive matching |
370
- | `comments` | absence, count | `ignore` (default) or `include` for matches inside comments |
371
- | `module` | import assertions | Which dependency, matched like `exclude` |
372
- | `types` | import assertions | `include` (default) or `ignore` for `import type` |
373
- | `allow-empty` | absence, count, import assertions | Tolerate a scope that holds no files. Off by default - see below |
374
- | `baseline` | absence, import-absence | Known violations that do not count: `path` or `path:count` |
375
- | `ratchet` | absence, import-absence | `two-sided` (default) or `one-way` - see below |
376
- | `reason` | all | Human-readable justification, printed on failure |
377
-
378
- Unknown attributes are an error, not a shrug: `expct="1"` fails the run instead
379
- of silently asserting nothing.
380
-
381
- ### `allow-empty` - an assertion that covers nothing is a failure
382
-
383
- ```md
384
- <!-- @assert-absence target="services" symbol="Legacy" glob="*.ts" -->
385
- ```
386
-
387
- If `services/` holds no `.ts` file, that rule passes every time, forever,
388
- without inspecting anything - and in the report it is indistinguishable from a
389
- rule that inspected a thousand files and found nothing. So it fails instead:
390
-
391
- ```text
392
- ✖ docs/adr.md:3 @assert-absence
393
- "Legacy" must not appear in services
394
- no files were inspected, so this assertion verified nothing
395
- (add allow-empty="true" if that is expected)
396
- ```
397
-
398
- The usual causes are a `glob` matching no extension in the tree, an `exclude`
399
- that swallowed the target, or a directory somebody emptied. Where covering
400
- nothing yet is the honest state of the world - a rule written before the code
401
- it guards - `allow-empty="true"` says so, and `--allow-empty-scope` says it for
402
- a whole run.
403
-
404
- ### `baseline` - adopting a rule the codebase already breaks
405
-
406
- A strict rule introduced into a mature codebase lands on violations that already
407
- exist. `exclude` turns those into a permanent blind spot, and `expected="5"`
408
- cannot tell "one fixed, one added" from "nothing happened". A baseline names
409
- them:
410
-
411
- ```md
412
- <!-- @assert-absence target="src" symbol="LegacyGateway"
413
- baseline="src/legacy/gateway.ts:2
414
- src/legacy/adapter.ts" -->
415
- ```
416
-
417
- The rule now holds when no file outside the list matches and no file inside it
418
- matches more than it declares. A bare path means one match.
419
-
420
- **It ratchets in both directions.** New debt fails, obviously. So does debt that
421
- has been *paid* and left on the list, because an entry the code no longer
422
- supports is a spec asserting something untrue:
423
-
424
- ```text
425
- expected no matches, found 0; the baseline is out of date and must be
426
- pruned: src/legacy/adapter.ts (no longer matches)
427
- ```
428
-
429
- The fix is deleting the line the message names. `ratchet="one-way"` relaxes that
430
- half to a report-only note. Every run says how many matches the baseline
431
- excluded, because a pass bought by an exemption is never silent.
432
-
433
- To adopt a rule on a codebase that already violates it:
434
-
435
- ```bash
436
- spec-guard docs/adr.md --print-baseline
437
- ```
438
-
439
- prints the attribute that would exempt exactly today's violations, for you to
440
- paste in. It prints; it does not edit. [ADR-0009](docs/adr/0009-debt-baselines.md)
441
- explains why that distinction is the whole design, and why there is no `--fix`
442
- for architecture rules.
443
-
444
- ## CLI
445
-
446
- ```bash
447
- spec-guard [patterns...] [options]
448
- ```
449
-
450
- | Option | Description |
451
- | --- | --- |
452
- | `-r, --root <path>` | Codebase root that assertions resolve against (default: cwd) |
453
- | `-v, --verbose` | Print passing assertions too |
454
- | `--fail-fast` | Stop at the first failing assertion |
455
- | `--json` | Machine-readable report on stdout (same as `--format json`) |
456
- | `--format <human\|json\|sarif>` | Output format. `sarif` uploads to GitHub code scanning |
457
- | `--engine <auto\|rg\|js>` | Search engine (default `auto`: scanner for small trees, ripgrep for big ones) |
458
- | `--strict` | Treat analysis that could not be completed as a failure |
459
- | `--allow-missing-targets` | Warn instead of failing when a `target` path does not exist |
460
- | `--allow-empty-scope` | Warn instead of failing when an assertion inspects no files |
461
- | `--print-baseline` | Print the `baseline="..."` that would exempt today's violations, and exit |
462
- | `--no-default-skips` | Search `.git`, `.hg`, `.svn` and `node_modules` too |
463
- | `--include-specs` | Also count matches inside the spec files themselves |
464
- | `--max-snippets <n>` | Failure snippets per assertion (default 5) |
465
- | `--concurrency <n>` | Search passes in flight at once (default 8) |
466
- | `--allow-empty` | Exit 0 when no spec file matched the patterns (about the run, not an assertion) |
467
- | `--color` / `--no-color` | Force colour on or off (`NO_COLOR` honoured) |
468
-
469
- Patterns are expanded by spec-guard itself, so quoted globs behave identically
470
- on Windows, macOS and Linux. A directory expands to the Markdown files in it.
471
-
472
- ### Exit codes
473
-
474
- | Code | Meaning |
475
- | --- | --- |
476
- | `0` | Every assertion held |
477
- | `1` | An assertion failed, or a directive was malformed |
478
- | `2` | spec-guard could not run: bad usage, no spec files matched, `--engine rg` with no ripgrep |
479
-
480
- ## CI integration
481
-
482
- ```yaml
483
- # .github/workflows/specs.yml
484
- name: Specs
485
- on: [push, pull_request]
486
-
487
- jobs:
488
- spec-guard:
489
- runs-on: ubuntu-latest
490
- steps:
491
- - uses: actions/checkout@v5
492
- - uses: actions/setup-node@v5
493
- with:
494
- node-version: '22'
495
- - run: npx @descent-vtt/spec-guard "docs/**/*.md" "README.md" --verbose
496
- ```
497
-
498
- That job needs nothing else installed. GitHub-hosted runners do **not** ship
499
- ripgrep on `PATH` - spec-guard's own CI reports `engine: javascript` there - so
500
- the fallback is what actually runs, and it produces identical results. If your
501
- repository is large enough that you want ripgrep's speed, install it first:
502
-
503
- ```yaml
504
- - run: sudo apt-get update && sudo apt-get install -y ripgrep
505
- ```
506
-
507
- Or point spec-guard at a binary you already have with `SPEC_GUARD_RG=/path/to/rg`.
508
-
509
- As a pre-commit hook (assuming a local install, so the bare command resolves):
510
-
511
- ```bash
512
- npx spec-guard "docs/**/*.md" --fail-fast
513
- ```
514
-
515
- ### Annotations on the pull request
516
-
517
- `--format sarif` writes a SARIF 2.1.0 document, which GitHub turns into a
518
- comment on the offending line:
519
-
520
- ```yaml
521
- - run: npx @descent-vtt/spec-guard "docs/**/*.md" --format sarif > spec-guard.sarif || true
522
- - uses: github/codeql-action/upload-sarif@v3
523
- with:
524
- sarif_file: spec-guard.sarif
525
- category: spec-guard
526
- ```
527
-
528
- That job needs `permissions: security-events: write`. One alert per broken rule
529
- rather than per match - the thing that broke is the rule - anchored on the first
530
- offending line, with the directive as a related location because that is often
531
- where the fix goes. Alerts carry a fingerprint derived from what the assertion
532
- is about rather than where its matches landed, so inserting a line above a
533
- violation does not close the alert and open a new one.
534
-
535
- **There is no language server, and that is a decision rather than a gap.**
536
- spec-guard's claims are about a whole repository - "this symbol appears nowhere
537
- in `src`" - and a language server is handed one buffer at a time. Answering a
538
- repository-wide question on every keystroke means rescanning the tree on every
539
- keystroke; the alternative is answering a smaller question and calling it the
540
- same one. The CLI does a full run of this repository in about 130 ms, so a
541
- pre-commit hook or a watch loop already closes the feedback gap without a
542
- daemon, an extension per editor, or a protocol version matrix.
543
-
544
- ## How it works
545
-
546
- ```text
547
- parser ──▶ Directive[] ──▶ runner ──▶ Assertion[] ──▶ engine ──▶ AssertionResult[] ──▶ reporter
548
- (pure) (I/O) (rg | js) (pure)
549
- ```
550
-
551
- 1. **Parse.** Markdown is scanned for `<!-- @assert-* -->` comments. Fenced code
552
- blocks and inline code spans are masked first, so documentation that shows
553
- the syntax (like this README) never executes it.
554
- 2. **Resolve.** Each directive becomes a typed assertion. Bad numbers, unknown
555
- booleans, unknown attributes and paths escaping `--root` are rejected here,
556
- before any I/O.
557
- 3. **Search.** Assertions that share a target list and flags are answered by a
558
- *single* pass over the tree.
559
- 4. **Report.** ANSI output with the spec location, the expectation, the observed
560
- count, and up to five real snippets; or JSON for tooling.
561
-
562
- ### Performance
563
-
564
- Scanning a tree costs about the same whether you look for one symbol or twenty,
565
- so spec-guard groups assertions by target set and flags and answers each group
566
- in one pass.
567
-
568
- `auto` then picks between the two engines **per search group**, because they
569
- have different shapes of cost: ripgrep is dominated by process startup and
570
- barely notices tree size, while the built-in scanner has no startup cost and
571
- grows linearly. Starting a process to search a handful of files is a bad trade.
572
- Re-measured for 0.4.0 on Windows 11 / Node 24, median of three, one symbol over
573
- a synthetic tree:
574
-
575
- | files | scanner | ripgrep | |
576
- | --- | --- | --- | --- |
577
- | 100 | **17 ms** | 214 ms | scanner, 12.9x |
578
- | 500 | **70 ms** | 263 ms | scanner, 3.8x |
579
- | 1,000 | 216 ms | **181 ms** | ripgrep, 1.2x |
580
- | 3,000 | 735 ms | **222 ms** | ripgrep, 3.3x |
581
- | 5,000 | 1,046 ms | **241 ms** | ripgrep, 4.3x |
582
-
583
- The decision uses a bounded enumeration as its probe: spec-guard walks the
584
- target set until it either finishes - in which case the file list is already in
585
- hand and the scanner runs against it, with no process and no second walk - or
586
- exceeds a budget, in which case ripgrep takes over. The crossover above sits
587
- between 500 and 1,000 files, which is what the Windows budget of 512 encodes;
588
- on Linux it is far lower, because what is really being measured is process
589
- spawn cost. `scripts/bench-engines.mjs` reproduces this, and
590
- [ADR-0004](docs/adr/0004-adaptive-engine.md) has the full tables.
591
-
592
- Treat single measurements from one machine with suspicion. On the development
593
- machine used here, an absence assertion over 2,000 files measures 303 ms with
594
- the scanner and 399 ms with ripgrep, but ripgrep's own spread across five runs
595
- was 379-1039 ms - wide enough that the two are not really distinguishable at
596
- that size. The order-of-magnitude differences at the ends of the table are the
597
- part worth trusting.
598
-
599
- 0.5.0 added four language analysers, a scope probe on every assertion and a
600
- baseline pass, so the obvious question is what that cost. Both versions were run
601
- against the same synthetic 1,000-file tree, alternating rounds inside one
602
- process so that this machine's load - which varies by a factor of two over a
603
- session - falls on both equally:
604
-
605
- | | median | min |
606
- | --- | --- | --- |
607
- | 0.4.0 | 908.8 ms | 493.7 ms |
608
- | 0.5.0 | 913.4 ms | 496.6 ms |
609
-
610
- 0.5% apart, against a run-to-run spread far wider than that. The two versions
611
- also returned identical counts for all five assertions, which is the more
612
- useful half of the result.
613
-
614
- 0.5.1 replaced three hand-maintained "are these the same question" field lists
615
- with one, and the surviving version serialises the exclude set where the old one
616
- compared it by identity - a hot path made slower on purpose, so it was measured
617
- rather than assumed. Same tree, same alternating rounds:
618
-
619
- | | median | min |
620
- | --- | --- | --- |
621
- | 0.5.0 | 925.1 ms | 432.1 ms |
622
- | 0.5.1 | 830.6 ms | 428.5 ms |
623
-
624
- The medians are 10% apart and the minima are level, which is this machine's
625
- noise band doing most of the talking - the honest reading is "no regression",
626
- not "10% faster". The plausible mechanism for any real gain is that the runner
627
- now groups by the engine's own definition of a shared pass, so a group of five
628
- where two requests disagree is run as three and two rather than being refused
629
- whole and run as five. Counts identical across all five assertions, again.
630
-
631
- Two honest caveats:
632
-
633
- - **The reported engine is the one that ran, not the one available.** On a small
634
- repository `--json` reports `"engine": "javascript"` even with ripgrep
635
- installed. That is the optimisation working, not a failure to find `rg`.
636
- - **The engines are one implementation, not two that agree.** This used to say
637
- that ripgrep was the reference and the scanner merely matched it "on
638
- everything the test suite covers", with a note that the two treated
639
- `.gitignore` and ignored directories differently and that you should pin an
640
- engine if it mattered. It did matter: on a tree with eight copies of a symbol
641
- the scanner found two and ripgrep found four. Since 0.4.0 ripgrep answers only
642
- *which files contain this text*, and the scanner does all the counting,
643
- classification and reporting for both. `--engine` now changes how long a run
644
- takes and nothing about its verdict, and the suite asserts that on trees built
645
- from every case that used to split them.
646
-
647
- ## Programmatic API
648
-
649
- ```ts
650
- import { runSpecGuard, formatReport } from '@descent-vtt/spec-guard';
651
-
652
- const report = await runSpecGuard({
653
- patterns: ['docs/**/*.md'],
654
- root: process.cwd(),
655
- });
656
-
657
- if (!report.ok) {
658
- console.error(formatReport(report, { color: true, verbose: false }));
659
- process.exitCode = 1;
660
- }
661
- ```
662
-
663
- `runSpecGuard` returns the full report: every assertion, its bounds, the
664
- observed count, match locations, warnings and timings. The parser
665
- (`parseDirectives`) and reporter (`formatReport`, `formatJson`) are pure
666
- functions you can use on their own.
667
-
668
- ## Design decisions
669
-
670
- This tool was specified loosely and built opinionatedly. Where the
671
- implementation departs from the obvious reading of the brief, here is why.
672
-
673
- **Spec files are excluded from their own searches.** An ADR that says
674
- "`LegacyGateway` must not appear" contains the string `LegacyGateway`. Without
675
- this rule, absence assertions would fail on the document asserting them - the
676
- single most confusing possible first-run experience. `--include-specs` restores
677
- the naive behaviour, and counts the prose of your specs; the directives
678
- themselves stay uncounted, because a directive is an HTML comment and a rule
679
- whose own text trips it can never be satisfied. `comments="include"` counts even
680
- those.
681
-
682
- **Fenced code and inline code are masked before parsing.** A README documenting
683
- the syntax must not execute it. Masking preserves byte offsets, so reported line
684
- numbers stay exact. (This is subtle: pairing backtick runs the naive way
685
- desynchronises after a stray unmatched run and un-masks real prose. spec-guard
686
- uses CommonMark's equal-length pairing rule, and there is a regression test.)
687
-
688
- **Malformed directives fail the run.** A typo like `expct="1"` could be ignored
689
- as "not a directive". It is instead an error, because a spec tool whose typos
690
- silently assert nothing is worse than no spec tool.
691
-
692
- **Four attributes were added beyond the original brief** - `word`, `regex`,
693
- `glob` and `ignore-case` - because `symbol="Primary"` matching `PrimaryButton`
694
- is the first thing every user hits, and `reason` because a failure message
695
- should say *why* the rule exists.
696
-
697
- **`min`/`max` are accepted on `@assert-absence` too.** "At most 5 TODOs" is an
698
- absence claim with a budget, and burning a budget down is a real workflow.
699
-
700
- **The engine is detected lazily.** Probing with `rg --version` up front costs a
701
- process spawn (~27 ms on Windows) on the critical path of every run, including
702
- runs where ripgrep is missing. Discovering its absence from the first real
703
- search is free.
704
-
705
- **Assertions that share a target list share one pass over the tree.** This used
706
- to need a proof: merging literals into a ripgrep alternation can lose matches
707
- through containment (`Primary` / `PrimaryButton`) and dovetailing (`abc` / `cd`
708
- in `abcd`), so spec-guard checked for both and fell back to separate passes.
709
- Since 0.4.0 it does not need the proof, because ripgrep no longer counts
710
- anything - it answers only *which files contain this text*, and the scanner
711
- counts each pattern separately over the shared file contents. The batching
712
- checks were deleted along with the risk they guarded.
713
-
714
- **Exit code 2 exists.** "Your specs failed" and "spec-guard could not run" are
715
- different facts, and CI should be able to tell them apart.
716
-
717
- **A missing `target` fails the run.** It used to warn and search what was left,
718
- which meant an assertion pointed at a renamed directory searched nothing, found
719
- nothing, and reported success — the exact shape of a green check that verified
720
- nothing. `--allow-missing-targets` restores the old behaviour for repositories
721
- where a path is legitimately optional.
722
-
723
- **Hidden directories are searched, and `.gitignore` is not consulted.** An
724
- audit of 0.3.0 found the tool reporting a clean pass on a repository whose
725
- forbidden symbol sat in `.github/workflows/ci.yml`; the scanner and ripgrep also
726
- disagreed with each other, finding two matches and four on the same tree. Scope
727
- is now one policy that both engines are driven by, the skip list is four names
728
- long, and anything spec-guard could not inspect is reported rather than assumed
729
- clean ([ADR-0007](docs/adr/0007-search-scope.md)).
730
-
731
- **Comments are excluded by default, and the exclusion is reported.** Counting
732
- the note that records a deletion as an occurrence of the thing deleted punishes
733
- the documentation this tool exists to keep honest ([ADR-0006](docs/adr/0006-comment-classification.md)).
734
- The reverse risk — a rule that quietly stops checking anything because every
735
- match now sits in a comment — is why every run says how many matches it dropped.
736
-
737
- **An assertion that inspected no files fails.** It is the same defect as a
738
- missing `target` seen from a different angle: a rule whose scope is empty passes
739
- forever and reads exactly like a rule that found nothing. Turning this on found
740
- a vacuous assertion inside this repository's own test suite on the first run.
741
- `allow-empty="true"` covers the honest case of a rule written before the code it
742
- guards.
743
-
744
- **Four more languages, and no parser.** Import assertions read Python, Go, Rust
745
- and C# as well as JavaScript. Not by adding four tokenizers - by reusing the
746
- comment and string lexer that already existed, masking the source with it, and
747
- reading statements off what is left. Tree-sitter would have been the modern
748
- answer and costs 94 MB unpacked against this package's 0.33 MB; the four things
749
- a real parser would genuinely see that this cannot are listed in
750
- [ADR-0008](docs/adr/0008-polyglot-imports.md) rather than glossed over.
751
-
752
- **There is no `--fix`.** Every edit a machine can make to a failing boundary
753
- assertion is an edit that records the rule no longer holding: widen the bound,
754
- add an exclusion, append to the baseline, insert an ignore comment. A one-flag
755
- path from red to green is a bad button for a person and a much worse one for an
756
- agent whose loop terminates on a green build. `--print-baseline` prints what you
757
- could paste; it does not paste it ([ADR-0009](docs/adr/0009-debt-baselines.md)).
758
-
759
- ## spec-guard checks itself
760
-
761
- The invariants in [`docs/adr/0001-invariants.md`](docs/adr/0001-invariants.md)
762
- and [`docs/adr/0002-directive-format.md`](docs/adr/0002-directive-format.md) are
763
- executed against this repository on every CI run. The CLI never calls
764
- `console.log`, nothing outside the engine spawns a process, and the parser and
765
- reporter never touch the filesystem - because those documents say so, and the
766
- build fails if they stop being true.
767
-
768
- This README is executable too:
769
-
770
- <!-- @assert-present file="src/parser.ts,src/engine.ts,src/reporter.ts,src/runner.ts,src/cli.ts,src/comments.ts" -->
771
-
772
- ## Development
773
-
774
- ```bash
775
- npm install
776
- npm run build # tsc -> dist/
777
- npm test # vitest
778
- npm run test:coverage
779
- npm run test:mutation # stryker (~16 min in CI; over two hours locally)
780
- npm run lint # tsc --noEmit
781
- npm run selfcheck # run spec-guard on its own docs
782
- ```
783
-
784
- The test suite runs every assertion case against **both** engines and asserts
785
- they agree, so the fallback cannot quietly drift from ripgrep. `@vscode/ripgrep`
786
- is a devDependency purely so that the ripgrep path is exercised on every
787
- platform in CI, including machines that have no `rg` on PATH; it ships a
788
- prebuilt binary and is never a runtime dependency.
789
-
790
- ### Mutation testing
791
-
792
- Coverage says a line ran. It does not say an assertion would notice if the line
793
- behaved differently. This repository measures the difference: line coverage is
794
- **100%**, and **97.65%** of 4,299 mutants are killed in CI, with no file below
795
- 95%. The second number is the one worth reading - and it is the ceiling, not a
796
- way-point: every surviving mutant has been checked individually and the 101 that
797
- are left produce byte-identical output.
798
-
799
- That gap is the point. The first run scored 77.23%, and the weakest file was the
800
- reporter at 65.48% - not because it lacked tests, but because its tests were
801
- almost all `toContain` against colourless output. A mutant could prepend a junk
802
- line, drop a colour, or turn `remaining > 0` into `remaining >= 0` and every
803
- test still passed. Exact whole-output comparison took it to 87.30%. The engine
804
- and the walker were then rewritten for testability rather than papered over with
805
- more tests, which is what moved them from 75%/78% to 83%/93%.
806
-
807
- The most recent pass took every module to its ceiling, and its value was not the
808
- number. Writing down four contracts that had only ever been checked through
809
- their effect on a match count turned up four wrong answers: two assertions on
810
- one symbol answered each other's comment handling, an unreadable directory went
811
- unreported on any repository small enough to scan in process, snippets from CRLF
812
- files carried a carriage return into the terminal, and an invalid pattern was
813
- reported with its error message twice. Seventeen branches turned out to be
814
- unable to decide anything and were deleted rather than pinned; thirty-five
815
- negative controls - each defect reintroduced one at a time - confirm the suite
816
- goes red for every one, and the changes meant to be invisible were checked
817
- against 6,976 files of real code, reference for reference, against the published
818
- 0.5.0 build.
819
-
820
- Then the survivors themselves were checked rather than excused. All 130 were
821
- applied one at a time and run through a fingerprint of 4,176 observations, and
822
- **29 of them turned out not to be equivalent at all** - a Rust brace counter
823
- that only matters when a second statement follows, a type-only test that marks
824
- `import A, { B } from 'x'` type-only when loosened, a comment check that opens a
825
- block comment on `2*3`. All 29 are now tested and all 29 die. All of it is in
826
- [ADR-0003](docs/adr/0003-mutation-testing.md), including a Stryker limitation
827
- found on the way: a mutant that stops a test file *loading* is reported as
828
- survived even though the suite is in fact killing it.
829
-
830
- CI runs it in **two tiers**, both gated at 97%. Branches and pull requests run
831
- Stryker incrementally, reusing the verdict for any mutant whose source and
832
- covering tests are both unchanged. Pushes to `main`, the weekly schedule and
833
- manual runs do the full sweep, which is the authoritative number and the one
834
- quoted above; a full sweep is also what publishes the cache the branches start
835
- from, so an incremental verdict can never be built on another incremental
836
- verdict.
837
-
838
- The full sweep was 6m54s at 2,118 mutants, 15m51s at 3,493, and is 17m at
839
- 4,299. That growth is why the tiers exist: a check that gets quietly more
840
- expensive every release is a check somebody eventually proposes lowering.
841
-
842
- Run it locally with `npm run test:mutation` if you like, but do not calibrate
843
- anything on the result: on the Windows machine this was developed on the same
844
- suite takes over two hours against 21 minutes hosted, and it scores *higher*,
845
- because far more mutants hang there and Stryker counts a hang as a kill. Linux
846
- CI is the measurement.
847
-
848
- Eight cautionary tales are in [ADR-0003](docs/adr/0003-mutation-testing.md): a
849
- run whose score was pure fiction because the mutants were never activated, a
850
- tuning knob that lifted the score six points without adding a test, the platform
851
- gap above, the baseline being re-anchored when a tokenizer arrived, and a score
852
- that rose partly because code carrying hard-to-kill mutants was deleted rather
853
- than because tests improved, and the 0.5.0 sweep that failed the build at 83.77%
854
- because a suite can be thorough about the thing it was written to test and
855
- silent about the machinery underneath it, and the excuse that let the tokenizer
856
- sit at 73% for three releases because "that kind of code carries more equivalent
857
- mutants" sounded like judgement rather than an unchecked assumption - along with
858
- the real bugs that chasing the gate uncovered, and a class of mutant the runner
859
- reports as survived while the suite is in fact killing it.
860
-
861
- ## Requirements
862
-
863
- - Node.js 22 or newer (native ESM)
864
- - ripgrep optional - used when present, replaced by a built-in scanner when not
865
-
866
- ## License
867
-
868
- MIT
1
+ # spec-guard
2
+
3
+ **Executable architecture assertions for Markdown specs & ADRs.**
4
+
5
+ Turn the claims your design documents make about your codebase into assertions
6
+ that run in CI, in milliseconds, with no runtime dependencies.
7
+
8
+ ```md
9
+ The retired payment gateway is gone from the service layer.
10
+
11
+ <!-- @assert-absence target="src/services" symbol="LegacyPaymentGateway" -->
12
+ ```
13
+
14
+ If someone reintroduces `LegacyPaymentGateway`, CI fails and points at the line
15
+ of the ADR that promised it was gone.
16
+
17
+ ---
18
+
19
+ ## Why this exists
20
+
21
+ Architecture Decision Records, RFCs and technical briefs are full of hard claims
22
+ about a repository:
23
+
24
+ - *"`LegacyPaymentGateway` no longer exists anywhere in the service layer."*
25
+ - *"There is exactly one `UserSessionManager`."*
26
+ - *"No secret is ever read from `process.env` outside `src/config`."*
27
+
28
+ Every one of those claims is true on the day it is written and unverified from
29
+ the day after. Prose does not fail a build. Six months later the document is
30
+ still confidently asserting something the code stopped doing in March.
31
+
32
+ That has always been an annoyance. It became a defect multiplier the moment
33
+ coding agents started reading these documents as ground truth. An agent that
34
+ reads *"module X does not exist yet"* will happily build a second X. A reviewer
35
+ who trusts *"this symbol occurs zero times"* skips the grep. Stale documentation
36
+ is now executable in the worst sense: it executes inside someone's head, or
37
+ inside a model's context window, and produces work that has to be thrown away.
38
+
39
+ spec-guard makes the claims executable in the good sense. It reads directives
40
+ written as ordinary HTML comments - invisible in every Markdown renderer - and
41
+ checks them against the real tree with ripgrep. A stale claim becomes a failing
42
+ build with a line number, not a landmine.
43
+
44
+ ## Quickstart
45
+
46
+ ```bash
47
+ npm install --save-dev @descent-vtt/spec-guard
48
+ npx spec-guard "docs/**/*.md"
49
+ ```
50
+
51
+ **The package is scoped; the command is not.** Once installed, the binary is
52
+ plain `spec-guard`, so `npx spec-guard`, `npm scripts` and a global install all
53
+ use that name:
54
+
55
+ ```bash
56
+ npm install -g @descent-vtt/spec-guard # then: spec-guard "docs/**/*.md"
57
+ ```
58
+
59
+ Without a local install, `npx` needs the full package name - `npx spec-guard`
60
+ on its own would resolve to a different package on the registry:
61
+
62
+ ```bash
63
+ npx @descent-vtt/spec-guard "docs/**/*.md"
64
+ ```
65
+
66
+ Add a directive to any Markdown file, directly under the sentence it makes
67
+ executable:
68
+
69
+ ```md
70
+ ## Decision
71
+
72
+ Session state has exactly one owner.
73
+
74
+ <!-- @assert-count target="src/" symbol="UserSessionManager" expected="1" -->
75
+ ```
76
+
77
+ Run it:
78
+
79
+ ```bash
80
+ npx spec-guard "docs/**/*.md" --verbose
81
+ ```
82
+
83
+ ```text
84
+ spec-guard 1 spec · 1 assertion · ripgrep
85
+
86
+ ✔ docs/adr/0007-sessions.md:5 @assert-count "UserSessionManager" (1 match) in src
87
+
88
+ 1 passed · 7ms
89
+ ✔ every spec assertion holds
90
+ ```
91
+
92
+ And when it drifts:
93
+
94
+ ```text
95
+ ✖ docs/adr/0007-sessions.md:5 @assert-count
96
+ "UserSessionManager" must appear exactly 1 time in src
97
+ expected exactly 1 match, found 3
98
+ src/auth/UserSessionManager.ts:12:14 export class UserSessionManager {
99
+ src/legacy/sessions.ts:4:22 import { UserSessionManager } from '../auth';
100
+ src/workers/refresh.ts:9:10 const manager = new UserSessionManager();
101
+
102
+ 0 passed · 1 failed · 9ms
103
+ ```
104
+
105
+ Exit code 1. The ADR is now a test.
106
+
107
+ ## Directives
108
+
109
+ Every directive is an HTML comment. It may span multiple lines. Attribute values
110
+ may use double or single quotes, and a bare attribute means `="true"`.
111
+
112
+ ### `@assert-absence` - this symbol is gone
113
+
114
+ ```md
115
+ <!-- @assert-absence target="src/controllers,src/services" symbol="LegacyPaymentGateway" -->
116
+ <!-- @assert-absence target="src/" symbol="STRIPE_SECRET_KEY" expected="0" -->
117
+ <!-- @assert-absence target="src/" symbol="TODO" expected="5" reason="burn down the backlog" -->
118
+ ```
119
+
120
+ Fails when the symbol occurs more than `expected` times (default `0`).
121
+ `max="..."` is accepted as a synonym for `expected`.
122
+
123
+ ### `exclude` - everywhere except
124
+
125
+ Most real rules are "nowhere except one place", not "not here":
126
+
127
+ ```md
128
+ <!-- @assert-absence target="src" symbol="process.env" exclude="src/config/**" -->
129
+ ```
130
+
131
+ That is the rule this README opens with - *no secret is ever read from
132
+ `process.env` outside `src/config`* - written as one assertion instead of a
133
+ hand-maintained list of every directory that is not `src/config`.
134
+
135
+ `exclude` follows gitignore rules, which are **not** the same as `glob`'s:
136
+
137
+ | Pattern | Excludes |
138
+ | --- | --- |
139
+ | `tests` | any directory or file named `tests`, at any depth, and everything under it |
140
+ | `src/config` | that directory and everything under it |
141
+ | `src/config/**` | the same, written explicitly |
142
+ | `*.test.ts` | any file with that name shape, at any depth |
143
+
144
+ The difference is deliberate. `glob="*.ts"` filters files, so basename matching
145
+ is what you want; `exclude="tests"` means the directory, because that is what
146
+ people mean when they write it - and it is what `rg -g '!tests'` does. Both
147
+ engines implement the same rule, and a parity matrix asserts they agree on it.
148
+
149
+ List attributes accept commas or whitespace, so both of these work:
150
+
151
+ ```md
152
+ <!-- @assert-absence target="src" symbol="TODO" exclude="src/legacy/** tests" -->
153
+ <!-- @assert-absence target="src" symbol="TODO" exclude="src/legacy/**,tests" -->
154
+ ```
155
+
156
+ (A path containing a space therefore cannot be written; there is no quoting
157
+ inside an attribute value.)
158
+
159
+ ### `comments` - the note about a deletion is not the deletion
160
+
161
+ You delete a symbol and leave the explanation where the next person will look:
162
+
163
+ ```ts
164
+ // LegacyThing was removed in ADR-398; do not reintroduce it.
165
+ ```
166
+
167
+ A plain text search reads that comment as an occurrence, so the assertion that
168
+ keeps the symbol deleted fails on the sentence proving it was deleted. Both
169
+ fixes are bad: delete the note and lose the reason, or delete the rule.
170
+
171
+ So matches inside comments do not count:
172
+
173
+ ```md
174
+ <!-- comments are ignored by default -->
175
+ <!-- @assert-absence target="src" symbol="LegacyThing" -->
176
+
177
+ <!-- ...unless you ask for them -->
178
+ <!-- @assert-absence target="src" symbol="Copyright" comments="include" -->
179
+ ```
180
+
181
+ `comments="include"` is right for assertions that really are about text — a
182
+ licence header, or a name that must appear nowhere in the repository at all.
183
+
184
+ Because this is the one thing that can turn a failing run green without anyone
185
+ touching code, a run that passed this way says so:
186
+
187
+ ```
188
+ ⚠ 1 match inside comments was not counted; add comments="include" to count it
189
+
190
+ 1 passed · 7ms
191
+ ✔ every spec assertion holds
192
+ ```
193
+
194
+ Comment syntax is known for around 59 extensions across 9 families (JS/TS, C,
195
+ C#, Rust, Go, Python-style `#`, SQL-style `--`, markup, and formats with no
196
+ comments at all). Strings are tracked too, because `//` inside a URL is not a
197
+ comment and reading it as one would hide real code. Where spec-guard is unsure —
198
+ an unknown extension, an unterminated literal — the text counts as code, and the
199
+ report says which files it could not classify. A match wrongly kept is a visible
200
+ failure you can argue with; a match wrongly dropped is a lie.
201
+
202
+ ## What gets searched
203
+
204
+ An assertion is worth exactly as much as the set of files behind it, so
205
+ spec-guard is explicit about that set and never quietly narrows it.
206
+
207
+ **Four directory names are skipped**, and nothing else:
208
+
209
+ | Skipped | Why |
210
+ | --- | --- |
211
+ | `.git`, `.hg`, `.svn` | version-control stores hold compressed copies of code you deleted on purpose |
212
+ | `node_modules` | code you did not write, which your architecture rules are not about |
213
+
214
+ Everything else is searched. That includes **hidden directories** - `.github`,
215
+ `.husky`, `.claude-rules`, `.agents` - because that is where CI, hooks and agent
216
+ rules live, and a rule that cannot see your workflow files is not enforcing much.
217
+ It also includes `dist`, `build`, `out` and `coverage`, because spec-guard
218
+ cannot tell build output from a directory of build scripts, and guessing wrong
219
+ means a rule silently stops covering anything.
220
+
221
+ `.gitignore` is not consulted. It describes what git should carry, not what a
222
+ rule covers - and ripgrep applies it only inside a git repository, so honouring
223
+ it made the same tree answer differently depending on whether a `.git` directory
224
+ happened to exist above it.
225
+
226
+ To narrow scope, say so in the assertion:
227
+
228
+ ```md
229
+ <!-- @assert-absence target="src" symbol="TODO" exclude="dist coverage" -->
230
+ ```
231
+
232
+ `--no-default-skips` removes even those four, for a run that has to be certain.
233
+
234
+ ### Nothing is skipped quietly
235
+
236
+ A file spec-guard could not read, or one whose bytes are not text but which
237
+ contained the symbol anyway, is a gap in the answer rather than a detail of it.
238
+ Those are reported, and `--strict` fails on them:
239
+
240
+ ```text
241
+ ✖ docs/adr.md:3 @assert-absence
242
+ "ApiKey" must not appear in .
243
+ expected no matches, found 0, and 1 file could not be inspected
244
+ ⚠ 1 match in 1 binary file not counted: build/app.bin
245
+ ```
246
+
247
+ The counts are in `--json` too, as `skipped` on each result.
248
+
249
+ ### `@assert-count` - this symbol occurs exactly / at least / at most N times
250
+
251
+ ```md
252
+ <!-- @assert-count target="src/" symbol="UserSessionManager" expected="1" -->
253
+ <!-- @assert-count target="src/ui/" symbol="PrimaryButton" min="1" -->
254
+ <!-- @assert-count target="src/core/" symbol="DeprecatedHelper" max="3" -->
255
+ <!-- @assert-count target="src/" symbol="Repository" min="2" max="10" -->
256
+ ```
257
+
258
+ Requires `expected`, or `min` and/or `max`. `expected` cannot be combined with
259
+ `min`/`max`.
260
+
261
+ An exact count is the most brittle assertion in the set: a test that merely
262
+ mentions the symbol will break it. spec-guard does not silently skip test files
263
+ for you - an assertion that quietly ignores part of the tree is the failure mode
264
+ this tool exists to prevent - so say what you mean:
265
+
266
+ ```md
267
+ <!-- @assert-count target="src" symbol="UserSessionManager" expected="1" exclude="*.test.ts *.spec.ts" -->
268
+ ```
269
+
270
+ `min="1"` is often the better rule anyway: it says "this exists" without
271
+ breaking every time someone writes a second test.
272
+
273
+ ### `@assert-import-absence` / `@assert-import-count` - dependencies, not text
274
+
275
+ The rules architecture documents actually contain are usually about
276
+ dependencies, and a text search answers those badly in both directions - it
277
+ matches a mention in a comment, and it misses `export * from '../db'`.
278
+
279
+ ```md
280
+ <!-- @assert-import-absence target="src/ui" module="src/db" reason="the UI talks to services, not storage" -->
281
+ <!-- @assert-import-count target="src" module="axios" max="1" -->
282
+ ```
283
+
284
+ These read the dependency rather than the text, in **five languages**:
285
+
286
+ | Language | Extensions | Reads |
287
+ | --- | --- | --- |
288
+ | JavaScript / TypeScript | `.js .mjs .cjs .jsx .ts .mts .cts .tsx` | `import`, `export ... from`, `require()`, `import()` |
289
+ | Python | `.py .pyi` | `import a.b`, `from .rel import x`, `importlib.import_module("x")` |
290
+ | Go | `.go` | `import "x"` and grouped `import ( ... )`, including aliases and `_` |
291
+ | Rust | `.rs` | `use a::{b, c}` with nested groups, `pub use`, `extern crate` |
292
+ | C# | `.cs .csx` | `using`, `using static`, `global using`, `using X = A.B` |
293
+
294
+ No parser and no new dependency. JavaScript gets a full tokenizer because a
295
+ module reference can appear anywhere in an expression; the other four are read
296
+ by masking comments and string literals with the classifier from
297
+ [ADR-0006](docs/adr/0006-comment-classification.md) and then reading statements
298
+ off what is left. Tree-sitter would have cost 94 MB unpacked against this
299
+ package's 0.33 MB - [ADR-0008](docs/adr/0008-polyglot-imports.md) has the
300
+ measurements, and the list of things a real parser would genuinely see that
301
+ this does not.
302
+
303
+ Module patterns are matched against the resolved `/`-separated path *and*
304
+ against the specifier as written, so `module="System.Text.Json"` and
305
+ `module="System/Text/Json"` both work. Matching is case-sensitive, so a rule
306
+ spanning C# and Go needs both conventions:
307
+ `module="app/db/** App/Db/**"`.
308
+
309
+ The unit counted is **files** that depend on the module. For JavaScript and
310
+ TypeScript it understands:
311
+
312
+ | Form | Counted |
313
+ | --- | --- |
314
+ | `import { A } from 'x'`, `import 'x'`, `import * as ns from 'x'` | yes |
315
+ | `import type { A } from 'x'` | yes, unless `types="ignore"` |
316
+ | `export * from 'x'`, `export { A } from 'x'` | yes - a re-export is a dependency, and a barrel file is how layering rules usually get broken |
317
+ | `require('x')` | yes, with a literal argument |
318
+ | `import('x')` | yes, with a literal argument |
319
+ | `import(name)`, `require(expr)` | **no - reported, see below** |
320
+
321
+ Relative specifiers are resolved by path arithmetic, so `module="src/db"`
322
+ matches `../db/client.js` seen from `src/ui/`, and Python's `from ..core import
323
+ x` resolves against the importing file the same way. There is deliberately no
324
+ filesystem resolution: tsconfig `paths` aliases and package `exports` maps are
325
+ not followed, so an alias is written out as itself (`module="@app/db"`).
326
+
327
+ **It tells you what it could not see.** Roughly 1% of module references in real
328
+ code are dynamic, and about 0.09% of files defeat the tokenizer outright. Both
329
+ are reported rather than counted as clean:
330
+
331
+ ```text
332
+ ✔ docs/adr.md:1 @assert-import-absence "src/db" (0 matches) in src/core
333
+ ⚠ 1 module reference could not be resolved statically
334
+ ⚠ src/core/plugin.ts:1 import(name)
335
+ ```
336
+
337
+ The count is still true of everything that could be seen; the warning is what
338
+ stops it being mistaken for a complete answer. `--strict` turns those warnings
339
+ into failures. Files in scope that are not JavaScript or TypeScript are counted
340
+ and reported too, so a rule pointed at the wrong tree says "analysed 2 of 3
341
+ files" rather than quietly passing - and if *none* of them can be read, the
342
+ assertion fails rather than passing on an empty analysis.
343
+
344
+ [ADR-0005](docs/adr/0005-import-assertions.md) has the measurements and the
345
+ reasoning behind each boundary.
346
+
347
+ ### `@assert-layers` - dependencies point one way
348
+
349
+ "Domain must not depend on infrastructure" across three layers is three
350
+ import rules, and across five it is ten - a triangle nobody writes down, and
351
+ the row somebody forgets when a layer is added. One directive states the order:
352
+
353
+ ```md
354
+ <!-- @assert-layers target="src" order="src/domain, src/application, src/infrastructure" -->
355
+ ```
356
+
357
+ **Order runs from the layer everything may depend on to the layer that may
358
+ depend on everything.** A file may import from its own layer and from any layer
359
+ listed before it; importing from a layer listed after it is a violation, and
360
+ the report says which way it went:
361
+
362
+ ```text
363
+ ✖ docs/architecture.md:12 @assert-layers
364
+ src must keep its layers in order, src/domain < src/application < src/infrastructure
365
+ expected no violating files, found 1
366
+ src/domain/user.ts:3:1 src/domain -> src/infrastructure: import ../infrastructure/db.js
367
+ ```
368
+
369
+ A layer is a pattern in the same language as `module=` and `exclude`, which is
370
+ why this works in all five languages above: `src/domain` is anchored at the
371
+ root, and a bare `domain` matches that segment anywhere - in `src/domain/user.ts`,
372
+ in Python's `app.domain.user`, in Rust's `crate::domain::user`. A layer may also
373
+ be a single file; `order="src/parser.ts, src/cli.ts"` understands that
374
+ `./parser.js` is `src/parser.ts`.
375
+
376
+ The unit is files, and `max`, `types`, `exclude` and `baseline` mean what they
377
+ mean on `@assert-import-absence`. Three things fail rather than pass, because
378
+ each is a rule that would otherwise check less than it says: a layer that
379
+ matches no file (`order="domain, aplication"`), a file two layers both claim,
380
+ and a target that does not exist. Files that no layer claims are left alone and
381
+ counted, so a directory nobody assigned shows up as a number rather than as
382
+ silence.
383
+
384
+ ### `@assert-import-cycle` - no file depends on itself
385
+
386
+ ```md
387
+ <!-- @assert-import-cycle target="src" types="ignore" -->
388
+ ```
389
+
390
+ A cycle is reported as one concrete loop, with the line of every import on it:
391
+
392
+ ```text
393
+ ✖ docs/architecture.md:20 @assert-import-cycle
394
+ src must have no import cycles (type-only imports ignored)
395
+ expected no import cycles, found 1
396
+ src/orders/cart.ts:4:1 src/orders/cart.ts:4 -> src/billing/invoice.ts:2 -> src/orders/cart.ts
397
+ ```
398
+
399
+ The count is **knots, not loops**: a set of files each reachable from the others
400
+ counts once however many routes run around it, so adding an import inside a
401
+ tangle that already exists does not move the number. `max="2"` adopts the rule
402
+ on a codebase that already has two, and fails on the third.
403
+
404
+ `types="ignore"` asks the runtime question - erased imports create no load-order
405
+ cycle - and the default asks the coupling one. This repository needs both: its
406
+ `src/` has exactly one cycle, and it is type-only.
407
+
408
+ **Cycles are JavaScript and TypeScript only**, because a cycle needs to know that
409
+ `./b.js` *is* `src/b.ts`, and in the other four languages an import names a
410
+ module, package or namespace rather than a file. A scope with no JavaScript or
411
+ TypeScript in it fails instead of reporting no cycles. The resolver is a short,
412
+ fixed table - the TypeScript source of an emitted `.js`, the path as written,
413
+ the path with each extension, the directory's `index` - checked against files
414
+ the walk already found, never against the disk.
415
+
416
+ It says what it could not follow. An import that should have become an edge and
417
+ did not - a relative path matching no file, or an alias like `@/db` or
418
+ `#internal/db` that cannot be a package - is reported, and fails the run under
419
+ `--strict`. An alias spelled like a real package, `@app/db` through `tsconfig`
420
+ paths, is the one thing it cannot see and cannot report: that would mean
421
+ reading `tsconfig.json`, which is the resolver
422
+ [ADR-0011](docs/adr/0011-layers-and-cycles.md) declines to become, for the
423
+ reasons it gives.
424
+
425
+ ### `@assert-present` - this file exists
426
+
427
+ ```md
428
+ <!-- @assert-present file="SECURITY.md" -->
429
+ <!-- @assert-present file="config/production.json,config/staging.json" -->
430
+ ```
431
+
432
+ Passes when every listed path exists relative to `--root`. Directories count.
433
+
434
+ ### Attributes
435
+
436
+ | Attribute | Applies to | Meaning |
437
+ | --- | --- | --- |
438
+ | `target` | absence, count | Comma-separated paths to search, relative to `--root`. Default `.` |
439
+ | `symbol` | absence, count | The literal string to search for (or a regex with `regex="true"`) |
440
+ | `file` | present | Comma-separated paths that must exist |
441
+ | `expected` | absence, count | Upper bound for absence; exact count for count |
442
+ | `min` / `max` | count (`max` also on absence) | Inclusive bounds |
443
+ | `glob` | absence, count | Include-only file filters, e.g. `*.ts,*.tsx` (ripgrep `-g` semantics) |
444
+ | `exclude` | absence, count | Paths to leave out, gitignore-style: `src/config/**`, `tests`, `*.test.ts` |
445
+ | `regex` | absence, count | Treat `symbol` as a regular expression |
446
+ | `word` | absence, count | Require word boundaries, so `Primary` does not match `PrimaryButton` |
447
+ | `ignore-case` | absence, count | Case-insensitive matching |
448
+ | `comments` | absence, count | `ignore` (default) or `include` for matches inside comments |
449
+ | `module` | import-absence, import-count | Which dependency, matched like `exclude` |
450
+ | `order` | layers | The layers, from the one everything may depend on to the one that may depend on everything |
451
+ | `types` | import assertions, layers, cycles | `include` (default) or `ignore` for `import type` |
452
+ | `allow-empty` | absence, count, import assertions, layers, cycles | Tolerate a scope that holds no files, or a layer that matches none. Off by default - see below |
453
+ | `baseline` | absence, import-absence, layers | Known violations that do not count: `path` or `path:count` |
454
+ | `ratchet` | absence, import-absence, layers | `two-sided` (default) or `one-way` - see below |
455
+ | `reason` | all | Human-readable justification, printed on failure |
456
+
457
+ Unknown attributes are an error, not a shrug: `expct="1"` fails the run instead
458
+ of silently asserting nothing.
459
+
460
+ ### `allow-empty` - an assertion that covers nothing is a failure
461
+
462
+ ```md
463
+ <!-- @assert-absence target="services" symbol="Legacy" glob="*.ts" -->
464
+ ```
465
+
466
+ If `services/` holds no `.ts` file, that rule passes every time, forever,
467
+ without inspecting anything - and in the report it is indistinguishable from a
468
+ rule that inspected a thousand files and found nothing. So it fails instead:
469
+
470
+ ```text
471
+ ✖ docs/adr.md:3 @assert-absence
472
+ "Legacy" must not appear in services
473
+ no files were inspected, so this assertion verified nothing
474
+ (add allow-empty="true" if that is expected)
475
+ ```
476
+
477
+ The usual causes are a `glob` matching no extension in the tree, an `exclude`
478
+ that swallowed the target, or a directory somebody emptied. Where covering
479
+ nothing yet is the honest state of the world - a rule written before the code
480
+ it guards - `allow-empty="true"` says so, and `--allow-empty-scope` says it for
481
+ a whole run.
482
+
483
+ ### `baseline` - adopting a rule the codebase already breaks
484
+
485
+ A strict rule introduced into a mature codebase lands on violations that already
486
+ exist. `exclude` turns those into a permanent blind spot, and `expected="5"`
487
+ cannot tell "one fixed, one added" from "nothing happened". A baseline names
488
+ them:
489
+
490
+ ```md
491
+ <!-- @assert-absence target="src" symbol="LegacyGateway"
492
+ baseline="src/legacy/gateway.ts:2
493
+ src/legacy/adapter.ts" -->
494
+ ```
495
+
496
+ The rule now holds when no file outside the list matches and no file inside it
497
+ matches more than it declares. A bare path means one match.
498
+
499
+ **It ratchets in both directions.** New debt fails, obviously. So does debt that
500
+ has been *paid* and left on the list, because an entry the code no longer
501
+ supports is a spec asserting something untrue:
502
+
503
+ ```text
504
+ expected no matches, found 0; the baseline is out of date and must be
505
+ pruned: src/legacy/adapter.ts (no longer matches)
506
+ ```
507
+
508
+ The fix is deleting the line the message names. `ratchet="one-way"` relaxes that
509
+ half to a report-only note. Every run says how many matches the baseline
510
+ excluded, because a pass bought by an exemption is never silent.
511
+
512
+ To adopt a rule on a codebase that already violates it:
513
+
514
+ ```bash
515
+ spec-guard docs/adr.md --print-baseline
516
+ ```
517
+
518
+ prints the attribute that would exempt exactly today's violations, for you to
519
+ paste in. It prints; it does not edit. [ADR-0009](docs/adr/0009-debt-baselines.md)
520
+ explains why that distinction is the whole design, and why there is no `--fix`
521
+ for architecture rules.
522
+
523
+ ## Document status - drafts and superseded ADRs
524
+
525
+ An ADR has a life. It is `Proposed` before anyone agrees to it, and `Superseded`
526
+ long before anyone deletes it - because deleting it deletes the reason a
527
+ decision was made.
528
+
529
+ spec-guard reads that status from the document and honours it. A document whose
530
+ status is `draft`, `proposed`, `rejected`, `deprecated` or `superseded` is
531
+ parsed, validated, reported - and not executed.
532
+
533
+ Three spellings are recognised, because three are in use:
534
+
535
+ <!-- Shown fenced on purpose: spec-guard masks code before reading a status. -->
536
+
537
+ ```md
538
+ ---
539
+ status: proposed
540
+ ---
541
+ ```
542
+
543
+ ```md
544
+ ## Status
545
+
546
+ Accepted (0.3.0).
547
+ ```
548
+
549
+ ```md
550
+ **Status:** accepted
551
+ ```
552
+
553
+ Front-matter wins, then the `## Status` section, then the bold label (which is
554
+ only read above the first `##`). Anything else - `Provisional`, `In review`, a
555
+ misspelled `Supersedded`, or no status at all - keeps enforcing. That asymmetry
556
+ is deliberate: an unanticipated word that keeps enforcing is a visible failure
557
+ with an obvious fix, while one that stops enforcing is a green build over a rule
558
+ nobody is checking.
559
+
560
+ Withholding is never quiet:
561
+
562
+ ```text
563
+ ○ docs/adr/0011-queues.md is Proposed. - 2 assertions not executed
564
+
565
+ 12 passed · 2 not in force · 48ms
566
+ ```
567
+
568
+ By name, not by count - the same fact reaches `--format json` as `inactiveSpecs`
569
+ and `--format sarif` as a note-level execution notification. A withheld
570
+ directive is still checked for typos and bad attributes, so a draft's mistake is
571
+ found on the day it is written rather than on the day it is accepted.
572
+ `--ignore-status` runs everything, which is how you ask whether a draft would
573
+ pass if you accepted it today.
574
+
575
+ [ADR-0010](docs/adr/0010-spec-status.md) has the full reasoning, including why
576
+ there is no per-directive `if-status` attribute.
577
+
578
+ ## CLI
579
+
580
+ ```bash
581
+ spec-guard [patterns...] [options]
582
+ ```
583
+
584
+ | Option | Description |
585
+ | --- | --- |
586
+ | `-r, --root <path>` | Codebase root that assertions resolve against (default: cwd) |
587
+ | `-v, --verbose` | Print passing assertions too |
588
+ | `--fail-fast` | Stop at the first failing assertion |
589
+ | `--json` | Machine-readable report on stdout (same as `--format json`) |
590
+ | `--format <human\|json\|sarif>` | Output format. `sarif` uploads to GitHub code scanning |
591
+ | `--engine <auto\|rg\|js>` | Search engine (default `auto`: scanner for small trees, ripgrep for big ones) |
592
+ | `--strict` | Treat analysis that could not be completed as a failure |
593
+ | `--allow-missing-targets` | Warn instead of failing when a `target` path does not exist |
594
+ | `--allow-empty-scope` | Warn instead of failing when an assertion inspects no files |
595
+ | `--print-baseline` | Print the `baseline="..."` that would exempt today's violations, and exit |
596
+ | `--no-default-skips` | Search `.git`, `.hg`, `.svn` and `node_modules` too |
597
+ | `--ignore-status` | Execute directives in draft, proposed and superseded documents too |
598
+ | `--include-specs` | Also count matches inside the spec files themselves |
599
+ | `--max-snippets <n>` | Failure snippets per assertion (default 5) |
600
+ | `--concurrency <n>` | Search passes in flight at once (default 8) |
601
+ | `--allow-empty` | Exit 0 when no spec file matched the patterns (about the run, not an assertion) |
602
+ | `--color` / `--no-color` | Force colour on or off (`NO_COLOR` honoured) |
603
+
604
+ Patterns are expanded by spec-guard itself, so quoted globs behave identically
605
+ on Windows, macOS and Linux. A directory expands to the Markdown files in it.
606
+
607
+ ### Exit codes
608
+
609
+ | Code | Meaning |
610
+ | --- | --- |
611
+ | `0` | Every assertion held |
612
+ | `1` | An assertion failed, or a directive was malformed |
613
+ | `2` | spec-guard could not run: bad usage, no spec files matched, `--engine rg` with no ripgrep |
614
+
615
+ ## CI integration
616
+
617
+ ```yaml
618
+ # .github/workflows/specs.yml
619
+ name: Specs
620
+ on: [push, pull_request]
621
+
622
+ jobs:
623
+ spec-guard:
624
+ runs-on: ubuntu-latest
625
+ steps:
626
+ - uses: actions/checkout@v5
627
+ - uses: actions/setup-node@v5
628
+ with:
629
+ node-version: '22'
630
+ - run: npx @descent-vtt/spec-guard "docs/**/*.md" "README.md" --verbose
631
+ ```
632
+
633
+ That job needs nothing else installed. GitHub-hosted runners do **not** ship
634
+ ripgrep on `PATH` - spec-guard's own CI reports `engine: javascript` there - so
635
+ the fallback is what actually runs, and it produces identical results. If your
636
+ repository is large enough that you want ripgrep's speed, install it first:
637
+
638
+ ```yaml
639
+ - run: sudo apt-get update && sudo apt-get install -y ripgrep
640
+ ```
641
+
642
+ Or point spec-guard at a binary you already have with `SPEC_GUARD_RG=/path/to/rg`.
643
+
644
+ As a pre-commit hook (assuming a local install, so the bare command resolves):
645
+
646
+ ```bash
647
+ npx spec-guard "docs/**/*.md" --fail-fast
648
+ ```
649
+
650
+ ### Annotations on the pull request
651
+
652
+ `--format sarif` writes a SARIF 2.1.0 document, which GitHub turns into a
653
+ comment on the offending line:
654
+
655
+ ```yaml
656
+ - run: npx @descent-vtt/spec-guard "docs/**/*.md" --format sarif > spec-guard.sarif || true
657
+ - uses: github/codeql-action/upload-sarif@v3
658
+ with:
659
+ sarif_file: spec-guard.sarif
660
+ category: spec-guard
661
+ ```
662
+
663
+ That job needs `permissions: security-events: write`. One alert per broken rule
664
+ rather than per match - the thing that broke is the rule - anchored on the first
665
+ offending line, with the directive as a related location because that is often
666
+ where the fix goes. Alerts carry a fingerprint derived from what the assertion
667
+ is about rather than where its matches landed, so inserting a line above a
668
+ violation does not close the alert and open a new one.
669
+
670
+ **There is no language server, and that is a decision rather than a gap.**
671
+ spec-guard's claims are about a whole repository - "this symbol appears nowhere
672
+ in `src`" - and a language server is handed one buffer at a time. Answering a
673
+ repository-wide question on every keystroke means rescanning the tree on every
674
+ keystroke; the alternative is answering a smaller question and calling it the
675
+ same one. The CLI does a full run of this repository in about 130 ms, so a
676
+ pre-commit hook or a watch loop already closes the feedback gap without a
677
+ daemon, an extension per editor, or a protocol version matrix.
678
+
679
+ ## How it works
680
+
681
+ ```text
682
+ parser ──▶ Directive[] ──▶ runner ──▶ Assertion[] ──▶ engine ──▶ AssertionResult[] ──▶ reporter
683
+ (pure) (I/O) (rg | js) (pure)
684
+ ```
685
+
686
+ 1. **Parse.** Markdown is scanned for `<!-- @assert-* -->` comments. Fenced code
687
+ blocks and inline code spans are masked first, so documentation that shows
688
+ the syntax (like this README) never executes it.
689
+ 2. **Resolve.** Each directive becomes a typed assertion. Bad numbers, unknown
690
+ booleans, unknown attributes and paths escaping `--root` are rejected here,
691
+ before any I/O.
692
+ 3. **Search.** Assertions that share a target list and flags are answered by a
693
+ *single* pass over the tree.
694
+ 4. **Report.** ANSI output with the spec location, the expectation, the observed
695
+ count, and up to five real snippets; or JSON for tooling.
696
+
697
+ ### Performance
698
+
699
+ Scanning a tree costs about the same whether you look for one symbol or twenty,
700
+ so spec-guard groups assertions by target set and flags and answers each group
701
+ in one pass.
702
+
703
+ `auto` then picks between the two engines **per search group**, because they
704
+ have different shapes of cost: ripgrep is dominated by process startup and
705
+ barely notices tree size, while the built-in scanner has no startup cost and
706
+ grows linearly. Starting a process to search a handful of files is a bad trade.
707
+ Re-measured for 0.4.0 on Windows 11 / Node 24, median of three, one symbol over
708
+ a synthetic tree:
709
+
710
+ | files | scanner | ripgrep | |
711
+ | --- | --- | --- | --- |
712
+ | 100 | **17 ms** | 214 ms | scanner, 12.9x |
713
+ | 500 | **70 ms** | 263 ms | scanner, 3.8x |
714
+ | 1,000 | 216 ms | **181 ms** | ripgrep, 1.2x |
715
+ | 3,000 | 735 ms | **222 ms** | ripgrep, 3.3x |
716
+ | 5,000 | 1,046 ms | **241 ms** | ripgrep, 4.3x |
717
+
718
+ The decision uses a bounded enumeration as its probe: spec-guard walks the
719
+ target set until it either finishes - in which case the file list is already in
720
+ hand and the scanner runs against it, with no process and no second walk - or
721
+ exceeds a budget, in which case ripgrep takes over. The crossover above sits
722
+ between 500 and 1,000 files, which is what the Windows budget of 512 encodes;
723
+ on Linux it is far lower, because what is really being measured is process
724
+ spawn cost. `scripts/bench-engines.mjs` reproduces this, and
725
+ [ADR-0004](docs/adr/0004-adaptive-engine.md) has the full tables.
726
+
727
+ Treat single measurements from one machine with suspicion. On the development
728
+ machine used here, an absence assertion over 2,000 files measures 303 ms with
729
+ the scanner and 399 ms with ripgrep, but ripgrep's own spread across five runs
730
+ was 379-1039 ms - wide enough that the two are not really distinguishable at
731
+ that size. The order-of-magnitude differences at the ends of the table are the
732
+ part worth trusting.
733
+
734
+ 0.5.0 added four language analysers, a scope probe on every assertion and a
735
+ baseline pass, so the obvious question is what that cost. Both versions were run
736
+ against the same synthetic 1,000-file tree, alternating rounds inside one
737
+ process so that this machine's load - which varies by a factor of two over a
738
+ session - falls on both equally:
739
+
740
+ | | median | min |
741
+ | --- | --- | --- |
742
+ | 0.4.0 | 908.8 ms | 493.7 ms |
743
+ | 0.5.0 | 913.4 ms | 496.6 ms |
744
+
745
+ 0.5% apart, against a run-to-run spread far wider than that. The two versions
746
+ also returned identical counts for all five assertions, which is the more
747
+ useful half of the result.
748
+
749
+ 0.5.1 replaced three hand-maintained "are these the same question" field lists
750
+ with one, and the surviving version serialises the exclude set where the old one
751
+ compared it by identity - a hot path made slower on purpose, so it was measured
752
+ rather than assumed. Same tree, same alternating rounds:
753
+
754
+ | | median | min |
755
+ | --- | --- | --- |
756
+ | 0.5.0 | 925.1 ms | 432.1 ms |
757
+ | 0.5.1 | 830.6 ms | 428.5 ms |
758
+
759
+ The medians are 10% apart and the minima are level, which is this machine's
760
+ noise band doing most of the talking - the honest reading is "no regression",
761
+ not "10% faster". The plausible mechanism for any real gain is that the runner
762
+ now groups by the engine's own definition of a shared pass, so a group of five
763
+ where two requests disagree is run as three and two rather than being refused
764
+ whole and run as five. Counts identical across all five assertions, again.
765
+
766
+ Two honest caveats:
767
+
768
+ - **The reported engine is the one that ran, not the one available.** On a small
769
+ repository `--json` reports `"engine": "javascript"` even with ripgrep
770
+ installed. That is the optimisation working, not a failure to find `rg`.
771
+ - **The engines are one implementation, not two that agree.** This used to say
772
+ that ripgrep was the reference and the scanner merely matched it "on
773
+ everything the test suite covers", with a note that the two treated
774
+ `.gitignore` and ignored directories differently and that you should pin an
775
+ engine if it mattered. It did matter: on a tree with eight copies of a symbol
776
+ the scanner found two and ripgrep found four. Since 0.4.0 ripgrep answers only
777
+ *which files contain this text*, and the scanner does all the counting,
778
+ classification and reporting for both. `--engine` now changes how long a run
779
+ takes and nothing about its verdict, and the suite asserts that on trees built
780
+ from every case that used to split them.
781
+
782
+ ## Programmatic API
783
+
784
+ ```ts
785
+ import { runSpecGuard, formatReport } from '@descent-vtt/spec-guard';
786
+
787
+ const report = await runSpecGuard({
788
+ patterns: ['docs/**/*.md'],
789
+ root: process.cwd(),
790
+ });
791
+
792
+ if (!report.ok) {
793
+ console.error(formatReport(report, { color: true, verbose: false }));
794
+ process.exitCode = 1;
795
+ }
796
+ ```
797
+
798
+ `runSpecGuard` returns the full report: every assertion, its bounds, the
799
+ observed count, match locations, warnings and timings. The parser
800
+ (`parseDirectives`) and reporter (`formatReport`, `formatJson`) are pure
801
+ functions you can use on their own.
802
+
803
+ ## Design decisions
804
+
805
+ This tool was specified loosely and built opinionatedly. Where the
806
+ implementation departs from the obvious reading of the brief, here is why.
807
+
808
+ **Spec files are excluded from their own searches.** An ADR that says
809
+ "`LegacyGateway` must not appear" contains the string `LegacyGateway`. Without
810
+ this rule, absence assertions would fail on the document asserting them - the
811
+ single most confusing possible first-run experience. `--include-specs` restores
812
+ the naive behaviour, and counts the prose of your specs; the directives
813
+ themselves stay uncounted, because a directive is an HTML comment and a rule
814
+ whose own text trips it can never be satisfied. `comments="include"` counts even
815
+ those.
816
+
817
+ **Fenced code and inline code are masked before parsing.** A README documenting
818
+ the syntax must not execute it. Masking preserves byte offsets, so reported line
819
+ numbers stay exact. (This is subtle: pairing backtick runs the naive way
820
+ desynchronises after a stray unmatched run and un-masks real prose. spec-guard
821
+ uses CommonMark's equal-length pairing rule, and there is a regression test.)
822
+
823
+ **A document's lifecycle status is read; a directive's is not.** Status is
824
+ document-level and visible in every rendered Markdown view. A per-directive
825
+ `if-status` attribute would be a switch disabling one assertion inside an
826
+ accepted ADR, invisible to anyone who does not read the raw source - and the
827
+ one thing a mechanism for not running an assertion must never be is invisible.
828
+
829
+ **Malformed directives fail the run.** A typo like `expct="1"` could be ignored
830
+ as "not a directive". It is instead an error, because a spec tool whose typos
831
+ silently assert nothing is worse than no spec tool.
832
+
833
+ **Four attributes were added beyond the original brief** - `word`, `regex`,
834
+ `glob` and `ignore-case` - because `symbol="Primary"` matching `PrimaryButton`
835
+ is the first thing every user hits, and `reason` because a failure message
836
+ should say *why* the rule exists.
837
+
838
+ **`min`/`max` are accepted on `@assert-absence` too.** "At most 5 TODOs" is an
839
+ absence claim with a budget, and burning a budget down is a real workflow.
840
+
841
+ **The engine is detected lazily.** Probing with `rg --version` up front costs a
842
+ process spawn (~27 ms on Windows) on the critical path of every run, including
843
+ runs where ripgrep is missing. Discovering its absence from the first real
844
+ search is free.
845
+
846
+ **Assertions that share a target list share one pass over the tree.** This used
847
+ to need a proof: merging literals into a ripgrep alternation can lose matches
848
+ through containment (`Primary` / `PrimaryButton`) and dovetailing (`abc` / `cd`
849
+ in `abcd`), so spec-guard checked for both and fell back to separate passes.
850
+ Since 0.4.0 it does not need the proof, because ripgrep no longer counts
851
+ anything - it answers only *which files contain this text*, and the scanner
852
+ counts each pattern separately over the shared file contents. The batching
853
+ checks were deleted along with the risk they guarded.
854
+
855
+ **Exit code 2 exists.** "Your specs failed" and "spec-guard could not run" are
856
+ different facts, and CI should be able to tell them apart.
857
+
858
+ **A missing `target` fails the run.** It used to warn and search what was left,
859
+ which meant an assertion pointed at a renamed directory searched nothing, found
860
+ nothing, and reported success — the exact shape of a green check that verified
861
+ nothing. `--allow-missing-targets` restores the old behaviour for repositories
862
+ where a path is legitimately optional.
863
+
864
+ **Hidden directories are searched, and `.gitignore` is not consulted.** An
865
+ audit of 0.3.0 found the tool reporting a clean pass on a repository whose
866
+ forbidden symbol sat in `.github/workflows/ci.yml`; the scanner and ripgrep also
867
+ disagreed with each other, finding two matches and four on the same tree. Scope
868
+ is now one policy that both engines are driven by, the skip list is four names
869
+ long, and anything spec-guard could not inspect is reported rather than assumed
870
+ clean ([ADR-0007](docs/adr/0007-search-scope.md)).
871
+
872
+ **Comments are excluded by default, and the exclusion is reported.** Counting
873
+ the note that records a deletion as an occurrence of the thing deleted punishes
874
+ the documentation this tool exists to keep honest ([ADR-0006](docs/adr/0006-comment-classification.md)).
875
+ The reverse risk — a rule that quietly stops checking anything because every
876
+ match now sits in a comment — is why every run says how many matches it dropped.
877
+
878
+ **An assertion that inspected no files fails.** It is the same defect as a
879
+ missing `target` seen from a different angle: a rule whose scope is empty passes
880
+ forever and reads exactly like a rule that found nothing. Turning this on found
881
+ a vacuous assertion inside this repository's own test suite on the first run.
882
+ `allow-empty="true"` covers the honest case of a rule written before the code it
883
+ guards.
884
+
885
+ **Four more languages, and no parser.** Import assertions read Python, Go, Rust
886
+ and C# as well as JavaScript. Not by adding four tokenizers - by reusing the
887
+ comment and string lexer that already existed, masking the source with it, and
888
+ reading statements off what is left. Tree-sitter would have been the modern
889
+ answer and costs 94 MB unpacked against this package's 0.33 MB; the four things
890
+ a real parser would genuinely see that this cannot are listed in
891
+ [ADR-0008](docs/adr/0008-polyglot-imports.md) rather than glossed over.
892
+
893
+ **There is no `--fix`.** Every edit a machine can make to a failing boundary
894
+ assertion is an edit that records the rule no longer holding: widen the bound,
895
+ add an exclusion, append to the baseline, insert an ignore comment. A one-flag
896
+ path from red to green is a bad button for a person and a much worse one for an
897
+ agent whose loop terminates on a green build. `--print-baseline` prints what you
898
+ could paste; it does not paste it ([ADR-0009](docs/adr/0009-debt-baselines.md)).
899
+
900
+ ## spec-guard checks itself
901
+
902
+ The invariants in [`docs/adr/0001-invariants.md`](docs/adr/0001-invariants.md)
903
+ and [`docs/adr/0002-directive-format.md`](docs/adr/0002-directive-format.md) are
904
+ executed against this repository on every CI run. The CLI never calls
905
+ `console.log`, nothing outside the engine spawns a process, and the parser and
906
+ reporter never touch the filesystem - because those documents say so, and the
907
+ build fails if they stop being true.
908
+
909
+ This README is executable too:
910
+
911
+ <!-- @assert-present file="src/parser.ts,src/engine.ts,src/reporter.ts,src/runner.ts,src/cli.ts,src/comments.ts" -->
912
+
913
+ ## Development
914
+
915
+ ```bash
916
+ npm install
917
+ npm run build # tsc -> dist/
918
+ npm test # vitest
919
+ npm run test:coverage
920
+ npm run test:mutation # stryker (~16 min in CI; over two hours locally)
921
+ npm run lint # tsc --noEmit
922
+ npm run selfcheck # run spec-guard on its own docs
923
+ ```
924
+
925
+ The test suite runs every assertion case against **both** engines and asserts
926
+ they agree, so the fallback cannot quietly drift from ripgrep. `@vscode/ripgrep`
927
+ is a devDependency purely so that the ripgrep path is exercised on every
928
+ platform in CI, including machines that have no `rg` on PATH; it ships a
929
+ prebuilt binary and is never a runtime dependency.
930
+
931
+ ### Mutation testing
932
+
933
+ Coverage says a line ran. It does not say an assertion would notice if the line
934
+ behaved differently. This repository measures the difference: line coverage is
935
+ **100%**, and **97.65%** of 4,299 mutants are killed in CI, with no file below
936
+ 95%. The second number is the one worth reading - and it is the ceiling, not a
937
+ way-point: every surviving mutant has been checked individually and the 101 that
938
+ are left produce byte-identical output.
939
+
940
+ That gap is the point. The first run scored 77.23%, and the weakest file was the
941
+ reporter at 65.48% - not because it lacked tests, but because its tests were
942
+ almost all `toContain` against colourless output. A mutant could prepend a junk
943
+ line, drop a colour, or turn `remaining > 0` into `remaining >= 0` and every
944
+ test still passed. Exact whole-output comparison took it to 87.30%. The engine
945
+ and the walker were then rewritten for testability rather than papered over with
946
+ more tests, which is what moved them from 75%/78% to 83%/93%.
947
+
948
+ The most recent pass took every module to its ceiling, and its value was not the
949
+ number. Writing down four contracts that had only ever been checked through
950
+ their effect on a match count turned up four wrong answers: two assertions on
951
+ one symbol answered each other's comment handling, an unreadable directory went
952
+ unreported on any repository small enough to scan in process, snippets from CRLF
953
+ files carried a carriage return into the terminal, and an invalid pattern was
954
+ reported with its error message twice. Seventeen branches turned out to be
955
+ unable to decide anything and were deleted rather than pinned; thirty-five
956
+ negative controls - each defect reintroduced one at a time - confirm the suite
957
+ goes red for every one, and the changes meant to be invisible were checked
958
+ against 6,976 files of real code, reference for reference, against the published
959
+ 0.5.0 build.
960
+
961
+ Then the survivors themselves were checked rather than excused. All 130 were
962
+ applied one at a time and run through a fingerprint of 4,176 observations, and
963
+ **29 of them turned out not to be equivalent at all** - a Rust brace counter
964
+ that only matters when a second statement follows, a type-only test that marks
965
+ `import A, { B } from 'x'` type-only when loosened, a comment check that opens a
966
+ block comment on `2*3`. All 29 are now tested and all 29 die. All of it is in
967
+ [ADR-0003](docs/adr/0003-mutation-testing.md), including a Stryker limitation
968
+ found on the way: a mutant that stops a test file *loading* is reported as
969
+ survived even though the suite is in fact killing it.
970
+
971
+ CI runs it in **two tiers**, both gated at 97%. Branches and pull requests run
972
+ Stryker incrementally, reusing the verdict for any mutant whose source and
973
+ covering tests are both unchanged. Pushes to `main`, the weekly schedule and
974
+ manual runs do the full sweep, which is the authoritative number and the one
975
+ quoted above; a full sweep is also what publishes the cache the branches start
976
+ from, so an incremental verdict can never be built on another incremental
977
+ verdict.
978
+
979
+ The full sweep was 6m54s at 2,118 mutants, 15m51s at 3,493, and is 17m at
980
+ 4,299. That growth is why the tiers exist: a check that gets quietly more
981
+ expensive every release is a check somebody eventually proposes lowering.
982
+
983
+ Run it locally with `npm run test:mutation` if you like, but do not calibrate
984
+ anything on the result: on the Windows machine this was developed on the same
985
+ suite takes over two hours against 21 minutes hosted, and it scores *higher*,
986
+ because far more mutants hang there and Stryker counts a hang as a kill. Linux
987
+ CI is the measurement.
988
+
989
+ Eight cautionary tales are in [ADR-0003](docs/adr/0003-mutation-testing.md): a
990
+ run whose score was pure fiction because the mutants were never activated, a
991
+ tuning knob that lifted the score six points without adding a test, the platform
992
+ gap above, the baseline being re-anchored when a tokenizer arrived, and a score
993
+ that rose partly because code carrying hard-to-kill mutants was deleted rather
994
+ than because tests improved, and the 0.5.0 sweep that failed the build at 83.77%
995
+ because a suite can be thorough about the thing it was written to test and
996
+ silent about the machinery underneath it, and the excuse that let the tokenizer
997
+ sit at 73% for three releases because "that kind of code carries more equivalent
998
+ mutants" sounded like judgement rather than an unchecked assumption - along with
999
+ the real bugs that chasing the gate uncovered, and a class of mutant the runner
1000
+ reports as survived while the suite is in fact killing it.
1001
+
1002
+ ## Requirements
1003
+
1004
+ - Node.js 22 or newer (native ESM)
1005
+ - ripgrep optional - used when present, replaced by a built-in scanner when not
1006
+
1007
+ ## License
1008
+
1009
+ MIT