@descent-vtt/spec-graph 0.4.0 → 0.6.0

package/CHANGELOG.md

@@ -3,6 +3,361 @@
 Notable changes, newest first. Versions follow [semver](https://semver.org):
 a patch fixes behaviour without asking anything of a repository that upgrades.
 
+## 0.6.0
+
+A diff of two graph exports that names only the changes it can tell apart,
+every glob moved onto the automaton `~=` already ran on, and a directive that
+annotates one item where it used to annotate every item in reach. The mutation
+figure is lower than 0.5.0's, and it is the first a release has carried from
+tests that no longer race each other on disk.
+
+### Fixed
+
+**A glob can no longer keep a run busy for minutes.** 0.5.0 moved `~=` onto an
+automaton that cannot backtrack and left globs on `RegExp`, on the argument
+that a glob has no nested quantifiers. Nobody had timed it. Stars that are not
+nested still divide a failing subject between them every possible way:
+
+| glob | subject | `RegExp` | now |
+| --- | --- | ---: | ---: |
+| `**/*-*-*-*.md` | a 643-character hyphenated file name | 2.3s | 0.2ms |
+| `*-*-*-x` | a 10,000-character reference target | 120s | 0.9ms |
+
+The second is the one that matters: `--ignore-ref` and `ignoreReferences` are
+matched against targets read out of documents, which nothing limits the length
+of. Every glob - the patterns to check, `--ignore`, `--ignore-ref`, `--history` -
+now runs on the same automaton as `~=`, verified against the `RegExp` it
+replaces on every glob and path in a differential corpus. See
+[ADR-0017](docs/adr/0017-a-predicate-must-finish.md).
+
+- **An invalid glob names itself.** `docs/{a` used to report
+  `Invalid regular expression: /^docs\/(?:a$/i: Unterminated group`, about a
+  parenthesis nobody typed. It now reports `invalid glob "docs/{a": unclosed "{"`.
+- **`--ignore-ref` reads case the same way on every platform.** It lower-cased
+  both sides, and on Windows alone also folded case the way the `i` flag does,
+  which made a handful of distinct characters - the micro sign and the Greek
+  mu - one target there and two everywhere else.
+- **A `@spec-item` directive annotates one item.** Each item used to look for
+  its own, so a directive was found by every item within 200 characters below
+  it and by every item enclosing it. The question after a declared one took the
+  same id and vanished from the graph without a word, and a parent could take
+  its child's directive in place of its own. A directive now binds to the item
+  directly below it, with nothing but blank lines and comments between, or else
+  to the innermost item it is written on or indented under. An id two items
+  still claim, by declaring it twice or by declaring another item's number, is
+  a parse problem, and the first item keeps it. `bindItemDirectives` is
+  exported; `directiveFor` is unchanged and still exported.
+
+### Added
+
+**`spec-graph diff <before.json> <after.json>` says what a change did to the
+decisions:** documents added, removed, moved or accepted, relations added or
+removed, and obligations resolved or reopened. It names only what it can tell
+apart. An obligation's id is its position in its section, so one inserted
+question renumbers every question below it, and a diff keyed on those ids would
+report a closed question as reopened when nobody touched it. An obligation is
+paired only by a declared id, or by its document, section and title when nothing
+else shares them; anything unpaired is reported as having appeared or
+disappeared. Human, `--format json` or `--format markdown`, and exit 0 either
+way. See [ADR-0020](docs/adr/0020-a-diff-names-what-it-can-tell-apart.md).
+
+- The JSON graph export names its `generator`, so a diff can warn when two
+  exports came from different versions, and each item says whether its id was
+  `declared`. Both are additions; the export is still version 1.
+- `compileGlob(pattern, { ignoreCase })`, the matcher the walk uses, and an
+  `ignoreCase` option on `compilePattern` for a pattern that must respect case.
+  `globToRegExp` is unchanged and still exported.
+
+### Changed
+
+One behaviour moves, and it can turn a run that passed into one that fails.
+
+- **A `@spec-item` directive with prose, a heading or code between it and the
+  item below binds to nothing.** It used to reach any item starting within 200
+  characters. Its id, state and title no longer apply: the item takes its
+  numbered id, a `state="moot"` stops closing it, and a link to the declared id
+  is a `broken-reference`. Move the directive down to the line above the item.
+
+### Verified
+
+| | |
+|---|---|
+| `npm run lint` | pass |
+| `npm test` | 921 passing, 23 files |
+| `npm run selfcheck` | pass, over 23 documents and 184 relations |
+| hosted full sweep, which governs | **75.25%** over 10,519 mutants, four shards of 22 to 33 minutes |
+| `break` | unchanged at 70 |
+
+0.5.0's 76.83% is not the figure to compare with. About three points of it were
+kills no assertion made: tests writing to fixed paths deleted each other's files
+under Stryker's parallel workers. Race-free, the sweep read 73.42% before
+`spec-graph diff` and 75.25% with this release: `diff.ts` at 98.35%, and
+`directives.ts` from 66.88% to 78.01%. Lose every timeout and it reads 71.89%.
+See [ADR-0007](docs/adr/0007-mutation-testing.md) and
+[ADR-0019](docs/adr/0019-the-sweep-runs-in-shards.md).
+
+The sweep after the directive fix moved `extract.ts` down 0.18 points, among the
+usual movement in files nobody touched, and this one was real. Four mutants
+that hand an item to every specification in a register had gone from killed to
+surviving: resolution now drops the duplicates they create, and the test that
+caught them only counted items. It now also asserts that no problem is
+reported.
+
+## 0.5.0
+
+One shipped feature turned out to have shipped a false positive, and the
+predicate that was documented as dangerous is now an automaton that cannot
+backtrack. Four of the ADR suite's open questions answered by building the
+thing, seven directions declined with a reason, one deferred with its design
+settled so it is not re-litigated.
+
+### Fixed
+
+**A register's decisions answer for the front matter above them.** This was a
+false positive in the feature 0.4.0 led with, and the worst kind: confident and
+unanswerable. A region answered *nothing* for `fm.owner`, `!=` against an absent
+value is a mismatch, so a repository writing
+
+```json
+"rules": { "owned": { "query": "document[fm.owner!=platform]" } }
+```
+
+was told twice that a register saying `owner: platform` is not owned by
+platform — with the message rendering `{0.fm.owner}` as the literal placeholder,
+because there was nothing to put there.
+
+A region now carries its file's front matter minus two kinds of key. A
+**relation** key stays behind: `supersedes:` at the top of a register supersedes
+on behalf of the register, not of each decision in it. So does a key the region
+**answers for itself** — its identifier, status, title and aliases, which are
+precisely the things a register exists to vary row by row.
+[ADR-0009](docs/adr/0009-a-specification-is-a-region.md) carried this as a
+narrowed open question and called it a gap. It was a defect, and it was found by
+pointing the binary at a register rather than by reading the code.
+
+Read from the other side, this can *add* findings: a rule phrased positively,
+`document[fm.deprecated]`, now matches a register's decisions as well as the
+register. That is the same correction - the decisions really are described by
+the front matter above them - and worth knowing before upgrading a repository
+that keeps registers and writes rules about front matter.
+
+### Added
+
+**`~=` is matched by an automaton that cannot backtrack.**
+[ADR-0016](docs/adr/0016-a-query-needs-a-sentence.md) documented this hazard
+with `(a+)+$`, which nobody writes, and priced the fix at "worth it only if a
+real repository hangs". Both were wrong. `^([A-Za-z0-9_]+[ ]?)+$` is what
+somebody writes to check that a title is words separated by single spaces:
+
+| subject | `RegExp` | now |
+| --- | ---: | ---: |
+| `the quick brown fox jumps over the lazy dog!` | 0.9s | 11us |
+| `... over the lazy dog and!` | 5.4s | 12us |
+| `... over the lazy dog and keeps!` | 103s | 13us |
+
+A word of k letters can be cut into pieces 2^(k-1) ways and a backtracking
+engine tries the product, so a sixty-character title does not finish this year —
+unattended, on every build, against a corpus whose titles the author of the rule
+has not read. The work is now exactly O(pattern x subject), and **no stage of
+this pipeline can take longer than its input.**
+
+It costs a dialect. Backreferences and lookaround are not regular, so they are
+refused while the selector is read, with the character pointed at — as are three
+things `RegExp` accepts and should not: an unknown letter escape, which `RegExp`
+reads as the letter, so a pattern meant as an anchor silently matches a capital
+A; a Unicode property escape, which needs a table this package will not carry;
+and octal escapes. Everything else works, and a pattern that does not parse is
+now a usage error rather than a pattern that silently matches nothing.
+
+Verified against the engine it replaces: **1.33 million pattern-subject pairs,
+zero disagreements**, 19,000 of them committed as a gate. It earned that twice
+over, on one clause of the language specification read backwards in two
+different places - the word characters `\b` recognises, and whether a character
+whose upper case is ASCII may match `[A-Z]`. The second was found by the
+mutation score pointing at a corpus with a blind spot in it rather than at a
+test. See [ADR-0017](docs/adr/0017-a-predicate-must-finish.md) and
+[ADR-0007](docs/adr/0007-mutation-testing.md).
+
+**Configuration is discovered upward, and the file holding it is the root.**
+
+```bash
+cd packages/auth && spec-graph check     # the repository's rules, the
+                                         # repository's patterns, the
+                                         # repository's paths
+```
+
+[ADR-0010](docs/adr/0010-configuration-belongs-to-the-repository.md) worried
+that discovery would make a run depend on where it started. Reading the working
+directory already did, and silently: a check from a package directory found no
+configuration, ran no project rules, used the default include patterns, and
+printed a verdict in the same shape as the real one. Because the directory
+holding the file becomes the root, and every path here is relative to the root,
+a run from anywhere inside the repository now produces **byte-identical** output
+to a run from the top. The walk stops at the repository — a directory holding
+`.git` — so a stray file in a home directory cannot reach it. `--root` names the
+root yourself and turns discovery off, and a path typed on the command line
+stays relative to where you typed it. See
+[ADR-0018](docs/adr/0018-the-configuration-file-is-the-root.md).
+
+**`spec-graph query project:<rule>` runs a registered rule by name.** The rule
+is already compiled by the time the command runs, and the alternative was
+copying its selector back out of the configuration file by hand. Several
+selectors are deduped the way the check dedupes them, so this prints the set the
+check reports on; `--verbose` names them, which is the question a team
+calibrating a convention is actually asking. Built-in rules are deliberately not
+addressable this way: a project rule *is* its selector, while a built-in is a
+selector plus judgement, and printing a set that differs from the findings would
+invite the wrong conclusion.
+
+**`spec-graph rules <rule-id> --explain` names the ADR that decided it.** A rule
+that fires is a claim about somebody's repository, and the reasoning behind the
+claim was in a document nobody could find from the message. The table holds a
+pointer rather than a paraphrase, because a second copy of the reasoning would
+drift from the first — and a test holds every pointer to a file this corpus
+checks, so a renamed ADR breaks the link in the same run that breaks the
+reference.
+
+**`--format markdown`** writes the same facts as a GitHub-flavoured table, for
+`$GITHUB_STEP_SUMMARY` or a pull-request comment:
+
+```yaml
+- run: npx spec-graph --format markdown >> "$GITHUB_STEP_SUMMARY"
+```
+
+SARIF puts a finding on the line that caused it, which is where somebody fixing
+one wants it, and says nothing at all to the person deciding whether to merge.
+This leads with the verdict, counts the corpus, gives every finding its hint,
+and *names* the stale baseline entries rather than counting them — on a pull
+request a count is the one thing a reader cannot act on. No environment
+detection: a format that changed because a variable was set would be a format
+nobody controls.
+
+### Changed
+
+Four behaviours move, none of them a rule. Three can turn a run that passed
+into a usage error, which is the point in each case; the fourth changes what a
+run from a subdirectory reports.
+
+- **A `~=` pattern that does not parse is a usage error**, with the character
+  pointed at, rather than a pattern that matches nothing. The old silence was
+  indistinguishable from a rule that ran and found none.
+- **A `~=` pattern using a backreference, lookaround, `\p{...}`, an octal
+  escape or an unknown letter escape is refused**, for the reasons above. If
+  you have one, the message says which construct and why; `^=`, `$=` and `*=`
+  cover the fixed-prefix, suffix and substring cases without a pattern at all.
+- **A run from a subdirectory now finds the repository's configuration**, and
+  reports repository-relative paths because the configuration's directory is
+  the root. A script that read paths out of a nested run will see them change.
+  `--root` keeps the old behaviour exactly.
+- **`spec-graph rules <word>` reads the word as a rule id** and reports one
+  that names nothing. It used to be collected as an include pattern, which the
+  command has no use for.
+
+### Declined
+
+Each with its reasoning in the ADR that owns the question, rather than in a
+roadmap nobody reads:
+
+- **An MCP server.** The capability is already here — `analyseSources()` takes
+  text, `--format json` is the answer — so what is left is a protocol
+  implementation, hand-written to keep the dependency count at zero, tracking a
+  specification that is still moving, to expose three tools that are three shell
+  commands an agent can already run.
+- **A language server.** The right shape for editor feedback and a second
+  package's worth of work, and SARIF already reaches the problems pane. Two of
+  the three hard parts are done — `analyseSources()` needs no file on disk, and
+  every finding carries a span — so it should start when somebody wants hover
+  and go-to-definition, not as a way of delivering diagnostics that already
+  arrive.
+- **A `--watch` daemon.** Declined again, with a better reason than last time:
+  the incremental half of it cannot work. Whether a bare `ADR-0099` in prose is
+  a citation depends on which families exist elsewhere, and a near-miss
+  suggestion is computed against every sibling — touch one file and the honest
+  set to recompute is all of them. Which is fine, because the pipeline is a pure
+  function and 60 ms long. And then the resident index buys nothing, and the
+  feature is a loop.
+- **Code frames in the terminal.** The report already prints
+  `file:line:column`, which every terminal in use turns into a click, and
+  `renderMatch` already prints the shape of a multi-hop path.
+- **Provenance and attestation metadata.** A timestamp in the output means no
+  two reports of one corpus are ever byte-identical, which is the property every
+  format here is built to have. The commit is something CI already knows.
+- **An interactive SVG.** A layout engine or a vendored library is a runtime
+  dependency wherever it sits in the tarball, and its bytes depend on a layout
+  pass. `dot` and `mermaid` hand layout to tools built for it, and both are
+  text. The other half of that ask - phase styling and typed edges in the
+  Mermaid export - has been there since the first release. What is left is a
+  `subgraph` per directory, which changes the shape of an export somebody's
+  documentation build is parsing, to group a corpus that is one directory in
+  most repositories.
+- **Definition lists as a register form.** Two unrelated syntaxes behind one
+  name: raw `<dl>`, which the scanner leaves as prose deliberately because
+  ADR-0001's masking guarantee is what keeps a code fence quiet, and the
+  PHP-Markdown-Extra form, which is not CommonMark and which no corpus here
+  contains. Either would need a new parser for a convention nobody has written
+  down.
+
+One is deferred rather than declined, with half its design recorded so it is not
+re-litigated: a **relational diff** between two states of the graph, taken
+between two `--graph-format json` exports rather than between two git revisions,
+because revisions mean spawning a binary underneath a pipeline that is a pure
+function of text. See
+[ADR-0015](docs/adr/0015-feedback-goes-where-the-tools-already-look.md).
+
+### Verified
+
+| | |
+|---|---|
+| `npm run lint` | pass |
+| `npm test` | 833 passing, 21 files |
+| `npm run selfcheck` | pass, over 21 documents and 159 relations |
+| `npm run test:mutation` | 80.54% over 9,697 mutants, 74m27s |
+| hosted full run, which governs | **76.83%**, 168m06s |
+| hosted incremental run, same commit | 76.43%, 64m27s |
+| `break` | unchanged at 70 |
+
+79.41% over 8,585 mutants at 0.4.0 against 80.54% over 9,697 here, on the same
+machine: 1,112 more mutants and a point higher. The new code carries itself -
+`regex.ts` at **90.61%**, `config.ts` at 92.37%, `project-rules.ts` still at
+99.50% - and lose all 343 timeouts and the figure reads 77.00%.
+
+The first full run of this release read 80.35%, and its survivor list is where
+the last defect came from. A mutant on the line stepping past `^` in a negated
+class survived 1.33 million differential comparisons, because the corpus only
+ever tested classes against two-character subjects; one-character subjects
+killed it and then caught `[A-Z]` matching U+017F. `report.ts` went from 66.30%
+to 70.84% on the same pass, once the Markdown tests asserted the table's
+structure - rectangular rows, a blank line before every block, every summary
+number present - rather than fragments of its text. What remains there is
+mostly the older formatters.
+
+Between those two runs `rules.ts` moved from 79.42% to 76.02% and
+`directives.ts` from 77.92% to 74.03%, with neither file touched: the tests added
+in between changed which tests `perTest` believes cover which mutants.
+[ADR-0007](docs/adr/0007-mutation-testing.md) has recorded that shape for two
+releases, and it is why a per-file drop is a question and not an answer.
+
+The hosted number is the one that governs, because it is where the build fails:
+**76.83%**, up from 75.40% at 0.4.0. Lose every timeout and it reads 73.31%, the
+first time the pessimistic figure has cleared the guard by more than three
+points.
+
+The incremental run on the same commit read 0.40 points low - the third
+observation after 3.18 high at 0.3.0 and 1.29 low at 0.4.0, and the reason
+[ADR-0007](docs/adr/0007-mutation-testing.md) calls it a signal rather than a
+figure. The tag's run also logged a cache save, which this entry first read as
+the cache-key fix from 0.4.0 doing its job. *Corrected 2026-09-13:* it saved a
+copy of the stale report it had restored, because the full step never wrote one,
+and an entry saved under a tag cannot be restored on `main` in any case. See
+[ADR-0007](docs/adr/0007-mutation-testing.md).
+
+**The rebuild took 168 of its 180 minutes.** 13% more mutants than 0.4.0, and 63%
+longer on the hosted runner against 15% longer locally. At 0.4.0's old cap of
+120 it would have been cancelled with nothing reported. The cause is not yet
+attributed - runner variance and the new differential test are both candidates -
+and the choice between raising the cap again and cutting per-mutant cost is left
+open in ADR-0007 until it is.
+
 ## 0.4.0
 
 Ten open questions across the ADR suite, answered. Five shipped, three declined

package/README.md

@@ -386,6 +386,23 @@ Items inherit their document's lifecycle, so `item[phase=retired]` means what yo
 expect. A query evaluates to **paths**, not endpoints, which is why a finding can
 name both ends and the line that connects them.
 
+`~=` is a regular expression matched by an automaton that cannot backtrack, so a
+predicate is linear in the subject whatever the pattern — `^([A-Za-z0-9_]+[ ]?)+$`
+against a fifty-four-character title takes `RegExp` 103 seconds and this 13
+microseconds. It reads the usual syntax minus backreferences and lookaround,
+which are not regular; both are refused when the selector is read, with the
+character pointed at. Globs - the patterns to check, `--ignore`, `--ignore-ref`,
+`--history` - are matched by the same automaton, because a glob with three stars
+in it is
+enough to keep `RegExp` busy for two minutes over a long reference target. See
+[ADR-0017](docs/adr/0017-a-predicate-must-finish.md).
+
+On a register, a region answers its file's front matter for every descriptive
+key — `fm.owner` on a decision inside a register is the register's owner — while
+the keys a region owns itself, its identifier and status and title, and any key
+that declares a relation, stay on the file. See
+[ADR-0009](docs/adr/0009-a-specification-is-a-region.md).
+
 ## Your own rules
 
 A convention spec-graph never anticipated is a selector plus a sentence. Write
@@ -432,6 +449,12 @@ escalated by `--strict`, listed by `spec-graph rules`, exempted on journals like
 every other rule about obligations, and emitted as a first-class `ruleId` in
 SARIF, where a project rule describes itself by the selector it is.
 
+To see what a rule matches without copying its selector back out of the file:
+
+```bash
+spec-graph query project:no-draft-dependency --verbose
+```
+
 Mistakes are reported when the file is read, not discovered as a rule that
 quietly finds nothing:
 
@@ -460,6 +483,11 @@ Directives are ordinary HTML comments — invisible in every Markdown renderer:
 A directive always wins, and the report says the state came from a directive, so
 an override is visible rather than mysterious.
 
+A `@spec-item` annotates one item: the one directly below it, with nothing but
+blank lines or other comments between, or else the one it is written on or
+indented under. An id that two items end up with is a parse problem, listed by
+`--verbose`, and the first item keeps it.
+
 ## Journals, changelogs and minutes
 
 A 2024 journal noting *"decision deferred to ADR-002"* is not delegating
@@ -594,6 +622,14 @@ command. spec-graph reads the first of `.spec-graph.json`,
 }
 ```
 
+The file is looked for in the working directory and then upward, as far as the
+repository — a directory holding `.git` — and **the directory holding it becomes
+the root**. So a run from `packages/auth` reports what a run from the top
+reports, byte for byte, because every path here is relative to the root. The
+nearest file wins, `--root` names the root yourself and turns discovery off, and
+a path typed on the command line stays relative to where you typed it. See
+[ADR-0018](docs/adr/0018-the-configuration-file-is-the-root.md).
+
 A flag always wins over the file, and list flags **add** to it rather than
 replacing it — a `--ignore-ref` on the command line is one more exclusion, not a
 decision to discard what the repository already declared. `--verbose` prints
@@ -644,10 +680,15 @@ already exists in the corpus.
 ```text
 spec-graph [check] [patterns...]     Validate the graph. The default.
 spec-graph query <selector>          Run a selector, print matching paths.
+spec-graph query project:<rule>      Run a registered rule by its id.
 spec-graph graph [patterns...]       Export for Graphviz, Mermaid or JSON.
-spec-graph rules [--explain]         List the diagnostics.
+spec-graph rules [rule-id]           List the diagnostics; --explain adds
+                                     each selector and the ADR behind it.
+spec-graph diff <before> <after>     Compare two JSON graph exports.
 
---root <dir>            Directory patterns resolve against
+--root <dir>            Directory patterns resolve against, and the root of the
+                        run. Without it the config file is discovered upward and
+                        its directory is the root
 --ignore <glob>         Skip paths (repeatable)
 --ignore-ref <glob>     Do not report these reference targets (repeatable)
 --family <name>         Families a bare identifier may name (repeatable)
@@ -657,7 +698,7 @@ spec-graph rules [--explain]         List the diagnostics.
 --record-baseline <f>   Write today's findings as accepted debt, exit 0
 --ratchet               Also fail when a baseline entry no longer occurs
 --no-config             Ignore .spec-graph.json and the package.json key
---format <fmt>          human | json | sarif
+--format <fmt>          human | json | sarif (check) | markdown (check, diff)
 --graph-format <fmt>    dot | mermaid | json
 --documents-only        Hide items; their relations lift onto their documents
 --rule <id>=<severity>  error | warn | info | off (repeatable)
@@ -680,6 +721,19 @@ rather than a silent success.
   run: npx spec-graph "docs/**/*.md" --format json > spec-graph.json
 ```
 
+`--format markdown` writes the same facts as a table, for the place reviewers
+actually look:
+
+```yaml
+- name: Summarise the specification graph
+  if: always()
+  run: npx spec-graph --format markdown >> "$GITHUB_STEP_SUMMARY"
+```
+
+It leads with the verdict, counts the corpus, gives every finding its hint, and
+*names* the baseline entries that no longer occur rather than counting them —
+on a pull request a count is the one thing a reader cannot act on.
+
 The JSON report is versioned, flat, and carries start and end positions for
 every finding, so a bot can annotate a diff:
 
@@ -704,6 +758,28 @@ every finding, so a bot can annotate a diff:
 }
 ```
 
+A check says what is wrong. `spec-graph diff` says what a pull request changed:
+documents added, removed, moved or accepted, relations added or removed, and
+obligations resolved or reopened. It compares two exports made by the same
+binary, so CI makes one from the base branch and one from the pull request:
+
+```yaml
+- run: git fetch --depth 1 origin "$GITHUB_BASE_REF" && git worktree add ../base FETCH_HEAD
+- run: npx spec-graph graph --root ../base --graph-format json > base.json
+- run: npx spec-graph graph --graph-format json > head.json
+- run: npx spec-graph diff base.json head.json --format markdown >> "$GITHUB_STEP_SUMMARY"
+```
+
+It names only what it can tell apart. A document keeps its id through a rename,
+so a moved document reads as moved. An obligation's id is its position in its
+section, so inserting a question renumbers every question below it; an
+obligation is therefore paired only by an id its author declared, or by its
+document, section and title when nothing else shares them. Anything unpaired
+is reported as having appeared or disappeared, never as resolved or reopened.
+Findings are left to `--baseline`, which already reports what is new. The exit
+code is `0` whether anything changed or not: a diff describes, and `check`
+gates. See [ADR-0020](docs/adr/0020-a-diff-names-what-it-can-tell-apart.md).
+
 ## Visualising
 
 ```bash
@@ -739,7 +815,7 @@ exported. Reporters, editor extensions and custom rules are all first-class.
 
 ## Design
 
-Twelve ADRs, which `spec-graph` validates on every CI run:
+Twenty ADRs, which `spec-graph` validates on every CI run:
 
 - [ADR-0001 — A hand-written Markdown scanner](docs/adr/0001-hand-written-markdown-scanner.md)
 - [ADR-0002 — A four-phase lifecycle lattice](docs/adr/0002-lifecycle-lattice.md)
@@ -756,11 +832,18 @@ Twelve ADRs, which `spec-graph` validates on every CI run:
 - [ADR-0013 — When the scanner is unsure, it hands back prose](docs/adr/0013-the-scanner-hands-back-prose.md)
 - [ADR-0014 — A relation is spelled in both directions](docs/adr/0014-a-relation-is-spelled-both-ways.md)
 - [ADR-0015 — Feedback goes where the tools already look](docs/adr/0015-feedback-goes-where-the-tools-already-look.md)
+- [ADR-0016 — A query needs a sentence before it is a rule](docs/adr/0016-a-query-needs-a-sentence.md)
+- [ADR-0017 — A predicate must finish](docs/adr/0017-a-predicate-must-finish.md)
+- [ADR-0018 — The configuration file is the root](docs/adr/0018-the-configuration-file-is-the-root.md)
+- [ADR-0019 — The sweep runs in shards](docs/adr/0019-the-sweep-runs-in-shards.md)
+- [ADR-0020 — A diff names only what it can tell apart](docs/adr/0020-a-diff-names-what-it-can-tell-apart.md)
 
 **Zero runtime dependencies.** Node 22+, native ESM, TypeScript strict with
 `noUncheckedIndexedAccess` and `exactOptionalPropertyTypes`. The Markdown
-scanner, the front-matter reader, the glob matcher and the query parser are all
-written here, so the whole package is auditable in an afternoon.
+scanner, the front-matter reader, the glob matcher, the query parser and the
+regular-expression matcher behind `~=` are all written here, so the whole
+package is auditable in an afternoon — and nothing in the pipeline can take
+longer than its input.
 
 **Verified, not just covered.** 411 tests; 94.2% statement and 96.9% line
 coverage. Coverage says a line ran, so the suite is also held to a mutation

package/dist/cli.d.ts

@@ -1,9 +1,10 @@
 /**
  * The command line.
  *
- * Four verbs, because there are four questions worth asking of a specification
+ * Five verbs, because there are five questions worth asking of a specification
  * graph: is it consistent (`check`), what does it contain (`query`), what does
- * it look like (`graph`), and what will you check for me (`rules`).
+ * it look like (`graph`), what will you check for me (`rules`), and what did a
+ * change do to it (`diff`).
  *
  * Exit codes are the contract with CI: `0` clean, `1` findings, `2` the tool
  * itself could not run. A usage mistake never masquerades as a passing build.
@@ -16,11 +17,19 @@ export declare const EXIT_ERROR = 2;
 export declare class UsageError extends Error {
     constructor(message: string);
 }
-export type Command = 'check' | 'query' | 'graph' | 'rules';
+export type Command = 'check' | 'query' | 'graph' | 'rules' | 'diff';
 export interface CliOptions {
     readonly command: Command;
     readonly patterns: readonly string[];
     readonly root: string;
+    /**
+     * Whether `--root` was given.
+     *
+     * When it was, the caller has named the root and configuration is read from
+     * exactly there. When it was not, the root is discovered - which is a
+     * different question from what the root currently is. See ADR-0018.
+     */
+    readonly rootExplicit: boolean;
     readonly ignore: readonly string[];
     /** Reference targets to leave unreported when they do not resolve. */
     readonly ignoreReferences: readonly string[];
@@ -45,7 +54,7 @@ export interface CliOptions {
     readonly ratchet: boolean;
     /** Skip the repository configuration file entirely. */
     readonly noConfig: boolean;
-    readonly format: 'human' | 'json' | 'sarif';
+    readonly format: 'human' | 'json' | 'sarif' | 'markdown';
     readonly graphFormat: GraphFormat;
     readonly severities: Partial<Record<AnyRuleId, Severity>>;
     readonly color: boolean | null;
@@ -68,7 +77,7 @@ export interface CliIO {
     readonly env?: Readonly<Record<string, string | undefined>> | undefined;
     readonly isTTY?: boolean | undefined;
 }
-export declare const HELP = "spec-graph - turn Markdown specifications into a verifiable graph\n\nUSAGE\n  spec-graph [check] [patterns...] [options]\n  spec-graph query <selector> [patterns...] [options]\n  spec-graph graph [patterns...] [--graph-format dot|mermaid|json]\n  spec-graph rules [--explain]\n\nCOMMANDS\n  check     Validate the specification graph. The default.\n  query     Run a selector and print the matching paths.\n  graph     Export the graph for Graphviz, Mermaid, or another tool.\n  rules     List the diagnostics that will run, built in and project.\n\nOPTIONS\n  --root <dir>            Directory the patterns resolve against (default: cwd)\n  --ignore <glob>         Skip paths. Repeatable.\n  --ignore-ref <glob>     Do not report these reference targets when they fail\n                          to resolve, for repositories where [[...]] tags a\n                          concept rather than naming a file. Repeatable.\n                          Suppresses findings only, never edges.\n  --family <name>         Families a bare identifier in prose may name. When\n                          given, everything else stays prose. Repeatable.\n  --ignore-family <name>  Families that are never citations - RFC when the repo\n                          cites RFC 2119 and keeps its own RFCs. Repeatable.\n  --history <glob>        Files that log what was decided rather than deciding\n                          it - journals, changelogs, minutes. Their links are\n                          still checked; their obligations are not. Repeatable.\n  --baseline <file>       Accept the findings recorded in this file and report\n                          only what is new since. Missing file = accept nothing.\n  --record-baseline <f>   Write today's findings to this file as accepted debt,\n                          and exit 0 without judging them.\n  --ratchet               Also fail when a baseline entry no longer occurs, so\n                          a paid-off exemption cannot outlive the defect.\n  --no-config             Ignore .spec-graph.json and the package.json key.\n  --format <fmt>          human, json, or sarif - the interchange format\n                          GitHub code scanning and editors already read\n                          (default: human)\n  --graph-format <fmt>    dot, mermaid or json (default: dot)\n  --documents-only        Leave items out of the exported graph\n  --rule <id>=<severity>  Override one rule: error, warn, info or off. A project\n                          rule is named in full: project:no-draft-dependency.\n                          Repeatable.\n  --max <n>               Show at most n findings (0 = no limit)\n  --max-warnings <n>      Fail when warnings exceed n (default: no limit)\n  --strict                Raise every warning to an error. An explicit --rule\n                          still wins, so --strict --rule x=warn exempts x.\n  --color / --no-color    Force colour on or off\n  --ascii                 Use ASCII glyphs only\n  --verbose               Include parse problems, per-file detail, and every\n                          reference --ignore-ref or --ignore-family silenced\n  -h, --help              Show this help\n  -v, --version           Show the version\n\nSELECTORS\n  A selector is a path through the graph.\n\n    item[openness=open] -delegates-to-> document[phase=retired]\n    document[phase=active] -assumes-> document[phase=retired]\n    document =supersedes=> document\n    *[path^=docs/adr] -contains-> item[state=accepted-debt]\n\n  Node types:  document, item, * (any)\n  Attributes:  id, kind, title, path, file, line, phase, status, receptivity,\n               alias, document, state (or disposition), openness, section,\n               text, body, evidence, conflicted, fm.<front-matter-key>\n  Operators:   = != ^= $= *= ~=   and [attr] for \"is present\"\n               ~= is a JavaScript regular expression, run once per node; ^= $=\n               and *= cover most cases and cannot backtrack\n  Relations:   -kind->  <-kind-   =kind=>  <=kind=   (= forms are transitive)\n\nEXIT CODES\n  0  clean\n  1  findings at error severity (or over --max-warnings)\n  2  the tool could not run\n\nCONFIGURATION\n  Anything repeated on every run belongs in the repository rather than in the\n  command. spec-graph reads the first of these that exists:\n\n    .spec-graph.json, spec-graph.config.json, or a \"spec-graph\" key in\n    package.json\n\n    { \"patterns\": [\"docs/**/*.md\"],\n      \"ignoreReferences\": [\"trap *\"],\n      \"ignoreFamilies\": [\"RFC\"],\n      \"historyPatterns\": [\"**/JOURNAL_*.md\", \"archive/**\"],\n      \"baseline\": \".spec-graph-baseline.json\",\n      \"severities\": { \"self-reference\": \"off\" },\n      \"strict\": true }\n\n  A flag always wins over the file, and list flags add to it rather than\n  replacing it.\n\nPROJECT RULES\n  A convention spec-graph never anticipated is a selector plus a sentence, and\n  belongs in the same file. Every rule declared here runs beside the built-ins\n  and is reported, baselined, escalated by --strict and silenced by --rule in\n  exactly the same way.\n\n    { \"rules\": {\n        \"no-draft-dependency\": {\n          \"query\": \"document[phase=active] -depends-on-> document[phase=draft]\",\n          \"message\": \"{0} depends on {1}, which is still a draft\",\n          \"hint\": \"wait for {1} to be accepted, or drop it from {0.path}\",\n          \"severity\": \"error\" } } }\n\n  query     One selector, or a list of them read as a union.\n  message   The headline. {0} is the first node on the path, {1} the next;\n            {1.phase} and {0.fm.owner} read any selector attribute.\n  hint      The next action. Optional, and templated the same way.\n  severity  error, warn, info or off. Defaults to warn - a rule a team has\n            just written has not yet earned the right to fail their build.\n\n  The id is the name with project: in front, which is why it can never collide\n  with a built-in. A selector that does not parse, or a {2} the query can never\n  reach, is reported when the file is read rather than found missing later.\n\nADOPTING THIS ON AN OLD REPOSITORY\n  Record what is already wrong, then report only what happens next:\n\n    spec-graph check --record-baseline .spec-graph-baseline.json\n    git add .spec-graph-baseline.json\n\n  The file is keyed by specification and citation, not by line number, so it\n  survives edits, moves and renames. Findings that stop occurring are reported\n  so the file can be tightened; nothing new gets in.\n\nEXAMPLES\n  spec-graph \"docs/**/*.md\"\n  spec-graph check --rule self-reference=off --format json\n  spec-graph query 'item[openness=open] -delegates-to-> document[phase=retired]'\n  spec-graph check --ignore-ref \"trap *\"    # [[trap 55]] tags a concept, not a file\n  spec-graph graph --documents-only --graph-format mermaid > graph.mmd\n  spec-graph check --history \"**/JOURNAL_*.md\"   # a log is not a specification\n  spec-graph check --baseline .spec-graph-baseline.json\n  spec-graph rules --explain                # including this repository's own\n  spec-graph check --rule project:no-draft-dependency=off\n";
+export declare const HELP = "spec-graph - turn Markdown specifications into a verifiable graph\n\nUSAGE\n  spec-graph [check] [patterns...] [options]\n  spec-graph query <selector|project:rule> [patterns...] [options]\n  spec-graph graph [patterns...] [--graph-format dot|mermaid|json]\n  spec-graph rules [rule-id] [--explain]\n  spec-graph diff <before.json> <after.json> [--format human|json|markdown]\n\nCOMMANDS\n  check     Validate the specification graph. The default.\n  query     Run a selector - or a registered project rule, by its id - and\n            print the matching paths.\n  graph     Export the graph for Graphviz, Mermaid, or another tool.\n  rules     List the diagnostics that will run, built in and project. Name one\n            to see only that one; --explain adds its selector and the ADR that\n            decided it.\n  diff      Compare two graph exports: documents added, removed, moved or\n            accepted; relations added or removed; and obligations resolved or\n            reopened where they can be told apart. Make both exports with the\n            same spec-graph. Exits 0 whether anything changed or not.\n\nOPTIONS\n  --root <dir>            Directory the patterns resolve against. Without it,\n                          .spec-graph.json is looked for in the working\n                          directory and then upward as far as the repository,\n                          and the directory holding it becomes the root - so a\n                          run from a subdirectory reports what a run from the\n                          top reports. Paths typed on the command line stay\n                          relative to where they were typed.\n  --ignore <glob>         Skip paths. Repeatable.\n  --ignore-ref <glob>     Do not report these reference targets when they fail\n                          to resolve, for repositories where [[...]] tags a\n                          concept rather than naming a file. Repeatable.\n                          Suppresses findings only, never edges.\n  --family <name>         Families a bare identifier in prose may name. When\n                          given, everything else stays prose. Repeatable.\n  --ignore-family <name>  Families that are never citations - RFC when the repo\n                          cites RFC 2119 and keeps its own RFCs. Repeatable.\n  --history <glob>        Files that log what was decided rather than deciding\n                          it - journals, changelogs, minutes. Their links are\n                          still checked; their obligations are not. Repeatable.\n  --baseline <file>       Accept the findings recorded in this file and report\n                          only what is new since. Missing file = accept nothing.\n  --record-baseline <f>   Write today's findings to this file as accepted debt,\n                          and exit 0 without judging them.\n  --ratchet               Also fail when a baseline entry no longer occurs, so\n                          a paid-off exemption cannot outlive the defect.\n  --no-config             Ignore .spec-graph.json and the package.json key.\n  --format <fmt>          human, json, sarif or markdown. sarif is the\n                          interchange format GitHub code scanning and editors\n                          already read, for check; markdown is a table for a\n                          pull-request comment or $GITHUB_STEP_SUMMARY, for check\n                          or diff (default: human)\n  --graph-format <fmt>    dot, mermaid or json (default: dot)\n  --documents-only        Leave items out of the exported graph\n  --rule <id>=<severity>  Override one rule: error, warn, info or off. A project\n                          rule is named in full: project:no-draft-dependency.\n                          Repeatable.\n  --max <n>               Show at most n findings (0 = no limit)\n  --max-warnings <n>      Fail when warnings exceed n (default: no limit)\n  --strict                Raise every warning to an error. An explicit --rule\n                          still wins, so --strict --rule x=warn exempts x.\n  --color / --no-color    Force colour on or off\n  --ascii                 Use ASCII glyphs only\n  --verbose               Include parse problems, per-file detail, and every\n                          reference --ignore-ref or --ignore-family silenced\n  -h, --help              Show this help\n  -v, --version           Show the version\n\nSELECTORS\n  A selector is a path through the graph.\n\n    item[openness=open] -delegates-to-> document[phase=retired]\n    document[phase=active] -assumes-> document[phase=retired]\n    document =supersedes=> document\n    *[path^=docs/adr] -contains-> item[state=accepted-debt]\n\n  Node types:  document, item, * (any)\n  Attributes:  id, kind, title, path, file, line, phase, status, receptivity,\n               alias, document, state (or disposition), openness, section,\n               text, body, evidence, conflicted, fm.<front-matter-key>\n  Operators:   = != ^= $= *= ~=   and [attr] for \"is present\"\n               ~= is a regular expression, matched by an automaton that cannot\n               backtrack: linear in the subject, whatever the pattern. It reads\n               the usual syntax minus backreferences and lookaround, which are\n               not regular - both are refused when the selector is read, with\n               the character pointed at\n  Relations:   -kind->  <-kind-   =kind=>  <=kind=   (= forms are transitive)\n\nEXIT CODES\n  0  clean\n  1  findings at error severity (or over --max-warnings)\n  2  the tool could not run\n\nCONFIGURATION\n  Anything repeated on every run belongs in the repository rather than in the\n  command. spec-graph reads the first of these that exists:\n\n    .spec-graph.json, spec-graph.config.json, or a \"spec-graph\" key in\n    package.json\n\n    { \"patterns\": [\"docs/**/*.md\"],\n      \"ignoreReferences\": [\"trap *\"],\n      \"ignoreFamilies\": [\"RFC\"],\n      \"historyPatterns\": [\"**/JOURNAL_*.md\", \"archive/**\"],\n      \"baseline\": \".spec-graph-baseline.json\",\n      \"severities\": { \"self-reference\": \"off\" },\n      \"strict\": true }\n\n  A flag always wins over the file, and list flags add to it rather than\n  replacing it.\n\nPROJECT RULES\n  A convention spec-graph never anticipated is a selector plus a sentence, and\n  belongs in the same file. Every rule declared here runs beside the built-ins\n  and is reported, baselined, escalated by --strict and silenced by --rule in\n  exactly the same way.\n\n    { \"rules\": {\n        \"no-draft-dependency\": {\n          \"query\": \"document[phase=active] -depends-on-> document[phase=draft]\",\n          \"message\": \"{0} depends on {1}, which is still a draft\",\n          \"hint\": \"wait for {1} to be accepted, or drop it from {0.path}\",\n          \"severity\": \"error\" } } }\n\n  query     One selector, or a list of them read as a union.\n  message   The headline. {0} is the first node on the path, {1} the next;\n            {1.phase} and {0.fm.owner} read any selector attribute.\n  hint      The next action. Optional, and templated the same way.\n  severity  error, warn, info or off. Defaults to warn - a rule a team has\n            just written has not yet earned the right to fail their build.\n\n  The id is the name with project: in front, which is why it can never collide\n  with a built-in. A selector that does not parse, or a {2} the query can never\n  reach, is reported when the file is read rather than found missing later.\n\n  To see what one matches without copying its selector out of the file:\n\n    spec-graph query project:no-draft-dependency --verbose\n\nADOPTING THIS ON AN OLD REPOSITORY\n  Record what is already wrong, then report only what happens next:\n\n    spec-graph check --record-baseline .spec-graph-baseline.json\n    git add .spec-graph-baseline.json\n\n  The file is keyed by specification and citation, not by line number, so it\n  survives edits, moves and renames. Findings that stop occurring are reported\n  so the file can be tightened; nothing new gets in.\n\nEXAMPLES\n  spec-graph \"docs/**/*.md\"\n  spec-graph check --rule self-reference=off --format json\n  spec-graph query 'item[openness=open] -delegates-to-> document[phase=retired]'\n  spec-graph check --ignore-ref \"trap *\"    # [[trap 55]] tags a concept, not a file\n  spec-graph graph --documents-only --graph-format mermaid > graph.mmd\n  spec-graph check --history \"**/JOURNAL_*.md\"   # a log is not a specification\n  spec-graph check --baseline .spec-graph-baseline.json\n  spec-graph rules --explain                # including this repository's own\n  spec-graph rules ghost-handover --explain      # and why it exists\n  spec-graph check --rule project:no-draft-dependency=off\n  spec-graph query project:no-draft-dependency   # what does that rule match?\n  spec-graph check --format markdown >> \"$GITHUB_STEP_SUMMARY\"\n  spec-graph diff base.json head.json --format markdown >> \"$GITHUB_STEP_SUMMARY\"\n";
 /** Parses argv into options. Throws {@link UsageError} on anything malformed. */
 export declare function parseArgs(argv: readonly string[], cwd: string): CliOptions;
 /** Runs the CLI and returns the process exit code. Never throws. */