@erclx/aitk 0.109.0 → 0.111.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -38,11 +38,15 @@ Nothing refreshes on its own. Claude Code ships auto-update off for third-party
38
38
  ```bash
39
39
  claude plugin marketplace update aitk
40
40
  claude plugin update aitk@aitk
41
- bun install --global @erclx/aitk
41
+ aitk upgrade
42
42
  ```
43
43
 
44
44
  The first two update the skills, the third updates the CLI, and they move independently. Restart Claude Code, or run `/reload-plugins`, to pick the skills up.
45
45
 
46
+ `aitk upgrade` reads the package manager off its own install path and reinstalls with that one, so you don't have to remember which put it there. It names what it detected before it runs anything, and it refuses a source checkout rather than reinstalling over your clone.
47
+
48
+ You don't have to wait until something breaks to find out you're behind. `aitk sync --check` and `aitk claude skills drift` both report the installed version against the newest published one, and neither changes its exit code over it, so an offline machine reads unknown rather than red.
49
+
46
50
  To stop doing this by hand, turn auto-update on once under `/plugin` in the Marketplaces tab. Confirm what you are running with `aitk --version` and `claude plugin list`.
47
51
 
48
52
  ## Why
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "aitk",
3
3
  "description": "Automated governance, versioning, and discovery tools for Claude Code.",
4
- "version": "0.109.0",
4
+ "version": "0.111.0",
5
5
  "author": {
6
6
  "name": "Eric Le",
7
7
  "url": "https://github.com/erclx"
@@ -0,0 +1,88 @@
1
+ ---
2
+ title: Audits
3
+ description: Running every health check as one set, what the single verdict means, the exit code each outcome takes, the retained baseline and the delta it reports, and which corpora are kept out of the record
4
+ ---
5
+
6
+ # Audits
7
+
8
+ `aitk audits run` runs every audit this repository owns, reports each one under a single verdict, and compares each count to the floor the last recorded run left behind. Nothing here is a new measure. Every number it prints comes from a verb that already published it, which is what makes the command cheap and what separates it from writing another check.
9
+
10
+ ```bash
11
+ aitk audits run
12
+ aitk audits run --json
13
+ aitk audits run --record
14
+ aitk audits list --json
15
+ ```
16
+
17
+ | Option | Behavior |
18
+ | --------------- | ----------------------------------------------------------------- |
19
+ | `--json` | Add a machine-readable record on stdout, keeping the frame |
20
+ | `--root <path>` | Measure this tree instead of the current worktree |
21
+ | `--record` | Write this run's tracked counts to `.claude/audits/baseline.json` |
22
+
23
+ ## What it runs
24
+
25
+ Twelve verbs, listed by `aitk audits list`. Each runs once in its fullest form, and the aggregate reads that verb's own record rather than imposing a shared envelope on it. Every one of those records already has consumers naming its keys, so a common shape would be a breaking change bought for tidiness.
26
+
27
+ The verbs walk separate trees and share no state, so they run together. Measured on the authoring machine, twelve runs finish in 0.8 seconds of wall clock against 4.4 seconds of processor, which is under every other stage in `bun run check`.
28
+
29
+ Each is invoked as the CLI the caller is running rather than as a global `aitk`. A globally installed binary resolves to the main checkout no matter which worktree is executing, so the aggregate would measure a tree the branch never touched and report a pass over it.
30
+
31
+ ## What gates and what reports
32
+
33
+ Three findings fail the run: an unresolved context citation, a banned character, word, or spelling, and a skill folder carrying no `REQUIREMENT.md`. Those are the same three `scripts/core/verify.sh` already fails a push on, and the aggregate adds nothing to the set. Each is a fact with no false-positive class behind it.
34
+
35
+ Everything else reports. A heavy bullet, a long entry, a board row nothing resolves, a degradation term in a comment, and an implementation reaching history ahead of its test are judgments a reader settles. A push failing on one of those teaches contributors to route around the stage, which is the split recorded across every audit here and the one this command inherits rather than moves.
36
+
37
+ Growth against the baseline reports too. The standards behind the largest measures set no hard cap, so a count that rose is a fact about the corpus and a judgment about whether it matters.
38
+
39
+ ## Exit codes
40
+
41
+ | Code | Meaning |
42
+ | ---- | -------------------------------------------------------------------- |
43
+ | `0` | Every audit reported and none carried a finding that is a fact |
44
+ | `1` | Refused, with the reason on stderr |
45
+ | `2` | An audit carries a finding that is a fact |
46
+ | `3` | An audit did not report, so the run measured less than the whole set |
47
+
48
+ `3` is a defect in the run rather than in the tree, which is why it takes a code of its own. An aggregate reporting a pass over a set it never finished measuring is the failure the command exists against, so an unparseable record, a verb that could not be started, and an exit no verb documents all land there instead of quietly counting as clean. The `markdown audit` empty-ban-set exit is the precedent: a corpus walked with nothing looked for is a broken check, not a clean tree.
49
+
50
+ The verdict is published as a field as well as an exit code. A reader branching on `summary.verdict` gets `clean`, `reported`, `findings`, or `incomplete` without inferring the difference between a quiet corpus and one nobody finished measuring.
51
+
52
+ ### An absent corpus is not an unmeasured one
53
+
54
+ A per-machine folder that is not on this disk reports as `absent` and moves neither the verdict nor the exit code. Every one of those folders is gitignored, so a fresh clone and every CI run carry none of them, and counting six expected absences as failures pins the verdict at `incomplete` forever. A signal that never changes is one nobody reads after the second time they see it.
55
+
56
+ The allowance is narrow. It covers a per-machine corpus refusing because its folder is missing, and nothing else. A tracked tree that cannot be found is a broken checkout, and a per-machine verb refusing for any other reason is a broken verb. Both stay `unmeasured`.
57
+
58
+ Every run states how many corpora it measured against how many it skipped, including a run with no findings at all. A count of what passed reads as a verdict on the whole set unless the run also says what it never reached.
59
+
60
+ ### What a shallow checkout changes
61
+
62
+ `aitk gov test-order` scopes its range against the trunk and falls back to the root commit when no trunk ref resolves. A depth-1 checkout has neither, so the range is empty and the verb reports zero of everything rather than refusing. The numbers are real for the history present, which means a shallow run under-reports against a baseline taken from a full clone and shows as shrinkage. That is the safe direction, and it is the reason a `test-order` delta is worth reading only from a clone carrying the history.
63
+
64
+ ## The retained baseline
65
+
66
+ `.claude/audits/baseline.json` holds the counts from the last run recorded with `--record`, alongside the day it was taken and the commit it was read at. Each later run reports which counts moved and by how much.
67
+
68
+ The file is committed. A per-machine record cannot answer the question this half exists for, which is whether a number grew since anyone last looked, and a fresh checkout has to inherit that answer rather than start over. The cost is that a branch moving a count either fixes it or re-records it and says why, which is the shape the consumed-copy assertion already carries.
69
+
70
+ It sits under the project root rather than beside the aggregator in `src/`. The numbers describe one repository's corpus, and `src/` ships to every project that installs the CLI, so a baseline in the package would hand a target these counts to measure its own tree against.
71
+
72
+ ### What is kept out of it
73
+
74
+ A gitignored record folder holds one machine's session scratch. Its counts describe somebody else's disk, so committing them writes a floor no other clone can reproduce and every contributor reads a regression against a number that was never about their tree. The board and five of the six record kinds are therefore reported on every run and recorded in none. `records-standards` is the exception, because that corpus is tracked.
75
+
76
+ An audit that did not report is left out rather than written as zero. Zero there records a clean corpus nobody measured, and the next run reads its real numbers as a regression against a floor that was never taken.
77
+
78
+ ### The first run
79
+
80
+ A run with no baseline behind it says so rather than showing a delta of zero. Those two states say the same thing to a reader and mean opposite things, which is a defect this repository has already had to fix twice elsewhere. The same reasoning covers a check the baseline never recorded, a key the run produced that the record does not carry, and a key the record carries that this run did not produce. Each is named for what it is.
81
+
82
+ A hand-edited baseline that does not parse refuses the whole run. Reading a broken record as an absent one would reset the floor the file exists to hold, silently.
83
+
84
+ ## In the verify pipeline
85
+
86
+ `bun run check` runs the set as a reporting stage after the three gating stages, and never fails on it. Those three stages keep their own specific remedies, so the aggregate reports the rest and the growth, and a fact still fails the push at the stage that names what to do about it.
87
+
88
+ The stage reads `summary`, a flat object of scalars published beside the nested arrays. Every key in it is unique across the whole record, so a shell stage greps one out without a JSON parser. The three verbs the gating stages already ran walk their trees a second time here, which is the 0.8 seconds measured above and the reason the whole set runs rather than only the part those stages skip: one verdict over every audit is the value, and a stage measuring a subset would report a health nobody took.
@@ -13,7 +13,7 @@ Full help: `aitk <command> --help`. Behavior notes for the install and sync verb
13
13
  | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
14
14
  | `aitk init [path]` | Bootstrap a project with selected toolkit domains |
15
15
  | `aitk sync [path]` | Sync all installed domains in a target project |
16
- | `aitk sync --check` | Report toolkit drift without writing (`--json`, `--exit-code`) |
16
+ | `aitk sync --check` | Report toolkit drift and the installed version against the newest published (`--json`, `--exit-code`) |
17
17
  | `aitk sandbox [cat:cmd]` | Run sandbox scenarios (interactive or routed), toolkit-only like the tree it reads |
18
18
  | `aitk sandbox reset` | Reset sandbox to baseline |
19
19
  | `aitk sandbox clean` | Wipe the sandbox |
@@ -44,9 +44,12 @@ Full help: `aitk <command> --help`. Behavior notes for the install and sync verb
44
44
  | `aitk context audit` | Report required sections, length, cited paths, reference form, catalog tables, provenance, superseded-decision narration, and index drift |
45
45
  | `aitk markdown audit` | Fail any markdown path on a banned character, word, or spelling, and report the structural checkpoints |
46
46
  | `aitk claude skills audit` | Report both skill corpora against the mechanical rules in `standards/skill.md` |
47
- | `aitk claude skills drift` | Name the shipped skill bodies rewritten between a given ref and `HEAD` (`--json`) |
47
+ | `aitk claude skills drift` | Name the shipped skill bodies rewritten between a given ref and `HEAD`, and the installed version against the newest published (`--json`) |
48
48
  | `aitk gov test-order` | Report where an implementation reached history ahead of the test covering it (`--json`) |
49
+ | `aitk audits run` | Run every audit as one set, report per check under one verdict, and compare each count to the recorded baseline (`--json`, `--record`) |
50
+ | `aitk audits list` | List every audit the set runs, with the corpus each reads and whether it gates (`--json`) |
49
51
  | `aitk capture [source]` | Render HTML capture sources to PNG, toolkit-only and absent from an installed package |
52
+ | `aitk upgrade` | Reinstall the CLI globally with the package manager the install path names (`--json`) |
50
53
 
51
54
  ## Domain commands
52
55
 
@@ -68,6 +71,7 @@ Each domain exposes a consistent shape where applicable: `list`, `install`, `syn
68
71
  | `comments` | `scan` |
69
72
  | `context` | `audit` |
70
73
  | `markdown` | `audit` |
74
+ | `audits` | `run`, `list` |
71
75
 
72
76
  Common patterns:
73
77
 
@@ -75,3 +79,17 @@ Common patterns:
75
79
  - `install <name> <path>` → install a specific entry into a target project.
76
80
  - `sync <path>` → reapply all installed entries in a target project.
77
81
  - `create [name]` → scaffold a new authoring entry in this repo.
82
+
83
+ ## Version skew
84
+
85
+ `aitk sync --check` and `aitk claude skills drift` are the two moments a target
86
+ already stops to reconcile with the toolkit, so each reports the installed
87
+ version against the newest published one. No other command performs the lookup,
88
+ which keeps a registry round trip out of the catalog reads an agent runs in a
89
+ loop.
90
+
91
+ The report carries three states and never changes an exit code. `behind` names
92
+ `aitk upgrade` as the remedy, `current` says so, and `unknown` carries the
93
+ reason the registry could not be reached. Branch on the `skew.state` field in
94
+ the JSON record rather than on the exit, since an offline machine has to read as
95
+ unmeasured rather than as a failing check.
@@ -8,6 +8,7 @@ category: Agent surface
8
8
 
9
9
  CLI catalog and invocation rules for agents, split by command domain. Start with overview.
10
10
 
11
+ - [Audits](audits.md): Running every health check as one set, what the single verdict means, the exit code each outcome takes, the retained baseline and the delta it reports, and which corpora are kept out of the record
11
12
  - [Capture](capture.md): Rendering committed HTML sources to PNG, what the command asserts about fonts, and why it is toolkit-only
12
13
  - [Command catalog](commands.md): Every project-level command and every domain subcommand, plus the shape each domain exposes
13
14
  - [Comments](comments.md): Comment density by language and kind, the two structural exclusions, and how the degradation sweep finds its vocabulary
@@ -11,15 +11,16 @@ What a skill or script reads to discover names at runtime, and how each domain i
11
11
 
12
12
  Use these to discover what's available instead of hardcoding names.
13
13
 
14
- | Command | Returns |
15
- | -------------------------------- | ---------------------------------------------------- |
16
- | `aitk tooling list --json` | Stacks, extends chain, dep and script counts |
17
- | `aitk snippets list --json` | Presets and categories with their slugs |
18
- | `aitk standards list --json` | Standards docs and the paths each governs |
19
- | `aitk gov list --json` | Governance stacks, rule sets, and unreferenced rules |
20
- | `aitk claude seeds list --json` | Seed doc sources with content |
21
- | `aitk claude skills list --json` | Plugin skills, descriptions, requirement flag |
22
- | `aitk docs list --json` | Consumer docs plus per-domain context |
14
+ | Command | Returns |
15
+ | -------------------------------- | ---------------------------------------------------------------- |
16
+ | `aitk tooling list --json` | Stacks, extends chain, dep and script counts |
17
+ | `aitk snippets list --json` | Presets and categories with their slugs |
18
+ | `aitk standards list --json` | Standards docs and the paths each governs |
19
+ | `aitk gov list --json` | Governance stacks, rule sets, and unreferenced rules |
20
+ | `aitk claude seeds list --json` | Seed doc sources with content |
21
+ | `aitk claude skills list --json` | Plugin skills, descriptions, requirement flag |
22
+ | `aitk docs list --json` | Consumer docs plus per-domain context |
23
+ | `aitk audits list --json` | Audits the set runs, the corpus each reads, and whether it gates |
23
24
 
24
25
  ### Catalog fields
25
26
 
@@ -129,4 +130,7 @@ AITK_NON_INTERACTIVE=1 aitk wiki init /path/to/project
129
130
 
130
131
  # Run a sandbox scenario non-interactively
131
132
  SANDBOX_SCENARIO=sync aitk sandbox infra:tooling
133
+
134
+ # Read every audit as one record. Exit 2 is a fact, 3 an audit that did not report
135
+ aitk audits run --json
132
136
  ```
@@ -114,6 +114,12 @@ When the toolkit updates, target projects pull changes per domain. There is one
114
114
 
115
115
  `aitk sync --check <path>` reports what has drifted without writing anything. It splits each difference by cause, which is the question that decides what to do next.
116
116
 
117
+ #### The binary first
118
+
119
+ The report opens by naming the binary running it. The installed version reads against the newest published one, and a version behind that points you at `aitk upgrade`, since every section under it is a reading taken by whichever toolkit you happen to have. An unreachable registry reports unknown with its reason rather than failing, so `--exit-code` still gates on the drift the check measured locally and an offline machine stays green.
120
+
121
+ #### Then the causes
122
+
117
123
  A `stale` file still matches what the toolkit installed, so the update is mechanical. A `customized` file carries local edits, so taking the upstream version is a decision and `aitk:claude-seed-sync` is the tool for it. A `stranded` file sits where an older toolkit installed it and the toolkit has since moved, which is what `aitk:migration-standards` handles.
118
124
 
119
125
  That attribution comes from `.claude/aitk.json`, a stamp every install and sync writes. Standards, snippets, and governance record a hash per installed file. Tooling records the stack chain it resolved instead, since its install runs no per-file walk to attribute.
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@erclx/aitk",
3
3
  "type": "module",
4
- "version": "0.109.0",
4
+ "version": "0.111.0",
5
5
  "description": "Infrastructure and quality tooling for developer workflows",
6
6
  "license": "MIT",
7
7
  "bin": {
@@ -25,6 +25,11 @@ SANDBOX_UNDECLARED_CEILING=47
25
25
  # a new arrival against, and a config file would absorb the arrival silently.
26
26
  GOV_EXPECTED_UNREFERENCED="260-shadcn 320-tanstack-query"
27
27
 
28
+ # The retained counts the audit stage compares each run against. Spelled here
29
+ # rather than derived, because this script only ever names the file in a remedy
30
+ # a reader has to be able to open, and `aitk audits run` owns writing it.
31
+ AUDITS_BASELINE=".claude/audits/baseline.json"
32
+
28
33
  check_dependencies() {
29
34
  command -v bun >/dev/null 2>&1 || log_error "bun is not installed"
30
35
  }
@@ -209,10 +214,11 @@ seed_entry_count() {
209
214
  awk '{ total += $1 } END { print total + 0 }'
210
215
  }
211
216
 
212
- # One numeric summary key out of a coverage report. The per-scenario objects carry
213
- # neither key this is called with, so the match reaches the top level alone and
214
- # the caller does not depend on the order the keys are emitted in.
215
- sandbox_summary_field() {
217
+ # One numeric summary key out of a command's JSON record. Every caller passes a
218
+ # name the nested objects in that record do not carry, so the match reaches the
219
+ # top level alone and the caller does not depend on the order the keys are
220
+ # emitted in.
221
+ json_summary_field() {
216
222
  printf '%s' "$2" | grep -o "\"$1\":[0-9]\+" | grep -o '[0-9]\+'
217
223
  }
218
224
 
@@ -466,8 +472,8 @@ main() {
466
472
  # `|| x=""` on both, because a grep that matches nothing exits non-zero and
467
473
  # errexit would take the script down at the assignment, before the guard
468
474
  # below could name what went missing.
469
- total=$(sandbox_summary_field totalScenarios "$coverage_output") || total=""
470
- armed=$(sandbox_summary_field armedScenarios "$coverage_output") || armed=""
475
+ total=$(json_summary_field totalScenarios "$coverage_output") || total=""
476
+ armed=$(json_summary_field armedScenarios "$coverage_output") || armed=""
471
477
  if [ -z "$total" ] || [ -z "$armed" ]; then
472
478
  log_error "The coverage report carried no scenario totals, so the stage measured nothing. Run bun src/cli.ts sandbox coverage --json."
473
479
  fi
@@ -478,6 +484,68 @@ main() {
478
484
  log_info "$armed of $total scenarios declare expectations, $undeclared undeclared against a ceiling of $SANDBOX_UNDECLARED_CEILING"
479
485
  fi
480
486
 
487
+ # The three stages above gate on the three findings here that are facts, and
488
+ # this stage reports the rest. It runs the whole set anyway rather than only
489
+ # what those stages skip, because the aggregate's own value is one verdict
490
+ # over every audit, and a stage measuring a subset would report a health this
491
+ # repository never took.
492
+ #
493
+ # The duplicate walk costs 0.8s wall against roughly 4.4s of processor,
494
+ # measured on the authoring machine at 12 verbs run together. That is under
495
+ # every other stage in this script, which is what settles the open question
496
+ # about whether the pipeline can afford it.
497
+ #
498
+ # This reports and never fails. Growth in a judgment count is the thing the
499
+ # baseline exists to make visible, and failing a push on one would teach a
500
+ # contributor to route around the stage, which is the split every audit stage
501
+ # here already keeps. A fact still fails the push, at the specific stage above
502
+ # that names its own remedy.
503
+ log_step "Audit set"
504
+ local audits_output audits_status=0 audits_grown audits_shrunk audits_facts audits_unmeasured audits_absent audits_unrecorded
505
+ audits_output=$(cd "$PROJECT_ROOT" && bun src/cli.ts audits run --json 2>/dev/null) || audits_status=$?
506
+ if [ -z "$audits_output" ]; then
507
+ log_warn "Skipped, the audit set did not report (exit $audits_status)"
508
+ else
509
+ audits_grown=$(json_summary_field grown "$audits_output") || audits_grown=""
510
+ audits_shrunk=$(json_summary_field shrunk "$audits_output") || audits_shrunk=""
511
+ audits_facts=$(json_summary_field facts "$audits_output") || audits_facts=""
512
+ audits_unmeasured=$(json_summary_field unmeasured "$audits_output") || audits_unmeasured=""
513
+ audits_absent=$(json_summary_field absent "$audits_output") || audits_absent=""
514
+ audits_unrecorded=$(json_summary_field unrecorded "$audits_output") || audits_unrecorded=""
515
+
516
+ # An absent field is a record this stage cannot read, which is not the same
517
+ # as a run with nothing to report. Reading it as zero would print a clean
518
+ # line over a summary nobody parsed.
519
+ if [ -z "$audits_grown" ] || [ -z "$audits_facts" ] || [ -z "$audits_unmeasured" ]; then
520
+ log_warn "The audit record carried no summary, so this stage measured nothing. Run bun src/cli.ts audits run."
521
+ else
522
+ # An absent per-machine folder is the ordinary state here rather than a
523
+ # finding, since every one of them is gitignored and CI carries none. It
524
+ # is still stated, because a stage naming only what it measured claims a
525
+ # coverage it does not have.
526
+ if [ -n "$audits_absent" ] && [ "$audits_absent" -gt 0 ]; then
527
+ log_info "$audits_absent per-machine corpus/corpora absent, so unmeasured here by design"
528
+ fi
529
+ if [ "$audits_unmeasured" -gt 0 ]; then
530
+ log_warn "$audits_unmeasured audit(s) did not report, so the set is incomplete. Run bun src/cli.ts audits run."
531
+ fi
532
+ if [ "$audits_facts" -gt 0 ]; then
533
+ log_warn "$audits_facts audit(s) carry a finding that is a fact. The stage above names the remedy."
534
+ fi
535
+ if [ -n "$audits_unrecorded" ] && [ "$audits_unrecorded" -gt 0 ]; then
536
+ log_warn "$audits_unrecorded tracked audit(s) have no recorded floor. Take one with bun src/cli.ts audits run --record."
537
+ fi
538
+ if [ "$audits_grown" -gt 0 ]; then
539
+ log_warn "$audits_grown measure(s) grew against $AUDITS_BASELINE. Run bun src/cli.ts audits run to see which, then fix them or re-record and say why."
540
+ else
541
+ log_info "No measure grew against $AUDITS_BASELINE"
542
+ fi
543
+ if [ -n "$audits_shrunk" ] && [ "$audits_shrunk" -gt 0 ]; then
544
+ log_info "$audits_shrunk measure(s) fell against $AUDITS_BASELINE"
545
+ fi
546
+ fi
547
+ fi
548
+
481
549
  # The plugin is the second delivery path and this is the only stage gating it,
482
550
  # so the skip below is for a contributor's machine rather than for the merge
483
551
  # gate. A runner installs the CLI as a workflow step, which makes an absent
@@ -0,0 +1,194 @@
1
+ import { mkdir, readFile, writeFile } from 'node:fs/promises'
2
+ import { dirname, join } from 'node:path'
3
+ import type { AuditResult } from '@/audits/catalog'
4
+
5
+ /**
6
+ * Where the retained counts live, relative to the project root.
7
+ *
8
+ * Committed rather than per-machine, because the question this half answers is
9
+ * whether a number grew since anyone last looked, and a fresh checkout has to
10
+ * inherit that answer. A per-machine record makes every contributor's first run
11
+ * a first run.
12
+ *
13
+ * Under the project root rather than beside the aggregator in `src/`, because
14
+ * the numbers describe one repository's corpus and `src/` ships to every target
15
+ * that installs the CLI. A baseline in the package would hand a target this
16
+ * repository's counts to measure its own tree against.
17
+ */
18
+ export const BASELINE_REL = join('.claude', 'audits', 'baseline.json')
19
+
20
+ export interface Baseline {
21
+ /** The day the record was taken, as `YYYY-MM-DD`. */
22
+ readonly recordedAt: string
23
+ /** The commit the counts were read at, so a reader can reproduce them. */
24
+ readonly commit: string
25
+ readonly checks: Readonly<Record<string, Readonly<Record<string, number>>>>
26
+ }
27
+
28
+ export interface Stamp {
29
+ readonly recordedAt: string
30
+ readonly commit: string
31
+ }
32
+
33
+ export interface MovedCount {
34
+ readonly key: string
35
+ readonly from: number
36
+ readonly to: number
37
+ readonly delta: number
38
+ }
39
+
40
+ export type Delta =
41
+ | {
42
+ readonly id: string
43
+ readonly kind: 'compared'
44
+ readonly moved: readonly MovedCount[]
45
+ /** Keys the run reproduced exactly. */
46
+ readonly steady: readonly string[]
47
+ /** Keys the run produced that the baseline never recorded. */
48
+ readonly added: readonly { key: string; to: number }[]
49
+ /** Keys the baseline holds that this run did not produce. */
50
+ readonly dropped: readonly { key: string; from: number }[]
51
+ }
52
+ /** No recorded floor, so a zero delta would be indistinguishable from quiet. */
53
+ | { readonly id: string; readonly kind: 'unrecorded' }
54
+ /** Gitignored scratch, whose counts are one machine's and answer nobody else. */
55
+ | { readonly id: string; readonly kind: 'per-machine' }
56
+ /** The audit did not report, so there is nothing to compare. */
57
+ | { readonly id: string; readonly kind: 'unmeasured' }
58
+
59
+ /**
60
+ * Builds the record a run leaves behind.
61
+ *
62
+ * Only a tracked corpus is retained. A gitignored record folder holds one
63
+ * machine's session scratch, so committing its counts writes a floor no other
64
+ * clone can reproduce, and every contributor would read a regression against a
65
+ * number that describes somebody else's disk.
66
+ *
67
+ * An audit that did not report is left out rather than written as zero. Zeros
68
+ * there record a clean corpus nobody measured, and the next run reads its real
69
+ * numbers as a regression against a floor that was never taken.
70
+ */
71
+ export function baselineFrom(
72
+ results: readonly AuditResult[],
73
+ stamp: Stamp,
74
+ ): Baseline {
75
+ const checks: Record<string, Record<string, number>> = {}
76
+
77
+ for (const result of results) {
78
+ if (!result.tracked || result.counts === undefined) continue
79
+ checks[result.id] = { ...result.counts }
80
+ }
81
+
82
+ return { recordedAt: stamp.recordedAt, commit: stamp.commit, checks }
83
+ }
84
+
85
+ /**
86
+ * Compares this run against the recorded floor, one audit at a time.
87
+ *
88
+ * A first run reports `unrecorded` rather than a delta of zero. A zero delta
89
+ * against an absent baseline says the same thing as a corpus that did not move,
90
+ * and those are the two states this repository has already had to separate
91
+ * twice elsewhere.
92
+ */
93
+ export function compareBaseline(
94
+ baseline: Baseline | undefined,
95
+ results: readonly AuditResult[],
96
+ ): Delta[] {
97
+ return results.map((result) => {
98
+ if (!result.tracked) return { id: result.id, kind: 'per-machine' as const }
99
+ if (result.counts === undefined) {
100
+ return { id: result.id, kind: 'unmeasured' as const }
101
+ }
102
+
103
+ const recorded = baseline?.checks[result.id]
104
+ if (recorded === undefined) {
105
+ return { id: result.id, kind: 'unrecorded' as const }
106
+ }
107
+
108
+ const moved: MovedCount[] = []
109
+ const steady: string[] = []
110
+ const added: { key: string; to: number }[] = []
111
+ const dropped: { key: string; from: number }[] = []
112
+
113
+ for (const [key, to] of Object.entries(result.counts)) {
114
+ const from = recorded[key]
115
+ if (from === undefined) {
116
+ added.push({ key, to })
117
+ } else if (from === to) {
118
+ steady.push(key)
119
+ } else {
120
+ moved.push({ key, from, to, delta: to - from })
121
+ }
122
+ }
123
+
124
+ for (const [key, from] of Object.entries(recorded)) {
125
+ if (!(key in result.counts)) dropped.push({ key, from })
126
+ }
127
+
128
+ return {
129
+ id: result.id,
130
+ kind: 'compared' as const,
131
+ moved,
132
+ steady,
133
+ added,
134
+ dropped,
135
+ }
136
+ })
137
+ }
138
+
139
+ function isBaseline(value: unknown): value is Baseline {
140
+ if (typeof value !== 'object' || value === null) return false
141
+ const record = value as Record<string, unknown>
142
+ return (
143
+ typeof record.recordedAt === 'string' &&
144
+ typeof record.commit === 'string' &&
145
+ typeof record.checks === 'object' &&
146
+ record.checks !== null &&
147
+ !Array.isArray(record.checks)
148
+ )
149
+ }
150
+
151
+ /**
152
+ * Reads the recorded floor, or `undefined` when none has been taken.
153
+ *
154
+ * An absent file and a broken one are different answers. Absent is a first run.
155
+ * Broken is a record someone hand-edited into a shape nothing can read, and
156
+ * reading that as absent would silently reset the floor the file exists to hold.
157
+ */
158
+ export async function readBaseline(
159
+ root: string,
160
+ ): Promise<Baseline | undefined> {
161
+ const path = join(root, BASELINE_REL)
162
+
163
+ let raw: string
164
+ try {
165
+ raw = await readFile(path, 'utf8')
166
+ } catch {
167
+ return undefined
168
+ }
169
+
170
+ let parsed: unknown
171
+ try {
172
+ parsed = JSON.parse(raw)
173
+ } catch {
174
+ throw new Error(`${BASELINE_REL} does not parse as JSON. Fix or delete it.`)
175
+ }
176
+
177
+ if (!isBaseline(parsed)) {
178
+ throw new Error(
179
+ `${BASELINE_REL} carries no recordedAt, commit, and checks. Fix or delete it.`,
180
+ )
181
+ }
182
+
183
+ return parsed
184
+ }
185
+
186
+ export async function writeBaseline(
187
+ root: string,
188
+ baseline: Baseline,
189
+ ): Promise<string> {
190
+ const path = join(root, BASELINE_REL)
191
+ await mkdir(dirname(path), { recursive: true })
192
+ await writeFile(path, `${JSON.stringify(baseline, null, 2)}\n`, 'utf8')
193
+ return path
194
+ }