@erclx/aitk 3.18.0 → 3.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "aitk",
3
3
  "description": "Automated governance, versioning, and discovery tools for Claude Code.",
4
- "version": "3.18.0",
4
+ "version": "3.19.0",
5
5
  "author": {
6
6
  "name": "Eric Le",
7
7
  "url": "https://github.com/erclx"
@@ -0,0 +1,68 @@
1
+ ---
2
+ title: Self-stated counts
3
+ description: Reading a sentence that asserts a closed catalog's size, how a match is decided, the plausibility filter that keeps a generic word from matching a subset, and why the sweep reports rather than gates
4
+ ---
5
+
6
+ # Self-stated counts
7
+
8
+ `aitk gov counts` reports a sentence stating how many members a closed catalog holds where the stated figure disagrees with what the tree actually counts. A document that reports its own catalog size carries a number nothing else compares against the tree, so it goes stale on the edit that moves the catalog and reads as current until a reviewer happens to recompute it.
9
+
10
+ ```bash
11
+ aitk gov counts
12
+ aitk gov counts --json
13
+ aitk gov counts --root ../my-app
14
+ ```
15
+
16
+ | Option | Behavior |
17
+ | --------------- | ---------------------------------------------------------- |
18
+ | `--root <path>` | Tree to read, defaulting to the current directory |
19
+ | `--json` | Add a machine-readable record on stdout, keeping the frame |
20
+
21
+ Under `--json` the record holds stdout alone and the frame still renders on stderr, refusals included, which is the split `output-shape.md` fixes for every mode.
22
+
23
+ ## The catalogs it reads
24
+
25
+ A closed set of six, each with a filesystem reader behind it rather than every list command this CLI ships:
26
+
27
+ - `skills`: shipped plugin skills, a folder per `claude/skills/*/SKILL.md`
28
+ - `rules`: authored governance rules, a file per `governance/rules/**/*.md`
29
+ - `standards`: the authoring corpus at `standards/`
30
+ - `snippets`: distinct entries across every `snippets/` category, matching what `regen-hero.sh` counts
31
+ - `commands`: top-level `aitk` commands, read off `src/cli.ts`'s registration imports
32
+ - `audits`: the registered set in `src/audits/catalog.ts`, this entry included
33
+
34
+ The set is closed rather than derived, so widening it is a deliberate change to this file rather than a side effect of a new list command shipping elsewhere. `commands` is the one catalog with no meaning outside this repository: a project installing this CLI carries no `src/cli.ts` of its own, and that catalog reads as not applicable there rather than as zero, since zero would report a finding against any such project stating its command count as anything but zero, forever.
35
+
36
+ ## How a match is decided
37
+
38
+ A sentence has to carry an assertion verb (`loads`, `ships`, `carries`, `holds`, `counts`, `totals`, `documents`, `declares`, `installs`, `lists`, `contains`, `comprises`) or an article (`the`, `a`, `an`) immediately ahead of the number, which is immediately ahead of the catalog noun, with one optional qualifying word between the number and the noun (`sixty-one shipped skills`). The number reads either as digits or as a spelled-out cardinal through ninety-nine, since this corpus states a catalog size in words as often as in digits.
39
+
40
+ The verb gate is not the first design tried. A bare number next to a catalog noun anywhere in the tracked corpus returned 290 findings against a repository whose actionable instance count was one. Reading that run showed why: `18 rules citing a standard`, `21 skill bodies`, and `eight internal skills` all pair a number with a catalog noun while naming a subset, an example, or a different catalog than the one matched, and that shape dominates ordinary prose. Every instance this sweep was written against reads the number as the direct object of a verb asserting the catalog's own total, and gating on that verb is what took the false-positive count from 290 to five on the same tree.
41
+
42
+ The article half was added after a first review of the shipped design found a live miss: `.claude/context/development/gates.md` stated a stale audit total while the tree held 20, and the verb gate never reached it, since that sentence puts its verb after the noun rather than ahead of the number. Widening the trigger set to admit an article ahead of the number closed that gap.
43
+
44
+ ## The plausibility filter
45
+
46
+ The verb gate alone still left four false positives standing: `carries two rules about a standard's own lifecycle`, `holds one rule or one fact` (twice, once per mirrored copy), and `documents two similar commands`. Each pairs an assertion verb with a catalog noun used for something the `rules` or `commands` catalog does not mean, and what tells those apart from the one live finding is magnitude: a catalog this sweep tracks drifts by a few members between the day a sentence was written and the day it is read, so a genuine staleness claim sits near the true count. `2` beside a true count of `59` is not a catalog that shrank, it is a different `rules` entirely.
47
+
48
+ A stated figure is read only when it sits within a factor of two of the true count, in either direction. That bound is tuned against the run this sweep was written against rather than reasoned to from first principles, the way `restated.ts` tunes its own document-frequency ceiling, and it is a property of the scale every catalog here sits at (the tens) rather than a universal rule.
49
+
50
+ The article gate is looser than the verb list and carries a real cost. Re-running against the tree once it widened caught the live miss above and also misread a passage in `.claude/context/standards/destinations.md` naming a real subset, a standard outside the retired `standards/bundled/` fan-out, as a claim about the whole standards catalog. The qualifying adjective there fills the same optional-word slot `sixty-one shipped skills` needs to match at all, and no syntactic rule tells a qualifier that narrows a catalog from one that only restates it. The finding could not be closed by correcting a figure, since the sentence was true when the corpus held it, only unmarked as a past state, which is the class the date exclusion already reads past. Naming the moment, the same way `.claude/ARCHITECTURE.md` dates its own figures, closed it with no change to the matcher.
51
+
52
+ ## What it does not measure
53
+
54
+ A delta phrased as a transition (`from fourteen to fifteen`), a fraction (`thirteen of sixteen`), and a total reached through an indirect noun (`denominator of sixty-one shipped skills`) are all catalog-size claims this corpus carries, and none matches the assertion-verb shape this reads. Each is a known gap rather than an oversight, left for a wider pass once this design's own false-positive rate is measured over more than one run.
55
+
56
+ A calendar date (`2026-08-21`) or a backticked commit reference in the same sentence reads the whole sentence past, since that is how this corpus already marks a figure as a historical record rather than a live claim. `.claude/ARCHITECTURE.md` and the context entries carry a figure this way deliberately, and every one of them stays correct forever.
57
+
58
+ The wider check over prose restatements is deliberately out of reach here. `aitk gov restated` already records a parser over prose as rejected, and this sweep reaches a number with a command behind it rather than two sentences a reader has to judge as agreeing.
59
+
60
+ ## Exit codes
61
+
62
+ Exit codes are `0` when no stated figure disagrees with the tree, `1` for a refusal, and `2` for at least one disagreement.
63
+
64
+ Nothing wires this into `bun run check` or into a hook. The false-positive rate is read off the first real run rather than assumed ahead of it, and gating a measure with an unmeasured rate is what teaches a contributor to route around the stage. `aitk gov restated` and `aitk gov test-order` are the siblings this reasoning already governs.
65
+
66
+ Both refusals are breaks rather than absences. A tree with no git history or no markdown file at all is a broken checkout, matching what `aitk markdown audit` already reads `no-git` and `no-markdown` as, since this sweep shares that corpus.
67
+
68
+ An exit code says nothing about a call made from a session, since a shell profile may wrap the binary in a function taking its status from a later command. Read the record's `findings` rather than the exit when a skill consumes this.
@@ -15,6 +15,7 @@ CLI catalog and invocation rules for agents, split by command domain. Start with
15
15
  - [Comments](comments.md): Comment density by language and kind, the two structural exclusions, and how the degradation sweep finds its vocabulary
16
16
  - [Context audit checks](context-audit-checks.md): What each non-gating check reports, the unit each checkpoint is measured in, the architecture record's length gate and claim coverage, which folders each check reaches, and what moved to the attribute tier
17
17
  - [Context audit](context-audit.md): Running the audit, its flags and folder scope, the exit codes, the citation gate, and the widened gate the seed stage runs
18
+ - [Self-stated counts](counts.md): Reading a sentence that asserts a closed catalog's size, how a match is decided, the plausibility filter that keeps a generic word from matching a subset, and why the sweep reports rather than gates
18
19
  - [Demo](demo.md): Compiling a screencast draft into a runnable plan, driving a served application to a recording and a still, the pointer the recording paints, and what each refusal reports
19
20
  - [Docs](docs.md): How aitk docs resolves the toolkit's own reference surface from an install root, and how a split domain is named
20
21
  - [Indexes](indexes.md): Flags, exit codes, and JSON shape for aitk indexes regen, plus when it auto-stages what it rewrote
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@erclx/aitk",
3
3
  "type": "module",
4
- "version": "3.18.0",
4
+ "version": "3.19.0",
5
5
  "description": "Infrastructure and quality tooling for developer workflows",
6
6
  "license": "MIT",
7
7
  "bin": {
@@ -21,7 +21,6 @@ PROJECT_ROOT="${PROJECT_ROOT:-$(cd "$SCRIPT_DIR/../.." && pwd)}"
21
21
 
22
22
  TEMPLATE="$PROJECT_ROOT/assets/hero.html.tmpl"
23
23
  OUTPUT="$PROJECT_ROOT/assets/hero.html"
24
- CLI_ENTRY="$PROJECT_ROOT/src/cli.ts"
25
24
  LISTED=10
26
25
 
27
26
  # `bun src/cli.ts` rather than `aitk`, since a globally linked binary resolves to
@@ -35,27 +34,23 @@ if [ ! -f "$TEMPLATE" ]; then
35
34
  exit 1
36
35
  fi
37
36
 
38
- # The commands have no `--json` catalog to read, so the count comes from the
39
- # registration block in the CLI entry point, which is the one place a command is
40
- # added. `--help` is hand-authored ASCII and would drift from what is registered.
41
- if [ ! -f "$CLI_ENTRY" ]; then
42
- echo "regen-hero: missing CLI entry point at $CLI_ENTRY" >&2
43
- exit 1
44
- fi
45
-
46
- COMMAND_COUNT="$(grep -c '^import { register as ' "$CLI_ENTRY" || true)"
47
- if [ "${COMMAND_COUNT:-0}" -eq 0 ]; then
48
- echo "regen-hero: no command registrations found in $CLI_ENTRY, refusing to write a zeroed hero" >&2
49
- exit 1
50
- fi
51
-
37
+ # The commands have no `--json` catalog of their own, so the count comes from
38
+ # `aitk gov counts`, which already reads the registration block in the CLI
39
+ # entry point to judge a document stating its own command count. Reading that
40
+ # figure here rather than re-deriving it with a second regex is what keeps the
41
+ # two readings of `src/cli.ts` from drifting apart unwatched, which is the
42
+ # defect `aitk gov counts` itself exists to catch.
43
+ # `gov counts` exits non-zero on an ordinary finding, unlike every `list` verb
44
+ # below, so its output is read past that under `set -e` rather than aborting
45
+ # a hero rebuild for a stale count somewhere unrelated in the tree.
46
+ COUNTS_JSON="$(catalog gov counts || true)"
52
47
  SKILLS_JSON="$(catalog claude skills list)"
53
48
  GOV_JSON="$(catalog gov list)"
54
49
  STANDARDS_JSON="$(catalog standards list)"
55
50
  SNIPPETS_JSON="$(catalog snippets list)"
56
51
  TOOLING_JSON="$(catalog tooling list)"
57
52
 
58
- for payload in "$SKILLS_JSON" "$GOV_JSON" "$STANDARDS_JSON" "$SNIPPETS_JSON" "$TOOLING_JSON"; do
53
+ for payload in "$COUNTS_JSON" "$SKILLS_JSON" "$GOV_JSON" "$STANDARDS_JSON" "$SNIPPETS_JSON" "$TOOLING_JSON"; do
59
54
  if [ -z "$payload" ]; then
60
55
  echo "regen-hero: a catalog returned nothing, refusing to write a zeroed hero" >&2
61
56
  exit 1
@@ -69,6 +64,7 @@ done
69
64
  PAYLOAD_DIR="$(mktemp -d)"
70
65
  trap 'rm -rf "$PAYLOAD_DIR"' EXIT
71
66
 
67
+ printf '%s' "$COUNTS_JSON" >"$PAYLOAD_DIR/counts.json"
72
68
  printf '%s' "$SKILLS_JSON" >"$PAYLOAD_DIR/skills.json"
73
69
  printf '%s' "$GOV_JSON" >"$PAYLOAD_DIR/gov.json"
74
70
  printf '%s' "$STANDARDS_JSON" >"$PAYLOAD_DIR/standards.json"
@@ -76,15 +72,16 @@ printf '%s' "$SNIPPETS_JSON" >"$PAYLOAD_DIR/snippets.json"
76
72
  printf '%s' "$TOOLING_JSON" >"$PAYLOAD_DIR/tooling.json"
77
73
 
78
74
  export PAYLOAD_DIR
79
- export TEMPLATE OUTPUT LISTED PROJECT_ROOT COMMAND_COUNT
75
+ export TEMPLATE OUTPUT LISTED PROJECT_ROOT
80
76
 
81
77
  bun --eval '
82
78
  const { readFileSync } = require("node:fs")
83
79
 
84
- const { PAYLOAD_DIR, TEMPLATE, OUTPUT, LISTED, PROJECT_ROOT, COMMAND_COUNT } = process.env
80
+ const { PAYLOAD_DIR, TEMPLATE, OUTPUT, LISTED, PROJECT_ROOT } = process.env
85
81
 
86
82
  const payload = (name) => readFileSync(PAYLOAD_DIR + "/" + name + ".json", "utf8")
87
83
 
84
+ const COUNTS_JSON = payload("counts")
88
85
  const SKILLS_JSON = payload("skills")
89
86
  const GOV_JSON = payload("gov")
90
87
  const STANDARDS_JSON = payload("standards")
@@ -108,6 +105,11 @@ const snippets = new Set(
108
105
  JSON.parse(SNIPPETS_JSON).categories.flatMap((category) => category.entries),
109
106
  )
110
107
  const toolingStacks = JSON.parse(TOOLING_JSON).stacks
108
+ const commandCount = JSON.parse(COUNTS_JSON).catalogs.commands
109
+ if (!commandCount) {
110
+ console.error("regen-hero: gov counts read no commands catalog, refusing to write a zeroed hero")
111
+ process.exit(1)
112
+ }
111
113
 
112
114
  const escape = (value) =>
113
115
  value.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;")
@@ -191,7 +193,7 @@ const values = {
191
193
  SNIPPET_COUNT: String(snippets.size),
192
194
  GOV_STACK_COUNT: String(gov.stacks.length),
193
195
  TOOLING_STACK_COUNT: String(toolingStacks.length),
194
- COMMAND_COUNT: String(Number(COMMAND_COUNT)),
196
+ COMMAND_COUNT: String(commandCount),
195
197
  SKILL_ENTRIES: markup(featured(skills, FEATURED_SKILLS)),
196
198
  RULE_ENTRIES: entries(deliveredRules),
197
199
  STANDARD_ENTRIES: entries(standards),
@@ -669,6 +669,18 @@ export const AUDITS: readonly AuditSpec[] = [
669
669
  ] satisfies RestatedRefusal[],
670
670
  counts: restatedCounts,
671
671
  },
672
+ {
673
+ id: 'counts',
674
+ label: 'Self-stated catalog counts',
675
+ argv: ['gov', 'counts', '--json'],
676
+ // Reports rather than gates, on the split this file already draws. The
677
+ // false-positive rate is read off the first real run rather than assumed
678
+ // ahead of it, and gating a measure with an unmeasured rate is what teaches
679
+ // a contributor to route around the stage.
680
+ gatingExits: [],
681
+ corpus: 'tracked',
682
+ counts: findingsOnly,
683
+ },
672
684
  {
673
685
  id: 'census',
674
686
  label: 'Codebase census',
@@ -2,6 +2,12 @@ import { existsSync } from 'node:fs'
2
2
  import { mkdir, writeFile } from 'node:fs/promises'
3
3
  import { basename, dirname, join, resolve } from 'node:path'
4
4
  import type { Command } from 'commander'
5
+ import {
6
+ type CountFinding,
7
+ type CountsRefusal,
8
+ type CountsReport,
9
+ scanCounts,
10
+ } from '@/counts/scan'
5
11
  import { PROJECT_ROOT } from '@/project-root'
6
12
  import { createGovAdapter } from '@/gov/adapter'
7
13
  import { regenConsumedRules } from '@/gov/consumed'
@@ -83,12 +89,23 @@ interface RestatedOptions {
83
89
  readonly json?: boolean
84
90
  }
85
91
 
92
+ interface CountsOptions {
93
+ readonly root?: string
94
+ readonly json?: boolean
95
+ }
96
+
86
97
  /** What a reader does about each way the sweep produced no reading. */
87
98
  const RESTATED_REFUSALS: Record<RestatedRefusal, string> = {
88
99
  'no-instructions': `No ${INSTRUCTIONS_REL} here, or it carries no bullet, so there is no instruction corpus to sweep.`,
89
100
  'no-surfaces': `Neither ${SEED_REL} nor ${SHIPPED_SKILLS_REL}/ is here, so no second surface exists to match against.`,
90
101
  }
91
102
 
103
+ const COUNTS_REFUSALS: Record<CountsRefusal, string> = {
104
+ 'no-git':
105
+ 'This tree is not a git checkout, so there is no tracked markdown corpus to read.',
106
+ 'no-markdown': 'No markdown file in this tree, so there is nothing to scan.',
107
+ }
108
+
92
109
  export function register(program: Command): void {
93
110
  const gov = program
94
111
  .command('gov')
@@ -213,6 +230,56 @@ export function register(program: Command): void {
213
230
  process.exitCode = runTestOrder(opts)
214
231
  })
215
232
 
233
+ gov
234
+ .command('counts')
235
+ .description(
236
+ 'Report a self-stated catalog count that disagrees with what the tree holds',
237
+ )
238
+ .helpOption('-h, --help', 'Show this help message')
239
+ .option('--root <path>', 'Tree to read, defaulting to the cwd')
240
+ .option('--json', 'Add a machine-readable record on stdout')
241
+ .addHelpText(
242
+ 'after',
243
+ [
244
+ '',
245
+ 'Reads every tracked markdown file for a sentence stating how many',
246
+ 'members a closed catalog holds, then compares the stated figure',
247
+ 'against what the tree actually counts.',
248
+ '',
249
+ 'Catalogs read: skills, governance rules, standards, snippets, CLI',
250
+ 'commands, and audit ids. Closed rather than derived, so widening the',
251
+ 'set is a deliberate change rather than a side effect of a new list',
252
+ 'command shipping elsewhere.',
253
+ '',
254
+ 'A sentence carrying a calendar date or a backticked commit reference',
255
+ 'is read past, since that is how this corpus already marks a figure as',
256
+ 'a historical record rather than a live claim.',
257
+ '',
258
+ 'What it does not measure:',
259
+ ' a delta phrased as a transition (from fourteen to fifteen) and a',
260
+ ' fraction (thirteen of sixteen) are both catalog-size claims this',
261
+ ' corpus carries, and neither matches the number-then-noun shape read here',
262
+ '',
263
+ 'Exit codes:',
264
+ ' 0 no stated figure disagrees with the tree',
265
+ ' 1 refused, with the reason on stderr or in the JSON record',
266
+ ' 2 at least one stated figure disagrees with the tree',
267
+ '',
268
+ 'It reports and never gates. The false-positive rate is read off the',
269
+ 'first run rather than assumed, and a push failing on a stated figure',
270
+ 'that reads correctly to a person would teach a contributor to route',
271
+ 'around the stage.',
272
+ '',
273
+ 'Examples:',
274
+ ' aitk gov counts',
275
+ ' aitk gov counts --json',
276
+ '',
277
+ ].join('\n'),
278
+ )
279
+ .action(async (opts: CountsOptions) => {
280
+ process.exitCode = await runCounts(opts)
281
+ })
282
+
216
283
  gov
217
284
  .command('superseded')
218
285
  .description(
@@ -575,6 +642,73 @@ function reportTestOrder(
575
642
  outro()
576
643
  }
577
644
 
645
+ /**
646
+ * Reports and never gates, matching the two sweeps above. The false-positive
647
+ * rate here is unmeasured before a first run against a real corpus, and
648
+ * gating a measure with an unknown false-positive rate is what teaches a
649
+ * contributor to route around the stage.
650
+ */
651
+ async function runCounts(opts: CountsOptions): Promise<number> {
652
+ const root = resolve(opts.root ?? process.cwd())
653
+ const report = await scanCounts(root)
654
+ const emitJson = opts.json ?? false
655
+
656
+ if (report.kind === 'unreadable') {
657
+ intro('aitk gov counts')
658
+ logStep('Refused')
659
+ logWarn(COUNTS_REFUSALS[report.reason])
660
+ outro()
661
+
662
+ if (emitJson) {
663
+ process.stdout.write(
664
+ `${JSON.stringify({
665
+ root,
666
+ reason: report.reason,
667
+ message: COUNTS_REFUSALS[report.reason],
668
+ })}\n`,
669
+ )
670
+ }
671
+
672
+ return 1
673
+ }
674
+
675
+ reportCounts(report, root)
676
+
677
+ if (emitJson) {
678
+ process.stdout.write(`${JSON.stringify({ root, ...report })}\n`)
679
+ }
680
+
681
+ return report.findings.length > 0 ? 2 : 0
682
+ }
683
+
684
+ function describeFinding(finding: CountFinding): string {
685
+ return `${finding.file}:${finding.line} states ${finding.stated} but the tree holds ${finding.actual} ${finding.catalog}`
686
+ }
687
+
688
+ function reportCounts(
689
+ report: Extract<CountsReport, { kind: 'measured' }>,
690
+ root: string,
691
+ ): void {
692
+ intro('aitk gov counts')
693
+
694
+ logStep('Corpus')
695
+ logInfo(`${report.filesScanned} tracked markdown file(s) in ${root}`)
696
+ logInfo(
697
+ Object.entries(report.catalogs)
698
+ .map(([id, count]) => `${id}: ${count ?? 'n/a'}`)
699
+ .join(', '),
700
+ )
701
+
702
+ logStep(report.findings.length === 0 ? 'Clean' : 'Findings')
703
+ if (report.findings.length === 0) {
704
+ logInfo('no self-stated catalog count disagrees with the tree')
705
+ } else {
706
+ for (const finding of report.findings) logWarn(describeFinding(finding))
707
+ }
708
+
709
+ outro()
710
+ }
711
+
578
712
  /**
579
713
  * Both selectors absent means both sections, which is the bash default. Naming
580
714
  * both is the same as naming neither rather than an error, since the two flags
@@ -0,0 +1,96 @@
1
+ import { existsSync, readFileSync } from 'node:fs'
2
+ import { join } from 'node:path'
3
+ import { AUDITS } from '@/audits/catalog'
4
+ import { listSkills } from '@/claude/skills-list'
5
+ import { buildGovCatalog } from '@/gov/list'
6
+ import { buildSnippetsCatalog } from '@/snippets/list'
7
+ import { listStandards } from '@/standards/read'
8
+
9
+ export interface Catalog {
10
+ readonly id: string
11
+ /** The noun form a prose sentence carries, singular first. */
12
+ readonly nouns: readonly [string, string]
13
+ /**
14
+ * `undefined` means this tree carries nothing this catalog reads, which is
15
+ * ordinary for every catalog but `commands`: a target installing this CLI
16
+ * has no `src/cli.ts` of its own, and reading that as zero would report a
17
+ * finding against every project stating its command count as anything but
18
+ * zero, forever.
19
+ */
20
+ readonly count: (root: string) => number | undefined
21
+ }
22
+
23
+ /** One line per registered top-level command. */
24
+ const REGISTER_IMPORT = /^import \{ register as /gm
25
+
26
+ /**
27
+ * Every top-level `aitk` command, read off the CLI entry point rather than a
28
+ * catalog verb, because a command carries no `--json` listing of its own
29
+ * siblings. `regen-hero.sh` reads its own `COMMAND_COUNT` through `gov counts`
30
+ * rather than a second regex on `src/cli.ts`, so this is the one place that
31
+ * pattern is written.
32
+ *
33
+ * This is the one catalog with no meaning outside this repository, the way
34
+ * `regen-hero.sh` documents itself as clone-only for the same reason.
35
+ */
36
+ function countCommands(root: string): number | undefined {
37
+ const cli = join(root, 'src', 'cli.ts')
38
+ if (!existsSync(cli)) return undefined
39
+ return [...readFileSync(cli, 'utf8').matchAll(REGISTER_IMPORT)].length
40
+ }
41
+
42
+ /**
43
+ * The distinct entries across every category, matching what `regen-hero.sh`
44
+ * counts. A snippet reachable from two categories is one snippet, not two.
45
+ */
46
+ function countSnippets(root: string): number {
47
+ const catalog = buildSnippetsCatalog(root)
48
+ return new Set(catalog.categories.flatMap((category) => category.entries))
49
+ .size
50
+ }
51
+
52
+ /**
53
+ * The closed set of catalogs this sweep counts a document against.
54
+ *
55
+ * Closed rather than derived from every list command this CLI ships, on the
56
+ * design the plan settled on: a wider set widens the false-positive class
57
+ * before the first run has measured how large that class already is. Each
58
+ * reader here walks the filesystem directly rather than shelling out to its
59
+ * own `aitk` verb, since this module already runs inside the process that
60
+ * would spawn it.
61
+ */
62
+ export const CATALOGS: readonly Catalog[] = [
63
+ {
64
+ id: 'skills',
65
+ nouns: ['skill', 'skills'],
66
+ count: (root) => listSkills(root).length,
67
+ },
68
+ {
69
+ id: 'rules',
70
+ nouns: ['rule', 'rules'],
71
+ count: (root) => buildGovCatalog(root).rules.length,
72
+ },
73
+ {
74
+ id: 'standards',
75
+ nouns: ['standard', 'standards'],
76
+ count: (root) => listStandards(root).length,
77
+ },
78
+ {
79
+ id: 'snippets',
80
+ nouns: ['snippet', 'snippets'],
81
+ count: countSnippets,
82
+ },
83
+ {
84
+ id: 'commands',
85
+ nouns: ['command', 'commands'],
86
+ count: countCommands,
87
+ },
88
+ {
89
+ id: 'audits',
90
+ nouns: ['audit', 'audits'],
91
+ // Reads the registered set living in this same process, so the count this
92
+ // row reports is the tree's own audit surface rather than a stale copy of
93
+ // it, this entry included.
94
+ count: () => AUDITS.length,
95
+ },
96
+ ]
@@ -0,0 +1,79 @@
1
+ const ONES: Record<string, number> = {
2
+ zero: 0,
3
+ one: 1,
4
+ two: 2,
5
+ three: 3,
6
+ four: 4,
7
+ five: 5,
8
+ six: 6,
9
+ seven: 7,
10
+ eight: 8,
11
+ nine: 9,
12
+ }
13
+
14
+ const TEENS: Record<string, number> = {
15
+ ten: 10,
16
+ eleven: 11,
17
+ twelve: 12,
18
+ thirteen: 13,
19
+ fourteen: 14,
20
+ fifteen: 15,
21
+ sixteen: 16,
22
+ seventeen: 17,
23
+ eighteen: 18,
24
+ nineteen: 19,
25
+ }
26
+
27
+ const TENS: Record<string, number> = {
28
+ twenty: 20,
29
+ thirty: 30,
30
+ forty: 40,
31
+ fifty: 50,
32
+ sixty: 60,
33
+ seventy: 70,
34
+ eighty: 80,
35
+ ninety: 90,
36
+ }
37
+
38
+ /**
39
+ * Every spelled-out cardinal from zero through ninety-nine, keyed lowercase.
40
+ *
41
+ * Every catalog this reads is under a hundred members, so the compound tens
42
+ * form (`sixty-one`) is the only multi-word shape worth building: the corpus
43
+ * this was written against spells a catalog size in words as often as in
44
+ * digits, and a matcher reading digits alone would miss half its own
45
+ * motivating cases.
46
+ */
47
+ export const WORD_NUMBERS: ReadonlyMap<string, number> = (() => {
48
+ const words = new Map<string, number>()
49
+ for (const [word, value] of Object.entries(ONES)) words.set(word, value)
50
+ for (const [word, value] of Object.entries(TEENS)) words.set(word, value)
51
+ for (const [tensWord, tensValue] of Object.entries(TENS)) {
52
+ words.set(tensWord, tensValue)
53
+ for (const [onesWord, onesValue] of Object.entries(ONES)) {
54
+ if (onesValue === 0) continue
55
+ words.set(`${tensWord}-${onesWord}`, tensValue + onesValue)
56
+ }
57
+ }
58
+ return words
59
+ })()
60
+
61
+ /**
62
+ * Longest key first, so the alternation matches `sixty-one` whole rather than
63
+ * stopping at `sixty` and leaving `-one` unconsumed.
64
+ */
65
+ const WORD_ALTERNATION = [...WORD_NUMBERS.keys()]
66
+ .sort((left, right) => right.length - left.length)
67
+ .join('|')
68
+
69
+ /** A bare digit run, capped at three since every catalog here is under 1,000. */
70
+ const DIGIT_PATTERN = '\\d{1,3}'
71
+
72
+ /** Either spelling a catalog size takes in this corpus, for a caller building its own regex around it. */
73
+ export const NUMBER_PATTERN = `(?:${DIGIT_PATTERN}|${WORD_ALTERNATION})`
74
+
75
+ /** The value a matched token names, or `undefined` for a word this map does not carry. */
76
+ export function parseNumber(token: string): number | undefined {
77
+ if (/^\d+$/.test(token)) return Number(token)
78
+ return WORD_NUMBERS.get(token.toLowerCase())
79
+ }
@@ -0,0 +1,258 @@
1
+ import { readFileSync } from 'node:fs'
2
+ import { join } from 'node:path'
3
+ import { type Catalog, CATALOGS } from '@/counts/catalogs'
4
+ import { NUMBER_PATTERN, parseNumber } from '@/counts/numbers'
5
+ import { resolveMarkdown } from '@/markdown/files'
6
+
7
+ export interface CountFinding {
8
+ readonly file: string
9
+ readonly line: number
10
+ readonly catalog: string
11
+ readonly stated: number
12
+ readonly actual: number
13
+ readonly sentence: string
14
+ }
15
+
16
+ /**
17
+ * Narrower than `MarkdownAuditRefusal`. This sweep takes no path arguments,
18
+ * so `resolveMarkdown` can never return its `no-match` reason here.
19
+ */
20
+ export type CountsRefusal = 'no-git' | 'no-markdown'
21
+
22
+ export type CountsReport =
23
+ | {
24
+ readonly kind: 'measured'
25
+ readonly filesScanned: number
26
+ /**
27
+ * The true count this run read for every catalog, keyed by id.
28
+ * `undefined` for a catalog this tree carries nothing to count, which
29
+ * `commands` is on every project but this repository.
30
+ */
31
+ readonly catalogs: Readonly<Record<string, number | undefined>>
32
+ readonly findings: readonly CountFinding[]
33
+ }
34
+ | { readonly kind: 'unreadable'; readonly reason: CountsRefusal }
35
+
36
+ /**
37
+ * A calendar date, which reads a sentence as a historical record rather than
38
+ * a live claim about the tree. `.claude/ARCHITECTURE.md` and the context
39
+ * entries carry a figure this way deliberately, and every one of them stays
40
+ * correct forever, so a sentence carrying one is read past rather than
41
+ * matched.
42
+ */
43
+ const DATE = /\b\d{4}-\d{2}-\d{2}\b/
44
+
45
+ /** A commit-length hex token in a backtick span, the other marker of a dated record. */
46
+ const COMMIT = /`[0-9a-f]{7,12}`/
47
+
48
+ function isDated(sentence: string): boolean {
49
+ return DATE.test(sentence) || COMMIT.test(sentence)
50
+ }
51
+
52
+ /**
53
+ * Splits a rendered line into the sentences a reader sees.
54
+ *
55
+ * A markdown paragraph in this corpus is written as one long line rather than
56
+ * hard-wrapped, so a line is routinely several sentences deep, and a date
57
+ * naming one of them must not excuse a live claim two sentences away in the
58
+ * same line.
59
+ */
60
+ function sentencesOf(line: string): string[] {
61
+ return line
62
+ .split(/(?<=[.!?])\s+/)
63
+ .map((sentence) => sentence.trim())
64
+ .filter((sentence) => sentence !== '')
65
+ }
66
+
67
+ /**
68
+ * A verb asserting the whole of something, immediately ahead of the number.
69
+ *
70
+ * The first design matched a bare number next to a catalog noun anywhere in
71
+ * the tracked corpus, and running it here returned 290 findings against a
72
+ * repository whose Files-to-touch names one live instance. Reading the
73
+ * corpus behind that run showed why: `18 rules citing a standard`, `21 skill
74
+ * bodies`, and `eight internal skills` all pair a number with a catalog noun
75
+ * while naming a subset, an example, or a different catalog than the one
76
+ * matched, and that shape dominates ordinary prose. Every instance this
77
+ * module was written against reads the number as the direct object of a verb
78
+ * asserting the catalog's own total, `the full entry loads 59 skills` being
79
+ * the live one, and gating the match on that verb is what a full re-read of
80
+ * the 290 showed removing all of them while keeping that one.
81
+ */
82
+ const ASSERTION_VERBS = [
83
+ 'loads',
84
+ 'ships',
85
+ 'carries',
86
+ 'holds',
87
+ 'counts',
88
+ 'totals',
89
+ 'documents',
90
+ 'declares',
91
+ 'installs',
92
+ 'lists',
93
+ 'contains',
94
+ 'comprises',
95
+ ]
96
+
97
+ /**
98
+ * An article immediately ahead of the number, admitting the shape a bare verb
99
+ * gate cannot reach: `the twelve audits read gitignored folders` asserts the
100
+ * catalog's own total with the verb sitting after the noun rather than ahead
101
+ * of the number.
102
+ *
103
+ * Found on this module's own first review, against a live instance the verb
104
+ * gate alone reported clean: `.claude/context/development/gates.md` stated
105
+ * `the twelve audits` while the tree held 20, sitting inside the plausibility
106
+ * bound this design already carries, so nothing but the missing shape kept it
107
+ * from reporting.
108
+ *
109
+ * The article is looser than the verb list, and re-running against the tree
110
+ * once it widened produced the class this trades for the miss above: `the 21
111
+ * flat standards` read a real subset (a standard outside the retired
112
+ * `standards/bundled/` fan-out) as a claim about the whole catalog, since
113
+ * `flat` fills the same optional-word slot `sixty-one shipped skills` needs to
114
+ * match at all. No syntactic rule tells a qualifier that narrows a catalog
115
+ * from one that only restates it, and the sentence was true when written, so
116
+ * the fix was not a narrower matcher: the clause was dated as a past state,
117
+ * the way `.claude/ARCHITECTURE.md` dates its own figures, which is the class
118
+ * the date exclusion below already reads past.
119
+ */
120
+ const ARTICLES = ['the', 'a', 'an']
121
+
122
+ /**
123
+ * Matches a catalog's stated size: an assertion verb or an article, the
124
+ * number, an optional single qualifying word, then the noun in either number.
125
+ *
126
+ * The optional word between the number and the noun is what reaches a form
127
+ * like `installs 11 shipped snippets` without also reaching past an
128
+ * intervening clause, since a wider gap would start pairing a number in one
129
+ * clause with a noun in the next. Requiring the noun's own plural form to
130
+ * double as its match narrows the corpus this reads without hand-listing
131
+ * every irregular plural, since none of these six is irregular.
132
+ *
133
+ * What this drops along with the false positives: `denominator of sixty-one
134
+ * shipped skills` and `exposed all 59` both state a real catalog total and
135
+ * neither puts a verb or an article directly ahead of the number, so both
136
+ * read past. The gap between the trigger and the number is a second axis
137
+ * this could widen once this design's own false-positive rate is measured
138
+ * over more than the two runs behind it so far.
139
+ */
140
+ function buildMatcher(catalog: Catalog): RegExp {
141
+ const [singular, plural] = catalog.nouns
142
+ const triggers = [...ASSERTION_VERBS, ...ARTICLES].join('|')
143
+ return new RegExp(
144
+ `\\b(?:${triggers})\\s+(${NUMBER_PATTERN})(?:\\s+[a-z]+)?\\s+(?:${singular}|${plural})\\b`,
145
+ 'gi',
146
+ )
147
+ }
148
+
149
+ const MATCHERS: readonly { catalog: Catalog; regex: RegExp }[] = CATALOGS.map(
150
+ (catalog) => ({ catalog, regex: buildMatcher(catalog) }),
151
+ )
152
+
153
+ /**
154
+ * A stated figure within a factor of two of the true count, in either
155
+ * direction.
156
+ *
157
+ * The verb gate alone still passed a run against this repository at four
158
+ * false positives to one real finding: `carries two rules about a
159
+ * standard's own lifecycle`, `holds one rule or one fact`, and `documents
160
+ * two similar commands` each pair an assertion verb with a catalog noun
161
+ * used for something this tree's `rules` or `commands` catalog does not
162
+ * mean. What tells those apart from the live finding is magnitude. A
163
+ * catalog this sweep tracks drifts by a few members between the day a
164
+ * sentence was written and the day it is read, so a genuine staleness claim
165
+ * sits near the true count, and `2` beside a true count of `59` is not a
166
+ * catalog that shrank, it is a different `rules` entirely.
167
+ *
168
+ * Tuned against the run this module was written against rather than
169
+ * reasoned to from first principles, the way `restated.ts` tunes its own
170
+ * document-frequency ceiling. Every catalog here sits in the tens, and the
171
+ * bound is a property of that scale rather than a universal one.
172
+ */
173
+ const PLAUSIBLE_RATIO = 0.5
174
+
175
+ function isPlausibleClaim(stated: number, actual: number): boolean {
176
+ if (actual === 0) return stated === 0
177
+ return (
178
+ stated / actual >= PLAUSIBLE_RATIO && actual / stated >= PLAUSIBLE_RATIO
179
+ )
180
+ }
181
+
182
+ /**
183
+ * Scans every tracked markdown file for a sentence stating how many members
184
+ * one of the closed catalogs holds, and compares the stated figure against
185
+ * what the tree actually counts.
186
+ *
187
+ * What it does not measure: a delta phrased as a transition (`from fourteen
188
+ * to fifteen`), a fraction (`thirteen of sixteen`), and a total reached
189
+ * through an indirect noun (`denominator of sixty-one shipped skills`) are
190
+ * all catalog-size claims this corpus carries, and none matches the
191
+ * assertion-verb shape this reads. Each is a known gap rather than an
192
+ * oversight, left for a wider pass once this one's false-positive rate is
193
+ * measured.
194
+ */
195
+ export async function scanCounts(root: string): Promise<CountsReport> {
196
+ const scope = await resolveMarkdown(root, [])
197
+ if (scope.kind === 'unavailable') {
198
+ return { kind: 'unreadable', reason: 'no-git' }
199
+ }
200
+ if (scope.files.length === 0) {
201
+ return { kind: 'unreadable', reason: 'no-markdown' }
202
+ }
203
+
204
+ const catalogs: Record<string, number | undefined> = {}
205
+ for (const catalog of CATALOGS) catalogs[catalog.id] = catalog.count(root)
206
+
207
+ const findings: CountFinding[] = []
208
+
209
+ for (const file of scope.files) {
210
+ const text = readFileSync(join(root, file), 'utf8')
211
+ let fenced = false
212
+
213
+ text.split('\n').forEach((line, index) => {
214
+ if (line.trimStart().startsWith('```')) {
215
+ fenced = !fenced
216
+ return
217
+ }
218
+ if (fenced) return
219
+
220
+ for (const sentence of sentencesOf(line)) {
221
+ if (isDated(sentence)) continue
222
+
223
+ for (const { catalog, regex } of MATCHERS) {
224
+ regex.lastIndex = 0
225
+ const match = regex.exec(sentence)
226
+ if (match === null) continue
227
+
228
+ const stated = parseNumber(match[1])
229
+ if (stated === undefined) continue
230
+
231
+ const actual = catalogs[catalog.id]
232
+ // No true count on this tree, which is the ordinary state of the
233
+ // `commands` catalog outside this repository. A stated figure has
234
+ // nothing to be judged stale against.
235
+ if (actual === undefined) continue
236
+ if (stated === actual) continue
237
+ if (!isPlausibleClaim(stated, actual)) continue
238
+
239
+ findings.push({
240
+ file,
241
+ line: index + 1,
242
+ catalog: catalog.id,
243
+ stated,
244
+ actual,
245
+ sentence,
246
+ })
247
+ }
248
+ }
249
+ })
250
+ }
251
+
252
+ return {
253
+ kind: 'measured',
254
+ filesScanned: scope.files.length,
255
+ catalogs,
256
+ findings,
257
+ }
258
+ }