@erclx/aitk 1.4.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/claude/.claude-plugin/plugin.json +1 -1
- package/docs/agents/context-audit-checks.md +18 -2
- package/docs/agents/context-audit.md +3 -3
- package/docs/agents/index.md +1 -1
- package/package.json +1 -1
- package/src/audits/catalog.ts +62 -1
- package/src/commands/context.ts +124 -6
- package/src/context/architecture.ts +364 -0
- package/src/context/gate.ts +19 -5
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Context audit checks
|
|
3
|
-
description: What each non-gating check reports, the unit each checkpoint is measured in, which folders each check reaches, and what moved to the attribute tier
|
|
3
|
+
description: What each non-gating check reports, the unit each checkpoint is measured in, the architecture record's length gate and claim coverage, which folders each check reaches, and what moved to the attribute tier
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Context audit checks
|
|
7
7
|
|
|
8
|
-
What each finding from `aitk context audit` means. The command surface, its flags, and the
|
|
8
|
+
What each finding from `aitk context audit` means. The command surface, its flags, and the two gating checks are in `context-audit.md`.
|
|
9
9
|
|
|
10
10
|
## Required sections
|
|
11
11
|
|
|
@@ -87,6 +87,22 @@ A rejected alternative is a back-reference in the past tense by construction, an
|
|
|
87
87
|
|
|
88
88
|
The JSON record carries the findings per entry as `entries[].narration` and the sets as `checkpoints.narration`, which is absent under `--citations-only` where the run never loads them and null where no rule publishes both.
|
|
89
89
|
|
|
90
|
+
## The architecture record
|
|
91
|
+
|
|
92
|
+
Two findings read `.claude/ARCHITECTURE.md` rather than a folder, and only the first is a fact.
|
|
93
|
+
|
|
94
|
+
The length check compares the record against the ceiling it derives for itself, and only a record that states its own allowances has one. No standard sets a length rule for this document, so the numbers belong to whichever record declares them. The check reads a frame allowance and an allowance per decision out of the record's own prose and puts the ceiling at the frame plus the allowance times the decision count. The JSON record carries what it read as `architecture.allowances` and the reading as `architecture.lines` against `architecture.ceiling`.
|
|
95
|
+
|
|
96
|
+
A record stating no such rule is measured and reported and never gated, which is the ordinary shape of a project that wrote an architecture record and no convention about its length. Holding a pair of numbers in the toolkit and gating every project against them would fail a record on a ceiling that project never agreed to, and the framed line would credit the rule to a file that does not state it.
|
|
97
|
+
|
|
98
|
+
What the derivation costs where it does apply is that the ceiling rises when a decision is added and falls for nothing, so the check passes exactly when the file grew. It gates anyway, because a record that states a limit for itself and makes it computable from a count has turned it into a fact, which is what separates it from every judgment below. A rewrite of the declaring sentence past the clauses the check reads falls back to reporting rather than to a stale ceiling, so the failure shows up in the run's own output.
|
|
99
|
+
|
|
100
|
+
The coverage report classifies each decision as carrying a countable claim, a structural invariant, or neither, then reports each testable entry against whether it names a check that exists. A countable claim carries a figure a run could recompute, and an invariant quantifies over a named tree closely enough that a walk could falsify it. A check is a `scripts/**.sh` path the entry spells that is on disk, or an `aitk` invocation matching a registered audit, so coverage reads the entry rather than the tree and a claim some check happens to cover without the entry saying so reads as unchecked.
|
|
101
|
+
|
|
102
|
+
Three limits are stated on every run rather than hidden. The countable signal reads digits alone, so a measured claim written in words reads as uncounted. Entries are counted by heading, and one heading holding several decisions counts once. Nothing is stored, so an entry rewritten tomorrow is classified afresh the next time the verb runs and no verdict goes stale.
|
|
103
|
+
|
|
104
|
+
The report gates nothing. Deciding whether a sentence states a claim is a judgment no parser settles, so the output names candidates for a reader. This answers a different question from the verification anchors `.claude/standards/architecture.md` describes, which record that one cited number was re-read. That mechanism says whether a marked figure held, and this one says how much of the record could be checked at all.
|
|
105
|
+
|
|
90
106
|
## Which folders each check reaches
|
|
91
107
|
|
|
92
108
|
The provenance, required-section, and narration checks cover `.claude/context/` alone, the reference-form check covers the split folders inside it, and length and the table finding reach every audited folder.
|
|
@@ -5,7 +5,7 @@ description: Running the audit, its flags and folder scope, the exit codes, the
|
|
|
5
5
|
|
|
6
6
|
# Context audit
|
|
7
7
|
|
|
8
|
-
`aitk context audit [path]` reports the structural state of the folders following the index-plus-entry contract, meaning a generated `index.md` beside entries carrying frontmatter. It reads and reports. Fixing what it finds is separate work. What each finding means is in `context-audit-checks.md`.
|
|
8
|
+
`aitk context audit [path]` reports the structural state of the folders following the index-plus-entry contract, meaning a generated `index.md` beside entries carrying frontmatter, and it measures `.claude/ARCHITECTURE.md` beside them. It reads and reports. Fixing what it finds is separate work. What each finding means is in `context-audit-checks.md`.
|
|
9
9
|
|
|
10
10
|
Findings stated over every markdown file rather than over a context entry are measured by `aitk markdown audit`, described in `markdown-audit.md`. That command resolves no folder, so it reaches trees this one refuses.
|
|
11
11
|
|
|
@@ -37,7 +37,7 @@ A run where no requested name resolves refuses, whichever list it read. Naming t
|
|
|
37
37
|
|
|
38
38
|
## Exit codes
|
|
39
39
|
|
|
40
|
-
Exit codes are `0` for a clean run, `1` for a refusal, and `2` for a gating finding. An unresolved citation gates under every mode.
|
|
40
|
+
Exit codes are `0` for a clean run, `1` for a refusal, and `2` for a gating finding. An unresolved citation gates under every mode. An architecture record that states its own line allowances gates when it is past the ceiling those derive, on any run that measures it, which is every mode except `--citations-only`, and a record stating none is reported and never gated. Entry length, reference form, table, provenance, narration, and the record's claim classification print and return `0` under every mode, because each is a judgment and failing a push on one would make the check something to route around. Narration is the weakest of the five, since whether two bullets share a subject is a call the measure approximates from structure alone, and one of the shapes it matches is the rejected alternative the standard asks an entry to keep.
|
|
41
41
|
|
|
42
42
|
Required-section and index findings sit between the two. Both are answerable from the file rather than weighed, so `--gate` promotes them to failing codes while a bare run leaves them advisory. The toolkit runs the bare form against itself and the widened form against the seed tree, described below.
|
|
43
43
|
|
|
@@ -47,7 +47,7 @@ Required-section and index findings sit between the two. Both are answerable fro
|
|
|
47
47
|
|
|
48
48
|
The widened gate is correct here and wrong at the project root. A seed is authored once and read by every target, while a context entry in a live project is edited under time pressure by the people who own it. A missing section in the first is a defect shipping outward, and in the second it is a threshold worth reporting and not worth blocking a push over.
|
|
49
49
|
|
|
50
|
-
Coverage follows the index-plus-entry contract, so it reaches seeded entries and the indexes beside them.
|
|
50
|
+
Coverage follows the index-plus-entry contract, so it reaches seeded entries and the indexes beside them. `DESIGN.md` and `REQUIREMENTS.md` sitting directly under `.claude/` belong to no audited folder and stay outside it, while `ARCHITECTURE.md` is measured on its own path. The seed record states no line allowance, so it is reported rather than gated, which is what a seed template showing the shape of a record should be.
|
|
51
51
|
|
|
52
52
|
The stage prints the entries it measured per root and warns on a root that measured none. A root can resolve an audited folder and hold no entry in it, which `tooling/claude/seeds` does today, so a single pass line over the set would report coverage of a tree nothing opened.
|
|
53
53
|
|
package/docs/agents/index.md
CHANGED
|
@@ -12,7 +12,7 @@ CLI catalog and invocation rules for agents, split by command domain. Start with
|
|
|
12
12
|
- [Capture](capture.md): Rendering committed HTML sources to PNG, what the command asserts about fonts, and why it is toolkit-only
|
|
13
13
|
- [Command catalog](commands.md): Every project-level command and every domain subcommand, plus the shape each domain exposes
|
|
14
14
|
- [Comments](comments.md): Comment density by language and kind, the two structural exclusions, and how the degradation sweep finds its vocabulary
|
|
15
|
-
- [Context audit checks](context-audit-checks.md): What each non-gating check reports, the unit each checkpoint is measured in, which folders each check reaches, and what moved to the attribute tier
|
|
15
|
+
- [Context audit checks](context-audit-checks.md): What each non-gating check reports, the unit each checkpoint is measured in, the architecture record's length gate and claim coverage, which folders each check reaches, and what moved to the attribute tier
|
|
16
16
|
- [Context audit](context-audit.md): Running the audit, its flags and folder scope, the exit codes, the citation gate, and the widened gate the seed stage runs
|
|
17
17
|
- [Docs](docs.md): How aitk docs resolves the toolkit's own reference surface from an install root, and how a split domain is named
|
|
18
18
|
- [Indexes](indexes.md): Flags, exit codes, and JSON shape for aitk indexes regen, plus when it auto-stages what it rewrote
|
package/package.json
CHANGED
package/src/audits/catalog.ts
CHANGED
|
@@ -103,6 +103,61 @@ function allOf(
|
|
|
103
103
|
return counts
|
|
104
104
|
}
|
|
105
105
|
|
|
106
|
+
/**
|
|
107
|
+
* Reads the architecture record's three measures, or nothing when the project
|
|
108
|
+
* carries no record.
|
|
109
|
+
*
|
|
110
|
+
* Three states rather than two, matching what the verb publishes. The key
|
|
111
|
+
* absent is a run that never opened the record, which the aggregate never asks
|
|
112
|
+
* for and so reads as a shape that moved. Null is a project entitled to carry
|
|
113
|
+
* no record, whose other context counts still stand, so it contributes no key
|
|
114
|
+
* rather than a zero that would read as a conforming record.
|
|
115
|
+
*/
|
|
116
|
+
function architectureCounts(
|
|
117
|
+
root: Record<string, unknown>,
|
|
118
|
+
): Record<string, number> | undefined | 'absent' {
|
|
119
|
+
if (!('architecture' in root)) return undefined
|
|
120
|
+
if (root.architecture === null) return 'absent'
|
|
121
|
+
|
|
122
|
+
const record = asObject(root.architecture)
|
|
123
|
+
const decisions = record?.decisions
|
|
124
|
+
if (
|
|
125
|
+
record === undefined ||
|
|
126
|
+
!Array.isArray(decisions) ||
|
|
127
|
+
typeof record.lines !== 'number'
|
|
128
|
+
) {
|
|
129
|
+
return undefined
|
|
130
|
+
}
|
|
131
|
+
|
|
132
|
+
// A record stating no length rule has no ceiling to be past, and reporting
|
|
133
|
+
// zero there would read as one measured and found conforming.
|
|
134
|
+
const ceiling =
|
|
135
|
+
typeof record.ceiling === 'number' ? record.ceiling : undefined
|
|
136
|
+
|
|
137
|
+
let unverifiable = 0
|
|
138
|
+
let unchecked = 0
|
|
139
|
+
for (const raw of decisions) {
|
|
140
|
+
const entry = asObject(raw)
|
|
141
|
+
const claim = entry?.claim
|
|
142
|
+
const checks = lengthOf(entry?.checks)
|
|
143
|
+
if (typeof claim !== 'string' || checks === undefined) return undefined
|
|
144
|
+
|
|
145
|
+
if (claim === 'neither') unverifiable += 1
|
|
146
|
+
else if (checks === 0) unchecked += 1
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
return {
|
|
150
|
+
// A boolean, counted so the aggregate reads it the way it reads every
|
|
151
|
+
// other measure. The verb gates on it separately, and the key is absent
|
|
152
|
+
// rather than zero on a record that declared no ceiling.
|
|
153
|
+
...(ceiling !== undefined && {
|
|
154
|
+
recordOverLength: record.lines > ceiling ? 1 : 0,
|
|
155
|
+
}),
|
|
156
|
+
recordUnverifiable: unverifiable,
|
|
157
|
+
recordUnchecked: unchecked,
|
|
158
|
+
}
|
|
159
|
+
}
|
|
160
|
+
|
|
106
161
|
function contextCounts(record: unknown): Record<string, number> | undefined {
|
|
107
162
|
const root = asObject(record)
|
|
108
163
|
if (root === undefined) return undefined
|
|
@@ -117,13 +172,19 @@ function contextCounts(record: unknown): Record<string, number> | undefined {
|
|
|
117
172
|
bareReferences += bare
|
|
118
173
|
}
|
|
119
174
|
|
|
120
|
-
|
|
175
|
+
const architecture = architectureCounts(root)
|
|
176
|
+
if (architecture === undefined) return undefined
|
|
177
|
+
|
|
178
|
+
const counts = allOf({
|
|
121
179
|
unresolvedCitations: lengthOf(asObject(root.citations)?.unresolved),
|
|
122
180
|
longEntries: lengthOf(root.length),
|
|
123
181
|
missingSections: lengthOf(root.missingSections),
|
|
124
182
|
indexDrift: lengthOf(root.indexDrift),
|
|
125
183
|
bareReferences,
|
|
126
184
|
})
|
|
185
|
+
|
|
186
|
+
if (counts === undefined) return undefined
|
|
187
|
+
return architecture === 'absent' ? counts : { ...counts, ...architecture }
|
|
127
188
|
}
|
|
128
189
|
|
|
129
190
|
function markdownCounts(record: unknown): Record<string, number> | undefined {
|
package/src/commands/context.ts
CHANGED
|
@@ -15,6 +15,14 @@ import {
|
|
|
15
15
|
REQUIRED_SECTIONS,
|
|
16
16
|
type SectionFinding,
|
|
17
17
|
} from '@/context/audit'
|
|
18
|
+
import {
|
|
19
|
+
type ArchitectureReport,
|
|
20
|
+
coveredCount,
|
|
21
|
+
isOverLength,
|
|
22
|
+
measureArchitecture,
|
|
23
|
+
RECORD_REL,
|
|
24
|
+
testableCount,
|
|
25
|
+
} from '@/context/architecture'
|
|
18
26
|
import { auditCitations, type CitationReport } from '@/context/citations'
|
|
19
27
|
import {
|
|
20
28
|
type AuditedFolder,
|
|
@@ -65,7 +73,7 @@ export function register(program: Command): void {
|
|
|
65
73
|
context
|
|
66
74
|
.command('audit')
|
|
67
75
|
.description(
|
|
68
|
-
'Report required sections, entry length, citations, reference form, catalog tables, provenance, superseded-decision narration, and
|
|
76
|
+
'Report required sections, entry length, citations, reference form, catalog tables, provenance, superseded-decision narration, index drift, and the architecture record against its own ceiling',
|
|
69
77
|
)
|
|
70
78
|
.argument('[path]', 'Project root, defaulting to the current directory')
|
|
71
79
|
.helpOption('-h, --help', 'Show this help message')
|
|
@@ -88,11 +96,14 @@ export function register(program: Command): void {
|
|
|
88
96
|
' 1 refused, with the reason on stderr',
|
|
89
97
|
' 2 a gating finding is present',
|
|
90
98
|
'',
|
|
91
|
-
'An unresolved citation always gates.
|
|
92
|
-
'
|
|
93
|
-
'
|
|
94
|
-
'
|
|
95
|
-
'
|
|
99
|
+
'An unresolved citation always gates. An architecture record that',
|
|
100
|
+
'states its own line allowances gates when it is past the ceiling',
|
|
101
|
+
'those derive, on any run except --citations-only, which never',
|
|
102
|
+
'measures it. A record stating no allowance is reported and never',
|
|
103
|
+
'gated. --gate widens the gate to the other two findings that are',
|
|
104
|
+
'facts rather than judgments: a missing required section and index',
|
|
105
|
+
'drift. Entry length, reference form, table, provenance, narration,',
|
|
106
|
+
'and the record claim classification are judgments under both.',
|
|
96
107
|
'',
|
|
97
108
|
'Depth and bullet weight are stated over every markdown file rather',
|
|
98
109
|
'than over a context entry, so `aitk markdown audit` measures them.',
|
|
@@ -205,6 +216,12 @@ async function runAudit(
|
|
|
205
216
|
const drift = gateOnly ? [] : await auditIndexes(folders)
|
|
206
217
|
const sections = gateOnly ? [] : missingSections(root, folders, entries)
|
|
207
218
|
const length = gateOnly ? undefined : lengthFindings(entries)
|
|
219
|
+
// Absent under `--citations-only` and null when the project carries no
|
|
220
|
+
// record, for the reason `checkpoints.narration` states about its own two
|
|
221
|
+
// absences. A run that never looked and a project with nothing to look at
|
|
222
|
+
// are different answers, and one value for both reports the second as the
|
|
223
|
+
// first.
|
|
224
|
+
const record = gateOnly ? undefined : await measureArchitecture(root)
|
|
208
225
|
|
|
209
226
|
if (gateOnly) {
|
|
210
227
|
reportGate(citations)
|
|
@@ -219,6 +236,7 @@ async function runAudit(
|
|
|
219
236
|
reportProvenance(entries, folders)
|
|
220
237
|
reportNarration(entries, folders, narration)
|
|
221
238
|
reportDrift(drift)
|
|
239
|
+
reportRecord(record)
|
|
222
240
|
outro()
|
|
223
241
|
}
|
|
224
242
|
|
|
@@ -251,6 +269,10 @@ async function runAudit(
|
|
|
251
269
|
length,
|
|
252
270
|
missingSections: sections,
|
|
253
271
|
indexDrift: drift,
|
|
272
|
+
// Null says the run opened the project and found no record, which a
|
|
273
|
+
// target that never wrote one is entitled to. Absent says the run
|
|
274
|
+
// never looked, which is `--citations-only`.
|
|
275
|
+
architecture: gateOnly ? undefined : (record ?? null),
|
|
254
276
|
checkpoints: {
|
|
255
277
|
lines: LENGTH_CHECKPOINT,
|
|
256
278
|
renderWidth: RENDER_WIDTH,
|
|
@@ -277,6 +299,7 @@ async function runAudit(
|
|
|
277
299
|
|
|
278
300
|
const gating = isGating({
|
|
279
301
|
unresolvedCitations: citations.unresolved.length,
|
|
302
|
+
recordOverLength: record !== undefined && isOverLength(record),
|
|
280
303
|
sections,
|
|
281
304
|
drift,
|
|
282
305
|
widened,
|
|
@@ -696,6 +719,101 @@ function reportNarration(
|
|
|
696
719
|
)
|
|
697
720
|
}
|
|
698
721
|
|
|
722
|
+
/** How each classification reads in the report. */
|
|
723
|
+
const CLAIM_LABEL: Record<string, string> = {
|
|
724
|
+
countable: 'countable claim',
|
|
725
|
+
invariant: 'structural invariant',
|
|
726
|
+
neither: 'reasoning only',
|
|
727
|
+
}
|
|
728
|
+
|
|
729
|
+
/**
|
|
730
|
+
* Reports the architecture record against the ceiling it states for itself and
|
|
731
|
+
* against what a machine could test in it.
|
|
732
|
+
*
|
|
733
|
+
* The length reading is a fact and gates. Everything below it names candidates
|
|
734
|
+
* a reader adjudicates, because deciding whether a sentence states a claim is a
|
|
735
|
+
* judgment no parser settles, and a stored verdict would age the way the
|
|
736
|
+
* anchors it sits beside already do. Nothing is stored: every run reclassifies,
|
|
737
|
+
* so an entry rewritten tomorrow is read as it stands then.
|
|
738
|
+
*/
|
|
739
|
+
function reportRecord(report: ArchitectureReport | undefined): void {
|
|
740
|
+
logStep('Architecture record')
|
|
741
|
+
|
|
742
|
+
if (report === undefined) {
|
|
743
|
+
logInfo(
|
|
744
|
+
`Out of scope. The project carries no ${RECORD_REL}, so there was no record to measure.`,
|
|
745
|
+
)
|
|
746
|
+
return
|
|
747
|
+
}
|
|
748
|
+
|
|
749
|
+
const decisions = report.decisions.length
|
|
750
|
+
const { allowances } = report
|
|
751
|
+
|
|
752
|
+
if (allowances === undefined) {
|
|
753
|
+
logInfo(
|
|
754
|
+
`Covers ${report.rel} alone. No standard sets a length rule for it and this record states none, so its ${plural(report.lines, 'line')} across ${plural(decisions, 'decision')} are reported and nothing is gated.`,
|
|
755
|
+
)
|
|
756
|
+
logInfo(
|
|
757
|
+
'A record declaring an allowance for its frame and one a decision is measured against the ceiling those two derive. That rule belongs to whichever record writes it, never to the toolkit.',
|
|
758
|
+
)
|
|
759
|
+
} else {
|
|
760
|
+
logInfo(
|
|
761
|
+
`Covers ${report.rel} alone, which states its own allowance of ${plural(allowances.frame, 'line')} for the frame and ${allowances.perDecision} a decision.`,
|
|
762
|
+
)
|
|
763
|
+
if (isOverLength(report)) {
|
|
764
|
+
logError(
|
|
765
|
+
`${report.lines} lines against a ceiling of ${report.ceiling} from ${plural(decisions, 'decision')}`,
|
|
766
|
+
)
|
|
767
|
+
} else {
|
|
768
|
+
logInfo(
|
|
769
|
+
`${report.lines} lines against a ceiling of ${report.ceiling} from ${plural(decisions, 'decision')}.`,
|
|
770
|
+
)
|
|
771
|
+
}
|
|
772
|
+
logInfo(
|
|
773
|
+
`The ceiling rises with the decision count, so adding a decision buys ${allowances.perDecision} lines and the check passes exactly when the file grew.`,
|
|
774
|
+
)
|
|
775
|
+
}
|
|
776
|
+
|
|
777
|
+
if (decisions === 0) {
|
|
778
|
+
logWarn('The record declares no decision, so nothing was classified.')
|
|
779
|
+
return
|
|
780
|
+
}
|
|
781
|
+
|
|
782
|
+
const testable = testableCount(report)
|
|
783
|
+
const covered = coveredCount(report)
|
|
784
|
+
|
|
785
|
+
logInfo(
|
|
786
|
+
'A countable claim carries a figure a run could recompute and an invariant quantifies over a named tree a walk could falsify. Both are candidates a reader settles, and neither gates.',
|
|
787
|
+
)
|
|
788
|
+
logInfo(
|
|
789
|
+
'A figure spelled in words reads as uncounted, since a cardinal in prose is pronominal more often than measured. Entries are counted by heading, so a heading carrying several decisions counts once and the total reads low by however many it holds.',
|
|
790
|
+
)
|
|
791
|
+
const line = `${testable} of ${decisions} carry a claim a machine could test, ${covered} of which name a check that exists`
|
|
792
|
+
// A record whose every testable claim names a check has nothing to act on,
|
|
793
|
+
// and so does one carrying no testable claim at all. Warning on both is how
|
|
794
|
+
// a section becomes one nobody reads after the second run.
|
|
795
|
+
if (testable > covered) logWarn(line)
|
|
796
|
+
else logInfo(`${line}.`)
|
|
797
|
+
logInfo(
|
|
798
|
+
'Coverage reads the entry rather than the tree, so a claim some check happens to cover without the entry naming it reads as unchecked.',
|
|
799
|
+
)
|
|
800
|
+
|
|
801
|
+
pipeOutput(
|
|
802
|
+
report.decisions
|
|
803
|
+
.map((entry) => {
|
|
804
|
+
const kind = CLAIM_LABEL[entry.claim] ?? entry.claim
|
|
805
|
+
const evidence =
|
|
806
|
+
entry.figures.length > 0 ? ` ${entry.figures.join(' ')}` : ''
|
|
807
|
+
const checks =
|
|
808
|
+
entry.checks.length > 0
|
|
809
|
+
? `\n checked by ${entry.checks.join(', ')}`
|
|
810
|
+
: ''
|
|
811
|
+
return `${report.rel}:${entry.line} ${kind}${evidence}\n ${entry.heading}${checks}`
|
|
812
|
+
})
|
|
813
|
+
.join('\n'),
|
|
814
|
+
)
|
|
815
|
+
}
|
|
816
|
+
|
|
699
817
|
function reportDrift(drift: readonly FolderDrift[]): void {
|
|
700
818
|
logStep('Index drift')
|
|
701
819
|
|
|
@@ -0,0 +1,364 @@
|
|
|
1
|
+
import { access, readFile } from 'node:fs/promises'
|
|
2
|
+
import { join } from 'node:path'
|
|
3
|
+
import { AUDITS } from '@/audits/catalog'
|
|
4
|
+
import { bodyLines } from '@/markdown/scan'
|
|
5
|
+
|
|
6
|
+
/**
|
|
7
|
+
* The record this measures, relative to the project root.
|
|
8
|
+
*
|
|
9
|
+
* One fixed path rather than a folder walk, because the standard governing it
|
|
10
|
+
* names one document, and the length rule this measures is stated by whichever
|
|
11
|
+
* record sits there rather than by the standard or by this file.
|
|
12
|
+
*/
|
|
13
|
+
export const RECORD_REL = '.claude/ARCHITECTURE.md'
|
|
14
|
+
|
|
15
|
+
/**
|
|
16
|
+
* The line allowances a record states for itself, absent when it states none.
|
|
17
|
+
*
|
|
18
|
+
* No standard sets a length rule for this document, so the numbers belong to
|
|
19
|
+
* whichever record declares them rather than to the toolkit. Holding a pair in
|
|
20
|
+
* code and gating every project against it audits a target against a rule it
|
|
21
|
+
* never adopted, which is the failure `canResolveAtRoot` already answers on the
|
|
22
|
+
* folder side. A record stating no rule is measured and never gated.
|
|
23
|
+
*/
|
|
24
|
+
export interface Allowances {
|
|
25
|
+
readonly frame: number
|
|
26
|
+
readonly perDecision: number
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
/**
|
|
30
|
+
* What a machine could do with the entry's reasoning, which is a candidate
|
|
31
|
+
* rather than a verdict.
|
|
32
|
+
*
|
|
33
|
+
* - `countable`: carries a figure over a tree, so a run could recompute it.
|
|
34
|
+
* - `invariant`: quantifies over a named tree, so a walk could falsify it.
|
|
35
|
+
* - `neither`: says why a choice was made, which nothing tests.
|
|
36
|
+
*/
|
|
37
|
+
export type ClaimKind = 'countable' | 'invariant' | 'neither'
|
|
38
|
+
|
|
39
|
+
export interface DecisionReport {
|
|
40
|
+
readonly heading: string
|
|
41
|
+
/** Line of the `###` heading, so a report line opens at the entry. */
|
|
42
|
+
readonly line: number
|
|
43
|
+
readonly claim: ClaimKind
|
|
44
|
+
/** The digit-spelled figures behind a `countable` reading, in order. */
|
|
45
|
+
readonly figures: readonly string[]
|
|
46
|
+
/** The first sentence behind an `invariant` reading, absent otherwise. */
|
|
47
|
+
readonly quantified?: string
|
|
48
|
+
/**
|
|
49
|
+
* Executable checks the entry names, which is the only coverage signal the
|
|
50
|
+
* record carries. Empty on an entry that names none, including one whose
|
|
51
|
+
* claim some check happens to cover without the entry saying so.
|
|
52
|
+
*/
|
|
53
|
+
readonly checks: readonly string[]
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
export interface ArchitectureReport {
|
|
57
|
+
readonly rel: string
|
|
58
|
+
readonly lines: number
|
|
59
|
+
/** What the record declared, absent when it states no length rule. */
|
|
60
|
+
readonly allowances?: Allowances
|
|
61
|
+
/** The frame plus the per-decision allowance, absent alongside it. */
|
|
62
|
+
readonly ceiling?: number
|
|
63
|
+
readonly decisions: readonly DecisionReport[]
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
const DECISION_HEADING = /^###\s+(.+?)\s*$/
|
|
67
|
+
const SECTION_HEADING = /^##\s+\S/
|
|
68
|
+
const CODE_SPAN = /`[^`]*`/g
|
|
69
|
+
/** Dropped ahead of the figure scan, since an anchor date is not a claim. */
|
|
70
|
+
const ISO_DATE = /\b\d{4}-\d{2}-\d{2}\b/g
|
|
71
|
+
const FIGURE = /\b\d+(?:,\d{3})*\b/g
|
|
72
|
+
const SENTENCE_SPLIT = /(?<=[.])\s+/
|
|
73
|
+
|
|
74
|
+
/** A code span naming a path, which is what a quantifier has to govern. */
|
|
75
|
+
const PATH_SPAN = String.raw`\`[^\`]*(?:/|\.md|\.ts|\.sh|\.json)[^\`]*\``
|
|
76
|
+
const QUANTIFIER = String.raw`\b(?:every|each|no|nothing|any|all|never|only)\b`
|
|
77
|
+
/** Six words is the widest gap the corpus puts between the two. */
|
|
78
|
+
const WINDOW = String.raw`(?:\s+\S+){0,6}?\s+`
|
|
79
|
+
|
|
80
|
+
const QUANTIFIES_PATH = new RegExp(
|
|
81
|
+
`${QUANTIFIER}${WINDOW}${PATH_SPAN}|${PATH_SPAN}${WINDOW}${QUANTIFIER}`,
|
|
82
|
+
'i',
|
|
83
|
+
)
|
|
84
|
+
|
|
85
|
+
/** A code span naming a shell check this repository could run. */
|
|
86
|
+
const SCRIPT_SPAN = /^scripts\/[\w./-]+\.sh$/
|
|
87
|
+
/** A code span invoking the CLI, which may or may not name a registered audit. */
|
|
88
|
+
const AITK_SPAN = /^aitk\s+(.+)$/
|
|
89
|
+
|
|
90
|
+
/**
|
|
91
|
+
* The audit invocations a decision could name, spelled as a reader writes them.
|
|
92
|
+
*
|
|
93
|
+
* Read off the catalog rather than listed, so a verb renamed at its source
|
|
94
|
+
* stops matching here instead of going on matching a string nobody maintains.
|
|
95
|
+
*/
|
|
96
|
+
const AUDIT_INVOCATIONS: readonly string[] = AUDITS.map((audit) =>
|
|
97
|
+
audit.argv.filter((arg) => arg !== '--json').join(' '),
|
|
98
|
+
)
|
|
99
|
+
|
|
100
|
+
function codeSpans(text: string): string[] {
|
|
101
|
+
return (text.match(CODE_SPAN) ?? []).map((span) => span.slice(1, -1))
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
/**
|
|
105
|
+
* Names the executable checks an entry spells.
|
|
106
|
+
*
|
|
107
|
+
* A script has to exist on disk and an `aitk` invocation has to match a
|
|
108
|
+
* registered audit, because an entry naming a check that was removed is an
|
|
109
|
+
* uncovered claim wearing a covered one's words.
|
|
110
|
+
*/
|
|
111
|
+
async function namedChecks(root: string, body: string): Promise<string[]> {
|
|
112
|
+
const spans = [...new Set(codeSpans(body))]
|
|
113
|
+
|
|
114
|
+
const verbs = spans.filter((span) => {
|
|
115
|
+
const invocation = span.match(AITK_SPAN)?.[1]
|
|
116
|
+
return invocation !== undefined && AUDIT_INVOCATIONS.includes(invocation)
|
|
117
|
+
})
|
|
118
|
+
|
|
119
|
+
const scripts = spans.filter((span) => SCRIPT_SPAN.test(span))
|
|
120
|
+
const present = await Promise.all(
|
|
121
|
+
scripts.map((span) =>
|
|
122
|
+
access(join(root, span)).then(
|
|
123
|
+
() => true,
|
|
124
|
+
() => false,
|
|
125
|
+
),
|
|
126
|
+
),
|
|
127
|
+
)
|
|
128
|
+
|
|
129
|
+
return [...verbs, ...scripts.filter((_, index) => present[index])]
|
|
130
|
+
}
|
|
131
|
+
|
|
132
|
+
/**
|
|
133
|
+
* Reads the figures a decision carries, which is the countable-claim signal.
|
|
134
|
+
*
|
|
135
|
+
* Digits alone. A cardinal spelled in words reads as pronominal far more often
|
|
136
|
+
* than as measured in this corpus, where "the alternative and it is one nobody
|
|
137
|
+
* passes" outnumbers "eleven copies", and admitting the spelled form classified
|
|
138
|
+
* 22 of 24 entries as countable, which distinguishes nothing. The cost is that
|
|
139
|
+
* a measured claim written in words reads as uncounted, which the report says.
|
|
140
|
+
*/
|
|
141
|
+
function figuresIn(body: string): string[] {
|
|
142
|
+
return body.replace(ISO_DATE, ' ').replace(CODE_SPAN, ' ').match(FIGURE) ?? []
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
/**
|
|
146
|
+
* The first sentence quantifying over a named tree, or undefined.
|
|
147
|
+
*
|
|
148
|
+
* The quantifier has to sit within a short window of the path so a sentence
|
|
149
|
+
* mentioning both without relating them does not read as a claim about the
|
|
150
|
+
* tree. Both orders are matched, since the corpus writes the property before
|
|
151
|
+
* the path as readily as after it.
|
|
152
|
+
*/
|
|
153
|
+
function quantifiedSentence(body: string): string | undefined {
|
|
154
|
+
return body
|
|
155
|
+
.replace(ISO_DATE, ' ')
|
|
156
|
+
.split(SENTENCE_SPLIT)
|
|
157
|
+
.map((sentence) => sentence.replace(/\n/g, ' ').trim())
|
|
158
|
+
.find((sentence) => QUANTIFIES_PATH.test(sentence))
|
|
159
|
+
}
|
|
160
|
+
|
|
161
|
+
/**
|
|
162
|
+
* Classifies one entry, with the countable reading taking precedence.
|
|
163
|
+
*
|
|
164
|
+
* An entry carrying both a figure and a quantified tree is recomputable, which
|
|
165
|
+
* is the stronger test, and the two readings are reported as one kind because
|
|
166
|
+
* the coverage split a reader wants is testable against unverifiable rather
|
|
167
|
+
* than a per-entry inventory of every claim in it.
|
|
168
|
+
*/
|
|
169
|
+
export function classifyDecision(body: string): {
|
|
170
|
+
claim: ClaimKind
|
|
171
|
+
figures: string[]
|
|
172
|
+
quantified?: string
|
|
173
|
+
} {
|
|
174
|
+
const figures = figuresIn(body)
|
|
175
|
+
if (figures.length > 0) return { claim: 'countable', figures }
|
|
176
|
+
|
|
177
|
+
const quantified = quantifiedSentence(body)
|
|
178
|
+
if (quantified !== undefined) {
|
|
179
|
+
return { claim: 'invariant', figures: [], quantified }
|
|
180
|
+
}
|
|
181
|
+
|
|
182
|
+
return { claim: 'neither', figures: [] }
|
|
183
|
+
}
|
|
184
|
+
|
|
185
|
+
interface RawDecision {
|
|
186
|
+
readonly heading: string
|
|
187
|
+
readonly line: number
|
|
188
|
+
readonly body: string
|
|
189
|
+
}
|
|
190
|
+
|
|
191
|
+
/**
|
|
192
|
+
* Splits the record into its `###` entries.
|
|
193
|
+
*
|
|
194
|
+
* A heading inside a fenced block is skipped, since the seed template shows the
|
|
195
|
+
* shape it asks a project to write and a template entry is not a decision.
|
|
196
|
+
*
|
|
197
|
+
* A heading carrying several decisions counts once, so the total reads low by
|
|
198
|
+
* however many it holds. The report states that rather than parsing for it,
|
|
199
|
+
* because splitting a decision from its heading needs a marker the standard
|
|
200
|
+
* does not ask a record to carry.
|
|
201
|
+
*/
|
|
202
|
+
export function splitDecisions(source: string): RawDecision[] {
|
|
203
|
+
const lines = bodyLines(source)
|
|
204
|
+
const decisions: RawDecision[] = []
|
|
205
|
+
let open: { heading: string; line: number; body: string[] } | undefined
|
|
206
|
+
|
|
207
|
+
for (const line of lines) {
|
|
208
|
+
if (line.fenced) {
|
|
209
|
+
open?.body.push(line.text)
|
|
210
|
+
continue
|
|
211
|
+
}
|
|
212
|
+
|
|
213
|
+
const heading = line.text.match(DECISION_HEADING)?.[1]
|
|
214
|
+
if (heading !== undefined) {
|
|
215
|
+
if (open) decisions.push({ ...open, body: open.body.join('\n') })
|
|
216
|
+
open = { heading, line: line.number, body: [] }
|
|
217
|
+
continue
|
|
218
|
+
}
|
|
219
|
+
|
|
220
|
+
// A decision runs to the next `###` or to the section that follows the
|
|
221
|
+
// decision list, so the risks below never read as the last entry's body.
|
|
222
|
+
if (SECTION_HEADING.test(line.text)) {
|
|
223
|
+
if (open) decisions.push({ ...open, body: open.body.join('\n') })
|
|
224
|
+
open = undefined
|
|
225
|
+
continue
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
open?.body.push(line.text)
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
if (open) decisions.push({ ...open, body: open.body.join('\n') })
|
|
232
|
+
return decisions
|
|
233
|
+
}
|
|
234
|
+
|
|
235
|
+
/** Cardinals a record spells rather than writes, which the corpus does for both. */
|
|
236
|
+
const SPELLED: Record<string, number> = {
|
|
237
|
+
one: 1,
|
|
238
|
+
two: 2,
|
|
239
|
+
three: 3,
|
|
240
|
+
four: 4,
|
|
241
|
+
five: 5,
|
|
242
|
+
six: 6,
|
|
243
|
+
seven: 7,
|
|
244
|
+
eight: 8,
|
|
245
|
+
nine: 9,
|
|
246
|
+
ten: 10,
|
|
247
|
+
}
|
|
248
|
+
|
|
249
|
+
const CARDINAL = String.raw`(\d+|${Object.keys(SPELLED).join('|')})`
|
|
250
|
+
const FRAME_CLAUSE = new RegExp(String.raw`${CARDINAL}-line frame`, 'i')
|
|
251
|
+
const PER_DECISION_CLAUSE = new RegExp(
|
|
252
|
+
String.raw`${CARDINAL}\s+lines?\s+a\s+decision`,
|
|
253
|
+
'i',
|
|
254
|
+
)
|
|
255
|
+
|
|
256
|
+
function readCardinal(token: string | undefined): number | undefined {
|
|
257
|
+
if (token === undefined) return undefined
|
|
258
|
+
const spelled = SPELLED[token.toLowerCase()]
|
|
259
|
+
if (spelled !== undefined) return spelled
|
|
260
|
+
|
|
261
|
+
const digits = Number.parseInt(token, 10)
|
|
262
|
+
return Number.isNaN(digits) ? undefined : digits
|
|
263
|
+
}
|
|
264
|
+
|
|
265
|
+
/**
|
|
266
|
+
* Reads the allowances a record declares for itself, or nothing.
|
|
267
|
+
*
|
|
268
|
+
* Both clauses have to be present, because half a formula is not one. A record
|
|
269
|
+
* whose wording drifts past these clauses falls back to reporting rather than
|
|
270
|
+
* to a stale ceiling held here, so the failure is visible in the run's own
|
|
271
|
+
* output instead of gating a project on a rule nobody can point at.
|
|
272
|
+
*/
|
|
273
|
+
export function readAllowances(source: string): Allowances | undefined {
|
|
274
|
+
const frame = readCardinal(source.match(FRAME_CLAUSE)?.[1])
|
|
275
|
+
const perDecision = readCardinal(source.match(PER_DECISION_CLAUSE)?.[1])
|
|
276
|
+
|
|
277
|
+
if (frame === undefined || perDecision === undefined) return undefined
|
|
278
|
+
return { frame, perDecision }
|
|
279
|
+
}
|
|
280
|
+
|
|
281
|
+
/** Whether a read failed because nothing sits at the path. */
|
|
282
|
+
function isMissing(error: unknown): boolean {
|
|
283
|
+
const code = (error as { code?: unknown }).code
|
|
284
|
+
return code === 'ENOENT' || code === 'ENOTDIR'
|
|
285
|
+
}
|
|
286
|
+
|
|
287
|
+
/** The ceiling the record's own formula derives from its decision count. */
|
|
288
|
+
export function ceilingFor(allowances: Allowances, decisions: number): number {
|
|
289
|
+
return allowances.frame + allowances.perDecision * decisions
|
|
290
|
+
}
|
|
291
|
+
|
|
292
|
+
/**
|
|
293
|
+
* Measures the record, or reports nothing when the project carries none.
|
|
294
|
+
*
|
|
295
|
+
* Absent rather than empty, for the reason the sibling checks state: a project
|
|
296
|
+
* with no record and one whose record holds no decision are different answers,
|
|
297
|
+
* and a zeroed report reads as the second.
|
|
298
|
+
*/
|
|
299
|
+
export async function measureArchitecture(
|
|
300
|
+
root: string,
|
|
301
|
+
): Promise<ArchitectureReport | undefined> {
|
|
302
|
+
const path = join(root, RECORD_REL)
|
|
303
|
+
|
|
304
|
+
let source: string
|
|
305
|
+
try {
|
|
306
|
+
source = await readFile(path, 'utf8')
|
|
307
|
+
} catch (error) {
|
|
308
|
+
// Only a record that is not there reads as absent. A record present and
|
|
309
|
+
// unreadable propagates the way every sibling reader here lets one
|
|
310
|
+
// propagate, since swallowing it reports a project with no record and the
|
|
311
|
+
// length gate passes over a file nobody opened.
|
|
312
|
+
if (!isMissing(error)) throw error
|
|
313
|
+
return undefined
|
|
314
|
+
}
|
|
315
|
+
|
|
316
|
+
const allowances = readAllowances(source)
|
|
317
|
+
const raw = splitDecisions(source)
|
|
318
|
+
const decisions = await Promise.all(
|
|
319
|
+
raw.map(async (entry) => {
|
|
320
|
+
const { claim, figures, quantified } = classifyDecision(entry.body)
|
|
321
|
+
return {
|
|
322
|
+
heading: entry.heading,
|
|
323
|
+
line: entry.line,
|
|
324
|
+
claim,
|
|
325
|
+
figures,
|
|
326
|
+
...(quantified !== undefined && { quantified }),
|
|
327
|
+
checks: await namedChecks(root, entry.body),
|
|
328
|
+
}
|
|
329
|
+
}),
|
|
330
|
+
)
|
|
331
|
+
|
|
332
|
+
return {
|
|
333
|
+
rel: RECORD_REL,
|
|
334
|
+
lines: source.replace(/\n$/, '').split('\n').length,
|
|
335
|
+
...(allowances !== undefined && {
|
|
336
|
+
allowances,
|
|
337
|
+
ceiling: ceilingFor(allowances, raw.length),
|
|
338
|
+
}),
|
|
339
|
+
decisions,
|
|
340
|
+
}
|
|
341
|
+
}
|
|
342
|
+
|
|
343
|
+
/**
|
|
344
|
+
* Whether the record is longer than the ceiling it derives for itself.
|
|
345
|
+
*
|
|
346
|
+
* False for a record declaring no allowances, which has no ceiling to be past.
|
|
347
|
+
* That is the answer rather than a gap, since the length rule is the record's
|
|
348
|
+
* own and a project that never wrote one owes nothing to it.
|
|
349
|
+
*/
|
|
350
|
+
export function isOverLength(report: ArchitectureReport): boolean {
|
|
351
|
+
return report.ceiling !== undefined && report.lines > report.ceiling
|
|
352
|
+
}
|
|
353
|
+
|
|
354
|
+
/** How many entries carry a claim a machine could test. */
|
|
355
|
+
export function testableCount(report: ArchitectureReport): number {
|
|
356
|
+
return report.decisions.filter((entry) => entry.claim !== 'neither').length
|
|
357
|
+
}
|
|
358
|
+
|
|
359
|
+
/** How many testable entries name a check that exists. */
|
|
360
|
+
export function coveredCount(report: ArchitectureReport): number {
|
|
361
|
+
return report.decisions.filter(
|
|
362
|
+
(entry) => entry.claim !== 'neither' && entry.checks.length > 0,
|
|
363
|
+
).length
|
|
364
|
+
}
|
package/src/context/gate.ts
CHANGED
|
@@ -4,6 +4,15 @@ import type { FolderDrift } from '@/context/index-drift'
|
|
|
4
4
|
export interface GateInput {
|
|
5
5
|
/** Cited paths that resolved to nothing, which gate under either mode. */
|
|
6
6
|
readonly unresolvedCitations: number
|
|
7
|
+
/**
|
|
8
|
+
* Whether the architecture record is longer than the ceiling it derives for
|
|
9
|
+
* itself, which gates under either mode for the reason a citation does.
|
|
10
|
+
*
|
|
11
|
+
* False when the project carries no record and false under
|
|
12
|
+
* `--citations-only`, which never measures it. That mode runs one check by
|
|
13
|
+
* construction, so widening it here would gate on a reading it never took.
|
|
14
|
+
*/
|
|
15
|
+
readonly recordOverLength: boolean
|
|
7
16
|
readonly sections: readonly SectionFinding[]
|
|
8
17
|
readonly drift: readonly FolderDrift[]
|
|
9
18
|
/**
|
|
@@ -24,19 +33,24 @@ export function hasDrift(drift: readonly FolderDrift[]): boolean {
|
|
|
24
33
|
/**
|
|
25
34
|
* Whether the audit found something that should fail the caller.
|
|
26
35
|
*
|
|
27
|
-
* An unresolved citation is a broken pointer and gates unconditionally
|
|
28
|
-
*
|
|
29
|
-
*
|
|
30
|
-
*
|
|
31
|
-
*
|
|
36
|
+
* An unresolved citation is a broken pointer and gates unconditionally, and so
|
|
37
|
+
* does a record past its own ceiling: the record states the limit for itself
|
|
38
|
+
* and derives it from a count, which makes it the one measure here that is a
|
|
39
|
+
* fact rather than a threshold a reader weighs. The two findings `--gate` adds
|
|
40
|
+
* are the ones answerable from the file itself: a required section it does not
|
|
41
|
+
* declare, and an index disagreeing with its folder. Entry length, depth,
|
|
42
|
+
* bullet, table, provenance, and the record's claim coverage are judgments, so
|
|
43
|
+
* they stay out under both modes.
|
|
32
44
|
*/
|
|
33
45
|
export function isGating({
|
|
34
46
|
unresolvedCitations,
|
|
47
|
+
recordOverLength,
|
|
35
48
|
sections,
|
|
36
49
|
drift,
|
|
37
50
|
widened,
|
|
38
51
|
}: GateInput): boolean {
|
|
39
52
|
if (unresolvedCitations > 0) return true
|
|
53
|
+
if (recordOverLength) return true
|
|
40
54
|
if (!widened) return false
|
|
41
55
|
|
|
42
56
|
return sections.length > 0 || hasDrift(drift)
|