@erclx/aitk 0.19.0 → 0.21.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/claude/.claude-plugin/plugin.json +1 -1
- package/docs/agents.md +39 -0
- package/package.json +1 -1
- package/scripts/core/verify.sh +9 -0
- package/scripts/lib/sandbox-path.sh +177 -0
- package/scripts/lib/ui.sh +9 -1
- package/scripts/manage-sandbox.sh +10 -4
- package/src/cli.ts +4 -0
- package/src/commands/context.ts +321 -0
- package/src/commands/sandbox.ts +25 -1
- package/src/context/audit.ts +227 -0
- package/src/context/citations.ts +167 -0
- package/src/context/folders.ts +91 -0
- package/src/context/index-drift.ts +64 -0
- package/src/sandbox/expect.ts +38 -0
package/docs/agents.md
CHANGED
|
@@ -80,6 +80,7 @@ Full help: `aitk <command> --help`.
|
|
|
80
80
|
| `aitk transcripts <url>` | Fetch a YouTube transcript with metadata frontmatter (needs `yt-dlp`) |
|
|
81
81
|
| `aitk tasks archive` | Move a shipped task off the board, clear its ordering row, and regenerate the index |
|
|
82
82
|
| `aitk comments scan` | Measure comment density by language and comment kind, with a trend recomputed from git |
|
|
83
|
+
| `aitk context audit` | Report entry length, depth, cited-path resolution, and index drift in the context folders |
|
|
83
84
|
| `aitk capture [source]` | Render HTML capture sources to PNG, toolkit-only and absent from an installed package |
|
|
84
85
|
|
|
85
86
|
### Domain commands
|
|
@@ -98,6 +99,7 @@ Each domain exposes a consistent shape where applicable: `list`, `install`, `syn
|
|
|
98
99
|
| `slides` | `render`, `list` |
|
|
99
100
|
| `tasks` | `archive` |
|
|
100
101
|
| `comments` | `scan` |
|
|
102
|
+
| `context` | `audit` |
|
|
101
103
|
|
|
102
104
|
Common patterns:
|
|
103
105
|
|
|
@@ -351,6 +353,43 @@ The degradation sweep reads its vocabulary from whichever rule publishes a `## D
|
|
|
351
353
|
|
|
352
354
|
`090-code-comments` is the rule that publishes the list, and it ships on the `base` stack. A project that installs or syncs governance for the first time after that rule landed gets a sweep that previously reported skipped, so hits appear where the command used to stay quiet. Edit the backticked terms in the installed copy to change what that project sweeps for. The sweep matches comment text, so a comment naming a term as an example is a hit, and a hit is a prompt to read the line rather than a verdict on it.
|
|
353
355
|
|
|
356
|
+
## Context audit
|
|
357
|
+
|
|
358
|
+
`aitk context audit [path]` reports the structural state of the folders following the index-plus-entry contract, meaning a generated `index.md` beside entries carrying frontmatter. It reads and reports. Fixing what it finds is separate work.
|
|
359
|
+
|
|
360
|
+
```bash
|
|
361
|
+
aitk context audit
|
|
362
|
+
aitk context audit --json
|
|
363
|
+
aitk context audit --citations-only
|
|
364
|
+
aitk context audit --folder context,diagrams
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
| Option | Behavior |
|
|
368
|
+
| ------------------ | ------------------------------------------------------------------------ |
|
|
369
|
+
| `--json` | Add a machine-readable record on stdout, keeping the frame |
|
|
370
|
+
| `--folder <list>` | Comma-separated folder names under `.claude/` (default: the three below) |
|
|
371
|
+
| `--citations-only` | Run the gating citation check alone, printing nothing when it passes |
|
|
372
|
+
|
|
373
|
+
Scope defaults to `context`, `diagrams`, and `wireframes`, and a folder the project does not carry is skipped rather than reported. A domain that outgrew one file and split into `<domain>/` is audited as its own folder, so a split entry measures at the same grain as a flat one.
|
|
374
|
+
|
|
375
|
+
Exit codes are `0` for a clean run, `1` for a refusal, and `2` for an unresolved citation. Only the citation check sets a failing code. Length, depth, table, and index findings print and return `0`, because each is a judgment threshold and failing a push on one would make the check something to route around.
|
|
376
|
+
|
|
377
|
+
### What each check reports
|
|
378
|
+
|
|
379
|
+
Length and depth quote their checkpoints from `.claude/standards/context.md`: roughly 150 lines for an entry, roughly 40 for a run of lines no heading breaks. Depth measures the longest such run rather than everything under one `##`, skips fenced blocks so a markdown example does not read as three headings, and exempts a run whose lines are all list items at one indent. Runs count blank lines, which the standard leaves open, so a hand reader who drops them lands a line or two lower. The report states the convention on every run.
|
|
380
|
+
|
|
381
|
+
The table check reports a catalog that grows a row per shipped thing, not a table count. A fixed comparison table never reflows, so its size costs nothing. A table qualifies at six or more body rows whose first column mostly carries a path, command, or link, which is what separates a catalog from a comparison without reading the prose.
|
|
382
|
+
|
|
383
|
+
Index drift compares an index against its siblings in both directions. An entry the index does not link is invisible to a session choosing what to open, and a linked name resolving to nothing sends one to a path that opens nothing.
|
|
384
|
+
|
|
385
|
+
### The citation gate
|
|
386
|
+
|
|
387
|
+
The citation check resolves every path into an audited folder that appears anywhere in the repository, and it is the half wired into `bun run check`. A stale reference has a silent failure mode: the session opens nothing and carries on.
|
|
388
|
+
|
|
389
|
+
Three exclusions keep it from firing on prose about paths. Fenced blocks are skipped in markdown, which covers a standard displaying a path as an example. Fixture and harness trees are skipped by location, covering sandbox scenarios that describe their own scratch tree, the eval harness naming its target project, and `*.test.ts`. A path into a folder the project does not carry is skipped, so a skill directing a reader to `.claude/wireframes/index.md` stays valid in a project that has wireframes and silent in one that does not.
|
|
390
|
+
|
|
391
|
+
What remains is a sentence naming a hypothetical entry to show the shape of a name, which no syntax separates from a real reference. Append `<!-- audit-ignore-citations -->` to that source line. The marker suppresses citation checking for its own line only.
|
|
392
|
+
|
|
354
393
|
## Runtime catalogs
|
|
355
394
|
|
|
356
395
|
Use these to discover what's available instead of hardcoding names.
|
package/package.json
CHANGED
package/scripts/core/verify.sh
CHANGED
|
@@ -148,6 +148,15 @@ main() {
|
|
|
148
148
|
run_check "bash $PROJECT_ROOT/scripts/core/check-plugin-boundary.sh" "Plugin ships toolkit-internal content."
|
|
149
149
|
log_info "Plugin boundary clean"
|
|
150
150
|
|
|
151
|
+
# Only the citation half of the audit gates. Length, depth, table, and index
|
|
152
|
+
# findings are judgment thresholds, and failing a push on one would make the
|
|
153
|
+
# stage something to route around. `bun src/cli.ts` rather than `aitk`, since a
|
|
154
|
+
# globally installed binary resolves to the main checkout no matter which
|
|
155
|
+
# worktree is running.
|
|
156
|
+
log_step "Context citations"
|
|
157
|
+
run_check "cd $PROJECT_ROOT && bun src/cli.ts context audit --citations-only" "A cited context path does not resolve. Run bun src/cli.ts context audit."
|
|
158
|
+
log_info "Context citations resolve"
|
|
159
|
+
|
|
151
160
|
log_step "Plugin manifests"
|
|
152
161
|
if ! command -v claude >/dev/null 2>&1; then
|
|
153
162
|
log_info "Skipped, claude is not installed"
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
|
|
3
|
+
# The sandbox tree lives outside the toolkit worktree. `scripts/sandbox/run.sh`
|
|
4
|
+
# sets cwd to it for `claude -p`, and every `CLAUDE.md` between that cwd and the
|
|
5
|
+
# filesystem root loads into the session. Under the repository the toolkit's own
|
|
6
|
+
# instructions join that chain beside the seeded copy the scenario installed,
|
|
7
|
+
# both carry the rule sending shared session scratch to the main worktree root,
|
|
8
|
+
# and nothing decides which root wins. A session picking the toolkit writes its
|
|
9
|
+
# output where no manifest reads it, so the run reports success while the verdict
|
|
10
|
+
# reports no writes at all. `scripts/eval/run.sh` keeps its fixture outside the
|
|
11
|
+
# repository for the same reason.
|
|
12
|
+
#
|
|
13
|
+
# Twin of `SANDBOX_DIR` in `src/commands/sandbox.ts`. The exec boundary rules out
|
|
14
|
+
# a shared constant, so a change to the default lands on both sides.
|
|
15
|
+
resolve_sandbox_dir() {
|
|
16
|
+
if [ -n "${AITK_SANDBOX_DIR:-}" ]; then
|
|
17
|
+
printf '%s\n' "$AITK_SANDBOX_DIR"
|
|
18
|
+
return 0
|
|
19
|
+
fi
|
|
20
|
+
|
|
21
|
+
printf '%s/aitk/sandbox\n' "${XDG_STATE_HOME:-$HOME/.local/state}"
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
# Collapses repeated separators, folds `.` and `..` segments, and strips every
|
|
25
|
+
# trailing separator, leaving a bare root as `/`. Every comparison below is a
|
|
26
|
+
# string test, so `//`, `$HOME//`, and `$HOME/../../usr` each read as a path no
|
|
27
|
+
# rule names until this runs, and an unresolved `..` defeats the allowlist and
|
|
28
|
+
# both directions of the repository test at once.
|
|
29
|
+
#
|
|
30
|
+
# The fold is lexical because the guard runs before provisioning creates the
|
|
31
|
+
# tree, which rules out `cd` with `pwd -P` and any resolution needing the path
|
|
32
|
+
# to exist. Nothing here follows a symlink, so a `..` below one resolves against
|
|
33
|
+
# the link's own path rather than its target. A `..` climbing past the root
|
|
34
|
+
# clamps to `/`, matching the kernel, which leaves the allowlist to refuse it
|
|
35
|
+
# under the rule that already covers every root path.
|
|
36
|
+
normalize_sandbox_path() {
|
|
37
|
+
local path="$1"
|
|
38
|
+
|
|
39
|
+
[ -n "$path" ] || return 0
|
|
40
|
+
|
|
41
|
+
while [ "$path" != "${path//\/\//\/}" ]; do
|
|
42
|
+
path="${path//\/\//\/}"
|
|
43
|
+
done
|
|
44
|
+
|
|
45
|
+
local root=""
|
|
46
|
+
if [ "${path#/}" != "$path" ]; then
|
|
47
|
+
root="/"
|
|
48
|
+
path="${path#/}"
|
|
49
|
+
fi
|
|
50
|
+
|
|
51
|
+
# `resolved` carries each kept segment behind its own separator, so a pop is
|
|
52
|
+
# one suffix removal and popping an empty stack is the no-op that clamps at
|
|
53
|
+
# the root. `climbed` holds the leading `..` a relative path has no segment to
|
|
54
|
+
# pop against, which dropping would change the directory it names.
|
|
55
|
+
local resolved="" climbed="" segment
|
|
56
|
+
local remaining=4096
|
|
57
|
+
|
|
58
|
+
while [ -n "$path" ] && [ "$remaining" -gt 0 ]; do
|
|
59
|
+
remaining=$((remaining - 1))
|
|
60
|
+
|
|
61
|
+
segment="${path%%/*}"
|
|
62
|
+
if [ "$segment" = "$path" ]; then
|
|
63
|
+
path=""
|
|
64
|
+
else
|
|
65
|
+
path="${path#*/}"
|
|
66
|
+
fi
|
|
67
|
+
|
|
68
|
+
case "$segment" in
|
|
69
|
+
"" | .) ;;
|
|
70
|
+
..)
|
|
71
|
+
if [ -n "$resolved" ]; then
|
|
72
|
+
resolved="${resolved%/*}"
|
|
73
|
+
elif [ -z "$root" ]; then
|
|
74
|
+
climbed="$climbed../"
|
|
75
|
+
fi
|
|
76
|
+
;;
|
|
77
|
+
*) resolved="$resolved/$segment" ;;
|
|
78
|
+
esac
|
|
79
|
+
done
|
|
80
|
+
|
|
81
|
+
if [ -n "$root" ]; then
|
|
82
|
+
path="${resolved:-/}"
|
|
83
|
+
else
|
|
84
|
+
path="${climbed}${resolved#/}"
|
|
85
|
+
path="${path:-.}"
|
|
86
|
+
fi
|
|
87
|
+
|
|
88
|
+
while [ "${#path}" -gt 1 ] && [ "${path%/}" != "$path" ]; do
|
|
89
|
+
path="${path%/}"
|
|
90
|
+
done
|
|
91
|
+
|
|
92
|
+
printf '%s' "$path"
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
# Whether `candidate` is `target` or a directory containing it. Removing the
|
|
96
|
+
# former removes the latter, which is what makes an ancestor as dangerous as an
|
|
97
|
+
# exact match.
|
|
98
|
+
is_at_or_above() {
|
|
99
|
+
local candidate="$1"
|
|
100
|
+
local target="$2"
|
|
101
|
+
|
|
102
|
+
[ "$candidate" = "$target" ] || [ "${target#"$candidate"/}" != "$target" ]
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
# Prints the reason a location is unusable and returns non-zero, so the caller
|
|
106
|
+
# reports one message naming the path rather than a bare refusal.
|
|
107
|
+
#
|
|
108
|
+
# Provisioning runs `rm -rf` on this path at three sites before staging, so the
|
|
109
|
+
# test is an allowlist rather than a list of paths to refuse. A blocklist has to
|
|
110
|
+
# name every system directory to be right once and stays wrong as soon as one is
|
|
111
|
+
# missed, while requiring a strict descendant of the home directory or of the
|
|
112
|
+
# temp root admits the default and every reasonable override and refuses `/`,
|
|
113
|
+
# `/usr`, `/etc`, and `$HOME` itself without naming any of them.
|
|
114
|
+
#
|
|
115
|
+
# The repository test is separate and runs both ways. A path under the worktree
|
|
116
|
+
# restores the ancestor chain the relocation removed, and a path above it is one
|
|
117
|
+
# `rm -rf` away from deleting the repository. It resolves the main worktree root
|
|
118
|
+
# rather than trusting `$PROJECT_ROOT`, since a linked worktree is itself inside
|
|
119
|
+
# that root.
|
|
120
|
+
assert_sandbox_dir_safe() {
|
|
121
|
+
local raw="$1"
|
|
122
|
+
local root="${2:-${PROJECT_ROOT:-$PWD}}"
|
|
123
|
+
|
|
124
|
+
if [ -z "$raw" ] || [ "${raw#/}" = "$raw" ]; then
|
|
125
|
+
printf 'AITK_SANDBOX_DIR must be an absolute path, got: %s\n' "${raw:-<empty>}"
|
|
126
|
+
return 1
|
|
127
|
+
fi
|
|
128
|
+
|
|
129
|
+
# `normalize_sandbox_path` stops at 4096 segments and drops the remainder,
|
|
130
|
+
# which would hand the tests below an ancestor of the path provisioning
|
|
131
|
+
# removes. Refusing past `PATH_MAX` keeps that bound out of reach, since a
|
|
132
|
+
# segment costs at least two bytes and no syscall accepts the string anyway.
|
|
133
|
+
if [ "${#raw}" -gt 4096 ]; then
|
|
134
|
+
printf 'Refusing the sandbox path. It is %s characters, past the longest path any filesystem here accepts.\n' "${#raw}"
|
|
135
|
+
return 1
|
|
136
|
+
fi
|
|
137
|
+
|
|
138
|
+
local dir home temp
|
|
139
|
+
dir="$(normalize_sandbox_path "$raw")"
|
|
140
|
+
home="$(normalize_sandbox_path "${HOME:-/root}")"
|
|
141
|
+
temp="$(normalize_sandbox_path "${TMPDIR:-/tmp}")"
|
|
142
|
+
|
|
143
|
+
# Every message below names `$raw`, which is what the operator set. The tests
|
|
144
|
+
# compare `$dir`, so a path carrying `..` is refused for a location its own
|
|
145
|
+
# spelling does not show.
|
|
146
|
+
local resolution=""
|
|
147
|
+
[ "$dir" = "$raw" ] || resolution=" It resolves to $dir."
|
|
148
|
+
|
|
149
|
+
if ! is_at_or_above "$home" "$dir" && ! is_at_or_above "$temp" "$dir"; then
|
|
150
|
+
printf 'Refusing %s as the sandbox. Provisioning removes the tree first, so the path has to sit under %s or %s.%s\n' "$raw" "$home" "$temp" "$resolution"
|
|
151
|
+
return 1
|
|
152
|
+
fi
|
|
153
|
+
|
|
154
|
+
if [ "$dir" = "$home" ] || [ "$dir" = "$temp" ]; then
|
|
155
|
+
printf 'Refusing %s as the sandbox. Provisioning removes the tree first, so the path has to sit under %s rather than be it.\n' "$raw" "$dir"
|
|
156
|
+
return 1
|
|
157
|
+
fi
|
|
158
|
+
|
|
159
|
+
local main_root
|
|
160
|
+
main_root="$(git -C "$root" worktree list --porcelain 2>/dev/null |
|
|
161
|
+
grep -m 1 '^worktree ' | cut -d' ' -f2-)"
|
|
162
|
+
main_root="$(normalize_sandbox_path "${main_root:-$root}")"
|
|
163
|
+
|
|
164
|
+
if is_at_or_above "$dir" "$main_root"; then
|
|
165
|
+
printf 'Refusing %s as the sandbox. Provisioning removes the tree first, and that path contains %s.%s\n' "$raw" "$main_root" "$resolution"
|
|
166
|
+
return 1
|
|
167
|
+
fi
|
|
168
|
+
|
|
169
|
+
case "$dir" in
|
|
170
|
+
"$main_root"/*)
|
|
171
|
+
printf 'Sandbox at %s sits inside %s, which puts the toolkit CLAUDE.md back on the session ancestor chain.%s Point AITK_SANDBOX_DIR outside the repository.\n' "$raw" "$main_root" "$resolution"
|
|
172
|
+
return 1
|
|
173
|
+
;;
|
|
174
|
+
esac
|
|
175
|
+
|
|
176
|
+
return 0
|
|
177
|
+
}
|
package/scripts/lib/ui.sh
CHANGED
|
@@ -6,6 +6,12 @@ if [ "${BASH_VERSINFO[0]:-0}" -lt 4 ]; then
|
|
|
6
6
|
exit 1
|
|
7
7
|
fi
|
|
8
8
|
|
|
9
|
+
# Resolved from this file's own location rather than `$PROJECT_ROOT`, because
|
|
10
|
+
# `require_project_root` is what several scripts call before anything has
|
|
11
|
+
# established a root.
|
|
12
|
+
# shellcheck source=/dev/null
|
|
13
|
+
source "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")/sandbox-path.sh"
|
|
14
|
+
|
|
9
15
|
GREEN='\033[0;32m'
|
|
10
16
|
RED='\033[0;31m'
|
|
11
17
|
YELLOW='\033[0;33m'
|
|
@@ -46,7 +52,9 @@ guard_root() {
|
|
|
46
52
|
}
|
|
47
53
|
|
|
48
54
|
require_project_root() {
|
|
49
|
-
|
|
55
|
+
local sandbox
|
|
56
|
+
sandbox="$(resolve_sandbox_dir)"
|
|
57
|
+
if [[ "$PWD" == "$sandbox" || "$PWD" == "$sandbox"/* ]]; then
|
|
50
58
|
echo -e "${GREY}┌${NC}" >&2
|
|
51
59
|
log_error "Execution restricted: Command cannot be run from inside the sandbox environment."
|
|
52
60
|
fi
|
|
@@ -8,6 +8,7 @@ export PROJECT_ROOT
|
|
|
8
8
|
|
|
9
9
|
source "$PROJECT_ROOT/scripts/config.sh"
|
|
10
10
|
source "$PROJECT_ROOT/scripts/lib/ui.sh"
|
|
11
|
+
source "$PROJECT_ROOT/scripts/lib/sandbox-path.sh"
|
|
11
12
|
source "$PROJECT_ROOT/scripts/lib/sandbox-git.sh"
|
|
12
13
|
source "$PROJECT_ROOT/scripts/lib/sandbox-fixtures.sh"
|
|
13
14
|
|
|
@@ -135,8 +136,8 @@ validate_environment() {
|
|
|
135
136
|
log_error "Sandbox directory not found at: $SANDBOX_DIR"
|
|
136
137
|
fi
|
|
137
138
|
|
|
138
|
-
if [[ "$PWD" ==
|
|
139
|
-
log_warn "Detected execution inside
|
|
139
|
+
if [[ "$PWD" == "$SANDBOX" || "$PWD" == "$SANDBOX"/* ]]; then
|
|
140
|
+
log_warn "Detected execution inside the sandbox. Switching to project root..."
|
|
140
141
|
cd "$PROJECT_ROOT" || log_error "Failed to switch to project root."
|
|
141
142
|
fi
|
|
142
143
|
}
|
|
@@ -324,7 +325,7 @@ finalize_sandbox_run() {
|
|
|
324
325
|
cmd_clean() {
|
|
325
326
|
log_step "Removing sandbox"
|
|
326
327
|
rm -rf "$SANDBOX"
|
|
327
|
-
log_rem "
|
|
328
|
+
log_rem "$SANDBOX"
|
|
328
329
|
trap - EXIT
|
|
329
330
|
close_timeline
|
|
330
331
|
echo "" >&2
|
|
@@ -422,9 +423,14 @@ main() {
|
|
|
422
423
|
log_error "Context error: you must run this command from inside the toolkit repository."
|
|
423
424
|
fi
|
|
424
425
|
|
|
425
|
-
SANDBOX="$
|
|
426
|
+
SANDBOX="$(resolve_sandbox_dir)"
|
|
426
427
|
SANDBOX_DIR="$PROJECT_ROOT/scripts/sandbox"
|
|
427
428
|
|
|
429
|
+
local unsafe
|
|
430
|
+
if ! unsafe="$(assert_sandbox_dir_safe "$SANDBOX" "$PROJECT_ROOT")"; then
|
|
431
|
+
log_error "$unsafe"
|
|
432
|
+
fi
|
|
433
|
+
|
|
428
434
|
if [[ "$1" == "reset" ]]; then
|
|
429
435
|
reset_sandbox
|
|
430
436
|
exit 0
|
package/src/cli.ts
CHANGED
|
@@ -19,6 +19,7 @@ import { register as feedback } from '@/commands/feedback'
|
|
|
19
19
|
import { register as transcripts } from '@/commands/transcripts'
|
|
20
20
|
import { register as tasks } from '@/commands/tasks'
|
|
21
21
|
import { register as comments } from '@/commands/comments'
|
|
22
|
+
import { register as context } from '@/commands/context'
|
|
22
23
|
|
|
23
24
|
const GREY = '\x1b[0;90m'
|
|
24
25
|
const WHITE = '\x1b[1;37m'
|
|
@@ -48,6 +49,7 @@ function showHelp(): void {
|
|
|
48
49
|
`${GREY}│${NC} transcripts <url> ${GREY}# Fetch a YouTube transcript with metadata frontmatter${NC}`,
|
|
49
50
|
`${GREY}│${NC} tasks [cmd] ${GREY}# Task board commands (archive)${NC}`,
|
|
50
51
|
`${GREY}│${NC} comments [cmd] ${GREY}# Measure comment density and trend (scan)${NC}`,
|
|
52
|
+
`${GREY}│${NC} context [cmd] ${GREY}# Report context folder health (audit)${NC}`,
|
|
51
53
|
`${GREY}│${NC}`,
|
|
52
54
|
`${GREY}│${NC} ${WHITE}Sandbox:${NC}`,
|
|
53
55
|
`${GREY}│${NC} aitk sandbox ${GREY}# Interactive scenario picker${NC}`,
|
|
@@ -79,6 +81,7 @@ function showHelp(): void {
|
|
|
79
81
|
`${GREY}│${NC} aitk transcripts https://youtu.be/VIDEO_ID`,
|
|
80
82
|
`${GREY}│${NC} aitk tasks archive --pull-request 673 --json`,
|
|
81
83
|
`${GREY}│${NC} aitk comments scan src --json`,
|
|
84
|
+
`${GREY}│${NC} aitk context audit --json`,
|
|
82
85
|
`${GREY}└${NC}`,
|
|
83
86
|
]
|
|
84
87
|
console.log(lines.join('\n'))
|
|
@@ -115,5 +118,6 @@ feedback(program)
|
|
|
115
118
|
transcripts(program)
|
|
116
119
|
tasks(program)
|
|
117
120
|
comments(program)
|
|
121
|
+
context(program)
|
|
118
122
|
|
|
119
123
|
program.parse()
|
|
@@ -0,0 +1,321 @@
|
|
|
1
|
+
import { resolve } from 'node:path'
|
|
2
|
+
import type { Command } from 'commander'
|
|
3
|
+
import {
|
|
4
|
+
type EntryReport,
|
|
5
|
+
LENGTH_CHECKPOINT,
|
|
6
|
+
measureFolders,
|
|
7
|
+
RUN_CHECKPOINT,
|
|
8
|
+
} from '@/context/audit'
|
|
9
|
+
import { auditCitations, type CitationReport } from '@/context/citations'
|
|
10
|
+
import {
|
|
11
|
+
type AuditedFolder,
|
|
12
|
+
DEFAULT_FOLDERS,
|
|
13
|
+
presentNames,
|
|
14
|
+
resolveFolders,
|
|
15
|
+
} from '@/context/folders'
|
|
16
|
+
import { auditIndexes, type FolderDrift } from '@/context/index-drift'
|
|
17
|
+
import {
|
|
18
|
+
frameError,
|
|
19
|
+
intro,
|
|
20
|
+
logError,
|
|
21
|
+
logInfo,
|
|
22
|
+
logStep,
|
|
23
|
+
logWarn,
|
|
24
|
+
outro,
|
|
25
|
+
pipeOutput,
|
|
26
|
+
} from '@/ui'
|
|
27
|
+
|
|
28
|
+
/** Returned when an unresolved citation is found, which is the gating check. */
|
|
29
|
+
const EXIT_UNRESOLVED = 2
|
|
30
|
+
|
|
31
|
+
/** A name of dots alone is `.` or `..`, both of which escape the audit root. */
|
|
32
|
+
const FOLDER_NAME = /^(?!\.+$)[A-Za-z0-9._-]+$/
|
|
33
|
+
|
|
34
|
+
interface AuditCommandOptions {
|
|
35
|
+
readonly json?: boolean
|
|
36
|
+
readonly folder?: string
|
|
37
|
+
readonly citationsOnly?: boolean
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
export function register(program: Command): void {
|
|
41
|
+
const context = program
|
|
42
|
+
.command('context')
|
|
43
|
+
.description('Report the structural state of the context folders')
|
|
44
|
+
.helpOption('-h, --help', 'Show this help message')
|
|
45
|
+
|
|
46
|
+
context
|
|
47
|
+
.command('audit')
|
|
48
|
+
.description('Report entry length, depth, citations, and index drift')
|
|
49
|
+
.argument('[path]', 'Project root, defaulting to the current directory')
|
|
50
|
+
.helpOption('-h, --help', 'Show this help message')
|
|
51
|
+
.option('--json', 'Add a machine-readable record on stdout')
|
|
52
|
+
.option('--folder <list>', `Comma-separated folder names under .claude/`)
|
|
53
|
+
.option('--citations-only', 'Run the gating citation check alone')
|
|
54
|
+
.addHelpText(
|
|
55
|
+
'after',
|
|
56
|
+
[
|
|
57
|
+
'',
|
|
58
|
+
'Exit codes:',
|
|
59
|
+
' 0 the audit completed with every cited path resolving',
|
|
60
|
+
' 1 refused, with the reason on stderr',
|
|
61
|
+
' 2 a cited path did not resolve',
|
|
62
|
+
'',
|
|
63
|
+
'Only unresolved citations set a failing exit code. Length, depth,',
|
|
64
|
+
'table, and index findings are advisory.',
|
|
65
|
+
'',
|
|
66
|
+
'Examples:',
|
|
67
|
+
' aitk context audit',
|
|
68
|
+
' aitk context audit --json',
|
|
69
|
+
' aitk context audit --citations-only',
|
|
70
|
+
' aitk context audit --folder context,diagrams',
|
|
71
|
+
'',
|
|
72
|
+
].join('\n'),
|
|
73
|
+
)
|
|
74
|
+
.action(async (path: string | undefined, opts: AuditCommandOptions) => {
|
|
75
|
+
process.exitCode = await runAudit(path, opts)
|
|
76
|
+
})
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
function parseFolders(list: string | undefined): string[] | string {
|
|
80
|
+
if (!list) return [...DEFAULT_FOLDERS]
|
|
81
|
+
|
|
82
|
+
const names = list
|
|
83
|
+
.split(',')
|
|
84
|
+
.map((name) => name.trim())
|
|
85
|
+
.filter(Boolean)
|
|
86
|
+
|
|
87
|
+
if (names.length === 0) return 'Empty --folder list. Pass at least one name.'
|
|
88
|
+
|
|
89
|
+
// `..` would resolve the audit root above `.claude/`, where `presentNames`
|
|
90
|
+
// has no folder name to slice out and hands the citation pattern undefined.
|
|
91
|
+
const invalid = names.filter((name) => !FOLDER_NAME.test(name))
|
|
92
|
+
if (invalid.length > 0) {
|
|
93
|
+
return `--folder takes folder names under .claude/, not paths: ${invalid.join(', ')}`
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
return names
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
async function runAudit(
|
|
100
|
+
path: string | undefined,
|
|
101
|
+
opts: AuditCommandOptions,
|
|
102
|
+
): Promise<number> {
|
|
103
|
+
const root = resolve(path ?? process.cwd())
|
|
104
|
+
const names = parseFolders(opts.folder)
|
|
105
|
+
const gateOnly = opts.citationsOnly ?? false
|
|
106
|
+
|
|
107
|
+
if (typeof names === 'string') return refuse(names, gateOnly)
|
|
108
|
+
|
|
109
|
+
const folders = await resolveFolders(root, names)
|
|
110
|
+
if (folders.length === 0) {
|
|
111
|
+
return refuse(
|
|
112
|
+
`No audited folder found under .claude/. Looked for: ${names.join(', ')}.`,
|
|
113
|
+
gateOnly,
|
|
114
|
+
)
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
const citations = await auditCitations(root, presentNames(folders))
|
|
118
|
+
if (citations.kind === 'unavailable') {
|
|
119
|
+
return refuse(
|
|
120
|
+
'git could not list the tree, so no citation was checked. Run inside a git repository.',
|
|
121
|
+
gateOnly,
|
|
122
|
+
)
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
const entries = gateOnly ? [] : await measureFolders(root, folders)
|
|
126
|
+
const drift = gateOnly ? [] : await auditIndexes(folders)
|
|
127
|
+
|
|
128
|
+
if (gateOnly) {
|
|
129
|
+
reportGate(citations)
|
|
130
|
+
} else {
|
|
131
|
+
intro('aitk context audit')
|
|
132
|
+
reportScope(folders)
|
|
133
|
+
reportCitations(citations)
|
|
134
|
+
reportLength(entries)
|
|
135
|
+
reportDepth(entries)
|
|
136
|
+
reportTables(entries)
|
|
137
|
+
reportDrift(drift)
|
|
138
|
+
outro()
|
|
139
|
+
}
|
|
140
|
+
|
|
141
|
+
if (opts.json) {
|
|
142
|
+
process.stdout.write(
|
|
143
|
+
`${JSON.stringify({
|
|
144
|
+
root,
|
|
145
|
+
folders: folders.map((folder) => ({
|
|
146
|
+
path: folder.rel,
|
|
147
|
+
entries: folder.entries.length,
|
|
148
|
+
})),
|
|
149
|
+
citations: {
|
|
150
|
+
scanned: citations.scanned,
|
|
151
|
+
total: citations.total,
|
|
152
|
+
unresolved: citations.unresolved,
|
|
153
|
+
},
|
|
154
|
+
entries,
|
|
155
|
+
indexDrift: drift,
|
|
156
|
+
checkpoints: {
|
|
157
|
+
lines: LENGTH_CHECKPOINT,
|
|
158
|
+
run: RUN_CHECKPOINT,
|
|
159
|
+
runCountsBlankLines: true,
|
|
160
|
+
},
|
|
161
|
+
})}\n`,
|
|
162
|
+
)
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
return citations.unresolved.length > 0 ? EXIT_UNRESOLVED : 0
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
function refuse(message: string, gateOnly: boolean): number {
|
|
169
|
+
if (gateOnly) {
|
|
170
|
+
frameError(message)
|
|
171
|
+
return 1
|
|
172
|
+
}
|
|
173
|
+
|
|
174
|
+
intro('aitk context audit')
|
|
175
|
+
logStep('Refused')
|
|
176
|
+
logWarn(message)
|
|
177
|
+
outro()
|
|
178
|
+
return 1
|
|
179
|
+
}
|
|
180
|
+
|
|
181
|
+
/**
|
|
182
|
+
* Prints nothing when every path resolves.
|
|
183
|
+
*
|
|
184
|
+
* `--citations-only` is what `verify.sh` runs on every push, and that script
|
|
185
|
+
* pipes a stage's whole output into its own frame. A passing gate that printed
|
|
186
|
+
* its frame would nest one inside the other on every contributor's push.
|
|
187
|
+
*/
|
|
188
|
+
function reportGate(report: ScannedCitations): void {
|
|
189
|
+
const count = report.unresolved.length
|
|
190
|
+
if (count === 0) return
|
|
191
|
+
|
|
192
|
+
intro('aitk context audit')
|
|
193
|
+
logError(
|
|
194
|
+
count === 1
|
|
195
|
+
? '1 cited path does not resolve'
|
|
196
|
+
: `${count} cited paths do not resolve`,
|
|
197
|
+
)
|
|
198
|
+
pipeOutput(
|
|
199
|
+
report.unresolved
|
|
200
|
+
.map((citation) => `${citation.file}:${citation.line} ${citation.path}`)
|
|
201
|
+
.join('\n'),
|
|
202
|
+
)
|
|
203
|
+
outro()
|
|
204
|
+
}
|
|
205
|
+
|
|
206
|
+
function plural(count: number, noun: string): string {
|
|
207
|
+
return `${count} ${noun}${count === 1 ? '' : 's'}`
|
|
208
|
+
}
|
|
209
|
+
|
|
210
|
+
function reportScope(folders: readonly AuditedFolder[]): void {
|
|
211
|
+
logStep('Scope')
|
|
212
|
+
|
|
213
|
+
for (const folder of folders) {
|
|
214
|
+
logInfo(`${folder.rel}: ${folder.entries.length} entries`)
|
|
215
|
+
}
|
|
216
|
+
}
|
|
217
|
+
|
|
218
|
+
type ScannedCitations = Extract<CitationReport, { kind: 'scanned' }>
|
|
219
|
+
|
|
220
|
+
function reportCitations(report: ScannedCitations): void {
|
|
221
|
+
logStep('Citations')
|
|
222
|
+
logInfo(
|
|
223
|
+
`${plural(report.total, 'cited path')} across ${plural(report.scanned, 'file')}, fixtures and fenced examples excluded`,
|
|
224
|
+
)
|
|
225
|
+
|
|
226
|
+
if (report.unresolved.length === 0) {
|
|
227
|
+
logInfo('Every cited path resolves.')
|
|
228
|
+
return
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
logWarn(`${report.unresolved.length} unresolved`)
|
|
232
|
+
pipeOutput(
|
|
233
|
+
report.unresolved
|
|
234
|
+
.map((citation) => `${citation.file}:${citation.line} ${citation.path}`)
|
|
235
|
+
.join('\n'),
|
|
236
|
+
)
|
|
237
|
+
}
|
|
238
|
+
|
|
239
|
+
function reportLength(entries: readonly EntryReport[]): void {
|
|
240
|
+
logStep('Length')
|
|
241
|
+
|
|
242
|
+
const over = entries
|
|
243
|
+
.filter((entry) => entry.lines > LENGTH_CHECKPOINT)
|
|
244
|
+
.sort((a, b) => b.lines - a.lines)
|
|
245
|
+
|
|
246
|
+
if (over.length === 0) {
|
|
247
|
+
logInfo(`No entry past the ${LENGTH_CHECKPOINT}-line checkpoint.`)
|
|
248
|
+
return
|
|
249
|
+
}
|
|
250
|
+
|
|
251
|
+
logWarn(`${over.length} past the ${LENGTH_CHECKPOINT}-line checkpoint`)
|
|
252
|
+
pipeOutput(
|
|
253
|
+
over.map((entry) => `${entry.rel} ${entry.lines} lines`).join('\n'),
|
|
254
|
+
)
|
|
255
|
+
}
|
|
256
|
+
|
|
257
|
+
/**
|
|
258
|
+
* Names the blank-line convention on every run.
|
|
259
|
+
*
|
|
260
|
+
* The standard settles heading level and fenced blocks and stops there, so a
|
|
261
|
+
* hand reader who drops blank lines lands a line or two below this number.
|
|
262
|
+
* Stating it is what keeps the two measurements reconcilable.
|
|
263
|
+
*/
|
|
264
|
+
function reportDepth(entries: readonly EntryReport[]): void {
|
|
265
|
+
logStep('Depth')
|
|
266
|
+
logInfo('Runs count blank lines. Fenced blocks and peer lists are excluded.')
|
|
267
|
+
|
|
268
|
+
const over = entries
|
|
269
|
+
.filter((entry) => entry.longestRun > RUN_CHECKPOINT)
|
|
270
|
+
.sort((a, b) => b.longestRun - a.longestRun)
|
|
271
|
+
|
|
272
|
+
if (over.length === 0) {
|
|
273
|
+
logInfo(`No run past the ${RUN_CHECKPOINT}-line checkpoint.`)
|
|
274
|
+
return
|
|
275
|
+
}
|
|
276
|
+
|
|
277
|
+
logWarn(`${over.length} past the ${RUN_CHECKPOINT}-line checkpoint`)
|
|
278
|
+
pipeOutput(
|
|
279
|
+
over
|
|
280
|
+
.map(
|
|
281
|
+
(entry) =>
|
|
282
|
+
`${entry.rel}:${entry.longestRunLine} ${entry.longestRun} lines unbroken`,
|
|
283
|
+
)
|
|
284
|
+
.join('\n'),
|
|
285
|
+
)
|
|
286
|
+
}
|
|
287
|
+
|
|
288
|
+
function reportTables(entries: readonly EntryReport[]): void {
|
|
289
|
+
logStep('Tables')
|
|
290
|
+
|
|
291
|
+
const candidates = entries.flatMap((entry) =>
|
|
292
|
+
entry.catalogTables.map(
|
|
293
|
+
(table) => `${entry.rel}:${table.line} ${table.rows} rows`,
|
|
294
|
+
),
|
|
295
|
+
)
|
|
296
|
+
|
|
297
|
+
if (candidates.length === 0) {
|
|
298
|
+
logInfo('No table reads as a catalog that grows a row per shipped thing.')
|
|
299
|
+
return
|
|
300
|
+
}
|
|
301
|
+
|
|
302
|
+
logWarn(`${plural(candidates.length, 'candidate')} for a bullet list`)
|
|
303
|
+
pipeOutput(candidates.join('\n'))
|
|
304
|
+
}
|
|
305
|
+
|
|
306
|
+
function reportDrift(drift: readonly FolderDrift[]): void {
|
|
307
|
+
logStep('Index drift')
|
|
308
|
+
|
|
309
|
+
const lines = drift.flatMap((folder) => [
|
|
310
|
+
...folder.unlisted.map((name) => `${folder.rel} unlisted: ${name}`),
|
|
311
|
+
...folder.missing.map((name) => `${folder.rel} missing: ${name}`),
|
|
312
|
+
])
|
|
313
|
+
|
|
314
|
+
if (lines.length === 0) {
|
|
315
|
+
logInfo('Every index agrees with its siblings.')
|
|
316
|
+
return
|
|
317
|
+
}
|
|
318
|
+
|
|
319
|
+
logWarn(plural(lines.length, 'disagreement'))
|
|
320
|
+
pipeOutput(lines.join('\n'))
|
|
321
|
+
}
|
package/src/commands/sandbox.ts
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import { existsSync, readFileSync, readdirSync } from 'node:fs'
|
|
2
|
+
import { homedir } from 'node:os'
|
|
2
3
|
import { join } from 'node:path'
|
|
3
4
|
import type { Command } from 'commander'
|
|
4
5
|
import { PROJECT_ROOT, execScript } from '@/exec'
|
|
@@ -28,6 +29,29 @@ import {
|
|
|
28
29
|
|
|
29
30
|
const SANDBOX_DIR = join(PROJECT_ROOT, 'scripts', 'sandbox')
|
|
30
31
|
|
|
32
|
+
/**
|
|
33
|
+
* The provisioned tree, as opposed to `SANDBOX_DIR` above, which holds the
|
|
34
|
+
* scenario scripts. It sits outside the toolkit worktree so the toolkit's own
|
|
35
|
+
* `CLAUDE.md` stays off the ancestor chain of the session `run.sh` spawns with
|
|
36
|
+
* cwd here.
|
|
37
|
+
*
|
|
38
|
+
* Twin of `resolve_sandbox_dir` in `scripts/lib/sandbox-path.sh`. The exec
|
|
39
|
+
* boundary rules out a shared constant, so a change to the default lands on both
|
|
40
|
+
* sides.
|
|
41
|
+
*/
|
|
42
|
+
function sandboxTree(): string {
|
|
43
|
+
const override = process.env.AITK_SANDBOX_DIR
|
|
44
|
+
if (override !== undefined && override !== '') return override
|
|
45
|
+
|
|
46
|
+
const state = process.env.XDG_STATE_HOME
|
|
47
|
+
const base =
|
|
48
|
+
state !== undefined && state !== ''
|
|
49
|
+
? state
|
|
50
|
+
: join(homedir(), '.local', 'state')
|
|
51
|
+
|
|
52
|
+
return join(base, 'aitk', 'sandbox')
|
|
53
|
+
}
|
|
54
|
+
|
|
31
55
|
/**
|
|
32
56
|
* Holds fixture content for scenarios rather than scenarios of its own.
|
|
33
57
|
* Twin of the `-not -name fixtures` filter in `scripts/manage-sandbox.sh`.
|
|
@@ -224,7 +248,7 @@ function runCheck(
|
|
|
224
248
|
// A sandbox that was never provisioned fails every path assertion, reading as a
|
|
225
249
|
// skill that did nothing rather than a caller that ran check too early. The
|
|
226
250
|
// whole point of a verdict is that it means what it says.
|
|
227
|
-
const sandboxDir =
|
|
251
|
+
const sandboxDir = sandboxTree()
|
|
228
252
|
if (!existsSync(sandboxDir)) {
|
|
229
253
|
logError(`No sandbox at ${sandboxDir}. Provision one with aitk sandbox.`)
|
|
230
254
|
outro()
|
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
import { readFile } from 'node:fs/promises'
|
|
2
|
+
import { relative } from 'node:path'
|
|
3
|
+
import type { AuditedFolder } from '@/context/folders'
|
|
4
|
+
|
|
5
|
+
/** Checkpoints quoted from `standards/context.md`. Neither is a cap. */
|
|
6
|
+
export const LENGTH_CHECKPOINT = 150
|
|
7
|
+
export const RUN_CHECKPOINT = 40
|
|
8
|
+
|
|
9
|
+
/**
|
|
10
|
+
* A table this size or larger whose first column mostly names artifacts reads
|
|
11
|
+
* as a catalog that grows a row per shipped thing, which is the shape the
|
|
12
|
+
* standard routes to a bullet list. Below it, a table is small enough that a
|
|
13
|
+
* reflow rewrites little.
|
|
14
|
+
*/
|
|
15
|
+
export const CATALOG_ROW_CHECKPOINT = 6
|
|
16
|
+
|
|
17
|
+
/** Share of first cells that must name an artifact for a table to qualify. */
|
|
18
|
+
const CATALOG_NAMED_RATIO = 0.6
|
|
19
|
+
|
|
20
|
+
const FRONTMATTER = /^---\n[\s\S]*?\n---\n?/
|
|
21
|
+
const FENCE = /^\s*(```|~~~)/
|
|
22
|
+
const HEADING = /^#{1,6}\s/
|
|
23
|
+
const LIST_ITEM = /^(\s*)([-*+]|\d+\.)\s+/
|
|
24
|
+
const TABLE_ROW = /^\s*\|/
|
|
25
|
+
const TABLE_SEPARATOR = /^\s*\|[\s:|-]+\|\s*$/
|
|
26
|
+
const NAMED_CELL = /`[^`]+`|\[[^\]]+\]\([^)]+\)/
|
|
27
|
+
|
|
28
|
+
export interface TableFinding {
|
|
29
|
+
readonly line: number
|
|
30
|
+
readonly rows: number
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
export interface EntryReport {
|
|
34
|
+
readonly rel: string
|
|
35
|
+
readonly lines: number
|
|
36
|
+
readonly longestRun: number
|
|
37
|
+
/** First line of the longest run, or 0 when the entry has no run at all. */
|
|
38
|
+
readonly longestRunLine: number
|
|
39
|
+
readonly catalogTables: readonly TableFinding[]
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
interface BodyLine {
|
|
43
|
+
readonly number: number
|
|
44
|
+
readonly text: string
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
/**
|
|
48
|
+
* Drops the frontmatter while keeping every surviving line's original number,
|
|
49
|
+
* so a finding points at the line an editor opens rather than at an offset into
|
|
50
|
+
* the body.
|
|
51
|
+
*/
|
|
52
|
+
function bodyLines(source: string): BodyLine[] {
|
|
53
|
+
const match = source.match(FRONTMATTER)
|
|
54
|
+
const offset = match ? match[0].split('\n').length - 1 : 0
|
|
55
|
+
|
|
56
|
+
return source
|
|
57
|
+
.slice(match ? match[0].length : 0)
|
|
58
|
+
.replace(/\n$/, '')
|
|
59
|
+
.split('\n')
|
|
60
|
+
.map((text, index) => ({ number: offset + index + 1, text }))
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
/**
|
|
64
|
+
* Reports whether a run is the peer list the standard exempts.
|
|
65
|
+
*
|
|
66
|
+
* Every non-blank line has to be a list item at one indent. Prose mixed into
|
|
67
|
+
* the run or a nested level inside it ends the exemption, because either one
|
|
68
|
+
* means the block is no longer a flat set a reader can skim.
|
|
69
|
+
*/
|
|
70
|
+
function isPeerList(run: readonly BodyLine[]): boolean {
|
|
71
|
+
const indents = new Set<number>()
|
|
72
|
+
|
|
73
|
+
for (const line of run) {
|
|
74
|
+
if (line.text.trim() === '') continue
|
|
75
|
+
|
|
76
|
+
const match = line.text.match(LIST_ITEM)
|
|
77
|
+
if (!match) return false
|
|
78
|
+
indents.add(match[1].length)
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
return indents.size === 1
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
/**
|
|
85
|
+
* Measures the longest run of lines no heading breaks.
|
|
86
|
+
*
|
|
87
|
+
* Fenced blocks are skipped rather than treated as breaks, per the standard:
|
|
88
|
+
* they leave the count without ending the run, so prose either side of an
|
|
89
|
+
* example still measures as the one stretch a reader scrolls through. Blank
|
|
90
|
+
* lines do count, since the checkpoint is about how far a reader travels
|
|
91
|
+
* between signposts and a blank line is distance like any other. A hand reader
|
|
92
|
+
* measuring without them lands one or two lines lower, which the report legend
|
|
93
|
+
* states.
|
|
94
|
+
*/
|
|
95
|
+
function longestRun(lines: readonly BodyLine[]): {
|
|
96
|
+
length: number
|
|
97
|
+
line: number
|
|
98
|
+
} {
|
|
99
|
+
let longest = 0
|
|
100
|
+
let longestLine = 0
|
|
101
|
+
let run: BodyLine[] = []
|
|
102
|
+
let fenced = false
|
|
103
|
+
|
|
104
|
+
const close = (): void => {
|
|
105
|
+
// The reported line is the run's first non-blank one, since that is what an
|
|
106
|
+
// editor should open. A run of nothing but blank lines is the gap between
|
|
107
|
+
// two headings rather than a stretch a reader travels, so it never counts.
|
|
108
|
+
const first = run.find((line) => line.text.trim() !== '')
|
|
109
|
+
|
|
110
|
+
if (first && run.length > longest && !isPeerList(run)) {
|
|
111
|
+
longest = run.length
|
|
112
|
+
longestLine = first.number
|
|
113
|
+
}
|
|
114
|
+
run = []
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
for (const line of lines) {
|
|
118
|
+
if (FENCE.test(line.text)) {
|
|
119
|
+
fenced = !fenced
|
|
120
|
+
continue
|
|
121
|
+
}
|
|
122
|
+
if (fenced) continue
|
|
123
|
+
|
|
124
|
+
if (HEADING.test(line.text)) {
|
|
125
|
+
close()
|
|
126
|
+
continue
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
run.push(line)
|
|
130
|
+
}
|
|
131
|
+
|
|
132
|
+
close()
|
|
133
|
+
|
|
134
|
+
return { length: longest, line: longestLine }
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
function firstCell(row: string): string {
|
|
138
|
+
return row.split('|').slice(1)[0] ?? ''
|
|
139
|
+
}
|
|
140
|
+
|
|
141
|
+
/**
|
|
142
|
+
* Finds the tables whose rows name shipped artifacts.
|
|
143
|
+
*
|
|
144
|
+
* A bare table count reports mostly fixed comparison tables, where a reflow
|
|
145
|
+
* costs nothing because no row is ever added. The reflow problem belongs to a
|
|
146
|
+
* catalog that gains a row per artifact, and a first column carrying a path,
|
|
147
|
+
* command, or link is what separates the two without reading the prose.
|
|
148
|
+
*/
|
|
149
|
+
function catalogTables(lines: readonly BodyLine[]): TableFinding[] {
|
|
150
|
+
const findings: TableFinding[] = []
|
|
151
|
+
let fenced = false
|
|
152
|
+
let index = 0
|
|
153
|
+
|
|
154
|
+
while (index < lines.length) {
|
|
155
|
+
const line = lines[index]
|
|
156
|
+
|
|
157
|
+
if (FENCE.test(line.text)) {
|
|
158
|
+
fenced = !fenced
|
|
159
|
+
index++
|
|
160
|
+
continue
|
|
161
|
+
}
|
|
162
|
+
if (fenced || !TABLE_ROW.test(line.text)) {
|
|
163
|
+
index++
|
|
164
|
+
continue
|
|
165
|
+
}
|
|
166
|
+
|
|
167
|
+
const separator = lines[index + 1]
|
|
168
|
+
if (!separator || !TABLE_SEPARATOR.test(separator.text)) {
|
|
169
|
+
index++
|
|
170
|
+
continue
|
|
171
|
+
}
|
|
172
|
+
|
|
173
|
+
const body: BodyLine[] = []
|
|
174
|
+
let cursor = index + 2
|
|
175
|
+
while (cursor < lines.length && TABLE_ROW.test(lines[cursor].text)) {
|
|
176
|
+
body.push(lines[cursor])
|
|
177
|
+
cursor++
|
|
178
|
+
}
|
|
179
|
+
|
|
180
|
+
const named = body.filter((row) => NAMED_CELL.test(firstCell(row.text)))
|
|
181
|
+
if (
|
|
182
|
+
body.length >= CATALOG_ROW_CHECKPOINT &&
|
|
183
|
+
named.length / body.length >= CATALOG_NAMED_RATIO
|
|
184
|
+
) {
|
|
185
|
+
findings.push({ line: line.number, rows: body.length })
|
|
186
|
+
}
|
|
187
|
+
|
|
188
|
+
index = cursor
|
|
189
|
+
}
|
|
190
|
+
|
|
191
|
+
return findings
|
|
192
|
+
}
|
|
193
|
+
|
|
194
|
+
export function measureEntry(rel: string, source: string): EntryReport {
|
|
195
|
+
const lines = bodyLines(source)
|
|
196
|
+
const run = longestRun(lines)
|
|
197
|
+
|
|
198
|
+
return {
|
|
199
|
+
rel,
|
|
200
|
+
lines: source.replace(/\n$/, '').split('\n').length,
|
|
201
|
+
longestRun: run.length,
|
|
202
|
+
longestRunLine: run.line,
|
|
203
|
+
catalogTables: catalogTables(lines),
|
|
204
|
+
}
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
/**
|
|
208
|
+
* Measures every entry in the audited folders. A generated `index.md` is not
|
|
209
|
+
* among them, since its body is rewritten on every regen and no checkpoint
|
|
210
|
+
* describes a catalog.
|
|
211
|
+
*/
|
|
212
|
+
export async function measureFolders(
|
|
213
|
+
root: string,
|
|
214
|
+
folders: readonly AuditedFolder[],
|
|
215
|
+
): Promise<EntryReport[]> {
|
|
216
|
+
const reports: EntryReport[] = []
|
|
217
|
+
|
|
218
|
+
for (const folder of folders) {
|
|
219
|
+
for (const path of folder.entries) {
|
|
220
|
+
reports.push(
|
|
221
|
+
measureEntry(relative(root, path), await readFile(path, 'utf8')),
|
|
222
|
+
)
|
|
223
|
+
}
|
|
224
|
+
}
|
|
225
|
+
|
|
226
|
+
return reports
|
|
227
|
+
}
|
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
import { existsSync } from 'node:fs'
|
|
2
|
+
import { readFile } from 'node:fs/promises'
|
|
3
|
+
import { resolve } from 'node:path'
|
|
4
|
+
import { $ } from 'bun'
|
|
5
|
+
import { gitEnv } from '@/git-env'
|
|
6
|
+
|
|
7
|
+
/**
|
|
8
|
+
* Suppresses citation checking for the source line carrying it.
|
|
9
|
+
*
|
|
10
|
+
* Prose that teaches a naming pattern displays a path rather than pointing at
|
|
11
|
+
* one, and no syntax separates the two: an illustration and a reference are
|
|
12
|
+
* both inline code in a sentence. Location covers the fixture trees and fenced
|
|
13
|
+
* examples, and this covers what is left, which is a sentence naming a
|
|
14
|
+
* hypothetical entry to show the shape of the name.
|
|
15
|
+
*/
|
|
16
|
+
export const IGNORE_MARKER = 'audit-ignore-citations'
|
|
17
|
+
|
|
18
|
+
/**
|
|
19
|
+
* Trees holding content authored to be parsed rather than followed.
|
|
20
|
+
*
|
|
21
|
+
* Sandbox scenarios describe paths inside their own scratch fixtures and the
|
|
22
|
+
* eval harness names paths in its target project. Neither is a reference into
|
|
23
|
+
* this repository, so an unresolved path there is correct rather than stale.
|
|
24
|
+
*/
|
|
25
|
+
const FIXTURE_TREES: readonly string[] = ['scripts/sandbox/', 'scripts/eval/']
|
|
26
|
+
|
|
27
|
+
const FIXTURE_SEGMENTS: readonly string[] = ['fixtures', '__fixtures__']
|
|
28
|
+
|
|
29
|
+
const FENCE = /^\s*(```|~~~)/
|
|
30
|
+
|
|
31
|
+
export interface Citation {
|
|
32
|
+
readonly file: string
|
|
33
|
+
readonly line: number
|
|
34
|
+
readonly path: string
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
export function isFixture(rel: string): boolean {
|
|
38
|
+
if (rel.endsWith('.test.ts')) return true
|
|
39
|
+
if (FIXTURE_TREES.some((tree) => rel.startsWith(tree))) return true
|
|
40
|
+
return rel.split('/').some((segment) => FIXTURE_SEGMENTS.includes(segment))
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
export function citationPattern(folders: readonly string[]): RegExp {
|
|
44
|
+
const names = folders.map((name) =>
|
|
45
|
+
name.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'),
|
|
46
|
+
)
|
|
47
|
+
return new RegExp(
|
|
48
|
+
`\\.claude/(?:${names.join('|')})/[A-Za-z0-9._/-]+\\.md`,
|
|
49
|
+
'g',
|
|
50
|
+
)
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
/**
|
|
54
|
+
* Pulls the cited paths out of one file's text.
|
|
55
|
+
*
|
|
56
|
+
* Fenced blocks are skipped for markdown only. A fence is markdown syntax, and
|
|
57
|
+
* applying it to a shell or TypeScript source would let a heredoc of triple
|
|
58
|
+
* backticks silently hide the rest of the file.
|
|
59
|
+
*/
|
|
60
|
+
export function collectCitations(
|
|
61
|
+
rel: string,
|
|
62
|
+
text: string,
|
|
63
|
+
pattern: RegExp,
|
|
64
|
+
): Citation[] {
|
|
65
|
+
const isMarkdown = rel.endsWith('.md')
|
|
66
|
+
const found: Citation[] = []
|
|
67
|
+
let fenced = false
|
|
68
|
+
|
|
69
|
+
for (const [index, line] of text.split('\n').entries()) {
|
|
70
|
+
if (isMarkdown && FENCE.test(line)) {
|
|
71
|
+
fenced = !fenced
|
|
72
|
+
continue
|
|
73
|
+
}
|
|
74
|
+
if (fenced || line.includes(IGNORE_MARKER)) continue
|
|
75
|
+
|
|
76
|
+
for (const match of line.matchAll(pattern)) {
|
|
77
|
+
found.push({ file: rel, line: index + 1, path: match[0] })
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
return found
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
/**
|
|
85
|
+
* Lists the files a citation can live in: tracked, plus untracked files git
|
|
86
|
+
* does not ignore. The untracked half is what keeps a new entry's references
|
|
87
|
+
* checked on the branch that adds it rather than one push later.
|
|
88
|
+
*
|
|
89
|
+
* Returns undefined when git cannot answer, which the caller reports rather
|
|
90
|
+
* than smoothing into an empty list. An empty list resolves every one of its
|
|
91
|
+
* zero citations, so a degraded git would otherwise turn the push gate into an
|
|
92
|
+
* unconditional pass with output indistinguishable from a clean tree.
|
|
93
|
+
*/
|
|
94
|
+
async function listFiles(root: string): Promise<string[] | undefined> {
|
|
95
|
+
const run = async (args: string[]): Promise<string[] | undefined> => {
|
|
96
|
+
const result = await $`git -C ${root} ${args}`
|
|
97
|
+
.env(gitEnv())
|
|
98
|
+
.quiet()
|
|
99
|
+
.nothrow()
|
|
100
|
+
if (result.exitCode !== 0) return undefined
|
|
101
|
+
return result.text().split('\n').filter(Boolean)
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
const tracked = await run(['ls-files'])
|
|
105
|
+
const untracked = await run(['ls-files', '--others', '--exclude-standard'])
|
|
106
|
+
if (!tracked || !untracked) return undefined
|
|
107
|
+
|
|
108
|
+
return [...new Set([...tracked, ...untracked])].sort()
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
/**
|
|
112
|
+
* `unavailable` is a distinct state from a clean scan.
|
|
113
|
+
*
|
|
114
|
+
* Finding nothing and being unable to look mean opposite things, and only one
|
|
115
|
+
* of them should let a push through.
|
|
116
|
+
*/
|
|
117
|
+
export type CitationReport =
|
|
118
|
+
| {
|
|
119
|
+
readonly kind: 'scanned'
|
|
120
|
+
readonly scanned: number
|
|
121
|
+
readonly total: number
|
|
122
|
+
readonly unresolved: readonly Citation[]
|
|
123
|
+
}
|
|
124
|
+
| { readonly kind: 'unavailable' }
|
|
125
|
+
|
|
126
|
+
/**
|
|
127
|
+
* Resolves every cited path outside the fixture trees.
|
|
128
|
+
*
|
|
129
|
+
* This is the only check that gates the repository check, which makes it the
|
|
130
|
+
* only one whose false positives cost a contributor anything. It reports a
|
|
131
|
+
* finding solely for a path that does not resolve on disk, and the exclusions
|
|
132
|
+
* above are what keep that from firing on prose about paths.
|
|
133
|
+
*/
|
|
134
|
+
export async function auditCitations(
|
|
135
|
+
root: string,
|
|
136
|
+
folders: readonly string[],
|
|
137
|
+
): Promise<CitationReport> {
|
|
138
|
+
const pattern = citationPattern(folders)
|
|
139
|
+
const listed = await listFiles(root)
|
|
140
|
+
if (!listed) return { kind: 'unavailable' }
|
|
141
|
+
|
|
142
|
+
const candidates = listed.filter((rel) => !isFixture(rel))
|
|
143
|
+
const citations: Citation[] = []
|
|
144
|
+
|
|
145
|
+
for (const rel of candidates) {
|
|
146
|
+
const path = resolve(root, rel)
|
|
147
|
+
if (!existsSync(path)) continue
|
|
148
|
+
|
|
149
|
+
let text: string
|
|
150
|
+
try {
|
|
151
|
+
text = await readFile(path, 'utf8')
|
|
152
|
+
} catch {
|
|
153
|
+
continue
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
citations.push(...collectCitations(rel, text, pattern))
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
return {
|
|
160
|
+
kind: 'scanned',
|
|
161
|
+
scanned: candidates.length,
|
|
162
|
+
total: citations.length,
|
|
163
|
+
unresolved: citations.filter(
|
|
164
|
+
(citation) => !existsSync(resolve(root, citation.path)),
|
|
165
|
+
),
|
|
166
|
+
}
|
|
167
|
+
}
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
import { existsSync } from 'node:fs'
|
|
2
|
+
import { dirname, relative, resolve } from 'node:path'
|
|
3
|
+
import { INDEX_FILE, listIndexes } from '@/indexes/walk'
|
|
4
|
+
|
|
5
|
+
/**
|
|
6
|
+
* Folder names under `.claude/` audited by default.
|
|
7
|
+
*
|
|
8
|
+
* The index-plus-entry contract on its own over-collects. `.claude/standards/`
|
|
9
|
+
* also carries a generated index beside entries with frontmatter, and auditing
|
|
10
|
+
* it would measure the consumed copy of `standards/` against a rule written for
|
|
11
|
+
* per-domain narrative. This list names the folders `standards/context.md`
|
|
12
|
+
* governs, and `--folder` admits another without an edit here.
|
|
13
|
+
*/
|
|
14
|
+
export const DEFAULT_FOLDERS: readonly string[] = [
|
|
15
|
+
'context',
|
|
16
|
+
'diagrams',
|
|
17
|
+
'wireframes',
|
|
18
|
+
]
|
|
19
|
+
|
|
20
|
+
export interface AuditedFolder {
|
|
21
|
+
/** Repo-relative folder path, used verbatim in every report line. */
|
|
22
|
+
readonly rel: string
|
|
23
|
+
readonly indexPath: string
|
|
24
|
+
/** Absolute paths of the folder's own entries, excluding its `index.md`. */
|
|
25
|
+
readonly entries: readonly string[]
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
/**
|
|
29
|
+
* Names the requested folders that actually exist, which is the citation
|
|
30
|
+
* check's scope.
|
|
31
|
+
*
|
|
32
|
+
* A skill or seed pointing into `.claude/wireframes/` is a live instruction for
|
|
33
|
+
* a project that carries the folder and says nothing about one that does not.
|
|
34
|
+
* Checking a path into an absent folder would fail eight shipped references
|
|
35
|
+
* here for the sole reason that this repository has no wireframes.
|
|
36
|
+
*/
|
|
37
|
+
export function presentNames(folders: readonly AuditedFolder[]): string[] {
|
|
38
|
+
return [...new Set(folders.map((folder) => folder.rel.split('/')[1]))]
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
async function readEntries(dir: string): Promise<string[]> {
|
|
42
|
+
const paths: string[] = []
|
|
43
|
+
|
|
44
|
+
for await (const name of new Bun.Glob('*.md').scan({
|
|
45
|
+
cwd: dir,
|
|
46
|
+
onlyFiles: true,
|
|
47
|
+
dot: true,
|
|
48
|
+
})) {
|
|
49
|
+
if (name === INDEX_FILE) continue
|
|
50
|
+
paths.push(resolve(dir, name))
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
return paths.sort()
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
/**
|
|
57
|
+
* Resolves the folders to audit under `root`.
|
|
58
|
+
*
|
|
59
|
+
* A named folder contributes its own entries plus those of every nested index
|
|
60
|
+
* folder beneath it, so a domain that outgrew one file and split is audited at
|
|
61
|
+
* the same grain as one that did not. Discovery of the nested folders runs
|
|
62
|
+
* through the shared walker, which is what keeps `.gitignore` and the vendored
|
|
63
|
+
* prune governing this scan as well as index regeneration.
|
|
64
|
+
*
|
|
65
|
+
* A requested folder that does not exist is dropped rather than reported. The
|
|
66
|
+
* default list names three folders and a project carrying one of them is the
|
|
67
|
+
* ordinary case.
|
|
68
|
+
*/
|
|
69
|
+
export async function resolveFolders(
|
|
70
|
+
root: string,
|
|
71
|
+
names: readonly string[] = DEFAULT_FOLDERS,
|
|
72
|
+
): Promise<AuditedFolder[]> {
|
|
73
|
+
const folders: AuditedFolder[] = []
|
|
74
|
+
|
|
75
|
+
for (const name of names) {
|
|
76
|
+
const dir = resolve(root, '.claude', name)
|
|
77
|
+
if (!existsSync(`${dir}/${INDEX_FILE}`)) continue
|
|
78
|
+
|
|
79
|
+
const dirs = [dir, ...(await listIndexes(dir)).map(dirname)]
|
|
80
|
+
|
|
81
|
+
for (const each of [...new Set(dirs)].sort()) {
|
|
82
|
+
folders.push({
|
|
83
|
+
rel: relative(root, each),
|
|
84
|
+
indexPath: `${each}/${INDEX_FILE}`,
|
|
85
|
+
entries: await readEntries(each),
|
|
86
|
+
})
|
|
87
|
+
}
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
return folders
|
|
91
|
+
}
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
import { existsSync } from 'node:fs'
|
|
2
|
+
import { readFile } from 'node:fs/promises'
|
|
3
|
+
import { basename, dirname, resolve } from 'node:path'
|
|
4
|
+
import type { AuditedFolder } from '@/context/folders'
|
|
5
|
+
|
|
6
|
+
const ENTRY_LINK = /^-\s+\[[^\]]*\]\(([^)]+)\)/
|
|
7
|
+
|
|
8
|
+
export interface FolderDrift {
|
|
9
|
+
readonly rel: string
|
|
10
|
+
/** Entry files present in the folder that the index does not link. */
|
|
11
|
+
readonly unlisted: readonly string[]
|
|
12
|
+
/** Names the index links that resolve to nothing on disk. */
|
|
13
|
+
readonly missing: readonly string[]
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
/** Collects the link targets of an index's catalog bullets. */
|
|
17
|
+
export function listedTargets(source: string): string[] {
|
|
18
|
+
const targets: string[] = []
|
|
19
|
+
|
|
20
|
+
for (const line of source.split('\n')) {
|
|
21
|
+
const match = line.match(ENTRY_LINK)
|
|
22
|
+
if (match) targets.push(match[1])
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
return targets
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
/**
|
|
29
|
+
* Compares one folder's index against its siblings in both directions.
|
|
30
|
+
*
|
|
31
|
+
* Both halves are reported because they fail differently. An unlisted entry is
|
|
32
|
+
* invisible to a session reading the catalog to choose what to open, while a
|
|
33
|
+
* listed name with no file sends one to a path that opens nothing. Neither
|
|
34
|
+
* surfaces from reading the index alone.
|
|
35
|
+
*/
|
|
36
|
+
export async function auditFolder(folder: AuditedFolder): Promise<FolderDrift> {
|
|
37
|
+
const dir = dirname(folder.indexPath)
|
|
38
|
+
const source = await readFile(folder.indexPath, 'utf8')
|
|
39
|
+
const targets = listedTargets(source)
|
|
40
|
+
|
|
41
|
+
// A sub-catalog is linked as `<name>/index.md` and its entries are audited
|
|
42
|
+
// as their own folder, so only the leading segment is compared here.
|
|
43
|
+
const listed = new Set(targets.map((target) => target.split('/')[0]))
|
|
44
|
+
|
|
45
|
+
const unlisted = folder.entries
|
|
46
|
+
.map((path) => basename(path))
|
|
47
|
+
.filter((name) => !listed.has(name))
|
|
48
|
+
|
|
49
|
+
const missing = targets.filter((target) => !existsSync(resolve(dir, target)))
|
|
50
|
+
|
|
51
|
+
return { rel: folder.rel, unlisted, missing }
|
|
52
|
+
}
|
|
53
|
+
|
|
54
|
+
export async function auditIndexes(
|
|
55
|
+
folders: readonly AuditedFolder[],
|
|
56
|
+
): Promise<FolderDrift[]> {
|
|
57
|
+
const drift: FolderDrift[] = []
|
|
58
|
+
|
|
59
|
+
for (const folder of folders) {
|
|
60
|
+
drift.push(await auditFolder(folder))
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
return drift
|
|
64
|
+
}
|
package/src/sandbox/expect.ts
CHANGED
|
@@ -141,9 +141,33 @@ function contentArray(value: unknown): ContentAssertion[] {
|
|
|
141
141
|
if (typeof entry !== 'object' || entry === null) continue
|
|
142
142
|
|
|
143
143
|
const record = entry as Record<string, unknown>
|
|
144
|
+
|
|
145
|
+
// Dropping a half-written entry rather than throwing is deliberate, and it
|
|
146
|
+
// reads as the opposite of the stray-key check below. An entry missing its
|
|
147
|
+
// path or pattern declares no assertion to lose, and `checkExpectation`
|
|
148
|
+
// fails an arm whose surviving declaration asserts nothing, so the vacuous
|
|
149
|
+
// pass is already closed one level up. A stray key is the reverse: the entry
|
|
150
|
+
// is well-formed and the declaration around it silently lost a key it
|
|
151
|
+
// appears to carry, which nothing downstream can see.
|
|
144
152
|
if (typeof record.path !== 'string' || record.path === '') continue
|
|
145
153
|
if (typeof record.pattern !== 'string' || record.pattern === '') continue
|
|
146
154
|
|
|
155
|
+
// A bare key written below a `[[content]]` header belongs to that table in
|
|
156
|
+
// TOML, not to the document, so a declaration listing `manual` or
|
|
157
|
+
// `max_turns` after its content blocks parses clean and silently asserts
|
|
158
|
+
// neither. The `claude/ui-test` arm shipped that way: a turn ceiling that
|
|
159
|
+
// never ran and five manual entries that never reached the unchecked count,
|
|
160
|
+
// while `aitk sandbox coverage` counted the arm as armed. Nothing at the
|
|
161
|
+
// top level can see the difference, so the check belongs here.
|
|
162
|
+
const stray = Object.keys(record).filter(
|
|
163
|
+
(key) => key !== 'path' && key !== 'pattern',
|
|
164
|
+
)
|
|
165
|
+
if (stray.length > 0) {
|
|
166
|
+
throw new Error(
|
|
167
|
+
`content entry for ${record.path} carries ${stray.join(', ')}. Move top-level keys above the first [[content]] block.`,
|
|
168
|
+
)
|
|
169
|
+
}
|
|
170
|
+
|
|
147
171
|
assertions.push({ path: record.path, pattern: record.pattern })
|
|
148
172
|
}
|
|
149
173
|
|
|
@@ -257,6 +281,20 @@ function checkWriteScope(
|
|
|
257
281
|
}
|
|
258
282
|
}
|
|
259
283
|
|
|
284
|
+
// A scope produces one result per write, so a run that wrote nothing produces
|
|
285
|
+
// none, and without this the declaration vanishes from the verdict entirely:
|
|
286
|
+
// no result, no skipped entry, and no contribution to the unchecked count that
|
|
287
|
+
// exists to surface exactly this. The `undefined` branch above cannot stand in,
|
|
288
|
+
// since `run.sh` always passes `--writes` and `readWrites` returns `[]` for an
|
|
289
|
+
// empty file. An arm whose output escaped the snapshot reads as a clean run,
|
|
290
|
+
// which is the vacuous pass the harness exists to remove.
|
|
291
|
+
if (writes.length === 0) {
|
|
292
|
+
return {
|
|
293
|
+
results: [],
|
|
294
|
+
skipped: ['write scope: the run wrote nothing, so no path was checked'],
|
|
295
|
+
}
|
|
296
|
+
}
|
|
297
|
+
|
|
260
298
|
const globs = expectation.writeScope.map((glob) => new Bun.Glob(glob))
|
|
261
299
|
|
|
262
300
|
return {
|