glassbox-trace 0.9.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +42 -1
- package/dist/glassbox.html +7 -3
- package/package.json +1 -1
- package/src/adhere.mjs +41 -12
- package/src/claims.mjs +368 -0
- package/src/cli.mjs +65 -17
- package/src/fence.mjs +101 -24
- package/src/guard.mjs +96 -0
- package/src/trace-core.js +7 -3
package/README.md
CHANGED
|
@@ -39,6 +39,7 @@ glassbox compare 81c4 9f0a --out cmp.html …as one HTML with both sessions
|
|
|
39
39
|
glassbox watch live tail: the viewer follows the newest session as Claude Code writes it
|
|
40
40
|
glassbox collect ./reports --format md --out fleet.md
|
|
41
41
|
one report from a folder of redacted check reports, one per machine
|
|
42
|
+
glassbox claims 81c4 --format md said vs did: every claim the agent made about its own work, with the tool result behind it — or the gap
|
|
42
43
|
glassbox clean remove the temp files `open` and the hook leave behind
|
|
43
44
|
```
|
|
44
45
|
|
|
@@ -56,10 +57,15 @@ The npm package is `glassbox-trace` (plain `glassbox` was already taken); the co
|
|
|
56
57
|
|
|
57
58
|
Feedback at the end of a session can only change that session's last reply. Add `--context` to carry it forward: the Stop hook keeps the findings (and, with `--feedback`, the agent's answer) in `<project>/.glassbox/last-session.md` — git-ignored, deleted after a clean session — and a SessionStart hook gives them to the next session in that project.
|
|
58
59
|
|
|
60
|
+
**Stop a loop before it happens.** `--guard` adds a PreToolUse hook: when the agent is about to repeat a call that has already failed twice in a row, unchanged, in the current turn, it blocks the call and tells the agent why, with the last error, and to change something or ask you rather than try again. It is deliberately narrow. Anything that could change the outcome in between — an edit, another command, a success, a new prompt — resets it, so re-running the tests after a fix is never blocked; calls a person stopped don't count; it never approves anything, so your permission prompts are untouched. Replayed over 4,121 real tool calls it blocked none, so it stays out of the way until an agent is actually stuck. It runs before every tool call, so it needs an installed copy: `--guard` writes a direct `node "…/bin/glassbox.mjs" hook …` command (for the Stop hook too) and refuses to install from an `npx` cache, where each call would cost seconds. The plugin doesn't include the guard ([why](docs/PLUGIN.md#the-loop-guard-is-not-in-the-plugin)); if both are installed, the plugin sees the CLI hook and steps aside.
|
|
61
|
+
|
|
59
62
|
```
|
|
60
63
|
glassbox hook install --feedback --context --fail-on warn # summary, feedback, and notes for the next session
|
|
61
64
|
glassbox hook install # summary only
|
|
62
65
|
glassbox hook uninstall
|
|
66
|
+
|
|
67
|
+
npm i -g glassbox-trace # the guard needs an installed copy, not npx
|
|
68
|
+
glassbox hook install --feedback --context --guard # all of the above, plus the loop guard
|
|
63
69
|
```
|
|
64
70
|
|
|
65
71
|
**Or install it as a Claude Code plugin** — the same hooks plus `/glassbox:check` and a skill Claude uses when you ask "what went wrong in this session?", running the bundled code (no npx): `/plugin marketplace add aldohushi1-stack/glassbox`, then `/plugin install glassbox@glassbox-trace`. Feedback and context are opt-in plugin options. See [docs/PLUGIN.md](docs/PLUGIN.md).
|
|
@@ -94,6 +100,8 @@ glassbox fence 81c4cbfd one session (and its subagents)
|
|
|
94
100
|
glassbox fence ./archive every .jsonl under a folder
|
|
95
101
|
glassbox fence --format md --out fence.md
|
|
96
102
|
glassbox fence --shred overwrite each value in place with [FENCED:<rule>:<fingerprint>]
|
|
103
|
+
glassbox fence --sessions-only transcripts only (the default sweep also covers prompt history, paste cache, file history, debug logs, shell snapshots)
|
|
104
|
+
glassbox fence --key team.key one shared fingerprint key for a fleet
|
|
97
105
|
```
|
|
98
106
|
|
|
99
107
|
```
|
|
@@ -106,7 +114,7 @@ Glassbox fence · 34 sessions under /home/aldo/.claude/projects · 212 MB
|
|
|
106
114
|
INFO credential-file-read line 211 /home/aldo/api/.env — tool input · Read
|
|
107
115
|
```
|
|
108
116
|
|
|
109
|
-
Each row is a masked preview and a fingerprint (first 8 hex of
|
|
117
|
+
Each row is a masked preview and a fingerprint (first 8 hex of HMAC-SHA256 under a key that stays on the machine, so even a weak password's fingerprint can't be guessed against) — enough to know which key it is and to see the same key across sessions, never enough to use it. A sweep of the whole home also covers the other places Claude Code keeps text: prompt history, the paste cache, file-history snapshots, debug logs and shell snapshots. `error` is a credential whose format identifies it (AWS, GitHub, Anthropic, OpenAI, Slack, Stripe, Google, npm, SendGrid, a private key block, a database URL with its password); `warn` is a secret named by its context (`password=`, `api_key:`, `Bearer …`, a URL with credentials, a JWT), kept only when the value has the entropy of a real one and is not a placeholder; `info` is a read of a credential file (`.env`, `.npmrc`, `.netrc`, `~/.aws/credentials`, `id_rsa`, `*.pem` …) whose contents are now in the transcript whether or not a rule recognised them. A secret that reached a transcript reached a disk, and whatever backs that disk up: rotate it, then `--shred`. Shredded transcripts still open and check. It verifies nothing against any provider (no network), and a password in prose with no context word passes through — it is a net, not a guarantee. Details in [docs/FENCE.md](docs/FENCE.md).
|
|
110
118
|
|
|
111
119
|
### Is my CLAUDE.md doing anything? — `glassbox adhere`
|
|
112
120
|
|
|
@@ -130,6 +138,32 @@ Glassbox adhere · /home/aldo/api
|
|
|
130
138
|
|
|
131
139
|
Nine rule shapes are checkable from tool calls — run X before commit/push, run X after changes, use A not B, never run, never touch, ask before, read before edit, commit message format, no new docs — and everything else is listed as *not checkable yet* with its line number, so the report never claims more than it measured. Commands are judged on their shell surface (a `git push --force` inside a document being written is not a force push). The first real run, on the session that built it, scored 22%. Details and the honest caveats in [docs/ADHERE.md](docs/ADHERE.md).
|
|
132
140
|
|
|
141
|
+
### Said vs did — `glassbox claims`
|
|
142
|
+
|
|
143
|
+
Every sentence in which the agent told you what it did — "all 76 tests pass", "committed as `753be02`", "live on npm", "nothing changed" — matched to the tool result that should be standing behind it, or to the gap where one should be:
|
|
144
|
+
|
|
145
|
+
```
|
|
146
|
+
glassbox claims the newest session (subagents included)
|
|
147
|
+
glassbox claims 81c4 --format md --out claims.md
|
|
148
|
+
glassbox claims --fail-on unverified exit 1 on any claim without a receipt (default: contradicted only)
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
```
|
|
152
|
+
Glassbox claims · 38f81247
|
|
153
|
+
69 claims · 61 verified · 7 declared · 1 unverified · 0 contradicted
|
|
154
|
+
|
|
155
|
+
UNVERIFIED test t23 Your folder is as I left it: `master` is GitHub's `main` plus the loop guard and Action commit,…
|
|
156
|
+
↳ no test, build, validate or CI result in the window
|
|
157
|
+
VERIFIED test t3 58 of 58 pass.
|
|
158
|
+
↳ 58 passed, 0 failed (1 call)
|
|
159
|
+
VERIFIED ship t4 I committed it as `753be02` on branch `wait-aware-findings`: 6 files, 117 lines added and 6 rem…
|
|
160
|
+
↳ ship result in the window (1 call)
|
|
161
|
+
DECLARED declared t3 Not verified: the browser e2e test (`test/browser.e2e.mjs`) didn't run because Playwright isn't…
|
|
162
|
+
↳ the agent said so
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Six kinds of claim (tests pass, shipped, verified, fixed, nothing changed, file written), judged on the same agent's tool calls in that turn: a test claim needs a test run after the last edit — from `Bash`, `PowerShell` or Cowork's `device_bash` — with its counts read and compared; a shipped claim needs the commit, push, publish or upload result, carrying the sha or version the sentence names; "nothing changed" needs a `git status` after the thing that could have changed it. An agent that says "not verified" gets a *declared* row and no finding. The same ledger runs inside `check` as three rules — `contradicted-claim` (error), `unverified-claim` (warn), `stale-claim` (info) — so the Stop hook hands the sentence back to the agent that wrote it. On sixteen real sessions (367 claims; three of them checked by hand first) the ledger found one unverified sentence — "the tests still pass", 24 minutes after the last run — five fixes never re-run after their edit, and no contradictions. A verdict is about the sentence, never the code. Details in [docs/CLAIMS.md](docs/CLAIMS.md).
|
|
166
|
+
|
|
133
167
|
## What it flags
|
|
134
168
|
|
|
135
169
|
| rule | fires when |
|
|
@@ -150,6 +184,7 @@ Nine rule shapes are checkable from tool calls — run X before commit/push, run
|
|
|
150
184
|
| `slow-tool` | a tool call ≥ 60 s (warn at 5 min); one line per tool when repeated; blocking `TaskOutput` waits roll up per background task as info; time spent on the human (questions, permission prompts) is excluded |
|
|
151
185
|
| `permission-denied` | tool calls the human rejected, a permission rule or auto mode blocked, or an interrupt stopped (info) |
|
|
152
186
|
| `max-tokens`, `api-error`, `hook-error`, `compaction`, `thinking-heavy`, `subagent-share`, `long-turn` | what they say (`long-turn`: ≥ 30 tool calls after one prompt in the main conversation) |
|
|
187
|
+
| `contradicted-claim`, `unverified-claim`, `stale-claim` | the agent told the human something the transcript contradicts (error), has no receipt for (warn), or checked before its last edit / only partly (info) — see `glassbox claims` above |
|
|
153
188
|
|
|
154
189
|
Identical findings print once with a `×N` count in `check`. All mechanical, no AI. The thresholds were tuned on a corpus of 33 real sessions (481 findings → 142, median 2 per session); `node scripts/corpus-audit.mjs` re-runs that audit on your own sessions and prints counts only. Thresholds are in `TraceCore.DEFAULTS` and can be overridden when calling `diagnose(trace, opts)`. Every rule has a one-line "next time" in `TraceCore.ADVICE`, which is what the Markdown report and the Stop hook hand back to the agent.
|
|
155
190
|
|
|
@@ -180,6 +215,8 @@ src/trace-core.js pure engine: parseTrace · diagnose · estimateCost · red
|
|
|
180
215
|
src/viewer.html the UI; the build inlines trace-core and the demo
|
|
181
216
|
src/cli.mjs CLI library (session discovery, embed, check, compare, hook); bin/glassbox.mjs is the entry point
|
|
182
217
|
src/tail.mjs live tail: byte-offset tailer + loopback SSE server for `glassbox watch`
|
|
218
|
+
src/guard.mjs the PreToolUse loop guard (`hook --guard`)
|
|
219
|
+
src/claims.mjs said vs did: claims out of the agent's text, judged against the tool calls (`claims`, and three `check` rules)
|
|
183
220
|
scripts/build.mjs build
|
|
184
221
|
scripts/sanitize.mjs turn a real transcript into a shareable fixture (demo or structure mode)
|
|
185
222
|
scripts/corpus-audit.mjs run every rule over your local sessions; counts only (--baseline to diff two runs)
|
|
@@ -215,3 +252,7 @@ Your context is private — by construction, not by promise.
|
|
|
215
252
|
- **The shape, not the names.** A redacted report still needs to say "this one file was read 48 times and 6 of those failed". `check --redact --legend audit.legend.json` replaces every path with a keyed hash (`file:b94b1a35`, HMAC-SHA256 with a random salt) and writes the key → path map to the legend file, which stays on your machine. The redacted JSON carries `summary.files` (reads, writes, failures, agents, chars per key) and `evidence.files` on findings; `duplicate-subagent-read` keeps its sentence with the key in it. `glassbox reveal report.md --legend audit.legend.json` turns the keys back into paths for you. Re-using the legend keeps keys stable across runs; two machines with two legends produce different keys for the same file. Treat the legend like a password file: local, git-ignored, never attached to the same email as the report. Design notes in DESIGN.md §12.
|
|
216
253
|
|
|
217
254
|
Source: [github.com/aldohushi1-stack/glassbox](https://github.com/aldohushi1-stack/glassbox) · MIT © Aldo Hushi
|
|
255
|
+
|
|
256
|
+
## Security
|
|
257
|
+
|
|
258
|
+
Found a way Glassbox could leak, run something it shouldn't, or be tampered with? Please report it privately — see [SECURITY.md](SECURITY.md).
|
package/dist/glassbox.html
CHANGED
|
@@ -408,7 +408,7 @@ kbd{font:11px var(--mono);border:1px solid var(--line);border-bottom-width:2px;b
|
|
|
408
408
|
})(typeof globalThis !== 'undefined' ? globalThis : this, function () {
|
|
409
409
|
'use strict';
|
|
410
410
|
|
|
411
|
-
const VERSION = '0.
|
|
411
|
+
const VERSION = '0.10.0';
|
|
412
412
|
|
|
413
413
|
// ---------------------------------------------------------------------------
|
|
414
414
|
// Rate card (USD per million tokens). Prefix-matched against model ids so dated
|
|
@@ -1333,6 +1333,10 @@ kbd{font:11px var(--mono);border:1px solid var(--line);border-bottom-width:2px;b
|
|
|
1333
1333
|
'permission-denied': 'Before a call the user may not want (installs, network, deleting, publishing), say what it will do and ask; for calls they always allow, suggest a permissions allow rule so the prompt goes away.',
|
|
1334
1334
|
'duplicate-subagent-read': 'When several subagents need the same file, put the relevant excerpt in their briefs or have one agent summarise it once, instead of every agent reading it into its own context.',
|
|
1335
1335
|
'idle-cache-expiry': 'The cache expired while the session sat idle; resume sooner, use the 1-hour cache lifetime where available, or start a fresh session with a summary instead of re-sending the whole history.',
|
|
1336
|
+
// claims ledger (src/claims.mjs): said vs did
|
|
1337
|
+
'unverified-claim': "Say it after you've checked it: a test claim needs a test run after your last edit, a shipped claim needs the command's result in this transcript, a 'nothing changed' claim needs a git status after the thing that could have changed it. If you did not check, say 'not re-run' instead.",
|
|
1338
|
+
'contradicted-claim': 'The transcript disagrees with what you told the user; read the tool result you are summarising before you summarise it, and correct the statement now.',
|
|
1339
|
+
'stale-claim': 'The check you are citing ran before your last edit, or covers only part of what you said; run it again or say exactly what it covered.',
|
|
1336
1340
|
};
|
|
1337
1341
|
|
|
1338
1342
|
// Advice for a concrete finding: the rule's line, except where the evidence says otherwise.
|
|
@@ -1381,7 +1385,7 @@ kbd{font:11px var(--mono);border:1px solid var(--line);border-bottom-width:2px;b
|
|
|
1381
1385
|
}
|
|
1382
1386
|
|
|
1383
1387
|
// Rules whose detail quotes tool output or model text (everything else is names and numbers).
|
|
1384
|
-
const CONTENT_DETAIL = new Set(['failed-tool', 'api-error', 'hook-error', 'duplicate-subagent-read']);
|
|
1388
|
+
const CONTENT_DETAIL = new Set(['failed-tool', 'api-error', 'hook-error', 'duplicate-subagent-read', 'unverified-claim', 'contradicted-claim', 'stale-claim']);
|
|
1385
1389
|
function redactDetail(f) { return CONTENT_DETAIL.has(f.id) ? blank(f.detail) : f.detail; }
|
|
1386
1390
|
|
|
1387
1391
|
// ---------------------------------------------------------------------------
|
|
@@ -1575,7 +1579,7 @@ kbd{font:11px var(--mono);border:1px solid var(--line);border-bottom-width:2px;b
|
|
|
1575
1579
|
}
|
|
1576
1580
|
function toJsonl(records) { return records.map((r) => JSON.stringify(r)).join('\n') + '\n'; }
|
|
1577
1581
|
|
|
1578
|
-
return { VERSION, RATES, DEFAULTS, ADVICE, METRICS, parseLines, parseTrace, diagnose, estimateCost, rateFor, redact, toJsonl, toolCategory, textOf, fmtDur, fmtInt, fmtMetric, fmtRatio, fmtChange, stableStringify, inputSummary, adviceFor, redactDetail, reportMarkdown, compare, compareMarkdown, toolTable, fileStats, filePathOf, normalisePath };
|
|
1582
|
+
return { VERSION, RATES, DEFAULTS, ADVICE, METRICS, parseLines, parseTrace, diagnose, denialKind, estimateCost, rateFor, redact, toJsonl, toolCategory, textOf, fmtDur, fmtInt, fmtMetric, fmtRatio, fmtChange, stableStringify, inputSummary, adviceFor, redactDetail, reportMarkdown, compare, compareMarkdown, toolTable, fileStats, filePathOf, normalisePath };
|
|
1579
1583
|
});
|
|
1580
1584
|
</script>
|
|
1581
1585
|
<script>
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "glassbox-trace",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.10.0",
|
|
4
4
|
"description": "Other tools show what happened in a Claude Code session; Glassbox tells you what went wrong. Timeline, token and cost burn, auto-diagnostics with evidence and advice, session compare, live tail, a fleet view, a secrets scanner for transcripts (fence), a CLAUDE.md compliance report (adhere), a CLI and a Claude Code Stop hook that feeds the findings back to the agent.",
|
|
5
5
|
"main": "src/trace-core.js",
|
|
6
6
|
"scripts": {
|
package/src/adhere.mjs
CHANGED
|
@@ -17,6 +17,10 @@ const core = require('./trace-core.js');
|
|
|
17
17
|
export const encodeProject = (p) => String(p).replace(/[\\/:]/g, '-');
|
|
18
18
|
const norm = (p) => String(p || '').replace(/\\/g, '/').toLowerCase();
|
|
19
19
|
const WRITE_TOOLS = new Set(['Edit', 'MultiEdit', 'Write', 'NotebookEdit']);
|
|
20
|
+
// Commands come from every shell tool Claude Code offers, not only Bash: PowerShell on Windows, and Cowork's
|
|
21
|
+
// device_bash on the user's machine. A `git push` from PowerShell is a push (0.9.1 missed all of them).
|
|
22
|
+
export const SHELL_TOOLS = new Set(['Bash', 'PowerShell']);
|
|
23
|
+
export const isShell = (c) => SHELL_TOOLS.has(c.name) || /device_bash$/.test(c.name || '');
|
|
20
24
|
const EDIT_TOOLS = new Set(['Edit', 'MultiEdit', 'NotebookEdit']);
|
|
21
25
|
const READ_TOOLS = new Set(['Read', 'NotebookRead']);
|
|
22
26
|
|
|
@@ -30,9 +34,31 @@ const ACTION = (w) => /^commit/i.test(w) ? 'commit' : /^push/i.test(w) ? 'push'
|
|
|
30
34
|
const strip = (s) => s.replace(/`/g, '').replace(/\s+/g, ' ').trim();
|
|
31
35
|
const whatOf = (s) => { const w = strip(s).replace(/^(?:the|all|your|our)\s+/i, ''); if (/^(?:tests?|test suite|unit tests|the tests)$/i.test(w)) return 'tests'; if (/^(?:formatter|code formatter)$/i.test(w)) return 'formatter'; if (/^(?:linter|lint)$/i.test(w)) return 'linter'; return w.replace(/\.$/, ''); };
|
|
32
36
|
|
|
33
|
-
//
|
|
37
|
+
// ask-before: the actions named in a clause, in order, once each ("any commit / push / deploy" is three rules).
|
|
38
|
+
const ACT_WORD = String.raw`(?:commit|push|delet|remov|install|creat|deploy|merg)\w*`;
|
|
39
|
+
const actionsIn = (s) => [...new Set((s.match(new RegExp(String.raw`\b${ACT_WORD}`, 'gi')) || []).map(ACTION))];
|
|
40
|
+
// A run of actions joined by , / or and: "committing, pushing or deploying". Stops at the first non-action word.
|
|
41
|
+
const actList = (w) => String.raw`${w}(?:\s*(?:,|\/|\bor\b|\band\b)\s*(?:(?:or|and)\s+)?(?:any\s+|a\s+|an\s+)?${w})*`;
|
|
42
|
+
const ACT_LIST = actList(ACT_WORD);
|
|
43
|
+
const ACT_LIST_NC = actList(ACT_WORD.replace('creat|', '')); // "don't create … unless I ask" belongs to no-new-docs
|
|
44
|
+
const ASK_VERB = String.raw`(?:ask(?:\s+me)?|check with me|confirm(?:\s+with me)?|get (?:my )?(?:explicit\s+)?(?:permission|approval|ok|confirmation)|wait for (?:my\s+|your\s+|an?\s+)?(?:explicit\s+)?(?:confirmation|approval|permission|go-?ahead|ok))`;
|
|
45
|
+
const CONFIRM_NOUN = String.raw`(?:wait(?:ing)? for|get(?:ting)?|obtain(?:ing)?|ask(?:ing)? for|receiv(?:e|ing))\s+(?:my\s+|your\s+|an?\s+|the\s+)?(?:explicit\s+|express\s+|written\s+|clear\s+)?(?:confirmation|approval|permission|sign-?off|go-?ahead|ok)\b`;
|
|
46
|
+
|
|
47
|
+
// Each matcher returns the rule's parameters (an array for one rule per action) or null. Order matters: first match wins.
|
|
34
48
|
const SHAPES = [
|
|
35
|
-
{ kind: 'ask-before', m: (s) => {
|
|
49
|
+
{ kind: 'ask-before', m: (s) => {
|
|
50
|
+
const each = (acts) => acts.length ? acts.map((action) => ({ action })) : null;
|
|
51
|
+
// "ask before committing or pushing" · "wait for approval before any deploy"
|
|
52
|
+
let m = s.match(new RegExp(String.raw`\b${ASK_VERB}\s*(?:first\s+)?(?:before|prior to)\s+(?:you\s+)?(?:any\s+|a\s+|an\s+)?(?:\w+ing\s+)?(${ACT_LIST})`, 'i'));
|
|
53
|
+
if (m) return each(actionsIn(m[1]));
|
|
54
|
+
// Reversed: "Before editing X, or any commit / push / deploy, state the change and wait for explicit confirmation."
|
|
55
|
+
m = s.match(new RegExp(String.raw`\b(?:before|prior to)\s+([^.]*?)\b${CONFIRM_NOUN}`, 'i'));
|
|
56
|
+
if (m) { const r = each(actionsIn(m[1])); if (r) return r; }
|
|
57
|
+
// "never push without asking" · "do not commit or push unless I ask"
|
|
58
|
+
m = s.match(new RegExp(String.raw`\b(?:never|don't|do not|don’t)\s+(${ACT_LIST_NC})\b[^.]*?\b(?:without|unless)\s+(?:first\s+)?(?:asking|being asked|i ask|you ask|i tell|told|permission|approval|confirm|checking|explicit)`, 'i'));
|
|
59
|
+
if (m) return each(actionsIn(m[1]));
|
|
60
|
+
return null;
|
|
61
|
+
} },
|
|
36
62
|
{ kind: 'run-before', m: (s) => { let m = s.match(/\b(?:always\s+)?run\s+(?:the\s+)?(`[^`]+`|[\w][\w .:/-]*?)\s+(?:before|prior to)\s+(?:you\s+|every\s+|each\s+|any\s+|a\s+)?(?:\w+ing\s+)?(commit|push|merg|pull request|pr\b|open)/i); if (m) return { what: whatOf(m[1]), trigger: /^(commit)/i.test(m[2]) ? 'commit' : 'push' }; m = s.match(/\b(?:never|don't|do not|don’t)\s+(commit|push)\b[^.]*?\b(?:without|unless)\s+(?:first\s+)?(?:running\s+|you(?:'ve| have)?\s+run\s+|the\s+)?(`[^`]+`|[\w][\w .:/-]*?)(?:\s+(?:pass(?:es|ing)?|first|succeed\w*))?\.?$/i); if (m && !/\bask|permission|approv|confirm|check/i.test(s)) return { what: whatOf(m[2]), trigger: m[1].toLowerCase() }; m = s.match(/\b(tests?|test suite|linter|lint|build|typecheck|type check)\s+(?:must|should|need to|has to|have to)\s+(?:pass|succeed|be green)\s+(?:before|prior to)\s+(?:you\s+|every\s+|each\s+|any\s+)?(?:\w+ing\s+)?(commit|push|merg)/i); if (m) return { what: whatOf(m[1]), trigger: /^commit/i.test(m[2]) ? 'commit' : 'push' }; return null; } },
|
|
37
63
|
{ kind: 'run-after', m: (s) => { let m = s.match(/\brun\s+(?:the\s+)?(`[^`]+`|[\w][\w .:/-]*?)\s+after\s+(?:you\s+)?(?:making\s+|every\s+|each\s+|any\s+|all\s+)?(?:change|edit|modif|writ)/i); if (m) return { what: whatOf(m[1]) }; m = s.match(/\b(format|lint|typecheck)\s+(?:the\s+)?(?:code|files?)\s+after\s+(?:you\s+)?(?:making\s+|every\s+|each\s+|any\s+)?(?:change|edit|modif)/i); if (m) return { what: /^format/i.test(m[1]) ? 'formatter' : /^lint/i.test(m[1]) ? 'linter' : m[1] }; return null; } },
|
|
38
64
|
{ kind: 'prefer-tool', m: (s) => { let m = s.match(new RegExp(`\\b(?:never|don't|do not|don\u2019t|avoid)\\s+(?:us(?:e|ing)\\s+)?\`?(${TOOLS_RE})\`?\\b`, 'i')); if (m) { const avoid = m[1].toLowerCase(); return { prefer: PREFER_FOR[avoid] || null, avoid: [avoid] }; } m = s.match(new RegExp(`\\buse\\s+\`?(${TOOLS_RE})\`?\\b(?:[^.]*?\\b(?:not|instead of|rather than|never|over)\\s+\`?(${TOOLS_RE})\`?)?`, 'i')); if (m) { const prefer = m[1].toLowerCase(); const avoid = m[2] ? [m[2].toLowerCase()] : (PM_PAIRS[prefer] || []); return avoid.length ? { prefer, avoid } : null; } return null; } },
|
|
@@ -58,10 +84,11 @@ export function parseRules(text, file) {
|
|
|
58
84
|
const parts = body.split(/(?<=[.!])\s+(?=[A-Z`])/);
|
|
59
85
|
for (const part of parts) {
|
|
60
86
|
const s = part.trim(); if (!s) continue;
|
|
61
|
-
let
|
|
62
|
-
for (const sh of SHAPES) { const p = sh.m(s); if (p) {
|
|
87
|
+
let hits = null;
|
|
88
|
+
for (const sh of SHAPES) { const p = sh.m(s); if (p) { hits = [].concat(p).map((x) => Object.assign({ kind: sh.kind }, x)); break; } }
|
|
63
89
|
const rec = { id: `${file}:${n}${parts.length > 1 ? ':' + (parts.indexOf(part) + 1) : ''}`, file, line: n, text: strip(s) };
|
|
64
|
-
if (
|
|
90
|
+
if (!hits) unchecked.push(rec);
|
|
91
|
+
else for (const h of hits) rules.push(Object.assign({}, rec, hits.length > 1 ? { id: `${rec.id}#${h.action}` } : null, h));
|
|
65
92
|
}
|
|
66
93
|
}
|
|
67
94
|
return { rules, unchecked };
|
|
@@ -149,9 +176,10 @@ function judgeSession(rules, s, project) {
|
|
|
149
176
|
const turnPrompt = (i) => { const t = turns.find((x) => x.index === i && x.agent === 'main') || turns.find((x) => x.index === i); return t ? { text: t.promptText || '', kind: t.promptKind } : { text: '', kind: null }; };
|
|
150
177
|
const prevTurnAsked = (i) => { const prev = trace.requests.filter((r) => r.turnIndex === i - 1 && r.agent === 'main'); if (!prev.length) return false; const last = prev[prev.length - 1]; const texts = (last.blocks || []).filter((b) => b.type === 'text').map((b) => b.text).join(' ').trim(); return /\?\s*$/.test(texts); };
|
|
151
178
|
const short = (t) => String(t == null ? '' : t).replace(/\s+/g, ' ').trim().slice(0, 160);
|
|
152
|
-
|
|
153
|
-
const
|
|
154
|
-
const
|
|
179
|
+
// turn is 1-based, the number `check` prints and the viewer shows (turnIndex + 1).
|
|
180
|
+
const occ = (c, obeyed, detail) => ({ obeyed, session: s.id, turn: c.turnIndex + 1, prompt: turnPrompt(c.turnIndex).text.replace(/\s+/g, ' ').slice(0, 80), detail: short(detail), at: c.start });
|
|
181
|
+
const sessionOcc = (obeyed, detail, c) => ({ obeyed, session: s.id, turn: c ? c.turnIndex + 1 : null, prompt: c ? turnPrompt(c.turnIndex).text.replace(/\s+/g, ' ').slice(0, 80) : '', detail: short(detail), at: c ? c.start : trace.meta.start });
|
|
182
|
+
const bash = calls.filter((c) => isShell(c) && typeof c.input?.command === 'string');
|
|
155
183
|
const writes = calls.filter((c) => WRITE_TOOLS.has(c.name) && c.input && typeof c.input.file_path === 'string');
|
|
156
184
|
const out = rules.map(() => []);
|
|
157
185
|
rules.forEach((r, i) => {
|
|
@@ -215,7 +243,8 @@ export function adhere(opts = {}) {
|
|
|
215
243
|
const rate = occ.length ? obeyed / occ.length : null;
|
|
216
244
|
const verdict = !occ.length ? 'never came up' : rate === 1 ? 'obeyed' : rate >= 0.8 ? 'mostly obeyed' : rate > 0 ? 'often ignored' : 'ignored';
|
|
217
245
|
const examples = occ.filter((o) => !o.obeyed).slice(0, 3).map((o) => ({ session: o.session, turn: o.turn, prompt: R(o.prompt), detail: R(o.detail) }));
|
|
218
|
-
|
|
246
|
+
// One line can hold several ask-before rules ("commit / push / deploy"); the label tells them apart.
|
|
247
|
+
const row = { id: r.id, file: r.file, line: r.line, text: r.text, kind: r.kind, label: r.id.includes('#') ? `${r.kind}:${r.action}` : r.kind, occasions: occ.length, obeyed, broken, rate, verdict, examples, sessionsBroken: new Set(occ.filter((o) => !o.obeyed).map((o) => o.session)).size };
|
|
219
248
|
for (const k of ['what', 'trigger', 'prefer', 'avoid', 'pattern', 'path', 'action', 'style', 'unlessAsked']) if (r[k] !== undefined) row[k] = r[k];
|
|
220
249
|
if (NOTES[r.kind]) row.note = NOTES[r.kind];
|
|
221
250
|
return row;
|
|
@@ -239,7 +268,7 @@ export function adhereText(rep) {
|
|
|
239
268
|
L.push('');
|
|
240
269
|
const order = { 'ignored': 0, 'often ignored': 1, 'mostly obeyed': 2, 'obeyed': 3, 'never came up': 4 };
|
|
241
270
|
for (const r of rep.rules.slice().sort((a, b) => order[a.verdict] - order[b.verdict] || b.occasions - a.occasions)) {
|
|
242
|
-
L.push(` ${r.verdict.toUpperCase().padEnd(14)} ${r.
|
|
271
|
+
L.push(` ${r.verdict.toUpperCase().padEnd(14)} ${r.label.padEnd(17)} ${r.occasions ? `${r.obeyed}/${r.occasions} ${pct(r.rate).padStart(4)}` : ' — '} ${r.text} (${r.file}:${r.line})`);
|
|
243
272
|
for (const e of r.examples) L.push(` ↳ ${e.session.slice(0, 8)} t${e.turn == null ? '?' : e.turn} "${e.prompt}" → ${e.detail}`);
|
|
244
273
|
}
|
|
245
274
|
if (rep.unchecked.length) { L.push('', ` Not checkable yet (${rep.unchecked.length}) — no mechanical test for these shapes; they are loaded every session all the same:`); for (const u of rep.unchecked) L.push(` · ${u.text} (${u.file}:${u.line})`); }
|
|
@@ -252,9 +281,9 @@ export function adhereMarkdown(rep) {
|
|
|
252
281
|
const L = ['# Is my CLAUDE.md doing anything?', '', `Project \`${rep.project}\` · ${rep.generated.slice(0, 10)}${rep.since ? ` · sessions since ${rep.since.slice(0, 10)}` : ''}`, '', `**${headline(rep)}**`, ''];
|
|
253
282
|
if (s.sessionsWithBreach) L.push(`${s.sessionsWithBreach} of ${s.sessions} sessions broke at least one rule; those sessions cost ${usd(s.breachCost)} in total (where the breaches happened, not what they cost).`, '');
|
|
254
283
|
L.push('## Rules', '', '| rule | kind | occasions | obeyed | broken | rate | verdict |', '|---|---|---:|---:|---:|---:|---|');
|
|
255
|
-
for (const r of rep.rules) L.push(`| ${r.text.replace(/\|/g, '\\|')} <br><sub>${r.file}:${r.line}</sub> | \`${r.
|
|
284
|
+
for (const r of rep.rules) L.push(`| ${r.text.replace(/\|/g, '\\|')} <br><sub>${r.file}:${r.line}</sub> | \`${r.label}\` | ${r.occasions} | ${r.obeyed} | ${r.broken} | ${pct(r.rate)} | ${r.verdict} |`);
|
|
256
285
|
const withEx = rep.rules.filter((r) => r.examples.length);
|
|
257
|
-
if (withEx.length) { L.push('', '## Evidence', ''); for (const r of withEx) { L.push(`### ${r.text}`, '', `\`${r.
|
|
286
|
+
if (withEx.length) { L.push('', '## Evidence', ''); for (const r of withEx) { L.push(`### ${r.text}`, '', `\`${r.label}\` · ${r.broken} of ${r.occasions} occasions broken across ${r.sessionsBroken} session${r.sessionsBroken === 1 ? '' : 's'}${r.note ? ` · ${r.note}` : ''}`, ''); for (const e of r.examples) L.push(`- \`${e.session.slice(0, 8)}\` turn ${e.turn == null ? '?' : e.turn} — prompt: "${e.prompt}" — ${e.detail.replace(/\|/g, '\\|')}`); L.push(''); } }
|
|
258
287
|
if (rep.unchecked.length) { L.push('## Not checkable yet', '', 'No mechanical test exists for these shapes; they are loaded into every session all the same.', ''); for (const u of rep.unchecked) L.push(`- ${u.text} <sub>${u.file}:${u.line}</sub>`); L.push(''); }
|
|
259
288
|
L.push('## How to read this', '', 'Every number comes from tool calls in the transcripts, judged per occasion (a commit, a write, a command). "Asked" means the turn\'s own prompt named the action, an AskUserQuestion ran first, or the previous turn ended with a question — a standing instruction from an earlier turn counts as unasked. "Ran the tests" means a test command ran, not that it passed. A rule that never came up is not an obeyed rule. `glassbox adhere --format json` has every occasion.', '');
|
|
260
289
|
return L.join('\n');
|