@smartmemory/compose 0.2.21-beta → 0.2.23-beta
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: context-budget
|
|
3
|
+
description: Use when the user wants to audit, measure, or trim the session-start context load — token cost of agents, skills, rules, MCP tool schemas, and the CLAUDE.md chain. Triggers on "context budget", "token audit", "what's loading my context", "trim my skills/agents", "how big is my context". Produces a ranked, classified cut list; never auto-applies cuts.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Context Budget
|
|
7
|
+
|
|
8
|
+
Audit the loaded surface a session pays for at startup, estimate its token cost, classify
|
|
9
|
+
every component into **always / sometimes / rarely needed**, and produce a ranked cut list
|
|
10
|
+
with estimated reclaim. Read-only — you surface recommendations; the user reviews and chooses.
|
|
11
|
+
|
|
12
|
+
## When to use
|
|
13
|
+
|
|
14
|
+
- "Run a context budget" / "audit my context" / "what's eating my context window"
|
|
15
|
+
- After adding skills, agents, rules, or MCP servers — measure the new cost
|
|
16
|
+
- Before trimming `~/.claude/skills/` or a project's `.claude/` surface
|
|
17
|
+
|
|
18
|
+
## How it works
|
|
19
|
+
|
|
20
|
+
The scan/classify/report logic lives in a tested module: `compose/lib/context-budget.js`
|
|
21
|
+
(pure ESM, covered by `compose/test/context-budget.test.js`). This skill is a thin wrapper —
|
|
22
|
+
run the module's CLI, then interpret the report for the user.
|
|
23
|
+
|
|
24
|
+
### Step 1 — Gather live MCP tool counts
|
|
25
|
+
|
|
26
|
+
`.mcp.json` lists servers but **not** how many tools each exposes. The session's own
|
|
27
|
+
tool-list (the deferred-tools / connected-MCP reminders you can see) is the source of truth.
|
|
28
|
+
Count the tools per server you actually have loaded, e.g. `compose=46,stratum=44`. Servers you
|
|
29
|
+
don't pass a count for are still listed but flagged `tool-count-unknown` and excluded from the
|
|
30
|
+
numeric total (the tool never fabricates a number).
|
|
31
|
+
|
|
32
|
+
### Step 2 — Run the audit
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
node <compose-root>/lib/context-budget.js <project-root> \
|
|
36
|
+
--tool-counts=compose=46,stratum=44,playwright=25,filesystem=14,memory=9
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
- `<project-root>` defaults to `cwd` if omitted.
|
|
40
|
+
- The CLI walks: `~/.claude/{agents,skills,rules,CLAUDE.md}` + the project's
|
|
41
|
+
`.claude/{agents,skills,rules}`, the `.mcp.json` servers, and the CLAUDE.md chain
|
|
42
|
+
(home → every `CLAUDE.md` from cwd up to the repo root).
|
|
43
|
+
- Token estimate is a dependency-free ~4-chars-per-token heuristic — **relative budgeting,
|
|
44
|
+
not billing-accurate**. Use it to rank, not to bill.
|
|
45
|
+
|
|
46
|
+
### Step 3 — Interpret the report
|
|
47
|
+
|
|
48
|
+
The report prints three buckets and a TOP 5 RECLAIMS list. Walk the user through:
|
|
49
|
+
|
|
50
|
+
- **ALWAYS** — referenced by name in the CLAUDE.md chain, or the CLAUDE.md chain itself. Keep.
|
|
51
|
+
- **SOMETIMES** — domain-specific and not referenced in CLAUDE.md. Includes active MCP servers
|
|
52
|
+
the chain doesn't mention: they're load-bearing *while configured*, but if this project doesn't
|
|
53
|
+
use a server, disabling it in `.mcp.json` is often the single biggest reclaim (schemas are tens
|
|
54
|
+
of K tokens). Candidates for on-demand activation / disable-if-unused rather than always-loaded.
|
|
55
|
+
- **RARELY** — duplicates across surfaces, overlapping rules, simple-CLI-wrapping MCP servers.
|
|
56
|
+
Recommend cutting.
|
|
57
|
+
|
|
58
|
+
Flags worth calling out explicitly:
|
|
59
|
+
- `duplicate` — same SKILL.md present in both `compose/.claude/skills/` and `~/.claude/skills/`
|
|
60
|
+
(the common real source of churn). Counted once; the copy is the cut.
|
|
61
|
+
- `wraps-simple-cli` — an MCP server whose command is `git`/`gh`/`npm`/etc.; the Bash tool can
|
|
62
|
+
do that work without the schema overhead.
|
|
63
|
+
- `over-N-lines` — an oversized agent (>200), skill (>400), or rule (>100) worth splitting.
|
|
64
|
+
|
|
65
|
+
## Heuristics are guides, not rules
|
|
66
|
+
|
|
67
|
+
Always surface the **reason** alongside each recommendation so the user can override. The buckets
|
|
68
|
+
force a lazy-load decision per component; they don't make it for you.
|
|
69
|
+
|
|
70
|
+
## Non-goals
|
|
71
|
+
|
|
72
|
+
- **Never auto-apply cuts.** Present the list; the user decides and acts.
|
|
73
|
+
- Not a replacement for manual review.
|
|
74
|
+
- No cross-session token tracking (separate work).
|
|
75
|
+
|
|
76
|
+
## Reference
|
|
77
|
+
|
|
78
|
+
- Logic + contract: `compose/lib/context-budget.js`
|
|
79
|
+
- Tests: `compose/test/context-budget.test.js`
|
|
80
|
+
- Feature: `docs/features/COMP-CTXBUDGET-1/`
|
|
81
|
+
- Source: ECC `affaan-m/everything-claude-code/skills/context-budget/SKILL.md`
|
|
@@ -0,0 +1,459 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* context-budget.js — Audit the session-start loaded surface and budget its token cost.
|
|
3
|
+
*
|
|
4
|
+
* Scans agents, skills, rules, MCP server tool schemas, and the CLAUDE.md chain;
|
|
5
|
+
* estimates per-component tokens; classifies each into always / sometimes / rarely
|
|
6
|
+
* needed; and renders a ranked cut list with estimated reclaim.
|
|
7
|
+
*
|
|
8
|
+
* Read-only. The human reviews and chooses cuts (auto-applying cuts is a non-goal).
|
|
9
|
+
* Backs the `/context-budget` skill — see compose/.claude/skills/context-budget/SKILL.md.
|
|
10
|
+
*
|
|
11
|
+
* Token estimates use a dependency-free ~4-chars-per-token heuristic. They are relative
|
|
12
|
+
* budgeting estimates, NOT billing-accurate; `estimateTokens` is pluggable so a real
|
|
13
|
+
* tokenizer can be swapped in later. COMP-CTXBUDGET-1.
|
|
14
|
+
*/
|
|
15
|
+
|
|
16
|
+
import { readFileSync, existsSync, readdirSync, statSync } from 'fs';
|
|
17
|
+
import { join, basename, dirname } from 'path';
|
|
18
|
+
import { createHash } from 'crypto';
|
|
19
|
+
|
|
20
|
+
const TOKENS_PER_MCP_TOOL = 500;
|
|
21
|
+
const SIMPLE_CLI_COMMANDS = new Set(['git', 'gh', 'npm', 'npx', 'cat', 'ls', 'grep', 'sed', 'awk']);
|
|
22
|
+
|
|
23
|
+
// Flag thresholds (lines) from the plan.
|
|
24
|
+
const FLAG_LINES = { agent: 200, skill: 400, rule: 100 };
|
|
25
|
+
|
|
26
|
+
/**
|
|
27
|
+
* Estimate tokens for a chunk of text. Dependency-free ~4-chars-per-token heuristic.
|
|
28
|
+
* @param {string} text
|
|
29
|
+
* @returns {number}
|
|
30
|
+
*/
|
|
31
|
+
export function estimateTokens(text) {
|
|
32
|
+
if (!text) return 0;
|
|
33
|
+
return Math.ceil(text.length / 4);
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
function readTextSafe(path) {
|
|
37
|
+
try {
|
|
38
|
+
return readFileSync(path, 'utf-8');
|
|
39
|
+
} catch {
|
|
40
|
+
return null;
|
|
41
|
+
}
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
function lineCount(text) {
|
|
45
|
+
if (!text) return 0;
|
|
46
|
+
// Count lines without a trailing-newline off-by-one surprise.
|
|
47
|
+
const n = text.split('\n').length;
|
|
48
|
+
return text.endsWith('\n') ? n - 1 : n;
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
function contentHash(text) {
|
|
52
|
+
return createHash('sha1').update(text || '').digest('hex');
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
function makeComponent(kind, path, label, text, extraFlags = []) {
|
|
56
|
+
const lines = lineCount(text);
|
|
57
|
+
const flags = [...extraFlags];
|
|
58
|
+
const threshold = FLAG_LINES[kind];
|
|
59
|
+
if (threshold && lines > threshold) flags.push(`over-${threshold}-lines`);
|
|
60
|
+
return {
|
|
61
|
+
kind,
|
|
62
|
+
path,
|
|
63
|
+
label,
|
|
64
|
+
lines,
|
|
65
|
+
tokens: estimateTokens(text),
|
|
66
|
+
hash: contentHash(text),
|
|
67
|
+
flags,
|
|
68
|
+
};
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
// ---------- Surface scanners ----------
|
|
72
|
+
|
|
73
|
+
function scanAgents(roots) {
|
|
74
|
+
const out = [];
|
|
75
|
+
for (const root of roots) {
|
|
76
|
+
for (const rel of ['.claude/agents', '.agents', 'agents']) {
|
|
77
|
+
const dir = join(root, rel);
|
|
78
|
+
if (!existsSync(dir)) continue;
|
|
79
|
+
let entries;
|
|
80
|
+
try {
|
|
81
|
+
entries = readdirSync(dir, { withFileTypes: true });
|
|
82
|
+
} catch {
|
|
83
|
+
continue;
|
|
84
|
+
}
|
|
85
|
+
for (const e of entries) {
|
|
86
|
+
if (!e.isFile() || !e.name.endsWith('.md')) continue;
|
|
87
|
+
const p = join(dir, e.name);
|
|
88
|
+
const text = readTextSafe(p);
|
|
89
|
+
if (text == null) continue;
|
|
90
|
+
out.push(makeComponent('agent', p, `agent:${basename(e.name, '.md')}`, text));
|
|
91
|
+
}
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
return out;
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
function scanSkills(roots) {
|
|
98
|
+
const out = [];
|
|
99
|
+
for (const root of roots) {
|
|
100
|
+
for (const rel of ['.claude/skills', 'skills']) {
|
|
101
|
+
const dir = join(root, rel);
|
|
102
|
+
if (!existsSync(dir)) continue;
|
|
103
|
+
let entries;
|
|
104
|
+
try {
|
|
105
|
+
entries = readdirSync(dir, { withFileTypes: true });
|
|
106
|
+
} catch {
|
|
107
|
+
continue;
|
|
108
|
+
}
|
|
109
|
+
for (const e of entries) {
|
|
110
|
+
if (!e.isDirectory()) continue;
|
|
111
|
+
const p = join(dir, e.name, 'SKILL.md');
|
|
112
|
+
const text = readTextSafe(p);
|
|
113
|
+
if (text == null) continue;
|
|
114
|
+
out.push(makeComponent('skill', p, `skill:${e.name}`, text));
|
|
115
|
+
}
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
return out;
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
function scanRules(roots) {
|
|
122
|
+
const out = [];
|
|
123
|
+
const seenPaths = new Set();
|
|
124
|
+
for (const root of roots) {
|
|
125
|
+
for (const rel of ['.claude/rules', 'rules']) {
|
|
126
|
+
const dir = join(root, rel);
|
|
127
|
+
if (!existsSync(dir)) continue;
|
|
128
|
+
walkMdFiles(dir).forEach((p) => {
|
|
129
|
+
if (seenPaths.has(p)) return;
|
|
130
|
+
seenPaths.add(p);
|
|
131
|
+
const text = readTextSafe(p);
|
|
132
|
+
if (text == null) return;
|
|
133
|
+
out.push(makeComponent('rule', p, `rule:${basename(p, '.md')}`, text));
|
|
134
|
+
});
|
|
135
|
+
}
|
|
136
|
+
}
|
|
137
|
+
// Mark content overlap: rules whose first heading line matches another rule's.
|
|
138
|
+
const byHeading = new Map();
|
|
139
|
+
for (const c of out) {
|
|
140
|
+
const head = (readTextSafe(c.path) || '').split('\n')[0].trim();
|
|
141
|
+
if (!head) continue;
|
|
142
|
+
if (byHeading.has(head)) {
|
|
143
|
+
c.flags.push('overlap');
|
|
144
|
+
byHeading.get(head).flags.push('overlap');
|
|
145
|
+
} else {
|
|
146
|
+
byHeading.set(head, c);
|
|
147
|
+
}
|
|
148
|
+
}
|
|
149
|
+
return out;
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
function walkMdFiles(dir) {
|
|
153
|
+
const out = [];
|
|
154
|
+
let entries;
|
|
155
|
+
try {
|
|
156
|
+
entries = readdirSync(dir, { withFileTypes: true });
|
|
157
|
+
} catch {
|
|
158
|
+
return out;
|
|
159
|
+
}
|
|
160
|
+
for (const e of entries) {
|
|
161
|
+
const p = join(dir, e.name);
|
|
162
|
+
if (e.isDirectory()) out.push(...walkMdFiles(p));
|
|
163
|
+
else if (e.isFile() && e.name.endsWith('.md')) out.push(p);
|
|
164
|
+
}
|
|
165
|
+
return out;
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
function scanMcpServers(mcpConfigPath, toolCounts = {}) {
|
|
169
|
+
const out = [];
|
|
170
|
+
if (!mcpConfigPath || !existsSync(mcpConfigPath)) return out;
|
|
171
|
+
let cfg;
|
|
172
|
+
try {
|
|
173
|
+
cfg = JSON.parse(readFileSync(mcpConfigPath, 'utf-8'));
|
|
174
|
+
} catch {
|
|
175
|
+
return out;
|
|
176
|
+
}
|
|
177
|
+
const servers = cfg.mcpServers || {};
|
|
178
|
+
for (const [name, spec] of Object.entries(servers)) {
|
|
179
|
+
const flags = [];
|
|
180
|
+
const cmd = basename(spec.command || '');
|
|
181
|
+
const firstArg = Array.isArray(spec.args) ? spec.args[0] || '' : '';
|
|
182
|
+
if (SIMPLE_CLI_COMMANDS.has(cmd) || SIMPLE_CLI_COMMANDS.has(basename(firstArg))) {
|
|
183
|
+
flags.push('wraps-simple-cli');
|
|
184
|
+
}
|
|
185
|
+
const count = toolCounts[name];
|
|
186
|
+
const hasCount = Number.isFinite(count) && count >= 0;
|
|
187
|
+
let tokens = 0;
|
|
188
|
+
if (hasCount) {
|
|
189
|
+
tokens = TOKENS_PER_MCP_TOOL * count;
|
|
190
|
+
} else {
|
|
191
|
+
flags.push('tool-count-unknown');
|
|
192
|
+
}
|
|
193
|
+
out.push({
|
|
194
|
+
kind: 'mcp-server',
|
|
195
|
+
path: mcpConfigPath,
|
|
196
|
+
label: `mcp-server:${name}`,
|
|
197
|
+
lines: 0,
|
|
198
|
+
tokens,
|
|
199
|
+
hash: contentHash(`mcp:${name}`),
|
|
200
|
+
flags,
|
|
201
|
+
toolCount: hasCount ? count : null,
|
|
202
|
+
});
|
|
203
|
+
}
|
|
204
|
+
return out;
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
/**
|
|
208
|
+
* Resolve the CLAUDE.md chain: home global + every CLAUDE.md from cwd upward to a repo
|
|
209
|
+
* boundary (a dir containing .git) or filesystem root.
|
|
210
|
+
*/
|
|
211
|
+
function claudeMdChain({ home, cwd }) {
|
|
212
|
+
const paths = [];
|
|
213
|
+
if (home) paths.push(join(home, '.claude', 'CLAUDE.md'));
|
|
214
|
+
let dir = cwd;
|
|
215
|
+
const seen = new Set();
|
|
216
|
+
while (dir && !seen.has(dir)) {
|
|
217
|
+
seen.add(dir);
|
|
218
|
+
paths.push(join(dir, 'CLAUDE.md'));
|
|
219
|
+
if (existsSync(join(dir, '.git'))) break;
|
|
220
|
+
const parent = dirname(dir);
|
|
221
|
+
if (parent === dir) break;
|
|
222
|
+
dir = parent;
|
|
223
|
+
}
|
|
224
|
+
const out = [];
|
|
225
|
+
const usedPaths = new Set();
|
|
226
|
+
for (const p of paths) {
|
|
227
|
+
if (usedPaths.has(p)) continue;
|
|
228
|
+
usedPaths.add(p);
|
|
229
|
+
const text = readTextSafe(p);
|
|
230
|
+
if (text == null) continue;
|
|
231
|
+
out.push(makeComponent('claude-md', p, `claude-md:${p}`, text));
|
|
232
|
+
}
|
|
233
|
+
return out;
|
|
234
|
+
}
|
|
235
|
+
|
|
236
|
+
/**
|
|
237
|
+
* Walk every surface into a flat inventory of components.
|
|
238
|
+
* @param {{cwd:string, home?:string, mcpConfigPath?:string, toolCounts?:object}} opts
|
|
239
|
+
* @returns {Array} components
|
|
240
|
+
*/
|
|
241
|
+
export function scanSurface({ cwd, home, mcpConfigPath, toolCounts = {} }) {
|
|
242
|
+
const roots = [home, cwd].filter(Boolean);
|
|
243
|
+
return [
|
|
244
|
+
...scanAgents(roots),
|
|
245
|
+
...scanSkills(roots),
|
|
246
|
+
...scanRules(roots),
|
|
247
|
+
...scanMcpServers(mcpConfigPath, toolCounts),
|
|
248
|
+
...claudeMdChain({ home, cwd }),
|
|
249
|
+
];
|
|
250
|
+
}
|
|
251
|
+
|
|
252
|
+
/**
|
|
253
|
+
* Collapse identical skill copies across surfaces (the real source of churn between
|
|
254
|
+
* compose/.claude/skills and ~/.claude/skills). Keeps the first occurrence; later
|
|
255
|
+
* identical copies are retained but marked `duplicateOf` and zeroed so totals don't
|
|
256
|
+
* double-count. Non-identical same-named skills are both kept.
|
|
257
|
+
*/
|
|
258
|
+
export function dedupeSkills(components) {
|
|
259
|
+
const seen = new Map(); // key: skill label + content hash -> kept component
|
|
260
|
+
return components.map((c) => {
|
|
261
|
+
if (c.kind !== 'skill') return c;
|
|
262
|
+
const key = `${c.label}::${c.hash}`;
|
|
263
|
+
if (seen.has(key)) {
|
|
264
|
+
return { ...c, duplicateOf: seen.get(key).path, tokens: 0, flags: [...c.flags, 'duplicate'] };
|
|
265
|
+
}
|
|
266
|
+
seen.set(key, c);
|
|
267
|
+
return c;
|
|
268
|
+
});
|
|
269
|
+
}
|
|
270
|
+
|
|
271
|
+
/**
|
|
272
|
+
* Classify a component into a budget bucket with an explaining reason.
|
|
273
|
+
* @param {object} component
|
|
274
|
+
* @param {{claudeMdText?:string, projectType?:string}} ctx
|
|
275
|
+
* @returns {{bucket:string, reason:string}}
|
|
276
|
+
*/
|
|
277
|
+
export function classifyComponent(component, ctx = {}) {
|
|
278
|
+
const { claudeMdText = '', projectType = '' } = ctx;
|
|
279
|
+
const flags = component.flags || [];
|
|
280
|
+
const name = component.label.split(':').slice(1).join(':');
|
|
281
|
+
|
|
282
|
+
// Duplicates are always a recommended cut.
|
|
283
|
+
if (component.duplicateOf) {
|
|
284
|
+
return { bucket: 'rarely', reason: `duplicate of ${component.duplicateOf}` };
|
|
285
|
+
}
|
|
286
|
+
|
|
287
|
+
// CLAUDE.md itself and MCP servers backing the project are load-bearing.
|
|
288
|
+
if (component.kind === 'claude-md') {
|
|
289
|
+
return { bucket: 'always', reason: 'CLAUDE.md chain is always loaded' };
|
|
290
|
+
}
|
|
291
|
+
|
|
292
|
+
if (nameReferenced(name, claudeMdText)) {
|
|
293
|
+
return { bucket: 'always', reason: `referenced by name in the CLAUDE.md chain` };
|
|
294
|
+
}
|
|
295
|
+
|
|
296
|
+
// Overlapping rules / unknown-or-CLI-wrapping MCP servers / over-size domain content.
|
|
297
|
+
if (flags.includes('overlap')) {
|
|
298
|
+
return { bucket: 'rarely', reason: 'content overlaps a sibling in the same module' };
|
|
299
|
+
}
|
|
300
|
+
if (component.kind === 'mcp-server' && flags.includes('wraps-simple-cli')) {
|
|
301
|
+
return { bucket: 'rarely', reason: 'wraps a simple CLI the Bash tool can call directly' };
|
|
302
|
+
}
|
|
303
|
+
|
|
304
|
+
if (component.kind === 'mcp-server') {
|
|
305
|
+
// A real server with tools, not referenced by name — load-bearing while configured,
|
|
306
|
+
// but a disable-if-unused candidate (schemas are the heaviest single line items).
|
|
307
|
+
return {
|
|
308
|
+
bucket: 'sometimes',
|
|
309
|
+
reason: 'active MCP server, not referenced in CLAUDE.md — disable in .mcp.json if unused',
|
|
310
|
+
};
|
|
311
|
+
}
|
|
312
|
+
|
|
313
|
+
// Default: domain-specific, not referenced => sometimes (consider lazy-load).
|
|
314
|
+
return { bucket: 'sometimes', reason: 'not referenced in CLAUDE.md — consider on-demand activation' };
|
|
315
|
+
}
|
|
316
|
+
|
|
317
|
+
/**
|
|
318
|
+
* Whole-word "referenced by name" check that tolerates hyphen/space variants.
|
|
319
|
+
* `compose` matches "the compose skill" but NOT "decompose"/"composed";
|
|
320
|
+
* `code-standards` matches "code standards", "code-standards", or "codestandards".
|
|
321
|
+
*/
|
|
322
|
+
export function nameReferenced(name, text) {
|
|
323
|
+
if (!name || !text) return false;
|
|
324
|
+
const n = name.toLowerCase();
|
|
325
|
+
const variants = new Set([n, n.replace(/-/g, ' '), n.replace(/-/g, '')]);
|
|
326
|
+
for (const v of variants) {
|
|
327
|
+
if (!v) continue;
|
|
328
|
+
const esc = v.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
329
|
+
const re = new RegExp(`(^|[^a-z0-9])${esc}([^a-z0-9]|$)`, 'i');
|
|
330
|
+
if (re.test(text)) return true;
|
|
331
|
+
}
|
|
332
|
+
return false;
|
|
333
|
+
}
|
|
334
|
+
|
|
335
|
+
function formatTokens(n) {
|
|
336
|
+
if (n >= 1000) return `${(n / 1000).toFixed(1)}K`;
|
|
337
|
+
return String(n);
|
|
338
|
+
}
|
|
339
|
+
|
|
340
|
+
/**
|
|
341
|
+
* Build the structured report (and rendered text) from a classified inventory.
|
|
342
|
+
* Accepts raw components; classifies internally if a ctx is provided, else expects
|
|
343
|
+
* components already carrying a `bucket`.
|
|
344
|
+
*/
|
|
345
|
+
export function buildReport(components, ctx = {}) {
|
|
346
|
+
// Ensure each component is classified.
|
|
347
|
+
const classified = components.map((c) => {
|
|
348
|
+
if (c.bucket) return c;
|
|
349
|
+
const { bucket, reason } = classifyComponent(c, ctx);
|
|
350
|
+
return { ...c, bucket, reason };
|
|
351
|
+
});
|
|
352
|
+
|
|
353
|
+
const buckets = { always: [], sometimes: [], rarely: [] };
|
|
354
|
+
for (const c of classified) buckets[c.bucket].push(c);
|
|
355
|
+
|
|
356
|
+
const totalTokens = classified.reduce((a, c) => a + c.tokens, 0);
|
|
357
|
+
|
|
358
|
+
// Top reclaims: highest-token candidates among sometimes+rarely.
|
|
359
|
+
const topReclaims = [...buckets.sometimes, ...buckets.rarely]
|
|
360
|
+
.filter((c) => c.tokens > 0)
|
|
361
|
+
.sort((a, b) => b.tokens - a.tokens)
|
|
362
|
+
.slice(0, 5);
|
|
363
|
+
|
|
364
|
+
const text = renderReport({ buckets, totalTokens, topReclaims });
|
|
365
|
+
return { totalTokens, buckets, topReclaims, classified, text };
|
|
366
|
+
}
|
|
367
|
+
|
|
368
|
+
function renderBucketLines(list) {
|
|
369
|
+
return list
|
|
370
|
+
.slice()
|
|
371
|
+
.sort((a, b) => b.tokens - a.tokens)
|
|
372
|
+
.map((c) => {
|
|
373
|
+
const flagStr = c.flags && c.flags.length ? ` [${c.flags.join(', ')}]` : '';
|
|
374
|
+
return ` - ${c.label} (${c.lines} lines, ~${formatTokens(c.tokens)} tokens) — ${c.reason}${flagStr}`;
|
|
375
|
+
})
|
|
376
|
+
.join('\n');
|
|
377
|
+
}
|
|
378
|
+
|
|
379
|
+
function bucketTotal(list) {
|
|
380
|
+
return list.reduce((a, c) => a + c.tokens, 0);
|
|
381
|
+
}
|
|
382
|
+
|
|
383
|
+
function renderReport({ buckets, totalTokens, topReclaims }) {
|
|
384
|
+
const lines = [];
|
|
385
|
+
lines.push(`CONTEXT BUDGET — current load: ~${formatTokens(totalTokens)} tokens`);
|
|
386
|
+
lines.push('');
|
|
387
|
+
lines.push(`ALWAYS NEEDED (keep, total ~${formatTokens(bucketTotal(buckets.always))} tokens)`);
|
|
388
|
+
lines.push(renderBucketLines(buckets.always) || ' (none)');
|
|
389
|
+
lines.push('');
|
|
390
|
+
lines.push(
|
|
391
|
+
`SOMETIMES NEEDED (consider lazy-load, total ~${formatTokens(bucketTotal(buckets.sometimes))} tokens)`
|
|
392
|
+
);
|
|
393
|
+
lines.push(renderBucketLines(buckets.sometimes) || ' (none)');
|
|
394
|
+
lines.push('');
|
|
395
|
+
lines.push(`RARELY NEEDED (recommend cut, total ~${formatTokens(bucketTotal(buckets.rarely))} tokens)`);
|
|
396
|
+
lines.push(renderBucketLines(buckets.rarely) || ' (none)');
|
|
397
|
+
lines.push('');
|
|
398
|
+
lines.push('TOP 5 RECLAIMS:');
|
|
399
|
+
if (topReclaims.length === 0) {
|
|
400
|
+
lines.push(' (none)');
|
|
401
|
+
} else {
|
|
402
|
+
topReclaims.forEach((c, i) => {
|
|
403
|
+
lines.push(` ${i + 1}. ${c.label} (~${formatTokens(c.tokens)} tokens) — ${c.reason}`);
|
|
404
|
+
});
|
|
405
|
+
}
|
|
406
|
+
const potential = bucketTotal(buckets.sometimes) + bucketTotal(buckets.rarely);
|
|
407
|
+
lines.push('');
|
|
408
|
+
lines.push(`Potential reclaim if all sometimes+rarely cut: ~${formatTokens(potential)} tokens`);
|
|
409
|
+
return lines.join('\n');
|
|
410
|
+
}
|
|
411
|
+
|
|
412
|
+
/**
|
|
413
|
+
* Top-level orchestrator: scan → dedupe → classify → report.
|
|
414
|
+
* @param {{cwd:string, home?:string, mcpConfigPath?:string, toolCounts?:object}} opts
|
|
415
|
+
*/
|
|
416
|
+
export function auditContextBudget({ cwd, home, mcpConfigPath, toolCounts = {} }) {
|
|
417
|
+
const resolvedMcp = mcpConfigPath || (cwd ? join(cwd, '.mcp.json') : undefined);
|
|
418
|
+
const raw = scanSurface({ cwd, home, mcpConfigPath: resolvedMcp, toolCounts });
|
|
419
|
+
const deduped = dedupeSkills(raw);
|
|
420
|
+
|
|
421
|
+
// Build the CLAUDE.md context text used for "referenced by name" classification.
|
|
422
|
+
const claudeMdText = deduped
|
|
423
|
+
.filter((c) => c.kind === 'claude-md')
|
|
424
|
+
.map((c) => readTextSafe(c.path) || '')
|
|
425
|
+
.join('\n');
|
|
426
|
+
|
|
427
|
+
return buildReport(deduped, { claudeMdText, projectType: 'node' });
|
|
428
|
+
}
|
|
429
|
+
|
|
430
|
+
// ---------- CLI guard ----------
|
|
431
|
+
export function parseToolCounts(arg) {
|
|
432
|
+
const out = {};
|
|
433
|
+
if (!arg) return out;
|
|
434
|
+
for (const pair of arg.split(',')) {
|
|
435
|
+
const [name, n] = pair.split('=');
|
|
436
|
+
if (!name || n == null || n.trim() === '') continue;
|
|
437
|
+
const num = Number(n);
|
|
438
|
+
if (Number.isFinite(num) && num >= 0) out[name.trim()] = num;
|
|
439
|
+
}
|
|
440
|
+
return out;
|
|
441
|
+
}
|
|
442
|
+
|
|
443
|
+
function isMainModule() {
|
|
444
|
+
try {
|
|
445
|
+
return process.argv[1] && import.meta.url === `file://${process.argv[1]}`;
|
|
446
|
+
} catch {
|
|
447
|
+
return false;
|
|
448
|
+
}
|
|
449
|
+
}
|
|
450
|
+
|
|
451
|
+
if (isMainModule()) {
|
|
452
|
+
const args = process.argv.slice(2);
|
|
453
|
+
const cwd = args.find((a) => !a.startsWith('--')) || process.cwd();
|
|
454
|
+
const home = process.env.HOME;
|
|
455
|
+
const tcArg = args.find((a) => a.startsWith('--tool-counts='));
|
|
456
|
+
const toolCounts = parseToolCounts(tcArg ? tcArg.split('=').slice(1).join('=') : '');
|
|
457
|
+
const report = auditContextBudget({ cwd, home, toolCounts });
|
|
458
|
+
process.stdout.write(report.text + '\n');
|
|
459
|
+
}
|
package/lib/feature-validator.js
CHANGED
|
@@ -32,7 +32,7 @@ import fs from 'node:fs';
|
|
|
32
32
|
import path from 'node:path';
|
|
33
33
|
import { fileURLToPath } from 'node:url';
|
|
34
34
|
import { FEATURE_CODE_RE_STRICT, validateCode } from './feature-code.js';
|
|
35
|
-
import { parseRoadmap } from './roadmap-parser.js';
|
|
35
|
+
import { parseRoadmap, splitRoadmapCells } from './roadmap-parser.js';
|
|
36
36
|
import { listFeatures, readFeature } from './feature-json.js';
|
|
37
37
|
import { loadExternalPrefixes } from './project-paths.js';
|
|
38
38
|
import { checkRoundtrip, LOSSY_LABELS } from './roadmap-roundtrip.js';
|
|
@@ -161,7 +161,9 @@ export function loadValidationContext(cwd, options = {}) {
|
|
|
161
161
|
const rowMatch = rawLine.match(/^\|(.+)\|\s*$/);
|
|
162
162
|
if (!rowMatch) { inTable = false; sawSeparator = false; continue; }
|
|
163
163
|
|
|
164
|
-
|
|
164
|
+
// Escaped-pipe-aware split (COMP-MCP-VALIDATE-4): a `\|` in a description
|
|
165
|
+
// cell must not shift status-column detection and read prose as the status.
|
|
166
|
+
const cols = splitRoadmapCells(rawLine);
|
|
165
167
|
|
|
166
168
|
// Detect header row by column names. Recognize common column-name variants
|
|
167
169
|
// (feature/code/item/name) and (status/state) so non-canonical tables that
|
|
@@ -19,6 +19,7 @@ import { join, resolve, dirname } from 'path';
|
|
|
19
19
|
import { fileURLToPath } from 'url';
|
|
20
20
|
import { SchemaValidator } from '../server/schema-validator.js';
|
|
21
21
|
import { FEATURE_CODE_RE_STRICT } from './feature-code.js';
|
|
22
|
+
import { splitRoadmapCells } from './roadmap-parser.js';
|
|
22
23
|
|
|
23
24
|
const __dirname = dirname(fileURLToPath(import.meta.url));
|
|
24
25
|
const SCHEMA_PATH = resolve(__dirname, '../contracts/feature-json.schema.json');
|
|
@@ -108,7 +109,9 @@ export function scanRoadmapRows(roadmapPath) {
|
|
|
108
109
|
if (/^##\s+/.test(rawLine)) { inTable = false; sawSeparator = false; codeIdx = statusIdx = -1; continue; }
|
|
109
110
|
const rowMatch = rawLine.match(/^\|(.+)\|\s*$/);
|
|
110
111
|
if (!rowMatch) { inTable = false; sawSeparator = false; continue; }
|
|
111
|
-
|
|
112
|
+
// Escaped-pipe-aware split (COMP-MCP-VALIDATE-4): keep column detection stable
|
|
113
|
+
// when a description cell contains `\|`.
|
|
114
|
+
const cols = splitRoadmapCells(rawLine);
|
|
112
115
|
const lower = cols.map((c) => c.toLowerCase());
|
|
113
116
|
const featureColIdx = lower.findIndex((c) => ['feature', 'code', 'item', 'name'].includes(c));
|
|
114
117
|
const statusColIdx = lower.findIndex((c) => ['status', 'state'].includes(c));
|
package/lib/roadmap-parser.js
CHANGED
|
@@ -21,6 +21,24 @@ const MILESTONE_HEADING_RE = /^###\s+(.+?)(?:\s*:\s*(.+))?$/;
|
|
|
21
21
|
const TABLE_ROW_RE = /^\|(.+)\|$/;
|
|
22
22
|
const FENCE_RE = /^```/;
|
|
23
23
|
|
|
24
|
+
/**
|
|
25
|
+
* Split a markdown table row into trimmed cells, honoring escaped pipes.
|
|
26
|
+
* Splits on UNESCAPED `|` only (`/(?<!\\)\|/`), drops the leading/trailing empty
|
|
27
|
+
* cells from the outer pipes, and unescapes `\|` → `|` in each cell. Symmetric
|
|
28
|
+
* with `escCell()` in roadmap-gen.js. For pipe-free rows this is identical to a
|
|
29
|
+
* naive `split('|')`.
|
|
30
|
+
*
|
|
31
|
+
* The canonical row splitter — every ROADMAP-row parse site (validator,
|
|
32
|
+
* write-guard, this parser) must use it so a `\|` in a description cell can never
|
|
33
|
+
* shift status-column detection (COMP-MCP-VALIDATE-4).
|
|
34
|
+
*
|
|
35
|
+
* @param {string} rawLine - A full table row line, e.g. "| a | b \\| c | PLANNED |"
|
|
36
|
+
* @returns {string[]} trimmed, unescaped cells
|
|
37
|
+
*/
|
|
38
|
+
export function splitRoadmapCells(rawLine) {
|
|
39
|
+
return rawLine.trim().split(/(?<!\\)\|/).slice(1, -1).map((c) => c.trim().replace(/\\\|/g, '|'));
|
|
40
|
+
}
|
|
41
|
+
|
|
24
42
|
/**
|
|
25
43
|
* @typedef {{ code: string, description: string, status: string, phaseId: string, position: number }} FeatureEntry
|
|
26
44
|
*/
|
|
@@ -112,10 +130,8 @@ export function parseRoadmap(text) {
|
|
|
112
130
|
continue;
|
|
113
131
|
}
|
|
114
132
|
|
|
115
|
-
//
|
|
116
|
-
|
|
117
|
-
// identical to split('|') (no backslashes, no lookbehind matches).
|
|
118
|
-
const cells = trimmed.split(/(?<!\\)\|/).slice(1, -1).map(c => c.trim().replace(/\\\|/g, '|'));
|
|
133
|
+
// Escaped-pipe-aware cell split (the canonical splitter — see splitRoadmapCells).
|
|
134
|
+
const cells = splitRoadmapCells(trimmed);
|
|
119
135
|
|
|
120
136
|
// Skip separator rows (|---|---|---|)
|
|
121
137
|
if (cells.every(c => /^[-:]+$/.test(c))) {
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@smartmemory/compose",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.23-beta",
|
|
4
4
|
"description": "Structured AI dev pipeline — goal-to-product orchestration with gates, iteration loops, and feature lifecycle management.",
|
|
5
5
|
"author": "SmartMemory",
|
|
6
6
|
"license": "MIT",
|
|
@@ -19,11 +19,11 @@
|
|
|
19
19
|
"dev:client": "vite",
|
|
20
20
|
"build": "vite build",
|
|
21
21
|
"preview": "vite preview",
|
|
22
|
-
"test": "node --import ./test/suppress-expected-drift.js --test test/*.test.js test/comp-obs-branch/*.test.js test/integration/*.test.js && npm run test:ui && npm run test:tracker",
|
|
22
|
+
"test": "node --import ./test/suppress-expected-drift.js --test --test-timeout=120000 test/*.test.js test/comp-obs-branch/*.test.js test/integration/*.test.js && npm run test:ui && npm run test:tracker",
|
|
23
23
|
"test:ui": "vitest run",
|
|
24
24
|
"test:tracker": "vitest run --config vitest.tracker.config.js",
|
|
25
|
-
"test:integration": "node --test test/integration/*.test.js",
|
|
26
|
-
"test:wave-6": "node --test test/wave-6-integration.test.js test/wave-6-contract-compliance.test.js",
|
|
25
|
+
"test:integration": "node --test --test-timeout=120000 test/integration/*.test.js",
|
|
26
|
+
"test:wave-6": "node --test --test-timeout=120000 test/wave-6-integration.test.js test/wave-6-contract-compliance.test.js",
|
|
27
27
|
"prepublishOnly": "npm run build"
|
|
28
28
|
},
|
|
29
29
|
"keywords": [
|