context-doctor 0.10.0 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -34,7 +34,7 @@ Findings (4)
34
34
  npx context-doctor install
35
35
  ```
36
36
 
37
- That single command is also all it takes to **set up context-doctor on anyone else's machine**. Prefer a global install, or want the unreleased `main`? Both work (Node 18+):
37
+ That single command is also all it takes to **set up context-doctor on anyone else's machine**. Prefer a global install, or want the unreleased `main`? Both work (Node 20+):
38
38
 
39
39
  ```bash
40
40
  npm install -g context-doctor && context-doctor install
@@ -89,10 +89,10 @@ Practical upshot: a developer who only wants cheaper, faster API calls never tou
89
89
  |---|---|
90
90
  | `context-doctor install` / `uninstall` | Wire (or remove) everything: MCP for Claude Desktop/Code/Cursor, the Agent Skill, the every-prompt hook |
91
91
  | `context-doctor analyze <file>` | Profile a conversation: token breakdown, findings, cost + latency estimates. `--fail-over-budget` exits 1 on a breach, for CI |
92
- | `context-doctor optimize <file>` | Apply the safe fixes; `--strategy prune-history` for consented lossy compaction |
92
+ | `context-doctor optimize <file>` | Apply the safe fixes; add `--strategy trim-tool-calls` for big inline file writes, `--strategy prune-history` for consented lossy compaction |
93
93
  | `context-doctor session [file]` | Profile a Claude Code session: live context, findings, **measured tokens and prompt-cache economics**. Also reads ChatGPT data exports (`conversations.json`) |
94
94
  | `context-doctor cursor [--list]` | Profile a chat from Cursor's local history (both storage formats) |
95
- | `context-doctor report` | Machine-wide impact report: exact proxy savings, hook activity, recoverable waste in recent sessions |
95
+ | `context-doctor report` | Machine-wide impact report (proxy savings persist across restarts): exact proxy savings, hook activity, recoverable waste in recent sessions |
96
96
  | `context-doctor proxy` | Always-on local proxy that optimizes every Anthropic/OpenAI API request in flight (`/stats` for cumulative savings) |
97
97
  | `context-doctor watch [file]` | Live monitor of a growing session/agent trace: token/cost line per change, findings as they appear |
98
98
  | `context-doctor doctor` | Self-check the whole installation — one pasteable ✓/✗ diagnosis with fixes |
@@ -261,8 +261,11 @@ const { conversation, tokensBefore, tokensAfter } = optimizeConversation(chatJso
261
261
  | `dedupe` — replace repeated content with a reference | No | ✅ |
262
262
  | `trim-tool-results` — truncate stale tool outputs | Mostly no | ✅ |
263
263
  | `strip-base64` — remove inline binary blobs | No (for the model) | ✅ |
264
+ | `trim-tool-calls` — shrink the arguments of calls that already ran | Mostly no | opt-in |
264
265
  | `prune-history` — collapse old turns into a stub for summarization | Yes | opt-in |
265
266
 
267
+ `trim-tool-calls` is the big one for agent sessions. Writing a file through a tool call puts the entire file in context permanently, so in file-heavy work the calls outweigh every tool result combined — on a real 278k-token session, the default set reached 248k and adding `trim-tool-calls` reached 102k. It is opt-in because it edits what the model itself wrote.
268
+
266
269
  Everything the optimizer does is inspectable: it prints exactly which messages changed and how many tokens each change saved.
267
270
 
268
271
  **Summarization without an API key:** when `prune-history` runs through the MCP tools, context-doctor hands a digest of the pruned turns back to the model that called it (the Claude/GPT already running in your app) and asks *it* to write the replacement summary — LLM-quality compaction, zero extra cost, no keys.
@@ -319,6 +322,16 @@ npx context-doctor analyze conversation.json --fail-over-budget
319
322
 
320
323
  Exits 1 when the `.contextdoctorrc` budget is breached, so a pull request can be gated on context size the same way it is gated on tests.
321
324
 
325
+ ## Sharing a profile safely
326
+
327
+ A profile quotes message previews and file paths, so pasting one into an issue pastes fragments of real work. `--redact` keeps every number and the finding structure but replaces content with `[redacted]` and masks paths:
328
+
329
+ ```bash
330
+ npx context-doctor session --redact
331
+ ```
332
+
333
+ `context-doctor doctor` is safe to paste as-is: it reports integration status, never conversation content.
334
+
322
335
  ## Performance: what context-doctor itself costs
323
336
 
324
337
  A tool that promises speed must be near-free. Measured overhead per touchpoint:
@@ -329,6 +342,7 @@ A tool that promises speed must be near-free. Measured overhead per touchpoint:
329
342
  | MCP server | Spawned once per app session | Tools run only when called; standing instructions cost **~110 tokens per conversation** — deliberately terse |
330
343
  | Proxy | Per API request | ~1–3ms of CPU (parse → optimize → re-serialize) against typical model latencies of hundreds of ms; responses stream through chunk-by-chunk, never buffered |
331
344
  | Skill | Loads only when relevant | ~1k tokens while active; its always-present description is ~60 tokens |
345
+ | Profiling a session | On demand, and on hook growth events | ~160ms for an 8.5MB / 1,855-message transcript (near-duplicate pairs that cannot clear the similarity bar are skipped without comparison) |
332
346
  | CLI / library | Only when you run it | Not in any hot path |
333
347
 
334
348
  Net effect is strongly negative overhead: the tokens these touchpoints save on every subsequent call dwarf what they cost.
package/dist/blob.d.ts ADDED
@@ -0,0 +1,15 @@
1
+ /**
2
+ * Inline base64 detection, shared by the profiler and the optimizer.
3
+ *
4
+ * A charset match alone is not enough: `"a".repeat(5000)` is valid base64
5
+ * alphabet, and so are long hex digests, IDs and minified identifiers. Calling
6
+ * those "base64" makes the profiler wrong and — far worse — makes the
7
+ * `strip-base64` strategy DELETE real content. So a candidate must also look
8
+ * like encoded binary: spanning most of the 64-symbol alphabet, with no single
9
+ * character dominating.
10
+ */
11
+ export declare const BASE64_PLACEHOLDER = "[context-doctor: base64 blob removed \u2014 use file/image APIs instead]";
12
+ /** True when the text carries at least one inline base64 blob. */
13
+ export declare function hasBase64Blob(text: string): boolean;
14
+ /** Replace real base64 blobs with a placeholder, leaving lookalikes intact. */
15
+ export declare function stripBase64Blobs(text: string): string;
package/dist/blob.js ADDED
@@ -0,0 +1,44 @@
1
+ /**
2
+ * Inline base64 detection, shared by the profiler and the optimizer.
3
+ *
4
+ * A charset match alone is not enough: `"a".repeat(5000)` is valid base64
5
+ * alphabet, and so are long hex digests, IDs and minified identifiers. Calling
6
+ * those "base64" makes the profiler wrong and — far worse — makes the
7
+ * `strip-base64` strategy DELETE real content. So a candidate must also look
8
+ * like encoded binary: spanning most of the 64-symbol alphabet, with no single
9
+ * character dominating.
10
+ */
11
+ /** Runs of base64 alphabet long enough to be worth reporting (~375 bytes+). */
12
+ const BASE64_RE = /(?:data:[\w/+.-]+;base64,)?[A-Za-z0-9+/]{500,}={0,2}/g;
13
+ export const BASE64_PLACEHOLDER = "[context-doctor: base64 blob removed — use file/image APIs instead]";
14
+ /** Distinct characters a genuine base64 payload is expected to span. */
15
+ const MIN_DISTINCT_CHARS = 24;
16
+ /** Above this share for one character, the run is padding or repetition. */
17
+ const MAX_SINGLE_CHAR_SHARE = 0.35;
18
+ function isBase64Blob(candidate) {
19
+ // An explicit data: URI declares its own encoding — no need to guess.
20
+ if (candidate.startsWith("data:"))
21
+ return true;
22
+ const counts = new Map();
23
+ for (const ch of candidate)
24
+ counts.set(ch, (counts.get(ch) ?? 0) + 1);
25
+ if (counts.size < MIN_DISTINCT_CHARS)
26
+ return false;
27
+ let max = 0;
28
+ for (const n of counts.values())
29
+ if (n > max)
30
+ max = n;
31
+ return max / candidate.length < MAX_SINGLE_CHAR_SHARE;
32
+ }
33
+ /** True when the text carries at least one inline base64 blob. */
34
+ export function hasBase64Blob(text) {
35
+ for (const m of text.matchAll(BASE64_RE)) {
36
+ if (isBase64Blob(m[0]))
37
+ return true;
38
+ }
39
+ return false;
40
+ }
41
+ /** Replace real base64 blobs with a placeholder, leaving lookalikes intact. */
42
+ export function stripBase64Blobs(text) {
43
+ return text.replace(BASE64_RE, (m) => (isBase64Blob(m) ? BASE64_PLACEHOLDER : m));
44
+ }
package/dist/cli.js CHANGED
@@ -64,12 +64,15 @@ Options:
64
64
  --model <name> Model name for window-size math (e.g. claude-sonnet-5, gpt-4o)
65
65
  --exact (analyze) Add an exact token count: Anthropic count-tokens API for
66
66
  Claude models (needs ANTHROPIC_API_KEY), tiktoken for GPT (if installed)
67
+ --redact Mask message previews and file paths in the report, so it can be
68
+ shared in a bug report without leaking conversation content
67
69
  --json Machine-readable output
68
70
  --fail-over-budget (analyze/session) Exit 1 when the .contextdoctorrc budget is
69
71
  exceeded — lets CI gate a pull request on context size
70
72
  --out <file> (optimize) Write result to file instead of stdout
71
73
  --strategy <id> (optimize) Strategy to run; repeatable.
72
- Available: dedupe, trim-tool-results, strip-base64, prune-history
74
+ Available: dedupe, trim-tool-results, trim-tool-calls, strip-base64,
75
+ prune-history
73
76
  Default: dedupe, trim-tool-results, strip-base64 (lossless-ish set)
74
77
  --keep-recent <n> (optimize) Messages at the tail to leave untouched (default 6)
75
78
  --max-tool-tokens <n> (optimize) Token budget for trimmed tool results (default 300)
@@ -89,7 +92,7 @@ Examples:
89
92
  export OPENAI_BASE_URL=http://localhost:8787/v1
90
93
  `;
91
94
  function parseArgs(argv) {
92
- const args = { json: false, strategies: [], list: false, exact: false, failOverBudget: false };
95
+ const args = { json: false, strategies: [], list: false, exact: false, redact: false, failOverBudget: false };
93
96
  const positional = [];
94
97
  for (let i = 0; i < argv.length; i++) {
95
98
  const a = argv[i];
@@ -107,6 +110,9 @@ function parseArgs(argv) {
107
110
  case "--exact":
108
111
  args.exact = true;
109
112
  break;
113
+ case "--redact":
114
+ args.redact = true;
115
+ break;
110
116
  case "--fail-over-budget":
111
117
  args.failOverBudget = true;
112
118
  break;
@@ -228,7 +234,7 @@ function main() {
228
234
  }
229
235
  else {
230
236
  console.log(`Cursor chat: ${chat.title ?? "(untitled)"}\nId: ${chat.composerId}\n`);
231
- console.log(renderProfile(profile));
237
+ console.log(renderProfile(profile, { redact: args.redact }));
232
238
  if (parsed.reportedInputTokens) {
233
239
  console.log("");
234
240
  console.log(`Measured context (reported by the API on the last request): ${parsed.reportedInputTokens} tokens.\n` +
@@ -275,7 +281,7 @@ function main() {
275
281
  console.log(`Note: ${parsed.compactedAway} earlier message(s) were compacted away and are NOT counted below — this is the live context the model still sees.`);
276
282
  }
277
283
  console.log("");
278
- console.log(renderProfile(profile));
284
+ console.log(renderProfile(profile, { redact: args.redact }));
279
285
  if (parsed.reportedInputTokens) {
280
286
  console.log("");
281
287
  console.log(`Measured context (reported by the API on the last request): ${parsed.reportedInputTokens} tokens.\n` +
@@ -339,7 +345,7 @@ function main() {
339
345
  if (args.command === "analyze") {
340
346
  const loaded = loadConfig(process.cwd(), (m) => console.error(`context-doctor: ${m}`));
341
347
  const profile = profileConversation(parseConversation(input), args.model ?? loaded.config.model);
342
- console.log(args.json ? JSON.stringify(profile, null, 2) : renderProfile(profile));
348
+ console.log(args.json ? JSON.stringify(profile, null, 2) : renderProfile(profile, { redact: args.redact }));
343
349
  if (!args.json)
344
350
  applyBudgetGate(printBudgetStatus(profile, loaded), args.failOverBudget);
345
351
  else
package/dist/doctor.js CHANGED
@@ -31,6 +31,16 @@ function checkMcpEntry(appName, configPath) {
31
31
  if (target && !existsSync(target)) {
32
32
  return { label: appName, status: "fail", detail: `MCP entry points at missing file ${target} — re-run: context-doctor install` };
33
33
  }
34
+ // Configs written before 0.12 pinned the exact node binary, which a Node
35
+ // upgrade removes; the app then silently loses the tools.
36
+ const cmd = String(entry.command ?? "");
37
+ if (cmd.includes("/") && !existsSync(cmd)) {
38
+ return {
39
+ label: appName,
40
+ status: "fail",
41
+ detail: `MCP command ${cmd} no longer exists (a Node upgrade moves version-pinned paths) — re-run: context-doctor install`,
42
+ };
43
+ }
34
44
  return { label: appName, status: "ok", detail: `MCP wired (${entry.command === "npx" ? "npx, tracks npm releases" : "local build"})` };
35
45
  }
36
46
  catch (e) {
package/dist/impact.js CHANGED
@@ -68,17 +68,24 @@ export async function buildImpactReport(proxyPort = 8787) {
68
68
  optimizeUsd += inputCostUsd(e.saved, pricing);
69
69
  }
70
70
  const proxy = await fetchProxyStats(proxyPort);
71
- const proxySaved = proxy?.tokensSaved ?? 0;
71
+ // Persisted checkpoints cover proxy runs that have since exited; the live
72
+ // process reports whatever it has not checkpointed yet.
73
+ const proxyEvents = ledger.filter((e) => e.ev === "proxy");
74
+ const proxyHistoric = proxyEvents.reduce((s, e) => s + (e.saved ?? 0), 0);
75
+ const proxySaved = proxyHistoric + (proxy?.tokensSaved ?? 0);
72
76
  // -- Headline: what context-doctor has saved ----------------------------------
73
77
  const totalSaved = proxySaved + optimizeSaved + totalReduction;
74
78
  lines.push("Tokens context-doctor saved (measured)");
75
79
  lines.push("─".repeat(56));
76
80
  lines.push(`TOTAL: ~${formatTokens(totalSaved)} tokens`);
77
81
  if (proxy) {
78
- lines.push(` · proxy (exact, current run): ${formatTokens(proxySaved)} across ${proxy.optimizedRequests}/${proxy.requests} requests ${formatUsd(proxy.estUsdSaved)}`);
82
+ lines.push(` · proxy (exact): ${formatTokens(proxySaved)} ${formatTokens(proxyHistoric)} from earlier runs, ` +
83
+ `${formatTokens(proxy.tokensSaved)} live across ${proxy.optimizedRequests}/${proxy.requests} requests ≈ ${formatUsd(proxy.estUsdSaved)}`);
79
84
  }
80
85
  else {
81
- lines.push(` · proxy: not running on :${proxyPort} (its exact savings appear here while it runs)`);
86
+ lines.push(proxyHistoric > 0
87
+ ? ` · proxy (exact, from ${proxyEvents.length} earlier run checkpoint(s)): ${formatTokens(proxyHistoric)} — not running now`
88
+ : ` · proxy: not running on :${proxyPort} (its exact savings appear here once it runs)`);
82
89
  }
83
90
  const familyNote = [...savedByFamily.entries()]
84
91
  .filter(([, v]) => v > 0)
package/dist/install.js CHANGED
@@ -8,7 +8,7 @@
8
8
  */
9
9
  import { existsSync, mkdirSync, readFileSync, writeFileSync, copyFileSync, rmSync } from "node:fs";
10
10
  import { homedir, platform } from "node:os";
11
- import { dirname, join } from "node:path";
11
+ import { delimiter, dirname, join, sep } from "node:path";
12
12
  import { fileURLToPath } from "node:url";
13
13
  function claudeDesktopConfigPath() {
14
14
  switch (platform()) {
@@ -28,16 +28,22 @@ function targets() {
28
28
  ];
29
29
  }
30
30
  /**
31
- * The server command to write into configs. When running from a published
32
- * install, npx keeps it auto-updating; from a local checkout, point at the
33
- * built file directly so it works before the package is on npm.
31
+ * The server command to write into configs.
32
+ *
33
+ * Never write `process.execPath`: on Homebrew, nvm and asdf that is a
34
+ * VERSION-PINNED path (…/node/25.6.0/bin/node), so the next Node upgrade
35
+ * silently breaks every config we wrote — the apps simply stop showing the
36
+ * tools, with no error to explain why. `node` from PATH survives upgrades.
37
+ *
38
+ * Installed from npm → npx, which also picks up package updates. Local
39
+ * checkout → the built file, so the repo works before/without publishing.
34
40
  */
35
41
  function serverEntry() {
36
42
  const selfDir = dirname(fileURLToPath(import.meta.url));
37
43
  const localMcp = join(selfDir, "mcp.js");
38
- const runningFromNpx = (process.env.npm_execpath ?? "").includes("npx") || selfDir.includes("_npx");
39
- if (!runningFromNpx && existsSync(localMcp)) {
40
- return { command: process.execPath, args: [localMcp] };
44
+ const fromPackage = selfDir.includes(`${sep}node_modules${sep}`) || selfDir.includes("_npx");
45
+ if (!fromPackage && existsSync(localMcp)) {
46
+ return { command: "node", args: [localMcp] };
41
47
  }
42
48
  return { command: "npx", args: ["-y", "context-doctor-mcp"] };
43
49
  }
@@ -57,17 +63,68 @@ function writeJsonWithBackup(path, data) {
57
63
  copyFileSync(path, path + ".context-doctor.backup");
58
64
  writeFileSync(path, JSON.stringify(data, null, 2));
59
65
  }
60
- /** Shell command used for the Claude Code every-prompt hook. */
66
+ /**
67
+ * Find an executable on PATH without shelling out (works on Windows too).
68
+ * Used to prefer a global `context-doctor` install for the hook: a stable
69
+ * location that survives both package updates and Node upgrades.
70
+ */
71
+ function binOnPath(name) {
72
+ const exts = platform() === "win32" ? [".cmd", ".exe", ".bat", ""] : [""];
73
+ for (const dir of (process.env.PATH ?? "").split(delimiter)) {
74
+ if (!dir)
75
+ continue;
76
+ for (const ext of exts) {
77
+ const candidate = join(dir, name + ext);
78
+ if (existsSync(candidate))
79
+ return candidate;
80
+ }
81
+ }
82
+ return null;
83
+ }
84
+ /**
85
+ * Shell command used for the Claude Code every-prompt hook.
86
+ *
87
+ * The hook runs on EVERY prompt, so the command must be both fast and durable.
88
+ * In preference order:
89
+ * 1. a local checkout's built cli.js — absolute, stable, zero resolution cost;
90
+ * 2. a global `context-doctor` binary on PATH — same, for npm -g installs;
91
+ * 3. `npx -y context-doctor hook` — last resort.
92
+ *
93
+ * Critically, a path inside npx's `_npx` cache is NEVER written: npm garbage-
94
+ * collects that directory, and the hook would then fail silently on every
95
+ * prompt. `node` (not process.execPath) keeps it alive across Node upgrades.
96
+ */
61
97
  function hookCommand() {
62
98
  const selfDir = dirname(fileURLToPath(import.meta.url));
63
99
  const localCli = join(selfDir, "cli.js");
64
- const runningFromNpx = (process.env.npm_execpath ?? "").includes("npx") || selfDir.includes("_npx");
65
- if (!runningFromNpx && existsSync(localCli)) {
66
- return `"${process.execPath}" "${localCli}" hook`;
67
- }
100
+ const ephemeral = selfDir.includes("_npx");
101
+ if (!ephemeral && existsSync(localCli))
102
+ return `node "${localCli}" hook`;
103
+ const global = binOnPath("context-doctor");
104
+ if (global)
105
+ return `"${global}" hook`;
68
106
  return "npx -y context-doctor hook";
69
107
  }
108
+ /** True when the hook had to fall back to npx — worth telling the user. */
109
+ function hookUsesNpx() {
110
+ return hookCommand().startsWith("npx ");
111
+ }
70
112
  const HOOK_MARKER = "context-doctor";
113
+ /**
114
+ * Is this settings.json hook entry ours?
115
+ *
116
+ * Usually the command contains "context-doctor" (npx form, or a path through
117
+ * the package directory). A repo cloned into a differently-named folder does
118
+ * not, so a command ending in `cli.js hook` counts too — specific enough not
119
+ * to claim an unrelated hook.
120
+ */
121
+ function isOurHookEntry(entry) {
122
+ const raw = JSON.stringify(entry ?? "");
123
+ if (raw.includes(HOOK_MARKER))
124
+ return true;
125
+ const command = String(entry?.hooks?.[0]?.command ?? "");
126
+ return /(cli\.js|context-doctor(\.cmd|\.exe|\.bat)?)"?\s+hook\s*$/.test(command);
127
+ }
71
128
  /**
72
129
  * Register the UserPromptSubmit hook in ~/.claude/settings.json so EVERY
73
130
  * Claude Code query gets a context-size check. Idempotent.
@@ -79,12 +136,28 @@ function installHook() {
79
136
  const settings = readJson(settingsPath);
80
137
  settings.hooks = settings.hooks ?? {};
81
138
  const entries = settings.hooks.UserPromptSubmit ?? [];
82
- const already = entries.some((e) => JSON.stringify(e).includes(HOOK_MARKER));
83
- if (!already) {
84
- entries.push({ hooks: [{ type: "command", command: hookCommand() }] });
85
- settings.hooks.UserPromptSubmit = entries;
139
+ const want = hookCommand();
140
+ // Re-running install must REPAIR a stale entry, not skip it. Earlier versions
141
+ // wrote a version-pinned node binary; if we only checked "is it present?" an
142
+ // upgrade would leave that broken command in place forever.
143
+ const ours = entries.filter(isOurHookEntry);
144
+ const current = ours[0]?.hooks?.[0]?.command;
145
+ if (ours.length === 0) {
146
+ entries.push({ hooks: [{ type: "command", command: want }] });
147
+ }
148
+ else if (current !== want) {
149
+ // Replace every entry of ours with exactly one correct entry.
150
+ const others = entries.filter((e) => !isOurHookEntry(e));
151
+ others.push({ hooks: [{ type: "command", command: want }] });
152
+ settings.hooks.UserPromptSubmit = others;
86
153
  writeJsonWithBackup(settingsPath, settings);
154
+ return settingsPath;
155
+ }
156
+ else {
157
+ return settingsPath; // already correct — leave the file untouched
87
158
  }
159
+ settings.hooks.UserPromptSubmit = entries;
160
+ writeJsonWithBackup(settingsPath, settings);
88
161
  return settingsPath;
89
162
  }
90
163
  function uninstallHook() {
@@ -95,7 +168,7 @@ function uninstallHook() {
95
168
  const entries = settings.hooks?.UserPromptSubmit;
96
169
  if (!entries)
97
170
  return;
98
- const filtered = entries.filter((e) => !JSON.stringify(e).includes(HOOK_MARKER));
171
+ const filtered = entries.filter((e) => !isOurHookEntry(e));
99
172
  if (filtered.length !== entries.length) {
100
173
  settings.hooks.UserPromptSubmit = filtered;
101
174
  if (filtered.length === 0)
@@ -128,6 +201,7 @@ export function runInstall() {
128
201
  try {
129
202
  const config = readJson(target.configPath);
130
203
  config.mcpServers = config.mcpServers ?? {};
204
+ // Always overwrite: re-running install is how a stale entry gets repaired.
131
205
  config.mcpServers["context-doctor"] = entry;
132
206
  writeJsonWithBackup(target.configPath, config);
133
207
  console.log(`✓ ${target.name}: MCP server added (${target.configPath})`);
@@ -140,8 +214,14 @@ export function runInstall() {
140
214
  if (skillPath)
141
215
  console.log(`✓ Agent Skill installed for Claude Code (${skillPath})`);
142
216
  const hookPath = installHook();
143
- if (hookPath)
217
+ if (hookPath) {
144
218
  console.log(`✓ Claude Code every-prompt hook installed (${hookPath}) — heavy sessions get automatic hygiene guidance`);
219
+ // npx resolves the package on every single prompt; a global install makes
220
+ // the hook a plain exec instead, which is both faster and update-proof.
221
+ if (hookUsesNpx()) {
222
+ console.log(" note: the hook falls back to npx. For a faster, permanent hook: npm i -g context-doctor && context-doctor install");
223
+ }
224
+ }
145
225
  console.log("\nDone. Restart the apps to pick up the new tools, then try:");
146
226
  console.log(' "What\'s eating my context?" — or paste a conversation and ask for a profile.');
147
227
  }
package/dist/ledger.d.ts CHANGED
@@ -7,14 +7,17 @@
7
7
  * check — hook deep-parsed a session {ev?: undefined|"check", sid, tok, warn}
8
8
  * (pre-0.3.6 hook entries have no `ev` field; treated as checks)
9
9
  * optimize — an optimization was applied {ev: "optimize", src: "cli"|"mcp", saved, model?}
10
+ * proxy — proxy savings checkpoint {ev: "proxy", saved, usd?, requests?}
10
11
  */
11
12
  export interface LedgerEntry {
12
13
  ts: number;
13
- ev?: "check" | "optimize";
14
+ ev?: "check" | "optimize" | "proxy";
14
15
  sid?: string;
15
16
  tok?: number;
16
17
  warn?: boolean;
17
- src?: "cli" | "mcp";
18
+ src?: "cli" | "mcp" | "proxy";
19
+ usd?: number;
20
+ requests?: number;
18
21
  saved?: number;
19
22
  model?: string;
20
23
  }
package/dist/ledger.js CHANGED
@@ -7,6 +7,7 @@
7
7
  * check — hook deep-parsed a session {ev?: undefined|"check", sid, tok, warn}
8
8
  * (pre-0.3.6 hook entries have no `ev` field; treated as checks)
9
9
  * optimize — an optimization was applied {ev: "optimize", src: "cli"|"mcp", saved, model?}
10
+ * proxy — proxy savings checkpoint {ev: "proxy", saved, usd?, requests?}
10
11
  */
11
12
  import { appendFileSync, existsSync, mkdirSync, readFileSync, statSync, writeFileSync } from "node:fs";
12
13
  import { homedir } from "node:os";
package/dist/mcp.js CHANGED
@@ -30,14 +30,14 @@ import { recordLedger } from "./ledger.js";
30
30
  // Kept deliberately terse: these ride in EVERY conversation's context, and a
31
31
  // context-saving tool must not itself be context overhead (~110 tokens).
32
32
  const SERVER_INSTRUCTIONS = `Context hygiene, always: summarize large pastes/tool results instead of carrying them verbatim; reference earlier content, don't re-quote; never inline base64. Past ~30 turns or several large pastes, proactively offer to run profile_context. Any question about tokens, cost, or latency: call profile_context, don't estimate. If optimize_context returns a pruned-turns digest, you write the ≤150-token replacement summary.`;
33
- const STRATEGY_IDS = ["dedupe", "trim-tool-results", "strip-base64", "prune-history"];
33
+ const STRATEGY_IDS = ["dedupe", "trim-tool-results", "trim-tool-calls", "strip-base64", "prune-history"];
34
34
  /**
35
35
  * Build a fully-configured server instance. A factory (not a singleton) so the
36
36
  * stateless HTTP mode can hand every request its own server, per the MCP SDK's
37
37
  * recommended pattern.
38
38
  */
39
39
  function createServer() {
40
- const server = new McpServer({ name: "context-doctor", version: "0.10.0" }, { instructions: SERVER_INSTRUCTIONS });
40
+ const server = new McpServer({ name: "context-doctor", version: "0.12.0" }, { instructions: SERVER_INSTRUCTIONS });
41
41
  server.tool("profile_context", "Profile an LLM conversation or prompt: token breakdown by category, largest messages, and actionable findings about wasted context (duplicates, oversized tool results, base64 blobs, cache-unfriendly ordering). Accepts OpenAI/Anthropic conversation JSON or raw text. Call this immediately whenever the user asks about token usage, context size, LLM cost, or latency — and proactively offer it once a conversation grows long or accumulates large pasted content.", {
42
42
  conversation: z.string().describe("Conversation JSON (OpenAI or Anthropic format, or bare message array) or raw prompt text"),
43
43
  model: z.string().optional().describe("Target model name for context-window math, e.g. claude-sonnet-5 or gpt-4o"),
@@ -48,7 +48,7 @@ function createServer() {
48
48
  server.tool("optimize_context", "Rewrite a conversation to reclaim tokens using deterministic strategies: dedupe repeated content, trim stale tool results, strip base64 blobs, optionally prune old history. Returns the slimmed conversation JSON plus a savings summary. No LLM calls — safe and inspectable. Call this after profile_context finds recoverable waste and the user wants it fixed; add the prune-history strategy only with the user's consent, then write the replacement summary yourself as the result instructs.", {
49
49
  conversation: z.string().describe("Conversation JSON (OpenAI or Anthropic format, or bare message array)"),
50
50
  strategies: z.array(z.enum(STRATEGY_IDS)).optional()
51
- .describe("Strategies to apply. Default: dedupe, trim-tool-results, strip-base64. Add prune-history for lossy compaction of old turns."),
51
+ .describe("Strategies to apply. Default: dedupe, trim-tool-results, strip-base64. Add trim-tool-calls to shrink big inline file writes, or prune-history for lossy compaction of old turns."),
52
52
  keep_recent: z.number().int().positive().optional().describe("Messages at the tail to leave untouched (default 6)"),
53
53
  max_tool_result_tokens: z.number().int().positive().optional().describe("Token budget for trimmed tool results (default 300)"),
54
54
  }, async ({ conversation, strategies, keep_recent, max_tool_result_tokens }) => {
@@ -6,12 +6,12 @@
6
6
  * Strategies operate on the ORIGINAL JSON structure (not the normalized view)
7
7
  * so the output is a drop-in replacement for the input conversation.
8
8
  */
9
- export type StrategyId = "dedupe" | "trim-tool-results" | "prune-history" | "strip-base64";
9
+ export type StrategyId = "dedupe" | "trim-tool-results" | "trim-tool-calls" | "prune-history" | "strip-base64";
10
10
  export interface OptimizeOptions {
11
11
  strategies?: StrategyId[];
12
12
  /** Tool results older than this many messages from the end get trimmed. */
13
13
  keepRecent?: number;
14
- /** Max tokens a trimmed tool result keeps. */
14
+ /** Max tokens a trimmed tool result — or tool-call argument set — keeps. */
15
15
  maxToolResultTokens?: number;
16
16
  }
17
17
  export interface AppliedChange {
package/dist/optimize.js CHANGED
@@ -8,12 +8,36 @@
8
8
  */
9
9
  import { createHash } from "node:crypto";
10
10
  import { estimateTokens } from "./tokens.js";
11
+ import { hasBase64Blob, stripBase64Blobs } from "./blob.js";
11
12
  const DEFAULTS = {
12
13
  strategies: ["dedupe", "trim-tool-results", "strip-base64"],
13
14
  keepRecent: 6,
14
15
  maxToolResultTokens: 300,
15
16
  };
16
- const BASE64_RE = /(?:data:[\w/+.-]+;base64,)?[A-Za-z0-9+/]{500,}={0,2}/g;
17
+ /**
18
+ * Shrink the arguments of a tool call that has already run.
19
+ *
20
+ * In file-heavy agent sessions the biggest single items in context are not
21
+ * tool RESULTS but tool CALLS: a Write or a `cat > file <<EOF` carries the
22
+ * whole file inline, forever. Once the call has returned, the live context
23
+ * only needs enough of the arguments to identify what was done.
24
+ *
25
+ * Keys are preserved (so the call still reads as itself) and only long string
26
+ * values are cut, with an explicit marker so nothing looks silently complete.
27
+ */
28
+ function trimCallArguments(input, maxTokens) {
29
+ const budgetChars = maxTokens * 4;
30
+ const out = {};
31
+ for (const [key, value] of Object.entries(input)) {
32
+ if (typeof value === "string" && value.length > budgetChars) {
33
+ out[key] = value.slice(0, budgetChars) + `\n[context-doctor: ${value.length - budgetChars} more chars trimmed — this call already ran]`;
34
+ }
35
+ else {
36
+ out[key] = value;
37
+ }
38
+ }
39
+ return out;
40
+ }
17
41
  function hash(text) {
18
42
  return createHash("sha1").update(text.replace(/\s+/g, " ").trim()).digest("hex");
19
43
  }
@@ -109,11 +133,10 @@ export function optimizeConversation(input, options = {}) {
109
133
  if (opts.strategies.includes("strip-base64")) {
110
134
  messages.forEach((m, i) => {
111
135
  const text = textOf(m.content);
112
- if (!BASE64_RE.test(text))
136
+ if (!hasBase64Blob(text))
113
137
  return;
114
- BASE64_RE.lastIndex = 0;
115
138
  const before = estimateTokens(text);
116
- const cleaned = text.replace(BASE64_RE, "[context-doctor: base64 blob removed — use file/image APIs instead]");
139
+ const cleaned = stripBase64Blobs(text);
117
140
  const saved = before - estimateTokens(cleaned);
118
141
  if (saved > 50) {
119
142
  m.content = replaceText(m.content, cleaned);
@@ -159,6 +182,46 @@ export function optimizeConversation(input, options = {}) {
159
182
  });
160
183
  });
161
184
  }
185
+ // -- trim-tool-calls: shrink the arguments of calls that already ran ----------
186
+ if (opts.strategies.includes("trim-tool-calls")) {
187
+ const cutoff = messages.length - opts.keepRecent;
188
+ messages.forEach((m, i) => {
189
+ if (i >= cutoff)
190
+ return;
191
+ let saved = 0;
192
+ // Anthropic shape: tool_use blocks with a structured `input`.
193
+ if (Array.isArray(m.content)) {
194
+ for (const b of m.content) {
195
+ if (b?.type !== "tool_use" || b.input == null || typeof b.input !== "object")
196
+ continue;
197
+ const before = estimateTokens(JSON.stringify(b.input));
198
+ if (before <= opts.maxToolResultTokens)
199
+ continue;
200
+ b.input = trimCallArguments(b.input, opts.maxToolResultTokens);
201
+ saved += before - estimateTokens(JSON.stringify(b.input));
202
+ }
203
+ }
204
+ // OpenAI shape: tool_calls[].function.arguments is a JSON string.
205
+ for (const tc of m.tool_calls ?? []) {
206
+ const args = tc?.function?.arguments;
207
+ if (typeof args !== "string")
208
+ continue;
209
+ const before = estimateTokens(args);
210
+ if (before <= opts.maxToolResultTokens)
211
+ continue;
212
+ tc.function.arguments = truncateToTokens(args, opts.maxToolResultTokens);
213
+ saved += before - estimateTokens(tc.function.arguments);
214
+ }
215
+ if (saved > 0) {
216
+ applied.push({
217
+ strategy: "trim-tool-calls",
218
+ messageIndex: i,
219
+ tokensSaved: saved,
220
+ note: "Arguments of a completed tool call truncated",
221
+ });
222
+ }
223
+ });
224
+ }
162
225
  // -- prune-history: replace the older half with a stub ------------------------
163
226
  // Opt-in only: it is lossy, so it is not in the default strategy set.
164
227
  let prunedDigest;
package/dist/parse.d.ts CHANGED
@@ -27,5 +27,11 @@ export interface NormalizedConversation {
27
27
  messages: NormalizedMessage[];
28
28
  /** Format detected, for reporting. */
29
29
  sourceFormat: "openai" | "anthropic" | "array" | "text";
30
+ /**
31
+ * Set when the input could not be read as a conversation. Silently profiling
32
+ * a broken file as one big "user message" produces a confident, wrong report
33
+ * — the caller should show this instead.
34
+ */
35
+ parseWarning?: string;
30
36
  }
31
37
  export declare function parseConversation(input: string): NormalizedConversation;
package/dist/parse.js CHANGED
@@ -90,16 +90,30 @@ export function parseConversation(input) {
90
90
  try {
91
91
  data = JSON.parse(input);
92
92
  }
93
- catch {
94
- // Not JSON — treat the whole thing as one user message so profiling still works.
93
+ catch (e) {
94
+ // Not JSON — treat the whole thing as one user message so profiling still
95
+ // works for raw prompts. But if it LOOKS like JSON, the user handed us a
96
+ // broken conversation file and deserves to be told, not given a report
97
+ // about a single 9-token "message".
98
+ const head = input.trimStart()[0];
99
+ const parseWarning = input.trim() === ""
100
+ ? "Input is empty — nothing to profile."
101
+ : head === "{" || head === "["
102
+ ? `Input starts like JSON but does not parse (${e.message}). Profiling it as raw text, which is almost certainly not what you want.`
103
+ : undefined;
95
104
  return {
96
105
  sourceFormat: "text",
97
- messages: [{ index: 0, role: "user", kind: "user", text: input, hasBinary: false }],
106
+ parseWarning,
107
+ // Empty input has no message: reporting "1 message, ~4 tokens" for it
108
+ // would be inventing content that is not there.
109
+ messages: input.trim() === "" ? [] : [{ index: 0, role: "user", kind: "user", text: input, hasBinary: false }],
98
110
  };
99
111
  }
100
112
  if (Array.isArray(data)) {
113
+ const looksLikeMessages = data.length === 0 || data.some((m) => m && typeof m === "object" && "role" in m);
101
114
  return {
102
115
  sourceFormat: "array",
116
+ parseWarning: looksLikeMessages ? undefined : "This is a JSON array, but no element has a `role` field — it does not look like a conversation.",
103
117
  messages: data.map((m, i) => normalizeMessage(m, i)),
104
118
  };
105
119
  }
@@ -114,5 +128,8 @@ export function parseConversation(input) {
114
128
  messages.push(...rawMessages.map((m, i) => normalizeMessage(m, i)));
115
129
  const isAnthropic = obj.system != null ||
116
130
  rawMessages.some((m) => Array.isArray(m.content) && m.content.some((b) => b?.type === "tool_use" || b?.type === "tool_result"));
117
- return { sourceFormat: isAnthropic ? "anthropic" : "openai", messages };
131
+ const parseWarning = messages.length === 0
132
+ ? "This JSON has no `messages` array (and no `system`) — it does not look like a conversation. Expected {\"messages\":[{\"role\":…,\"content\":…}]}."
133
+ : undefined;
134
+ return { sourceFormat: isAnthropic ? "anthropic" : "openai", parseWarning, messages };
118
135
  }
package/dist/profile.d.ts CHANGED
@@ -12,7 +12,7 @@ export interface MessageProfile {
12
12
  preview: string;
13
13
  toolName?: string;
14
14
  }
15
- export type FindingId = "large_tool_result" | "duplicate_content" | "near_duplicate" | "repeated_tool_call" | "repeated_file_read" | "retained_error_output" | "base64_blob" | "long_history" | "large_system_prompt" | "cache_ordering" | "near_window_limit";
15
+ export type FindingId = "large_tool_result" | "large_tool_call" | "duplicate_content" | "near_duplicate" | "repeated_tool_call" | "repeated_file_read" | "retained_error_output" | "base64_blob" | "long_history" | "large_system_prompt" | "cache_ordering" | "near_window_limit";
16
16
  export interface Finding {
17
17
  id: FindingId;
18
18
  severity: "info" | "warn" | "high";
@@ -51,5 +51,7 @@ export interface ContextProfile {
51
51
  /** Present when the model has a known price. All figures are estimates. */
52
52
  cost?: CostEstimate;
53
53
  sourceFormat: string;
54
+ /** Propagated from parsing: input could not be read as a conversation. */
55
+ parseWarning?: string;
54
56
  }
55
57
  export declare function profileConversation(conv: NormalizedConversation, model?: string): ContextProfile;
package/dist/profile.js CHANGED
@@ -5,6 +5,7 @@
5
5
  import { createHash } from "node:crypto";
6
6
  import { contextWindowFor, estimateTokens, MESSAGE_OVERHEAD_TOKENS, providerFor } from "./tokens.js";
7
7
  import { estimatedTtftSeconds, inputCostUsd, pricingFor } from "./pricing.js";
8
+ import { hasBase64Blob } from "./blob.js";
8
9
  function categoryOf(m) {
9
10
  switch (m.kind) {
10
11
  case "system": return "system";
@@ -56,7 +57,6 @@ function preview(text, len = 90) {
56
57
  function contentHash(text) {
57
58
  return createHash("sha1").update(text.replace(/\s+/g, " ").trim()).digest("hex");
58
59
  }
59
- const BASE64_RE = /(?:data:[\w/+.-]+;base64,|[A-Za-z0-9+/]{500,}={0,2})/;
60
60
  /** FNV-1a — cheap deterministic hash for shingle sampling. */
61
61
  function fnv1a(s) {
62
62
  let h = 0x811c9dc5;
@@ -84,6 +84,8 @@ function sampledShingles(text) {
84
84
  }
85
85
  return out;
86
86
  }
87
+ /** Similarity at or above which two messages count as near-duplicates. */
88
+ const SIMILARITY_THRESHOLD = 0.6;
87
89
  function jaccard(a, b) {
88
90
  if (a.size === 0 || b.size === 0)
89
91
  return 0;
@@ -119,6 +121,24 @@ export function profileConversation(conv, model) {
119
121
  });
120
122
  }
121
123
  }
124
+ // -- Large individual tool calls --------------------------------------------
125
+ // Writing a file through a tool call puts the ENTIRE file contents in the
126
+ // context permanently, and in file-heavy agent sessions these outweigh every
127
+ // tool result put together. Unlike a result, the argument cannot be trimmed
128
+ // before the call — but once the call has returned, the live context only
129
+ // needs a reference to what was written.
130
+ for (const p of perMessage) {
131
+ if (p.msg.kind === "tool_call" && p.tokens > 2000) {
132
+ findings.push({
133
+ id: "large_tool_call",
134
+ severity: p.tokens > 8000 ? "high" : "warn",
135
+ estSavings: Math.round(p.tokens * 0.8),
136
+ message: `Tool call at message #${p.msg.index}${p.msg.toolName ? ` (${p.msg.toolName})` : ""} carries ~${p.tokens} tokens of arguments.`,
137
+ suggestion: "Usually a whole file being written inline. Once it has run, replace the arguments with a reference (\"wrote <path>\") — or write files in smaller pieces so no single call carries the entire content.",
138
+ messages: [p.msg.index],
139
+ });
140
+ }
141
+ }
122
142
  // -- Exact duplicate content ------------------------------------------------
123
143
  const seen = new Map();
124
144
  for (const p of perMessage) {
@@ -149,6 +169,11 @@ export function profileConversation(conv, model) {
149
169
  .sort((a, b) => b.tokens - a.tokens)
150
170
  .slice(0, 150);
151
171
  const shingleSets = candidates.map((p) => sampledShingles(p.msg.text));
172
+ // Jaccard has a hard ceiling of |smaller| / |larger|: two shingle sets of
173
+ // very different sizes CANNOT reach the threshold, so those pairs are
174
+ // skipped without intersecting them. Exact, not heuristic — it changes
175
+ // runtime, never results.
176
+ const sizes = shingleSets.map((set) => set.size);
152
177
  const exactDup = new Set(findings.filter((f) => f.id === "duplicate_content").flatMap((f) => f.messages));
153
178
  for (let i = 0; i < candidates.length; i++) {
154
179
  for (let j = i + 1; j < candidates.length; j++) {
@@ -156,8 +181,12 @@ export function profileConversation(conv, model) {
156
181
  const b = candidates[j];
157
182
  if (exactDup.has(a.msg.index) && exactDup.has(b.msg.index))
158
183
  continue; // already flagged exactly
184
+ const small = Math.min(sizes[i], sizes[j]);
185
+ const large = Math.max(sizes[i], sizes[j]);
186
+ if (large === 0 || small / large < SIMILARITY_THRESHOLD)
187
+ continue; // cannot clear the bar
159
188
  const sim = jaccard(shingleSets[i], shingleSets[j]);
160
- if (sim >= 0.6) {
189
+ if (sim >= SIMILARITY_THRESHOLD) {
161
190
  const smaller = Math.min(a.tokens, b.tokens);
162
191
  const [first, second] = a.msg.index <= b.msg.index ? [a, b] : [b, a];
163
192
  findings.push({
@@ -248,7 +277,7 @@ export function profileConversation(conv, model) {
248
277
  }
249
278
  // -- Base64 / binary blobs ---------------------------------------------------
250
279
  for (const p of perMessage) {
251
- if (BASE64_RE.test(p.msg.text)) {
280
+ if (hasBase64Blob(p.msg.text)) {
252
281
  findings.push({
253
282
  id: "base64_blob",
254
283
  severity: "high",
@@ -348,5 +377,6 @@ export function profileConversation(conv, model) {
348
377
  totalEstSavings,
349
378
  cost,
350
379
  sourceFormat: conv.sourceFormat,
380
+ parseWarning: conv.parseWarning,
351
381
  };
352
382
  }
package/dist/proxy.js CHANGED
@@ -15,6 +15,7 @@ import http from "node:http";
15
15
  import { optimizeConversation } from "./optimize.js";
16
16
  import { formatTokens } from "./tokens.js";
17
17
  import { formatUsd, inputCostUsd, pricingFor } from "./pricing.js";
18
+ import { recordLedger } from "./ledger.js";
18
19
  /** Connection-level headers that must not be forwarded. */
19
20
  const SKIP_REQUEST_HEADERS = new Set(["host", "content-length", "connection", "transfer-encoding", "accept-encoding", "expect"]);
20
21
  const SKIP_RESPONSE_HEADERS = new Set(["content-length", "content-encoding", "transfer-encoding", "connection"]);
@@ -208,6 +209,36 @@ export function startProxy(opts = {}) {
208
209
  res.end(JSON.stringify({ error: `context-doctor proxy: ${e.message}` }));
209
210
  }
210
211
  });
212
+ // Savings live in memory, so a restart would erase the record the dashboard
213
+ // and report draw on. Checkpoint the delta to the ledger periodically and on
214
+ // shutdown, so the history survives the process.
215
+ let checkpointedTokens = 0;
216
+ let checkpointedUsd = 0;
217
+ let checkpointedRequests = 0;
218
+ const checkpoint = () => {
219
+ const savedDelta = stats.tokensSaved - checkpointedTokens;
220
+ if (savedDelta <= 0)
221
+ return;
222
+ recordLedger({
223
+ ev: "proxy",
224
+ src: "proxy",
225
+ saved: savedDelta,
226
+ usd: Number((stats.estUsdSaved - checkpointedUsd).toFixed(6)),
227
+ requests: stats.optimizedRequests - checkpointedRequests,
228
+ });
229
+ checkpointedTokens = stats.tokensSaved;
230
+ checkpointedUsd = stats.estUsdSaved;
231
+ checkpointedRequests = stats.optimizedRequests;
232
+ };
233
+ const checkpointTimer = setInterval(checkpoint, 60_000);
234
+ checkpointTimer.unref?.(); // never hold the process open on our account
235
+ for (const signal of ["SIGINT", "SIGTERM"]) {
236
+ process.once(signal, () => {
237
+ checkpoint();
238
+ process.exit(0);
239
+ });
240
+ }
241
+ server.on("close", checkpoint);
211
242
  const host = opts.host ?? "127.0.0.1";
212
243
  server.listen(port, host, () => {
213
244
  console.error(`context-doctor proxy listening on http://${host}:${port}`);
package/dist/report.d.ts CHANGED
@@ -4,4 +4,12 @@
4
4
  * anywhere (terminals, issues, chat).
5
5
  */
6
6
  import { ContextProfile } from "./profile.js";
7
- export declare function renderProfile(profile: ContextProfile): string;
7
+ export interface RenderOptions {
8
+ /**
9
+ * Replace anything quoted from the conversation with a placeholder, so a
10
+ * profile can be pasted into a bug report without leaking content. Numbers
11
+ * and structure — the parts that make a report useful — are kept.
12
+ */
13
+ redact?: boolean;
14
+ }
15
+ export declare function renderProfile(profile: ContextProfile, options?: RenderOptions): string;
package/dist/report.js CHANGED
@@ -18,11 +18,24 @@ function bar(fraction, width = 28) {
18
18
  const filled = Math.round(fraction * width);
19
19
  return "█".repeat(filled) + "░".repeat(width - filled);
20
20
  }
21
- export function renderProfile(profile) {
21
+ /** Mask filesystem paths and quoted fragments inside a finding's text. */
22
+ function redactText(text) {
23
+ return text
24
+ .replace(/(?:\/[\w.@ -]+){2,}/g, "[path]")
25
+ .replace(/\b[\w.-]+\.(ts|tsx|js|jsx|py|go|rs|java|rb|md|json|ya?ml|sql|sh)\b/gi, "[file]");
26
+ }
27
+ export function renderProfile(profile, options = {}) {
22
28
  const lines = [];
23
29
  const p = profile;
24
30
  lines.push("CONTEXT DOCTOR — profile");
25
31
  lines.push("═".repeat(56));
32
+ // A malformed or non-conversation input still profiles (as raw text), but a
33
+ // report that does not say so reads as a confident answer to the wrong
34
+ // question. Lead with the warning.
35
+ if (p.parseWarning) {
36
+ lines.push(`⚠ ${p.parseWarning}`);
37
+ lines.push("");
38
+ }
26
39
  lines.push(`Total: ~${formatTokens(p.totalTokens)} tokens across ${p.messageCount} messages (${p.sourceFormat} format)`);
27
40
  if (p.model) {
28
41
  const windowNote = p.contextWindow
@@ -52,17 +65,41 @@ export function renderProfile(profile) {
52
65
  lines.push("─".repeat(56));
53
66
  for (const m of p.largestMessages) {
54
67
  const label = m.toolName ? `${m.kind}:${m.toolName}` : m.kind;
55
- lines.push(` #${m.index} [${label}] ~${formatTokens(m.tokens)} ${m.preview}`);
68
+ // The preview is the only place raw conversation text reaches the report.
69
+ const body = options.redact ? "[redacted]" : m.preview;
70
+ lines.push(` #${m.index} [${label}] ~${formatTokens(m.tokens)} ${body}`);
56
71
  }
57
72
  lines.push("");
58
73
  // Findings
59
74
  if (p.findings.length > 0) {
60
75
  lines.push(`Findings (${p.findings.length})`);
61
76
  lines.push("─".repeat(56));
77
+ // A file-heavy session can produce a dozen findings of one kind, each with
78
+ // the same advice. Printing them all buries the other kinds, so show the
79
+ // worst few per kind and total the rest into one line.
80
+ const MAX_PER_KIND = 3;
81
+ const shownPerKind = new Map();
82
+ const heldPerKind = new Map();
62
83
  for (const f of p.findings) {
84
+ const shown = shownPerKind.get(f.id) ?? 0;
85
+ if (shown >= MAX_PER_KIND) {
86
+ const held = heldPerKind.get(f.id) ?? { count: 0, savings: 0 };
87
+ heldPerKind.set(f.id, { count: held.count + 1, savings: held.savings + f.estSavings });
88
+ continue;
89
+ }
90
+ shownPerKind.set(f.id, shown + 1);
63
91
  const savings = f.estSavings > 0 ? ` [save ~${formatTokens(f.estSavings)}]` : "";
64
- lines.push(`${SEVERITY_MARK[f.severity]} ${f.message}${savings}`);
92
+ lines.push(`${SEVERITY_MARK[f.severity]} ${options.redact ? redactText(f.message) : f.message}${savings}`);
65
93
  lines.push(` → ${f.suggestion}`);
94
+ const held = heldPerKind.get(f.id);
95
+ if (shown + 1 === MAX_PER_KIND && held === undefined)
96
+ heldPerKind.set(f.id, { count: 0, savings: 0 });
97
+ }
98
+ for (const [id, held] of heldPerKind) {
99
+ if (held.count === 0)
100
+ continue;
101
+ const more = held.savings > 0 ? `, ~${formatTokens(held.savings)} more recoverable` : "";
102
+ lines.push(` … and ${held.count} more of the same kind (${id})${more}. Use --json for the full list.`);
66
103
  }
67
104
  lines.push("");
68
105
  if (p.totalEstSavings > 0) {
package/dist/session.js CHANGED
@@ -8,7 +8,8 @@
8
8
  * where `message` is in Anthropic Messages format. Everything else
9
9
  * (titles, mode changes, hook records) is metadata and skipped.
10
10
  */
11
- import { readdirSync, readFileSync, statSync, existsSync } from "node:fs";
11
+ import { readdirSync, readFileSync, statSync, existsSync, openSync, readSync, closeSync } from "node:fs";
12
+ import { StringDecoder } from "node:string_decoder";
12
13
  import { homedir } from "node:os";
13
14
  import { join } from "node:path";
14
15
  function projectsDir() {
@@ -72,12 +73,60 @@ function parseChatGPTExport(data, path) {
72
73
  path,
73
74
  };
74
75
  }
76
+ /**
77
+ * Read a JSONL transcript line by line without ever materializing the whole
78
+ * file as one string.
79
+ *
80
+ * Agent sessions with large tool results reach hundreds of MB, and those are
81
+ * exactly the sessions that most need analysis — but V8 refuses to build a
82
+ * string past ~512MB, so readFileSync would throw on them (and in the hook,
83
+ * throw *silently*). Streaming has no such ceiling and keeps peak memory at
84
+ * one chunk. StringDecoder carries partial UTF-8 sequences across chunk
85
+ * boundaries so multi-byte characters are never corrupted.
86
+ */
87
+ function forEachLine(path, onLine) {
88
+ const fd = openSync(path, "r");
89
+ const decoder = new StringDecoder("utf8");
90
+ const buf = Buffer.allocUnsafe(4 * 1024 * 1024);
91
+ let pending = "";
92
+ try {
93
+ for (;;) {
94
+ const bytes = readSync(fd, buf, 0, buf.length, null);
95
+ if (bytes === 0)
96
+ break;
97
+ pending += decoder.write(buf.subarray(0, bytes));
98
+ let nl;
99
+ while ((nl = pending.indexOf("\n")) !== -1) {
100
+ onLine(pending.slice(0, nl));
101
+ pending = pending.slice(nl + 1);
102
+ }
103
+ }
104
+ pending += decoder.end();
105
+ if (pending)
106
+ onLine(pending);
107
+ }
108
+ finally {
109
+ closeSync(fd);
110
+ }
111
+ }
112
+ /** Peek at the first bytes to tell a ChatGPT export (JSON array) from JSONL. */
113
+ function startsWithArray(path) {
114
+ const fd = openSync(path, "r");
115
+ try {
116
+ const buf = Buffer.allocUnsafe(64);
117
+ const bytes = readSync(fd, buf, 0, 64, 0);
118
+ return buf.subarray(0, bytes).toString("utf8").trimStart().startsWith("[");
119
+ }
120
+ finally {
121
+ closeSync(fd);
122
+ }
123
+ }
75
124
  export function parseSessionFile(path) {
76
- const raw = readFileSync(path, "utf8");
77
- // ChatGPT exports are one big JSON array, not JSONL.
78
- if (raw.trimStart().startsWith("[")) {
125
+ // ChatGPT exports are one big JSON array, not JSONL — and small enough to
126
+ // read whole. Only peek first, so multi-hundred-MB JSONL is never slurped.
127
+ if (startsWithArray(path)) {
79
128
  try {
80
- const data = JSON.parse(raw);
129
+ const data = JSON.parse(readFileSync(path, "utf8"));
81
130
  if (Array.isArray(data) && data.some((c) => c && typeof c.mapping === "object")) {
82
131
  return parseChatGPTExport(data, path);
83
132
  }
@@ -93,15 +142,15 @@ export function parseSessionFile(path) {
93
142
  let lastCompactIndex = -1;
94
143
  /** Newest API-reported input size, if the transcript carries usage. */
95
144
  let reportedInputTokens;
96
- for (const line of raw.split("\n")) {
145
+ forEachLine(path, (line) => {
97
146
  if (!line.trim())
98
- continue;
147
+ return;
99
148
  let entry;
100
149
  try {
101
150
  entry = JSON.parse(line);
102
151
  }
103
152
  catch {
104
- continue;
153
+ return;
105
154
  }
106
155
  // Titles are metadata lines; the last one wins.
107
156
  if (entry.type === "custom-title" && entry.customTitle)
@@ -109,12 +158,12 @@ export function parseSessionFile(path) {
109
158
  if (entry.type === "ai-title" && entry.aiTitle && !title)
110
159
  title = entry.aiTitle;
111
160
  if ((entry.type !== "user" && entry.type !== "assistant") || !entry.message)
112
- continue;
161
+ return;
113
162
  if (entry.isSidechain)
114
- continue; // subagent traffic has its own context window
163
+ return; // subagent traffic has its own context window
115
164
  const message = entry.message;
116
165
  if (!message.role || message.content == null)
117
- continue;
166
+ return;
118
167
  if (typeof message.model === "string")
119
168
  model = message.model;
120
169
  const usage = message.usage;
@@ -126,7 +175,7 @@ export function parseSessionFile(path) {
126
175
  if (entry.isCompactSummary)
127
176
  lastCompactIndex = messages.length;
128
177
  messages.push({ role: message.role, content: message.content });
129
- }
178
+ });
130
179
  // A compaction replaces everything before it: the summary entry IS the live
131
180
  // history from that point on. Counting the pre-compaction turns would
132
181
  // overstate context, cost per message and window fill — sometimes hugely.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "context-doctor",
3
- "version": "0.10.0",
3
+ "version": "0.12.0",
4
4
  "description": "Profile and optimize LLM context windows. See what's eating your tokens and fix it — works with Claude, GPT, Gemini, and any MCP-capable AI app.",
5
5
  "keywords": [
6
6
  "llm",
@@ -36,13 +36,13 @@
36
36
  "LICENSE"
37
37
  ],
38
38
  "engines": {
39
- "node": ">=18"
39
+ "node": ">=20"
40
40
  },
41
41
  "scripts": {
42
42
  "build": "tsc && node -e \"const fs=require('fs');['dist/cli.js','dist/mcp.js'].forEach(f=>fs.chmodSync(f,0o755))\"",
43
43
  "prepublishOnly": "npm run build",
44
44
  "dev": "tsc --watch",
45
- "test": "npm run build && node --test dist/test/smoke.test.js dist/test/proxy.test.js dist/test/hook.test.js dist/test/mcp-http.test.js dist/test/doctor.test.js dist/test/watch.test.js dist/test/chatgpt-export.test.js dist/test/config.test.js dist/test/dashboard.test.js dist/test/cursor.test.js dist/test/cache.test.js"
45
+ "test": "npm run build && node --test dist/test/smoke.test.js dist/test/proxy.test.js dist/test/hook.test.js dist/test/mcp-http.test.js dist/test/doctor.test.js dist/test/watch.test.js dist/test/chatgpt-export.test.js dist/test/config.test.js dist/test/dashboard.test.js dist/test/cursor.test.js dist/test/cache.test.js dist/test/session.test.js"
46
46
  },
47
47
  "dependencies": {
48
48
  "@modelcontextprotocol/sdk": "^1.0.0",
@@ -27,7 +27,7 @@ Proactively (do not wait to be asked):
27
27
 
28
28
  If the `context-doctor` MCP tools are available:
29
29
  - `profile_context` — pass a conversation JSON (OpenAI or Anthropic format) or raw text; returns a token breakdown, largest messages, findings with estimated savings.
30
- - `optimize_context` — applies deterministic fixes (dedupe, trim stale tool results, strip base64; opt-in `prune-history`). When the result contains pruned-turn source material and asks for a summary, **you write that summary** (≤150 tokens, dense, factual) and place it where the stub indicates — this is how summarization works without any API key.
30
+ - `optimize_context` — applies deterministic fixes (dedupe, trim stale tool results, strip base64; opt-in `trim-tool-calls` for big inline file writes and `prune-history`). When the result contains pruned-turn source material and asks for a summary, **you write that summary** (≤150 tokens, dense, factual) and place it where the stub indicates — this is how summarization works without any API key.
31
31
  - `context_best_practices` — provider-specific checklist to share with the user.
32
32
 
33
33
  If the tools are not connected, the CLI does the same: `npx context-doctor analyze <file> --model <model>` and `npx context-doctor optimize <file>`. For always-on optimization of the user's own apps: `npx context-doctor proxy` then point `ANTHROPIC_BASE_URL` / `OPENAI_BASE_URL` at it.