token-pilot 0.18.0 → 0.19.1

package/CHANGELOG.md

@@ -5,6 +5,24 @@ All notable changes to Token Pilot will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.19.1] - 2026-04-15
+
+### Added
+- **`decisions` field in `session_snapshot`** — stores key decisions with reasoning (e.g., "removed sysfee step — caused double counting"). Prevents the model from revisiting rejected approaches after context compaction.
+
+## [0.19.0] - 2026-04-15
+
+### Added
+- **`session_snapshot` tool** — capture current session state (goal, confirmed facts, files, blockers, next step) as a compact markdown block (<200 tokens). Call before context compaction or when switching direction in long sessions.
+- **`max_tokens` parameter** on `smart_read` and `smart_read_many` — token budget per read. Output auto-downgrades through three levels: full content → structural outline → compact (symbol names + line ranges only). Enables context-constrained sessions.
+- **Session compaction advisory** — policy engine now tracks total tool calls and tokens returned. Advises calling `session_snapshot()` when thresholds are reached (default: every 15 calls or after 8,000 tokens). Configurable via `compactionCallThreshold` and `compactionTokenThreshold`.
+- **"Why This Approach Works"** section in README explaining the 3-level optimization strategy.
+
+### Changed
+- **21 tools** (was 20) — added `session_snapshot`.
+- **MCP instructions** updated with `session_snapshot` workflow and `max_tokens` guidance.
+- Benchmark numbers updated: 55 files, 102K raw → 9K outline tokens (91% savings).
+
 ## [0.18.0] - 2026-04-05
 
 ### Added

package/README.md

@@ -21,16 +21,28 @@ Measured on public open-source repos using the regex fallback parser (no ast-ind
 
 | Repo | Files | Raw Tokens | Outline Tokens | Savings |
 |------|------:|----------:|--------------:|--------:|
-| [token-pilot](https://github.com/Digital-Threads/token-pilot) (TS) | 48 | 88,920 | 8,123 | **91%** |
+| [token-pilot](https://github.com/Digital-Threads/token-pilot) (TS) | 55 | 102,086 | 8,992 | **91%** |
 | [express](https://github.com/expressjs/express) (JS) | 6 | 14,421 | 193 | **99%** |
 | [fastify](https://github.com/fastify/fastify) (JS) | 23 | 50,000 | 3,161 | **94%** |
 | [flask](https://github.com/pallets/flask) (Python) | 20 | 78,236 | 7,418 | **91%** |
-| **Total** | **97** | **231,577** | **18,895** | **92%** |
+| **Total** | **104** | **244,743** | **19,764** | **92%** |
 
 > This measures `smart_read` structural outline savings only. Real sessions also benefit from session cache, dedup reminders, `read_symbol` targeted loading, and `read_for_edit` minimal context.
 >
 > Run the benchmark yourself: `npx tsx scripts/benchmark.ts`
 
+## Why This Approach Works
+
+The biggest source of token waste in AI coding sessions isn't verbose prompts — it's **redundant context**. Every time a model re-reads a file, re-sends conversation history, or loads code it doesn't need, you pay for tokens that add no value.
+
+Token Pilot attacks this at three levels:
+
+1. **Symbol-first reading** — load outlines instead of full files, drill into specific functions on demand. This alone saves 60-90% on most reads.
+2. **Context budget control** — `max_tokens` parameter on `smart_read` auto-downgrades output (full → outline → compact) to fit within a token budget per step.
+3. **Session state management** — `session_snapshot` captures session state as a compact markdown block (<200 tokens), enabling clean context compaction without losing track of what you're doing.
+
+These aren't theoretical gains. In real sessions, the combination of structural reading + targeted symbol access + session snapshots consistently reduces token usage by 80-90% compared to raw file reads.
+
 ## Installation
 
 ### Quick Start (recommended)
@@ -152,6 +164,7 @@ WHEN TO USE TOKEN PILOT (saves up to 80% tokens):
 • Reading code files → smart_read (returns structure, not raw content)
 • Need one function/class → read_symbol (loads only that symbol)
 • Exploring a directory → outline (all symbols in one call)
+• Long session? → session_snapshot (capture state before compaction)
 ...
 WHEN TO USE DEFAULT TOOLS (Token Pilot adds no value):
 • Regex/pattern search → use Grep/ripgrep, NOT find_usages
@@ -168,13 +181,13 @@ For more control, you can add rules to your project:
 - **Cursor** → `.cursorrules` in project root
 - **Codex** → `AGENTS.md` in project root
 
-## MCP Tools (19)
+## MCP Tools (21)
 
 ### Core Reading
 
 | Tool | Instead of | Description |
 |------|-----------|-------------|
-| `smart_read` | `Read` | AST structural overview: classes, functions, methods with signatures. Up to 90% savings on large files. Framework-aware: shows HTTP routes, column types, validation rules. |
+| `smart_read` | `Read` | AST structural overview: classes, functions, methods with signatures. Up to 90% savings on large files. Framework-aware: shows HTTP routes, column types, validation rules. `max_tokens` param for budget-constrained sessions. |
 | `read_symbol` | `Read` + scroll | Load source of a specific symbol. Supports `Class.method`. `show` param: full/head/tail/outline. |
 | `read_symbols` | N x `read_symbol` | Batch read multiple symbols from one file in a single call (max 10). One round-trip instead of N. |
 | `read_for_edit` | `Read` before `Edit` | Minimal RAW code around a symbol — copy directly as `old_string` for Edit tool. Batch mode: pass `symbols` array for multiple edit contexts. |
@@ -198,10 +211,11 @@ For more control, you can add rules to your project:
 | `smart_log` | raw `git log` | Structured commit history with category detection (feat/fix/refactor/docs), file stats, author breakdown. Filters by path and ref. |
 | `test_summary` | raw test output | Run tests and get structured summary: total/passed/failed + failure details. Supports vitest, jest, pytest, phpunit, go, cargo, rspec, mocha. |
 
-### Analytics
+### Session & Analytics
 
 | Tool | Description |
 |------|-------------|
+| `session_snapshot` | Capture session state as a compact markdown block (<200 tokens): goal, decisions (with reasoning), confirmed facts, relevant files, blockers, next step. Decisions field prevents revisiting rejected approaches after compaction. |
 | `session_analytics` | Token savings report: total saved, per-tool breakdown, top files, per-intent breakdown, decision insights, policy advisories. |
 
 ## CLI Commands

package/dist/config/defaults.js

@@ -59,6 +59,8 @@ export const DEFAULT_CONFIG = {
         maxFullFileReads: 10,
         warnOnLargeReads: true,
         largeReadThreshold: 2000,
+        compactionCallThreshold: 15,
+        compactionTokenThreshold: 8000,
     },
     ignore: [
         'node_modules/**',

package/dist/core/policy-engine.d.ts

@@ -16,6 +16,10 @@ export interface PolicyConfig {
     warnOnLargeReads: boolean;
     /** Token threshold for large read warning */
     largeReadThreshold: number;
+    /** Suggest compaction after N total tool calls (0 = disabled) */
+    compactionCallThreshold: number;
+    /** Suggest compaction after N total tokens returned (0 = disabled) */
+    compactionTokenThreshold: number;
 }
 export declare const DEFAULT_POLICIES: PolicyConfig;
 export interface PolicyCheckContext {
@@ -23,6 +27,8 @@ export interface PolicyCheckContext {
     tokensReturned: number;
     readForEditCalled?: Set<string>;
     editTargetPath?: string;
+    totalCallCount?: number;
+    totalTokensReturned?: number;
 }
 export interface PolicyAdvisory {
     level: 'info' | 'warn';

package/dist/core/policy-engine.js

@@ -10,6 +10,8 @@ export const DEFAULT_POLICIES = {
     maxFullFileReads: 10,
     warnOnLargeReads: true,
     largeReadThreshold: 2000,
+    compactionCallThreshold: 15,
+    compactionTokenThreshold: 8000,
 };
 /** Full-file read tools that count toward maxFullFileReads */
 const FULL_READ_TOOLS = new Set([
@@ -64,6 +66,28 @@ export function checkPolicy(policy, tool, context) {
             message: `POLICY: Consider using read_for_edit("${context.editTargetPath}") before editing to get precise edit context.`,
         };
     }
+    // 5. Session compaction advisory — by call count
+    if (policy.compactionCallThreshold > 0 &&
+        context.totalCallCount !== undefined &&
+        context.totalCallCount > 0 &&
+        context.totalCallCount % policy.compactionCallThreshold === 0) {
+        return {
+            level: 'info',
+            message: `COMPACTION: ${context.totalCallCount} tool calls this session. Consider calling session_snapshot() to capture state, then compact context.`,
+        };
+    }
+    // 6. Session compaction advisory — by total tokens
+    if (policy.compactionTokenThreshold > 0 &&
+        context.totalTokensReturned !== undefined &&
+        context.totalTokensReturned > policy.compactionTokenThreshold &&
+        context.totalCallCount !== undefined &&
+        context.totalCallCount % 5 === 0 // don't spam every call, check every 5th
+    ) {
+        return {
+            level: 'info',
+            message: `COMPACTION: ~${context.totalTokensReturned} tokens returned this session. Consider calling session_snapshot() to capture state, then compact context.`,
+        };
+    }
     return null;
 }
 /**

package/dist/core/validation.d.ts

@@ -12,6 +12,7 @@ export declare function validateSmartReadArgs(args: unknown): {
     show_docs?: boolean;
     show_references?: boolean;
     depth?: number;
+    max_tokens?: number;
 };
 /**
  * Validate read_symbol arguments.
@@ -67,6 +68,7 @@ export declare function validateFindUsagesArgs(args: unknown): FindUsagesArgs;
  */
 export declare function validateSmartReadManyArgs(args: unknown): {
     paths: string[];
+    max_tokens?: number;
 };
 /**
  * Validate read_for_edit arguments.

package/dist/core/validation.js

@@ -28,6 +28,7 @@ export function validateSmartReadArgs(args) {
         show_docs: optionalBool(a.show_docs, 'show_docs'),
         show_references: optionalBool(a.show_references, 'show_references'),
         depth: optionalNumber(a.depth, 'depth'),
+        max_tokens: optionalNumber(a.max_tokens, 'max_tokens'),
     };
 }
 /**
@@ -194,7 +195,7 @@ export function validateSmartReadManyArgs(args) {
             throw new Error('Each path in "paths" must be a non-empty string.');
         }
     }
-    return { paths: a.paths };
+    return { paths: a.paths, max_tokens: optionalNumber(a.max_tokens, 'max_tokens') };
 }
 function optionalString(val, name) {
     if (val === undefined || val === null)

package/dist/handlers/session-snapshot.d.ts

@@ -0,0 +1,15 @@
+export interface SessionSnapshotArgs {
+    goal: string;
+    decisions?: string[];
+    confirmed?: string[];
+    files?: string[];
+    blocked?: string;
+    next?: string;
+}
+export declare function handleSessionSnapshot(args: SessionSnapshotArgs): {
+    content: {
+        type: 'text';
+        text: string;
+    }[];
+};
+//# sourceMappingURL=session-snapshot.d.ts.map

package/dist/handlers/session-snapshot.js

@@ -0,0 +1,28 @@
+export function handleSessionSnapshot(args) {
+    const lines = ['## Session State'];
+    lines.push(`**Goal:** ${args.goal}`);
+    if (args.decisions?.length) {
+        lines.push('**Decisions:**');
+        for (const item of args.decisions) {
+            lines.push(`- ${item}`);
+        }
+    }
+    if (args.confirmed?.length) {
+        lines.push('**Confirmed:**');
+        for (const item of args.confirmed) {
+            lines.push(`- ${item}`);
+        }
+    }
+    if (args.files?.length) {
+        lines.push(`**Files:** ${args.files.join(', ')}`);
+    }
+    if (args.blocked) {
+        lines.push(`**Blocked:** ${args.blocked}`);
+    }
+    if (args.next) {
+        lines.push(`**Next:** ${args.next}`);
+    }
+    const text = lines.join('\n');
+    return { content: [{ type: 'text', text }] };
+}
+//# sourceMappingURL=session-snapshot.js.map

package/dist/handlers/smart-read-many.d.ts

@@ -4,6 +4,7 @@ import type { ContextRegistry } from '../core/context-registry.js';
 import type { TokenPilotConfig } from '../types.js';
 export interface SmartReadManyArgs {
     paths: string[];
+    max_tokens?: number;
 }
 export declare function handleSmartReadMany(args: SmartReadManyArgs, projectRoot: string, astIndex: AstIndexClient, fileCache: FileCache, contextRegistry: ContextRegistry, config: TokenPilotConfig): Promise<{
     content: Array<{

package/dist/handlers/smart-read-many.js

@@ -32,7 +32,7 @@ export async function handleSmartReadMany(args, projectRoot, astIndex, fileCache
                 const fullTokens = await estimateFullFileTokens(projectRoot, path);
                 return { path, text: reminderText + `\nFor full re-read: smart_read("${path}")`, fullTokens };
             }
-            const result = await handleSmartRead({ path }, projectRoot, astIndex, fileCache, contextRegistry, config);
+            const result = await handleSmartRead({ path, max_tokens: args.max_tokens }, projectRoot, astIndex, fileCache, contextRegistry, config);
             const text = result.content[0]?.text ?? '';
             const fullTokens = await estimateFullFileTokens(projectRoot, path);
             return { path, text, fullTokens };

package/dist/handlers/smart-read.d.ts

@@ -9,6 +9,7 @@ export interface SmartReadArgs {
     show_references?: boolean;
     depth?: number;
     scope?: 'full' | 'nav' | 'exports';
+    max_tokens?: number;
 }
 export declare function handleSmartRead(args: SmartReadArgs, projectRoot: string, astIndex: AstIndexClient, fileCache: FileCache, contextRegistry: ContextRegistry, config: TokenPilotConfig): Promise<{
     content: Array<{

package/dist/handlers/smart-read.js

@@ -28,31 +28,35 @@ export async function handleSmartRead(args, projectRoot, astIndex, fileCache, co
     if (lines.length <= config.smartRead.smallFileThreshold) {
         const hash = createHash('sha256').update(content).digest('hex');
         const tokens = estimateTokens(content);
-        contextRegistry.trackLoad(absPath, {
-            type: 'full',
-            startLine: 1,
-            endLine: lines.length,
-            tokens,
-        });
-        contextRegistry.setContentHash(absPath, hash);
-        // Cache for read_diff baseline (so read_diff works after external edits)
-        if (!fileCache.get(absPath)) {
-            const fileStat = await stat(absPath);
-            fileCache.set(absPath, {
-                structure: {
-                    path: absPath, language: 'unknown',
-                    meta: { lines: lines.length, bytes: content.length, lastModified: fileStat.mtimeMs, contentHash: hash },
-                    imports: [], exports: [], symbols: [],
-                },
-                content, lines, mtime: fileStat.mtimeMs, hash, lastAccess: Date.now(),
+        // Budget check: if full content exceeds max_tokens, skip pass-through and use outline path
+        if (!args.max_tokens || tokens <= args.max_tokens) {
+            contextRegistry.trackLoad(absPath, {
+                type: 'full',
+                startLine: 1,
+                endLine: lines.length,
+                tokens,
             });
+            contextRegistry.setContentHash(absPath, hash);
+            // Cache for read_diff baseline (so read_diff works after external edits)
+            if (!fileCache.get(absPath)) {
+                const fileStat = await stat(absPath);
+                fileCache.set(absPath, {
+                    structure: {
+                        path: absPath, language: 'unknown',
+                        meta: { lines: lines.length, bytes: content.length, lastModified: fileStat.mtimeMs, contentHash: hash },
+                        imports: [], exports: [], symbols: [],
+                    },
+                    content, lines, mtime: fileStat.mtimeMs, hash, lastAccess: Date.now(),
+                });
+            }
+            return {
+                content: [{
+                        type: 'text',
+                        text: `FILE: ${args.path} (${lines.length} lines — returned in full, below threshold)\n\n${content}`,
+                    }],
+            };
         }
-        return {
-            content: [{
-                    type: 'text',
-                    text: `FILE: ${args.path} (${lines.length} lines — returned in full, below threshold)\n\n${content}`,
-                }],
-        };
+        // else: fall through to outline path even for small files
     }
     // 3. Check cache
     let cached = fileCache.get(absPath);
@@ -174,7 +178,7 @@ export async function handleSmartRead(args, projectRoot, astIndex, fileCache, co
     // 6b. Adaptive fallback: if outline is not significantly smaller than raw, return raw
     const structureTokens = estimateTokens(output);
     const fullTokens = estimateTokens(content);
-    if (structureTokens >= fullTokens * 0.7) {
+    if (structureTokens >= fullTokens * 0.7 && (!args.max_tokens || fullTokens <= args.max_tokens)) {
         contextRegistry.trackLoad(absPath, {
             type: 'full',
             startLine: 1,
@@ -192,11 +196,30 @@ export async function handleSmartRead(args, projectRoot, astIndex, fileCache, co
                 }],
         };
     }
-    // 7. Add token savings
+    // 7. Budget enforcement: if outline exceeds max_tokens, return compact version
+    if (args.max_tokens && structureTokens > args.max_tokens) {
+        const symbols = cached.structure.symbols;
+        const compactLines = [
+            `FILE: ${args.path} (${lines.length} lines — compact, budget: ${args.max_tokens} tokens)`,
+            `Imports: ${cached.structure.imports?.length ?? 0} | Exports: ${cached.structure.exports?.length ?? 0} | Symbols: ${symbols.length}`,
+            '',
+        ];
+        for (const sym of symbols) {
+            compactLines.push(`  ${sym.kind} ${sym.name} [L${sym.location.startLine}-${sym.location.endLine}]`);
+        }
+        compactLines.push('', `Use read_symbol("${args.path}", "<name>") to drill into any symbol.`);
+        const compactText = compactLines.join('\n');
+        const compactTokens = estimateTokens(compactText);
+        contextRegistry.trackLoad(absPath, { type: 'structure', startLine: 1, endLine: cached.structure.meta.lines, tokens: compactTokens });
+        contextRegistry.setContentHash(absPath, cached.hash);
+        contextRegistry.trackStructureSymbols(absPath, symbols.map(s => s.name));
+        return { content: [{ type: 'text', text: compactText }] };
+    }
+    // 8. Add token savings
     const savings = config.display.showTokenSavings
         ? '\n' + formatSavings(structureTokens, fullTokens)
         : '';
-    // 8. Track
+    // 9. Track
     contextRegistry.trackLoad(absPath, {
         type: 'structure',
         startLine: 1,
@@ -205,7 +228,7 @@ export async function handleSmartRead(args, projectRoot, astIndex, fileCache, co
     });
     contextRegistry.setContentHash(absPath, cached.hash);
     contextRegistry.trackStructureSymbols(absPath, cached.structure.symbols.map(s => s.name));
-    // 9. Confidence metadata
+    // 10. Confidence metadata
     const confidenceMeta = assessConfidence({
         symbolResolved: (cached.structure.symbols?.length ?? 0) > 0,
         fullFile: false,

package/dist/server/tool-definitions.d.ts

@@ -30,6 +30,10 @@ export declare const TOOL_DEFINITIONS: ({
                 enum: string[];
                 description: string;
             };
+            max_tokens: {
+                type: string;
+                description: string;
+            };
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -65,6 +69,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -103,6 +113,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbols?: undefined;
             start_line?: undefined;
             end_line?: undefined;
@@ -133,6 +144,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -170,6 +187,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             include_edit_context?: undefined;
             start_line?: undefined;
@@ -201,6 +219,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -226,6 +250,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -259,6 +284,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -280,6 +311,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -314,6 +346,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -335,6 +373,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -369,6 +408,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -421,6 +466,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             context_before?: undefined;
             context_after?: undefined;
             show?: undefined;
@@ -448,6 +494,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -464,6 +516,10 @@ export declare const TOOL_DEFINITIONS: ({
                 };
                 description: string;
             };
+            max_tokens: {
+                type: string;
+                description: string;
+            };
             path?: undefined;
             show_imports?: undefined;
             show_docs?: undefined;
@@ -503,6 +559,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -547,6 +609,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_imports?: undefined;
             show_docs?: undefined;
             depth?: undefined;
+            max_tokens?: undefined;
             context_before?: undefined;
             context_after?: undefined;
             show?: undefined;
@@ -576,6 +639,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -598,6 +667,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -632,6 +702,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required?: undefined;
     };
@@ -649,6 +725,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -684,6 +761,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -709,6 +792,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -742,6 +826,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -760,6 +850,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -794,6 +885,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required?: undefined;
     };
@@ -820,6 +917,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -852,6 +950,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required?: undefined;
     };
@@ -887,6 +991,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -917,6 +1022,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -940,6 +1051,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -973,6 +1085,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -998,6 +1116,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_imports?: undefined;
             show_docs?: undefined;
             depth?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -1032,6 +1151,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required?: undefined;
     };
@@ -1057,6 +1182,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -1091,6 +1217,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required: string[];
     };
@@ -1116,6 +1248,7 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -1149,6 +1282,12 @@ export declare const TOOL_DEFINITIONS: ({
             command?: undefined;
             runner?: undefined;
             timeout?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
         };
         required?: undefined;
     };
@@ -1176,6 +1315,93 @@ export declare const TOOL_DEFINITIONS: ({
             show_docs?: undefined;
             depth?: undefined;
             scope?: undefined;
+            max_tokens?: undefined;
+            symbol?: undefined;
+            context_before?: undefined;
+            context_after?: undefined;
+            show?: undefined;
+            include_edit_context?: undefined;
+            symbols?: undefined;
+            start_line?: undefined;
+            end_line?: undefined;
+            heading?: undefined;
+            context_lines?: undefined;
+            line?: undefined;
+            context?: undefined;
+            include_callers?: undefined;
+            include_tests?: undefined;
+            include_changes?: undefined;
+            section?: undefined;
+            paths?: undefined;
+            kind?: undefined;
+            limit?: undefined;
+            lang?: undefined;
+            mode?: undefined;
+            include?: undefined;
+            recursive?: undefined;
+            max_depth?: undefined;
+            verbose?: undefined;
+            module?: undefined;
+            export_only?: undefined;
+            check?: undefined;
+            pattern?: undefined;
+            name?: undefined;
+            ref?: undefined;
+            count?: undefined;
+            goal?: undefined;
+            decisions?: undefined;
+            confirmed?: undefined;
+            files?: undefined;
+            blocked?: undefined;
+            next?: undefined;
+        };
+        required: string[];
+    };
+} | {
+    name: string;
+    description: string;
+    inputSchema: {
+        type: "object";
+        properties: {
+            goal: {
+                type: string;
+                description: string;
+            };
+            decisions: {
+                type: string;
+                items: {
+                    type: string;
+                };
+                description: string;
+            };
+            confirmed: {
+                type: string;
+                items: {
+                    type: string;
+                };
+                description: string;
+            };
+            files: {
+                type: string;
+                items: {
+                    type: string;
+                };
+                description: string;
+            };
+            blocked: {
+                type: string;
+                description: string;
+            };
+            next: {
+                type: string;
+                description: string;
+            };
+            path?: undefined;
+            show_imports?: undefined;
+            show_docs?: undefined;
+            depth?: undefined;
+            scope?: undefined;
+            max_tokens?: undefined;
             symbol?: undefined;
             context_before?: undefined;
             context_after?: undefined;
@@ -1208,6 +1434,9 @@ export declare const TOOL_DEFINITIONS: ({
             name?: undefined;
             ref?: undefined;
             count?: undefined;
+            command?: undefined;
+            runner?: undefined;
+            timeout?: undefined;
         };
         required: string[];
     };

package/dist/server/tool-definitions.js

@@ -29,6 +29,8 @@ export const MCP_INSTRUCTIONS = [
     '17. Module architecture → module_info (deps, dependents, public API)',
     '18. Read markdown/yaml/json/csv section → read_section (loads one heading/key/row-range, NOT the whole file)',
     '   - For editing sections: read_for_edit(path, section="Section Name")',
+    '19. Long session / before compaction → session_snapshot (capture goal, decisions, confirmed facts, files, next step as <200 token block)',
+    '   - Budget-constrained? Use smart_read(max_tokens=N) to auto-downgrade output size',
     '',
     'USE DEFAULT TOOLS ONLY FOR: regex text search → Grep | exact raw content → Read | non-code configs → Read',
     '',
@@ -38,6 +40,7 @@ export const MCP_INSTRUCTIONS = [
     '• Docs: smart_read (outline) → read_section → read_for_edit(section=) → Edit → read_diff',
     '• Refactor: find_usages → read_symbols → read_for_edit → Edit → test_summary',
     '• Audit: code_audit + find_unused + Grep (for regex patterns)',
+    '• Long session: session_snapshot → compact context → continue with minimal state',
 ].join('\n');
 export const TOOL_DEFINITIONS = [
     // --- Core reading tools ---
@@ -56,6 +59,7 @@ export const TOOL_DEFINITIONS = [
                     enum: ['full', 'nav', 'exports'],
                     description: 'Output scope: full (default, all details), nav (names + lines only, 2-3x smaller), exports (public API only)',
                 },
+                max_tokens: { type: 'number', description: 'Token budget. If output exceeds this, auto-downgrades: full → outline → compact. Use for context-constrained sessions.' },
             },
             required: ['path'],
         },
@@ -172,6 +176,7 @@ export const TOOL_DEFINITIONS = [
                     items: { type: 'string' },
                     description: 'Array of file paths',
                 },
+                max_tokens: { type: 'number', description: 'Token budget per file. If a file exceeds this, auto-downgrades to compact outline.' },
             },
             required: ['paths'],
         },
@@ -349,5 +354,22 @@ export const TOOL_DEFINITIONS = [
             required: ['command'],
         },
     },
+    // --- Session ---
+    {
+        name: 'session_snapshot',
+        description: 'Capture current session state as a compact markdown block (<200 tokens). Call before compaction, when switching direction, or periodically in long sessions. Model provides the facts, tool formats them.',
+        inputSchema: {
+            type: 'object',
+            properties: {
+                goal: { type: 'string', description: 'Session goal — what and why' },
+                decisions: { type: 'array', items: { type: 'string' }, description: 'Key decisions made and why (e.g., "removed sysfee step — caused double counting"). Prevents revisiting rejected approaches.' },
+                confirmed: { type: 'array', items: { type: 'string' }, description: 'Established facts (what has been verified)' },
+                files: { type: 'array', items: { type: 'string' }, description: 'Relevant file paths' },
+                blocked: { type: 'string', description: 'Current blocker or obstacle' },
+                next: { type: 'string', description: 'Next step to take' },
+            },
+            required: ['goal'],
+        },
+    },
 ];
 //# sourceMappingURL=tool-definitions.js.map

package/dist/server.js

@@ -36,6 +36,7 @@ import { handleSmartDiff } from './handlers/smart-diff.js';
 import { handleExploreArea } from './handlers/explore-area.js';
 import { handleSmartLog } from './handlers/smart-log.js';
 import { handleTestSummary } from './handlers/test-summary.js';
+import { handleSessionSnapshot } from './handlers/session-snapshot.js';
 import { handleReadSection } from './handlers/read-section.js';
 import { detectContextMode } from './integration/context-mode-detector.js';
 import { estimateTokens } from './core/token-estimator.js';
@@ -154,6 +155,8 @@ export async function createServer(projectRoot, options) {
         : null;
     // Policy engine state
     let fullFileReadsCount = 0;
+    let totalCallCount = 0;
+    let totalTokensReturned = 0;
     const readForEditCalled = new Set();
     // Detect context-mode companion
     const cmEnabled = config.contextMode.enabled;
@@ -224,6 +227,8 @@ export async function createServer(projectRoot, options) {
             }),
         });
         // Policy tracking
+        totalCallCount++;
+        totalTokensReturned += rest.tokensReturned;
         if (isFullReadTool(rest.tool)) {
             fullFileReadsCount++;
         }
@@ -235,6 +240,8 @@ export async function createServer(projectRoot, options) {
             fullFileReadsCount,
             tokensReturned: rest.tokensReturned,
             readForEditCalled,
+            totalCallCount,
+            totalTokensReturned,
         });
         return advisory ? `\n${advisory.message}` : null;
     }
@@ -575,6 +582,17 @@ export async function createServer(projectRoot, options) {
                     recordWithTrace({ tool: 'test_summary', path: tsArgs.command, tokensReturned: tsTokens, tokensWouldBe: tsResult.rawTokens || tsTokens, timestamp: Date.now(), savingsCategory: 'compression', args: tsArgs });
                     return { content: tsResult.content };
                 }
+                case 'session_snapshot': {
+                    const snapshotArgs = args;
+                    if (!snapshotArgs.goal) {
+                        return { content: [{ type: 'text', text: 'Error: goal is required' }], isError: true };
+                    }
+                    const snapshotResult = handleSessionSnapshot(snapshotArgs);
+                    const snapshotText = snapshotResult.content[0]?.text ?? '';
+                    const snapshotTokens = estimateTokens(snapshotText);
+                    recordWithTrace({ tool: 'session_snapshot', tokensReturned: snapshotTokens, tokensWouldBe: snapshotTokens, timestamp: Date.now(), savingsCategory: 'compression' });
+                    return { content: snapshotResult.content };
+                }
                 default:
                     return {
                         content: [{ type: 'text', text: `Unknown tool: ${name}` }],

package/dist/types.d.ts

@@ -137,6 +137,8 @@ export interface TokenPilotConfig {
         maxFullFileReads: number;
         warnOnLargeReads: boolean;
         largeReadThreshold: number;
+        compactionCallThreshold: number;
+        compactionTokenThreshold: number;
     };
     ignore: string[];
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "token-pilot",
-  "version": "0.18.0",
+  "version": "0.19.1",
   "description": "Save up to 80% tokens when AI reads code — MCP server for token-efficient code navigation, AST-aware structural reading instead of dumping full files into context window",
   "type": "module",
   "main": "dist/index.js",