token-pilot 0.29.0 → 0.30.1

package/docs/configuration.md

@@ -0,0 +1,117 @@
+# Configuration & Tool Profiles
+
+## .token-pilot.json
+
+Drop `.token-pilot.json` in your project root. All fields are optional.
+
+```json
+{
+  "hooks": { "mode": "deny-enhanced", "denyThreshold": 300 },
+  "sessionStart": { "enabled": true, "showStats": false, "maxReminderTokens": 250 },
+  "agents": { "scope": null, "reminder": true },
+  "smartRead": { "smallFileThreshold": 200 },
+  "cache": { "maxSizeMB": 100, "watchFiles": true },
+  "policies": { "maxFullFileReads": 10, "largeReadThreshold": 2000 },
+  "astIndex": { "binaryPath": null },
+  "updates": { "checkOnStartup": true, "autoUpdate": false },
+  "ignore": ["node_modules/**", "dist/**", ".git/**"]
+}
+```
+
+| Option | Default | What it does |
+|--------|---------|--------------|
+| `hooks.mode` | `"deny-enhanced"` | Read hook mode: `off` / `advisory` / `deny-enhanced` |
+| `hooks.denyThreshold` | `300` | Line count above which the hook intervenes on unbounded `Read` |
+| `sessionStart.enabled` | `true` | Re-inject MCP-rules reminder at every new session / `/clear` / `/compact` |
+| `agents.scope` | `null` | Persisted scope of last `install-agents` run; reused silently |
+| `agents.reminder` | `true` | Show the "agents not installed" startup nudge |
+| `smartRead.smallFileThreshold` | `200` | Files with fewer lines bypass AST overhead and are returned in full |
+| `cache.maxSizeMB` | `100` | File cache ceiling (LRU eviction) |
+| `policies.maxFullFileReads` | `10` | Warn after N full-file reads in session |
+| `policies.largeReadThreshold` | `2000` | Token threshold above which a read is flagged as "large" in analytics |
+
+## Tool Profiles
+
+Trim the advertised `tools/list` to save ~2 k tokens per session. Set via `TOKEN_PILOT_PROFILE` in your MCP server env block.
+
+| Profile | Tools | ~Tokens | Use when |
+|---------|------:|--------:|----------|
+| `full` *(default)* | 22 | ~4 150 | All capabilities |
+| `edit` | 16 | ~3 120 | Code-change workflows (nav + batch reads + `read_for_edit`) |
+| `nav` | 10 | ~1 910 | Read-only exploration / subagents that only navigate |
+
+Handlers remain active regardless of profile — a subagent that explicitly names a filtered-out tool still gets served. The profile only controls what appears in `tools/list` at session start.
+
+### Setting a profile
+
+**In `.mcp.json`:**
+```json
+{
+  "mcpServers": {
+    "token-pilot": {
+      "command": "npx",
+      "args": ["-y", "token-pilot"],
+      "env": { "TOKEN_PILOT_PROFILE": "nav" }
+    }
+  }
+}
+```
+
+**Via shell:**
+```bash
+TOKEN_PILOT_PROFILE=edit npx token-pilot
+```
+
+## CLI Reference
+
+```bash
+token-pilot                              # start MCP server
+token-pilot init                         # create/merge .mcp.json; prompt about subagents
+token-pilot install-agents [--scope=user|project] [--force]
+token-pilot uninstall-agents --scope=user|project
+token-pilot bless-agents                 # extend third-party agents with token-pilot MCP
+token-pilot unbless-agents <name>... | --all
+token-pilot install-hook                 # install PreToolUse hooks
+token-pilot uninstall-hook
+token-pilot stats                        # totals + top files from hook-events.jsonl
+token-pilot stats --session[=<id>] | --by-agent
+token-pilot tool-audit                   # per-tool savings distribution
+token-pilot tool-audit --json
+token-pilot doctor                       # diagnostics (ast-index, config, upstream drift)
+token-pilot doctor --check=env           # env var check only
+token-pilot install-ast-index            # download ast-index binary (auto on first run)
+```
+
+## Architecture
+
+```
+src/
+  index.ts              — CLI entry + MCP server bootstrap
+  server.ts             — MCP server: 22 tool definitions + enforcement mode
+  server/
+    enforcement-mode.ts — TOKEN_PILOT_MODE parsing (advisory / deny / strict)
+  ast-index/            — ast-index binary client + auto-install
+  core/
+    event-log.ts        — hook-events.jsonl + rotation + retention
+    session-analytics.ts, policy-engine.ts, intent-classifier.ts
+  hooks/
+    installer.ts        — hook install/uninstall for Claude Code
+    pre-bash.ts         — PreToolUse:Bash advisor (Bash/sh/eval/loop patterns)
+    pre-grep.ts         — PreToolUse:Grep advisor (symbol-like pattern detection)
+    session-start.ts    — SessionStart reminder handler
+    summary-pipeline.ts — ast-index → regex → head+tail → pass-through
+  cli/
+    install-agents.ts, uninstall-agents.ts
+    bless-agents.ts, unbless-agents.ts, doctor-drift.ts
+    stats.ts, tool-audit.ts
+  templates/agent-builder.ts
+  config/loader.ts, defaults.ts
+  handlers/             — 22 MCP tool handlers
+  git/                  — HEAD + file watchers (cache invalidation)
+
+scripts/
+  build-agents.mjs      — render templates/ → dist/agents/
+  bench-hook.mjs        — hook latency benchmark
+
+templates/agents/       — source for tp-* family + shared preamble + contract
+```

package/docs/hooks.md

@@ -0,0 +1,99 @@
+# Hooks & Enforcement Modes
+
+Token Pilot installs two categories of PreToolUse hooks in Claude Code:
+
+1. **Read hook** — intercepts large `Read` calls (configurable threshold, default 300 lines) and returns a structural summary in the denial reason.
+2. **Grep / Bash hooks** — block heavy recursive patterns (`grep -r`, `find /`, `cat <file.ts>`, unbounded `git log`, bare `git diff`) and redirect to token-pilot MCP equivalents.
+
+## TOKEN_PILOT_MODE — Enforcement Mode
+
+Controls how aggressively both hook categories behave:
+
+| Value | Grep/Bash hooks | MCP output |
+|-------|----------------|------------|
+| `advisory` | Pass all through (no blocking) | No caps |
+| `deny` *(default)* | Block heavy patterns, allow bounded variants | No caps |
+| `strict` | Same as deny, plus auto-cap MCP output (see below) | Capped |
+
+```bash
+# Set in your MCP server env block or shell profile:
+TOKEN_PILOT_MODE=strict npx token-pilot
+```
+
+### Strict-mode MCP output caps
+
+When `TOKEN_PILOT_MODE=strict` and the caller has not set the parameter explicitly:
+
+| Tool | Auto-injected default | Note appended |
+|------|-----------------------|---------------|
+| `smart_read` | `max_tokens: 2000` | Yes |
+| `explore_area` | `include: ["outline"]` | Yes |
+| `find_usages` | `mode: "list"` | Yes |
+| `smart_log` | `count: 20` | Yes |
+
+Pass the parameter explicitly to override the cap.
+
+## Read Hook Modes
+
+The PreToolUse:Read hook has its own mode (separate from enforcement mode). Set in `.token-pilot.json`:
+
+| Mode | Behaviour |
+|------|-----------|
+| `off` | Hook is inert — all `Read` calls pass through |
+| `advisory` | Denies unbounded Read with a short tip pointing at `smart_read` / `read_for_edit` |
+| `deny-enhanced` *(default)* | Denies the Read and returns a full structural summary (imports, exports, declarations) **inside** the denial reason. Works for subagents that lack MCP access. |
+
+```json
+{ "hooks": { "mode": "deny-enhanced", "denyThreshold": 300 } }
+```
+
+## Grep / Bash Hook Rules
+
+The Grep hook redirects symbol-like patterns to `find_usages`. The Bash hook blocks:
+
+| Pattern | Blocked when | Allowed when |
+|---------|-------------|--------------|
+| `grep -r`/`-R` | Always (unbounded) | Has `-m N` bound |
+| `find /`, `find ~` | No `-maxdepth` | Has `-maxdepth N` |
+| `cat <file.ts>` | Code file, no pipeline | In pipeline (`cat … \| head`) or non-code file |
+| `git log` | No count limit | Has `-n N`, `--max-count`, or `\| head` |
+| `git diff` | Bare (no path/flag) | Has path arg or `--stat` |
+| `bash -c "…"`, `eval "…"` | Inner command is heavy | Inner command is benign |
+
+## Installing / Removing Hooks
+
+```bash
+npx token-pilot install-hook      # register PreToolUse hooks in Claude Code
+npx token-pilot uninstall-hook    # remove hooks
+```
+
+Hooks are auto-installed on first server start inside Claude Code. The Claude Code plugin path installs hooks automatically:
+
+```bash
+claude plugin marketplace add https://github.com/Digital-Threads/token-pilot
+claude plugin install token-pilot@token-pilot
+```
+
+## Environment Variables
+
+| Var | Effect |
+|-----|--------|
+| `TOKEN_PILOT_MODE` | `advisory` / `deny` (default) / `strict` — enforcement level for Grep/Bash hooks and MCP output caps |
+| `TOKEN_PILOT_BYPASS=1` | Pass every Read through (Read hook only) |
+| `TOKEN_PILOT_DENY_THRESHOLD=<n>` | Override `hooks.denyThreshold` (default 300) |
+| `TOKEN_PILOT_ADAPTIVE_THRESHOLD=true` | Enable adaptive curve as session burns |
+| `TOKEN_PILOT_DEBUG=1` | Verbose hook logging to stderr |
+| `TOKEN_PILOT_NO_AGENT_REMINDER=1` | Suppress the "tp-* not installed" stderr nudge |
+| `TOKEN_PILOT_SUBAGENT=1` | Mark the MCP server as running inside a subagent |
+
+## Analytics & Audit
+
+```bash
+token-pilot stats                          # totals + top files from hook-events.jsonl
+token-pilot stats --session[=<id>]         # filter by session
+token-pilot stats --by-agent              # grouped by agent
+token-pilot tool-audit                    # per-tool savings distribution (cumulative)
+token-pilot tool-audit --json             # machine-readable output
+```
+
+Hook events accumulate in `.token-pilot/hook-events.jsonl`. The `session_analytics` MCP tool provides per-tool breakdown within the current session.

package/docs/installation.md

@@ -0,0 +1,169 @@
+# Installation Guide
+
+## TL;DR
+
+**Claude Code — plugin (recommended):**
+```bash
+# New machine:
+claude plugin marketplace add https://github.com/Digital-Threads/token-pilot
+claude plugin install token-pilot@token-pilot
+
+# Update:
+claude plugin update token-pilot
+```
+
+**Other clients (Cursor, Codex, Cline, …):**
+```bash
+# New machine:
+npx -y token-pilot init
+
+# Update — npx always pulls fresh, just restart your client.
+# Or if installed globally:
+npm i -g token-pilot@latest
+npx token-pilot install-hook
+npx token-pilot install-agents --scope=user --force
+```
+
+---
+
+## First-time setup (full walkthrough)
+
+```bash
+npx -y token-pilot init
+```
+
+Writes `.mcp.json` (or merges into an existing one), adds `token-pilot` + [`context-mode`](https://github.com/mksglu/claude-context-mode), then prompts to install `tp-*` subagents. Restart your AI assistant to activate.
+
+---
+
+## Claude Code
+
+Three paths — pick one, they're mutually exclusive.
+
+### A. Plugin (one-step: hooks + MCP registered together)
+
+```bash
+claude plugin marketplace add https://github.com/Digital-Threads/token-pilot
+claude plugin install token-pilot@token-pilot
+```
+
+Claude Code clones the repo into `~/.claude/plugins/cache/token-pilot/`, sets `CLAUDE_PLUGIN_ROOT`, and registers the MCP server + all hooks declared in `.claude-plugin/hooks/hooks.json`. No `install-hook` call needed. Run `install-agents` separately for the `tp-*` subagents.
+
+### B. MCP config (npm-based, no plugin system)
+
+```bash
+claude mcp add token-pilot -- npx -y token-pilot
+# or for a specific scope:
+claude mcp add --scope user    token-pilot -- npx -y token-pilot
+claude mcp add --scope project token-pilot -- npx -y token-pilot
+```
+
+Or edit `.mcp.json` (project-level) / `~/.mcp.json` (user-level) directly:
+
+```json
+{
+  "mcpServers": {
+    "token-pilot": { "command": "npx", "args": ["-y", "token-pilot"] },
+    "context-mode": { "command": "npx", "args": ["-y", "claude-context-mode"] }
+  }
+}
+```
+
+Then:
+```bash
+npx token-pilot install-hook                   # register PreToolUse hooks
+npx token-pilot install-agents --scope=user    # install tp-* subagents
+```
+
+### C. One-liner
+
+```bash
+npx -y token-pilot init
+```
+
+Writes path B config for you, then prompts about subagents.
+
+---
+
+## Cursor
+
+Cursor reads `.cursor/mcp.json` (project) or `~/.cursor/mcp.json` (global):
+
+```json
+{
+  "mcpServers": {
+    "token-pilot": { "command": "npx", "args": ["-y", "token-pilot"] }
+  }
+}
+```
+
+---
+
+## Codex CLI
+
+Codex reads `~/.codex/config.toml`:
+
+```toml
+[mcp_servers.token-pilot]
+command = "npx"
+args = ["-y", "token-pilot"]
+```
+
+---
+
+## Cline (VS Code)
+
+Cline reads `cline_mcp_settings.json` (accessible via Cline panel → MCP Servers → Edit):
+
+```json
+{
+  "mcpServers": {
+    "token-pilot": { "command": "npx", "args": ["-y", "token-pilot"] }
+  }
+}
+```
+
+---
+
+## Gemini CLI
+
+Add to `~/.gemini/settings.json` or follow Gemini CLI MCP documentation for your version.
+
+---
+
+## Any MCP-compatible client
+
+The server is a plain stdio process:
+
+```
+command: npx
+args:    -y token-pilot
+```
+
+No env vars required. Common optional overrides:
+
+| Env var | Default | Purpose |
+|---------|---------|---------|
+| `TOKEN_PILOT_MODE` | `deny` | `advisory` / `deny` / `strict` — enforcement level |
+| `TOKEN_PILOT_PROFILE` | `full` | `nav` / `edit` / `full` — trims `tools/list` payload |
+| `TOKEN_PILOT_DENY_THRESHOLD` | `300` | Line count above which the Read hook intervenes |
+| `TOKEN_PILOT_ADAPTIVE_THRESHOLD` | `false` | Enable adaptive curve as session burns |
+| `TOKEN_PILOT_BYPASS` | unset | Set to `1` to disable the Read hook for one session |
+| `TOKEN_PILOT_SKIP_POSTINSTALL` | unset | Skip `ast-index` safety-net install at `npm install` time |
+
+---
+
+## From source (contributors / vendored installs)
+
+```bash
+git clone https://github.com/Digital-Threads/token-pilot.git
+cd token-pilot && npm install && npm run build
+# Point your client's config at dist/index.js:
+#   "command": "node", "args": ["/abs/path/to/token-pilot/dist/index.js"]
+```
+
+---
+
+## Non-Claude clients
+
+`install-agents` detects non-Claude clients via env vars + filesystem markers (`CURSOR_TRACE_ID`, `~/.codex/`, `~/.gemini/`, etc.) and **skips installing subagents** unless you pass `--scope=user|project` explicitly. Cursor, Codex, Gemini, and Cline users get all 22 MCP tools + PreToolUse hooks without the `tp-*` agents.

package/docs/tools.md

@@ -0,0 +1,61 @@
+# MCP Tools Reference
+
+Token Pilot exposes 22 MCP tools. All handlers remain active regardless of [tool profile](configuration.md#tool-profiles) — the profile only trims what appears in `tools/list` at session start.
+
+## Reading
+
+| Tool | Instead of | Purpose |
+|------|-----------|---------|
+| `smart_read` | `Read` | AST outline; 90% fewer tokens on large files |
+| `read_symbol` | `Read` + scroll | One class/function by name (`Class.method` supported) |
+| `read_symbols` | N × `read_symbol` | Batch up to 10 symbols from one file |
+| `read_for_edit` | `Read` before `Edit` | Minimal raw code around a symbol — copy directly as `old_string` |
+| `read_range` | `Read` offset | Specific line range |
+| `read_section` | `Read` | Section by heading (Markdown) or key (YAML/JSON/CSV) |
+| `read_diff` | re-`Read` after edit | Changed hunks since last `smart_read` |
+| `smart_read_many` | multiple `Read` | Batch smart_read for up to 20 files |
+
+## Search & Navigation
+
+| Tool | Instead of | Purpose |
+|------|-----------|---------|
+| `find_usages` | `Grep` (refs) | All usages of a symbol; filters by scope/kind/lang/mode |
+| `project_overview` | `ls` + explore | Project type, frameworks, architecture, directory map |
+| `related_files` | manual | Import graph: imports, importers, test files |
+| `outline` | multiple `smart_read` | Compact symbol overview of all code in a directory |
+| `find_unused` | manual | Dead code detection — unreferenced exported symbols |
+| `code_audit` | multiple `Grep` | TODOs, deprecated symbols, structural patterns |
+| `module_info` | manual | Deps, dependents, public API, unused deps |
+| `smart_diff` | raw `git diff` | Structural diff with symbol mapping |
+| `explore_area` | 3–5 calls | Structure + imports + tests + recent changes in one call |
+| `smart_log` | raw `git log` | Structured commits with category detection |
+| `test_summary` | raw test output | Run tests → pass/fail summary + failure details |
+
+## Session
+
+| Tool | Purpose |
+|------|---------|
+| `session_snapshot` | Compact markdown snapshot (<200 tokens) of goal, decisions, facts, blockers, next step. Auto-persisted to `.token-pilot/snapshots/latest.md`. |
+| `session_budget` | Hook-suppression pressure for this session: saved tokens, burn fraction, effective denyThreshold, time-to-compact projection. |
+| `session_analytics` | Token savings: per-tool breakdown, top files, policy advisories. |
+
+## `find_usages` modes
+
+`find_usages` accepts a `mode` parameter:
+
+| Mode | Output | Tokens |
+|------|--------|--------|
+| `full` | Symbol usages with surrounding code context | ~5–20× more |
+| `list` *(strict default)* | File:line pairs only — 5–10× smaller | smallest |
+
+In `TOKEN_PILOT_MODE=strict`, `mode` defaults to `"list"` when not set by the caller. Pass `mode: "full"` explicitly to override.
+
+## `smart_read` scopes
+
+| Scope | Output |
+|-------|--------|
+| *(default)* | Full AST outline with types, signatures, docs |
+| `nav` | Names + line numbers only (2–3× smaller) |
+| `exports` | Public API surface only |
+
+In `TOKEN_PILOT_MODE=strict`, `max_tokens` defaults to 2 000 when not set by the caller.

package/hooks/hooks.json

@@ -19,6 +19,24 @@
           }
         ]
       },
+      {
+        "matcher": "MultiEdit",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "node ${CLAUDE_PLUGIN_ROOT}/dist/index.js hook-edit"
+          }
+        ]
+      },
+      {
+        "matcher": "Write",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "node ${CLAUDE_PLUGIN_ROOT}/dist/index.js hook-edit"
+          }
+        ]
+      },
       {
         "matcher": "Bash",
         "hooks": [

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "token-pilot",
-  "version": "0.29.0",
-  "description": "Save up to 80% tokens when AI reads code \u2014 MCP server for token-efficient code navigation, AST-aware structural reading instead of dumping full files into context window",
+  "version": "0.30.1",
+  "description": "Save up to 80% tokens when AI reads code — MCP server for token-efficient code navigation, AST-aware structural reading instead of dumping full files into context window",
   "type": "module",
   "main": "dist/index.js",
   "bin": {

package/start.sh

@@ -5,24 +5,34 @@
 set -e
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+
+# Capture user's working directory BEFORE `cd $SCRIPT_DIR` — this is the
+# project root fallback when CLAUDE_PROJECT_DIR is not set. Without this,
+# `$(pwd)` below would resolve to the plugin cache dir and every relative
+# file path would be resolved inside the plugin install, not the project.
+USER_CWD="${PWD:-$(pwd)}"
+
 cd "$SCRIPT_DIR"
 
 # 1. Install runtime dependencies when the package is incomplete.
 if [ ! -f "node_modules/@modelcontextprotocol/sdk/package.json" ]; then
-  echo "[token-pilot] Installing runtime dependencies..." >&2
-  npm install --production --no-audit --no-fund 2>&1 >&2
+	echo "[token-pilot] Installing runtime dependencies..." >&2
+	npm install --production --no-audit --no-fund 2>&1 >&2
 fi
 
 # 2. Build if dist/ is missing.
 if [ ! -f "dist/index.js" ]; then
-  echo "[token-pilot] Building..." >&2
-  if [ ! -f "node_modules/typescript/bin/tsc" ]; then
-    npm install --no-audit --no-fund 2>&1 >&2
-  fi
-  npm run build 2>&1 >&2
+	echo "[token-pilot] Building..." >&2
+	if [ ! -f "node_modules/typescript/bin/tsc" ]; then
+		npm install --no-audit --no-fund 2>&1 >&2
+	fi
+	npm run build 2>&1 >&2
 fi
 
 # 3. Start the MCP server.
-# Pass CLAUDE_PROJECT_DIR as project root if available, otherwise cwd.
-PROJECT_ROOT="${CLAUDE_PROJECT_DIR:-$(pwd)}"
+# Priority for project root:
+#   1. CLAUDE_PROJECT_DIR  — set by Claude Code for the active workspace
+#   2. USER_CWD            — the user's shell working directory (captured pre-cd)
+# Passing the plugin cache dir here would poison every relative path lookup.
+PROJECT_ROOT="${CLAUDE_PROJECT_DIR:-$USER_CWD}"
 exec node "$SCRIPT_DIR/dist/index.js" "$PROJECT_ROOT"