context-compress 2026.3.22 → 2026.6.0

package/docs/token-reduction-report.md

@@ -6,7 +6,7 @@
 > provides a detailed before/after comparison for 12 common operations,
 > and addresses the natural question: "doesn't less tokens mean losing context?"
 
-**Version**: 2026.3.21 | **Last updated**: 2026-03-22
+**Version**: 2026.5.0 | **Last updated**: 2026-05-10
 
 ---
 
@@ -46,10 +46,28 @@ The worst part: **99% of that tool output is noise** — import statements, boil
 
 ---
 
-## The Solution: 3-Layer Architecture
+## The Solution: 4-Layer Architecture
 
 context-compress doesn't delete data — it **defers** it. All data is preserved and searchable. Only the relevant parts enter context.
 
+### Layer 0: Command-Specific Output Filters (v2026.3.22)
+
+Before generic compression, output passes through command-aware filters that strip noise specific to each tool:
+
+| Command | What's Stripped | Typical Savings |
+|:--|:--|:--|
+| `git push/pull/fetch/clone` | Remote progress lines, object counting, delta resolution | 40-60% |
+| `git status` | Hint lines (`use "git add"...`), blank lines | 20-30% |
+| `npm/yarn/pnpm install` | Deprecation warnings, funding prompts, tree-drawing chars | 30-50% |
+| `npm test / jest / vitest / pytest` | Passing test details, keeps only failures + summary | 70-95% |
+| `cargo build / make / gradle` | Download progress, "Compiling X/Y", lock waits | 50-70% |
+| `docker build` | Layer hash lines (` ---> abc123`), build context transfer | 30-50% |
+| `ls -R / find / tree` | Directory grouping for large listings (50+ files) | 60-80% |
+
+Additionally, all output passes through:
+- **ANSI stripping**: Terminal escape codes (colors, cursor movement) are always removed — pure noise for LLMs
+- **Progress line removal**: Spinner characters, percentage bars, download/ETA lines are filtered
+
 ### Layer 1: Sandbox Execution
 
 The agent writes code to process data. Only `console.log()` output enters context. 11 languages supported: JavaScript, TypeScript, Python, Shell, Ruby, Go, Rust, PHP, Perl, R, Elixir.
@@ -204,18 +222,28 @@ The following comparison uses realistic output sizes measured from the context-c
 | **After** | 200 | ~50 | `execute` with `intent: "errors"` → only issues shown |
 | **Saved** | | **~3,700** | **98.7% reduction** |
 
+### 13. npm test with ANSI + verbose output (~8KB, v2026.3.22)
+
+| | Bytes | Tokens (est.) | Method |
+|:--|--:|--:|:--|
+| **Before** | 8,000 | ~2,000 | `Bash npm test` → full ANSI-colored verbose output in context |
+| **After** | 350 | ~88 | Command filter strips ANSI + passing tests → only failures + summary |
+| **Saved** | | **~1,912** | **95.6% reduction** |
+
+**Pipeline**: ANSI stripping → command filter (test runner detection) → progress line removal → deduplication → smart truncation. All 5 layers applied automatically.
+
 ---
 
 ## Session Totals
 
-Combining all 12 operations from a single coding session:
+Combining all 13 operations from a single coding session:
 
 ```
-BEFORE:  1,043 KB  →  ~261K tokens consumed (bytes/4 midpoint)
-AFTER:       9 KB  →    ~2.2K tokens consumed
+BEFORE:  1,051 KB  →  ~263K tokens consumed (bytes/4 midpoint)
+AFTER:       9 KB  →    ~2.3K tokens consumed
                        ────────────────────────
-SAVED:   1,035 KB  →  ~259K tokens
-REDUCTION:                99.2%
+SAVED:   1,042 KB  →  ~260K tokens
+REDUCTION:                99.1%
 ```
 
 ---
@@ -229,21 +257,21 @@ Claude Code uses a 200K token context window.
 │                   200,000 token context window               │
 │                                                              │
 │  WITHOUT context-compress:                                   │
-│  ████████████████████████████████████████████████████ ~131%  │
-│  ← 12 operations OVERFLOW the window. Conversation lost.     │
+│  ████████████████████████████████████████████████████ ~132%  │
+│  ← 13 operations OVERFLOW the window. Conversation lost.     │
 │                                                              │
 │  WITH context-compress:                                      │
-│  █ ~1.1%                                                     │
-│  ← 12 operations use ~1.1%. ~98.9% free for conversation.   │
+│  █ ~1.2%                                                     │
+│  ← 13 operations use ~1.2%. ~98.8% free for conversation.   │
 └─────────────────────────────────────────────────────────────┘
 ```
 
 | Metric | Before | After |
 |:--|--:|--:|
-| Tokens consumed (est.) | ~261,000 | ~2,200 |
-| % of context window | ~131% | ~1.1% |
-| Operations before compaction | ~9 | **~1,100** |
-| Conversation longevity | Short | **~119x longer** |
+| Tokens consumed (est.) | ~263,000 | ~2,300 |
+| % of context window | ~132% | ~1.2% |
+| Operations before compaction | ~9 | **~1,080** |
+| Conversation longevity | Short | **~117x longer** |
 
 ---
 
@@ -379,7 +407,7 @@ The other 55,701 bytes are still in FTS5 — fully searchable. Need the order ta
 
 ## Security and Reliability
 
-context-compress v2026.3.21 includes comprehensive security and reliability features:
+context-compress v2026.5.0 includes comprehensive security and reliability features:
 
 ### Security
 
@@ -399,7 +427,8 @@ context-compress v2026.3.21 includes comprehensive security and reliability feat
 |:--|:--|
 | Graceful shutdown | Active subprocess tracking, SIGTERM/SIGINT cleanup, uncaughtException handling |
 | DB resilience | In-memory fallback on disk-full. WAL mode for crash recovery. Stale DB cleanup |
-| Output processing | Line deduplication, error grouping, smart 60/40 head/tail truncation |
+| Output processing | ANSI stripping, progress line removal, command-specific filters, line deduplication, error grouping, smart 60/40 head/tail truncation |
+| Cumulative stats | Cross-session token savings persisted to `stats.json` when `persistDb` is enabled |
 | Search fallback | 3-layer: Porter stemming → trigram (lazy) → Levenshtein fuzzy correction |
 | Configuration | ENV > file > defaults with Zod validation and sanity clamping |
 
@@ -501,6 +530,6 @@ The core principle:
 
 ---
 
-*Generated from real benchmarks on the context-compress v2026.3.21 codebase.*
+*Generated from real benchmarks on the context-compress v2026.5.0 codebase.*
 *Token estimates use bytes/4 midpoint. Actual token counts may vary by 20-30% depending on content type.*
 *See SECURITY.md for the full trust model and security architecture.*

package/hooks/claude-codex-hooks.json

@@ -0,0 +1,19 @@
+{
+  "description": "context-compress PreToolUse hooks for Claude-compatible plugin hosts",
+  "hooks": {
+    "PreToolUse": [
+      {
+        "matcher": "Bash|Read|Grep|WebFetch|Task",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "command -v node >/dev/null 2>&1 && CONTEXT_COMPRESS_FILTER_BASH=1 CONTEXT_COMPRESS_BIN=\"node ${CLAUDE_PLUGIN_ROOT}/dist/cli/index.js\" node \"${CLAUDE_PLUGIN_ROOT}/hooks/pretooluse.mjs\" || exit 0",
+            "commandWindows": "if (Get-Command node -ErrorAction SilentlyContinue) { $env:CONTEXT_COMPRESS_FILTER_BASH='1'; $env:CONTEXT_COMPRESS_BIN='context-compress'; node \"$env:CLAUDE_PLUGIN_ROOT\\hooks\\pretooluse.mjs\" }",
+            "timeout": 5,
+            "statusMessage": "Protecting context window..."
+          }
+        ]
+      }
+    ]
+  }
+}

package/hooks/pretooluse.mjs

@@ -6,6 +6,36 @@ var blockCurl = process.env.CONTEXT_COMPRESS_BLOCK_CURL !== "0";
 var blockWebFetch = process.env.CONTEXT_COMPRESS_BLOCK_WEBFETCH !== "0";
 var nudgeOnRead = process.env.CONTEXT_COMPRESS_NUDGE_READ !== "0";
 var nudgeOnGrep = process.env.CONTEXT_COMPRESS_NUDGE_GREP !== "0";
+var filterBash = process.env.CONTEXT_COMPRESS_FILTER_BASH === "1";
+var ccBin = process.env.CONTEXT_COMPRESS_BIN ?? "context-compress";
+var ccMode = process.env.CONTEXT_COMPRESS_MODE;
+var WRAP_TARGETS = [
+  /^git\s+(status|log|diff|show|blame|branch|stash\s+list|grep|ls-files)/,
+  /^(npm|yarn|pnpm|bun)\s+(install|i|add|test|run\s|update|outdated|audit|list|ls|view|info)/,
+  /^cargo\s+(build|test|check|run|clippy|tree|search|metadata)/,
+  /^(pytest|jest|mocha|vitest|tap|bats)\b/,
+  /^(find|grep|rg|fd|ag|ripgrep)\b/,
+  /^ls\s+(-R|-la|-al)/,
+  /^docker\s+(build|ps|logs|images|inspect|stats)/,
+  /^kubectl\s+(get|describe|logs|top|api-resources)/,
+  /^terraform\s+(plan|show|state\s+list|state\s+show|validate)/,
+  /^helm\s+(list|status|history|get)/,
+  /^(make|gradle|bazel|nx|turbo)\b/,
+  /^ps\s+(aux|-ef)/,
+  /^(top|htop)\b/,
+  /^(df|du)\b/,
+  /^(go|rustc)\s+(test|build|vet|run)/
+];
+function shouldWrap(cmd) {
+  const trimmed = cmd.trim();
+  if (/(?:^|\s)(?:>|>>|\d?>&)\s*\S/.test(trimmed)) return false;
+  if (/\|/.test(trimmed)) return false;
+  if (/&&|\|\||;/.test(trimmed)) return false;
+  return WRAP_TARGETS.some((re) => re.test(trimmed));
+}
+function shellQuote(s) {
+  return `'${s.replace(/'/g, "'\\''")}'`;
+}
 var raw = "";
 process.stdin.setEncoding("utf-8");
 for await (const chunk of process.stdin) raw += chunk;
@@ -37,6 +67,14 @@ if (tool === "Bash") {
       }
     });
   }
+  if (filterBash && shouldWrap(command)) {
+    const modeFlag = ccMode ? ` --mode ${ccMode}` : "";
+    respond({
+      updatedInput: {
+        command: `${ccBin} wrap${modeFlag} ${shellQuote(command)}`
+      }
+    });
+  }
   process.exit(0);
 }
 if (tool === "Read" && nudgeOnRead) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "context-compress",
-  "version": "2026.3.22",
+  "version": "2026.6.0",
   "description": "Context-aware MCP server that compresses tool output for Claude Code",
   "type": "module",
   "main": "dist/server.js",
@@ -12,12 +12,13 @@
     "build:hooks": "esbuild src/hooks/pretooluse.ts --bundle --platform=node --format=esm --outfile=hooks/pretooluse.mjs --target=node18 && node -e \"const{createHash}=require('crypto'),{readFileSync,writeFileSync}=require('fs');const h=createHash('sha256').update(readFileSync('hooks/pretooluse.mjs')).digest('hex');writeFileSync('hooks/pretooluse.sha256',h+'\\n')\"",
     "dev": "tsx src/index.ts",
     "typecheck": "tsc --noEmit",
-    "lint": "biome check src/",
-    "lint:fix": "biome check --write src/",
+    "lint": "biome check --max-diagnostics=200 src/",
+    "lint:fix": "biome check --write --max-diagnostics=200 src/",
     "test": "node --import tsx --test tests/**/*.test.ts",
     "test:unit": "node --import tsx --test tests/unit/*.test.ts",
     "test:integration": "node --import tsx --test tests/integration/*.test.ts",
-    "clean": "rm -rf dist",
+    "clean": "rm -rf dist dist-bin",
+    "build:bin": "bun build --compile --target=bun-darwin-arm64 ./src/cli/lite.ts --outfile=./dist-bin/cc-lite-darwin-arm64 && bun build --compile --target=bun-darwin-x64 ./src/cli/lite.ts --outfile=./dist-bin/cc-lite-darwin-x64 && bun build --compile --target=bun-linux-x64 ./src/cli/lite.ts --outfile=./dist-bin/cc-lite-linux-x64 && bun build --compile --target=bun-linux-arm64 ./src/cli/lite.ts --outfile=./dist-bin/cc-lite-linux-arm64",
     "prepublishOnly": "npm run lint && npm run test && npm run build"
   },
   "engines": {
@@ -37,6 +38,9 @@
     "typescript": "^5.7.0"
   },
   "files": [
+    ".codex-plugin/",
+    ".claude-plugin/",
+    ".mcp.json",
     "dist/",
     "docs/",
     "hooks/",
@@ -45,10 +49,10 @@
     "README.md"
   ],
   "license": "MIT",
-  "repository": {
-    "type": "git",
-    "url": "https://github.com/Open330/context-compress"
-  },
+	"repository": {
+		"type": "git",
+		"url": "git+https://github.com/Open330/context-compress.git"
+	},
   "keywords": [
     "mcp",
     "claude",

package/skills/context-compress-audit/SKILL.md

@@ -0,0 +1,49 @@
+---
+name: context-compress-audit
+description: Audit a repository, plugin setup, or agent session for raw Bash, Read, WebFetch, and MCP outputs that should be routed through context-compress instead. Use when asked to find context waste, raw tool output waste, missing plugin routing, or places where large command/file/web output still enters the agent context window.
+---
+
+# Context-Compress Audit
+
+Find places where large raw output can still enter the agent context window.
+
+## Procedure
+
+1. Run `mcp__context-compress__stats` first if the tool is available.
+2. Inspect setup surfaces that control routing:
+   - `.codex-plugin/plugin.json`
+   - `.claude-plugin/plugin.json`
+   - `.mcp.json`
+   - `hooks/`
+   - `skills/`
+   - README install instructions
+3. Search for risky guidance or examples that encourage raw output:
+   - `Bash` for tests, logs, `git log`, `git diff`, `curl`, `kubectl`, `docker`, `npm test`
+   - `Read` for large logs, bundled files, snapshots, CSV/JSON dumps
+   - `WebFetch` for documentation pages that should use `fetch_and_index`
+   - Playwright snapshots without a file/index/search path
+4. Report only actionable findings. Prefer one-line fixes that route work through:
+   - `batch_execute` for several commands plus searches
+   - `execute` for command/API output that must be analyzed first
+   - `execute_file` for large local files
+   - `fetch_and_index` plus `search` for web documentation
+
+## Output
+
+Use this format:
+
+```md
+## Context-Compress Audit
+
+- [severity] file:line - raw-output risk. Replace with <tool/workflow>.
+- [severity] file:line - missing install/routing coverage. Add <specific fix>.
+
+Summary: <N> findings, estimated impact <low|medium|high>.
+```
+
+Severity:
+- `high`: large output can enter context by default.
+- `medium`: docs/examples teach a wasteful path.
+- `low`: minor wording, missing cross-link, or optional setup gap.
+
+If nothing meaningful is found, say `No raw-output waste found. Routing looks covered.`

package/skills/context-compress-audit/agents/openai.yaml

@@ -0,0 +1,13 @@
+interface:
+  display_name: "Context Compress Audit"
+  short_description: "Find raw-output context waste."
+  default_prompt: "Use $context-compress-audit to audit this project for raw tool output waste and missing routing."
+
+dependencies:
+  tools:
+    - type: "mcp"
+      value: "context-compress"
+      description: "Context-compress MCP server for stats and indexed search."
+
+policy:
+  allow_implicit_invocation: true