compendium-mcp 0.6.2 → 0.7.0

package/README.md

@@ -4,32 +4,15 @@
   <img src="assets/logo.svg" alt="Compendium" width="420" />
 </p>
 
-MCP server that **minimizes LLM token usage** by compressing, summarizing, filtering, and chunk-referencing large context before it reaches the model.
+MCP server that **shrinks noisy context before it reaches the model**: filter logs, compress bulky text/JSON, redact secrets, chunk huge files, and keep only what a question needs.
 
-Built in Rust with the official [`rmcp`](https://crates.io/crates/rmcp) SDK.
+One tool (`compendium`), one `action` field, deterministic heuristics that need **no model**. An optional local Ollama model improves the two "smart" actions.
 
-## Why / when to use Compendium
+## Install (Cursor / Claude Desktop)
 
-Use it when an agent is about to paste **large or noisy context** into the model (build logs, test dumps, API JSON, untrusted web/tool text, long chat, or a fresh workspace). The goal is a **smaller, safer, still-useful** prompt — not another planner or agent runtime.
+Requires Node.js 18+. No Rust toolchain needed — the launcher downloads a prebuilt binary once.
 
-| Situation | Call |
-|-----------|------|
-| Unsure which action | `catalog` → `help` + `id` (or read `cmp://skill/…`) |
-| New task in a repo | `brief` with a short `query` |
-| Noisy terminal / CLI dump | `filter` (generic) or `compress_output` (cargo/npm/docker/git/…) |
-| Bulky text/JSON to densify | `compress` (small inputs bypass unless `force`) |
-| Untrusted paste / secrets / IPI | `sanitize` (or `sanitize_input: true` on the next action) |
-| Guided recipe | `playbooks` → `playbook` |
-
-Heuristic paths work with **no local model**. Optional loopback LLM improves `summarize_smart` / hybrid `rerank` / smart `filter_relevant`.
-
-## Quick start (Cursor)
-
-You need **Node.js 18+**. Compendium itself arrives via npm — no Rust install required.
-
-### 1. Add the MCP server
-
-Open Cursor MCP settings (`~/.cursor/mcp.json` or the project `.cursor/mcp.json`) and add:
+Add to `~/.cursor/mcp.json` (or the project's `.cursor/mcp.json`, or Claude Desktop's MCP config):
 
 ```json
 {
@@ -42,124 +25,44 @@ Open Cursor MCP settings (`~/.cursor/mcp.json` or the project `.cursor/mcp.json`
 }
 ```
 
-Restart MCP / reload Cursor. You should see one tool named **`compendium`**.
-
-That alone is enough: filter, compress, summarize, cache, and BM25 actions all work **without** a local model (fast heuristics).
+Reload MCP. You should see a single tool named `compendium`. That's it.
 
-### 2. (Optional) Smarter summaries with Ollama
+Prebuilt binaries: macOS (arm64, x64), Linux (x64, arm64), Windows (x64). Other platforms: `cargo build --release` and set `COMPENDIUM_BINARY=/path/to/compendium` in the server `env`.
 
-Want better `summarize_smart` / `filter_relevant`? Run a small model on your machine and point Compendium at it.
+### Optional: smarter summaries with Ollama
 
-1. Install [Ollama](https://ollama.com/) and start it (default: `http://127.0.0.1:11434`).
-2. Pull a chat model, for example:
+`summarize_smart` and `filter_relevant` use a small local model when one is configured; otherwise they fall back to heuristics. One command sets it up:
 
 ```bash
-ollama pull qwen:latest
-# or a smaller one: ollama pull qwen2.5:3b
-```
-
-3. Extend the MCP `env` block (URL must stay on **localhost** — Compendium blocks remote hosts on purpose):
-
-```json
-{
-  "mcpServers": {
-    "compendium": {
-      "command": "npx",
-      "args": ["-y", "compendium-mcp"],
-      "env": {
-        "COMPENDIUM_LOCAL_LLM_URL": "http://127.0.0.1:11434/v1",
-        "COMPENDIUM_LOCAL_LLM_MODEL": "qwen:latest"
-      }
-    }
-  }
-}
+npx -y compendium-mcp setup-ollama --write-mcp
 ```
 
-4. Reload MCP, then ask the agent to call `compendium` with `action: "summarize_smart"`.  
-   In the result, `"backend": "local_llm"` means Ollama answered; `"heuristic"` means it fell back (Ollama down, wrong model name, or URL missing).
-
-**Notes**
+It detects (or `--install`s) [Ollama](https://ollama.com/), pulls `qwen2.5:3b`, probes `http://127.0.0.1:11434/v1`, and writes `COMPENDIUM_LOCAL_LLM_URL` / `COMPENDIUM_LOCAL_LLM_MODEL` into `~/.cursor/mcp.json` (`--project` for `.cursor/mcp.json`). Reload MCP and call `{"action":"llm_status"}` — `reachable: true` means smart actions will report `backend: "local_llm"`.
 
-- Package name on npm is **`compendium-mcp`** (`compendium` was already taken). The CLI binary name is still `compendium`.
-- First Ollama reply can be slow while the model loads; later calls are faster.
-- Other local OpenAI-compatible servers work the same way (e.g. Lemonade `http://127.0.0.1:13305/api/v1`). See [Environment](#environment).
+Any OpenAI-compatible **loopback** server works (llama.cpp, Lemonade, …): set the two env vars yourself. Remote hosts are rejected on purpose.
 
-Smoke-check from a terminal (any folder **except** this git repo root is fine):
-
-```bash
-npx -y compendium-mcp --help
-```
+## Actions
 
-Binary packaging details for maintainers: [npm/DISTRIBUTION.md](npm/DISTRIBUTION.md).
+| `action` | Use it for | Main fields |
+|----------|------------|-------------|
+| `filter` | Strip ANSI/boilerplate, collapse whitespace, keep/drop regex lines | `text`, `filter?`, `query?` |
+| `compress_output` | Domain-aware scrub of cargo / npm / docker / git / kubectl output | `text`, `output?` |
+| `compress` | Dense rewrite of bulky text, code, or JSON | `text`, `compress?` |
+| `summarize` | Outline / conversation / file-tree summary | `text`, `summarize?` |
+| `summarize_smart` | Local-model summary (heuristic fallback) | `text`, `smart?` |
+| `filter_relevant` | Keep only lines relevant to a question (local model or BM25) | `text`, `query` |
+| `sanitize` | Redact secrets, neutralize prompt-injection phrases | `text`, `sanitize?` |
+| `rerank` | BM25-rank candidates for a query | `query`, `items` / `text` / chunk `map` |
+| `chunk` / `resolve` | Split a huge corpus into `cmp://` chunks; fetch one by id | `text`, `chunk?` / `id` |
+| `cache_store` / `cache_get` / `cache_invalidate` | Park a blob outside the prompt under a short key | `text`, `cache?` / `key` |
+| `count_tokens` | Measure size | `text` |
+| `stats` | Session savings per action | `reset?` |
+| `llm_status` | Is the local model configured and reachable? | `force?` |
+| `help` | List actions, or fields + example for one action | `id?` |
 
-## Community
+Every response is `{ "ok": true, "action": "filter", "result_json": "{…}" }` — parse `result_json` for the action payload. Add `"sanitize_input": true` to any text action to scrub secrets first. `compress` / `summarize` leave inputs under 1000 characters untouched unless `force: true`.
 
-- [Contributing](CONTRIBUTING.md)
-- [Changelog](CHANGELOG.md)
-- [Architecture](docs/architecture.md)
-- [Code of Conduct](CODE_OF_CONDUCT.md)
-- [Security policy](SECURITY.md)
-- [Support](SUPPORT.md)
-
-## Transports
-
-| Mode | Command | Notes |
-|------|---------|-------|
-| **stdio** (default) | `compendium` / `compendium stdio` | Cursor / Claude Desktop — dual-compat (legacy initialize or modern connect) |
-| **Streamable HTTP** | `compendium http [BIND]` | Requires `--features http`. Endpoint: `http://{bind}/mcp`. Sessionless (`2026-07-28`); JSON preferred, SSE fallback |
-
-Default HTTP bind: `127.0.0.1:8788` (override with arg or `COMPENDIUM_HTTP_BIND`). App cache (`COMPENDIUM_CACHE_DIR`) is not an MCP session — set it for multi-request HTTP. See playbook `http-transport`.
-
-## Tools
-
-Single MCP tool: **`compendium`**. Choose the operation with `action`. Prefer the [Why / when](#why--when-to-use-compendium) table for the first call; use the full list below only when you need a specific field.
-
-| `action` | Purpose | Main fields |
-|----------|---------|-------------|
-| `filter` | Strip ANSI, boilerplate, whitespace; densify JSON; keep/drop regexes | `text`, `filter` — not for cargo/npm dumps (`compress_output`) |
-| `compress` | Dense representation of text/code/logs | `text`, `compress` — soft inputs under ~1000 chars bypass unless `force` |
-| `compress_output` | Domain-aware stdout/stderr scrub (git, cargo, npm, docker, …) | `text`, `output` — prefer when CLI domain is known |
-| `summarize` | Hierarchical summary (conversation / file tree / outline) | `text`, `summarize` |
-| `summarize_smart` | Local-SLM dense summary (heuristic fallback if unset/fails) | `text`, `smart?`, `summarize?` |
-| `filter_relevant` | Query-aware keep of relevant lines (local SLM + heuristic fallback) | `text`, `query`, `smart?` |
-| `prune_history` | Drop filler / compress older chat turns | `text` or `messages`, `prune` |
-| `chunk` | Split into `cmp://` chunks (session-cached) | `text`, `chunk` |
-| `resolve` | Fetch chunk content by id | `id` (+ optional `map` / `text`) |
-| `count_tokens` | Measure tokens | `text` |
-| `stats` | Session savings + latency/bypass/backend telemetry | `reset?` — see playbook `stats-debug` |
-| `cache_store` | Park bulky payload outside the prompt | `text`, `cache` |
-| `cache_get` | Retrieve by key | `key` |
-| `cache_invalidate` | Drop one key or clear cache | `key?` |
-| `sanitize` | Redact secrets + neutralize IPI phrases | `text`, `sanitize?` — or `sanitize_input` |
-| `rerank` | BM25 (+ optional loopback embeddings + opt-in SLM cross-encoder) rank candidates / chunks | `query`, `items` or `text` or chunk `map`, `rerank?` |
-| `brief` | Scan a workspace; pack a structured starter briefing + cache key | `query`, `brief?` (`root`, caps), optional `text` hint |
-| `catalog` | Short action (+ playbook) ads; prefer before guessing | _(none)_ — call first when unsure |
-| `help` | Usage notes for one action (default **compressed**; `force: true` → full) | `id`, `force?` |
-| `playbooks` | List playbook ads | _(none)_ |
-| `playbook` | Load one playbook body | `id` |
-| `pack` | Zip text/files into a bounded archive | `text` or `items`, `pack?` |
-| `unpack` | Unpack zip with size caps into chunks (never runs scripts) | `text` or `key`, `pack?` |
-| `llm_status` | Probe configured local LLM (models; `force` = chat ping) | `force?` — when smart/hybrid unexpectedly heuristic |
-
-### Progressive disclosure (skills)
-
-Tool description/instructions stay thin. Discover details on demand:
-
-- **Tool bridge:** `action=catalog` → `action=help` with `id`, or `playbooks` → `playbook`
-- **MCP resources:** `resources/list` / `resources/read` on:
-  - `cmp://skill/index` — JSON index of actions + playbooks
-  - `cmp://skill/action/{name}` — full action help (markdown)
-  - `cmp://skill/playbook/{id}` — playbook body
-
-Bundled playbooks live under [`playbooks/`](playbooks/). Override/extend with `COMPENDIUM_PLAYBOOKS_DIR` (same `id` wins). Archives honor `COMPENDIUM_ARCHIVE_MAX_BYTES` / `_UNCOMPRESSED` / `_FILES` (defaults 2 MiB / 4 MiB / 50).
-
-Optional on most text actions: `sanitize_input: true` scrubs before processing. Soft payloads under `COMPENDIUM_SIGNAL_MIN_CHARS` (default 1000) bypass `compress` / `summarize` / `summarize_smart` unless `force: true`.
-
-`filter` accepts optional `query` (top-level or `filter.query`) for BM25 line keep. `prune_history` supports `prune.strategy: "afm"` (Critical / Thematic / Distant tiers; distant blob cached for `cache_get`).
-
-`brief` walks `brief.root` (default: process cwd) with `.gitignore` / `.ignore`, BM25-ranks paths/chunks, window-reads oversized files (not head-truncate), and returns a structured `briefing`: **Task / Status / Evidence / Caveats / Sources / Read next**, plus `cache_key`. Status uses a local SLM when `COMPENDIUM_LOCAL_LLM_URL` is set (`backend: local_llm`); otherwise heuristic bullets. Caveats flag truncated files and docs older than selected code. **Read next** includes source paths plus suggested `cmp://skill/playbook/…` / action URIs. Optional `COMPENDIUM_BRIEF_ROOT` restricts allowed roots. Briefings are sanitized by default.
-
-Example — noisy log (canonical first call after install):
+Example:
 
 ```json
 {
@@ -169,254 +72,45 @@ Example — noisy log (canonical first call after install):
 }
 ```
 
-Discover more without reading this README: `{"action":"catalog"}` then `{"action":"help","id":"compress_output"}`. Sample payloads: [`examples/`](examples/).
-
-Response envelope: `{ "ok": true, "action": "filter", "result_json": "{...}" }`. Parse `result_json` as JSON for the action-specific payload.
-
-## Project layout
-
-```
-assets/                # brand mark (SVG/PNG); baked into MCP icons via data URI
-docs/                  # architecture notes
-examples/              # sample MCP tool-call JSON payloads
-testdata/              # eval fixtures (logs, audit, PR JSON, untrusted paste, …)
-src/
-  main.rs              # CLI: stdio | http
-  lib.rs
-  brand.rs             # SEP-973 icons for serverInfo + tool
-  config.rs            # COMPENDIUM_* env config
-  server/              # MCP tool + resources + action handlers (rmcp)
-  http.rs              # Streamable HTTP, sessionless (feature = "http")
-  pipeline/
-    brief/             # workspace brief (walk / window / pack / synthesize)
-    tokens.rs          # heuristic or tiktoken BPE (feature = "real-tokens")
-    filter.rs
-    compress.rs
-    summarize.rs
-    smart.rs           # summarize_smart + filter_relevant
-    local_llm.rs       # OpenAI-compatible local SLM client (+ embed cache)
-    chunk.rs           # chunk + resolve
-    cache.rs           # session key/value cache (+ optional disk / embed vectors)
-    catalog.rs         # action ads + help (progressive disclosure)
-    playbook.rs        # bundled / dir playbooks
-    pack.rs            # zip pack/unpack with size caps
-    stats.rs           # session savings counters
-    prune.rs           # conversation history pruning
-    output.rs          # domain-aware compress_output
-playbooks/             # embedded skill-md playbooks
-tests/
-  integration.rs
-  e2e_smoke.rs         # spawns binary, MCP handshake, tools + resources
-  eval_regression.rs   # B1 heuristic quality + latency smoke
-CHANGELOG.md
-REPORT.md              # design essay + Shipped (A–C) / Next ops / Deferred roadmap
-```
-
-## Build
-
-```bash
-# Default: heuristic tokens + stdio only
-cargo build --release
-
-# Exact BPE token counts (tiktoken-rs)
-cargo build --release --features real-tokens
-
-# Streamable HTTP transport
-cargo build --release --features http
-
-# Everything
-cargo build --release --features real-tokens,http
-```
-
-Binary: `target/release/compendium`
-
-## Configure (advanced)
-
-The [Quick start](#quick-start-cursor) config is enough for most people. Extra options:
-
-### Claude Desktop
-
-Same `command` / `args` / `env` as Cursor, in Claude’s MCP config file.
-
-### Optional tuning env
-
-```json
-"env": {
-  "RUST_LOG": "compendium=info",
-  "COMPENDIUM_DEFAULT_MAX_TOKENS": "2048",
-  "COMPENDIUM_TOKENIZER": "cl100k_base",
-  "COMPENDIUM_LOCAL_LLM_URL": "http://127.0.0.1:11434/v1",
-  "COMPENDIUM_LOCAL_LLM_MODEL": "qwen:latest"
-}
-```
-
-### Local Cargo binary (developers)
-
-After code changes, rebuild and **reload MCP** so the live tool schema matches source (avoid stale `npx`/Release binaries during development):
-
-```bash
-cargo build --release --features real-tokens,http
-```
-
-```json
-{
-  "mcpServers": {
-    "compendium": {
-      "command": "/absolute/path/to/Compendium/target/release/compendium",
-      "env": {
-        "RUST_LOG": "compendium=info",
-        "COMPENDIUM_DEFAULT_MAX_TOKENS": "2048"
-      }
-    }
-  }
-}
-```
-
-### Remote / sidecar (HTTP)
-
-```bash
-cargo run --features http -- http 127.0.0.1:8788
-# MCP endpoint: http://127.0.0.1:8788/mcp
-```
-
-Point an MCP streamable-HTTP client at that URL (e.g. `StreamableHttpClientTransport::from_uri`).
+More samples in [`examples/`](examples/). Unsure which action fits? Call `{"action":"help"}`.
 
 ## Environment
 
+All optional.
+
 | Variable | Default | Meaning |
 |----------|---------|---------|
-| `COMPENDIUM_CHARS_PER_TOKEN` | `4.0` | Heuristic chars÷tokens (ignored with `real-tokens`) |
-| `COMPENDIUM_TOKENIZER` | `cl100k_base` | BPE encoding: `cl100k_base` or `o200k_base` (`real-tokens`) |
-| `COMPENDIUM_DEFAULT_MAX_TOKENS` | `2048` | Soft cap for compress |
-| `COMPENDIUM_MAX_BLANK_LINES` | `1` | Blank-line collapse limit |
-| `COMPENDIUM_SIMILARITY_THRESHOLD` | `0.85` | Jaccard line-dedupe threshold |
-| `COMPENDIUM_HTTP_BIND` | `127.0.0.1:8788` | Default HTTP listen address |
-| `COMPENDIUM_LOCAL_LLM_URL` | _(unset)_ | OpenAI-compatible base URL (e.g. `http://127.0.0.1:11434/v1` or `http://127.0.0.1:13305/api/v1`). Enables smart actions. |
-| `COMPENDIUM_LOCAL_LLM_MODEL` | `Qwen3-4B-GGUF` | Model id on that server (Ollama: e.g. `qwen:latest`) |
-| `COMPENDIUM_LOCAL_EMBED_MODEL` | _(same as chat)_ | Embeddings model for hybrid `rerank` / `brief` (e.g. `nomic-embed-text`) |
-| `COMPENDIUM_HYBRID_ALPHA` | `0.55` | BM25 weight in hybrid score (0–1); remainder is embedding cosine |
-| `COMPENDIUM_RERANK_CROSS_ENCODER` | _(off)_ | When `1`/`true`, `rerank` SLM-rescores top-N after BM25/hybrid |
-| `COMPENDIUM_CROSS_ENCODER_TOP_N` | `16` | Candidates passed to cross-encoder (clamped 4–64) |
-| `COMPENDIUM_AUDIT_PATH` | _(unset)_ | Append-only JSONL audit log (action metadata only; no payloads) |
-| `COMPENDIUM_LOCAL_LLM_API_KEY` | _(unset)_ | Optional bearer token for locked loopback servers |
+| `COMPENDIUM_LOCAL_LLM_URL` | _(unset)_ | OpenAI-compatible loopback base URL; enables smart actions |
+| `COMPENDIUM_LOCAL_LLM_MODEL` | `qwen2.5:3b` | Model id on that server |
+| `COMPENDIUM_LOCAL_LLM_API_KEY` | _(unset)_ | Bearer token for locked loopback servers |
 | `COMPENDIUM_LOCAL_LLM_TIMEOUT_SECS` | `120` | HTTP timeout (first model load can be slow) |
+| `COMPENDIUM_LOCAL_LLM_REASONING_EFFORT` | `low` | `low`/`medium`/`high`/`off`; `omit` drops the field for strict servers |
+| `COMPENDIUM_DEFAULT_MAX_TOKENS` | `2048` | Soft output cap for compress/summarize |
 | `COMPENDIUM_SIGNAL_MIN_CHARS` | `1000` | Bypass compress/summarize below this length (`0` disables) |
-| `COMPENDIUM_BRIEF_ROOT` | _(unset)_ | When set, `action=brief` may only scan roots under this canonical path |
-| `COMPENDIUM_PLAYBOOKS_DIR` | _(unset)_ | Extra/override playbook `*.md` directory (same `id` replaces embedded) |
-| `COMPENDIUM_ARCHIVE_MAX_BYTES` | `2097152` | Max compressed archive size for pack/unpack |
-| `COMPENDIUM_ARCHIVE_MAX_UNCOMPRESSED` | `4194304` | Max total uncompressed bytes for pack/unpack |
-| `COMPENDIUM_ARCHIVE_MAX_FILES` | `50` | Max files per archive |
-| `COMPENDIUM_SKILL_TTL_MS` | `300000` | Soft TTL (ms) on skill `resources/read` responses |
-| `COMPENDIUM_CACHE_DIR` | _(unset)_ | Persist session cache (chunks/cache keys) across restarts; default size cap 64 MiB. Multiple MCP processes may share one dir — **no** cross-process lock; TTL/eviction are best-effort. Prefer a dedicated dir per user/host. |
-| `COMPENDIUM_CACHE_MAX_BYTES` | _(unset / 64MiB with dir)_ | Soft cap on total cached payload bytes |
-| `RUST_LOG` | `compendium=info` | Logs on **stderr** only |
-
-## Example tool calls
-
-All calls use the single tool **`compendium`** with an `action` field.
-
-**Filter noisy terminal output**
-
-```json
-{
-  "action": "filter",
-  "text": "\u001b[31mERROR\u001b[0m boom\n\n\nINFO ok",
-  "filter": {
-    "strip_ansi": true,
-    "keep_patterns": ["ERROR|WARN"]
-  }
-}
-```
-
-**Compress a large log**
-
-```json
-{
-  "action": "compress",
-  "text": "...",
-  "compress": {
-    "content_type": "log",
-    "max_tokens": 512
-  }
-}
-```
+| `COMPENDIUM_CHARS_PER_TOKEN` | `4.0` | Heuristic token estimate |
+| `COMPENDIUM_TOKENIZER` | `cl100k_base` | BPE encoding when built with `--features real-tokens` |
+| `COMPENDIUM_BINARY` | _(unset)_ | npm launcher: use this binary instead of downloading |
+| `RUST_LOG` | `compendium=info` | Logs go to stderr only |
 
-**Chunk a document into references**
-
-```json
-{
-  "action": "chunk",
-  "text": "... huge file ...",
-  "chunk": {
-    "source": "file:///path/to/doc.md",
-    "chunk_tokens": 400,
-    "overlap_tokens": 40
-  }
-}
-```
-
-Prefer the returned `index_text` in the model context; pull individual chunk contents by id only when needed.
-
-**Query-aware filter (local SLM or heuristic fallback)**
-
-```json
-{
-  "action": "filter_relevant",
-  "text": "... noisy cargo/test log ...",
-  "query": "why did the auth tests fail",
-  "smart": { "max_tokens": 512, "fallback": true }
-}
-```
-
-Without `COMPENDIUM_LOCAL_LLM_URL`, `summarize_smart` / `filter_relevant` automatically use heuristics and set `backend: "heuristic"` plus `fallback_reason` in the result.
-
-**Pack a workspace briefing for a fresh agent turn**
-
-```json
-{
-  "action": "brief",
-  "query": "fix the OAuth refresh token path",
-  "brief": {
-    "root": "/path/to/repo",
-    "max_files": 40,
-    "top_k_chunks": 12,
-    "max_brief_tokens": 2048
-  }
-}
-```
-
-Start the new turn with the returned `briefing` (or `cache_get` the `cache_key`). The host should not paste the whole tree into the prompt first. Treat Status as a starter synthesis — verify Caveats and Read next before large edits.
-
-## Local small language model
-
-Follow [Quick start §2](#2-optional-smarter-summaries-with-ollama) for Ollama.
-
-Rules of thumb:
-
-- **Only loopback** URLs (`127.0.0.1`, `::1`, `localhost`) — no cloud endpoints.
-- Without `COMPENDIUM_LOCAL_LLM_URL`, smart actions use heuristics and set `backend: "heuristic"`.
-- Calls use `temperature=0` and `seed=0` for stable outputs.
-- Lemonade example: `COMPENDIUM_LOCAL_LLM_URL=http://127.0.0.1:13305/api/v1` and `COMPENDIUM_LOCAL_LLM_MODEL=Qwen3-4B-GGUF`.
-- llama.cpp OpenAI server: same pattern — set URL to its `/v1` base and the served model id.
-
-## Develop / test
+## Build from source
 
 ```bash
+cargo build --release                        # heuristic token counts
+cargo build --release --features real-tokens # exact BPE counts via tiktoken
 cargo test
-cargo test --features real-tokens
-cargo test --features http --test http_smoke
-cargo test --test e2e_smoke
-cargo run --features http -- http 127.0.0.1:8788
 ```
 
-`e2e_smoke` spawns `CARGO_BIN_EXE_compendium`, completes MCP connect (legacy initialize) over stdio, lists tools, then calls gateway actions. `http_smoke` (requires `--features http`) exercises sessionless streamable HTTP in-process.
+Point your MCP config at `target/release/compendium` directly, or keep `npx` and set `COMPENDIUM_BINARY`.
 
-## Design notes
+## Design
+
+- **Deterministic by default.** Heuristic paths need no network and produce byte-identical output, which keeps prompt prefix caches warm.
+- **Local-first.** Smart actions only ever call a loopback URL; there is no cloud path.
+- **stdout is JSON-RPC.** All logging goes to stderr.
+
+## Community
 
-- **Deterministic by default** — heuristic pipeline needs no network; smart actions only call a configured **local** OpenAI-compatible URL and fall back to heuristics when unset or failing.
-- **Token backends** — fast heuristic by default; opt into exact BPE with `real-tokens`.
-- **Zero stdout pollution** (stdio mode) — tracing goes to stderr so JSON-RPC framing stays clean.
-- **Release profile** — LTO + stripped binary for low footprint.
+[Contributing](CONTRIBUTING.md) · [Changelog](CHANGELOG.md) · [Architecture](docs/architecture.md) · [Security](SECURITY.md) · [Support](SUPPORT.md) · [Code of Conduct](CODE_OF_CONDUCT.md)
 
 ## License
 

package/bin/run.js

@@ -2,25 +2,17 @@
 'use strict';
 
 /**
- * Compendium MCP dispatcher.
+ * Compendium MCP launcher.
  *
- * Resolves the native binary and exec's it with inherited stdio so Cursor /
- * Claude Desktop / `npx compendium-mcp` talk JSON-RPC over stdin/stdout.
- *
- * Resolution order:
- *   1. COMPENDIUM_BINARY env
- *   2. Dev build: <repo>/target/release|debug/compendium(.exe)
- *   3. optionalDependency platform package (compendium-mcp-<platform>)
- *   4. Cached GitHub Release download (~/.cache/compendium-mcp/<version>/)
+ * Finds (or downloads once) the native `compendium` binary for this platform
+ * and runs it with inherited stdio, so MCP clients talk JSON-RPC over stdin/stdout.
  */
 
 const { spawn } = require('child_process');
 const fs = require('fs');
 const https = require('https');
-const http = require('http');
 const path = require('path');
 const { pipeline } = require('stream/promises');
-const { createWriteStream } = require('fs');
 
 const {
   packageVersion,
@@ -31,46 +23,28 @@ const {
   platformKey,
 } = require('../npm/lib/platform');
 
-function existsExecutable(file) {
+function isExecutable(file) {
   try {
     fs.accessSync(file, fs.constants.F_OK);
-    // On Windows, X_OK is unreliable; presence is enough.
-    if (process.platform !== 'win32') {
-      fs.accessSync(file, fs.constants.X_OK);
-    }
+    if (process.platform !== 'win32') fs.accessSync(file, fs.constants.X_OK);
     return true;
   } catch {
     return false;
   }
 }
 
-function tryRequirePlatformBinary() {
-  const spec = currentPlatform();
-  try {
-    // optionalDependency installs next to this package when available.
-    const pkgRoot = path.dirname(require.resolve(`${spec.pkg}/package.json`));
-    const candidate = path.join(pkgRoot, 'bin', binaryName(spec));
-    if (existsExecutable(candidate)) return candidate;
-  } catch {
-    // optional dep missing — fall through
-  }
-  return null;
-}
-
-function tryDevBinary() {
+function devBinary() {
   const repoRoot = path.resolve(__dirname, '..');
-  const name = process.platform === 'win32' ? 'compendium.exe' : 'compendium';
-  const release = path.join(repoRoot, 'target', 'release', name);
-  const debug = path.join(repoRoot, 'target', 'debug', name);
-  if (existsExecutable(release)) return release;
-  if (existsExecutable(debug)) return debug;
+  for (const profile of ['release', 'debug']) {
+    const candidate = path.join(repoRoot, 'target', profile, binaryName());
+    if (isExecutable(candidate)) return candidate;
+  }
   return null;
 }
 
-function download(url, dest) {
+function download(url, dest, hops = 0) {
   return new Promise((resolve, reject) => {
-    const client = url.startsWith('https:') ? https : http;
-    const req = client.get(
+    const req = https.get(
       url,
       {
         headers: {
@@ -79,14 +53,10 @@ function download(url, dest) {
         },
       },
       (res) => {
-        // Follow one redirect hop (GitHub release assets).
-        if (
-          res.statusCode >= 300 &&
-          res.statusCode < 400 &&
-          res.headers.location
-        ) {
+        if (res.statusCode >= 300 && res.statusCode < 400 && res.headers.location) {
           res.resume();
-          download(res.headers.location, dest).then(resolve, reject);
+          if (hops >= 5) return reject(new Error('too many redirects'));
+          download(res.headers.location, dest, hops + 1).then(resolve, reject);
           return;
         }
         if (res.statusCode !== 200) {
@@ -94,19 +64,16 @@ function download(url, dest) {
           reject(
             new Error(
               `Download failed (${res.statusCode}) for ${url}. ` +
-                `Ensure release assets exist for ${platformKey()}.`
+                `No release asset for ${platformKey()}? Build from source and set COMPENDIUM_BINARY.`
             )
           );
           return;
         }
         const tmp = `${dest}.partial`;
-        const out = createWriteStream(tmp);
-        pipeline(res, out)
+        pipeline(res, fs.createWriteStream(tmp))
           .then(() => {
             fs.renameSync(tmp, dest);
-            if (process.platform !== 'win32') {
-              fs.chmodSync(dest, 0o755);
-            }
+            if (process.platform !== 'win32') fs.chmodSync(dest, 0o755);
             resolve(dest);
           })
           .catch((err) => {
@@ -123,44 +90,29 @@ function download(url, dest) {
   });
 }
 
-async function ensureDownloadedBinary() {
+async function downloadedBinary() {
   const spec = currentPlatform();
   const version = packageVersion();
   const dir = cacheDir(version);
   fs.mkdirSync(dir, { recursive: true });
-  const dest = path.join(dir, binaryName(spec));
-  if (existsExecutable(dest)) return dest;
-
-  const repo = githubRepo();
-  const tag = `v${version}`;
-  const url = `https://github.com/${repo}/releases/download/${tag}/${spec.asset}`;
+  const dest = path.join(dir, binaryName());
+  if (isExecutable(dest)) return dest;
 
-  process.stderr.write(
-    `[compendium-mcp] Downloading ${spec.asset} (${tag})…\n`
-  );
+  const url = `https://github.com/${githubRepo()}/releases/download/v${version}/${spec.asset}`;
+  process.stderr.write(`[compendium-mcp] downloading ${spec.asset} v${version}…\n`);
   await download(url, dest);
-  process.stderr.write(`[compendium-mcp] Cached at ${dest}\n`);
   return dest;
 }
 
 async function resolveBinary() {
   if (process.env.COMPENDIUM_BINARY) {
     const forced = path.resolve(process.env.COMPENDIUM_BINARY);
-    if (!existsExecutable(forced)) {
+    if (!isExecutable(forced)) {
       throw new Error(`COMPENDIUM_BINARY not executable: ${forced}`);
     }
     return forced;
   }
-
-  // Prefer a local Cargo build when developing from this repo so `cargo build
-  // --release` is picked up immediately (optional platform packages can lag).
-  const fromDev = tryDevBinary();
-  if (fromDev) return fromDev;
-
-  const fromOptional = tryRequirePlatformBinary();
-  if (fromOptional) return fromOptional;
-
-  return ensureDownloadedBinary();
+  return devBinary() || downloadedBinary();
 }
 
 async function main() {
@@ -172,28 +124,24 @@ async function main() {
     process.exit(1);
   }
 
-  process.stderr.write(`[compendium-mcp] using ${binary}\n`);
-
   const child = spawn(binary, process.argv.slice(2), {
     stdio: 'inherit',
     windowsHide: true,
   });
 
   child.on('error', (err) => {
-    process.stderr.write(`[compendium-mcp] failed to spawn ${binary}: ${err.message}\n`);
+    process.stderr.write(`[compendium-mcp] failed to start ${binary}: ${err.message}\n`);
     process.exit(1);
   });
 
-  const forward = (signal) => {
-    if (!child.killed) child.kill(signal);
-  };
-  process.on('SIGINT', () => forward('SIGINT'));
-  process.on('SIGTERM', () => forward('SIGTERM'));
+  for (const signal of ['SIGINT', 'SIGTERM']) {
+    process.on(signal, () => {
+      if (!child.killed) child.kill(signal);
+    });
+  }
 
   child.on('exit', (code, signal) => {
-    if (signal) {
-      process.exit(signal === 'SIGINT' ? 130 : 1);
-    }
+    if (signal) process.exit(signal === 'SIGINT' ? 130 : 1);
     process.exit(code ?? 1);
   });
 }

package/npm/lib/platform.js

@@ -1,157 +1,59 @@
 'use strict';
 
 /**
- * Shared platform → package / release-asset mapping for Compendium npm distribution.
+ * Platform → GitHub Release asset mapping for the `compendium-mcp` npm wrapper.
  *
- * Distribution strategy (hybrid):
- * 1. Prefer optionalDependency platform packages (esbuild-style) — offline, fast.
- * 2. Fall back to downloading the matching GitHub Release asset into a local cache.
- * 3. Dev override: COMPENDIUM_BINARY or ./target/release/compendium.
- * 4. Force key: COMPENDIUM_PLATFORM (e.g. linux-x64-musl).
+ * Binary resolution (see bin/run.js):
+ *   1. COMPENDIUM_BINARY env override
+ *   2. Dev build: <repo>/target/release|debug/compendium
+ *   3. Cached download from GitHub Releases (~/.cache/compendium-mcp/<version>/)
  */
 
-const fs = require('fs');
 const os = require('os');
 const path = require('path');
-const { execSync } = require('child_process');
 
-/** @typedef {{ pkg: string, asset: string, rustTarget: string }} PlatformSpec */
-
-/** @type {Record<string, PlatformSpec>} */
+/** @type {Record<string, { asset: string }>} */
 const PLATFORMS = {
-  'darwin-arm64': {
-    pkg: 'compendium-mcp-darwin-arm64',
-    asset: 'compendium-darwin-arm64',
-    rustTarget: 'aarch64-apple-darwin',
-  },
-  'darwin-x64': {
-    pkg: 'compendium-mcp-darwin-x64',
-    asset: 'compendium-darwin-x64',
-    rustTarget: 'x86_64-apple-darwin',
-  },
-  'linux-x64': {
-    pkg: 'compendium-mcp-linux-x64',
-    asset: 'compendium-linux-x64',
-    rustTarget: 'x86_64-unknown-linux-gnu',
-  },
-  'linux-x64-musl': {
-    pkg: 'compendium-mcp-linux-x64-musl',
-    asset: 'compendium-linux-x64-musl',
-    rustTarget: 'x86_64-unknown-linux-musl',
-  },
-  'linux-arm64': {
-    pkg: 'compendium-mcp-linux-arm64',
-    asset: 'compendium-linux-arm64',
-    rustTarget: 'aarch64-unknown-linux-gnu',
-  },
-  'win32-x64': {
-    pkg: 'compendium-mcp-win32-x64',
-    asset: 'compendium-win32-x64.exe',
-    rustTarget: 'x86_64-pc-windows-msvc',
-  },
-  'win32-arm64': {
-    pkg: 'compendium-mcp-win32-arm64',
-    asset: 'compendium-win32-arm64.exe',
-    rustTarget: 'aarch64-pc-windows-msvc',
-  },
+  'darwin-arm64': { asset: 'compendium-darwin-arm64' },
+  'darwin-x64': { asset: 'compendium-darwin-x64' },
+  'linux-x64': { asset: 'compendium-linux-x64' },
+  'linux-arm64': { asset: 'compendium-linux-arm64' },
+  'win32-x64': { asset: 'compendium-win32-x64.exe' },
 };
 
 function packageVersion() {
-  // Keep in sync with root package.json / Cargo.toml at publish time.
   return require('../../package.json').version;
 }
 
 function githubRepo() {
-  return (
-    process.env.COMPENDIUM_GITHUB_REPO ||
-    process.env.GITHUB_REPOSITORY ||
-    'hocestnonsatis/Compendium'
-  );
+  return process.env.COMPENDIUM_GITHUB_REPO || 'hocestnonsatis/Compendium';
 }
 
-/** Best-effort musl detection (Alpine / static libc). Default is glibc. */
-function isLinuxMusl() {
-  if (process.platform !== 'linux') return false;
-  try {
-    if (typeof process.report?.getReport === 'function') {
-      const report = process.report.getReport();
-      if (report?.header?.glibcVersionRuntime) return false;
-    }
-  } catch (_) {
-    /* ignore */
-  }
-  try {
-    if (fs.existsSync('/etc/alpine-release')) return true;
-  } catch (_) {
-    /* ignore */
-  }
-  try {
-    const out = execSync('ldd --version 2>&1 || true', {
-      encoding: 'utf8',
-      stdio: ['ignore', 'pipe', 'pipe'],
-    });
-    if (/musl/i.test(out)) return true;
-  } catch (_) {
-    /* ignore */
-  }
-  return false;
-}
-
-/**
- * Normalize Node's process.platform + process.arch into our key.
- * Override with COMPENDIUM_PLATFORM when needed (e.g. linux-x64-musl).
- * @returns {string}
- */
 function platformKey() {
   const forced = (process.env.COMPENDIUM_PLATFORM || '').trim();
   if (forced) return forced;
-
-  const platform = process.platform;
-  let arch = process.arch;
-  // Rosetta / rare aliases
-  if (arch === 'ia32') arch = 'x64';
-
-  if (platform === 'linux' && arch === 'x64' && isLinuxMusl()) {
-    return 'linux-x64-musl';
-  }
-  return `${platform}-${arch}`;
+  const arch = process.arch === 'ia32' ? 'x64' : process.arch;
+  return `${process.platform}-${arch}`;
 }
 
-/**
- * @returns {PlatformSpec}
- */
 function currentPlatform() {
   const key = platformKey();
   const spec = PLATFORMS[key];
   if (!spec) {
-    const supported = Object.keys(PLATFORMS).join(', ');
     throw new Error(
-      `Unsupported platform "${key}". Supported: ${supported}.\n` +
-        `Set COMPENDIUM_BINARY to a local build, COMPENDIUM_PLATFORM to a known key, or open an issue for this target.`
+      `Unsupported platform "${key}". Supported: ${Object.keys(PLATFORMS).join(', ')}.\n` +
+        `Build from source (cargo build --release) and set COMPENDIUM_BINARY to the binary.`
     );
   }
   return spec;
 }
 
-/**
- * Binary filename inside a platform package / cache.
- * @param {PlatformSpec} [spec]
- */
-function binaryName(spec = currentPlatform()) {
-  return process.platform === 'win32' || spec.asset.endsWith('.exe')
-    ? 'compendium.exe'
-    : 'compendium';
+function binaryName() {
+  return process.platform === 'win32' ? 'compendium.exe' : 'compendium';
 }
 
-/**
- * Cache directory for lazily downloaded binaries.
- * @param {string} version
- */
 function cacheDir(version = packageVersion()) {
-  const base =
-    process.env.COMPENDIUM_CACHE_DIR ||
-    process.env.XDG_CACHE_HOME ||
-    path.join(os.homedir(), '.cache');
+  const base = process.env.XDG_CACHE_HOME || path.join(os.homedir(), '.cache');
   return path.join(base, 'compendium-mcp', version);
 }
 
@@ -163,5 +65,4 @@ module.exports = {
   currentPlatform,
   binaryName,
   cacheDir,
-  isLinuxMusl,
 };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "compendium-mcp",
-  "version": "0.6.2",
+  "version": "0.7.0",
   "description": "MCP server that compresses, summarizes, and filters context to minimize LLM token usage",
   "license": "MIT",
   "keywords": [
@@ -34,21 +34,7 @@
     "LICENSE"
   ],
   "scripts": {
-    "check-npm-gates": "node npm/scripts/check-versions-selftest.js && node npm/scripts/check-versions.js && node npm/scripts/check-residual-npm.js",
-    "check-versions": "node npm/scripts/check-versions.js",
-    "check-versions-selftest": "node npm/scripts/check-versions-selftest.js",
-    "check-residual-npm": "node npm/scripts/check-residual-npm.js",
-    "prepack": "node npm/scripts/check-wrapper.js && node npm/scripts/check-versions.js",
-    "prepare": "node npm/scripts/link-bins.js"
-  },
-  "optionalDependencies": {
-    "compendium-mcp-darwin-arm64": "0.6.2",
-    "compendium-mcp-darwin-x64": "0.6.2",
-    "compendium-mcp-linux-x64": "0.6.2",
-    "compendium-mcp-linux-x64-musl": "0.6.2",
-    "compendium-mcp-linux-arm64": "0.6.2",
-    "compendium-mcp-win32-x64": "0.6.2",
-    "compendium-mcp-win32-arm64": "0.6.2"
-  },
-  "preferUnplugged": true
+    "check-version": "node npm/scripts/check-version.js",
+    "prepack": "node npm/scripts/check-version.js"
+  }
 }