compendium-mcp 0.1.1 → 0.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +136 -44
  2. package/package.json +10 -8
package/README.md CHANGED
@@ -1,32 +1,88 @@
1
1
  # Compendium
2
2
 
3
+ <p align="center">
4
+ <img src="assets/logo.svg" alt="Compendium" width="420" />
5
+ </p>
6
+
3
7
  MCP server that **minimizes LLM token usage** by compressing, summarizing, filtering, and chunk-referencing large context before it reaches the model.
4
8
 
5
9
  Built in Rust with the official [`rmcp`](https://crates.io/crates/rmcp) SDK.
6
10
 
7
- ## Install via npm / npx
11
+ ## Quick start (Cursor)
12
+
13
+ You need **Node.js 18+**. Compendium itself arrives via npm — no Rust install required.
14
+
15
+ ### 1. Add the MCP server
16
+
17
+ Open Cursor MCP settings (`~/.cursor/mcp.json` or the project `.cursor/mcp.json`) and add:
18
+
19
+ ```json
20
+ {
21
+ "mcpServers": {
22
+ "compendium": {
23
+ "command": "npx",
24
+ "args": ["-y", "compendium-mcp"]
25
+ }
26
+ }
27
+ }
28
+ ```
29
+
30
+ Restart MCP / reload Cursor. You should see one tool named **`compendium`**.
31
+
32
+ That alone is enough: filter, compress, summarize, cache, and BM25 actions all work **without** a local model (fast heuristics).
8
33
 
9
- Package name: **`compendium-mcp`** (the npm name `compendium` is taken). CLI bin: `compendium`.
34
+ ### 2. (Optional) Smarter summaries with Ollama
35
+
36
+ Want better `summarize_smart` / `filter_relevant`? Run a small model on your machine and point Compendium at it.
37
+
38
+ 1. Install [Ollama](https://ollama.com/) and start it (default: `http://127.0.0.1:11434`).
39
+ 2. Pull a chat model, for example:
10
40
 
11
41
  ```bash
12
- npx -y compendium-mcp --help
13
- # or: npm install -g compendium-mcp
42
+ ollama pull qwen:latest
43
+ # or a smaller one: ollama pull qwen2.5:3b
14
44
  ```
15
45
 
16
- **Cursor / Claude Desktop**
46
+ 3. Extend the MCP `env` block (URL must stay on **localhost** — Compendium blocks remote hosts on purpose):
17
47
 
18
48
  ```json
19
49
  {
20
50
  "mcpServers": {
21
51
  "compendium": {
22
52
  "command": "npx",
23
- "args": ["-y", "compendium-mcp"]
53
+ "args": ["-y", "compendium-mcp"],
54
+ "env": {
55
+ "COMPENDIUM_LOCAL_LLM_URL": "http://127.0.0.1:11434/v1",
56
+ "COMPENDIUM_LOCAL_LLM_MODEL": "qwen:latest"
57
+ }
24
58
  }
25
59
  }
26
60
  }
27
61
  ```
28
62
 
29
- Distribution uses **optional platform packages** (`compendium-mcp-darwin-arm64`, …) with a **GitHub Releases download fallback**. See [npm/DISTRIBUTION.md](npm/DISTRIBUTION.md) for local testing, CI release, and publishing.
63
+ 4. Reload MCP, then ask the agent to call `compendium` with `action: "summarize_smart"`.
64
+ In the result, `"backend": "local_llm"` means Ollama answered; `"heuristic"` means it fell back (Ollama down, wrong model name, or URL missing).
65
+
66
+ **Notes**
67
+
68
+ - Package name on npm is **`compendium-mcp`** (`compendium` was already taken). The CLI binary name is still `compendium`.
69
+ - First Ollama reply can be slow while the model loads; later calls are faster.
70
+ - Other local OpenAI-compatible servers work the same way (e.g. Lemonade `http://127.0.0.1:13305/api/v1`). See [Environment](#environment).
71
+
72
+ Smoke-check from a terminal (any folder **except** this git repo root is fine):
73
+
74
+ ```bash
75
+ npx -y compendium-mcp --help
76
+ ```
77
+
78
+ Binary packaging details for maintainers: [npm/DISTRIBUTION.md](npm/DISTRIBUTION.md).
79
+
80
+ ## Community
81
+
82
+ - [Contributing](CONTRIBUTING.md)
83
+ - [Code of Conduct](CODE_OF_CONDUCT.md)
84
+ - [Security policy](SECURITY.md)
85
+ - [Support](SUPPORT.md)
30
86
 
31
87
  ## Transports
32
88
 
@@ -59,11 +115,32 @@ Single MCP tool: **`compendium`**. Choose the operation with `action`:
59
115
  | `cache_invalidate` | Drop one key or clear cache | `key?` |
60
116
  | `sanitize` | Redact secrets + neutralize IPI phrases | `text`, `sanitize?` |
61
117
  | `rerank` | BM25-rank candidates / chunks for a query | `query`, `items` or `text` or chunk `map`, `rerank?` |
118
+ | `brief` | Scan a workspace; pack a structured starter briefing + cache key | `query`, `brief?` (`root`, caps), optional `text` hint |
119
+ | `catalog` | Short action (+ playbook) ads; prefer before guessing | _(none)_ |
120
+ | `help` | Usage notes for one action (default **compressed**; `force: true` → full) | `id`, `force?` |
121
+ | `playbooks` | List bundled token-hygiene playbook ads | _(none)_ |
122
+ | `playbook` | Load one playbook body (sanitized) | `id` |
123
+ | `pack` | Zip text/files into a size-capped archive (cache and/or base64) | `text` or `items`, `pack?` |
124
+ | `unpack` | Unpack zip with caps → chunks (**never runs scripts**) | `key` or base64 `text`, `pack?` |
125
+
126
+ ### Progressive disclosure (skills)
127
+
128
+ Tool description/instructions stay thin. Discover details on demand:
129
+
130
+ - **Tool bridge:** `action=catalog` → `action=help` with `id`, or `playbooks` → `playbook`
131
+ - **MCP resources:** `resources/list` / `resources/read` on:
132
+ - `cmp://skill/index` — JSON index of actions + playbooks
133
+ - `cmp://skill/action/{name}` — full action help (markdown)
134
+ - `cmp://skill/playbook/{id}` — playbook body
135
+
136
+ Bundled playbooks live under [`playbooks/`](playbooks/). Override/extend with `COMPENDIUM_PLAYBOOKS_DIR` (same `id` wins). Archives honor `COMPENDIUM_ARCHIVE_MAX_BYTES` / `_UNCOMPRESSED` / `_FILES` (defaults 2 MiB / 4 MiB / 50).
62
137
 
63
138
  Optional on most text actions: `sanitize_input: true` scrubs before processing. Soft payloads under `COMPENDIUM_SIGNAL_MIN_CHARS` (default 1000) bypass `compress` / `summarize` / `summarize_smart` unless `force: true`.
64
139
 
65
140
  `filter` accepts optional `query` (top-level or `filter.query`) for BM25 line keep. `prune_history` supports `prune.strategy: "afm"` (Critical / Thematic / Distant tiers; distant blob cached for `cache_get`).
66
141
 
142
+ `brief` walks `brief.root` (default: process cwd) with `.gitignore` / `.ignore`, BM25-ranks paths/chunks, window-reads oversized files (not head-truncate), and returns a structured `briefing`: **Task / Status / Evidence / Caveats / Sources / Read next**, plus `cache_key`. Status uses a local SLM when `COMPENDIUM_LOCAL_LLM_URL` is set (`backend: local_llm`); otherwise heuristic bullets. Caveats flag truncated files and docs older than selected code. **Read next** includes source paths plus suggested `cmp://skill/playbook/…` / action URIs. Optional `COMPENDIUM_BRIEF_ROOT` restricts allowed roots. Briefings are sanitized by default.
143
+
67
144
  Example:
68
145
 
69
146
  ```json
@@ -79,14 +156,13 @@ Response envelope: `{ "ok": true, "action": "filter", "result_json": "{...}" }`.
79
156
  ## Project layout
80
157
 
81
158
  ```
82
- package.json / bin/run.js # npm wrapper for npx compendium-mcp
83
- npm/ # platform packages + distribution docs
84
- .github/workflows/ # release cross-compile + npm publish
159
+ assets/ # brand mark (SVG/PNG); baked into MCP icons via data URI
85
160
  src/
86
161
  main.rs # CLI: stdio | http
87
162
  lib.rs
163
+ brand.rs # SEP-973 icons for serverInfo + tool
88
164
  config.rs # COMPENDIUM_* env config
89
- server.rs # MCP tool handlers (rmcp macros)
165
+ server.rs # MCP tool + resources handlers (rmcp)
90
166
  http.rs # Streamable HTTP/SSE (feature = "http")
91
167
  pipeline/
92
168
  tokens.rs # heuristic or tiktoken BPE (feature = "real-tokens")
@@ -97,12 +173,16 @@ src/
97
173
  local_llm.rs # OpenAI-compatible local SLM client
98
174
  chunk.rs # chunk + resolve
99
175
  cache.rs # session key/value cache
176
+ catalog.rs # action ads + help (progressive disclosure)
177
+ playbook.rs # bundled / dir playbooks
178
+ pack.rs # zip pack/unpack with size caps
100
179
  stats.rs # session savings counters
101
180
  prune.rs # conversation history pruning
102
181
  output.rs # domain-aware compress_output
182
+ playbooks/ # embedded skill-md playbooks
103
183
  tests/
104
184
  integration.rs
105
- e2e_smoke.rs # spawns binary, MCP handshake, all tools
185
+ e2e_smoke.rs # spawns binary, MCP handshake, tools + resources
106
186
  ```
107
187
 
108
188
  ## Build
@@ -123,27 +203,27 @@ cargo build --release --features real-tokens,http
123
203
 
124
204
  Binary: `target/release/compendium`
125
205
 
126
- ## Configure (Cursor / Claude Desktop)
206
+ ## Configure (advanced)
207
+
208
+ The [Quick start](#quick-start-cursor) config is enough for most people. Extra options:
209
+
210
+ ### Claude Desktop
211
+
212
+ Same `command` / `args` / `env` as Cursor, in Claude’s MCP config file.
127
213
 
128
- ### Cursor (`~/.cursor/mcp.json` or project `.cursor/mcp.json`)
214
+ ### Optional tuning env
129
215
 
130
216
  ```json
131
- {
132
- "mcpServers": {
133
- "compendium": {
134
- "command": "npx",
135
- "args": ["-y", "compendium-mcp"],
136
- "env": {
137
- "RUST_LOG": "compendium=info",
138
- "COMPENDIUM_DEFAULT_MAX_TOKENS": "2048",
139
- "COMPENDIUM_TOKENIZER": "cl100k_base"
140
- }
141
- }
142
- }
217
+ "env": {
218
+ "RUST_LOG": "compendium=info",
219
+ "COMPENDIUM_DEFAULT_MAX_TOKENS": "2048",
220
+ "COMPENDIUM_TOKENIZER": "cl100k_base",
221
+ "COMPENDIUM_LOCAL_LLM_URL": "http://127.0.0.1:11434/v1",
222
+ "COMPENDIUM_LOCAL_LLM_MODEL": "qwen:latest"
143
223
  }
144
224
  ```
145
225
 
146
- Or point at a local release binary:
226
+ ### Local Cargo binary (developers)
147
227
 
148
228
  ```json
149
229
  {
@@ -152,9 +232,7 @@ Or point at a local release binary:
152
232
  "command": "/absolute/path/to/Compendium/target/release/compendium",
153
233
  "env": {
154
234
  "RUST_LOG": "compendium=info",
155
- "COMPENDIUM_DEFAULT_MAX_TOKENS": "2048",
156
- "COMPENDIUM_CHARS_PER_TOKEN": "4.0",
157
- "COMPENDIUM_TOKENIZER": "cl100k_base"
235
+ "COMPENDIUM_DEFAULT_MAX_TOKENS": "2048"
158
236
  }
159
237
  }
160
238
  }
@@ -181,10 +259,16 @@ Point an MCP streamable-HTTP client at that URL (e.g. `StreamableHttpClientTrans
181
259
  | `COMPENDIUM_SIMILARITY_THRESHOLD` | `0.85` | Jaccard line-dedupe threshold |
182
260
  | `COMPENDIUM_HTTP_BIND` | `127.0.0.1:8788` | Default HTTP listen address |
183
261
  | `COMPENDIUM_LOCAL_LLM_URL` | _(unset)_ | OpenAI-compatible base URL (e.g. `http://127.0.0.1:11434/v1` or `http://127.0.0.1:13305/api/v1`). Enables smart actions. |
184
- | `COMPENDIUM_LOCAL_LLM_MODEL` | `Qwen3-4B-GGUF` | Model id accepted by the local server |
262
+ | `COMPENDIUM_LOCAL_LLM_MODEL` | `Qwen3-4B-GGUF` | Model id on that server (Ollama: e.g. `qwen:latest`) |
185
263
  | `COMPENDIUM_LOCAL_LLM_API_KEY` | _(unset)_ | Optional bearer token for locked loopback servers |
186
264
  | `COMPENDIUM_LOCAL_LLM_TIMEOUT_SECS` | `120` | HTTP timeout (first model load can be slow) |
187
265
  | `COMPENDIUM_SIGNAL_MIN_CHARS` | `1000` | Bypass compress/summarize below this length (`0` disables) |
266
+ | `COMPENDIUM_BRIEF_ROOT` | _(unset)_ | When set, `action=brief` may only scan roots under this canonical path |
267
+ | `COMPENDIUM_PLAYBOOKS_DIR` | _(unset)_ | Extra/override playbook `*.md` directory (same `id` replaces embedded) |
268
+ | `COMPENDIUM_ARCHIVE_MAX_BYTES` | `2097152` | Max compressed archive size for pack/unpack |
269
+ | `COMPENDIUM_ARCHIVE_MAX_UNCOMPRESSED` | `4194304` | Max total uncompressed bytes for pack/unpack |
270
+ | `COMPENDIUM_ARCHIVE_MAX_FILES` | `50` | Max files per archive |
271
+ | `COMPENDIUM_SKILL_TTL_MS` | `300000` | Soft TTL (ms) on skill `resources/read` responses |
188
272
  | `RUST_LOG` | `compendium=info` | Logs on **stderr** only |
189
273
 
190
274
  ## Example tool calls
@@ -250,26 +334,34 @@ Prefer the returned `index_text` in the model context; pull individual chunk con
250
334
 
251
335
  Without `COMPENDIUM_LOCAL_LLM_URL`, `summarize_smart` / `filter_relevant` automatically use heuristics and set `backend: "heuristic"` plus `fallback_reason` in the result.
252
336
 
253
- ## Local small language model
254
-
255
- Smart actions call a **loopback-only** OpenAI-compatible chat endpoint — never a cloud API. `COMPENDIUM_LOCAL_LLM_URL` must be `127.0.0.1`, `::1`, or `localhost` (SSRF guard). Requests use `temperature=0` and `seed=0` for stable outputs. Point Compendium at Ollama, Lemonade, or llama.cpp:
337
+ **Pack a workspace briefing for a fresh agent turn**
256
338
 
257
339
  ```json
258
340
  {
259
- "mcpServers": {
260
- "compendium": {
261
- "command": "npx",
262
- "args": ["-y", "compendium-mcp"],
263
- "env": {
264
- "COMPENDIUM_LOCAL_LLM_URL": "http://127.0.0.1:11434/v1",
265
- "COMPENDIUM_LOCAL_LLM_MODEL": "qwen2.5:3b"
266
- }
267
- }
341
+ "action": "brief",
342
+ "query": "fix the OAuth refresh token path",
343
+ "brief": {
344
+ "root": "/path/to/repo",
345
+ "max_files": 40,
346
+ "top_k_chunks": 12,
347
+ "max_brief_tokens": 2048
268
348
  }
269
349
  }
270
350
  ```
271
351
 
272
- Lemonade example: `COMPENDIUM_LOCAL_LLM_URL=http://127.0.0.1:13305/api/v1` and `COMPENDIUM_LOCAL_LLM_MODEL=Qwen3-4B-GGUF`.
352
+ Start the new turn with the returned `briefing` (or `cache_get` the `cache_key`). The host should not paste the whole tree into the prompt first. Treat Status as a starter synthesis — verify Caveats and Read next before large edits.
353
+
354
+ ## Local small language model
355
+
356
+ Follow [Quick start §2](#2-optional-smarter-summaries-with-ollama) for Ollama.
357
+
358
+ Rules of thumb:
359
+
360
+ - **Only loopback** URLs (`127.0.0.1`, `::1`, `localhost`) — no cloud endpoints.
361
+ - Without `COMPENDIUM_LOCAL_LLM_URL`, smart actions use heuristics and set `backend: "heuristic"`.
362
+ - Calls use `temperature=0` and `seed=0` for stable outputs.
363
+ - Lemonade example: `COMPENDIUM_LOCAL_LLM_URL=http://127.0.0.1:13305/api/v1` and `COMPENDIUM_LOCAL_LLM_MODEL=Qwen3-4B-GGUF`.
364
+ - llama.cpp OpenAI server: same pattern — set URL to its `/v1` base and the served model id.
273
365
 
274
366
  ## Develop / test
275
367
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "compendium-mcp",
3
- "version": "0.1.1",
3
+ "version": "0.1.3",
4
4
  "description": "MCP server that compresses, summarizes, and filters context to minimize LLM token usage",
5
5
  "license": "MIT",
6
6
  "keywords": [
@@ -24,7 +24,8 @@
24
24
  "node": ">=18"
25
25
  },
26
26
  "bin": {
27
- "compendium": "bin/run.js"
27
+ "compendium": "bin/run.js",
28
+ "compendium-mcp": "bin/run.js"
28
29
  },
29
30
  "files": [
30
31
  "bin/",
@@ -33,14 +34,15 @@
33
34
  "LICENSE"
34
35
  ],
35
36
  "scripts": {
36
- "prepack": "node npm/scripts/check-wrapper.js"
37
+ "prepack": "node npm/scripts/check-wrapper.js",
38
+ "prepare": "node npm/scripts/link-bins.js"
37
39
  },
38
40
  "optionalDependencies": {
39
- "compendium-mcp-darwin-arm64": "0.1.1",
40
- "compendium-mcp-darwin-x64": "0.1.1",
41
- "compendium-mcp-linux-x64": "0.1.1",
42
- "compendium-mcp-linux-arm64": "0.1.1",
43
- "compendium-mcp-win32-x64": "0.1.1"
41
+ "compendium-mcp-darwin-arm64": "0.1.3",
42
+ "compendium-mcp-darwin-x64": "0.1.3",
43
+ "compendium-mcp-linux-x64": "0.1.3",
44
+ "compendium-mcp-linux-arm64": "0.1.3",
45
+ "compendium-mcp-win32-x64": "0.1.3"
44
46
  },
45
47
  "preferUnplugged": true
46
48
  }