tanuki-context 0.2.0 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/README.md +252 -196
  2. package/dist/agent.js +1 -1
  3. package/dist/cli.js +2 -2
  4. package/package.json +1 -1
package/README.md CHANGED
@@ -1,36 +1,44 @@
1
1
  # tanuki-context
2
2
 
3
3
  [![npm](https://img.shields.io/npm/v/tanuki-context)](https://www.npmjs.com/package/tanuki-context)
4
- Zero dependencies. 0.98 MB tarball. Runs anywhere Node >= 18 runs. MIT.
4
+ Zero dependencies. 0.98 MB install. Runs anywhere Node >= 18 runs. MIT.
5
5
 
6
- A model pays about 1 token per 4 characters to read text. A PNG page is
7
- billed by pixel dimensions (Anthropic's 28x28-pixel patch grid), and a dense
8
- page fits 3+ characters per token. [pxpipe](https://github.com/teamchong/pxpipe)
9
- measured ~3.1 chars per image-token on real Claude Code traffic and built a
10
- transparent proxy on that gap. tanuki-context is the same imaging engine with
11
- a different contract: the model calls a tool when it wants the cut, sees
12
- exactly what got imaged, and nothing is rewritten behind its back.
6
+ **AI models charge you for every word they read. tanuki-context turns the
7
+ bulky parts of a conversation — logs, command output, long documents — into
8
+ compact images the same model reads for a fraction of the price.**
13
9
 
14
- It takes bulky context (logs, docs, command output), optionally distills and
15
- compresses it, then renders it as dense PNG pages. You get the numbers before
16
- committing: `estimate` prices both routes and says which one wins.
10
+ That sounds like a trick, but it is just how the pricing works. Text costs
11
+ about 1 token per 4 characters. An image costs a fixed amount set by its
12
+ size in pixels, no matter how much text is drawn inside it. So if you pack
13
+ 28,000 characters of text into one dense 1568x728 image, the model reads it
14
+ for 1,456 tokens instead of about 7,000. [pxpipe](https://github.com/teamchong/pxpipe)
15
+ discovered how far that gap stretches on real traffic (~3.1 characters per
16
+ image token, against ~1 for text) and built a proxy on it. tanuki-context is
17
+ the same imaging engine, packaged so the model itself decides when to use it
18
+ — plus a relay mode for when you can't change anything.
17
19
 
18
- ## Measured
20
+ This is a real page, straight out of the pipeline — 200 KB of system log,
21
+ cleaned up and drawn. A vision model reads it directly:
19
22
 
20
- A 200 KB slice of this machine's system journal: 1,370 lines from tailscaled,
21
- NetworkManager, the kernel, wpa_supplicant, and friends. Not a synthetic
22
- fixture, and not cherry-picked to repeat itself (a plain `journalctl` tail is
23
- so repetitive that distill flattens it to 5 lines, which looks fake even
24
- though it isn't).
23
+ ![a rendered page: dense 5x8-pixel text, 312 columns of system log](https://raw.githubusercontent.com/Osyna/tanuki-context/main/docs/example-page.png)
25
24
 
26
- | how it enters context | tokens | saved |
27
- | -------------------------------- | -----: | --------: |
28
- | pasted as raw text | 51,200 | 0 |
29
- | rendered as pages (level 0) | 10,752 | **-79%** |
30
- | distilled first, then rendered | 5,320 | **-90%** |
31
- | distilled + codebook + tiny font | 2,352 | **-95%** |
25
+ *Four pages like this replace 51,200 tokens of raw text with 5,320 —
26
+ a 90% cut. Zoom in: it's all still there, errors verbatim.*
32
27
 
33
- Reproduce on your own log; every row is one command:
28
+ ## What you save
29
+
30
+ Measured on that same 200 KB slice of a real system journal (1,370 lines
31
+ from tailscaled, NetworkManager, the kernel, and friends — mixed on purpose;
32
+ a plain `journalctl` tail is so repetitive the result looks fake):
33
+
34
+ | how the log enters the conversation | tokens | saved |
35
+ | ----------------------------------- | -----: | --------: |
36
+ | pasted as raw text | 51,200 | 0 |
37
+ | drawn as image pages | 10,752 | **-79%** |
38
+ | noise removed first, then drawn | 5,320 | **-90%** |
39
+ | plus codebook and tiny font | 2,352 | **-95%** |
40
+
41
+ Every row is one command on your own file:
34
42
 
35
43
  ```
36
44
  npx tanuki-context estimate your.log 0
@@ -38,76 +46,84 @@ npx tanuki-context estimate your.log 0 --distill
38
46
  npx tanuki-context estimate your.log 0 --distill --codebook --font tiny
39
47
  ```
40
48
 
41
- Savings depend on the content. Repetitive logs crush well. Dense unique text
42
- still gets the flat ~3-4x from imaging alone. Small inputs technically win
43
- too, but by so little it is rarely worth the modality switch; see
44
- [Limits](#limits-plainly).
49
+ `estimate` is instant, never renders anything, and tells you honestly when
50
+ plain text would be cheaper.
45
51
 
46
- ## Quick start
52
+ ## Try it in 30 seconds
53
+
54
+ Claude Code:
47
55
 
48
56
  ```
49
- npx tanuki-context
57
+ claude mcp add tanuki-context -- npx -y tanuki-context
50
58
  ```
51
59
 
52
- That is the MCP stdio server. Register it in any client:
60
+ Any other MCP client:
53
61
 
54
62
  ```json
55
63
  { "mcpServers": { "tanuki-context": { "command": "npx", "args": ["-y", "tanuki-context"] } } }
56
64
  ```
57
65
 
58
- Or poke at the CLI first. Real output, same 200 KB journal slice:
66
+ No AI client at all, just curious what it would save you:
59
67
 
60
68
  ```
61
- npx tanuki-context estimate journal.log 0
62
- # { "imageTokens": 10752, "verdict": "PIPELINE cheaper",
63
- # "recommend": { "distill": true, "codebook": true, "imageTokens": 3920,
64
- # "pages": 3, "tinyImageTokens": 2352 }, ... }
65
- npx tanuki-context render journal.log 2 ./pages --codebook
66
- # ./pages/page0.png ...
69
+ npx tanuki-context estimate some-big-file.log 0
67
70
  ```
68
71
 
69
- ## Clients
72
+ ## The two modes
70
73
 
71
- **OMP (oh-my-pi)** - `~/.omp/agent/mcp.json`, or `.omp/mcp.json` in the project:
74
+ **Explicit (MCP tools) — the default and the recommendation.** Your AI gets
75
+ five tools. It calls `tanuki_estimate` on bulky text, reads the verdict, and
76
+ renders only when the pipeline wins. The model stays in charge and can see
77
+ exactly what happened to every byte. Since 0.2.0 the estimate answer includes
78
+ a `recommend` field — the cheapest safe settings, already priced — so one
79
+ call replaces a whole round of trial and error:
72
80
 
73
- ```json
74
- { "mcpServers": { "tanuki-context": { "command": "npx", "args": ["-y", "tanuki-context"] } } }
75
81
  ```
76
-
77
- **jcode** - `~/.jcode/mcp.json`, or `.jcode/mcp.json` in the project. jcode
78
- speaks stdio only; `"shared": true` reuses one server across sessions:
79
-
80
- ```json
81
- { "mcpServers": { "tanuki-context": { "command": "npx", "args": ["-y", "tanuki-context"], "shared": true } } }
82
+ npx tanuki-context estimate journal.log 0
83
+ # { "imageTokens": 10752, "verdict": "PIPELINE cheaper",
84
+ # "recommend": { "distill": true, "codebook": true, "imageTokens": 3920,
85
+ # "pages": 3, "tinyImageTokens": 2352 }, ... }
82
86
  ```
83
87
 
84
- **pi** - pi has no MCP layer, so this package doubles as a pi extension (the
85
- `"pi"` manifest field in package.json):
88
+ **Implicit (proxy) — for clients you can't modify.** A small local relay.
89
+ Point your client's Anthropic URL at it and oversized blocks get swapped for
90
+ image pages on the way out, under strict rules (listed below — the short
91
+ version is: it never touches your prompt, your tools, or your latest
92
+ message, and it never rewrites anything unless the numbers clearly win):
86
93
 
87
94
  ```
88
- pi install npm:tanuki-context
95
+ npx tanuki-context proxy # listens on 127.0.0.1:8484
96
+ export ANTHROPIC_BASE_URL=http://127.0.0.1:8484
89
97
  ```
90
98
 
91
- The five `tanuki_*` tools register natively; the extension spawns this
92
- package's own stdio server per session.
99
+ Both modes run the same engine and log their savings to the same file, which
100
+ `tanuki_stats` summarizes.
93
101
 
94
- **Claude Code**:
102
+ ## What people actually use it for
95
103
 
96
- ```
97
- claude mcp add tanuki-context -- npx -y tanuki-context
98
- ```
104
+ - **Pasting a build log into a chat.** 200 KB of CI output becomes a few
105
+ pages; the error lines survive verbatim, the 4,000 repeats don't.
106
+ - **Agents that read big files.** An agent reviewing code calls `estimate`
107
+ first, renders when it wins, and keeps thousands of tokens of headroom.
108
+ - **Long sessions near the context limit.** Old bulky turns can be imaged by
109
+ the proxy while recent ones stay text.
110
+ - **Fleets of agents.** One in-process server shared by a whole team via the
111
+ Agent SDK glue (below), with a canned instruction block that teaches every
112
+ agent the estimate-first habit.
99
113
 
100
- **Rust instead of Node?** Same pipeline, same numbers, one 5.7 MB static
101
- binary:
114
+ ## The five tools
102
115
 
103
- ```
104
- cargo install --git https://github.com/Osyna/tanuki-context --branch rust
105
- ```
116
+ | tool | what it does | when to reach for it |
117
+ | ----------------- | ----------------------------------------------------------- | ---------------------------------------- |
118
+ | `tanuki_estimate` | prices both routes, returns `recommend` | always first; it's free and instant |
119
+ | `tanuki_render` | the full pipeline, returns PNG pages + a breakdown | when estimate says the pipeline wins |
120
+ | `tanuki_distill` | de-noises a log, output stays text | when you want greppable text, not images |
121
+ | `tanuki_compress` | text-only compression, five levels | prose you'll paraphrase anyway |
122
+ | `tanuki_stats` | totals from the session's savings log | end-of-session accounting |
106
123
 
107
- Then `"command": "tanuki-context", "args": []` in any snippet above, or
108
- `TANUKI_BIN=~/.cargo/bin/tanuki-context` for the pi extension. The two
109
- engines are parity-tested against each other on every change; see
110
- [Two engines](#two-engines) below.
124
+ ---
125
+
126
+ Everything below is for readers who want the machinery.
111
127
 
112
128
  ## How it works
113
129
 
@@ -121,132 +137,182 @@ flowchart LR
121
137
  E --> F["PNG pages<br/>tokens = 28-px patches"]
122
138
  ```
123
139
 
124
- Three stages. The first two are optional.
125
-
126
- **Stage 0, distill.** Built for logs. Repeated lines and repeated multi-line
127
- blocks collapse to one exemplar plus an exact `xN` count. Near-duplicates
128
- that differ only in timestamps, ids, or numbers fold into a template, also
129
- counted. Error, warning, and fail lines are never touched. A `query` regex
130
- returns only the slice you care about, with context. On the journal slice
131
- above: 1,371 lines to 621, half the characters gone, all 318 error/warn
132
- lines kept verbatim.
133
-
134
- **Stage 1, the ladder.** Five text-compression levels, 0 (off) to 4 (gist
135
- only). From level 2 up an exact-recall guard keeps code, identifiers, hashes,
136
- paths, and indented lines verbatim, so loss is confined to prose. Honest
137
- consequence: on technical files there is little prose to cut. This repo's
138
- README loses 10% of its characters at level 2 and 14% at level 4; DESIGN.md
139
- loses 6% at level 4. The ladder earns its keep on wordy prose (meeting notes,
140
- chat transcripts, tickets); on code-heavy content it mostly just guards.
141
- Level 1 (whitespace cleanup) is lossless and always safe.
142
-
143
- **Stage 2, imaging.** The renderer packs text into 312-column pages of 5x8
144
- antialiased cells (1568x728 px max; short pages get width-trimmed) and
145
- encodes grayscale PNGs. This is where the big cut comes from, and it is flat:
146
- raw text pays ~1 token per 4 chars, a full page carries ~28,000 chars for
147
- 1,456 image tokens. Full Unicode, 92,812 codepoints: CJK, Cyrillic, emoji,
148
- astral planes. Unassigned codepoints render as readable `[U+HEX]` escapes.
149
- Nothing disappears silently.
150
-
151
- ## Density knobs
152
-
153
- Measured by `reference/methods-report.mjs` on three corpora; regenerate the
154
- table with `bun reference/methods-report.mjs`. Percentages are against the
155
- baseline renderer on the same content.
156
-
157
- | knob | what it does | code | prose | log |
158
- | ------------------- | ----------------------------------------------------------------------------- | ---: | ----: | ---: |
159
- | `pack` (default on) | single-cell tabs, `⇥N` indent runs, width-trimmed pages; byte-exact | -5% | 0% | 0% |
160
- | `codebook` | repeated tokens and path prefixes become 1-cell sigils plus a `·legend·` line | -9% | 0% | -38% |
161
- | `font: "tiny"` | glyphs box-filtered into 4x6 cells; opt-in, see limits below | -36% | -38% | -40% |
162
- | all three stacked | | **-45%** | **-38%** | **-62%** |
163
-
164
- The three corpora: `src/main.ts` (code), `DESIGN.md` (prose), a path-heavy
165
- synthetic log. Knobs compose but do not add up linearly; `codebook` needs
166
- repetition to bite, `pack` needs indentation.
167
-
168
- ## Tools (MCP, stdio)
169
-
170
- | tool | arguments | returns |
171
- | ----------------- | --------------------------------------------------------- | ------------------------------------------------ |
172
- | `tanuki_render` | `text, level?, distill?, query?, pack?, font?, codebook?` | PNG page blocks + savings breakdown |
173
- | `tanuki_estimate` | same as render | page geometry, token math, and a `recommend` field: the cheapest safe knob set, priced |
174
- | `tanuki_distill` | `text, query?` | stage 0 alone; output stays greppable text |
175
- | `tanuki_compress` | `text, level` | stage 1 alone |
176
- | `tanuki_stats` | none | savings summary from the session event log |
177
-
178
- `estimate` walks the safe knob ladder server-side (plain, codebook, distill,
179
- both, at level 0) and returns the first rung that holds as `recommend`, with
180
- the tiny-font price alongside. Probing those combos by hand costs about 590
181
- tokens of tool chatter; the walk is one call.
182
-
183
- `estimate` never touches pixel data. Call it on everything; render only when
184
- the verdict says the pipeline wins.
140
+ **Stage 0 — distill.** Built for logs. Repeated lines and repeated
141
+ multi-line blocks collapse to one example plus an exact count.
142
+ Near-duplicates that differ only in timestamps, ids, or numbers fold into a
143
+ template, also counted. Errors and warnings are never touched. Here is the
144
+ whole idea in eleven lines — real output:
145
+
146
+ ```
147
+ input (11 lines): distilled (deterministic):
148
+
149
+ 12:00:01 INFO sync started 12:00:01 INFO sync started
150
+ 12:00:02 INFO uploaded photo_001.jpg ok 12:00:02 INFO uploaded photo_001.jpg ok
151
+ 12:00:02 INFO uploaded photo_002.jpg ok 12:00:02 INFO uploaded photo_002.jpg ok
152
+ 12:00:03 INFO uploaded photo_003.jpg ok 12:00:03 INFO uploaded photo_003.jpg ok
153
+ 12:00:03 INFO uploaded photo_004.jpg ok 12:00:04 ERROR upload failed photo_006.jpg: connection reset
154
+ 12:00:04 INFO uploaded photo_005.jpg ok 12:00:06 INFO sync finished: 8 ok, 1 failed
155
+ 12:00:04 ERROR upload failed photo_006.jpg: connection reset
156
+ 12:00:05 INFO uploaded photo_007.jpg ok ── 5 repeated lines suppressed (0 exact ×N,
157
+ 12:00:05 INFO uploaded photo_008.jpg ok 5 same-template; first occurrences kept above) ──
158
+ 12:00:06 INFO uploaded photo_009.jpg ok ×8 (template) 12:00:02 INFO uploaded photo_001.jpg ok
159
+ 12:00:06 INFO sync finished: 8 ok, 1 failed
160
+ ```
161
+
162
+ The ERROR line survived untouched, the count is exact, and the same input
163
+ always produces the same output. On the 200 KB journal: 1,371 lines to 621,
164
+ half the characters gone, all 318 error/warn lines kept verbatim.
165
+
166
+ **Stage 1 — the ladder.** Five text-compression levels, 0 (off) to 4 (gist
167
+ only). From level 2 up, an exact-recall guard keeps code, identifiers,
168
+ hashes, paths, and indented lines verbatim, so loss is confined to prose.
169
+ Honest consequence: technical files barely shrink, because the guard
170
+ protects most of them. This repo's README loses 10% of its characters at
171
+ level 2 and 14% at level 4. The ladder earns its keep on wordy prose;
172
+ on code it mostly just guards. Level 1 (whitespace cleanup) is lossless.
173
+
174
+ **Stage 2 — imaging.** Text is packed into 312-column pages of 5x8
175
+ antialiased glyphs (1568x728 px max; short pages are width-trimmed) and
176
+ encoded as grayscale PNGs. Anthropic bills images by 28x28-pixel patches:
177
+ a full page is 56x26 = 1,456 tokens carrying up to 28,080 characters —
178
+ about 19 characters per token, where text gets 4. Full Unicode: 92,812
179
+ codepoints, CJK to emoji. Unassigned codepoints become readable `[U+HEX]`
180
+ escapes. Nothing disappears silently.
181
+
182
+ ## The techniques, named
183
+
184
+ | technique | what it does | why it saves |
185
+ | --- | --- | --- |
186
+ | pixel pricing | pages billed by 28-px patch grid, not content | the core ~4x gap text never gets |
187
+ | distill | dedupe with exact counts, errors verbatim | noisy logs are mostly repetition |
188
+ | exact-recall guard | levels 2-4 can't touch code/ids/paths | loss stays confined to prose |
189
+ | pack | single-cell tabs, `⇥N` indent runs, width-trimmed pages | -5% on code, byte-exact |
190
+ | codebook | repeated tokens/paths become 1-cell sigils + a `·legend·` line | -38% on path-heavy logs, reversible |
191
+ | tiny font | glyphs box-filtered to 4x6 cells | -36 to -40% more, opt-in (lossy read-back) |
192
+ | `recommend` | the estimate call walks all safe knob combos server-side | saves ~590 tokens of tool-call probing |
193
+ | proxy dedupe | byte-identical repeats become a one-line pointer | ~1,400 tokens per repeated 30 KB block |
194
+ | append-stable pages | appending text never changes earlier pages | prompt caching keeps pricing them at cache rates |
195
+
196
+ ## Benchmarks
197
+
198
+ Density knobs, measured by `reference/methods-report.mjs` (regenerate it
199
+ yourself with `bun reference/methods-report.mjs`; percentages vs the
200
+ baseline renderer on the same content):
201
+
202
+ | knob | code | prose | log |
203
+ | ------------------- | ---: | ----: | ---: |
204
+ | `pack` (default on) | -5% | 0% | 0% |
205
+ | `codebook` | -9% | 0% | -38% |
206
+ | `font: "tiny"` | -36% | -38% | -40% |
207
+ | all three stacked | **-45%** | **-38%** | **-62%** |
208
+
209
+ Corpora: `src/main.ts` (code), `DESIGN.md` (prose), a path-heavy synthetic
210
+ log. Knobs need something to bite on: codebook needs repetition, pack needs
211
+ indentation.
212
+
213
+ Footprint, measured 2026-07-26 on a Ryzen 7 9700X (median of 5 cold starts;
214
+ the reference server is the original node MCP wrapping pxpipe's library):
215
+
216
+ | metric | node | bun | rust | node ref |
217
+ | ----------------------- | ------- | ------- | ------- | -------- |
218
+ | spawn to first response | 35 ms | 27 ms | 0.4 ms | 158 ms |
219
+ | idle server RSS | 87 MB | 50 MB | 3.8 MB | 177 MB |
220
+ | distill a 12 MB log | 0.42 s | 0.31 s | 0.28 s | - |
221
+ | install | 0.98 MB tarball, zero deps | same | 5.7 MB static binary | node_modules tree |
222
+
223
+ Correctness: 39 TypeScript tests, 23 Rust tests, and a 92-check parity
224
+ harness that holds the two engines to byte-identical JSON and
225
+ pixel-identical PNGs on every knob combination, including distilled
226
+ renders and a full MCP session with error paths.
185
227
 
186
228
  ## Limits, plainly
187
229
 
188
230
  - **Rendering is exact; reading is a model skill.** Pages are pixel-faithful
189
- to pxpipe's production renderer, and pxpipe has receipts on how well models
190
- read them: near-perfect on arithmetic, gist, and state, but 13/15 on exact
191
- 12-char hex strings for claude-fable-5, and misses are silent. Their
192
- numbers: [pxpipe benchmarks](https://github.com/teamchong/pxpipe#benchmark-results-and-receipts).
231
+ to pxpipe's production renderer, and pxpipe has receipts on how well
232
+ models read them: near-perfect on arithmetic, gist, and state, but 13/15
233
+ on exact 12-char hex strings for claude-fable-5 — and misses are silent.
234
+ Their numbers: [pxpipe benchmarks](https://github.com/teamchong/pxpipe#benchmark-results-and-receipts).
193
235
  Keep byte-exact-critical values (secrets, hashes you must retype) in text.
194
236
  - **`font: "tiny"` trades legibility for tokens.** We measured 99.7%
195
- character read-back, and `M`/`H` is the confusable pair. Fine for logs and
196
- prose; skip it when the model must transcribe identifiers exactly.
197
- - **Levels 2-4 reword prose.** The guard keeps code and identifiers verbatim,
198
- but prose comes back paraphrase-shaped. Level 4 is gist only. Do not use it
199
- for anything you may need to quote.
237
+ character read-back; `M`/`H` is the confusable pair. Fine for logs, skip
238
+ it when the model must transcribe identifiers exactly.
239
+ - **Levels 2-4 reword prose.** Level 4 is gist only. Never use it for
240
+ anything you may need to quote.
200
241
  - **Tiny wins are not worth taking.** Width-trimming makes even small pages
201
242
  cheap on paper (12 short lines: 56 image tokens vs 115 as text), but a
202
- 50-token win buys you an image the model has to read back instead of text
203
- it can quote. The proxy gate defaults to at least 25% and 300 tokens saved
204
- for exactly this reason; in tool mode, just leave small things alone.
243
+ 50-token win buys you an image instead of quotable text. The proxy gate
244
+ requires at least 25% and 300 tokens saved for exactly this reason.
205
245
 
206
- ## Implicit mode (proxy)
246
+ ## Proxy mode, the fine print
207
247
 
208
- If you cannot touch the client, tanuki also runs as a local middlebox, the
209
- way pxpipe deploys. We left the proxy model early on because rewriting the
210
- system prompt into a user turn reads exactly like a prompt injection (an
211
- agent once flagged it as an attack, and it was right to). The middlebox
212
- came back with rules that remove that exact failure:
248
+ We left the proxy model early in this project because rewriting the system
249
+ prompt into a user turn reads exactly like a prompt injection — an agent
250
+ once flagged it as an attack, and it was right to. The proxy came back only
251
+ with rules that remove that failure:
213
252
 
253
+ - the system prompt and tool definitions are never touched
254
+ - nothing moves between roles or positions; a block is replaced in place by
255
+ a visible `[tanuki-context: ...]` marker plus its pages
256
+ - the latest message is never imaged (you may need to quote it)
257
+ - blocks carrying `cache_control` are never touched
258
+ - anything that doesn't clearly win forwards byte-for-byte
259
+
260
+ One reuse rule on top: when the same oversized block appears twice in one
261
+ request (agents re-read files constantly), the second byte-identical copy
262
+ becomes a one-line pointer to the pages above. Exact repeats only.
263
+
264
+ Responses stream through untouched. Savings land in
265
+ `~/.pxpipe/events.jsonl`, with the baseline named: what was billed plus
266
+ what the imaged blocks would have cost as text.
267
+
268
+ Knobs: `--port N` `--upstream URL` `--level 0-4` `--distill` `--codebook`
269
+ `--font tiny` `--min-chars N` `--ratio X` `--min-save N` `--max-pages N`.
270
+ Defaults are conservative: level 0, nothing lossy on.
271
+
272
+ ## Clients
273
+
274
+ **OMP (oh-my-pi)** — `~/.omp/agent/mcp.json`, or `.omp/mcp.json` in the project:
275
+
276
+ ```json
277
+ { "mcpServers": { "tanuki-context": { "command": "npx", "args": ["-y", "tanuki-context"] } } }
214
278
  ```
215
- npx tanuki-context proxy # listens on 127.0.0.1:8484
216
- export ANTHROPIC_BASE_URL=http://127.0.0.1:8484
279
+
280
+ **jcode** — `~/.jcode/mcp.json`, or `.jcode/mcp.json` in the project. jcode
281
+ speaks stdio only; `"shared": true` reuses one server across sessions:
282
+
283
+ ```json
284
+ { "mcpServers": { "tanuki-context": { "command": "npx", "args": ["-y", "tanuki-context"], "shared": true } } }
217
285
  ```
218
286
 
219
- Oversized text blocks in user messages and tool results are replaced in
220
- place by a short visible marker plus PNG blocks, only when `estimate` says
221
- imaging wins by a clear margin (default: at least 25% and 300 tokens). What
222
- it never does:
287
+ **pi** — pi has no MCP layer, so this package doubles as a pi extension (the
288
+ `"pi"` manifest field in package.json):
223
289
 
224
- - touch the system prompt or tool definitions
225
- - move content between roles or positions
226
- - image the latest message (you may need to quote it)
227
- - rewrite blocks carrying `cache_control` (that would break their cache)
228
- - rewrite anything when text is cheaper; those requests forward byte-for-byte
290
+ ```
291
+ pi install npm:tanuki-context
292
+ ```
229
293
 
230
- One thing it deliberately reuses: when the same oversized block appears
231
- twice in one request (agents re-read files constantly), the second
232
- byte-identical copy becomes a one-line pointer to the pages above instead
233
- of the same pages again. Exact repeats only; a one-byte difference images
234
- independently.
294
+ **Claude Code**:
235
295
 
236
- Responses stream through untouched. Savings land in `~/.pxpipe/events.jsonl`
237
- (the same file `tanuki_stats` reads), with the baseline named: what Anthropic
238
- billed, plus what the imaged blocks would have cost as text.
296
+ ```
297
+ claude mcp add tanuki-context -- npx -y tanuki-context
298
+ ```
239
299
 
240
- Knobs: `--port N` `--upstream URL` `--level 0-4` `--distill` `--codebook`
241
- `--font tiny` `--min-chars N` `--ratio X` `--min-save N` `--max-pages N`.
242
- Defaults are conservative: level 0, nothing lossy on.
300
+ **Rust instead of Node?** Same pipeline, same numbers, one 5.7 MB static
301
+ binary:
302
+
303
+ ```
304
+ cargo install --git https://github.com/Osyna/tanuki-context --branch rust
305
+ ```
306
+
307
+ Then `"command": "tanuki-context", "args": []` in any snippet above, or
308
+ `TANUKI_BIN=~/.cargo/bin/tanuki-context` for the pi extension.
243
309
 
244
310
  ## Claude Agent SDK
245
311
 
246
312
  `tanuki-context/agent` wires the pipeline into agents built on the
247
313
  [Claude Agent SDK](https://github.com/anthropics/claude-agent-sdk-typescript).
248
314
 
249
- External (subprocess per session, zero extra dependencies):
315
+ One agent (subprocess per session, zero extra dependencies):
250
316
 
251
317
  ```ts
252
318
  import { query } from "@anthropic-ai/claude-agent-sdk";
@@ -257,7 +323,7 @@ for await (const msg of query({ prompt: task, options: withTanuki({ model: "clau
257
323
  }
258
324
  ```
259
325
 
260
- In-process (one server instance shared by every agent in the process):
326
+ A team of agents (one in-process server shared by all of them):
261
327
 
262
328
  ```ts
263
329
  import { tanukiSdkServer, tanukiAllowedTools, TANUKI_INSTRUCTIONS } from "tanuki-context/agent";
@@ -271,11 +337,11 @@ const options = {
271
337
  // hand the same `options` to every agent in the team
272
338
  ```
273
339
 
274
- `withTanuki(options?)` merges into existing options without clobbering other
275
- servers or tools. `TANUKI_INSTRUCTIONS` is a canned prompt block that teaches
276
- agents the estimate-first workflow and the page decode grammar. The core
277
- package stays zero-dependency: the SDK and zod are optional peers, touched
278
- only inside `tanukiSdkServer()`.
340
+ `withTanuki(options?)` merges into your existing options without clobbering
341
+ other servers or tools. `TANUKI_INSTRUCTIONS` teaches agents the
342
+ estimate-first workflow, the `recommend` shortcut, and the page decode
343
+ grammar. The core package stays zero-dependency: the SDK and zod are
344
+ optional peers, touched only inside `tanukiSdkServer()`.
279
345
 
280
346
  Python SDK, plain dict config:
281
347
 
@@ -289,40 +355,30 @@ options = ClaudeAgentOptions(
289
355
  )
290
356
  ```
291
357
 
292
- ## CLI
358
+ ## CLI reference
293
359
 
294
360
  ```
295
361
  npx tanuki-context # MCP stdio server (default)
296
- npx tanuki-context proxy [--port 8484] [--upstream URL] [knobs] # implicit middlebox
362
+ npx tanuki-context proxy [--port 8484] [--upstream URL] [knobs] # implicit relay
297
363
  npx tanuki-context distill <file> [query] # stats JSON to stdout
298
364
  npx tanuki-context estimate <file> [level] [--distill] [--no-pack] [--font tiny] [--codebook]
299
- npx tanuki-context render <file> [level] [outdir] [--no-pack] [--font tiny] [--codebook]
365
+ npx tanuki-context render <file> [level] [outdir] [--distill] [--no-pack] [--font tiny] [--codebook]
300
366
  npx tanuki-context bench <file> <distill|pipeline> [level] [runs] # in-process timing
301
367
  ```
302
368
 
303
- ## Footprint
304
-
305
- Measured 2026-07-26 on a Ryzen 7 9700X, Linux. Spawn time is the median of 5
306
- cold starts to the first MCP response; the reference server is the original
307
- node MCP that wraps pxpipe's library (kept in `reference/node-mcp/`).
308
-
309
- | metric | node | bun | rust | node ref (pxpipe lib) |
310
- | ----------------------- | ------- | ------- | ------- | --------------------- |
311
- | spawn to first response | 35 ms | 27 ms | 0.4 ms | 158 ms |
312
- | idle server RSS | 87 MB | 50 MB | 3.8 MB | 177 MB |
313
- | distill a 12 MB log | 0.42 s | 0.31 s | 0.28 s | - |
314
- | install | 0.98 MB tarball, zero deps | same | 5.7 MB static binary | node_modules tree |
369
+ The example page above is one command:
370
+ `npx tanuki-context render your.log 0 ./pages --distill`.
315
371
 
316
372
  ## Two engines
317
373
 
318
374
  `main` is this TypeScript package. The [`rust` branch](../../tree/rust) is
319
375
  the same pipeline in Rust: same patch-grid token model, same escapes, same
320
376
  glyph atlas, same proxy rules. `reference/parity-ts.mjs` holds them to
321
- byte-identical JSON and pixel-identical PNGs on every knob combination, plus
322
- a full MCP session including error paths. When one engine changes, the
323
- other follows, or the parity harness fails loudly. The npm packaging and
324
- the Agent SDK / pi glue are TypeScript-only; everything the model sees is
325
- identical.
377
+ byte-identical JSON and pixel-identical PNGs — 92 checks, including
378
+ distilled renders (a check that exists because it once caught the Rust
379
+ engine emitting its dedupe summary in nondeterministic order; fixed, tested,
380
+ and now permanently covered). The npm packaging and the Agent SDK / pi glue
381
+ are TypeScript-only; everything the model sees is identical.
326
382
 
327
383
  The imaging engine is a remake of [pxpipe](https://github.com/teamchong/pxpipe):
328
384
  page geometry and glyphs come from pxpipe's own generated atlas (Spleen 5x8
@@ -344,7 +400,7 @@ unassigned codepoints.
344
400
  | `src/ladder.ts` | stage 1: levels 0-4 with the exact-recall guard |
345
401
  | `src/codebook.ts` | repeated tokens and path prefixes to sigils plus `·legend·` (opt-in) |
346
402
  | `src/render.ts` | stage 2: reflow, pack, wrap, page split, AA blit, tiny 4x6 font |
347
- | `src/proxy.ts` | implicit mode: local Anthropic middlebox, in-place block imaging |
403
+ | `src/proxy.ts` | implicit mode: local Anthropic middlebox, in-place block imaging, dedupe |
348
404
  | `src/atlas.ts` | glyph atlas (92,812 codepoints): metadata eager, pixels inflated lazily |
349
405
  | `src/png.ts` | minimal grayscale PNG encoder (`node:zlib`, filter-0 rows) |
350
406
  | `src/stats.ts` | event log summary |
@@ -363,7 +419,7 @@ Node-compatible files:
363
419
  ```
364
420
  bun run build # dist/cli.js + dist/agent.js + dist/pi.js
365
421
  bun test # 39 tests
366
- bun run parity # TS vs rust binary, byte/pixel-exact (needs TANUKI_BIN)
422
+ bun run parity # TS vs rust binary, 92 checks (needs TANUKI_BIN)
367
423
  ```
368
424
 
369
425
  Regenerating glyphs after a pxpipe atlas rebuild (needs a pxpipe checkout
package/dist/agent.js CHANGED
@@ -65,7 +65,7 @@ class ZJ{value;constructor(J){this.value=J}}function nJ(){let J=process.env.TANU
65
65
  //! tanuki-context estimate <file> [level] [--distill]
66
66
  //! tanuki-context render <file> [level] [outdir]
67
67
  //! tanuki-context proxy [--port N] [--upstream URL] [knobs] (implicit mode)
68
- var cJ="0.2.0",NJ=6;function JZ(J,Q){let Z=Math.min(J.length,Q.length),$=0;while($<Z&&J.charCodeAt($)===Q.charCodeAt($))$++;if($>=Z)return J.length-Q.length;return J.codePointAt($)-Q.codePointAt($)}function C(J,Q,Z=""){if(J===null||J===void 0)return"null";if(J instanceof ZJ){let z=J.value;return Number.isFinite(z)&&Number.isInteger(z)?z.toFixed(1):String(z)}let $=typeof J;if($==="string")return JSON.stringify(J);if($==="number"||$==="boolean")return String(J);if(Array.isArray(J)){if(J.length===0)return"[]";if(!Q){let q="[";for(let X=0;X<J.length;X++){if(X>0)q+=",";q+=C(J[X],!1)}return q+"]"}let z=Z+" ",K=`[
68
+ var cJ="0.2.1",NJ=6;function JZ(J,Q){let Z=Math.min(J.length,Q.length),$=0;while($<Z&&J.charCodeAt($)===Q.charCodeAt($))$++;if($>=Z)return J.length-Q.length;return J.codePointAt($)-Q.codePointAt($)}function C(J,Q,Z=""){if(J===null||J===void 0)return"null";if(J instanceof ZJ){let z=J.value;return Number.isFinite(z)&&Number.isInteger(z)?z.toFixed(1):String(z)}let $=typeof J;if($==="string")return JSON.stringify(J);if($==="number"||$==="boolean")return String(J);if(Array.isArray(J)){if(J.length===0)return"[]";if(!Q){let q="[";for(let X=0;X<J.length;X++){if(X>0)q+=",";q+=C(J[X],!1)}return q+"]"}let z=Z+" ",K=`[
69
69
  `;for(let q=0;q<J.length;q++){if(q>0)K+=`,
70
70
  `;K+=z+C(J[q],!0,z)}return K+`
71
71
  `+Z+"]"}let V=J,G=Object.keys(V).sort(JZ);if(G.length===0)return"{}";if(!Q){let z="{";for(let K=0;K<G.length;K++){if(K>0)z+=",";z+=JSON.stringify(G[K])+":"+C(V[G[K]],!1)}return z+"}"}let Y=Z+" ",H=`{
package/dist/cli.js CHANGED
@@ -71,7 +71,7 @@ var c={level:0,distill:!1,codebook:!1,font:"normal",minChars:4000,ratio:0.75,min
71
71
  //! tanuki-context estimate <file> [level] [--distill]
72
72
  //! tanuki-context render <file> [level] [outdir]
73
73
  //! tanuki-context proxy [--port N] [--upstream URL] [knobs] (implicit mode)
74
- var NZ="0.2.0",TJ=6;function jZ(J,Q){let Z=Math.min(J.length,Q.length),$=0;while($<Z&&J.charCodeAt($)===Q.charCodeAt($))$++;if($>=Z)return J.length-Q.length;return J.codePointAt($)-Q.codePointAt($)}function E(J,Q,Z=""){if(J===null||J===void 0)return"null";if(J instanceof e){let H=J.value;return Number.isFinite(H)&&Number.isInteger(H)?H.toFixed(1):String(H)}let $=typeof J;if($==="string")return JSON.stringify(J);if($==="number"||$==="boolean")return String(J);if(Array.isArray(J)){if(J.length===0)return"[]";if(!Q){let B="[";for(let X=0;X<J.length;X++){if(X>0)B+=",";B+=E(J[X],!1)}return B+"]"}let H=Z+" ",K=`[
74
+ var NZ="0.2.1",TJ=6;function jZ(J,Q){let Z=Math.min(J.length,Q.length),$=0;while($<Z&&J.charCodeAt($)===Q.charCodeAt($))$++;if($>=Z)return J.length-Q.length;return J.codePointAt($)-Q.codePointAt($)}function E(J,Q,Z=""){if(J===null||J===void 0)return"null";if(J instanceof e){let H=J.value;return Number.isFinite(H)&&Number.isInteger(H)?H.toFixed(1):String(H)}let $=typeof J;if($==="string")return JSON.stringify(J);if($==="number"||$==="boolean")return String(J);if(Array.isArray(J)){if(J.length===0)return"[]";if(!Q){let B="[";for(let X=0;X<J.length;X++){if(X>0)B+=",";B+=E(J[X],!1)}return B+"]"}let H=Z+" ",K=`[
75
75
  `;for(let B=0;B<J.length;B++){if(B>0)K+=`,
76
76
  `;K+=H+E(J[B],!0,H)}return K+`
77
77
  `+Z+"]"}let V=J,z=Object.keys(V).sort(jZ);if(z.length===0)return"{}";if(!Q){let H="{";for(let K=0;K<z.length;K++){if(K>0)H+=",";H+=JSON.stringify(z[K])+":"+E(V[z[K]],!1)}return H+"}"}let Y=Z+" ",G=`{
@@ -83,7 +83,7 @@ vs ~${H} text-tokens raw = TOTAL -${QJ(H,z)}%`,Z.protectedLines>0)X+=` · ${Z.pr
83
83
  `)}function EZ(){let J=Buffer.alloc(0);S.stdin.on("data",(Q)=>{let Z=J.length>0?Buffer.concat([J,Q]):Q,$=0,V;while((V=Z.indexOf(10,$))!==-1)QQ(Z.toString("utf8",$,V)),$=V+1;J=Z.subarray($)}),S.stdin.on("end",()=>{if(J.length>0)QQ(J.toString("utf8")),J=Buffer.alloc(0)})}function k(J){S.stderr.write(J+`
84
84
  `),S.exit(101)}function UJ(J){try{return FZ(J,"utf8")}catch{k("read file")}}function XJ(J,Q){if(!/^\+?[0-9]+$/.test(J))return null;let Z=BigInt(J.startsWith("+")?J.slice(1):J);return Z>Q?null:Number(Z)}var ZQ=255n,$Q=0xffffffffffffffffn;function YQ(){let J=S.argv.slice(1);switch(J[1]){case"distill":{let Q=J[2]??k("usage: tanuki-context distill <file> [query]"),Z=UJ(Q),$=b(Z,J[3]??null,2);S.stdout.write(E($.stats,!1)+`
85
85
  `);break}case"estimate":{let Q=J[2]??k("usage: tanuki-context estimate <file> [level] [--distill] [--no-pack] [--font tiny] [--codebook]"),Z=UJ(Q),$=J.slice(3).filter((H)=>!H.startsWith("--")),V=$.length>0?XJ($[0],$Q)??0:0,z=J.indexOf("--font"),Y=z!==-1&&J[z+1]!==void 0?J[z+1]:"normal",G=GQ({text:Z,level:V,distill:J.includes("--distill"),pack:!J.includes("--no-pack"),font:Y,codebook:J.includes("--codebook")});S.stdout.write(E(G,!1)+`
86
- `);break}case"render":{let Q=J[2]??k("usage: tanuki-context render <file> [level] [outdir] [--no-pack] [--font tiny] [--codebook]"),Z=UJ(Q),$=J.slice(3).filter((O)=>!O.startsWith("--")),V=$.length>0?XJ($[0],ZQ)??0:0,z=!J.includes("--no-pack"),Y=J.includes("--codebook"),G=J.indexOf("--font"),H=g(G!==-1&&J[G+1]!==void 0?J[G+1]:"normal"),K=BJ(Z,V,!1,null,Y),B=p(K.compressed,!0,z,H),X=B.tokens;S.stdout.write(E({pages:B.pages.length,imageTokens:X,dropped:B.dropped,rawTextTokens:JJ(h(Z))},!1)+`
86
+ `);break}case"render":{let Q=J[2]??k("usage: tanuki-context render <file> [level] [outdir] [--distill] [--no-pack] [--font tiny] [--codebook]"),Z=UJ(Q),$=J.slice(3).filter((O)=>!O.startsWith("--")),V=$.length>0?XJ($[0],ZQ)??0:0,z=!J.includes("--no-pack"),Y=J.includes("--codebook"),G=J.indexOf("--font"),H=g(G!==-1&&J[G+1]!==void 0?J[G+1]:"normal"),K=BJ(Z,V,J.includes("--distill"),null,Y),B=p(K.compressed,!0,z,H),X=B.tokens;S.stdout.write(E({pages:B.pages.length,imageTokens:X,dropped:B.dropped,rawTextTokens:JJ(h(Z))},!1)+`
87
87
  `);let U=$[1];if(U!==void 0){try{MZ(U,{recursive:!0})}catch{k("mkdir")}for(let O=0;O<B.pages.length;O++)try{WZ(`${U}/page${O}.png`,B.pages[O].png)}catch{k("write png")}}break}case"bench":{let Q=J[2]??k("usage: tanuki-context bench <file> <op> [level] [runs] [--distill]"),Z=J[3]??"pipeline",$=J[4]!==void 0?XJ(J[4],ZQ)??0:0,V=J[5]!==void 0?XJ(J[5],$Q)??3:3,z=J.includes("--distill"),Y=UJ(Q),G=[],H=null;for(let K=0;K<=V;K++){let B=performance.now();if(Z==="distill")H=b(Y,null,2).stats;else{let X=BJ(Y,$,z,null,!1),U=p(X.compressed,!0,!1,"normal");H={pages:U.pages.length,imageTokens:U.tokens,stage1Chars:h(X.compressed),dropped:U.dropped}}if(K>0)G.push(performance.now()-B)}if(G.sort((K,B)=>K-B),G.length===0)k("index out of bounds: the len is 0 but the index is 0");S.stdout.write(E({medianMs:new e(G[Math.floor(G.length/2)]),runs:V,result:H},!1)+`
88
88
  `);break}case"proxy":{let Q=(V,z)=>{let Y=J.indexOf(V);if(Y===-1||J[Y+1]===void 0)return z;let G=Number(J[Y+1]);return Number.isFinite(G)?G:z},Z=J.indexOf("--upstream"),$=J.indexOf("--font");JQ({port:Q("--port",8484),upstream:Z!==-1&&J[Z+1]!==void 0?J[Z+1]:S.env.TANUKI_UPSTREAM??"https://api.anthropic.com",level:Q("--level",c.level),distill:J.includes("--distill"),codebook:J.includes("--codebook"),font:g($!==-1&&J[$+1]!==void 0?J[$+1]:"normal"),minChars:Q("--min-chars",c.minChars),ratio:Q("--ratio",c.ratio),minSave:Q("--min-save",c.minSave),maxPages:Q("--max-pages",c.maxPages)});break}case"serve":case void 0:EZ();break;default:S.stderr.write(`unknown command: ${J[1]}
89
89
  usage: tanuki-context [serve|proxy|distill|estimate|render] ...
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "tanuki-context",
3
- "version": "0.2.0",
3
+ "version": "0.2.1",
4
4
  "description": "Token-cutting context pipeline as an MCP server: distill logs, compress text, render dense PNG pages (pxpipe imaging engine). Zero dependencies.",
5
5
  "license": "MIT",
6
6
  "type": "module",