@tokmizer/plugin 0.9.60 → 0.9.61

package/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "tokmizer",
-  "version": "0.9.60",
+  "version": "0.9.61",
   "description": "Tokmizer reduces token consumption for AI agents.",
   "author": {
     "name": "Tokmizer",

package/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "tokmizer",
-  "version": "0.9.60",
+  "version": "0.9.61",
   "description": "Tokmizer reduces token consumption for OpenAI Codex.",
   "author": {
     "name": "Tokmizer",

package/README.md

@@ -1,6 +1,6 @@
 # Tokmizer
 
-Tokmizer reduces token consumption for AI coding tools: Claude Code, OpenAI Codex, Cursor, Aider, Gemini CLI, Cline, Windsurf, and GitHub Copilot. Works on macOS, Linux, and Windows.
+Tokmizer reduces token consumption for 21 AI coding agents, including Claude Code, OpenAI Codex, Cursor, Gemini CLI and GitHub Copilot. Aider, Cline and Windsurf run through `tkr wrap`. Works on macOS, Linux, and Windows.
 
 It shortens command output before it reaches the model, reducing the space it takes in the context window. The reduction depends on the output and the chosen mode. This is not a guarantee of a lower bill or faster coding, rendering, or builds.
 
@@ -15,7 +15,14 @@ In Claude Code:
 
 Then restart Claude Code once, hooks only load at startup. Verify with `/tkr-status`, you should see Status active.
 
-Any other tool (universal shell-shim):
+Any other agent: install the CLI with npm (Node.js 20.11 or newer), then set up every agent found on this machine:
+
+```bash
+npm install -g @tokmizer/plugin
+tkr setup --all
+```
+
+Without Node.js, the install script uses npm when Node.js 20.11 or newer is there and otherwise downloads a standalone build for macOS or Linux, checked against its SHA-256. It leaves your shell profile unchanged. Then run `tkr setup --all`.
 
 ```bash
 curl -fsSL https://tokmizer.com/install.sh | bash
@@ -25,7 +32,7 @@ Windows (PowerShell):
 
 ```powershell
 npm.cmd install -g @tokmizer/plugin
-tkr.cmd shim install
+tkr.cmd setup --all
 ```
 
 See the [installation guide](https://tokmizer.com/en/docs#installation) for per-tool notes.
@@ -36,7 +43,7 @@ Codex, after installing with npm or Homebrew:
 tkr setup-codex
 ```
 
-Restart Codex, then approve the hook in `/hooks`.
+Restart Codex, then approve the hook in `/hooks`. `tkr setup --all` includes Codex.
 
 If Codex reports `PreToolUse hook returned updatedInput without permissionDecision:allow`, it is running the Claude Code hooks: an older Tokmizer plugin installed in Codex, or hooks copied over by Codex `/import`. Run `tkr doctor --fix`, then approve the Tokmizer hook in `/hooks` and restart Codex.
 
@@ -93,7 +100,9 @@ Each command adds one `PreToolUse` hook for the agent's shell tool. It rewrites
 
 Permission rules stay in force. Agents match their allow, ask and deny rules against the command text, so Tokmizer leaves a command untouched whenever one of your deny or ask rules matches it. Tokmizer reads those rules for Claude Code (`permissions.deny` and `permissions.ask` in user, project, local and managed settings; a command matched by one of your `permissions.allow` rules is left untouched too, so that rule keeps approving it), OpenCode (`permission` in `opencode.json`), Cursor CLI (`permissions.deny`), Gemini CLI (policy files and `tools.exclude`) and Antigravity CLI (`permissions.deny` and `permissions.ask`). For the other agents, and on top of the Cursor, Gemini and Antigravity rules, only inspection, build and test commands (`cat`, `rg`, `git log`, `npm test`, `cargo build` and similar; inspection only for Codex) are routed; anything else, such as `git push` or `rm`, runs exactly as typed. If a settings file cannot be read, nothing is routed. Shell functions, aliases, builtins and anything that is not a program on your `PATH` always run exactly as typed.
 
-In Claude Code, a command is rewritten only when that adds no permission prompt: in `bypassPermissions` mode, or when a plain `Bash` allow rule in your user or managed settings applies and the session is in default, `acceptEdits` or `dontAsk` mode. Otherwise it runs exactly as typed, so Claude Code still approves read-only commands such as `ls`, `cat` or `git status` on its own, and Tokmizer shortens the output once the command has succeeded.
+In Claude Code, a command is rewritten only when that adds no permission prompt: in `bypassPermissions` mode, or when a plain `Bash` allow rule in your user or managed settings applies and the session is in default, `acceptEdits` or `dontAsk` mode. Otherwise it runs exactly as typed, so Claude Code still approves read-only commands such as `ls`, `cat` or `git status` on its own, and Tokmizer shortens the output once the command has succeeded. Claude Code runs that after-command hook only for successful commands, so in this default mode a failing command reaches Claude in full. When commands are rewritten, failing output is shortened too, with its failure details kept.
+
+Aider, Cline and Windsurf have no hooks. Start them through Tokmizer, for example `tkr wrap aider`: only that session goes through Tokmizer's command shims. Your own terminals are not changed; `tkr shim global-path on` routes every terminal through the shims by adding a PATH line to your shell profile, and `tkr shim global-path off` removes it.
 
 Every agent found on this machine at once (each `tkr setup-…` above runs in turn, agents that are not installed are skipped, and a summary ends the run):
 
@@ -111,10 +120,10 @@ Homebrew:
 
 ```bash
 brew install norwelltw/tokmizer/tokmizer
-tkr shim install
+tkr setup --all
 ```
 
-Node.js 20.11 or newer is required for npm installation. Open a new terminal after setting up shims, then run `tkr status`. An offline installation reports that free activation is pending; run `tkr status` again when online.
+The formula brings Node.js with it. Installed with npm or Homebrew, tkr updates itself when a new version is out; `tkr auto-update off` stops that. Restart each agent after its setup, then run `tkr status`. An offline installation reports that free activation is pending; run `tkr status` again when online.
 
 ## Free access
 
@@ -139,7 +148,7 @@ In Claude Code:
 - `/tkr-telemetry` views or changes usage stats (command names and token counts; on by default, off in one command)
 - `/tkr-update` updates the plugin when a new version is out
 
-From any terminal (after the shell-shim install, or `npm install -g @tokmizer/plugin`):
+From any terminal (after installing with npm, the install script or Homebrew):
 
 - `tkr status` shows your account and engine status
 - `tkr gain` shows the estimated reduction in command-output tokens
@@ -151,6 +160,9 @@ From any terminal (after the shell-shim install, or `npm install -g @tokmizer/pl
 - `tkr settings show|set|configure` manages settings and syncs them across your machines
 - `tkr network off|on|status` blocks or allows every connection to tokmizer.com
 - `tkr migrate rtk` moves your rtk filters, settings and history over and turns off rtk hooks where Tokmizer runs (see [Switching from rtk to Tokmizer](#switching-from-rtk-to-tokmizer))
+- `tkr setup [--all]` sets up the agents found on this machine; `tkr setup-<agent> --remove` takes one out
+- `tkr doctor [--fix]` shows what is detected and repairs outdated shims and hooks
+- `tkr uninstall` removes every hook, shim and PATH line Tokmizer added (see [Uninstall](#uninstall))
 
 ## /power and the reducer in other agents
 
@@ -187,7 +199,7 @@ Never overwritten: an existing `power` command or skill that Tokmizer did not wr
 
 Choose `light` for gentle reduction, `balanced` (the default) for more compact output, or `max` for the strongest standard reduction. The aliases `normal`, `compact`, and `minimal` map to these levels respectively. Short or exact output can be identical across levels.
 
-In the next release, Light keeps recognized file reads complete by default, including large reads. Balanced and Max can still fold code bodies in large files. Choose Light when you need implementation details. Explicit outlining remains available to callers that request it. Host limits and the CLI's output buffer still apply.
+Since 0.9.54, Light keeps recognized file reads of up to 16 KB complete; larger reads are shortened again. Balanced and Max can fold code bodies. Choose Light when you need implementation details. Explicit outlining remains available to callers that request it. Host limits and the CLI's output buffer still apply.
 
 Since 0.9.52, short code reads and small explicit excerpts stay complete by default in all three standard modes. Larger reads can still be shortened. Recognized failures retain their diagnostics; Max can summarize repeated log lines with an occurrence count. Distinct diagnostic messages and command exit codes are preserved.
 
@@ -212,7 +224,7 @@ Terminal commands managed by Tokmizer use the mode selected when execution start
 
 `tkr gain` measures output size before and after compression and estimates tokens as UTF-8 bytes divided by four. Token and cost figures are estimates about command output, not your actual bill or a measure of task quality or completion speed.
 
-In the next release, successful `tkr restore` calls are counted as additional output in the local gain history, even with telemetry disabled. They reduce the net saving; repeated restorations count again and can make it negative. The display shows restored tokens separately. `gain --json` exposes the net estimate in `observed.savedTokens` and restoration totals in `restoration`; the existing top-level `savedTokens` remains the total for reduced outputs only. Earlier restorations cannot be reconstructed from local history. Manually repeated commands, prompts, replies and replayed conversation context are not attributed to restoration costs.
+Since 0.9.53, successful `tkr restore` calls are counted as additional output in the local gain history, even with telemetry disabled. They reduce the net saving; repeated restorations count again and can make it negative. The display shows restored tokens separately. `gain --json` exposes the net estimate in `observed.savedTokens` and restoration totals in `restoration`; the existing top-level `savedTokens` remains the total for reduced outputs only. Earlier restorations cannot be reconstructed from local history. Manually repeated commands, prompts, replies and replayed conversation context are not attributed to restoration costs.
 
 Other local views: `tkr gain --by-rule` groups savings by rule (your custom filters by name, built-in rules by command), `tkr gain --failures` lists commands whose output went through unchanged and why (no rule, nothing to cut, failed run kept in full), and `tkr gain --quota [pro|5x|20x]` estimates the share of a Claude plan budget the savings represent. The quota view is an estimate, not a plan limit: tokens saved per day × 30, divided by an assumed 44,000 tokens per 5-hour window × 144 windows × the plan multiplier (1, 5 or 20). Anthropic does not publish plan limits in tokens. All three combine with a period, `--project` and `--json`.
 
@@ -236,7 +248,7 @@ Entries are kept 30 days, up to 8 MB each and 200 MB in total; the oldest go fir
 
 On October 9, 2026, a fixed set of 30 real and synthetic command outputs contained 774,863 tokens before compression. The table counts tokens with the cl100k tokenizer, excluding restoration notices. It is a test corpus, not a sample of customer usage.
 
-These are historical results for the earlier mode behavior, not measurements of the upcoming Light read policy.
+These are historical results for the earlier mode behavior, not measurements of the Light read policy introduced in 0.9.54.
 
 | Mode | Output tokens | Preservation checks passed |
 | --- | ---: | ---: |
@@ -255,6 +267,10 @@ In a separate controlled test for 0.9.52, a repetitive failure log went from 1,0
 
 These tokenizer counts are separate from the byte-based estimates in `tkr gain`. They measure output volume, not your bill, an agent's task success, or execution speed. See the [mode guide](https://tokmizer.com/en/docs#configuration) for usage and limits.
 
+### Benchmark of October 17, 2026
+
+Each tool's Claude Code hook chain replayed 68 real command outputs offline (network cut, empty home folder, no account, median of 3 runs) with Tokmizer 0.9.60 and its default settings, against rtk 1.0.1. Tokmizer removed 84% of tokens in Claude Code's default mode and up to 93% when commands are rewritten (`bypassPermissions` or a plain `Bash` allow rule); rtk removed 55%. All 307 failure details were kept in both Tokmizer modes. The full results are on [tokmizer.com/en/compare/rtk](https://tokmizer.com/en/compare/rtk).
+
 ## Custom filters
 
 Add your own output rules as `*.toml` files in `~/.tokmizer/filters/` (or `$TOKMIZER_HOME/filters/`). They load next to the built-in rules and win over them for the commands they match. They stay on your machine and are never uploaded. A file or rule that fails validation is skipped; `tkr doctor` shows what was loaded and why something was skipped.
@@ -319,10 +335,10 @@ What leaves your machine, and when:
 |---|---|---|
 | First online run, no account | A random install ID, your computer name, OS and version, CPU model and core count, a SHA-256 hash of your network card (MAC) address, a SHA-256 hash of your user name, a fingerprint (SHA-256 of all of these), your time zone and the plugin version | Create the install token used to download rules |
 | `tkr link <key>` | The same, plus your key, and your email only if you pass `--email` | Attach this machine to your account |
-| Regular check-in (heartbeat) | Install ID, fingerprint, plugin version, local date and time zone | Refresh access and learn that a new version is out |
+| Regular check-in (heartbeat) | Install ID, fingerprint, plugin version, local date and time zone; while usage stats are on, also install health counts, for example outdated shims, broken hooks, repairs and commands that ran without Tokmizer because its files were missing | Refresh access, learn that a new version is out and spot broken installs |
 | Rule downloads | Install ID, fingerprint, plugin version | Get the latest compression rules and the `/power` and reducer instructions |
 | Settings sync, linked accounts only | Your Tokmizer settings | Keep them in sync with the dashboard |
-| Usage stats, only while they are on (with or without an account) | Command names such as `npm install` (never arguments; Git and GitHub commands excluded), token counts, rule IDs with output sizes, session ID, agent name | See which commands to optimize next |
+| Usage stats, only while they are on (with or without an account) | Command names such as `npm install` (never arguments; Git and GitHub commands only as the tool and its action, such as `git push` or `gh pr`), token counts, rule IDs with output sizes, session ID, agent name | See which commands to optimize next |
 | `tkr unlink` | Install ID, fingerprint | Remove this device from your account |
 | CLI self-update, after a check-in reports a new version | Runs `npm i -g @tokmizer/plugin@<version>` or `brew upgrade tokmizer`, which contact the npm registry, Homebrew and GitHub | Keep the CLI current (`tkr auto-update off` stops only this) |
 
@@ -351,7 +367,7 @@ tkr network on
 
 In CI or for one shell, set `TOKMIZER_NETWORK=0`. `tkr status` and `tkr doctor` show the current state.
 
-While it is off there is no sign-up, heartbeat, rule download, settings sync, usage stats or update check. Tokmizer keeps compressing locally: with the last downloaded rules for up to 7 days, then with its core built-in rules. Settings are saved on this machine only. `tkr link` and `tkr heartbeat` explain how to turn the network back on.
+While it is off there is no sign-up, heartbeat, rule download, settings sync, usage stats or update check. Tokmizer keeps compressing locally: with the last downloaded rules for up to 7 days, then with the rules built into the CLI. Settings are saved on this machine only. `tkr link` and `tkr heartbeat` explain how to turn the network back on.
 
 The switch does not cover Claude Code's own marketplace auto-update, which `/tkr-update` turns on (set `extraKnownMarketplaces.tokmizer.autoUpdate` to `false` in `~/.claude/settings.json` to stop it), nor commands you run yourself such as the installer, `/tkr-update`, npm or brew.
 
@@ -361,6 +377,21 @@ The switch does not cover Claude Code's own marketplace auto-update, which `/tkr
 - The public [redaction helpers](https://github.com/norwelltw/Tokmizer/blob/main/src/learn/redact.ts) are available for inspection.
 - Local state lives in `~/.tokmizer/`.
 
+## Uninstall
+
+```bash
+tkr uninstall
+```
+
+`tkr uninstall` removes every hook, shim and PATH line Tokmizer added, `/power` and the reducer in the other agents, and the config backups. It does not remove the program itself. Then run the command for the way you installed it:
+
+```bash
+npm uninstall -g @tokmizer/plugin
+brew uninstall tokmizer
+```
+
+In Claude Code: `/plugin uninstall tokmizer@tokmizer`. Your savings history stays in `~/.tokmizer` until you delete that folder. To pause Tokmizer without uninstalling: `tkr settings set enabled false`.
+
 ## License
 
 Proprietary. See [LICENSE](LICENSE) and the EULA at [tokmizer.com/eula](https://tokmizer.com/eula).

package/command-templates/tkr-status.md

@@ -12,7 +12,7 @@ node "${CLAUDE_PLUGIN_ROOT}/dist/bin/tkr.js" status
 
 Print the output as is. It includes the network state, the token state, whether the device is linked and whether `/power` is available right now. Tokmizer is free: there is no plan or subscription to mention.
 
-If the user asks what Tokmizer sends over the network, answer plainly. No account or email is needed. Tokmizer contacts tokmizer.com to keep its rules up to date and to improve the product. It sends a random install ID, the computer name, OS and CPU model, a hash of the network card address and of the user name, and the plugin version. This is not anonymous: the same machine always gives the same values. Command contents and output are never sent. Usage stats (command names, token counts and install health counts) are sent only if the user turned them on.
+If the user asks what Tokmizer sends over the network, answer plainly. No account or email is needed. Tokmizer contacts tokmizer.com to keep its rules up to date and to improve the product. It sends a random install ID, the computer name, OS and CPU model, a hash of the network card address and of the user name, and the plugin version. This is not anonymous: the same machine always gives the same values. Command contents and output are never sent. Usage stats (command names, token counts and install health counts) are on by default and sent only while they are on; `tkr telemetry off` stops them. Git and GitHub commands are recorded only as the tool and its action, such as `git push`.
 
 If the output shows `Network: off`, explain that nothing leaves the machine and Tokmizer keeps compressing with the rules already on it. To turn it back on: `tkr network on`. To turn everything off: `tkr network off`; usage stats only: `tkr telemetry off`.