@tokmizer/plugin 0.9.50 → 0.9.52

package/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "tokmizer",
-  "version": "0.9.50",
+  "version": "0.9.52",
   "description": "Tokmizer reduces token consumption for AI agents.",
   "author": {
     "name": "Tokmizer",

package/README.md

@@ -2,7 +2,7 @@
 
 Tokmizer reduces token consumption for AI coding tools: Claude Code, OpenAI Codex, Cursor, Aider, Gemini CLI, Cline, Windsurf, and GitHub Copilot. Works on macOS, Linux, and Windows.
 
-It compresses noisy command output (logs, diffs, listings, test runs) before it reaches the model. Your agent does more work with the same context window, and you pay for fewer tokens.
+It shortens command output before it reaches the model, reducing the space it takes in the context window. The reduction depends on the output and the chosen mode. This is not a guarantee of a lower bill or faster coding, rendering, or builds.
 
 ## Install
 
@@ -28,7 +28,7 @@ npm.cmd install -g @tokmizer/plugin
 tkr.cmd shim install
 ```
 
-See [scripts/README-INSTALL.md](scripts/README-INSTALL.md) for per-tool notes.
+See the [installation guide](https://tokmizer.com/en/docs#installation) for per-tool notes.
 
 Codex, after installing with npm or Homebrew:
 
@@ -62,9 +62,10 @@ In Claude Code:
 - `/tkr` shows or changes settings
 - `/tkr-link tokmizer_yourkey` connects an optional account
 - `/tkr-unlink` unlinks this machine from your account
-- `/tkr-gain` shows tokens saved
+- `/tkr-gain` shows the estimated reduction in command-output tokens
 - `/tkr-status` shows your current plan and usage
 - `/tkr-compression` picks the compression level (light, balanced, max)
+- `/tkr-yolo on` explicitly enables YOLO on this machine; `/tkr-yolo off` disables it
 - `/tkr-reducer` makes the assistant's replies shorter (off, light, balanced, max)
 - `/tkr-telemetry` views or changes anonymous telemetry
 - `/tkr-update` updates the plugin when a new version is out
@@ -72,28 +73,75 @@ In Claude Code:
 From any terminal (after the shell-shim install, or `npm install -g @tokmizer/plugin`):
 
 - `tkr status` shows your current plan and status
-- `tkr gain` shows tokens saved
+- `tkr gain` shows the estimated reduction in command-output tokens
 - `tkr link <key>` connects an optional account
 - `tkr unlink` unlinks this machine
 - `tkr compression light|balanced|max` picks how hard output gets compressed (default balanced)
+- `tkr yolo on|off|status` enables, disables, or checks the local YOLO mode
 - `tkr reducer off|light|balanced|max` makes the assistant's replies shorter (default off)
 - `tkr settings show|set|configure` manages settings and syncs them across your machines
 
-## What's in this repository
+## Choosing a compression mode
 
-The user-facing parts of the Tokmizer plugin:
+Choose `light` for gentle reduction, `balanced` (the default) for more compact output, or `max` for the strongest standard reduction. The aliases `normal`, `compact`, and `minimal` map to these levels respectively. Short or exact output can be identical across levels.
 
-- Plugin manifests for Claude Code (`.claude-plugin/`, `hooks/`) and Codex (`codex-plugin/`)
-- The `/tkr` slash commands (`commands/`)
-- Install scripts for macOS, Linux, and Windows (`scripts/`)
-- Selected plugin sources (`src/`): session UI, local stats and usage tracking, host detection, and the telemetry redaction rules that strip secrets before anything leaves your machine
+Since 0.9.52, short code reads and small explicit excerpts stay complete by default in all three standard modes. Larger reads can still be shortened. Recognized failures retain their diagnostics; Max can summarize repeated log lines with an occurrence count. Distinct diagnostic messages and command exit codes are preserved.
 
-The optimization engine and the API client are proprietary. They ship compiled with the [`@tokmizer/plugin`](https://www.npmjs.com/package/@tokmizer/plugin) npm package and are not part of this repository.
+Without a linked account, standard settings stay on this machine and work offline after initial activation. With a linked account, changes require a connection and sync across machines. Linking an account restores its server settings.
+
+The separate `reducer` setting provides response-style instructions for Claude Code, starting with the next session. Automatic injection is not implemented for Codex or other agents. The instructions ask the model to preserve code, errors, and security information; compliance and actual savings depend on the model. If the ruleset is unavailable, the CLI reports it instead of claiming activation. Reconnect and run `tkr heartbeat` before starting a new session.
+
+YOLO is off by default and requires an explicit `on`. It applies only to this machine, is not synchronized, and cannot be enabled from the dashboard.
+
+| Command | Effect |
+| --- | --- |
+| `tkr yolo on` | Enable more aggressive compression on this machine |
+| `tkr yolo off` | Return to the usual compression mode |
+| `tkr yolo status` | Show whether YOLO is enabled |
+| `tkr compression light`, `tkr compression balanced`, or `tkr compression max` | Disable YOLO locally. Linked accounts need a connection to sync the standard level |
+
+**YOLO can remove errors, list entries, and useful details.** It creates no new restoration copies of compressed output. Existing cached output is not deleted, but turning YOLO off cannot recover details discarded while it was on. Command exit codes remain unchanged.
+
+Use YOLO for repeated commands only when you accept incomplete output and can check the result independently. Turn it off for debugging, diagnosis, exhaustive code review, audits, or any task that needs the complete output. A preserved exit code alone does not verify the result. YOLO is not a promise of faster coding, rendering, or builds.
+
+Terminal commands managed by Tokmizer use the mode selected when execution starts. Native agent tools use the mode active when their output is received. Output processing that has already started in YOLO remains without a restoration copy after YOLO is turned off. Turning YOLO on blocks new restoration copies. Change modes between commands. This concerns Tokmizer's copies, not any history kept by your agent or terminal.
+
+`tkr gain` measures output size before and after compression and estimates tokens as UTF-8 bytes divided by four. Token and cost figures are estimates about command output, not your actual bill or a measure of task quality or completion speed.
+
+## Measured comparison
+
+On October 9, 2026, a fixed set of 30 real and synthetic command outputs contained 774,863 tokens before compression. The table counts tokens with the cl100k tokenizer, excluding restoration notices. It is a test corpus, not a sample of customer usage.
+
+| Mode | Output tokens | Preservation checks passed |
+| --- | ---: | ---: |
+| `light` | 112,816 | 85 / 85 |
+| `balanced` | 102,052 | 85 / 85 |
+| `max` | 93,131 | 85 / 85 |
+| YOLO | 1,925 | 44 / 85 |
+
+The 85 checks cover 13 of the 30 cases. The other 17 have no complete preservation checks, so these results do not prove universal fidelity. YOLO's smaller output comes with observed information loss.
+
+The result also depends on the output. In a separate set of five JSON examples, `light` produced 285 tokens, `balanced` 253, and both `max` and YOLO 197. YOLO does not always save more than `max`.
+
+In one real failed test, `max` kept 302 tokens and YOLO kept 77. Both retained the actual and expected values. YOLO removed the explanation about invoice 42 and the location `payment.test.mjs:4:10`, which were retained by `max`. That missing context can matter when fixing the failure.
+
+In a separate controlled test for 0.9.52, a repetitive failure log went from 1,094 to 84 tokens in Max (92.3% fewer, counted with cl100k). Every distinct diagnostic and the repetition counts were retained. This result applies to that example, not to typical usage.
+
+These tokenizer counts are separate from the byte-based estimates in `tkr gain`. They measure output volume, not your bill, an agent's task success, or execution speed. See the [mode guide](https://tokmizer.com/en/docs#configuration) for usage and limits.
+
+## Distribution
+
+This README is shared by the npm package and the public GitHub repository.
+
+- The npm package includes the CLI, Claude Code hooks and command templates, and the Codex manifest.
+- The public repository contains installation scripts, distribution manifests, the `/tkr` entry command, and selected UI, usage, and redaction helpers.
+
+The optimization engine and API client ship compiled with [`@tokmizer/plugin`](https://www.npmjs.com/package/@tokmizer/plugin). Their source code is not in the public repository.
 
 ## Privacy
 
-- Telemetry is on by default (`telemetryOptIn`) so your online dashboard can count your savings. We only receive command names (like `git log`), never your code, arguments, or output. Turn it off anytime with `tkr telemetry off`.
-- Before any usage event is recorded, [`src/learn/redact.ts`](src/learn/redact.ts) strips passwords, API keys, tokens, and cloud credentials. You can audit those rules here.
+- When telemetry is enabled, Tokmizer reports command names, usage counters, and session metadata, without your source code, arguments, or command output. Turn it off with `tkr telemetry off`.
+- The public [redaction helpers](https://github.com/norwelltw/Tokmizer/blob/main/src/learn/redact.ts) are available for inspection.
 - Local state lives in `~/.tokmizer/`.
 
 ## License

package/codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "tokmizer",
-  "version": "0.9.50",
+  "version": "0.9.52",
   "description": "Tokmizer reduces token consumption for OpenAI Codex.",
   "author": "Tokmizer",
   "homepage": "https://tokmizer.com"

package/command-templates/tkr-compression.md

@@ -15,7 +15,7 @@ Then ask the user to pick a level using the AskUserQuestion tool (single choice,
 
 - `light` — gentle: standard cleanup only
 - `balanced` — default: also collapses repeated output and blank runs
-- `max` — strongest: also drops verbose log noise (errors and results always stay)
+- `max` — strongest standard level: also drops verbose log noise
 
 Mark the current level in the option labels (e.g. "balanced (current)"). After they pick, apply it:
 
@@ -23,6 +23,8 @@ Mark the current level in the option labels (e.g. "balanced (current)"). After t
 node "${CLAUDE_PLUGIN_ROOT}/dist/bin/tkr.js" compression <level>
 ```
 
-Print the output as is. The change syncs to all the user's machines and applies to their next commands. Whatever the level, failed commands keep their full output.
+Print the output as is. For a linked account, standard levels sync across machines and changing them requires a connection. Without a linked account, they are saved locally and can be changed offline after initial activation. The aliases normal, compact and minimal map to light, balanced and max. Selecting any standard level first disables YOLO on this machine, even if syncing then fails. Report a sync failure separately from YOLO being disabled. Short or exact output can be identical across levels.
+
+YOLO is a separate, optional local mode, managed with `/tkr-yolo` or `tkr yolo on|off|status`. It can discard content and diagnostics, including failed-command output; discarded output cannot be restored. Command exit codes are preserved. Never enable YOLO automatically because the user asks to save more tokens. Only activate it after an explicit request for YOLO, and explain the warning before running the command. The standard levels keep failed-command output when YOLO is off.
 
 If the command fails because the plugin files are missing, tokmizer was uninstalled. Offer to clean up: delete the tokmizer-managed `tkr*.md` copies (they end with a `managed-by: tokmizer` marker) from `~/.claude/commands/`.

package/command-templates/tkr-gain.md

@@ -1,9 +1,9 @@
 ---
-description: Show how many tokens and dollars Tokmizer saved you.
+description: Show estimated token reductions in recorded tool outputs.
 argument-hint: [--day | --week]
 ---
 
-Show the user their Tokmizer savings.
+Show the user Tokmizer's recorded tool-output reductions.
 
 Run the bundled CLI:
 
@@ -11,8 +11,17 @@ Run the bundled CLI:
 node "${CLAUDE_PLUGIN_ROOT}/dist/bin/tkr.js" gain
 ```
 
-Print the output as is. If the user asks for today only or this session, run `session` instead of `gain`.
+Choose the scope the user requested:
 
-If the output shows zero activity, tell the user the savings build up as they keep working and there is nothing to configure.
+- Default: `gain` for the retained local history.
+- Today or `--day`: `gain --day` for the current local calendar day.
+- Last seven days or `--week`: `gain --week`.
+- This session: `session` for session coverage. This command reports routing counts, not token or dollar savings.
+
+Print the output as is, including its measurement scope and caveats. Token counts are estimates from output bytes, not model tokenizer counts. Dollar values use reference input prices and do not measure a bill reduction. Percentages apply only to the recorded tool outputs, not all tokens used by the agent.
+
+Keep modeled reducer estimates separate from observed output reductions. Do not add them together, invent replay or repeated-context savings, or infer avoided model calls. Do not turn a period's estimate into a monthly projection or paid-plan payback. A comparison with another optimizer requires measurements on the same workload; these local totals alone do not establish one.
+
+If the output shows zero activity, report that no activity was recorded in the selected scope. This does not establish whether the optimizer is configured or running.
 
 If the command fails because the plugin files are missing, tokmizer was uninstalled. Offer to clean up: delete the tokmizer-managed `tkr*.md` copies (they end with a `managed-by: tokmizer` marker) from `~/.claude/commands/`.

package/command-templates/tkr-yolo.md

@@ -0,0 +1,32 @@
+---
+description: Show, enable or disable the optional local YOLO mode. Content and diagnostics can be lost.
+argument-hint: on | off | status
+---
+
+Manage YOLO on this machine only. It is off by default and does not sync to other machines.
+
+Describe YOLO as aggressive output reduction for repeated commands where the user accepts incomplete output and can check the result independently. It is unsuitable for debugging, diagnosis, exhaustive code review, audits, or any task that needs complete output. Do not promise faster coding, rendering, or builds. A preserved exit code alone does not verify the result.
+
+Without an explicit `on` or `off` request, show its state:
+
+```bash
+node "${CLAUDE_PLUGIN_ROOT}/dist/bin/tkr.js" yolo status
+```
+
+Enable YOLO only when the user explicitly requests YOLO activation. A general request to save tokens, compress more or choose the strongest standard level is not permission to enable YOLO. Before activation, explain: YOLO can discard content and diagnostics, including failed-command output. It creates no new restoration copies and leaves older caches unchanged. Turning it off cannot recover discarded details. Command exit codes are preserved. Then run:
+
+```bash
+node "${CLAUDE_PLUGIN_ROOT}/dist/bin/tkr.js" yolo on
+```
+
+Disable it immediately when requested:
+
+```bash
+node "${CLAUDE_PLUGIN_ROOT}/dist/bin/tkr.js" yolo off
+```
+
+These commands work offline and do not require an account. Print stdout and any warning from stderr. Prefer changing modes between commands. Wrapped terminal commands read the mode before execution; native tool responses use the mode active when their output is received. Output processing that already started in YOLO stays without restoration copies after it is turned off. Previous output caches stay unchanged. Turning YOLO off does not recover discarded content; rerun the original command with YOLO off if the full output is needed and rerunning is safe.
+
+To return to a standard level, use `tkr compression light`, `balanced` or `max`. Each disables YOLO locally before attempting to sync the standard level. If the sync fails, report that failure and the local YOLO state separately.
+
+If the command fails because the plugin files are missing, tokmizer was uninstalled. Offer to clean up: delete the tokmizer-managed `tkr*.md` copies (they end with a `managed-by: tokmizer` marker) from `~/.claude/commands/`.

package/command-templates/tkr.md

@@ -39,6 +39,6 @@ Valid keys:
 
 After any change, run `tkr settings show` and print the result.
 
-Changes sync to all the user's machines instantly.
+With a linked account, changes are saved to the account and other machines receive them during refresh. Without a linked account, changes are saved only on this machine and work offline.
 
 If the command fails because the plugin files are missing, tokmizer was uninstalled. Offer to clean up: delete the tokmizer-managed `tkr*.md` copies (they end with a `managed-by: tokmizer` marker) from `~/.claude/commands/`.