token-save-mcp 0.1.0__tar.gz

token_save_mcp-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,57 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e .
+
+      - name: Server tests (offline, no API calls)
+        run: python tests/test_server.py
+
+      - name: Ensure jq (the hook needs it)
+        run: |
+          command -v jq >/dev/null || {
+            if [ "$RUNNER_OS" = "macOS" ]; then brew install jq;
+            else sudo apt-get update && sudo apt-get install -y jq; fi
+          }
+
+      - name: Hook tests
+        run: bash tests/test_hook.sh
+
+      - name: CLI smoke test
+        run: |
+          token-save-mcp --help
+          token-save-mcp doctor --offline || true
+
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install build twine
+      - run: python -m build
+      - run: twine check dist/*

token_save_mcp-0.1.0/.github/workflows/publish.yml

@@ -0,0 +1,31 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write        # trusted publishing — no API token stored here
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Build
+        run: |
+          python -m pip install --upgrade pip build twine
+          python -m build
+          twine check dist/*
+
+      - name: Publish
+        uses: pypa/gh-action-pypi-publish@release/v1

token_save_mcp-0.1.0/.gitignore

@@ -0,0 +1,10 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+build/
+dist/
+.venv/
+venv/
+.env
+.pytest_cache/
+.ruff_cache/

token_save_mcp-0.1.0/CHANGELOG.md

@@ -0,0 +1,29 @@
+# Changelog
+
+## [0.1.0] — 2026-09-18
+
+First public release.
+
+### Added
+- `bulk_read` — delegate reading files to a worker model; only the answer
+  enters the agent's context.
+- `code_write` — generate code from a spec and a reference file, optionally
+  written straight to disk so the generated code never enters context.
+- `status` — print the live configuration and probe the worker.
+- PreToolUse hook that blocks `Read` on files over a line or byte threshold
+  and redirects to `bulk_read`. Opt-in, reversible, and available in a softer
+  `warn` mode that allows the read but reports what it cost.
+- `token-save-mcp init` — one-command setup, no hand-edited JSON.
+- `token-save-mcp doctor` — diagnoses config, dependencies, registration and
+  makes a live worker call.
+- Provider presets: Ollama Cloud, OpenRouter, DeepSeek, Groq, local runtimes.
+  Any OpenAI-compatible endpoint via `TOKENSAVE_BASE_URL`.
+- `token-save-mcp stats` — a local JSONL ledger of every call, with a
+  shareable badge. Never leaves the machine; opt out with `TOKENSAVE_NO_LEDGER`.
+- 95 offline tests plus 19 hook routing tests; the transport is stubbed, so the
+  suite costs nothing to run.
+
+### Notes
+- Works with both `mcp` 1.x (`FastMCP`) and 2.x (`MCPServer`).
+- A SOCKS proxy without `httpx[socks]` now produces an actionable message
+  instead of a stack trace from inside httpx.

token_save_mcp-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,27 @@
+# Contributing
+
+Bug reports and patches are welcome.
+
+## Running the tests
+
+```bash
+pip install -e .
+python tests/test_server.py    # 95 offline tests — no API calls
+bash tests/test_hook.sh        # 19 hook routing tests (needs jq)
+```
+
+The transport is stubbed, so the suite is free to run and safe in CI.
+
+## Ground rules
+
+- **A new behaviour needs a test.** The suite exists because several real
+  defects (a dangling-symlink write, a corpus that ate all memory, a hook that
+  emitted invalid JSON on odd filenames) were found by review and would
+  otherwise have come back.
+- **Don't pin a model roster.** Any model id the provider serves is valid; a
+  hardcoded list goes stale the moment a provider ships something new.
+- **Savings must stay measured.** The footer reports the provider's own `usage`
+  numbers. Don't replace them with estimates.
+- **The hook fails open.** If anything is uncertain — no jq, malformed input,
+  unreadable file — it allows the read. Blocking by accident is worse than
+  missing a saving.

token_save_mcp-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Habartru
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

token_save_mcp-0.1.0/PKG-INFO

@@ -0,0 +1,326 @@
+Metadata-Version: 2.5
+Name: token-save-mcp
+Version: 0.1.0
+Summary: MCP server that keeps large files out of your coding agent's context by delegating reads to a cheap worker model. Enforced by a hook, measured per call.
+Project-URL: Homepage, https://github.com/Habartru/token_save_mcp
+Project-URL: Issues, https://github.com/Habartru/token_save_mcp/issues
+License: MIT License
+        
+        Copyright (c) 2026 Habartru
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Keywords: ai-coding,claude,claude-code,context-window,cursor,developer-tools,llm,mcp,model-context-protocol,ollama,openrouter,token-optimization
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.10
+Requires-Dist: mcp>=1.2
+Requires-Dist: openai>=1.60
+Description-Content-Type: text/markdown
+
+# token-save-mcp
+
+**Your coding agent burns its context reading files. This stops it — and shows you the receipt.**
+
+[![CI](https://github.com/Habartru/token_save_mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/Habartru/token_save_mcp/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/token-save-mcp.svg)](https://pypi.org/project/token-save-mcp/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/Habartru/token_save_mcp/blob/main/LICENSE)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
+
+An MCP server that sends big files to a cheap worker model and returns only the
+answer. The file bytes are paid for once, in the worker's context — not
+permanently in your agent's.
+
+![token-save-mcp in action](https://raw.githubusercontent.com/Habartru/token_save_mcp/main/docs/demo.gif)
+
+The hook **blocks** the expensive read and redirects it. The answer comes back
+with the worker's **real token usage from the API response** — not an estimate,
+a receipt:
+
+```
+─────────────────────────────────────────────────────────────
+token-save: 1 file, 606 lines | direct read ≈7,042 tok →
+into context ≈234 tok (saved 6,808 · 97%)
+worker: glm-5.3-flash | 6,155 in / 278 out | 4.0s
+```
+
+---
+
+## Install
+
+```bash
+pip install token-save-mcp
+export OPENROUTER_API_KEY=...            # or OLLAMA_API_KEY, DEEPSEEK_API_KEY…
+token-save-mcp init --provider openrouter --hook
+```
+
+That's it — `init` registers the MCP server and installs the hook. No
+hand-edited JSON. Drop `--hook` if you want the tools without enforcement.
+
+Verify with `token-save-mcp doctor`, which checks the config and makes one
+live call to prove the worker answers:
+
+```
+✓ provider: openrouter -> https://openrouter.ai/api/v1
+✓ worker model: deepseek/deepseek-chat
+✓ MCP server registered and connected
+✓ enforcement hook installed
+✓ worker replied in 2.0s (21 in / 21 out)
+```
+
+<details>
+<summary>Providers</summary>
+
+Any OpenAI-compatible endpoint works. Presets:
+
+| `--provider` | Endpoint | Key variable |
+|---|---|---|
+| `ollama` *(default)* | ollama.com | `OLLAMA_API_KEY` |
+| `openrouter` | openrouter.ai | `OPENROUTER_API_KEY` |
+| `deepseek` | api.deepseek.com | `DEEPSEEK_API_KEY` |
+| `groq` | api.groq.com | `GROQ_API_KEY` |
+| `local` | localhost:11434 | *(none)* |
+
+Anything else: set `TOKENSAVE_BASE_URL` and `TOKENSAVE_API_KEY` directly.
+With `local` your code never leaves the machine.
+
+The transport is the plain OpenAI SDK pointed at a `base_url`, so any
+OpenAI-compatible endpoint works. The measurements below were taken against
+Ollama Cloud; the other presets are configured and exercised by the test suite
+but their numbers will differ with the model you pick.
+
+</details>
+
+---
+
+## The part nobody else does: enforcement
+
+Every token-saving tool has the same failure mode — **the agent forgets to use
+it**. A tool the model may ignore gets ignored, and your savings are whatever
+the model felt like that day.
+
+`token-save-mcp install-hook` registers a `PreToolUse` hook that **blocks**
+`Read` on files over the threshold and redirects the agent to `bulk_read`:
+
+```
+Read("src/server.py")
+→ BLOCKED: This file is 606 lines (threshold: 350).
+  Use bulk_read to delegate this read instead.
+  Need exact content to EDIT? Re-read with offset/limit — that passes through.
+```
+
+What still passes through, by design:
+
+- **Targeted reads** (`offset`/`limit`) — editing needs exact text
+- **Small files** — under the threshold, delegating costs more than it saves
+- **Binaries** and missing files — nothing to summarise
+
+**Not ready to be told no?** Install it in warn mode instead — the read goes
+through, but you see what it cost:
+
+```bash
+token-save-mcp install-hook --hook-mode warn
+```
+
+Enforcement is **opt-in** and reversible: `token-save-mcp uninstall-hook`.
+
+> The hook is Claude Code only. The `bulk_read` / `code_write` tools are plain
+> MCP and work in any client — Cursor, Cline, Windsurf, Codex — just without
+> the enforcement layer.
+
+---
+
+## Tools
+
+### `bulk_read(question, paths, model?, effort?)`
+
+Read files without pulling them into context.
+
+```
+bulk_read(
+  question="Which methods touch the database, and where is auth enforced?",
+  paths=["src/service.py", "src/handlers.py"]
+)
+```
+
+**Use for:** surveying unfamiliar code, "what does this do", tracing a flow
+across files, finding where something is handled.
+
+**Don't use for:** editing (you need exact text — use a targeted read),
+debugging that needs your own reasoning over raw code, or files under ~350
+lines where delegation overhead exceeds the saving. The tool tells you when
+you've crossed that line rather than silently burning a call.
+
+### `code_write(spec, reference, target?, model?, effort?)`
+
+Generate boilerplate matching an existing file's style. With `target`, the code
+is written **straight to disk** and only a confirmation returns — the generated
+code never enters your context at all.
+
+```
+code_write(
+  spec="pytest suite for clamp(value, lo, hi), covering both bounds and lo>hi",
+  reference=["tests/test_total.py"],
+  target="tests/test_clamp.py"
+)
+→ Wrote tests/test_clamp.py (32 lines). Not read into your context.
+```
+
+Never overwrites: the target is created with `O_EXCL`, which also refuses to
+follow a dangling symlink.
+
+### `status()`
+
+Prints the live configuration and makes one tiny call to prove the worker is
+actually reachable.
+
+### `token-save-mcp stats`
+
+Every call appends one line to a local ledger, so you can see what the tool has
+actually saved you. Example output after a few weeks of use:
+
+```
+$ token-save-mcp stats --badge
+
+  token-save-mcp — all time
+
+  148 calls · 71,204 lines of code read by a worker
+  context saved: 812,455 tokens (94%)
+  worker time:   612s total
+
+  Markdown badge:
+  ![token-save](https://img.shields.io/badge/context%20saved-812K%20tokens-brightgreen)
+```
+
+The ledger is a plain JSONL file in `~/.token-save/` and never leaves your
+machine. `--since 7` limits the window; `TOKENSAVE_NO_LEDGER=1` turns recording
+off entirely.
+
+---
+
+## Measured savings
+
+Real runs, not projections. Each number is the footer from an actual call:
+
+| What | Size | Direct read | Via token-save | Saved |
+|---|---|---|---|---|
+| This project's own server.py | 606 lines | ≈7,042 tok | ≈234 tok | **97%** |
+| A large TypeScript handler | 602 lines | ≈13,340 tok | ≈689 tok | **95%** |
+| Production Python service | 443 lines | ≈5,788 tok | ≈684 tok | **88%** |
+| 4 files across a codebase | 1,910 lines | ≈28,379 tok | ≈304 tok | **99%** |
+| Code generation to disk | 58 lines written | — | 0 tok | **100%** |
+
+**Method:** "direct read" is the file's own size at ~3.6 chars/token (source
+code is denser than prose); "via token-save" is the returned answer measured the
+same way. The worker's in/out numbers come from the provider's `usage` field.
+Reproduce any row by running the same call — the footer prints on every one.
+
+**Where it's weaker, honestly:** on a 281-line diff the saving was 67%, because
+a short input with a long answer is the worst case. The tool says so in its own
+output. Savings are best where the file is big and the question is narrow.
+
+---
+
+## How it compares
+
+Different tools solve "too many tokens" in genuinely different ways:
+
+| | Approach | Enforced? | Savings figure |
+|---|---|---|---|
+| **token-save-mcp** | LLM worker reads, returns an answer | **Yes** — hook blocks Read | Measured per call |
+| Static AST tools | Parse the tree, return exact symbols | No | Deterministic |
+| Other delegation MCPs | LLM worker, single provider | No | Usually estimated |
+
+**Static AST tools are better than this one** at "give me the exact body of
+`handleRequest`" — they're free, instant, and can't hallucinate. Reach for them
+for symbol lookup.
+
+This tool is for **semantic questions over large files** — "what does this
+service do", "where does auth happen", "which of these files handle retries" —
+where you want an answer, not an extract. That costs a worker call and a few
+seconds, and a worker can be wrong. Use both.
+
+---
+
+## Configuration
+
+| Variable | Default | Purpose |
+|---|---|---|
+| `TOKENSAVE_PROVIDER` | `ollama` | Preset: ollama, openrouter, deepseek, groq, local |
+| `TOKENSAVE_API_KEY` | — | Overrides the preset's key variable |
+| `TOKENSAVE_BASE_URL` | preset | Any OpenAI-compatible endpoint |
+| `TOKENSAVE_MODEL` | preset | Worker model id |
+| `TOKENSAVE_MIN_LINES` | `350` | Hook threshold, and the "too small" warning |
+| `TOKENSAVE_HOOK_MODE` | `block` | `warn` allows the read but flags the cost |
+| `TOKENSAVE_HOOK_MAX_BYTES` | `100000` | Also block on size — catches minified files |
+| `TOKENSAVE_MAX_CORPUS_BYTES` | `2000000` | Ceiling on one request |
+| `TOKENSAVE_TIMEOUT` | `600` | Seconds per call |
+| `TOKENSAVE_MAX_RETRIES` | `4` | Retries on transient failures |
+| `TOKENSAVE_MAX_CONCURRENCY` | `3` | Match your provider's limit |
+| `TOKENSAVE_LEDGER` | `~/.token-save/ledger.jsonl` | Where `stats` reads from |
+| `TOKENSAVE_NO_LEDGER` | unset | Set to disable local recording |
+
+---
+
+## When not to use this
+
+Being clear about this is the point, not a disclaimer:
+
+- **You need exact text to edit.** Use a targeted read. The hook lets those through.
+- **You're debugging subtle behaviour.** Summaries lose the detail that matters.
+- **The file is small.** Under ~350 lines, reading directly is cheaper and faster.
+- **The worker can be wrong.** It's an LLM. For anything you'll act on blindly,
+  verify against the source. Static tools don't have this failure mode.
+
+---
+
+## Development
+
+```bash
+git clone https://github.com/Habartru/token_save_mcp
+cd token_save_mcp
+pip install -e ".[dev]"
+
+python tests/test_server.py    # 83 offline tests, no API calls
+bash tests/test_hook.sh        # 19 hook routing tests
+```
+
+The test suite stubs the transport, so it costs nothing to run and is safe in
+CI. It covers the retry loop, corpus assembly, fence stripping, the disk-write
+guards, and every hook routing decision.
+
+---
+
+## Credits
+
+The delegation-plus-hook pattern is adapted from the `shunt` plugin in
+[spotify/portal-ai-plugins](https://github.com/spotify/portal-ai-plugins)
+(Apache-2.0), which routes the same kind of work through Spotify's internal
+Portal CLI. This project keeps the idea and swaps the transport for any
+OpenAI-compatible provider, so no corporate Portal instance is required. Files
+also travel in-process rather than through `argv`, which removes the 128 KiB
+per-argument limit on Linux.
+
+MIT licensed.