kvleak 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,22 @@
1
+ name: ci
2
+ on:
3
+ push:
4
+ branches: [main]
5
+ pull_request:
6
+ jobs:
7
+ test:
8
+ runs-on: ${{ matrix.os }}
9
+ strategy:
10
+ fail-fast: false
11
+ matrix:
12
+ os: [ubuntu-latest, macos-latest, windows-latest]
13
+ python: ["3.9", "3.11", "3.13"]
14
+ steps:
15
+ - uses: actions/checkout@v4
16
+ - uses: actions/setup-python@v5
17
+ with:
18
+ python-version: ${{ matrix.python }}
19
+ - run: pip install -e ".[dev]"
20
+ - run: ruff check src tests
21
+ - run: pytest -q
22
+ - run: kvleak breakeven 150k --no-color
@@ -0,0 +1,17 @@
1
+ name: release
2
+ on:
3
+ push:
4
+ tags: ["v*"]
5
+ jobs:
6
+ pypi:
7
+ runs-on: ubuntu-latest
8
+ environment: pypi
9
+ permissions:
10
+ id-token: write # trusted publishing, no API token needed
11
+ steps:
12
+ - uses: actions/checkout@v4
13
+ - uses: actions/setup-python@v5
14
+ with:
15
+ python-version: "3.12"
16
+ - run: pip install build && python -m build
17
+ - uses: pypa/gh-action-pypi-publish@release/v1
@@ -0,0 +1,9 @@
1
+ .venv/
2
+ __pycache__/
3
+ *.egg-info/
4
+ dist/
5
+ build/
6
+ .pytest_cache/
7
+ .ruff_cache/
8
+ *.html
9
+ !docs/**/*.html
@@ -0,0 +1,8 @@
1
+ # Changelog
2
+
3
+ ## 0.1.0 (2026-10-19)
4
+
5
+ - First release: `audit`, `session`, `breakeven`, `statusline`, `pointer`, `models`.
6
+ - Claude Code reader (incl. subagents, compaction markers, 5m/1h TTL breakdown) and generic JSONL reader
7
+ (Anthropic / OpenAI / Gemini / flat agent-framework usage keys).
8
+ - Terminal, JSON, Markdown and self-contained HTML reports; `--fail-over` CI gate.
@@ -0,0 +1,31 @@
1
+ # Contributing
2
+
3
+ Thanks for helping! kvleak stays **zero-dependency** and **local-only**; please keep it that way.
4
+
5
+ ## Dev setup
6
+
7
+ ```bash
8
+ git clone https://github.com/shangliy/kvleak && cd kvleak
9
+ python -m venv .venv && . .venv/bin/activate
10
+ pip install -e ".[dev]"
11
+ pytest -q && ruff check src tests
12
+ python examples/make_demo.py demo && kvleak --path demo
13
+ ```
14
+
15
+ ## Adding a log format
16
+
17
+ 1. Write `read_<harness>(path) -> Transcript` in `src/kvleak/sources.py`. It must yield one
18
+ `Step` per *billed* model call, deduped, in chronological order.
19
+ 2. Teach `detect_format` to recognise it from the first few lines.
20
+ 3. Add a small fixture-based test in `tests/`.
21
+
22
+ ## Adding prices
23
+
24
+ Add a `Price(...)` row to `BUILTIN` in `src/kvleak/pricing.py` and cite the provider's public
25
+ pricing page in the PR. Keys match model ids by substring, and the longest match wins.
26
+
27
+ ## Good first issues
28
+
29
+ - Readers for other agent CLIs (Codex CLI, Gemini CLI, Aider, OpenHands, Cline, ...)
30
+ - A `--watch` mode that re-renders the audit every N seconds
31
+ - Per-project rollups in the HTML report
kvleak-0.1.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 kvleak contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
kvleak-0.1.0/PKG-INFO ADDED
@@ -0,0 +1,219 @@
1
+ Metadata-Version: 2.5
2
+ Name: kvleak
3
+ Version: 0.1.0
4
+ Summary: Find where your AI coding agent leaks prompt-cache money (Claude Code, multi-agent, any JSONL) and how to stop it.
5
+ Project-URL: Homepage, https://github.com/shangliy/kvleak
6
+ Project-URL: Issues, https://github.com/shangliy/kvleak/issues
7
+ Author: kvleak contributors
8
+ License-Expression: MIT
9
+ License-File: LICENSE
10
+ Keywords: claude-code,cost,kv-cache,llm,multi-agent,observability,prompt-caching,tokens
11
+ Classifier: Development Status :: 4 - Beta
12
+ Classifier: Environment :: Console
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: License :: OSI Approved :: MIT License
15
+ Classifier: Programming Language :: Python :: 3
16
+ Classifier: Topic :: Software Development :: Libraries
17
+ Classifier: Topic :: Utilities
18
+ Requires-Python: >=3.9
19
+ Provides-Extra: dev
20
+ Requires-Dist: pytest>=7; extra == 'dev'
21
+ Requires-Dist: ruff>=0.5; extra == 'dev'
22
+ Description-Content-Type: text/markdown
23
+
24
+ # kvleak
25
+
26
+ **Find out where your AI coding agent leaks prompt-cache money, and how to stop it.** · [中文](README.zh-CN.md)
27
+
28
+ [![CI](https://github.com/shangliy/kvleak/actions/workflows/ci.yml/badge.svg)](https://github.com/shangliy/kvleak/actions/workflows/ci.yml)
29
+ [![PyPI](https://img.shields.io/pypi/v/kvleak)](https://pypi.org/project/kvleak/)
30
+ ![Python](https://img.shields.io/badge/python-3.9%2B-blue)
31
+ ![Dependencies](https://img.shields.io/badge/dependencies-0-brightgreen)
32
+ ![License](https://img.shields.io/badge/license-MIT-green)
33
+
34
+ > **A 99.0% cache hit rate, and the session still leaked $82.** On Claude Fable 5.1 one
35
+ > cache miss costs as much as 80 hits. In a 30-day audit of real Claude Code use, 2% of calls
36
+ > carried more than half of all uncached input. [See the numbers →](docs/benchmarks.md)
37
+
38
+ Your agent probably reports a 90%+ cache hit rate, and you are still overpaying.
39
+ A cache miss costs about **12× more than a hit**, and the misses are rare but huge.
40
+ They come from idle gaps that outlive the 5-minute TTL, from model switches, and
41
+ from subagents that start cold. In real audits, **about 5% of calls burn about
42
+ half of all uncached input**.
43
+
44
+ `kvleak` reads the transcripts that are already on your disk, finds every
45
+ expensive miss, says *why* it happened, and puts a dollar figure on it.
46
+
47
+ ```bash
48
+ uvx kvleak # or: pipx run kvleak · pip install kvleak
49
+ ```
50
+
51
+ No API key, no config, no network calls, no dependencies. It reads
52
+ `~/.claude/projects` by default and any JSONL with token usage via `--path`.
53
+
54
+ ```text
55
+ kvleak · prompt-cache leak report
56
+ 14 sessions + 18 subagents · 1,594 model calls · 10.0k min miss size
57
+
58
+ Cache hit rate 92.1% ██████████████████████
59
+ Input spend $145.09 caching already saved you $791.64
60
+ Leaked to misses $69.54 47.9% of input spend went to cache rebuilds and cold starts
61
+ Recoverable $27.00 with break-even-capped keep-alive pings alone
62
+
63
+ Where the leak comes from
64
+ cause misses uncached leak share
65
+ Idle gap outlived the cache TTL 48 10.0M $38.40 55.2% ████████▊
66
+ Model switched mid-session 21 5.2M $18.34 26.4% ████▏
67
+ Prefix changed within the TTL 16 2.6M $11.14 16.0% ██▌
68
+ New session cold start 14 336k $1.61 2.3% ▎
69
+ Subagent cold start 18 468k $0.054 0.1%
70
+
71
+ What to do about it
72
+ 1. Idle gaps are your biggest leak class. Reply before the TTL ring empties, ...
73
+ 2. Switching models throws away the whole cache (KV state is per model). ...
74
+ ```
75
+
76
+ <sub>Output from the bundled synthetic demo: `python examples/make_demo.py demo && kvleak --path demo`</sub>
77
+
78
+ ---
79
+
80
+ ## Real numbers
81
+
82
+ | | |
83
+ |---|---|
84
+ | 30 days of interactive Claude Code sessions | **97.5%** hit rate, but **27.5%** of input spend went to cache rebuilds |
85
+ | Share of that leak caused by idle gaps | **82%** (median gap 2.8 h; Claude Code now uses the 1 h TTL) |
86
+ | Cost of one miss on a 200k context | **$3.95** on Fable 5.1, $1.56 on Opus 5.5 (1 h TTL) |
87
+ | 6-worker swarm, pointers + warm workers + pinned models | **−86%** input cost in simulation |
88
+
89
+ Methodology and reproduction commands: [docs/benchmarks.md](docs/benchmarks.md).
90
+
91
+ <p align="center"><img src="https://raw.githubusercontent.com/shangliy/kvleak/main/docs/assets/report.png" width="720" alt="kvleak HTML report"></p>
92
+
93
+ ## What you get
94
+
95
+ | Command | What it does |
96
+ |---|---|
97
+ | `kvleak` | Audits every session: hit rate, dollars leaked, leak broken down by cause, leakiest sessions, worst single misses, and concrete next steps. |
98
+ | `kvleak --since 7d --html report.html` | A self-contained HTML report (light/dark, no JS) you can share with your team. `--markdown` and `--json` also work. |
99
+ | `kvleak session <id>` | Call-by-call timeline of one session, with each miss flagged and explained. |
100
+ | `kvleak statusline --install` | Adds a live **cache TTL countdown** to Claude Code: `◕ cache 3:41 · 182k ctx · rebuild $0.89`. |
101
+ | `kvleak breakeven 150k --idle 20m` | Tells you whether to keep this cache alive or let it die. |
102
+ | `kvleak pointer build.log --status FAIL` | Turns a huge file into a ~60-token hand-off message for multi-agent boards. |
103
+ | `kvleak --fail-over 15` | Exits with code 2 when more than 15% of input spend leaked, so it can gate CI or nightly jobs. |
104
+
105
+ ### Every miss gets a cause
106
+
107
+ | Cause | What happened | Typical fix |
108
+ |---|---|---|
109
+ | **Idle gap outlived the TTL** | You were away for more than 5 min (or 1 h), so the next message rebuilt everything. | Watch the statusline countdown, batch work, or use capped keep-alive pings. |
110
+ | **Prefix changed within the TTL** | Something near the top of the prompt changed: a timestamp, unsorted JSON, or a different tool list. | Make the prefix deterministic. |
111
+ | **Model switched mid-session** | KV caches are per model, so a switch always means a full-price rebuild. | Pin one model per session or worker and hand off with pointers. |
112
+ | **History compacted** | `/compact` or auto-compact rewrote the history. | Compact once at a natural break, not mid-task. |
113
+ | **Cold start / subagent cold start** | A fresh agent paid full price for its system prompt and tool schemas. | Reuse warm workers; keep system prompts identical. |
114
+
115
+ Only the **avoidable** part counts as leak: the context the cache already held,
116
+ priced at write cost minus read cost. A new user message or new tool output
117
+ would be paid for anyway, so it is not counted. Details:
118
+ [docs/how-it-works.md](docs/how-it-works.md).
119
+
120
+ ## Live TTL countdown in Claude Code
121
+
122
+ ```bash
123
+ kvleak statusline --install # backs up settings.json; refuses to overwrite an existing statusLine without --force
124
+ ```
125
+
126
+ ```text
127
+ ◕ cache 3:41 · 182k ctx · rebuild $0.89 (green → yellow → red as the TTL drains)
128
+ ○ cache cold · next msg rebuilds 182k ($0.89)
129
+ ```
130
+
131
+ When the ring is almost empty, sending your next message *now* refreshes the
132
+ TTL at the cost of a cache read. If you already have a statusline script, call
133
+ `kvleak statusline` from it; it reads the same JSON on stdin.
134
+
135
+ ## Should I keep this cache alive?
136
+
137
+ ```text
138
+ $ kvleak breakeven 100k --model claude-sonnet-5 --idle 20m
139
+
140
+ Cached prefix 100k on claude-sonnet-5 (5m TTL)
141
+ Rebuild if it expires 125k tok-equiv ($0.250)
142
+ ...of which avoidable 115k tok-equiv ($0.230)
143
+ One keep-alive ping 10.0k tok-equiv ($0.020)
144
+ Break-even 11 pings ≈ 54m30s of idle time
145
+
146
+ Idle 20m00s: 4 pings cost 40.0k tok-equiv ($0.080), saving 75.0k tok-equiv ($0.150). Keep it alive.
147
+ ```
148
+
149
+ The rule is `K_max = floor((write − read) / read)`: 11 pings at 0.10× reads,
150
+ 24 at 0.05×, 49 at 0.025×. Past that, let the cache expire.
151
+
152
+ ## Running multi-agent swarms?
153
+
154
+ Multi-agent and multi-model setups multiply leaks in three ways:
155
+
156
+ * caches never cross models
157
+ * shared boards make every agent re-read everyone's logs, which costs O(N²)
158
+ * one-shot subagents each pay a cold-start tax
159
+
160
+ **[The multi-agent cache playbook](docs/playbook.md)** describes six layers of
161
+ practice: pinned models with pointer hand-offs, a script → small model → large
162
+ model funnel, warm workers with TTL batching, a slim board contract,
163
+ deterministic prefixes, and deliberate compaction. It includes a contract you
164
+ can paste straight into your swarm's instructions, and `kvleak` measures each
165
+ layer.
166
+
167
+ ```bash
168
+ $ kvleak pointer /tmp/build.log --status FAIL --note "3 tests failing in //pkg"
169
+ {"bytes":40211,"lines":512,"note":"3 tests failing in //pkg","path":"/tmp/build.log","sha256":"9f2c1e07a1b4c3d2","status":"FAIL","tail":[...]}
170
+ ```
171
+
172
+ ## Supported logs
173
+
174
+ | Source | How |
175
+ |---|---|
176
+ | **Claude Code** (incl. subagents) | automatic: `~/.claude/projects`, respects `$CLAUDE_CONFIG_DIR` |
177
+ | **Any JSONL with usage counters** | `--path <dir-or-file>`. Auto-detects Anthropic (`cache_read_input_tokens`), OpenAI (`prompt_tokens_details.cached_tokens`), Gemini (`cachedContentTokenCount`) and flat agent-framework keys (`input_tokens`, `cache_read_tokens`, `created_at`). |
178
+
179
+ Prices for current Claude models are built in (`kvleak models`). For other
180
+ providers, add a pricing file; see [`examples/pricing.json`](examples/pricing.json).
181
+ Models without a price still get full token-level analysis.
182
+
183
+ Want your agent harness supported natively? A reader is one small function
184
+ that yields `Step`s. See `src/kvleak/sources.py` and [CONTRIBUTING.md](CONTRIBUTING.md).
185
+
186
+ ## Use it as a library
187
+
188
+ ```python
189
+ from kvleak import analyze, load
190
+
191
+ report = analyze([load("session.jsonl")])
192
+ print(report.hit_rate, report.leak_usd, report.buckets["ttl_expiry"])
193
+ ```
194
+
195
+ ## FAQ
196
+
197
+ **Is my data sent anywhere?** No. kvleak only reads local files and makes no
198
+ network calls.
199
+
200
+ **I'm on a subscription, not the API. Does this matter?** Yes. Cache misses
201
+ use up your rate-limit headroom the same way they would cost money on the API.
202
+ Read the dollar figures as relative weights.
203
+
204
+ **My hit rate is 95%. Am I fine?** Maybe not. Hit rate counts tokens, not
205
+ dollars, and one 800k-token rebuild weighs as much as thousands of cheap hits.
206
+ Look at the *Leaked* line instead.
207
+
208
+ **How accurate is the cause detection?** It follows the rules in
209
+ [docs/how-it-works.md](docs/how-it-works.md). The main blind spot is very long
210
+ generations: timestamps are written when a response ends, so a few TTL
211
+ expiries can show up as prefix busts.
212
+
213
+ ## Credits
214
+
215
+ Inspired by the cache-countdown idea in
216
+ [olddonkey/cache-refresher](https://github.com/olddonkey/cache-refresher) and by
217
+ production audits of multi-agent swarms.
218
+
219
+ MIT licensed. Contributions welcome!
kvleak-0.1.0/README.md ADDED
@@ -0,0 +1,196 @@
1
+ # kvleak
2
+
3
+ **Find out where your AI coding agent leaks prompt-cache money, and how to stop it.** · [中文](README.zh-CN.md)
4
+
5
+ [![CI](https://github.com/shangliy/kvleak/actions/workflows/ci.yml/badge.svg)](https://github.com/shangliy/kvleak/actions/workflows/ci.yml)
6
+ [![PyPI](https://img.shields.io/pypi/v/kvleak)](https://pypi.org/project/kvleak/)
7
+ ![Python](https://img.shields.io/badge/python-3.9%2B-blue)
8
+ ![Dependencies](https://img.shields.io/badge/dependencies-0-brightgreen)
9
+ ![License](https://img.shields.io/badge/license-MIT-green)
10
+
11
+ > **A 99.0% cache hit rate, and the session still leaked $82.** On Claude Fable 5.1 one
12
+ > cache miss costs as much as 80 hits. In a 30-day audit of real Claude Code use, 2% of calls
13
+ > carried more than half of all uncached input. [See the numbers →](docs/benchmarks.md)
14
+
15
+ Your agent probably reports a 90%+ cache hit rate, and you are still overpaying.
16
+ A cache miss costs about **12× more than a hit**, and the misses are rare but huge.
17
+ They come from idle gaps that outlive the 5-minute TTL, from model switches, and
18
+ from subagents that start cold. In real audits, **about 5% of calls burn about
19
+ half of all uncached input**.
20
+
21
+ `kvleak` reads the transcripts that are already on your disk, finds every
22
+ expensive miss, says *why* it happened, and puts a dollar figure on it.
23
+
24
+ ```bash
25
+ uvx kvleak # or: pipx run kvleak · pip install kvleak
26
+ ```
27
+
28
+ No API key, no config, no network calls, no dependencies. It reads
29
+ `~/.claude/projects` by default and any JSONL with token usage via `--path`.
30
+
31
+ ```text
32
+ kvleak · prompt-cache leak report
33
+ 14 sessions + 18 subagents · 1,594 model calls · 10.0k min miss size
34
+
35
+ Cache hit rate 92.1% ██████████████████████
36
+ Input spend $145.09 caching already saved you $791.64
37
+ Leaked to misses $69.54 47.9% of input spend went to cache rebuilds and cold starts
38
+ Recoverable $27.00 with break-even-capped keep-alive pings alone
39
+
40
+ Where the leak comes from
41
+ cause misses uncached leak share
42
+ Idle gap outlived the cache TTL 48 10.0M $38.40 55.2% ████████▊
43
+ Model switched mid-session 21 5.2M $18.34 26.4% ████▏
44
+ Prefix changed within the TTL 16 2.6M $11.14 16.0% ██▌
45
+ New session cold start 14 336k $1.61 2.3% ▎
46
+ Subagent cold start 18 468k $0.054 0.1%
47
+
48
+ What to do about it
49
+ 1. Idle gaps are your biggest leak class. Reply before the TTL ring empties, ...
50
+ 2. Switching models throws away the whole cache (KV state is per model). ...
51
+ ```
52
+
53
+ <sub>Output from the bundled synthetic demo: `python examples/make_demo.py demo && kvleak --path demo`</sub>
54
+
55
+ ---
56
+
57
+ ## Real numbers
58
+
59
+ | | |
60
+ |---|---|
61
+ | 30 days of interactive Claude Code sessions | **97.5%** hit rate, but **27.5%** of input spend went to cache rebuilds |
62
+ | Share of that leak caused by idle gaps | **82%** (median gap 2.8 h; Claude Code now uses the 1 h TTL) |
63
+ | Cost of one miss on a 200k context | **$3.95** on Fable 5.1, $1.56 on Opus 5.5 (1 h TTL) |
64
+ | 6-worker swarm, pointers + warm workers + pinned models | **−86%** input cost in simulation |
65
+
66
+ Methodology and reproduction commands: [docs/benchmarks.md](docs/benchmarks.md).
67
+
68
+ <p align="center"><img src="https://raw.githubusercontent.com/shangliy/kvleak/main/docs/assets/report.png" width="720" alt="kvleak HTML report"></p>
69
+
70
+ ## What you get
71
+
72
+ | Command | What it does |
73
+ |---|---|
74
+ | `kvleak` | Audits every session: hit rate, dollars leaked, leak broken down by cause, leakiest sessions, worst single misses, and concrete next steps. |
75
+ | `kvleak --since 7d --html report.html` | A self-contained HTML report (light/dark, no JS) you can share with your team. `--markdown` and `--json` also work. |
76
+ | `kvleak session <id>` | Call-by-call timeline of one session, with each miss flagged and explained. |
77
+ | `kvleak statusline --install` | Adds a live **cache TTL countdown** to Claude Code: `◕ cache 3:41 · 182k ctx · rebuild $0.89`. |
78
+ | `kvleak breakeven 150k --idle 20m` | Tells you whether to keep this cache alive or let it die. |
79
+ | `kvleak pointer build.log --status FAIL` | Turns a huge file into a ~60-token hand-off message for multi-agent boards. |
80
+ | `kvleak --fail-over 15` | Exits with code 2 when more than 15% of input spend leaked, so it can gate CI or nightly jobs. |
81
+
82
+ ### Every miss gets a cause
83
+
84
+ | Cause | What happened | Typical fix |
85
+ |---|---|---|
86
+ | **Idle gap outlived the TTL** | You were away for more than 5 min (or 1 h), so the next message rebuilt everything. | Watch the statusline countdown, batch work, or use capped keep-alive pings. |
87
+ | **Prefix changed within the TTL** | Something near the top of the prompt changed: a timestamp, unsorted JSON, or a different tool list. | Make the prefix deterministic. |
88
+ | **Model switched mid-session** | KV caches are per model, so a switch always means a full-price rebuild. | Pin one model per session or worker and hand off with pointers. |
89
+ | **History compacted** | `/compact` or auto-compact rewrote the history. | Compact once at a natural break, not mid-task. |
90
+ | **Cold start / subagent cold start** | A fresh agent paid full price for its system prompt and tool schemas. | Reuse warm workers; keep system prompts identical. |
91
+
92
+ Only the **avoidable** part counts as leak: the context the cache already held,
93
+ priced at write cost minus read cost. A new user message or new tool output
94
+ would be paid for anyway, so it is not counted. Details:
95
+ [docs/how-it-works.md](docs/how-it-works.md).
96
+
97
+ ## Live TTL countdown in Claude Code
98
+
99
+ ```bash
100
+ kvleak statusline --install # backs up settings.json; refuses to overwrite an existing statusLine without --force
101
+ ```
102
+
103
+ ```text
104
+ ◕ cache 3:41 · 182k ctx · rebuild $0.89 (green → yellow → red as the TTL drains)
105
+ ○ cache cold · next msg rebuilds 182k ($0.89)
106
+ ```
107
+
108
+ When the ring is almost empty, sending your next message *now* refreshes the
109
+ TTL at the cost of a cache read. If you already have a statusline script, call
110
+ `kvleak statusline` from it; it reads the same JSON on stdin.
111
+
112
+ ## Should I keep this cache alive?
113
+
114
+ ```text
115
+ $ kvleak breakeven 100k --model claude-sonnet-5 --idle 20m
116
+
117
+ Cached prefix 100k on claude-sonnet-5 (5m TTL)
118
+ Rebuild if it expires 125k tok-equiv ($0.250)
119
+ ...of which avoidable 115k tok-equiv ($0.230)
120
+ One keep-alive ping 10.0k tok-equiv ($0.020)
121
+ Break-even 11 pings ≈ 54m30s of idle time
122
+
123
+ Idle 20m00s: 4 pings cost 40.0k tok-equiv ($0.080), saving 75.0k tok-equiv ($0.150). Keep it alive.
124
+ ```
125
+
126
+ The rule is `K_max = floor((write − read) / read)`: 11 pings at 0.10× reads,
127
+ 24 at 0.05×, 49 at 0.025×. Past that, let the cache expire.
128
+
129
+ ## Running multi-agent swarms?
130
+
131
+ Multi-agent and multi-model setups multiply leaks in three ways:
132
+
133
+ * caches never cross models
134
+ * shared boards make every agent re-read everyone's logs, which costs O(N²)
135
+ * one-shot subagents each pay a cold-start tax
136
+
137
+ **[The multi-agent cache playbook](docs/playbook.md)** describes six layers of
138
+ practice: pinned models with pointer hand-offs, a script → small model → large
139
+ model funnel, warm workers with TTL batching, a slim board contract,
140
+ deterministic prefixes, and deliberate compaction. It includes a contract you
141
+ can paste straight into your swarm's instructions, and `kvleak` measures each
142
+ layer.
143
+
144
+ ```bash
145
+ $ kvleak pointer /tmp/build.log --status FAIL --note "3 tests failing in //pkg"
146
+ {"bytes":40211,"lines":512,"note":"3 tests failing in //pkg","path":"/tmp/build.log","sha256":"9f2c1e07a1b4c3d2","status":"FAIL","tail":[...]}
147
+ ```
148
+
149
+ ## Supported logs
150
+
151
+ | Source | How |
152
+ |---|---|
153
+ | **Claude Code** (incl. subagents) | automatic: `~/.claude/projects`, respects `$CLAUDE_CONFIG_DIR` |
154
+ | **Any JSONL with usage counters** | `--path <dir-or-file>`. Auto-detects Anthropic (`cache_read_input_tokens`), OpenAI (`prompt_tokens_details.cached_tokens`), Gemini (`cachedContentTokenCount`) and flat agent-framework keys (`input_tokens`, `cache_read_tokens`, `created_at`). |
155
+
156
+ Prices for current Claude models are built in (`kvleak models`). For other
157
+ providers, add a pricing file; see [`examples/pricing.json`](examples/pricing.json).
158
+ Models without a price still get full token-level analysis.
159
+
160
+ Want your agent harness supported natively? A reader is one small function
161
+ that yields `Step`s. See `src/kvleak/sources.py` and [CONTRIBUTING.md](CONTRIBUTING.md).
162
+
163
+ ## Use it as a library
164
+
165
+ ```python
166
+ from kvleak import analyze, load
167
+
168
+ report = analyze([load("session.jsonl")])
169
+ print(report.hit_rate, report.leak_usd, report.buckets["ttl_expiry"])
170
+ ```
171
+
172
+ ## FAQ
173
+
174
+ **Is my data sent anywhere?** No. kvleak only reads local files and makes no
175
+ network calls.
176
+
177
+ **I'm on a subscription, not the API. Does this matter?** Yes. Cache misses
178
+ use up your rate-limit headroom the same way they would cost money on the API.
179
+ Read the dollar figures as relative weights.
180
+
181
+ **My hit rate is 95%. Am I fine?** Maybe not. Hit rate counts tokens, not
182
+ dollars, and one 800k-token rebuild weighs as much as thousands of cheap hits.
183
+ Look at the *Leaked* line instead.
184
+
185
+ **How accurate is the cause detection?** It follows the rules in
186
+ [docs/how-it-works.md](docs/how-it-works.md). The main blind spot is very long
187
+ generations: timestamps are written when a response ends, so a few TTL
188
+ expiries can show up as prefix busts.
189
+
190
+ ## Credits
191
+
192
+ Inspired by the cache-countdown idea in
193
+ [olddonkey/cache-refresher](https://github.com/olddonkey/cache-refresher) and by
194
+ production audits of multi-agent swarms.
195
+
196
+ MIT licensed. Contributions welcome!
@@ -0,0 +1,50 @@
1
+ # kvleak
2
+
3
+ **找出你的 AI 编程 Agent 在 Prompt Cache 上漏掉了多少钱,以及怎么堵上。** · [English](README.md)
4
+
5
+ 表面缓存命中率 90%+,账单却依然很贵?因为缓存**未命中比命中贵约 12 倍**,而且未命中虽然少、单次却很大:空闲超过 5 分钟 TTL、会话中途切模型、一次性子 Agent 冷启动……真实审计中,**约 5% 的调用吃掉了约一半的全价输入 Token**。
6
+
7
+ > **缓存命中率 99.0% 的会话,照样漏了 82 美元。** 在 Claude Fable 5.1 上,一次缓存未命中的成本相当于 80 次命中。一份 30 天的真实 Claude Code 使用审计显示:2% 的调用占了一半以上的全价输入。[查看数据 →](docs/benchmarks.md)
8
+
9
+ `kvleak` 直接读取本机已有的会话记录,找出每一次昂贵的缓存击穿,告诉你**为什么**发生,并折算成美元。
10
+
11
+ ```bash
12
+ uvx kvleak # 或:pipx run kvleak · pip install kvleak
13
+ ```
14
+
15
+ 零依赖、零配置、不需要 API Key、不联网。默认读取 `~/.claude/projects`,其他带 token usage 的 JSONL 用 `--path` 指定。
16
+
17
+ ## 真实数据
18
+
19
+ - 30 天交互式 Claude Code 会话:命中率 **97.5%**,但 **27.5%** 的输入花费用在了缓存重建上
20
+ - 其中 **82%** 的漏损来自空闲间隔(中位数 2.8 小时;Claude Code 现在默认使用 1 小时 TTL)
21
+ - 200k 上下文一次未命中的成本:Fable 5.1 上 **$3.95**,Opus 5.5 上 $1.56
22
+ - 6 个 Worker 的 Swarm 模拟:指针交接 + 常驻 Worker + 锁定模型,输入成本 **−86%**
23
+
24
+ ## 功能
25
+
26
+ | 命令 | 作用 |
27
+ |---|---|
28
+ | `kvleak` | 全量审计:命中率、漏损金额、按原因拆解、最漏的会话、最贵的单次未命中、可执行建议 |
29
+ | `kvleak --since 7d --html report.html` | 生成可分享的自包含 HTML 报告(亦支持 `--markdown` / `--json`) |
30
+ | `kvleak session <id>` | 单个会话逐步时间线,标出每一次未命中及原因 |
31
+ | `kvleak statusline --install` | 在 Claude Code 状态栏显示**缓存 TTL 倒计时**:`◕ cache 3:41 · 182k ctx · rebuild $0.89` |
32
+ | `kvleak breakeven 150k --idle 20m` | 保活还是放弃?按盈亏平衡点 `K_max = floor((write − read) / read)` 计算 |
33
+ | `kvleak pointer build.log --status FAIL` | 把大文件变成约 60 Token 的「指针消息」,用于多 Agent 看板交接 |
34
+ | `kvleak --fail-over 15` | 漏损超过输入花费的 15% 时以退出码 2 结束,可作为 CI 门禁 |
35
+
36
+ ## 未命中原因分类
37
+
38
+ | 原因 | 含义 | 修复方向 |
39
+ |---|---|---|
40
+ | 空闲超过 TTL | 离开超过 5 分钟(或 1 小时),下一条消息全量重建 | 看倒计时及时回复、批量处理、按盈亏平衡点限制保活次数 |
41
+ | TTL 内前缀变化 | 时间戳、未排序 JSON、工具列表变化导致前缀击穿 | 前缀确定性:`json.dumps(..., sort_keys=True)`、静态 System Prompt |
42
+ | 会话中途切模型 | KV Cache 按模型物理隔离,切换必然全价重算 | 每个 Worker 锁定单一模型,跨模型用文件指针交接 |
43
+ | 历史压缩 | `/compact` 或自动压缩重写历史 | 在自然断点主动压缩一次 |
44
+ | 冷启动 / 子 Agent 冷启动 | 新 Agent 全价加载系统提示词与工具 Schema | 复用常驻 Worker,保持 System Prompt 完全一致 |
45
+
46
+ ## 多 Agent / 多模型
47
+
48
+ [多 Agent 缓存手册(英文)](docs/playbook.md) 介绍六层优化:单 Worker 锁定模型并用指针交接、「脚本 → 小模型 → 大模型」算力漏斗、常驻 Worker 与 TTL 窗口批处理、公共看板瘦身契约(15 行以内)、确定性前缀、按水位线主动压缩。手册里附有一段可以直接粘贴进 Swarm 指令的契约。
49
+
50
+ MIT 协议,欢迎贡献!
Binary file
@@ -0,0 +1,85 @@
1
+ # Numbers
2
+
3
+ Three sets of numbers, each labelled by where it comes from. Re-run all of them
4
+ yourself; every command is on this page.
5
+
6
+ ## 1. Case study: 30 days of one heavy Claude Code user
7
+
8
+ The maintainer's own `~/.claude/projects`, from 2026-09-21 to 2026-10-20,
9
+ audited with `kvleak`. Interactive sessions are those with 5 or more calls,
10
+ plus their subagents. Headless one-shot `claude -p` runs are listed separately
11
+ because they behave differently.
12
+
13
+ | | interactive sessions | headless one-shot runs |
14
+ |---|---:|---:|
15
+ | transcripts | 289 (257 of them subagents) | 20,530 |
16
+ | billed model calls | 13,110 | 20,592 |
17
+ | cache hit rate (tokens) | **97.5%** | 59.4% |
18
+ | input spend at list price | $1,522 | $468 |
19
+ | what that input would cost with no caching | $18,085 | $548 |
20
+ | **leaked to cache rebuilds** | **$418 (27.5% of input spend)** | $65 (13.8%) |
21
+ | heavy misses as a share of calls | **2.2%** | 4.4% |
22
+ | uncached input inside those misses | **53.7%** | 11.0% |
23
+
24
+ Where the interactive leak came from:
25
+
26
+ | cause | misses | leak | share |
27
+ |---|---:|---:|---:|
28
+ | Idle gap outlived the cache TTL | 124 | $345 | **82.4%** |
29
+ | Subagent cold start | 127 | $27 | 6.5% |
30
+ | Prefix changed within the TTL | 17 | $20 | 4.8% |
31
+ | Model switched mid-session | 7 | $20 | 4.7% |
32
+ | History compacted | 8 | $4 | 0.9% |
33
+ | New session cold start | 4 | $3 | 0.7% |
34
+
35
+ Takeaways:
36
+
37
+ * **A high hit rate hides the leak.** The leakiest session had a **99.0%**
38
+ token hit rate and still leaked **$82**, because one rebuild of 800k tokens
39
+ outweighs thousands of cheap hits.
40
+ * **About 2% of calls carry more than half of all uncached input.**
41
+ * **Claude Code now writes about 90% of its cache with the 1-hour TTL**, so the
42
+ 5-minute expiry is no longer the problem. The leak is the lunch, meeting and
43
+ overnight gaps: the median idle gap before a TTL miss was 2.8 h. 45 of the
44
+ 124 expiries were under 2 h and worth bridging with a keep-alive, which would
45
+ have recovered **$90**.
46
+ * Cold starts dominate headless runs, but each one is small. They're cheap
47
+ per call and only add up at volume.
48
+
49
+ Reproduce on your own machine: `kvleak --since 30d`.
50
+
51
+ > This is one person's usage, at list API prices. Your mix will differ, and
52
+ > that's the point of running it yourself.
53
+
54
+ ## 2. What one cache miss costs, by model
55
+
56
+ Avoidable cost of rebuilding a **200k-token** context instead of reading it
57
+ (`(write − read) × 200k`), at list prices. Source: `kvleak models` and
58
+ `kvleak breakeven 200k --model <m> --ttl <ttl>`.
59
+
60
+ | model | one cache hit | one miss, 5-min TTL | one miss, 1-hour TTL | miss ÷ hit (1h) | keep-alive break-even (5m / 1h) |
61
+ |---|---:|---:|---:|---:|---:|
62
+ | Claude Fable 5.1 | $0.05 | $2.45 | **$3.95** | **80×** | 49 / 79 pings |
63
+ | Claude Opus 5.5 | $0.04 | $0.96 | $1.56 | 40× | 24 / 39 pings |
64
+ | Claude Sonnet 5.5 | $0.04 | $0.46 | $0.76 | 20× | 11 / 19 pings |
65
+ | Claude Haiku 5.5 | $0.002 | $0.02 | $0.04 | 20× | 11 / 19 pings |
66
+
67
+ Newer models make cache reads cheaper, so each miss hurts *more* relative to a
68
+ hit. On Fable 5.1, a single miss costs as much as 80 hits.
69
+
70
+ ## 3. Simulation: a 6-worker swarm, 40 rounds
71
+
72
+ `python examples/swarm_sim.py`. This is a transparent toy model, not a
73
+ measurement. Every worker re-reads its context each round, and one failing
74
+ build log (~8k tokens) is produced per round. Prices are Opus 5.5 with a
75
+ 5-minute TTL.
76
+
77
+ | scenario | input (tok-equiv) | cost | vs naive |
78
+ |---|---:|---:|---:|
79
+ | naive: paste logs on the board, respawn the fixer, reviewer on another model | 17.8M | $71.10 | — |
80
+ | + post pointers instead of logs (`kvleak pointer`) | 6.8M | $27.37 | **−62%** |
81
+ | + keep the fixer warm instead of respawning | 5.8M | $23.18 | −67% |
82
+ | + one model per worker (the [playbook](playbook.md)) | 2.4M | $9.63 | **−86%** |
83
+
84
+ The board change alone removes most of the O(N²) broadcast cost, because one
85
+ pasted log is re-read by every worker on every later turn.