cachemaxxing 0.1.0__tar.gz

cachemaxxing-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Ashhad Ahsan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

cachemaxxing-0.1.0/PKG-INFO

@@ -0,0 +1,84 @@
+Metadata-Version: 2.4
+Name: cachemaxxing
+Version: 0.1.0
+Summary: Stop paying full price for the same tokens: measure, diagnose and fix prompt-cache misses on Anthropic, OpenAI, Gemini and LiteLLM.
+Keywords: llm,prompt-caching,anthropic,openai,gemini,litellm,cost
+Author: Ashhad Ahsan
+Author-email: Ashhad Ahsan <ashhadahsan@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
+Classifier: Typing :: Typed
+Requires-Dist: anthropic>=2.3 ; extra == 'anthropic'
+Requires-Dist: google-genai>=2.34 ; extra == 'gemini'
+Requires-Dist: litellm>=1.106 ; extra == 'litellm'
+Requires-Dist: openai>=3.41 ; extra == 'openai'
+Requires-Python: >=3.11
+Project-URL: Homepage, https://ashhadahsan.github.io/cachemaxxing/
+Project-URL: Source, https://github.com/ashhadahsan/cachemaxxing
+Project-URL: Changelog, https://github.com/ashhadahsan/cachemaxxing/blob/main/CHANGELOG.md
+Provides-Extra: anthropic
+Provides-Extra: gemini
+Provides-Extra: litellm
+Provides-Extra: openai
+Description-Content-Type: text/markdown
+
+# cachemaxxing
+
+[![PyPI](https://img.shields.io/pypi/v/cachemaxxing)](https://pypi.org/project/cachemaxxing/)
+[![Python](https://img.shields.io/pypi/pyversions/cachemaxxing)](https://pypi.org/project/cachemaxxing/)
+[![CI](https://github.com/ashhadahsan/cachemaxxing/actions/workflows/ci.yml/badge.svg)](https://github.com/ashhadahsan/cachemaxxing/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/ashhadahsan/cachemaxxing/blob/main/LICENSE)
+
+**Stop paying full price for the same tokens.**
+
+Anthropic, OpenAI and Gemini bill a repeated prompt prefix at a fraction of the input price, but only when the bytes really repeat. cachemaxxing finds where they don't, tells you what it costs, and lays your prompts out so they do.
+
+- **Measure.** `usage()` normalizes every provider's usage, and `cachemaxxing report` gives hit rate and $ per call site.
+- **Diagnose.** `cachemaxxing diff` shows the first byte where two requests diverge, and why.
+- **Fix.** `Prompt` orders segments from most to least stable and places each provider's cache markers.
+
+```python
+from cachemaxxing import Prompt, usage
+
+p = Prompt(call_site="support.answer")
+p.tools(TOOLS).system(SYSTEM).context(DOCS).history(messages).user(question)
+
+resp = client.messages.create(max_tokens=1024, **p.anthropic("claude-haiku-5-5"))
+
+u = usage(resp)
+print(f"{u.hit_rate:.0%} of input from cache, ~${u.est_cost or 0:.4f}")
+```
+
+A full multi-turn RAG chat lives in [`examples/rag_chat.py`](examples/rag_chat.py).
+The API works the same with `p.openai(...)`, `p.openai_chat(...)`, `p.gemini(...)` and `p.litellm(...)`. It is a thin library, not a proxy, and has zero runtime dependencies.
+
+## Benchmark
+
+![Cached input tokens per 8-turn chat, naive vs cachemaxxing](https://raw.githubusercontent.com/ashhadahsan/cachemaxxing/main/docs/assets/benchmark.svg)
+
+| Provider | Model | Cached input | $ per conversation | Saving |
+|---|---|---|---|---|
+| Anthropic | `claude-haiku-5-5` | 0.0% → **94.8%** | $0.0092 → $0.0017 | **82%** |
+| OpenAI | `gpt-6-luna` | 0.0% → **93.1%** | $0.0070 → $0.0012 | **83%** |
+| Gemini | `gemini-3.8-flash` | 0.0% → **97.5%** | $0.0484 → $0.0094 | **80%** |
+
+The same 8-turn conversation, 3 customers per arm, starting from a cold cache, with costs estimated at list price. The naive arm has four common habits: a timestamp at the top of the prompt, tool order that varies, JSON key order that varies, and history re-rendered every turn. The Gemini row uses an explicit cache, creation and storage cost included; with implicit caching alone it reached 40.0%. Data: [`benchmarks/results/`](benchmarks/results/), 2026-10-30.
+
+## Install
+
+```console
+pip install cachemaxxing        # or: uv add cachemaxxing
+pip install "cachemaxxing[anthropic]"   # extras: anthropic, openai, gemini, litellm
+```
+
+**Docs:** https://ashhadahsan.github.io/cachemaxxing/ covers the quickstart, how caching works per provider, diagnosing a low hit rate, the API reference, and how it compares.
+
+MIT licensed.

cachemaxxing-0.1.0/README.md

@@ -0,0 +1,52 @@
+# cachemaxxing
+
+[![PyPI](https://img.shields.io/pypi/v/cachemaxxing)](https://pypi.org/project/cachemaxxing/)
+[![Python](https://img.shields.io/pypi/pyversions/cachemaxxing)](https://pypi.org/project/cachemaxxing/)
+[![CI](https://github.com/ashhadahsan/cachemaxxing/actions/workflows/ci.yml/badge.svg)](https://github.com/ashhadahsan/cachemaxxing/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/ashhadahsan/cachemaxxing/blob/main/LICENSE)
+
+**Stop paying full price for the same tokens.**
+
+Anthropic, OpenAI and Gemini bill a repeated prompt prefix at a fraction of the input price, but only when the bytes really repeat. cachemaxxing finds where they don't, tells you what it costs, and lays your prompts out so they do.
+
+- **Measure.** `usage()` normalizes every provider's usage, and `cachemaxxing report` gives hit rate and $ per call site.
+- **Diagnose.** `cachemaxxing diff` shows the first byte where two requests diverge, and why.
+- **Fix.** `Prompt` orders segments from most to least stable and places each provider's cache markers.
+
+```python
+from cachemaxxing import Prompt, usage
+
+p = Prompt(call_site="support.answer")
+p.tools(TOOLS).system(SYSTEM).context(DOCS).history(messages).user(question)
+
+resp = client.messages.create(max_tokens=1024, **p.anthropic("claude-haiku-5-5"))
+
+u = usage(resp)
+print(f"{u.hit_rate:.0%} of input from cache, ~${u.est_cost or 0:.4f}")
+```
+
+A full multi-turn RAG chat lives in [`examples/rag_chat.py`](examples/rag_chat.py).
+The API works the same with `p.openai(...)`, `p.openai_chat(...)`, `p.gemini(...)` and `p.litellm(...)`. It is a thin library, not a proxy, and has zero runtime dependencies.
+
+## Benchmark
+
+![Cached input tokens per 8-turn chat, naive vs cachemaxxing](https://raw.githubusercontent.com/ashhadahsan/cachemaxxing/main/docs/assets/benchmark.svg)
+
+| Provider | Model | Cached input | $ per conversation | Saving |
+|---|---|---|---|---|
+| Anthropic | `claude-haiku-5-5` | 0.0% → **94.8%** | $0.0092 → $0.0017 | **82%** |
+| OpenAI | `gpt-6-luna` | 0.0% → **93.1%** | $0.0070 → $0.0012 | **83%** |
+| Gemini | `gemini-3.8-flash` | 0.0% → **97.5%** | $0.0484 → $0.0094 | **80%** |
+
+The same 8-turn conversation, 3 customers per arm, starting from a cold cache, with costs estimated at list price. The naive arm has four common habits: a timestamp at the top of the prompt, tool order that varies, JSON key order that varies, and history re-rendered every turn. The Gemini row uses an explicit cache, creation and storage cost included; with implicit caching alone it reached 40.0%. Data: [`benchmarks/results/`](benchmarks/results/), 2026-10-30.
+
+## Install
+
+```console
+pip install cachemaxxing        # or: uv add cachemaxxing
+pip install "cachemaxxing[anthropic]"   # extras: anthropic, openai, gemini, litellm
+```
+
+**Docs:** https://ashhadahsan.github.io/cachemaxxing/ covers the quickstart, how caching works per provider, diagnosing a low hit rate, the API reference, and how it compares.
+
+MIT licensed.

cachemaxxing-0.1.0/pyproject.toml

@@ -0,0 +1,91 @@
+[project]
+name = "cachemaxxing"
+version = "0.1.0"
+description = "Stop paying full price for the same tokens: measure, diagnose and fix prompt-cache misses on Anthropic, OpenAI, Gemini and LiteLLM."
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+requires-python = ">=3.11"
+dependencies = []
+keywords = [
+    "llm",
+    "prompt-caching",
+    "anthropic",
+    "openai",
+    "gemini",
+    "litellm",
+    "cost",
+]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3 :: Only",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+    "Programming Language :: Python :: 3.15",
+    "Typing :: Typed",
+]
+
+[[project.authors]]
+name = "Ashhad Ahsan"
+email = "ashhadahsan@gmail.com"
+
+[project.optional-dependencies]
+anthropic = ["anthropic>=2.3"]
+openai = ["openai>=3.41"]
+gemini = ["google-genai>=2.34"]
+litellm = ["litellm>=1.106"]
+
+[project.urls]
+Homepage = "https://ashhadahsan.github.io/cachemaxxing/"
+Source = "https://github.com/ashhadahsan/cachemaxxing"
+Changelog = "https://github.com/ashhadahsan/cachemaxxing/blob/main/CHANGELOG.md"
+
+[project.scripts]
+cachemaxxing = "cachemaxxing.cli:main"
+
+[build-system]
+requires = ["uv_build>=0.13.6,<0.14"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = [
+    "pytest>=9.2",
+    "ruff>=0.17.3",
+    "mypy>=2.4.1",
+    "anthropic>=2.3",
+    "openai>=3.41",
+    "google-genai>=2.34",
+    "httpx>=0.28",
+]
+docs = [
+    "mkdocstrings-python>=2.0.10",
+    "zensical==0.0.73",
+]
+
+[tool.ruff]
+line-length = 120
+
+[tool.ruff.format]
+exclude = ["docs/*.md"]
+
+[tool.ruff.lint]
+select = [
+    "E",
+    "F",
+    "I",
+    "UP",
+    "B",
+]
+
+[tool.ruff.lint.per-file-ignores]
+"benchmarks/chart.py" = ["E501"]
+
+[tool.mypy]
+strict = true
+files = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

cachemaxxing-0.1.0/pyproject.toml.orig

@@ -0,0 +1,66 @@
+[project]
+name = "cachemaxxing"
+version = "0.1.0"
+description = "Stop paying full price for the same tokens: measure, diagnose and fix prompt-cache misses on Anthropic, OpenAI, Gemini and LiteLLM."
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Ashhad Ahsan", email = "ashhadahsan@gmail.com" }]
+requires-python = ">=3.11"
+dependencies = []
+keywords = ["llm", "prompt-caching", "anthropic", "openai", "gemini", "litellm", "cost"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Developers",
+  "Programming Language :: Python :: 3 :: Only",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Programming Language :: Python :: 3.14",
+  "Programming Language :: Python :: 3.15",
+  "Typing :: Typed",
+]
+
+[project.optional-dependencies]
+anthropic = ["anthropic>=2.3"]
+openai = ["openai>=3.41"]
+gemini = ["google-genai>=2.34"]
+litellm = ["litellm>=1.106"]  # never 1.82.7/1.82.8 (compromised 2026-03-24)
+
+[project.urls]
+Homepage = "https://ashhadahsan.github.io/cachemaxxing/"
+Source = "https://github.com/ashhadahsan/cachemaxxing"
+Changelog = "https://github.com/ashhadahsan/cachemaxxing/blob/main/CHANGELOG.md"
+
+[project.scripts]
+cachemaxxing = "cachemaxxing.cli:main"
+
+[build-system]
+requires = ["uv_build>=0.13.6,<0.14"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = ["pytest>=9.2", "ruff>=0.17.3", "mypy>=2.4.1", "anthropic>=2.3", "openai>=3.41", "google-genai>=2.34", "httpx>=0.28"]
+docs = [
+    "mkdocstrings-python>=2.0.10",
+    "zensical==0.0.73",
+]
+
+[tool.ruff]
+line-length = 120
+
+[tool.ruff.format]
+exclude = ["docs/*.md"]  # hand-aligned examples and --8<-- snippet includes
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B"]
+
+[tool.ruff.lint.per-file-ignores]
+"benchmarks/chart.py" = ["E501"]  # SVG markup lines
+
+[tool.mypy]
+strict = true
+files = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

cachemaxxing-0.1.0/src/cachemaxxing/__init__.py

@@ -0,0 +1,23 @@
+"""Stop paying full price for the same tokens."""
+
+from .canon import canonical_json
+from .diff import Divergence, diff
+from .economics import Decision, should_cache
+from .gemini import gemini_cache, gemini_cache_async
+from .prompt import Prompt
+from .usage import Stats, Usage, record, usage
+
+__all__ = [
+    "Decision",
+    "Divergence",
+    "Prompt",
+    "Stats",
+    "Usage",
+    "canonical_json",
+    "diff",
+    "gemini_cache",
+    "gemini_cache_async",
+    "record",
+    "should_cache",
+    "usage",
+]

cachemaxxing-0.1.0/src/cachemaxxing/canon.py

@@ -0,0 +1,42 @@
+"""Deterministic serialization helpers."""
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+
+def canonical_json(obj: Any) -> str:
+    """Same data, same bytes: sorted keys, fixed separators, non-ASCII kept."""
+    return json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
+
+
+def normalize_ws(text: str) -> str:
+    """LF line endings, no trailing whitespace per line, no blank lines at either end."""
+    lines = text.replace("\r\n", "\n").replace("\r", "\n").split("\n")
+    return "\n".join(line.rstrip() for line in lines).strip("\n")
+
+
+def plain(obj: Any) -> Any:
+    """An SDK (pydantic) object as a plain dict; anything else unchanged."""
+    dump = getattr(obj, "model_dump", None)
+    return dump(mode="json", exclude_none=True) if callable(dump) else obj
+
+
+def jsonable(obj: Any) -> Any:
+    """`plain`, applied recursively; tuples become lists."""
+    obj = plain(obj)
+    if isinstance(obj, dict):
+        return {k: jsonable(v) for k, v in obj.items()}
+    if isinstance(obj, list | tuple):
+        return [jsonable(v) for v in obj]
+    return obj
+
+
+def tool_name(tool: Any) -> str:
+    """Name of an Anthropic, OpenAI (Responses or Chat) or Gemini function tool; '' if none."""
+    t = plain(tool)
+    if not isinstance(t, dict):
+        return ""
+    fn = t.get("function")
+    return str(fn.get("name", "") if isinstance(fn, dict) else t.get("name", ""))

cachemaxxing-0.1.0/src/cachemaxxing/cli.py

@@ -0,0 +1,73 @@
+"""cachemaxxing diff A.json B.json | cachemaxxing report LOG.jsonl [--min-hit-rate X]"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import Counter
+from pathlib import Path
+from typing import Any
+
+from .diff import diff
+from .usage import Stats, usage
+
+NO_READ = "prefix ok, no cache read (TTL expired, below minimum, or routing)"
+
+
+def _load(path: str) -> Any:
+    return json.loads(Path(path).read_text(encoding="utf-8"))
+
+
+def _report(log: str, min_hit_rate: float | None) -> int:
+    stats = Stats()
+    last: dict[str, Any] = {}
+    drift: dict[str, Counter[str]] = {}
+    for line in Path(log).read_text(encoding="utf-8").splitlines():
+        if not line.strip():
+            continue
+        rec = json.loads(line)
+        site, u = rec.get("call_site", "default"), usage(rec["response"])
+        stats.add(site, u)
+        if site in last:
+            div = diff(last[site], rec["request"])
+            if (div is None or div.hint == "last turn") and u.cache_read == 0:
+                drift.setdefault(site, Counter())[NO_READ] += 1
+            elif div is not None and div.hint != "last turn":
+                path = re.sub(r"\[\d+\]", "[]", div.path)  # group growing conversations together
+                drift.setdefault(site, Counter())[f"{path} ({div.hint or '?'})"] += 1
+        last[site] = rec["request"]
+    if not stats.totals:
+        raise ValueError(f"no records in {log}")
+    notes = {site: f"{key} x{n}" for site, c in drift.items() for key, n in c.most_common(1)}
+    print(stats.table(notes))
+    low = sorted(s for s, u in stats.totals.items() if min_hit_rate is not None and u.hit_rate < min_hit_rate)
+    for s in low:
+        print(f"{s}: hit rate {stats.totals[s].hit_rate:.1%} is below --min-hit-rate {min_hit_rate}", file=sys.stderr)
+    return 1 if low else 0
+
+
+def main(argv: list[str] | None = None) -> int:
+    ap = argparse.ArgumentParser(prog="cachemaxxing", description="Find and price prompt-cache misses.")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    d = sub.add_parser("diff", help="first point where request B stops repeating request A")
+    d.add_argument("a")
+    d.add_argument("b")
+    r = sub.add_parser("report", help="hit rate, $ and top drift per call site from a JSONL log")
+    r.add_argument("log")
+    r.add_argument("--min-hit-rate", type=float, help="exit 1 if any call site is below this (0-1)")
+    args = ap.parse_args(argv)
+    try:
+        if args.cmd == "diff":
+            div = diff(_load(args.a), _load(args.b))
+            print(div if div else "no divergence: B repeats all of A")
+            return 1 if div else 0
+        return _report(args.log, args.min_hit_rate)
+    except (OSError, ValueError, KeyError) as e:
+        print(f"cachemaxxing: {e}", file=sys.stderr)
+        return 2
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

cachemaxxing-0.1.0/src/cachemaxxing/diff.py

@@ -0,0 +1,127 @@
+"""Find where request b stops repeating request a's cacheable prefix."""
+
+from __future__ import annotations
+
+import json
+import os
+import re
+from collections.abc import Iterator
+from dataclasses import dataclass
+from typing import Any
+
+from .canon import jsonable
+
+_ORDER = ("model", "tools", "system", "instructions", "config", "messages", "input", "contents")
+_GEMINI_PREFIX = ("cached_content", "system_instruction", "tools", "tool_config")
+_LISTS = ("messages", "input", "contents")
+_MARKERS = {"cache_control", "prompt_cache_breakpoint"}
+_TEXT_TYPES = {"text", "input_text", "output_text"}
+_TOOL_NAME = re.compile(r"(?:config\.)?tools\[\d+\](?:\.function|\.function_declarations\[\d+\])?\.name")
+_HINTS = (
+    ("uuid", re.compile(r"[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}")),
+    ("timestamp", re.compile(r"\d{4}-\d\d-\d\d[T ]\d\d:\d\d|\b\d{1,2}:\d\d:\d\d\b|\b1\d{9}(?:\d{3})?\b")),
+    ("hex id", re.compile(r"\b[0-9a-f]{16,}\b")),
+)
+
+
+@dataclass(frozen=True)
+class Divergence:
+    path: str  # where the two requests differ, e.g. "system[0].text"
+    offset: int  # characters into that value
+    a: str  # text around the divergence in request a
+    b: str  # ... and in request b
+    hint: str | None  # model, tool order, key order, removed, last turn, whitespace, uuid, timestamp, hex id
+    prefix_chars: int  # identical characters before it (about 4 per token)
+
+    def __str__(self) -> str:
+        why = f" ({self.hint})" if self.hint else ""
+        return (
+            f"diverges at {self.path} char {self.offset}{why}, after ~{self.prefix_chars // 4:,} matching tokens\n"
+            f"  a: {self.a!r}\n  b: {self.b!r}"
+        )
+
+
+def _leaves(x: Any, path: str) -> Iterator[tuple[str, str]]:
+    if isinstance(x, dict):
+        if isinstance(x.get("content"), str):  # "hi" renders like [{"type": "text", "text": "hi"}]
+            x = {**x, "content": [{"type": "text", "text": x["content"]}]}
+        for k, v in x.items():
+            if k not in _MARKERS and not (k == "type" and v in _TEXT_TYPES):
+                yield from _leaves(v, f"{path}.{k}")
+    elif isinstance(x, list):
+        for i, v in enumerate(x):
+            yield from _leaves(v, f"{path}[{i}]")
+    else:
+        yield path, x if isinstance(x, str) else json.dumps(x)
+
+
+def _snake(key: str) -> str:
+    return re.sub(r"(?<!^)([A-Z])", r"_\1", key).lower()
+
+
+def _request(r: Any) -> dict[str, Any]:
+    r = jsonable(r)
+    if not isinstance(r, dict):
+        raise ValueError("expected a request object (a JSON dict) or a record() line")
+    inner = r.get("request")
+    return inner if isinstance(inner, dict) else r
+
+
+def _flat(req: dict[str, Any]) -> list[tuple[str, str]]:
+    out: list[tuple[str, str]] = []
+    for k in _ORDER:
+        if k in req:
+            v = req[k]
+            if k == "config" and isinstance(v, dict):  # google-genai accepts camelCase keys too
+                snake = {_snake(key): x for key, x in v.items()}
+                v = {c: snake[c] for c in _GEMINI_PREFIX if c in snake}
+                if isinstance(v.get("system_instruction"), str):
+                    v["system_instruction"] = {"parts": [{"text": v["system_instruction"]}]}
+            elif k == "system" and isinstance(v, str):  # "S" renders like [{"type": "text", "text": "S"}]
+                v = [{"type": "text", "text": v}]
+            out += _leaves(v, k)
+    return out
+
+
+def diff(a: Any, b: Any) -> Divergence | None:
+    """First point where b stops repeating a's prompt; None when b repeats all of a.
+    Accepts request kwargs, SDK objects, or record() lines."""
+    ra, rb = _request(a), _request(b)
+    fa, fb = _flat(ra), _flat(rb)
+    seen = 0
+    for i, (pa, va) in enumerate(fa):
+        if i == len(fb):
+            return Divergence(pa, 0, va[:60], "", "removed", seen)
+        pb, vb = fb[i]
+        if pa != pb:
+            parent = pa.rsplit(".", 1)[0]
+            same = pb.rsplit(".", 1)[0] == parent and {x for x in fa if x[0].startswith(parent)} == {
+                x for x in fb if x[0].startswith(parent)
+            }
+            return Divergence(f"{pa} / {pb}", 0, va[:60], vb[:60], "key order" if same else None, seen)
+        if va != vb:
+            off = len(os.path.commonprefix([va, vb]))
+            lo = max(0, off - 30)
+            hint = _hint(pa, va, vb, off, fa, fb, ra)
+            return Divergence(pa, off, va[lo : off + 30], vb[lo : off + 30], hint, seen + off)
+        seen += len(va)
+    return None
+
+
+def _hint(
+    path: str, va: str, vb: str, off: int, fa: list[tuple[str, str]], fb: list[tuple[str, str]], ra: dict[str, Any]
+) -> str | None:
+    if path == "model":
+        return "model"
+    names_a = [v for p, v in fa if _TOOL_NAME.fullmatch(p)]
+    names_b = [v for p, v in fb if _TOOL_NAME.fullmatch(p)]
+    if "tools" in path.split("[")[0] and names_a != names_b and sorted(names_a) == sorted(names_b):
+        return "tool order"
+    for k in _LISTS:
+        items = ra.get(k)
+        if isinstance(items, list) and items and path.startswith(f"{k}[{len(items) - 1}]"):
+            return "last turn"
+    if va.split() == vb.split():
+        return "whitespace"
+    wa, wb = va[max(0, off - 40) : off + 40], vb[max(0, off - 40) : off + 40]
+    return next((name for name, rx in _HINTS if rx.search(wa) and rx.search(wb)), None)

cachemaxxing-0.1.0/src/cachemaxxing/economics.py

@@ -0,0 +1,51 @@
+"""Does explicit caching pay? Pure arithmetic over facts.MODELS."""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass
+
+from . import facts
+
+TTL_SECONDS = {"5m": 300, "1h": 3600, "30m": 1800, "12h": 43200}
+
+
+@dataclass(frozen=True)
+class Decision:
+    cache: bool
+    ttl: str | None  # "5m"/"1h" (Anthropic), "30m"/"12h" (OpenAI), "<n>s" (Gemini explicit cache)
+    saving: float  # expected USD saved versus not caching; negative means caching loses money
+    reason: str
+
+
+def should_cache(
+    prefix_tokens: int, expected_reuses: int, provider: str, model: str, *, interval_s: float = 60.0
+) -> Decision:
+    """Whether caching a prefix of `prefix_tokens` pays when it is reused `expected_reuses` times,
+    `interval_s` seconds apart (start to start)."""
+    m = facts.find(model)
+    if m is None or m.provider != provider:
+        raise ValueError(f"unknown {provider} model {model!r}; add it to cachemaxxing.facts.MODELS")
+    if m.long_context and prefix_tokens > m.long_context[0]:
+        m = m.long_context[1]
+    if prefix_tokens < m.min_tokens:
+        return Decision(False, None, 0.0, f"{prefix_tokens} tokens is below the {m.min_tokens}-token minimum")
+    t, n = prefix_tokens / 1e6, expected_reuses
+    ttl: str | None
+    if provider == "gemini":
+        if m.storage_hour is None:
+            return Decision(False, None, 0.0, f"{model} has no explicit caching; implicit caching is automatic")
+        seconds = max(60, math.ceil(interval_s * n))
+        ttl, cost = f"{seconds}s", t * m.input + n * t * m.cache_read + t * m.storage_hour * seconds / 3600
+    elif not m.cache_write:  # automatic caching without a write premium
+        ttl, cost = None, t * m.input + n * t * m.cache_read
+    else:
+        fits = [k for k in m.cache_write if TTL_SECONDS[k] >= interval_s]
+        if not fits:
+            return Decision(False, None, 0.0, f"reuses {interval_s:g}s apart outlive the longest TTL")
+        ttl = min(fits, key=TTL_SECONDS.__getitem__)
+        cost = t * m.cache_write[ttl] + n * t * m.cache_read
+    saving = (1 + n) * t * m.input - cost
+    if saving <= 0:
+        return Decision(False, None, saving, f"caching costs ${-saving:.6f} more than not caching")
+    return Decision(True, ttl, saving, f"saves ${saving:.6f} over {n} reuses")

cachemaxxing-0.1.0/src/cachemaxxing/facts.py

@@ -0,0 +1,114 @@
+"""Provider prompt-caching facts. Prices are USD per 1M tokens; every row links to its source.
+
+Checked on VERIFIED. Add or correct models by editing MODELS (it is a plain dict)."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+
+VERIFIED = "2026-10-30"
+
+
+@dataclass(frozen=True)
+class Provider:
+    max_breakpoints: int | None  # explicit markers per request; None = none (implicit caching)
+    ttls: tuple[str, ...]  # shortest first
+    docs: str
+
+
+@dataclass(frozen=True)
+class Model:
+    provider: str
+    input: float
+    output: float
+    cache_read: float
+    cache_write: dict[str, float] = field(default_factory=dict)  # TTL -> price, shortest first
+    min_tokens: int = 1024
+    storage_hour: float | None = None  # Gemini explicit caches, per 1M tokens per hour
+    long_context: tuple[int, Model] | None = None  # (prompt tokens above which, these prices)
+    source: str = ""
+
+
+PROVIDERS = {
+    "anthropic": Provider(4, ("5m", "1h"), "https://platform.claude.com/docs/en/build-with-claude/prompt-caching"),
+    "openai": Provider(4, ("30m", "12h"), "https://developers.openai.com/api/docs/guides/prompt-caching"),
+    "gemini": Provider(None, (), "https://ai.google.dev/gemini-api/docs/caching"),
+}
+
+_A = "https://platform.claude.com/docs/en/about-claude/pricing"
+_O = "https://developers.openai.com/api/docs/pricing"
+_G = "https://ai.google.dev/gemini-api/docs/pricing"
+
+MODELS: dict[str, Model] = {
+    "claude-haiku-5-5": Model(
+        "anthropic",
+        0.10,
+        0.50,
+        0.01,
+        {"5m": 0.125, "1h": 0.20},
+        512,
+        source=_A,
+        long_context=(100_000, Model("anthropic", 0.50, 2.50, 0.05, {"5m": 0.625, "1h": 1.00}, 512, source=_A)),
+    ),
+    "claude-sonnet-5-5": Model("anthropic", 2.00, 10.00, 0.10, {"5m": 2.50, "1h": 4.00}, 512, source=_A),
+    "claude-opus-5-5": Model("anthropic", 4.00, 20.00, 0.20, {"5m": 5.00, "1h": 8.00}, 512, source=_A),
+    "claude-fable-5-1": Model("anthropic", 10.00, 50.00, 0.25, {"5m": 12.50, "1h": 20.00}, 512, source=_A),
+    "gpt-6-luna": Model("openai", 0.10, 0.50, 0.01, {"30m": 0.125, "12h": 0.25}, 1024, source=_O),
+    "gpt-6.1-sol": Model("openai", 2.00, 10.00, 0.10, {"30m": 2.50, "12h": 5.00}, 1024, source=_O),
+    "gpt-6-astra": Model("openai", 10.00, 50.00, 1.00, {"30m": 12.50, "12h": 25.00}, 1024, source=_O),
+    # Gemini caches implicitly; storage_hour marks explicit-cache support.
+    # gemini-3.8-flash prices double on 2027-01-01, gemini-4-argon's on 2026-12-01: patch releases then.
+    "gemini-3.8-flash": Model("gemini", 0.75, 3.75, 0.075, min_tokens=4096, storage_hour=0.50, source=_G),
+    "gemini-3.1-flash-lite": Model("gemini", 0.25, 1.50, 0.025, min_tokens=4096, storage_hour=1.00, source=_G),
+    "gemini-4-argon": Model("gemini", 2.00, 10.00, 0.10, min_tokens=4096, source=_G),
+}
+
+
+def find(model: str) -> Model | None:
+    """Longest-prefix match on the id, ignoring 'provider/' and 'models/' prefixes and dated suffixes."""
+    name = model.rsplit("/", 1)[-1]
+    best = max((k for k in MODELS if name.startswith(k)), key=len, default=None)
+    return MODELS[best] if best else None
+
+
+def provider_of(model: str) -> str:
+    """'anthropic', 'openai', 'gemini' or 'other' for a model id, LiteLLM 'route/model' ids included."""
+    if (m := find(model)) is not None:
+        return m.provider
+    prefix, _, rest = model.partition("/")
+    if rest and prefix in PROVIDERS:
+        return prefix
+    name = rest or prefix
+    if "claude" in name:
+        return "anthropic"
+    if name.startswith("gemini"):
+        return "gemini"
+    if not rest and name.startswith(("gpt-", "o1", "o3", "o4")):
+        return "openai"
+    return "other"
+
+
+def markdown() -> str:
+    """The facts as Markdown; docs/_facts.md is this output."""
+    out = ["| Provider | Max breakpoints | TTLs | Docs |", "|---|---|---|---|"]
+    for name, p in PROVIDERS.items():
+        ttls = ", ".join(p.ttls) or "explicit caches take any TTL"
+        out.append(f"| {name} | {p.max_breakpoints or 'implicit only'} | {ttls} | <{p.docs}> |")
+    out += [
+        "",
+        "| Model | Input | Cache read | Cache write | Output | Min tokens | Storage/h | Source |",
+        "|---|--:|--:|---|--:|--:|--:|---|",
+    ]
+    for name, m in MODELS.items():
+        write = ", ".join(f"{ttl}: {price:g}" for ttl, price in m.cache_write.items()) or "—"
+        storage = "—" if m.storage_hour is None else f"{m.storage_hour:g}"
+        out.append(
+            f"| `{name}` | {m.input:g} | {m.cache_read:g} | {write} | {m.output:g} "
+            f"| {m.min_tokens:,} | {storage} | [pricing]({m.source}) |"
+        )
+    out += ["", f"USD per 1M tokens. Verified {VERIFIED}."]
+    return "\n".join(out) + "\n"
+
+
+if __name__ == "__main__":
+    print(markdown(), end="")

cachemaxxing-0.1.0/src/cachemaxxing/gemini.py

@@ -0,0 +1,84 @@
+"""Gemini explicit caches through your own google-genai client: the library's only I/O."""
+
+from __future__ import annotations
+
+import time
+from typing import Any
+
+from .economics import should_cache
+from .prompt import Prompt
+
+_LIVE: dict[str, tuple[str, float]] = {}  # ponytail: per-process memo; other processes match on display_name
+_MARGIN = 30.0  # seconds a cache must still live to be reused
+
+
+def _plan(
+    p: Prompt, model: str, reuses: int, interval_s: float, tokens: int | None
+) -> tuple[str, dict[str, Any]] | None:
+    if any(callable(t) for t in p._tools):  # python-function tools can't live in an explicit cache
+        return None
+    d = should_cache(tokens or p.est_prefix_tokens(), reuses, "gemini", model, interval_s=interval_s)
+    if not d.cache or d.ttl is None:
+        return None
+    key = f"cmx-{p.call_site}-{model.rsplit('/', 1)[-1]}-{p.prefix_hash()}"
+    return key, {"display_name": key, "ttl": d.ttl, **p.gemini_prefix()}
+
+
+def _alive(c: Any, key: str) -> bool:
+    exp = getattr(c, "expire_time", None)
+    return getattr(c, "display_name", None) == key and exp is not None and exp.timestamp() - time.time() > _MARGIN
+
+
+def _remember(key: str, c: Any) -> str:
+    _LIVE[key] = (str(c.name), c.expire_time.timestamp())
+    return str(c.name)
+
+
+def _memo(key: str) -> str | None:
+    hit = _LIVE.get(key)
+    return hit[0] if hit and hit[1] - time.time() > _MARGIN else None
+
+
+def gemini_cache(
+    client: Any,
+    p: Prompt,
+    model: str,
+    *,
+    expected_reuses: int,
+    interval_s: float = 60.0,
+    prefix_tokens: int | None = None,
+) -> str | None:
+    """Name of a live explicit cache holding p's static prefix, created only if should_cache() says it pays.
+    Use it as p.gemini(model, cached_content=name). None means stay on implicit caching."""
+    plan = _plan(p, model, expected_reuses, interval_s, prefix_tokens)
+    if plan is None:
+        return None
+    key, config = plan
+    if name := _memo(key):
+        return name
+    for c in client.caches.list():
+        if _alive(c, key):
+            return _remember(key, c)
+    return _remember(key, client.caches.create(model=model, config=config))
+
+
+async def gemini_cache_async(
+    client: Any,
+    p: Prompt,
+    model: str,
+    *,
+    expected_reuses: int,
+    interval_s: float = 60.0,
+    prefix_tokens: int | None = None,
+) -> str | None:
+    """Async twin of gemini_cache, using client.aio."""
+    plan = _plan(p, model, expected_reuses, interval_s, prefix_tokens)
+    if plan is None:
+        return None
+    key, config = plan
+    if name := _memo(key):
+        return name
+    async for c in await client.aio.caches.list():
+        if _alive(c, key):
+            return _remember(key, c)
+    return _remember(key, await client.aio.caches.create(model=model, config=config))

cachemaxxing-0.1.0/src/cachemaxxing/prompt.py

@@ -0,0 +1,295 @@
+"""Prompt: segments ordered from most to least stable, rendered as request kwargs per provider."""
+
+from __future__ import annotations
+
+import hashlib
+import re
+from collections.abc import Mapping, Sequence
+from typing import Any, Self
+
+from . import facts
+from .canon import canonical_json, normalize_ws, plain, tool_name
+
+Json = dict[str, Any]
+Parts = str | Mapping[str, Any] | Sequence[str | Mapping[str, Any]]
+
+
+def _parts(x: Parts) -> list[str]:
+    """Text parts: strings verbatim, anything else as canonical JSON."""
+    items: Sequence[Any] = [x] if isinstance(x, str | Mapping) else x
+    return [p if isinstance(p, str) else canonical_json(p) for p in items]
+
+
+def _sort_tool(t: Any) -> Any:
+    t = plain(t)
+    if isinstance(t, dict):
+        for key in ("function_declarations", "functionDeclarations"):
+            if isinstance(t.get(key), list):
+                return {**t, key: sorted(t[key], key=tool_name)}
+    return t
+
+
+_NOT_MARKABLE = {"thinking", "redacted_thinking"}
+_LITELLM_ROUTES = {  # LiteLLM route prefixes that honour each provider's cache fields
+    "anthropic": {"", "anthropic", "bedrock", "vertex_ai"},
+    "openai": {"", "openai"},
+    "gemini": {"", "gemini", "vertex_ai"},
+}
+
+
+def _cc(ttl: str) -> Json:
+    return {"type": "ephemeral"} if ttl == "5m" else {"type": "ephemeral", "ttl": ttl}
+
+
+def _drop(t: Any, key: str) -> Any:
+    d = plain(t)
+    return {k: v for k, v in d.items() if k != key} if isinstance(d, dict) and key in d else t
+
+
+def _drop_fn(t: Any) -> Any:
+    """A LiteLLM tool without a stale cache_control inside its function object."""
+    d = plain(t)
+    if isinstance(d, dict) and isinstance(d.get("function"), dict) and "cache_control" in d["function"]:
+        return {**d, "function": _drop(d["function"], "cache_control")}
+    return t
+
+
+def _check_ttl(provider: str, ttl: str) -> str:
+    if ttl not in facts.PROVIDERS[provider].ttls:
+        raise ValueError(f"{provider} cache TTL must be one of {facts.PROVIDERS[provider].ttls}, not {ttl!r}")
+    return ttl
+
+
+def _unmark(msg: Any, key: str) -> Any:
+    """Copy of msg without `key` markers on its blocks. Markers aren't content; stale ones would pile up."""
+    m = plain(msg)
+    if not (isinstance(m, dict) and isinstance(m.get("content"), list)):
+        return msg
+    blocks = [plain(b) for b in m["content"]]
+    if not any(isinstance(b, dict) and key in b for b in blocks):
+        return msg
+    return {**m, "content": [{k: v for k, v in b.items() if k != key} if isinstance(b, dict) else b for b in blocks]}
+
+
+def openai_explicit(model: str) -> bool:
+    """GPT-5.6 and later take explicit cache breakpoints; older models cache implicitly."""
+    m = re.search(r"gpt-(\d+)(?:\.(\d+))?", model)
+    return m is not None and (int(m[1]), int(m[2] or 0)) >= (5, 6)
+
+
+def _mark(msg: Any, key: str, marker: Json, text_type: str, only_text: bool) -> Any:
+    """Copy of msg with `marker` on its last markable block, or None if it has none. Text never changes."""
+    m = dict(plain(msg))
+    content = m.get("content")
+    if isinstance(content, str):
+        if not content.strip():  # providers reject markers on empty text
+            return None
+        m["content"] = [{"type": text_type, "text": content, key: marker}]
+        return m
+    if isinstance(content, list):
+        blocks = [plain(b) for b in content]
+        for i in range(len(blocks) - 1, -1, -1):
+            kind = blocks[i].get("type") if isinstance(blocks[i], dict) else None
+            ok = kind == text_type if only_text else kind is not None and kind not in _NOT_MARKABLE
+            if ok and not (kind in ("text", "input_text") and not str(blocks[i].get("text", "")).strip()):
+                blocks[i] = {**blocks[i], key: marker}
+                m["content"] = blocks
+                return m
+    return None
+
+
+class Prompt:
+    """Collect segments in any order; render with an adapter. Output order is fixed:
+    tools, system, context, examples, history, retrieved, user."""
+
+    def __init__(self, call_site: str = "default") -> None:
+        self.call_site = call_site
+        self._tools: list[Any] = []
+        self._system: list[str] = []
+        self._context: list[str] = []
+        self._examples: list[str] = []
+        self._history: list[Any] = []
+        self._retrieved: list[str] = []
+        self._user: list[str] = []
+
+    def tools(self, tools: Sequence[Any], *, sort: bool = True) -> Self:
+        """Tool definitions in your provider's format; sorted by name unless sort=False."""
+        self._tools = sorted(map(_sort_tool, tools), key=tool_name) if sort else list(tools)
+        return self
+
+    def system(self, text: Parts) -> Self:
+        self._system = [t for t in map(normalize_ws, _parts(text)) if t]
+        return self
+
+    def context(self, docs: Parts, *, reorderable: bool = False) -> Self:
+        """Static reference material. reorderable=True sorts the parts so their order can't drift."""
+        parts = [t for t in map(normalize_ws, _parts(docs)) if t]
+        self._context = sorted(parts) if reorderable else parts
+        return self
+
+    def examples(self, examples: Parts, *, reorderable: bool = False) -> Self:
+        parts = [t for t in map(normalize_ws, _parts(examples)) if t]
+        self._examples = sorted(parts) if reorderable else parts
+        return self
+
+    def history(self, messages: Sequence[Any]) -> Self:
+        """Earlier turns in your provider's message format. Append-only; never modified."""
+        self._history = list(messages)
+        return self
+
+    def retrieved(self, chunks: Parts) -> Self:
+        """Per-request context (RAG chunks, time, user profile). Placed after history, verbatim."""
+        self._retrieved = [t for t in _parts(chunks) if t.strip()]
+        return self
+
+    def user(self, text: str) -> Self:
+        self._user = [text] if text.strip() else []
+        return self
+
+    @property
+    def static(self) -> list[str]:
+        return self._system + self._context + self._examples
+
+    def _prefix_blob(self) -> str:
+        tools = [t.__qualname__ if callable(t) else plain(t) for t in self._tools]  # python-function tools: by name
+        return canonical_json([tools, self.static])
+
+    def prefix_hash(self) -> str:
+        """16 hex chars that change exactly when the tools or the static text change."""
+        return hashlib.sha256(self._prefix_blob().encode()).hexdigest()[:16]
+
+    def est_prefix_tokens(self) -> int:
+        # ponytail: ~4 chars/token estimate; pass exact counts to should_cache() when you have them
+        return len(self._prefix_blob()) // 4
+
+    def _tail(
+        self,
+        marker: Json | None,
+        key: str,
+        text_type: str,
+        roles: tuple[str, ...],
+        *,
+        only_text: bool,
+        count: int = 1,
+    ) -> list[Any]:
+        """History, then the final user turn, with `count` markers on the last markable positions: the final
+        user block (unless a per-request retrieved block follows it), then eligible history messages.
+        Stale markers in history are stripped first so they never pile up. Anthropic needs one tail marker
+        (its 20-block lookback finds earlier entries); OpenAI only looks up at breakpoints present in the
+        request, so count=2 also re-marks the position the previous request wrote."""
+        msgs = [_unmark(m, key) for m in self._history]
+        final: list[Json] = [{"type": text_type, "text": t} for t in self._retrieved + self._user]
+        left = count if marker is not None else 0
+        if left and final and not self._retrieved:
+            final[-1][key] = marker
+            left -= 1
+        for i in range(len(msgs) - 1, -1, -1):
+            if not left:
+                break
+            m = plain(msgs[i])
+            if isinstance(m, dict) and m.get("role") in roles and marker is not None:
+                marked = _mark(m, key, marker, text_type, only_text)
+                if marked is not None:
+                    msgs[i] = marked
+                    left -= 1
+        if final:
+            msgs.append({"role": "user", "content": final})
+        return msgs
+
+    def anthropic(self, model: str, *, ttl: str = "5m") -> Json:
+        """kwargs for anthropic `client.messages.create(max_tokens=..., **kwargs)`."""
+        cc = _cc(_check_ttl("anthropic", ttl))
+        kw: Json = {"model": model}
+        if self._tools:
+            tools = [_drop(t, "cache_control") for t in self._tools]
+            kw["tools"] = [*tools[:-1], {**plain(tools[-1]), "cache_control": cc}]
+        if self.static:
+            kw["system"] = [{"type": "text", "text": t} for t in self.static]
+            kw["system"][-1]["cache_control"] = cc
+        kw["messages"] = self._tail(cc, "cache_control", "text", ("user", "assistant"), only_text=False)
+        return kw
+
+    def openai(self, model: str, *, ttl: str = "30m", cache_key: str | None = None) -> Json:
+        """kwargs for openai `client.responses.create(**kwargs)`."""
+        return self._openai(model, ttl, cache_key, field="input", role="developer", text_type="input_text")
+
+    def openai_chat(self, model: str, *, ttl: str = "30m", cache_key: str | None = None) -> Json:
+        """kwargs for openai `client.chat.completions.create(**kwargs)` and OpenAI-compatible servers."""
+        if ttl == "12h":
+            raise ValueError("the 12h cache TTL is Responses-only; use Prompt.openai()")
+        return self._openai(model, ttl, cache_key, field="messages", role="system", text_type="text")
+
+    def _openai(self, model: str, ttl: str, cache_key: str | None, *, field: str, role: str, text_type: str) -> Json:
+        explicit = openai_explicit(model)
+        if explicit:
+            _check_ttl("openai", ttl)
+        bp: Json | None = {"mode": "explicit"} if explicit else None
+        kw: Json = {"model": model}
+        if self._tools:
+            kw["tools"] = list(self._tools)
+        msgs: list[Any] = []
+        if self.static:
+            parts: list[Json] = [{"type": text_type, "text": t} for t in self.static]
+            if bp is not None:
+                parts[-1]["prompt_cache_breakpoint"] = bp
+            msgs.append({"role": role, "content": parts})
+        roles = ("user", "developer", "system")
+        msgs += self._tail(bp, "prompt_cache_breakpoint", text_type, roles, only_text=True, count=2)
+        kw[field] = msgs
+        if explicit:
+            kw["prompt_cache_options"] = {"mode": "explicit", "ttl": ttl}
+        if cache_key or not explicit:
+            kw["prompt_cache_key"] = cache_key or f"{self.call_site}:{self.prefix_hash()}"
+        return kw
+
+    def gemini(self, model: str, *, cached_content: str | None = None) -> Json:
+        """kwargs for google-genai `client.models.generate_content(**kwargs)`. Implicit caching unless
+        cached_content (from gemini_cache) is given; then the cached fields are left out."""
+        config: Json = {"cached_content": cached_content} if cached_content else self.gemini_prefix()
+        contents = list(self._history)
+        final = [{"text": t} for t in self._retrieved + self._user]
+        if final:
+            contents.append({"role": "user", "parts": final})
+        return {"model": model, "contents": contents, "config": config}
+
+    def gemini_prefix(self) -> Json:
+        """The cacheable part of a Gemini request: what an explicit cache holds."""
+        prefix: Json = {}
+        if self.static:
+            prefix["system_instruction"] = {"parts": [{"text": t} for t in self.static]}
+        if self._tools:
+            prefix["tools"] = list(self._tools)
+        return prefix
+
+    def litellm(self, model: str, *, ttl: str | None = None, explicit: bool = False) -> Json:
+        """kwargs for `litellm.completion(**kwargs)` (or the LiteLLM Proxy via the OpenAI SDK), model included.
+        Cache fields follow the route: Claude gets cache_control, openai/ gets openai_chat(), gemini/ stays
+        implicit unless explicit=True (LiteLLM then creates an explicit cache for the static prefix)."""
+        route = model.split("/", 1)[0] if "/" in model else ""
+        provider = facts.provider_of(model)
+        if route not in _LITELLM_ROUTES.get(provider, set()):
+            provider = "other"
+        if provider == "openai":
+            return self.openai_chat(model, ttl=ttl or "30m")
+        if provider == "gemini" and explicit and (fm := facts.find(model)) is not None and fm.storage_hour is None:
+            raise ValueError(f"{model} has no explicit caching; drop explicit=True")
+        mark = provider == "anthropic" or (provider == "gemini" and explicit)
+        marker = _cc(_check_ttl("anthropic", ttl or "5m")) if mark else None
+        kw: Json = {"model": model}
+        if self._tools:
+            tools = [_drop_fn(t) for t in self._tools]
+            last = plain(tools[-1])
+            if provider == "anthropic" and isinstance(last, dict) and isinstance(last.get("function"), dict):
+                tools[-1] = {**last, "function": {**last["function"], "cache_control": marker}}
+            kw["tools"] = tools
+        msgs: list[Any] = []
+        if self.static:
+            parts: list[Json] = [{"type": "text", "text": t} for t in self.static]
+            if marker is not None:
+                parts[-1]["cache_control"] = marker
+            msgs.append({"role": "system", "content": parts})
+        # Gemini explicit caches cover the static prefix only; a history marker would create a cache per turn.
+        tail_marker = marker if provider == "anthropic" else None
+        msgs += self._tail(tail_marker, "cache_control", "text", ("user", "assistant"), only_text=True)
+        kw["messages"] = msgs
+        return kw

cachemaxxing-0.1.0/src/cachemaxxing/usage.py

@@ -0,0 +1,139 @@
+"""Normalize provider usage into one shape, price it, aggregate it per call site."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, replace
+from typing import Any
+
+from . import facts
+from .canon import jsonable, plain
+
+
+@dataclass(frozen=True)
+class Usage:
+    provider: str
+    model: str
+    input: int = 0  # every prompt token: uncached + cache read + cache write
+    cache_read: int = 0
+    cache_write: int = 0
+    output: int = 0  # includes thinking / reasoning tokens
+    est_cost: float | None = None  # USD at list price as of facts.VERIFIED; None for unknown models
+
+    @property
+    def hit_rate(self) -> float:
+        return self.cache_read / self.input if self.input else 0.0
+
+    def __add__(self, other: Usage) -> Usage:
+        cost = None if self.est_cost is None or other.est_cost is None else self.est_cost + other.est_cost
+        return replace(
+            self,
+            input=self.input + other.input,
+            cache_read=self.cache_read + other.cache_read,
+            cache_write=self.cache_write + other.cache_write,
+            output=self.output + other.output,
+            est_cost=cost,
+        )
+
+
+def _camel(key: str) -> str:
+    head, *rest = key.split("_")
+    return head + "".join(w.title() for w in rest)
+
+
+def _n(d: Any, *keys: str) -> int:
+    """First non-zero count among keys (snake_case or camelCase) of dict d; 0 if none."""
+    if isinstance(d, dict):
+        for k in keys:
+            for key in (k, _camel(k)):
+                if d.get(key):
+                    return int(d[key])
+    return 0
+
+
+def usage(response: Any, model: str | None = None) -> Usage:
+    """Normalize an Anthropic, OpenAI (Responses or Chat), Gemini or LiteLLM response (object or dict)."""
+    d = plain(response)
+    if not isinstance(d, dict):
+        raise ValueError("unrecognised response: expected an SDK response object or dict")
+    name = str(model or d.get("model") or d.get("model_version") or d.get("modelVersion") or "")
+    meta = d.get("usage_metadata") or d.get("usageMetadata")
+    if meta is not None or "candidates" in d:  # Gemini: prompt_token_count includes cached tokens
+        total = _n(meta, "prompt_token_count") + _n(meta, "tool_use_prompt_token_count")
+        out = _n(meta, "candidates_token_count") + _n(meta, "thoughts_token_count")
+        return _priced("gemini", name, total, _n(meta, "cached_content_token_count"), 0, out)
+    u = d.get("usage") or {}
+    if d.get("type") == "message":  # Anthropic: input_tokens EXCLUDES cache tokens
+        read, write = _n(u, "cache_read_input_tokens"), _n(u, "cache_creation_input_tokens")
+        long_write = _n(u.get("cache_creation"), "ephemeral_1h_input_tokens")
+        total = _n(u, "input_tokens") + read + write
+        return _priced("anthropic", name, total, read, write, _n(u, "output_tokens"), long_write)
+    if "prompt_tokens" in u:  # OpenAI Chat and LiteLLM: prompt_tokens includes cache tokens
+        det = u.get("prompt_tokens_details") or {}
+        write = _n(det, "cache_write_tokens", "cache_creation_tokens") or _n(u, "cache_creation_input_tokens")
+        long_write = _n(det.get("cache_write_tokens_details"), "ttl_12h_tokens")
+        read, out = _n(det, "cached_tokens"), _n(u, "completion_tokens")
+        return _priced(facts.provider_of(name), name, _n(u, "prompt_tokens"), read, write, out, long_write)
+    if "input_tokens" in u:  # OpenAI Responses: input_tokens includes cache tokens
+        det = u.get("input_tokens_details") or {}
+        long_write = _n(det.get("cache_write_tokens_details"), "ttl_12h_tokens")
+        read, write = _n(det, "cached_tokens"), _n(det, "cache_write_tokens")
+        return _priced("openai", name, _n(u, "input_tokens"), read, write, _n(u, "output_tokens"), long_write)
+    raise ValueError("unrecognised response: expected Anthropic, OpenAI, Gemini or LiteLLM")
+
+
+def _priced(provider: str, model: str, total: int, read: int, write: int, out: int, long_write: int = 0) -> Usage:
+    m = facts.find(model)
+    cost = None
+    if m is not None:
+        if m.long_context and total > m.long_context[0]:
+            m = m.long_context[1]
+        writes = list(m.cache_write.values()) or [m.input]  # [short TTL, long TTL]
+        uncached = total - read - write
+        cost = (
+            uncached * m.input
+            + read * m.cache_read
+            + (write - long_write) * writes[0]
+            + long_write * writes[-1]
+            + out * m.output
+        ) / 1e6
+    return Usage(provider, model, total, read, write, out, cost)
+
+
+_HEADER = ("call site", "calls", "hit rate", "input", "cache read", "cache write", "output", "est $", "top drift")
+
+
+class Stats:
+    """Usage totals per call site, in memory."""
+
+    def __init__(self) -> None:
+        self.calls: dict[str, int] = {}
+        self.totals: dict[str, Usage] = {}
+
+    def add(self, call_site: str, u: Usage) -> None:
+        self.calls[call_site] = self.calls.get(call_site, 0) + 1
+        self.totals[call_site] = self.totals[call_site] + u if call_site in self.totals else u
+
+    def table(self, notes: dict[str, str] | None = None) -> str:
+        rows = [list(_HEADER)]
+        for site, u in sorted(self.totals.items()):
+            cost = "n/a" if u.est_cost is None else f"{u.est_cost:.4f}"
+            rows.append(
+                [
+                    site,
+                    str(self.calls[site]),
+                    f"{u.hit_rate:.1%}",
+                    f"{u.input:,}",
+                    f"{u.cache_read:,}",
+                    f"{u.cache_write:,}",
+                    f"{u.output:,}",
+                    cost,
+                    (notes or {}).get(site, ""),
+                ]
+            )
+        widths = [max(len(r[i]) for r in rows) for i in range(len(rows[0]))]
+        return "\n".join("  ".join(c.ljust(w) for c, w in zip(r, widths, strict=True)).rstrip() for r in rows)
+
+
+def record(call_site: str, request: Any, response: Any) -> dict[str, Any]:
+    """One JSONL line for `cachemaxxing report`. The library never writes files; you do."""
+    return {"call_site": call_site, "request": jsonable(request), "response": jsonable(response)}