cache-doctor 0.1.0__tar.gz

cache_doctor-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 hao li
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

cache_doctor-0.1.0/PKG-INFO

@@ -0,0 +1,140 @@
+Metadata-Version: 2.4
+Name: cache-doctor
+Version: 0.1.0
+Summary: Semantic-cache health diagnostics: hit rate, expiry, and cache-pollution analysis for LLM response caches
+Author: hao li
+License: MIT
+Project-URL: Homepage, https://github.com/hahahahahahahahah6/cache-doctor
+Project-URL: Repository, https://github.com/hahahahahahahahah6/cache-doctor
+Project-URL: Issues, https://github.com/hahahahahahahahah6/cache-doctor/issues
+Keywords: llm,semantic-cache,cache,diagnostics,prompt-caching
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# cache-doctor
+
+The checkup your semantic cache never had.
+
+LLM gateways all ship semantic caches now — but every tool on the market only
+*operates* the cache (store, lookup, evict). Nobody diagnoses it. **cache-doctor
+is the first CLI that performs a health check on a semantic cache**: it reads
+your cache event log and reports hit rate, expiry waste, key fragmentation, and
+— the silent killer — **cache pollution**, where a lightly rewritten prompt
+matches a cached response it should never have matched.
+
+## Differentiation
+
+| | Cache products (Redis, GPTCache, LangChain caches, …) | cache-doctor |
+|---|---|---|
+| Job | Store and serve cached LLM responses | **Diagnose** the cache |
+| Hit rate | You compute it yourself | Measured, with rolling hourly trend |
+| Expiry | Configured once, forgotten | Expired-miss share + stale-serving detection |
+| Pollution | Invisible | Borderline-hit risk + ground-truth `flag_wrong` detection |
+| Key design | Whatever you happened to key on | Fragmentation detection (same prompt, many keys) |
+| Advice | None | Prioritized, severity-ranked suggestions |
+
+Think of it as the relationship between a database and a database health
+monitor: the cache is the patient, cache-doctor is the checkup instrument.
+
+## Install
+
+```bash
+pip install cache-doctor
+```
+
+Zero dependencies, stdlib only. Python ≥ 3.9. Works fully offline — your
+prompts never leave the machine.
+
+## Usage
+
+Point it at a JSONL cache log — one event per line:
+
+```jsonc
+{"ts": "2026-10-11T01:00:01Z", "prompt": "What is the capital of France?",
+ "cache_key": "k001", "hit": false, "miss_reason": "no_key",
+ "threshold": 0.9, "ttl_seconds": 3600}
+{"ts": "2026-10-11T01:00:22Z", "prompt": "Capital of France?",
+ "cache_key": "k001", "hit": true, "similarity": 0.91,
+ "threshold": 0.9, "ttl_seconds": 3600, "cache_age_seconds": 60}
+```
+
+```bash
+# Full health report + suggestions
+cache-doctor analyze cache-log.jsonl
+
+# Machine-readable metrics
+cache-doctor analyze cache-log.jsonl --json
+
+# Suggestions only (from a log, or from saved --state)
+cache-doctor suggest --log cache-log.jsonl
+cache-doctor analyze cache-log.jsonl --state state.json
+cache-doctor suggest --state state.json
+```
+
+### What it reports
+
+- **Hit rate** — overall plus a rolling hourly trend, so you can see *when* it degraded.
+- **Miss reasons** — `no_key` / `expired` / `low_similarity`, with expired-miss share.
+- **Pollution signals** — borderline hits (similarity within +0.05 of threshold),
+  ground-truth wrong hits (`flag_wrong: true`), and near misses.
+- **Key fragmentation** — the same normalized prompt stored under multiple keys
+  (a prompt-normalization bug wasting storage and diluting hits).
+- **Stale serving** — hits served from entries past 90% of their TTL.
+- **Suggestions** — severity-ranked fixes (`POLLUTION_CONFIRMED`,
+  `THRESHOLD_STRICT`, `EXPIRY_WASTE`, `KEY_FRAGMENTATION`, `STALE_SERVING`, …).
+
+### Log schema
+
+| Field | Required | Meaning |
+|---|---|---|
+| `prompt` | yes | Full prompt text |
+| `cache_key` | no | The key used for lookup |
+| `hit` | yes | Was it served from cache? |
+| `similarity` | no | Similarity score of the hit / best candidate |
+| `threshold` | no | Configured similarity threshold (default 0.9) |
+| `ttl_seconds` | no | TTL of the entry |
+| `cache_age_seconds` | no | Age of the served entry (hits) |
+| `miss_reason` | no | `no_key`, `expired`, `low_similarity` |
+| `flag_wrong` | no | Ground truth: the hit was a false positive |
+| `ts` | no | ISO-8601 timestamp (enables hourly trend) |
+
+## Example
+
+```
+$ cache-doctor analyze examples/sample-log.jsonl
+
+cache-doctor  health report
+============================================
+events analyzed :         16
+cache hits      :          6     37.5%  [###########-------------------]
+cache misses    :         10
+similarity thr. :        0.9
+
+miss reasons:
+  expired             2    20.0%
+  low_similarity      3    30.0%
+  no_key              5    50.0%
+
+pollution signals:
+  borderline hits :      3  ( 50.0% of hits)
+  wrong hits      :      1   <- flag_wrong=true
+  near misses     :      3  ( 30.0% of misses)
+...
+```
+
+## Development
+
+```bash
+python3 -m unittest discover -s tests -p 'test_*.py'
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

cache_doctor-0.1.0/README.md

@@ -0,0 +1,120 @@
+# cache-doctor
+
+The checkup your semantic cache never had.
+
+LLM gateways all ship semantic caches now — but every tool on the market only
+*operates* the cache (store, lookup, evict). Nobody diagnoses it. **cache-doctor
+is the first CLI that performs a health check on a semantic cache**: it reads
+your cache event log and reports hit rate, expiry waste, key fragmentation, and
+— the silent killer — **cache pollution**, where a lightly rewritten prompt
+matches a cached response it should never have matched.
+
+## Differentiation
+
+| | Cache products (Redis, GPTCache, LangChain caches, …) | cache-doctor |
+|---|---|---|
+| Job | Store and serve cached LLM responses | **Diagnose** the cache |
+| Hit rate | You compute it yourself | Measured, with rolling hourly trend |
+| Expiry | Configured once, forgotten | Expired-miss share + stale-serving detection |
+| Pollution | Invisible | Borderline-hit risk + ground-truth `flag_wrong` detection |
+| Key design | Whatever you happened to key on | Fragmentation detection (same prompt, many keys) |
+| Advice | None | Prioritized, severity-ranked suggestions |
+
+Think of it as the relationship between a database and a database health
+monitor: the cache is the patient, cache-doctor is the checkup instrument.
+
+## Install
+
+```bash
+pip install cache-doctor
+```
+
+Zero dependencies, stdlib only. Python ≥ 3.9. Works fully offline — your
+prompts never leave the machine.
+
+## Usage
+
+Point it at a JSONL cache log — one event per line:
+
+```jsonc
+{"ts": "2026-10-11T01:00:01Z", "prompt": "What is the capital of France?",
+ "cache_key": "k001", "hit": false, "miss_reason": "no_key",
+ "threshold": 0.9, "ttl_seconds": 3600}
+{"ts": "2026-10-11T01:00:22Z", "prompt": "Capital of France?",
+ "cache_key": "k001", "hit": true, "similarity": 0.91,
+ "threshold": 0.9, "ttl_seconds": 3600, "cache_age_seconds": 60}
+```
+
+```bash
+# Full health report + suggestions
+cache-doctor analyze cache-log.jsonl
+
+# Machine-readable metrics
+cache-doctor analyze cache-log.jsonl --json
+
+# Suggestions only (from a log, or from saved --state)
+cache-doctor suggest --log cache-log.jsonl
+cache-doctor analyze cache-log.jsonl --state state.json
+cache-doctor suggest --state state.json
+```
+
+### What it reports
+
+- **Hit rate** — overall plus a rolling hourly trend, so you can see *when* it degraded.
+- **Miss reasons** — `no_key` / `expired` / `low_similarity`, with expired-miss share.
+- **Pollution signals** — borderline hits (similarity within +0.05 of threshold),
+  ground-truth wrong hits (`flag_wrong: true`), and near misses.
+- **Key fragmentation** — the same normalized prompt stored under multiple keys
+  (a prompt-normalization bug wasting storage and diluting hits).
+- **Stale serving** — hits served from entries past 90% of their TTL.
+- **Suggestions** — severity-ranked fixes (`POLLUTION_CONFIRMED`,
+  `THRESHOLD_STRICT`, `EXPIRY_WASTE`, `KEY_FRAGMENTATION`, `STALE_SERVING`, …).
+
+### Log schema
+
+| Field | Required | Meaning |
+|---|---|---|
+| `prompt` | yes | Full prompt text |
+| `cache_key` | no | The key used for lookup |
+| `hit` | yes | Was it served from cache? |
+| `similarity` | no | Similarity score of the hit / best candidate |
+| `threshold` | no | Configured similarity threshold (default 0.9) |
+| `ttl_seconds` | no | TTL of the entry |
+| `cache_age_seconds` | no | Age of the served entry (hits) |
+| `miss_reason` | no | `no_key`, `expired`, `low_similarity` |
+| `flag_wrong` | no | Ground truth: the hit was a false positive |
+| `ts` | no | ISO-8601 timestamp (enables hourly trend) |
+
+## Example
+
+```
+$ cache-doctor analyze examples/sample-log.jsonl
+
+cache-doctor  health report
+============================================
+events analyzed :         16
+cache hits      :          6     37.5%  [###########-------------------]
+cache misses    :         10
+similarity thr. :        0.9
+
+miss reasons:
+  expired             2    20.0%
+  low_similarity      3    30.0%
+  no_key              5    50.0%
+
+pollution signals:
+  borderline hits :      3  ( 50.0% of hits)
+  wrong hits      :      1   <- flag_wrong=true
+  near misses     :      3  ( 30.0% of misses)
+...
+```
+
+## Development
+
+```bash
+python3 -m unittest discover -s tests -p 'test_*.py'
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

cache_doctor-0.1.0/pyproject.toml

@@ -0,0 +1,31 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "cache-doctor"
+version = "0.1.0"
+description = "Semantic-cache health diagnostics: hit rate, expiry, and cache-pollution analysis for LLM response caches"
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+authors = [{ name = "hao li" }]
+keywords = ["llm", "semantic-cache", "cache", "diagnostics", "prompt-caching"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+
+[project.urls]
+Homepage = "https://github.com/hahahahahahahahah6/cache-doctor"
+Repository = "https://github.com/hahahahahahahahah6/cache-doctor"
+Issues = "https://github.com/hahahahahahahahah6/cache-doctor/issues"
+
+[project.scripts]
+cache-doctor = "cache_doctor.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]

cache_doctor-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

cache_doctor-0.1.0/src/cache_doctor/__init__.py

@@ -0,0 +1,4 @@
+"""cache-doctor: semantic-cache health diagnostics."""
+
+__version__ = "0.1.0"
+__all__ = ["__version__"]

cache_doctor-0.1.0/src/cache_doctor/analyze.py

@@ -0,0 +1,169 @@
+"""Parse semantic-cache event logs and compute health metrics.
+
+Log format: JSONL, one event per line. Fields (all except ``prompt``
+optional)::
+
+    {
+      "ts": "2026-10-11T01:00:00Z",   # ISO-8601 timestamp
+      "prompt": "What is the capital of France?",
+      "cache_key": "9f3a...",
+      "hit": true,                    # served from cache?
+      "similarity": 0.93,             # similarity score of the hit, if any
+      "threshold": 0.90,              # configured similarity threshold
+      "ttl_seconds": 3600,
+      "cache_age_seconds": 1200,      # age of the entry served (hits only)
+      "miss_reason": "no_key",        # "no_key" | "expired" | "low_similarity"
+      "flag_wrong": false             # ground truth: the hit was a false positive
+    }
+
+Everything is stdlib-only.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+from collections import Counter, defaultdict
+from datetime import datetime
+from typing import Any
+
+MISS_REASONS = ("no_key", "expired", "low_similarity")
+BORDERLINE_MARGIN = 0.05  # hits within +margin of threshold are "borderline"
+STALE_FRACTION = 0.9      # hits served past 90% of TTL are "stale"
+
+_WS_RE = re.compile(r"\s+")
+
+
+def norm_prompt(prompt: str) -> str:
+    """Canonical prompt form used for key-fragmentation detection."""
+    return _WS_RE.sub(" ", (prompt or "").strip()).casefold()
+
+
+def _parse_ts(value: Any) -> datetime | None:
+    if not value:
+        return None
+    try:
+        return datetime.fromisoformat(str(value).replace("Z", "+00:00"))
+    except ValueError:
+        return None
+
+
+def parse_log(path: str) -> list[dict]:
+    """Read a JSONL cache log. Raises ValueError on malformed lines."""
+    events: list[dict] = []
+    with open(path, "r", encoding="utf-8") as fh:
+        for lineno, line in enumerate(fh, start=1):
+            line = line.strip()
+            if not line:
+                continue
+            try:
+                obj = json.loads(line)
+            except json.JSONDecodeError as exc:
+                raise ValueError(f"line {lineno}: invalid JSON ({exc.msg})") from exc
+            if not isinstance(obj, dict):
+                raise ValueError(f"line {lineno}: expected a JSON object")
+            events.append(obj)
+    return events
+
+
+def _bucket_hour(ts: datetime | None) -> str:
+    return ts.strftime("%Y-%m-%dT%H:00") if ts else "unknown"
+
+
+def compute_metrics(events: list[dict]) -> dict:
+    """Compute the full health-metrics dict for a list of cache events."""
+    total = len(events)
+    hits = [e for e in events if e.get("hit")]
+    misses = [e for e in events if not e.get("hit")]
+    hit_count = len(hits)
+    miss_count = len(misses)
+
+    miss_reasons: Counter = Counter()
+    for e in misses:
+        reason = e.get("miss_reason") or "no_key"
+        miss_reasons[reason if reason in MISS_REASONS else "no_key"] += 1
+
+    thresholds = [e["threshold"] for e in events
+                  if isinstance(e.get("threshold"), (int, float))]
+    threshold = (sorted(thresholds)[len(thresholds) // 2]
+                 if thresholds else 0.9)
+
+    hit_sims = [e["similarity"] for e in hits
+                if isinstance(e.get("similarity"), (int, float))]
+    miss_sims = [e["similarity"] for e in misses
+                 if isinstance(e.get("similarity"), (int, float))]
+
+    # Borderline hits: barely above the threshold -> false-positive risk.
+    borderline_hits = [e for e in hits
+                       if isinstance(e.get("similarity"), (int, float))
+                       and e["similarity"] < threshold + BORDERLINE_MARGIN]
+
+    # Ground-truth confirmed pollution: paraphrase served the wrong response.
+    wrong_hits = [e for e in hits if e.get("flag_wrong")]
+
+    # Key fragmentation: the same normalized prompt stored under many keys.
+    by_prompt: dict[str, set] = defaultdict(set)
+    for e in events:
+        p = e.get("prompt")
+        k = e.get("cache_key")
+        if p and k:
+            by_prompt[norm_prompt(p)].add(str(k))
+    frag_groups = {p: sorted(keys) for p, keys in by_prompt.items()
+                   if len(keys) > 1}
+    frag_requests = sum(1 for e in events
+                        if e.get("prompt") and e.get("cache_key")
+                        and norm_prompt(e["prompt"]) in frag_groups)
+
+    # Stale serving: hits from entries that were nearly expired.
+    stale_hits = [e for e in hits
+                  if isinstance(e.get("cache_age_seconds"), (int, float))
+                  and isinstance(e.get("ttl_seconds"), (int, float))
+                  and e["ttl_seconds"] > 0
+                  and e["cache_age_seconds"] >= STALE_FRACTION * e["ttl_seconds"]]
+
+    # Rolling hit rate per hour (chronological).
+    buckets: dict[str, list[int]] = defaultdict(lambda: [0, 0])
+    for e in events:
+        b = _bucket_hour(_parse_ts(e.get("ts")))
+        buckets[b][0] += 1
+        buckets[b][1] += 1 if e.get("hit") else 0
+    rolling = {b: round(h / t, 4) for b, (t, h) in sorted(buckets.items())}
+
+    # Near misses: misses whose best similarity almost reached the threshold.
+    near_misses = [e for e in misses
+                   if isinstance(e.get("similarity"), (int, float))
+                   and threshold - 0.05 <= e["similarity"] < threshold]
+
+    expired_misses = miss_reasons.get("expired", 0)
+
+    def _pctl(values: list[float], q: float) -> float | None:
+        if not values:
+            return None
+        s = sorted(values)
+        return round(s[min(len(s) - 1, int(q * len(s)))], 4)
+
+    return {
+        "total": total,
+        "hits": hit_count,
+        "misses": miss_count,
+        "hit_rate": round(hit_count / total, 4) if total else 0.0,
+        "miss_reasons": dict(miss_reasons),
+        "expired_misses": expired_misses,
+        "expired_miss_share": round(expired_misses / miss_count, 4) if miss_count else 0.0,
+        "threshold": round(threshold, 4),
+        "borderline_hits": len(borderline_hits),
+        "borderline_share": round(len(borderline_hits) / hit_count, 4) if hit_count else 0.0,
+        "wrong_hits": len(wrong_hits),
+        "near_misses": len(near_misses),
+        "near_miss_share": round(len(near_misses) / miss_count, 4) if miss_count else 0.0,
+        "frag_groups": len(frag_groups),
+        "frag_requests": frag_requests,
+        "frag_share": round(frag_requests / total, 4) if total else 0.0,
+        "stale_hits": len(stale_hits),
+        "stale_share": round(len(stale_hits) / hit_count, 4) if hit_count else 0.0,
+        "avg_sim_hit": round(sum(hit_sims) / len(hit_sims), 4) if hit_sims else None,
+        "p10_sim_hit": _pctl(hit_sims, 0.10),
+        "avg_sim_miss": round(sum(miss_sims) / len(miss_sims), 4) if miss_sims else None,
+        "rolling_hourly": rolling,
+        "events_analyzed": total,
+    }

cache_doctor-0.1.0/src/cache_doctor/cli.py

@@ -0,0 +1,105 @@
+"""CLI entry point for cache-doctor."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+
+from . import __version__
+from .analyze import compute_metrics, parse_log
+from .report import render_metrics, render_suggestions
+from .suggest import suggest
+
+
+def _cmd_analyze(args: argparse.Namespace) -> int:
+    try:
+        events = parse_log(args.log)
+    except (OSError, ValueError) as exc:
+        print(f"cache-doctor: error: {exc}", file=sys.stderr)
+        return 2
+    metrics = compute_metrics(events)
+    if args.json:
+        print(json.dumps(metrics, indent=2))
+    else:
+        print(render_metrics(metrics))
+        print()
+        print(render_suggestions(suggest(metrics)))
+    if args.state:
+        try:
+            with open(args.state, "w", encoding="utf-8") as fh:
+                json.dump(metrics, fh, indent=2)
+        except OSError as exc:
+            print(f"cache-doctor: warning: cannot write state: {exc}",
+                  file=sys.stderr)
+    return 0
+
+
+def _load_metrics_for_suggest(args: argparse.Namespace) -> dict | None:
+    if args.log:
+        try:
+            return compute_metrics(parse_log(args.log))
+        except (OSError, ValueError) as exc:
+            print(f"cache-doctor: error: {exc}", file=sys.stderr)
+            return None
+    if args.state:
+        try:
+            with open(args.state, encoding="utf-8") as fh:
+                data = json.load(fh)
+            if not isinstance(data, dict):
+                raise ValueError("state file must contain a JSON object")
+            return data
+        except (OSError, ValueError) as exc:
+            print(f"cache-doctor: error: {exc}", file=sys.stderr)
+            return None
+    print("cache-doctor: error: suggest needs --log or --state",
+          file=sys.stderr)
+    return None
+
+
+def _cmd_suggest(args: argparse.Namespace) -> int:
+    metrics = _load_metrics_for_suggest(args)
+    if metrics is None:
+        return 2
+    out = suggest(metrics)
+    if args.json:
+        print(json.dumps(out, indent=2))
+    else:
+        print(render_suggestions(out))
+    return 0
+
+
+def build_parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(
+        prog="cache-doctor",
+        description="Semantic-cache health diagnostics: hit rate, expiry, "
+                    "and cache-pollution analysis with optimization advice.")
+    p.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
+
+    sub = p.add_subparsers(dest="command", required=True)
+
+    a = sub.add_parser("analyze", help="Analyze a cache event log (JSONL).")
+    a.add_argument("log", help="Path to the JSONL cache log.")
+    a.add_argument("--json", action="store_true",
+                   help="Emit metrics as JSON instead of a text report.")
+    a.add_argument("--state", default=None, metavar="PATH",
+                   help="Also save metrics JSON to PATH for later `suggest`.")
+    a.set_defaults(func=_cmd_analyze)
+
+    s = sub.add_parser("suggest", help="Show optimization suggestions.")
+    s.add_argument("--log", default=None, help="Path to the JSONL cache log.")
+    s.add_argument("--state", default=None,
+                   help="Path to a metrics JSON saved by `analyze --state`.")
+    s.add_argument("--json", action="store_true",
+                   help="Emit suggestions as JSON.")
+    s.set_defaults(func=_cmd_suggest)
+    return p
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = build_parser().parse_args(argv)
+    return args.func(args)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

cache_doctor-0.1.0/src/cache_doctor/report.py

@@ -0,0 +1,69 @@
+"""Human-readable report rendering for cache-doctor."""
+
+from __future__ import annotations
+
+_BAR_WIDTH = 30
+
+
+def _bar(frac: float) -> str:
+    filled = int(round(max(0.0, min(1.0, frac)) * _BAR_WIDTH))
+    return "#" * filled + "-" * (_BAR_WIDTH - filled)
+
+
+def _pct(frac: float | None) -> str:
+    return f"{frac * 100:6.1f}%" if frac is not None else "     n/a"
+
+
+def render_metrics(m: dict) -> str:
+    lines = [
+        "cache-doctor  health report",
+        "=" * 44,
+        f"events analyzed : {m['total']:>10}",
+        f"cache hits      : {m['hits']:>10}   {_pct(m['hit_rate'])}  [{_bar(m['hit_rate'])}]",
+        f"cache misses    : {m['misses']:>10}",
+        f"similarity thr. : {m['threshold']:>10}",
+        "",
+        "miss reasons:",
+    ]
+    for reason, count in sorted(m.get("miss_reasons", {}).items()):
+        share = count / m["misses"] if m["misses"] else 0.0
+        lines.append(f"  {reason:<15} {count:>6}  {_pct(share)}")
+    lines += [
+        "",
+        "pollution signals:",
+        f"  borderline hits : {m['borderline_hits']:>6}  ({_pct(m['borderline_share'])} of hits)",
+        f"  wrong hits      : {m['wrong_hits']:>6}   <- flag_wrong=true",
+        f"  near misses     : {m['near_misses']:>6}  ({_pct(m['near_miss_share'])} of misses)",
+        "",
+        "policy signals:",
+        f"  expired misses  : {m['expired_misses']:>6}  ({_pct(m['expired_miss_share'])} of misses)",
+        f"  stale hits      : {m['stale_hits']:>6}  ({_pct(m['stale_share'])} of hits)",
+        f"  frag. groups    : {m['frag_groups']:>6}  ({m['frag_requests']} requests)",
+        "",
+        "similarity stats:",
+        f"  avg hit sim     : {m['avg_sim_hit'] if m['avg_sim_hit'] is not None else 'n/a':>10}",
+        f"  p10 hit sim     : {m['p10_sim_hit'] if m['p10_sim_hit'] is not None else 'n/a':>10}",
+        f"  avg miss sim    : {m['avg_sim_miss'] if m['avg_sim_miss'] is not None else 'n/a':>10}",
+    ]
+    rolling = m.get("rolling_hourly") or {}
+    if rolling:
+        lines += ["", "rolling hourly hit rate:"]
+        for hour, rate in rolling.items():
+            lines.append(f"  {hour}  {_pct(rate)}  [{_bar(rate)}]")
+    return "\n".join(lines)
+
+
+_SEV_MARK = {"critical": "!!", "high": "**", "medium": "*", "low": "-", "info": "ok"}
+
+
+def render_suggestions(suggestions: list[dict]) -> str:
+    lines = ["optimization suggestions", "=" * 44]
+    for i, s in enumerate(suggestions, 1):
+        mark = _SEV_MARK.get(s["severity"], "?")
+        lines += [
+            f"{i}. [{mark}] {s['severity'].upper():8} {s['title']}",
+            f"   {s['detail']}",
+            f"   -> {s['action']}",
+            "",
+        ]
+    return "\n".join(lines).rstrip()

cache_doctor-0.1.0/src/cache_doctor/suggest.py

@@ -0,0 +1,124 @@
+"""Rule-based optimization suggestions from cache health metrics."""
+
+from __future__ import annotations
+
+SEVERITY_ORDER = {"critical": 0, "high": 1, "medium": 2, "low": 3, "info": 4}
+
+
+def _mk(severity: str, code: str, title: str, detail: str, action: str) -> dict:
+    return {"severity": severity, "code": code, "title": title,
+            "detail": detail, "action": action}
+
+
+def suggest(metrics: dict) -> list[dict]:
+    """Return prioritized suggestions, most severe first."""
+    out: list[dict] = []
+    total = metrics.get("total", 0) or 0
+    hits = metrics.get("hits", 0) or 0
+    misses = metrics.get("misses", 0) or 0
+    hit_rate = metrics.get("hit_rate", 0.0) or 0.0
+    threshold = metrics.get("threshold", 0.9)
+
+    if total == 0:
+        return [_mk("info", "NO_DATA", "No cache events to analyze",
+                    "The log contained zero events.",
+                    "Point cache-doctor at a JSONL log with at least a few "
+                    "dozen requests for meaningful diagnostics.")]
+
+    # 1. Confirmed pollution: paraphrases served the wrong cached response.
+    wrong = metrics.get("wrong_hits", 0) or 0
+    if wrong:
+        out.append(_mk(
+            "critical", "POLLUTION_CONFIRMED",
+            f"{wrong} cache hit(s) served the WRONG response",
+            f"{wrong} hit(s) were flagged (flag_wrong=true): a lightly "
+            f"rewritten prompt matched a cached response it should not have. "
+            f"This is cache pollution - silent, wrong answers served fast.",
+            f"Raise the similarity threshold above {threshold}, require "
+            "stricter embedding agreement (e.g. rerank top-3 candidates), "
+            "and audit prompts with similarity below "
+            f"{round(threshold + 0.05, 2)}."))
+
+    # 2. Borderline hits: high false-positive risk.
+    bshare = metrics.get("borderline_share", 0.0) or 0.0
+    if bshare >= 0.10 and hits:
+        out.append(_mk(
+            "high", "POLLUTION_RISK",
+            f"{round(bshare * 100, 1)}% of hits are borderline",
+            f"{metrics.get('borderline_hits', 0)} hits landed within "
+            f"+0.05 of the threshold ({threshold}). Slight prompt rewrites "
+            "slipping through are the classic pollution vector.",
+            "Bump the threshold by 0.02-0.05 and re-run analyze to confirm "
+            "the hit rate does not collapse; consider embedding-model "
+            "upgrade if legitimate paraphrases then stop matching."))
+
+    # 3. Low hit rate overall.
+    if hit_rate < 0.35:
+        out.append(_mk(
+            "high", "LOW_HIT_RATE",
+            f"Hit rate {round(hit_rate * 100, 1)}% is low",
+            f"Only {hits} of {total} requests were served from cache. The "
+            "cache is paying storage and lookup cost for little benefit.",
+            "Check prompt normalization (whitespace/case), widen the "
+            "similarity threshold cautiously, and confirm the cache is "
+            "populated on miss (write-through)."))
+
+    # 4. Near misses: threshold may be too strict.
+    nshare = metrics.get("near_miss_share", 0.0) or 0.0
+    if nshare >= 0.15 and hit_rate < 0.6:
+        out.append(_mk(
+            "medium", "THRESHOLD_STRICT",
+            f"{round(nshare * 100, 1)}% of misses almost matched",
+            f"{metrics.get('near_misses', 0)} misses scored within 0.05 "
+            f"below the threshold ({threshold}) - likely legitimate "
+            "paraphrases being rejected.",
+            "Lower the threshold by 0.02-0.03 and watch wrong_hits; pair "
+            "with a reranker so looser matching stays safe."))
+
+    # 5. Expiry waste.
+    eshare = metrics.get("expired_miss_share", 0.0) or 0.0
+    if eshare >= 0.15 and misses:
+        out.append(_mk(
+            "medium", "EXPIRY_WASTE",
+            f"{round(eshare * 100, 1)}% of misses are expired entries",
+            f"{metrics.get('expired_misses', 0)} requests missed only "
+            "because the entry had expired - the content was cacheable "
+            "and would have been a hit.",
+            "Increase TTL (or switch to sliding TTL) for stable content; "
+            "split TTLs by content volatility if some entries go stale."))
+
+    # 6. Key fragmentation: same prompt, many keys.
+    fgroups = metrics.get("frag_groups", 0) or 0
+    if fgroups:
+        out.append(_mk(
+            "medium", "KEY_FRAGMENTATION",
+            f"{fgroups} prompt group(s) stored under multiple keys",
+            f"{metrics.get('frag_requests', 0)} requests hit distinct cache "
+            "keys for prompts that are identical after normalization - "
+            "duplicate storage and diluted hit rates.",
+            "Normalize prompts before keying: trim, collapse whitespace, "
+            "casefold, and strip volatile prefixes (timestamps, session "
+            "ids) from the cache key."))
+
+    # 7. Stale serving.
+    sshare = metrics.get("stale_share", 0.0) or 0.0
+    if sshare >= 0.10 and hits:
+        out.append(_mk(
+            "low", "STALE_SERVING",
+            f"{round(sshare * 100, 1)}% of hits served near-expiry entries",
+            f"{metrics.get('stale_hits', 0)} hits were served from entries "
+            "past 90% of their TTL - fresh on the wire, stale in spirit.",
+            "Add background refresh (stale-while-revalidate) or shorten "
+            "TTL for fast-changing content."))
+
+    # 8. All clear.
+    if not out:
+        out.append(_mk(
+            "info", "HEALTHY",
+            f"Cache looks healthy ({round(hit_rate * 100, 1)}% hit rate)",
+            f"{hits} hits / {total} requests, no pollution signals, no "
+            "fragmentation, expiry under control.",
+            "Keep monitoring; re-run after traffic mix changes or "
+            "embedding-model upgrades."))
+
+    return sorted(out, key=lambda s: SEVERITY_ORDER.get(s["severity"], 9))

cache_doctor-0.1.0/src/cache_doctor.egg-info/PKG-INFO

@@ -0,0 +1,140 @@
+Metadata-Version: 2.4
+Name: cache-doctor
+Version: 0.1.0
+Summary: Semantic-cache health diagnostics: hit rate, expiry, and cache-pollution analysis for LLM response caches
+Author: hao li
+License: MIT
+Project-URL: Homepage, https://github.com/hahahahahahahahah6/cache-doctor
+Project-URL: Repository, https://github.com/hahahahahahahahah6/cache-doctor
+Project-URL: Issues, https://github.com/hahahahahahahahah6/cache-doctor/issues
+Keywords: llm,semantic-cache,cache,diagnostics,prompt-caching
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# cache-doctor
+
+The checkup your semantic cache never had.
+
+LLM gateways all ship semantic caches now — but every tool on the market only
+*operates* the cache (store, lookup, evict). Nobody diagnoses it. **cache-doctor
+is the first CLI that performs a health check on a semantic cache**: it reads
+your cache event log and reports hit rate, expiry waste, key fragmentation, and
+— the silent killer — **cache pollution**, where a lightly rewritten prompt
+matches a cached response it should never have matched.
+
+## Differentiation
+
+| | Cache products (Redis, GPTCache, LangChain caches, …) | cache-doctor |
+|---|---|---|
+| Job | Store and serve cached LLM responses | **Diagnose** the cache |
+| Hit rate | You compute it yourself | Measured, with rolling hourly trend |
+| Expiry | Configured once, forgotten | Expired-miss share + stale-serving detection |
+| Pollution | Invisible | Borderline-hit risk + ground-truth `flag_wrong` detection |
+| Key design | Whatever you happened to key on | Fragmentation detection (same prompt, many keys) |
+| Advice | None | Prioritized, severity-ranked suggestions |
+
+Think of it as the relationship between a database and a database health
+monitor: the cache is the patient, cache-doctor is the checkup instrument.
+
+## Install
+
+```bash
+pip install cache-doctor
+```
+
+Zero dependencies, stdlib only. Python ≥ 3.9. Works fully offline — your
+prompts never leave the machine.
+
+## Usage
+
+Point it at a JSONL cache log — one event per line:
+
+```jsonc
+{"ts": "2026-10-11T01:00:01Z", "prompt": "What is the capital of France?",
+ "cache_key": "k001", "hit": false, "miss_reason": "no_key",
+ "threshold": 0.9, "ttl_seconds": 3600}
+{"ts": "2026-10-11T01:00:22Z", "prompt": "Capital of France?",
+ "cache_key": "k001", "hit": true, "similarity": 0.91,
+ "threshold": 0.9, "ttl_seconds": 3600, "cache_age_seconds": 60}
+```
+
+```bash
+# Full health report + suggestions
+cache-doctor analyze cache-log.jsonl
+
+# Machine-readable metrics
+cache-doctor analyze cache-log.jsonl --json
+
+# Suggestions only (from a log, or from saved --state)
+cache-doctor suggest --log cache-log.jsonl
+cache-doctor analyze cache-log.jsonl --state state.json
+cache-doctor suggest --state state.json
+```
+
+### What it reports
+
+- **Hit rate** — overall plus a rolling hourly trend, so you can see *when* it degraded.
+- **Miss reasons** — `no_key` / `expired` / `low_similarity`, with expired-miss share.
+- **Pollution signals** — borderline hits (similarity within +0.05 of threshold),
+  ground-truth wrong hits (`flag_wrong: true`), and near misses.
+- **Key fragmentation** — the same normalized prompt stored under multiple keys
+  (a prompt-normalization bug wasting storage and diluting hits).
+- **Stale serving** — hits served from entries past 90% of their TTL.
+- **Suggestions** — severity-ranked fixes (`POLLUTION_CONFIRMED`,
+  `THRESHOLD_STRICT`, `EXPIRY_WASTE`, `KEY_FRAGMENTATION`, `STALE_SERVING`, …).
+
+### Log schema
+
+| Field | Required | Meaning |
+|---|---|---|
+| `prompt` | yes | Full prompt text |
+| `cache_key` | no | The key used for lookup |
+| `hit` | yes | Was it served from cache? |
+| `similarity` | no | Similarity score of the hit / best candidate |
+| `threshold` | no | Configured similarity threshold (default 0.9) |
+| `ttl_seconds` | no | TTL of the entry |
+| `cache_age_seconds` | no | Age of the served entry (hits) |
+| `miss_reason` | no | `no_key`, `expired`, `low_similarity` |
+| `flag_wrong` | no | Ground truth: the hit was a false positive |
+| `ts` | no | ISO-8601 timestamp (enables hourly trend) |
+
+## Example
+
+```
+$ cache-doctor analyze examples/sample-log.jsonl
+
+cache-doctor  health report
+============================================
+events analyzed :         16
+cache hits      :          6     37.5%  [###########-------------------]
+cache misses    :         10
+similarity thr. :        0.9
+
+miss reasons:
+  expired             2    20.0%
+  low_similarity      3    30.0%
+  no_key              5    50.0%
+
+pollution signals:
+  borderline hits :      3  ( 50.0% of hits)
+  wrong hits      :      1   <- flag_wrong=true
+  near misses     :      3  ( 30.0% of misses)
+...
+```
+
+## Development
+
+```bash
+python3 -m unittest discover -s tests -p 'test_*.py'
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

cache_doctor-0.1.0/src/cache_doctor.egg-info/SOURCES.txt

@@ -0,0 +1,14 @@
+LICENSE
+README.md
+pyproject.toml
+src/cache_doctor/__init__.py
+src/cache_doctor/analyze.py
+src/cache_doctor/cli.py
+src/cache_doctor/report.py
+src/cache_doctor/suggest.py
+src/cache_doctor.egg-info/PKG-INFO
+src/cache_doctor.egg-info/SOURCES.txt
+src/cache_doctor.egg-info/dependency_links.txt
+src/cache_doctor.egg-info/entry_points.txt
+src/cache_doctor.egg-info/top_level.txt
+tests/test_smoke.py

cache_doctor-0.1.0/src/cache_doctor.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

cache_doctor-0.1.0/src/cache_doctor.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+cache-doctor = cache_doctor.cli:main

cache_doctor-0.1.0/src/cache_doctor.egg-info/top_level.txt

@@ -0,0 +1 @@
+cache_doctor

cache_doctor-0.1.0/tests/test_smoke.py

@@ -0,0 +1,198 @@
+"""Smoke tests for cache-doctor (stdlib unittest only)."""
+
+import io
+import json
+import os
+import sys
+import tempfile
+import unittest
+from contextlib import redirect_stdout, redirect_stderr
+
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "src"))
+
+from cache_doctor.analyze import compute_metrics, norm_prompt, parse_log
+from cache_doctor.cli import main
+from cache_doctor.suggest import suggest
+
+
+def ev(**kw):
+    base = {"ts": "2026-10-11T01:00:00Z", "threshold": 0.9,
+            "ttl_seconds": 3600}
+    base.update(kw)
+    return base
+
+
+def write_jsonl(events):
+    fd, path = tempfile.mkstemp(suffix=".jsonl")
+    with os.fdopen(fd, "w", encoding="utf-8") as fh:
+        for e in events:
+            fh.write(json.dumps(e) + "\n")
+    return path
+
+
+class TestParse(unittest.TestCase):
+    def test_parse_ok(self):
+        path = write_jsonl([ev(prompt="a", hit=False), ev(prompt="b", hit=True)])
+        try:
+            self.assertEqual(len(parse_log(path)), 2)
+        finally:
+            os.unlink(path)
+
+    def test_parse_bad_json_raises(self):
+        fd, path = tempfile.mkstemp(suffix=".jsonl")
+        with os.fdopen(fd, "w") as fh:
+            fh.write('{"ok": 1}\nnot json\n')
+        try:
+            with self.assertRaises(ValueError):
+                parse_log(path)
+        finally:
+            os.unlink(path)
+
+    def test_parse_empty_file(self):
+        fd, path = tempfile.mkstemp(suffix=".jsonl")
+        os.close(fd)
+        try:
+            self.assertEqual(parse_log(path), [])
+        finally:
+            os.unlink(path)
+
+
+class TestMetrics(unittest.TestCase):
+    def test_hit_rate(self):
+        events = [ev(hit=True, similarity=0.95) for _ in range(3)]
+        events += [ev(hit=False, miss_reason="no_key") for _ in range(2)]
+        m = compute_metrics(events)
+        self.assertEqual(m["total"], 5)
+        self.assertEqual(m["hit_rate"], 0.6)
+        self.assertEqual(m["miss_reasons"]["no_key"], 2)
+
+    def test_borderline_hits_detected(self):
+        events = [ev(hit=True, similarity=0.92)]  # within +0.05 of 0.9
+        m = compute_metrics(events)
+        self.assertEqual(m["borderline_hits"], 1)
+        self.assertEqual(m["borderline_share"], 1.0)
+
+    def test_wrong_hits_detected(self):
+        events = [ev(hit=True, similarity=0.95, flag_wrong=True)]
+        m = compute_metrics(events)
+        self.assertEqual(m["wrong_hits"], 1)
+
+    def test_expiry_share(self):
+        events = [ev(hit=False, miss_reason="expired") for _ in range(3)]
+        events += [ev(hit=False, miss_reason="no_key")]
+        m = compute_metrics(events)
+        self.assertEqual(m["expired_miss_share"], 0.75)
+
+    def test_key_fragmentation(self):
+        events = [
+            ev(prompt="What is AI?", cache_key="k1", hit=False),
+            ev(prompt="what  is ai?", cache_key="k2", hit=False),
+            ev(prompt="unrelated", cache_key="k9", hit=False),
+        ]
+        m = compute_metrics(events)
+        self.assertEqual(m["frag_groups"], 1)
+        self.assertEqual(m["frag_requests"], 2)
+
+    def test_stale_hits(self):
+        events = [ev(hit=True, similarity=0.95,
+                     cache_age_seconds=3500, ttl_seconds=3600)]
+        m = compute_metrics(events)
+        self.assertEqual(m["stale_hits"], 1)
+
+    def test_empty_events(self):
+        m = compute_metrics([])
+        self.assertEqual(m["total"], 0)
+        self.assertEqual(m["hit_rate"], 0.0)
+
+    def test_norm_prompt(self):
+        self.assertEqual(norm_prompt("  What IS  AI? "), "what is ai?")
+
+
+class TestSuggest(unittest.TestCase):
+    def test_pollution_is_critical(self):
+        m = compute_metrics([ev(hit=True, similarity=0.93, flag_wrong=True)])
+        s = suggest(m)
+        self.assertEqual(s[0]["code"], "POLLUTION_CONFIRMED")
+        self.assertEqual(s[0]["severity"], "critical")
+
+    def test_low_hit_rate_suggests(self):
+        events = [ev(hit=False, miss_reason="no_key") for _ in range(9)]
+        events += [ev(hit=True, similarity=0.95)]
+        s = suggest(compute_metrics(events))
+        self.assertTrue(any(x["code"] == "LOW_HIT_RATE" for x in s))
+
+    def test_healthy_cache_reports_info(self):
+        events = [ev(hit=True, similarity=0.99) for _ in range(9)]
+        events += [ev(hit=False, miss_reason="no_key", similarity=0.5)]
+        s = suggest(compute_metrics(events))
+        self.assertEqual(s, [s[0]])
+        self.assertEqual(s[0]["code"], "HEALTHY")
+
+    def test_expiry_waste_suggests(self):
+        events = [ev(hit=False, miss_reason="expired") for _ in range(4)]
+        events += [ev(hit=False, miss_reason="no_key")]
+        s = suggest(compute_metrics(events))
+        self.assertTrue(any(x["code"] == "EXPIRY_WASTE" for x in s))
+
+    def test_empty_metrics(self):
+        s = suggest(compute_metrics([]))
+        self.assertEqual(s[0]["code"], "NO_DATA")
+
+
+class TestCLI(unittest.TestCase):
+    def test_analyze_runs(self):
+        path = write_jsonl([ev(prompt="a", hit=True, similarity=0.95),
+                            ev(prompt="b", hit=False, miss_reason="no_key")])
+        buf = io.StringIO()
+        try:
+            with redirect_stdout(buf):
+                rc = main(["analyze", path])
+        finally:
+            os.unlink(path)
+        self.assertEqual(rc, 0)
+        self.assertIn("hit rate", buf.getvalue())
+
+    def test_analyze_json_output(self):
+        path = write_jsonl([ev(prompt="a", hit=True, similarity=0.95)])
+        buf = io.StringIO()
+        try:
+            with redirect_stdout(buf):
+                rc = main(["analyze", path, "--json"])
+        finally:
+            os.unlink(path)
+        self.assertEqual(rc, 0)
+        data = json.loads(buf.getvalue())
+        self.assertEqual(data["total"], 1)
+
+    def test_suggest_from_log(self):
+        path = write_jsonl([ev(prompt="a", hit=False, miss_reason="no_key")])
+        buf = io.StringIO()
+        try:
+            with redirect_stdout(buf):
+                rc = main(["suggest", "--log", path, "--json"])
+        finally:
+            os.unlink(path)
+        self.assertEqual(rc, 0)
+        codes = [s["code"] for s in json.loads(buf.getvalue())]
+        self.assertIn("LOW_HIT_RATE", codes)
+
+    def test_analyze_missing_file_errors(self):
+        err = io.StringIO()
+        with redirect_stderr(err):
+            rc = main(["analyze", "/nonexistent/file.jsonl"])
+        self.assertEqual(rc, 2)
+
+    def test_sample_log_end_to_end(self):
+        sample = os.path.join(os.path.dirname(__file__), "..",
+                              "examples", "sample-log.jsonl")
+        buf = io.StringIO()
+        with redirect_stdout(buf):
+            rc = main(["analyze", os.path.abspath(sample)])
+        self.assertEqual(rc, 0)
+        out = buf.getvalue()
+        self.assertIn("pollution signals", out)  # the flag_wrong event
+        self.assertIn("frag. groups", out)       # capital-of-france k001..k003
+
+
+if __name__ == "__main__":
+    unittest.main(verbosity=2)