tokenmizer 0.3.0__tar.gz → 0.3.2__tar.gz

{tokenmizer-0.3.0 → tokenmizer-0.3.2}/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "tokenmizer",
-  "version": "0.2.3",
+  "version": "0.3.2",
   "description": "Never lose your AI context again. Graph-backed memory, session checkpointing, and file intelligence for Claude Code and any LLM.",
   "author": {
     "name": "Shweta Mishra",

{tokenmizer-0.3.0 → tokenmizer-0.3.2}/.github/workflows/ci.yml

@@ -8,10 +8,22 @@ on:
 
 jobs:
   test:
-    runs-on: ubuntu-latest
+    # Windows is in the matrix because the 2026-07-10 audit found THREE
+    # Windows-only bugs Linux CI could never catch: a cp1252
+    # UnicodeEncodeError crashing `tokenmizer --help`, a leaked SQLite
+    # handle blocking corrupt-DB recovery (WinError 32), and console
+    # encoding issues in scripts. One Windows leg (latest Python only —
+    # Windows runners are slow) keeps that whole bug class from ever
+    # shipping again.
+    runs-on: ${{ matrix.os }}
     strategy:
+      fail-fast: false
       matrix:
+        os: [ubuntu-latest]
         python-version: ["3.10", "3.11", "3.12"]
+        include:
+          - os: windows-latest
+            python-version: "3.12"
 
     steps:
       - uses: actions/checkout@v4
@@ -20,14 +32,7 @@ jobs:
         uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
-
-      - name: Cache pip
-        uses: actions/cache@v4
-        with:
-          path: ~/.cache/pip
-          key: ${{ runner.os }}-pip-${{ hashFiles('pyproject.toml') }}
-          restore-keys: |
-            ${{ runner.os }}-pip-
+          cache: pip
 
       - name: Install dependencies
         run: |
@@ -40,16 +45,21 @@ jobs:
       # Coverage threshold lives in pyproject.toml [tool.coverage.report]
       # fail_under — single source of truth, do NOT override it here.
       - name: Run tests with coverage
-        run: |
-          pytest tests/ \
-            --cov=tokenmizer \
-            --cov-report=term-missing \
-            --cov-report=xml \
-            -v
+        run: pytest tests/ --cov=tokenmizer --cov-report=term-missing --cov-report=xml -v
+
+      # Regression guard for the cp1252 crash: --help must work on a
+      # non-UTF-8 Windows console, not just under pytest's captured IO.
+      - name: CLI smoke test (Windows encoding regression)
+        if: runner.os == 'Windows'
+        run: python -m tokenmizer.cli --help
+
+      - name: MCP stdio e2e
+        if: matrix.python-version == '3.12'
+        run: python scripts/mcp_e2e_check.py
 
       - name: Upload coverage
         uses: codecov/codecov-action@v4
-        if: matrix.python-version == '3.12'
+        if: matrix.python-version == '3.12' && runner.os == 'Linux'
         with:
           file: ./coverage.xml
         continue-on-error: true

{tokenmizer-0.3.0 → tokenmizer-0.3.2}/.github/workflows/release.yml

@@ -24,6 +24,20 @@ jobs:
         with:
           python-version: "3.12"
 
+      # The release tag (v0.3.2) must match the package version — otherwise
+      # a stale tag publishes the wrong code under the wrong number. Version
+      # pins across server.json / plugin.json / app.py are enforced by
+      # tests/unit/test_version_consistency.py in the test run below.
+      - name: Verify tag matches package version
+        run: |
+          pip install -e .
+          PKG_VERSION=$(python -c "import tokenmizer; print(tokenmizer.__version__)")
+          TAG_VERSION="${GITHUB_REF_NAME#v}"
+          if [ "$PKG_VERSION" != "$TAG_VERSION" ]; then
+            echo "::error::Tag $GITHUB_REF_NAME does not match package version $PKG_VERSION"
+            exit 1
+          fi
+
       - name: Run tests first — never publish a broken build
         run: |
           pip install -e ".[dev]"

{tokenmizer-0.3.0 → tokenmizer-0.3.2}/CHANGELOG.md

@@ -1,5 +1,167 @@
 # Changelog
 
+## [0.3.2] — 2026-07-10 — full-repo audit: graph memory, MCP server, visualization
+
+### Critical — the LLM extraction path never worked
+- **`api/app.py`:** the background extraction called
+  `HybridExtractor(provider_fn=_pfn)` — a kwarg `__init__` never accepted —
+  so it raised `TypeError` on EVERY call, and even without that,
+  `ext.extract(_msgs)` omitted `provider_fn`, which defaults to `None` and
+  skips the LLM pass. Net effect: **`use_llm_extraction: true` has never
+  once produced an LLM extraction** — every call raised, was caught by the
+  broad except, and logged as if it were a transient provider failure.
+  Fixed (`HybridExtractor()` + `extract(_msgs, provider_fn=_pfn)`) with a
+  regression test that replicates the exact call pattern.
+
+### Graph memory — topic classifier
+- **"Go" the verb collided with Go the language:** "Go with tRPC for the
+  API layer" classified as `language` (first-single-word-hit-wins never
+  reached "trpc"), so a later "use gRPC instead" never superseded it.
+  Bare "go" removed from language keywords; unambiguous forms kept
+  ("golang", "in go", "use go", ...).
+- **Vocabulary drift:** hybrid_extractor.py knew supabase/clerk/etc., the
+  classifier didn't — those decisions classified as None and supersession
+  silently never fired. New buckets: backend_platform, auth_provider,
+  payments, observability, state_management, package_manager, styling.
+- **Multi-topic decisions collapsed to one topic:** "Use FastAPI with
+  SQLAlchemy and PostgreSQL" returned only `web_framework`; a later
+  Postgres→SQLite switch was never detected as contradicting it. New
+  `classify_topics()` returns ALL matched topics; contradiction detection
+  is now set-intersection. Bigrams match first and consume their words
+  ("session store" no longer leaks a spurious `auth_mechanism`).
+  `classify_topic()` kept as a backward-compatible wrapper.
+- **`ARCHIVED` was unreachable:** documented, fully wired into decay/
+  prune/query logic — and nothing ever set it. The README advertised a
+  4-state model whose 4th state could not occur. SUPERSEDED decisions now
+  age into ARCHIVED after 7 days (from supersession time, not creation).
+- `/api/decision/invalidate` substring-matched across ALL decision nodes
+  regardless of status — a short label could flip already-SUPERSEDED
+  history nodes to INVALIDATED, destroying their supersession record.
+  Now only ACTIVE decisions are eligible, and the response lists exactly
+  which nodes were affected.
+
+### MCP server — hardened like a client integration
+- **`isError` was string-sniffing** (`result_text.startswith("❌")`): a
+  missing required argument produced `"Tool error: 'session_id'"` with
+  `isError: false` — MCP clients saw a *successful* result. Handlers now
+  return `(text, is_error)` structurally; missing/invalid args produce
+  typed validation errors with `isError: true`.
+- **Three server-killer inputs fixed:** an exception inside `initialize`
+  crashed the whole stdio loop with no JSON-RPC error (client hung, then
+  watched the subprocess die); a valid-JSON-but-not-an-object line
+  (`[1,2]`) raised AttributeError and killed the loop; malformed JSON was
+  silently dropped (`continue` — the request hung forever, no log, no
+  error). Per-request try/except now returns `-32603`, non-objects get
+  `-32600`, parse failures get `-32700` + a warning log.
+- **`logger` was dead code** — defined, never called once. Every caught
+  exception was invisible to the operator. `run_stdio_server()` now
+  configures stderr logging (stdout stays protocol-only), controlled by
+  `TOKENMIZER_MCP_LOG_LEVEL`.
+- Input validation: `session_id`/`file_path` presence+type, `level` enum,
+  `token_budget` positive-int (bool explicitly rejected — it's an int
+  subclass), non-dict `arguments`.
+- 18 new unit tests (`tests/unit/test_mcp_server.py`) covering all of the
+  above; `scripts/mcp_e2e_check.py` still ALL PASS.
+- `server.json` version synced (was 0.2.6, three releases stale).
+
+### Silent failures made visible (continuing the v0.2.x hardening)
+- `hybrid_extractor.llm_extract`: debug→warning (a provider outage
+  silently degraded every turn to heuristic-only).
+- `compression/engine.filter_json`: swallowed ALL exceptions with zero
+  logging (returned unfiltered content). Real failures now log at
+  warning; the benign not-JSON case stays quiet.
+- `graph._load_transitions`: DB corruption looked identical to the benign
+  first-run case (both debug). Corruption now warns; first-run stays debug.
+- `file_intelligence.detect_file_type`: sniff failure silently reclassified
+  files as "text" (worse extraction, no signal) — now warns. PDFs where
+  NO page yields text now warn once (scanned/image-only detection).
+- `CheckpointManager`: new `persistence_broken` flag — previously the
+  constructor swallowed a triple init failure and reported healthy while
+  every subsequent save was doomed. `get_latest()` now raises
+  `StorageError` on DB read failure instead of returning None ("no
+  checkpoint found, 404" vs "your checkpoints exist but are unreadable"
+  are different problems; callers can finally tell them apart).
+- **`_db_connect` leaked the connection when the WAL PRAGMA failed on a
+  corrupt DB file** (both graph.py and manager.py) — on Windows the open
+  handle blocked the documented delete-and-recreate recovery path
+  (WinError 32). Found because the chaos test failed for the RIGHT reason
+  once get_latest stopped swallowing errors.
+
+### Security — redaction gaps
+- URL-embedded credentials (`postgres://admin:pass@host/db`) matched NO
+  pattern — no `password=` literal, no recognized prefix. Now redacted
+  (credential part only; host survives for readability).
+- Added OpenRouter / Hugging Face / xAI / Together key patterns.
+- Checkpoint `next_action` (a raw 200-char message slice persisted to
+  SQLite) is now independently redacted — defense-in-depth so the
+  single-point-of-application assumption in chat_completions() is not the
+  only thing between a pasted key and the checkpoint DB.
+
+### Visualization — redesigned (was a generic node soup)
+- The old shareable HTML exported `transitions` (the supersession
+  history — the one thing this product tracks that a generic graph view
+  doesn't) and then never rendered them. It also loaded D3 from a CDN, so
+  the "self-contained" artifact broke offline.
+- New artifact (zero external deps, hand-rolled force layout):
+  supersession arcs (dashed red, old→new, arrowheads), a clickable
+  "Decision history" timeline panel (struck-through old label → new label
+  with trigger/reason/timestamp; click = spotlight both nodes + center),
+  glow rings on active decisions vs dashed rings + strikethrough on
+  superseded/archived, red rings on invalidated, per-type filter chips,
+  "Active only" toggle, text search, wheel-zoom/pan, one-click PNG export.
+- Verified in a real browser (not just unit-tested): filters, search,
+  timeline spotlight, and layout all exercised via DOM inspection.
+
+### CLI
+- `tokenmizer --help` crashed on Windows (cp1252) with UnicodeEncodeError
+  from the 🧠 emoji — found by dry-running the README quick start. Same
+  UTF-8 reconfigure fix the benchmark runners got in v0.2.4; the CLI had
+  been missed.
+
+### Round 2 — extraction-quality residuals (same audit, follow-up pass)
+- **Near-duplicate decision nodes merged instead of self-superseding:**
+  one message ("Decided: use React for the frontend.") could emit two
+  decision variants ("Use React" + "use React for the frontend.") via
+  different regex passes; they became two nodes and one superseded the
+  other — a bogus "Changed:" line in every resume. `_is_same_decision`
+  now recognizes containment (smaller label's words ⊆ larger's, min 2
+  words so "Use PostgreSQL" is NOT collapsed into "Switch from PostgreSQL
+  to SQLite"), and `add_node` fuzzy-merges same-decision variants: keeps
+  the existing node, upgrades to the longer label, backfills the summary,
+  never resurrects SUPERSEDED status. Verified end-to-end: the demo
+  scenario now produces exactly one transition (React→Next.js) and a
+  clean resume block.
+- **Validator now honors extractor corroboration confidence:** it used to
+  recompute confidence purely from label length/wording, so a doubly-
+  corroborated short decision (0.95) could be rejected while a verbose
+  weakly-sourced one passed. `validate()` gains `extractor_confidence`;
+  final = max(heuristic, (heuristic+extractor)/2) — monotone (evidence
+  only raises), not an override (heuristic-only 0.65 still fails a 0.65
+  threshold), and hard rejects remain absolute (0.95 cannot resurrect
+  junk). Wired from `add_node` via the existing confidence≠0.7 sentinel.
+- **`HybridExtractor.min_confidence` is no longer dead code:** extract()
+  now filters merged output by merge()'s confidence tiers — default 0.55
+  keeps every tier (behavior unchanged); 0.7 drops heuristic-only items;
+  0.9 keeps only corroborated ones. Decisions filter per-item, simple
+  lists per-category.
+- Test fixture fix: `test_prune_preserves_decisions` used ten decisions
+  differing only by a trailing digit — 83% word-overlap, which
+  `_is_same_decision` always considered the same decision; the merge fix
+  made that judgment consequential, collapsing the fixture. Replaced with
+  ten genuinely distinct decisions across different topic buckets.
+
+### Tests
+- 220 → 275 (55 new: MCP server 18, classifier+dedup 18, archival 3,
+  invalidate-scope 2, redaction 5, LLM-pass regression 1, validator
+  blending 4, min_confidence filter 4).
+
+## [0.3.1] — 2026-07-03 — shareable graph visualization
+
+- NEW `GET /api/graph/{session_id}/html` — self-contained dark interactive
+  force-graph page (D3): drag, zoom, glow-styled typed nodes, legend, live
+  stats header. Open in any browser, screenshot, share. Your session's
+  memory, visible.
+
 ## [0.3.0] — 2026-07-02 — true SSE streaming
 
 - **`stream: true` now works** — real passthrough streaming in OpenAI
@@ -104,7 +266,9 @@
   removed; benchmark table updated to freshly measured v0.2.4 numbers with
   date and sample-size caveat.
 
-## [Unreleased] — tokenizer, cache, and version consistency fixes
+## Shipped in [0.2.4] — tokenizer, cache, and version consistency fixes
+
+(Header fixed 2026-07-10: this section was left titled "[Unreleased]" after it shipped.)
 
 ### Correctness — core value proposition
 - **`core/tokenizer.py`:** Claude/Anthropic models were counted with tiktoken
@@ -168,7 +332,9 @@
   Quick Start code example (previously only mentioned deep in the CLI
   section) — Cursor/Continue.dev users were hitting an unexplained HTTP 501.
 
-## [Unreleased] — security/correctness audit pass
+## Shipped in [0.2.4] — security/correctness audit pass
+
+(Header fixed 2026-07-10: this section was left titled "[Unreleased]" after it shipped.)
 
 Full senior-level audit covering security, silent failures, dead code,
 and benchmark honesty. See `TESTING.md` for what's verified vs. not, and

{tokenmizer-0.3.0 → tokenmizer-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tokenmizer
-Version: 0.3.0
+Version: 0.3.2
 Summary: Reduce AI context loss by 2x. Graph-backed checkpoint and resume for any LLM session.
 Project-URL: Homepage, https://github.com/Shweta-Mishra-ai/tokenmizer
 Project-URL: Repository, https://github.com/Shweta-Mishra-ai/tokenmizer
@@ -92,6 +92,7 @@ Description-Content-Type: text/markdown
     <a href="https://registry.modelcontextprotocol.io/v0/servers?search=tokenmizer"><img src="https://img.shields.io/badge/MCP%20Registry-published-5ee7c8?style=flat-square" alt="MCP Registry"/></a>
     <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-4ade80?style=flat-square"/></a>
     <a href="https://github.com/Shweta-Mishra-ai/tokenmizer/stargazers"><img src="https://img.shields.io/github/stars/Shweta-Mishra-ai/tokenmizer?style=flat-square&color=f9d84a" alt="Stars"/></a>
+    <a href="https://glama.ai/mcp/servers/Shweta-Mishra-ai/tokenmizer"><img src="https://glama.ai/mcp/servers/Shweta-Mishra-ai/tokenmizer/badges/score.svg" alt="Glama Score"/></a>
   </p>
 
   <p>
@@ -150,35 +151,122 @@ Your App  →  TokenMizer (:8000)  →  Claude / GPT / Gemini / any LLM
 | 🟢 `ACTIVE` | Current — in effect | ✅ Always |
 | 🟡 `SUPERSEDED` | Replaced by newer decision | ⚠️ 7 days |
 | 🔴 `INVALIDATED` | Explicitly wrong/cancelled | ⚠️ Always (warning) |
-| ⬜ `ARCHIVED` | Old but valid, not relevant | ❌ Never |
+| ⬜ `ARCHIVED` | Superseded >7 days ago — aged out | ❌ Never |
 
 History is **never deleted**. "Why did we switch from React to Next.js?" — always answerable.
 
+> Honesty note (2026-07-10 audit): before v0.3.2, `ARCHIVED` was advertised
+> here but **unreachable** — the decay/prune/query logic all handled it, yet
+> no code path ever set it. Superseded decisions now age into `ARCHIVED`
+> automatically after 7 days (`GraphMemory.ARCHIVE_SUPERSEDED_AFTER_DAYS`),
+> which is what makes the "⚠️ 7 days" row above actually true.
+
 ---
 
 ## Quick Start
 
+<details>
+<summary><b>🟢 Complete step-by-step setup (start here if you're new — 5 minutes, no code reading needed)</b></summary>
+
+<br/>
+
+**Step 0 — Check Python** (need 3.10 or newer)
+
+Open a terminal (Windows: press Win, type "PowerShell", Enter · Mac: Cmd+Space, type "Terminal"):
+
+```
+python --version
+```
+
+You should see `Python 3.10` or higher. If not: install from [python.org/downloads](https://python.org/downloads) (Windows: tick **"Add Python to PATH"** during install).
+
+**Step 1 — Install TokenMizer**
+
+```
+pip install "tokenmizer[anthropic,cache]"
+```
+
+✅ You should see: `Successfully installed tokenmizer-...`
+
+**Step 2 — Add your API key** (get one at [console.anthropic.com](https://console.anthropic.com) → API Keys)
+
+Windows PowerShell:
+```powershell
+setx TOKENMIZER_ANTHROPIC_API_KEY "sk-ant-YOUR-KEY"
+```
+then **close and reopen** the terminal.
+
+Mac/Linux:
+```bash
+export TOKENMIZER_ANTHROPIC_API_KEY=sk-ant-YOUR-KEY
+```
+
+*(No key? Use free local Ollama instead — see "No API key?" below.)*
+
+**Step 3 — Start TokenMizer**
+
+```
+tokenmizer serve
+```
+
+✅ You should see: `Proxy: http://localhost:8000/v1/chat/completions`
+Leave this terminal open — TokenMizer runs here.
+
+**Step 4 — Verify it's alive**
+
+Open [http://localhost:8000](http://localhost:8000) in your browser → the TokenMizer dashboard appears. That's it — the proxy works.
+
+**Step 5 — Connect your tool** (pick yours)
+
+- **Cursor:** Settings → Models → OpenAI API → Base URL: `http://localhost:8000/v1`
+- **Claude Desktop / Claude Code:** see [Claude Code Integration](#claude-code-integration) below (copy one JSON block, restart the app)
+- **Your own Python code:** see "Use — change one line" below
+
+**Something failed?** `pip` not found → reinstall Python with "Add to PATH". Port 8000 busy → `tokenmizer serve --port 8001`. Anything else → [open an issue](https://github.com/Shweta-Mishra-ai/tokenmizer/issues) with the error text — median response < 1 day.
+
+</details>
+
 ### 1. Install
 
+Works on **Windows, macOS, and Linux** (Python 3.10+). Same command everywhere:
+
 ```bash
 # Recommended
 pip install "tokenmizer[anthropic,cache]"
 
 # All providers
 pip install "tokenmizer[anthropic,openai,gemini,cohere,cache]"
+```
+
+<details>
+<summary><b>No API key? Use Ollama (free, local)</b></summary>
+
+```bash
+# macOS:   brew install ollama
+# Windows: winget install Ollama.Ollama   (or download from ollama.com)
+# Linux:   curl -fsSL https://ollama.com/install.sh | sh
 
-# No key? Use Ollama (free, local)
-brew install ollama && ollama pull llama3
+ollama pull llama3
 pip install tokenmizer
+# then set provider: ollama in tokenmizer.yaml
 ```
+</details>
 
 ### 2. Set your API key
 
+**macOS / Linux (bash, zsh):**
 ```bash
 export TOKENMIZER_ANTHROPIC_API_KEY=sk-ant-...
-# or: TOKENMIZER_OPENAI_API_KEY, TOKENMIZER_GEMINI_API_KEY, etc.
 ```
 
+**Windows (PowerShell):**
+```powershell
+$env:TOKENMIZER_ANTHROPIC_API_KEY = "sk-ant-..."      # current session
+setx TOKENMIZER_ANTHROPIC_API_KEY "sk-ant-..."         # persistent (new terminals)
+```
+
+Other providers: `TOKENMIZER_OPENAI_API_KEY`, `TOKENMIZER_GEMINI_API_KEY`, etc. — full table in [Supported Providers](#supported-providers).
+
 ### 3. Start
 
 ```bash
@@ -233,11 +321,11 @@ Then use skills directly:
 /tokenmizer:stats                      → token savings report
 ```
 
-### Option B — MCP server
+### Option B — MCP server (Claude Desktop, Claude Code, Cursor, VS Code, Zed)
 
 mcp-name: io.github.Shweta-Mishra-ai/tokenmizer
 
-Add to `~/.claude/settings.json`:
+Add this `mcpServers` block to your client's MCP config file:
 
 ```json
 {
@@ -250,7 +338,31 @@ Add to `~/.claude/settings.json`:
 }
 ```
 
-(`tokenmizer-mcp` is installed with the package; `python3 -m tokenmizer.mcp.server` also works.)
+Where the config file lives:
+
+| Client | Config file |
+|---|---|
+| **Claude Desktop** (Windows) | `%APPDATA%\Claude\claude_desktop_config.json` |
+| **Claude Desktop** (macOS) | `~/Library/Application Support/Claude/claude_desktop_config.json` |
+| **Claude Code** | `.mcp.json` in your project, or `~/.claude/settings.json` |
+| **Cursor** | Settings → MCP → Add server (same JSON) |
+| **VS Code / Zed** | their MCP settings — same `command` + `env` |
+| **OpenAI Codex CLI** | `~/.codex/config.toml` — TOML format, see below |
+
+<details>
+<summary>Codex CLI config (TOML, not JSON)</summary>
+
+```toml
+[mcp_servers.tokenmizer]
+command = "tokenmizer-mcp"
+env = { TOKENMIZER_URL = "http://localhost:8000" }
+```
+</details>
+
+Then restart the client. Keep `tokenmizer serve` running for the
+checkpoint/resume/stats tools (file analysis works without it).
+If `tokenmizer-mcp` isn't on your PATH, use `"command": "python"`,
+`"args": ["-m", "tokenmizer.mcp.server"]` instead.
 
 ---
 
@@ -348,7 +460,13 @@ default_model: claude-sonnet-4-6
 graph_checkpoint:
   enabled: true
   trigger_at_percent: 0.85
-  use_llm_extraction: false     # true = 80%+ recall, needs key (~$0.001/turn)
+  use_llm_extraction: false     # true = hybrid LLM+heuristic extraction
+                                # (needs a provider key, ~$0.001/turn).
+                                # NOTE: before v0.3.2 this flag silently did
+                                # nothing — the call site passed provider_fn
+                                # to the wrong function and raised TypeError
+                                # on every turn, falling back to heuristics.
+                                # Fixed + regression-tested; see CHANGELOG.
 
 compression:
   enabled: true
@@ -388,6 +506,7 @@ TOKENMIZER_API_KEY=strong-key docker-compose up
 | `/api/checkpoint` | POST | Manual checkpoint |
 | `/api/decision/invalidate` | POST | Mark decision as invalid |
 | `/api/graph/{id}` | GET | Session graph stats |
+| `/api/graph/{id}/html` | GET | **Interactive graph page** — decision-history timeline, supersession arcs, type/status filters, search, zoom/pan, PNG export. Zero external dependencies (works offline) |
 | `/api/stats` | GET | Token savings analytics |
 | `/health` | GET | Health check |
 | `/docs` | GET | Swagger UI |
@@ -400,7 +519,13 @@ TOKENMIZER_API_KEY=strong-key docker-compose up
 - Secret/PII redaction applied once at ingestion, before graph storage,
   checkpoint storage, AND every LLM call (main chat *and* the background
   extraction model — these are separate, the redaction gap between them
-  was a real bug, now fixed)
+  was a real bug, now fixed). Patterns cover Anthropic/OpenAI/Google/
+  GitHub/AWS/Slack/Stripe/JWT/OpenRouter/HF/xAI keys, URL-embedded
+  credentials (`postgres://user:pass@host` — a 2026-07 audit gap, fixed),
+  and generic `key=`/`password=` assignments. Best-effort by nature —
+  an unrecognized format with no keyword context can still slip through.
+  The checkpoint layer independently re-redacts what it persists
+  (defense-in-depth, added in the same audit).
 - Session-isolated cache (sensitive data never shared across sessions)
 - Basic prompt-injection keyword filter — catches copy-pasted jailbreak
   templates only; **not** a security boundary against a motivated
@@ -534,7 +659,7 @@ Contributions welcome — this project merges fast (median PR review < 1 day).
 git clone https://github.com/Shweta-Mishra-ai/tokenmizer
 cd tokenmizer
 pip install -e ".[dev]"
-pytest tests/ -v && ruff check tokenmizer/     # 218 tests, must stay green
+pytest tests/ -v && ruff check tokenmizer/     # 262 tests, must stay green
 python scripts/mcp_e2e_check.py                # full-pipeline e2e check
 ```
 
@@ -567,5 +692,5 @@ MIT © [Shweta Mishra](https://github.com/Shweta-Mishra-ai)
 <div align="center">
   <sub>Built for developers who spend too much time re-explaining their projects to AI.</sub>
   <br/><br/>
-  <a href="https://github.com/Shweta-Mishra-ai/tokenmizer"><img src="https://img.shields.io/github/stars/Shweta-Mishra-ai/tokenmizer?style=social" alt="GitHub stars"/></a>
+  <a href="https://github.com/Shweta-Mishra-ai/tokenmizer/stargazers"><img src="https://img.shields.io/github/stars/Shweta-Mishra-ai/tokenmizer?style=flat-square&color=f9d84a&label=%E2%AD%90%20Star%20on%20GitHub" alt="GitHub stars"/></a>
 </div>