anonproxy 0.1.1__tar.gz

anonproxy-0.1.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Nethemba
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

anonproxy-0.1.1/PKG-INFO

@@ -0,0 +1,544 @@
+Metadata-Version: 2.4
+Name: anonproxy
+Version: 0.1.1
+Summary: Reversible anonymization proxy for sending pentest data to LLMs
+License: MIT
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fastapi>=0.110
+Requires-Dist: uvicorn[standard]>=0.27
+Requires-Dist: httpx>=0.27
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Provides-Extra: gliner
+Requires-Dist: gliner>=0.2.13; extra == "gliner"
+Provides-Extra: gliner2
+Requires-Dist: gliner2>=0.1; extra == "gliner2"
+Provides-Extra: piiranha
+Requires-Dist: transformers>=4.40; extra == "piiranha"
+Requires-Dist: torch>=2.2; extra == "piiranha"
+Provides-Extra: openai-pii
+Requires-Dist: transformers>=4.40; extra == "openai-pii"
+Requires-Dist: torch>=2.2; extra == "openai-pii"
+Provides-Extra: all
+Requires-Dist: gliner>=0.2.13; extra == "all"
+Requires-Dist: gliner2>=0.1; extra == "all"
+Requires-Dist: transformers>=4.40; extra == "all"
+Requires-Dist: torch>=2.2; extra == "all"
+Dynamic: license-file
+
+# Anonproxy
+
+**A reversible anonymization layer for sending pentest data to LLMs — built to be
+more reliable than match/replace.**
+
+Anonproxy sits between your tools (Claude Code, the OpenAI SDK, Burp Suite) and
+the LLM API. It strips IPs, hostnames, credentials, hashes, tokens, org names and
+PII out of everything *before* it leaves the machine, and restores the real
+values in the reply. The model reasons over realistic surrogates; you read real
+data.
+
+It is a ground-up reimplementation inspired by
+[`DontFeedTheAI`](https://github.com/zeroc00I/DontFeedTheAI), focused on the part
+that was only working ~75% of the time: **the round trip**.
+
+> **In plain terms:** you run a small program on your laptop. Your AI tools talk
+> to it instead of talking to the cloud directly. It swaps client data for
+> realistic fakes before anything leaves your machine, and swaps the real values
+> back into the answer. The cloud AI never sees real client data; you never see
+> the fakes.
+>
+> **New here? Read [QUICKSTART.md](QUICKSTART.md) first** — three steps, no jargon.
+
+---
+
+## Why the old approach hit ~75%
+
+Match/replace (and the original's exact-substring restore) breaks the moment the
+model touches a surrogate. Given a surrogate `host-ab12cd9`, models routinely
+write it back as:
+
+| What the model writes | Exact `str.replace` restores? |
+|---|:---:|
+| `host-ab12cd9` | ✅ |
+| `**host-ab12cd9**` (bold) | ✅ (wrapper is outside) |
+| `` `host-ab12cd9` `` (inline code) | ✅ |
+| `HOST-AB12CD9` (case changed) | ❌ |
+| `` `host-`ab12cd9 `` (emphasis *inside* the token) | ❌ |
+| `host‑ab12cd9` (non-breaking hyphen / line wrap) | ❌ |
+| surrogate split across two streaming chunks | ❌ |
+
+Add detection gaps (regex can't see a bare hostname) and a small local model's
+inconsistency, and you land around three-quarters. Our reproducible benchmark
+puts naive exact-replace at **78%** and Anonproxy's restorer at **100%** on the
+same mangling patterns:
+
+```
+$ python3 scripts/benchmark_roundtrip.py
+
+mangling         naive str.replace   tolerant restorer
+------------------------------------------------------
+verbatim      7/    7  (  100%)       7/7  (  100%)
+uppercase     2/    7  (   29%)       7/7  (  100%)
+lowercase     6/    7  (   86%)       7/7  (  100%)
+intra_code    0/    7  (    0%)       7/7  (  100%)
+bold_segment  3/    7  (   43%)       7/7  (  100%)
+...
+OVERALL       60/   77  (   78%)      77/77  (  100%)
+```
+
+## What makes it more reliable
+
+**1. Tolerant restoration.** Restoration matches a surrogate against a normalized
+projection of the reply (markdown noise *inside* the token removed, whitespace
+collapsed, case folded, unicode hyphens normalized) while keeping an index map
+back to the original text, so it recovers the value even when the model
+reformats it. It deliberately does **not** expand outward to eat markdown
+*around* a matched span (e.g. `**`/backticks hugging it) — those may be genuine
+original content rather than model-added formatting, and swallowing them
+corrupted real pages that happened to use `*emphasis*` near a redacted value.
+(`anonproxy/restorer.py`)
+
+**2. Streaming-safe.** A per-content-block hold-back buffer reassembles a
+surrogate split across SSE deltas before restoring it, so you get real values
+*as the response streams*, not after a full buffer. (`anonproxy/proxy/streaming.py`)
+
+**3. Consistency by construction.** Surrogates are deterministic (HMAC keyed on
+the engagement id) *and* vault-backed, so the same original always maps to the
+same surrogate on an exact re-sighting — even across restarts or a lost vault.
+A consistency rescan re-detects anything the vault has ever seen, so an entity
+caught once is caught every time. (Two *different* casings of the same
+real-world value, e.g. `WordPress.org` and `wordpress.org` both appearing in
+one page, are treated as distinct sightings with their own surrogates — the
+alternative, collapsing them, can only remember one original spelling and gets
+the other one's restoration wrong.) (`anonproxy/surrogates.py`, `anonproxy/vault.py`)
+
+**4. Format-preserving surrogates.** A hash surrogate is hex of the same length;
+an AWS key keeps its `AKIA` prefix; an IP is a valid RFC 5737 TEST-NET address; a
+payment card stays 16 Luhn-valid digits with the same grouping. The model treats
+them as the real thing and has no reason to "correct" them — which also *prevents*
+mangling in the first place. Precise floor spans also win over a broader model
+span that merely wraps them, so structure like a `PHPSESSID=` cookie name is kept
+while the value is swapped.
+
+**5. Layered, pluggable detection.** A deterministic regex floor (IPs, hashes,
+JWTs, cloud keys, FQDNs, MACs) plus a *configurable chain* of contextual backends
+and the consistency rescan. Contextual backends are additive: if one is down,
+regex + rescan carry on. See [Detection backends](#detection-backends).
+(`anonproxy/detectors/`)
+
+## Architecture
+
+```
+            real data                      surrogates only
+ client  ───────────────▶  Anonproxy  ───────────────────▶  LLM API
+ (Claude Code /            (engine +                         (Anthropic /
+  OpenAI SDK / Burp)        proxy)    ◀───────────────────   OpenAI-compatible)
+                              ▲          response w/ surrogates
+                              │ restored (tolerant, streaming)
+                              ▼
+                    per-engagement vault (SQLite, isolated)
+```
+
+One engine, three front ends:
+
+* **Library** — `from anonproxy import Engine`
+* **Proxy** — `python -m anonproxy serve` (Anthropic + OpenAI shapes)
+* **Burp extension** — `burp/` delegates to the engine's local API, so Burp
+  shares the same vault and tolerant restorer (see `burp/README.md`)
+
+## Quick start (all local)
+
+Everything runs on your machine — no VPS, no remote, nothing sensitive leaves
+the host. Pick whichever setup you prefer.
+
+**A. macOS app (recommended)**
+
+Download `Anonbar-macos.zip` from the project's
+[Releases](../../releases) page, unzip, drag **Anonbar.app** to
+Applications, and open it (unsigned local build: right-click → *Open* the
+first time, or `xattr -cr /Applications/Anonbar.app`). The 🛡️ icon appears
+in your menu bar.
+
+Requirements: any Python ≥ 3.10 on the machine (`brew install python@3.12`).
+Nothing else — on first **Start** the app creates its own isolated
+environment (`~/.anonproxy/venv`) and installs its dependencies for you.
+Building from source instead? See
+[Fast workflow](#fast-workflow-profiles-scripts_anon-macos-menubar).
+
+**B. Guided wizard**
+
+```bash
+# one-time install (Python 3.10+): gives you the `anonproxy` command
+pipx install anonproxy        # or: pip install anonproxy  /  pip install .
+anonproxy wizard
+```
+
+From source instead?
+
+```bash
+git clone https://github.com/jfreemansh/Anonproxy && cd Anonproxy
+pip install .                 # or: pip install -r requirements.txt + python -m anonproxy
+python -m anonproxy wizard
+```
+
+It asks for the engagement name, finds/pulls an Ollama model, writes a `.env`,
+and offers to launch the proxy.
+
+**C. Manual**
+
+```bash
+pip install -r requirements.txt
+ollama pull qwen3:4b                         # optional, for best recall
+python -m anonproxy serve --engagement acme-2026
+```
+
+**D. Docker (bundles Ollama)**
+
+```bash
+ENGAGEMENT_ID=acme-2026 docker compose up -d
+docker compose exec ollama ollama pull qwen3:4b     # one-time
+```
+
+Then point your client at it:
+
+```bash
+# Claude Code
+export ANTHROPIC_BASE_URL=http://127.0.0.1:8099
+claude
+
+# OpenAI SDK / OpenRouter
+#   base_url = "http://127.0.0.1:8099/v1"
+```
+
+### Fast workflow: profiles (`scripts/anon`, macOS menubar)
+
+> **The proxy listens on 8099, not 8080** — Burp's upstream proxy owns 8080,
+> and this tool ships a Burp extension that talks to a *running* Anonproxy.
+> Keeping the two apart means both are up at once with zero configuration.
+
+One JSON profile per client/test carries the whole context — engagement id
+(→ vault isolation), scope seed, detector chain, model, port. Spin-up between
+tests is one command instead of re-typed flags:
+
+```bash
+scripts/anon profile new acme-web --scope "acme.com,portal.acme.com,DC01" --notes "web app test"
+scripts/anon up acme-web --daemon      # detached; logs under ~/.anonproxy/logs/
+scripts/anon env acme-web --copy       # client export lines → clipboard
+scripts/anon stop                      # reap the daemon
+scripts/anon close acme-web            # export JSON+CSV evidence + wipe vault
+```
+
+`profile list|show|edit|rm` round it out (profiles live in
+`~/.anonproxy/profiles/`; `ANONPROXY_PROFILE_DIR` overrides). `up <profile>`
+without `--daemon` behaves like `serve` with the profile applied.
+
+On macOS there's also a **native status-bar app** (AppKit — no Python UI deps,
+no app bundle required): pick an engagement, Start/Stop, ＋ **New engagement…**
+(a real form: name, scope terms, port, notes, ephemeral toggle), copy client
+env, open `/audit`, run `verify`, and one-click **Export & archive vault…**
+close-out.
+
+```bash
+scripts/install_anonbar.sh      # one-time: builds + installs /Applications/Anonbar.app
+open -a Anonbar                 # or Spotlight: ⌘Space → "Anonbar"
+```
+
+Rebuild/reinstall only when `anonbar.swift` itself changes. For auto-start at
+login: System Settings → General → Login Items → add Anonbar.
+
+It shells out to the same CLI, so profiles created in either place just show
+up in the other. Env overrides: `ANONPROXY_HOME`, `PYTHON_BIN`.
+
+> Ephemeral profiles (`--ephemeral`) leave nothing on disk — but that also
+> means a close-out *after* stopping finds no mappings. Run close-out while
+> the proxy still holds them, or keep persistence on when evidence matters.
+
+Quick offline check without a client:
+
+```bash
+echo 'Host dc01.acmecorp.local at 10.20.0.10, NTLM 8846f7eaee8fb117ad06bdd830b7586c' \
+  | python -m anonproxy anon --engagement acme-2026
+```
+
+## End-to-end example (no Burp)
+
+Send real tool output to a real LLM through the proxy and get a useful answer
+back — while the API only ever sees fake data.
+
+```bash
+# 1. start the proxy
+python -m anonproxy serve --engagement acme-2026 &
+
+# 2. run a tool, then ask Claude about it THROUGH the proxy
+SCAN=$(nmap -sV dc01.acmecorp.local)
+
+curl -s http://127.0.0.1:8099/v1/messages \
+  -H "x-api-key: $ANTHROPIC_API_KEY" \
+  -H "anthropic-version: 2023-06-01" \
+  -H "content-type: application/json" \
+  -d "$(jq -n --arg scan "$SCAN" '{
+        model: "claude-sonnet-4-6",
+        max_tokens: 1024,
+        messages: [{role:"user", content:("Analyse this scan and suggest next steps:\n" + $scan)}]
+      }')"
+```
+
+What just happened:
+
+- **Anthropic only saw surrogates** — e.g. `203.0.113.47` and
+  `host-ab12cd9.pentest.local`, never `10.20.0.10` or `dc01.acmecorp.local`.
+- **Your reply has the real values back** — even if the model wrote them in bold
+  or changed their case.
+- Open `http://127.0.0.1:8099/audit` to see exactly what was swapped.
+
+The same works for OpenAI-style clients — just call
+`http://127.0.0.1:8099/v1/chat/completions` with your usual OpenAI payload.
+
+## Audit dashboard
+
+Open `http://127.0.0.1:8099/audit` (or `python -m anonproxy audit`) to review
+every `original → surrogate` mapping live during an engagement — filterable by
+type, with counts and CSV export. It binds to localhost and honours
+`ANONPROXY_API_TOKEN` if set; disable it with `ANONPROXY_AUDIT=false`. It exposes
+the reverse lookup, so treat it as an operator-only debug view.
+
+If a detector backend errors mid-engagement (a floor-detector crash, a contextual
+backend throwing), the stats bar shows a red `⚠ <name> failed ×N` pill instead of
+failing silently into a log line — check it if coverage looks off partway through
+a session.
+
+## Engagement workflow
+
+1. **One engagement id per client** (`--engagement acme-2026`). This isolates the
+   vault so surrogates never cross between clients.
+2. Run the proxy locally on your testing machine. Everything — proxy, vault, and
+   the Ollama detector — stays on the host.
+3. Work normally — every request is anonymized, every reply restored. Watch
+   coverage live at `/audit`.
+4. At session close, `python -m anonproxy export --engagement acme-2026` (or the
+   audit page's CSV export) dumps the full `original → surrogate` map for your
+   evidence/audit trail, then archive or delete the vault. Use
+   `ANONPROXY_EPHEMERAL=1` for in-memory-only (no disk persistence).
+
+## Configuration
+
+| Variable | Default | Description |
+|---|---|---|
+| `ENGAGEMENT_ID` | `default` | **Change per client.** Isolates the vault. |
+| `ANONPROXY_SCOPE` | *(empty)* | Comma list of client domains/hostnames/orgs always anonymized. |
+| `ANONPROXY_SCOPE_FILE` | *(empty)* | File of scope terms (one per line, optional `value=TYPE`). |
+| `ANONPROXY_CONTEXTUAL_MIN_LEN` | `4` | Contextual single-token findings shorter than this are dropped as noise (`db`, `sql`). Scope terms and regex matches bypass it. |
+| `ANONPROXY_DETECTORS` | `regex,ollama` | Backend chain (see [Detection backends](#detection-backends)). |
+| `LLM_ENABLED` | `true` | `false` = drop the Ollama backend. |
+| `OLLAMA_HOST` | `http://localhost:11434` | Ollama endpoint. |
+| `OLLAMA_MODEL` | `qwen3:4b` | Local detection model. Any Ollama model works (`--model`). |
+| `GLINER_MODEL` | `urchade/gliner_multi_pii-v1` | GLiNER model (if enabled). |
+| `PIIRANHA_MODEL` | `iiiorg/piiranha-v1-detect-personal-information` | Piiranha model (if enabled). |
+| `ANONYMIZER_SLM_MODEL` | `anonymizer-slm` | Ollama name of the imported Anonymizer SLM. |
+| `ANONPROXY_TOLERANT` | `true` | Tolerant restoration (vs. exact). |
+| `ANONPROXY_EPHEMERAL` | `false` | In-memory vault, nothing on disk. |
+| `ANONPROXY_AUDIT` | `true` | Serve the `/audit` dashboard. |
+| `ANONPROXY_PROFILE_DIR` | `~/.anonproxy/profiles` | Where engagement profiles (one JSON each) live. |
+| `ANONPROXY_EXPORTS_DIR` | `~/.anonproxy/exports` | Close-out evidence output root. |
+| `PORT` / `HOST` | `8099` / `127.0.0.1` | Proxy listen address. |
+| `ANTHROPIC_UPSTREAM` | `https://api.anthropic.com` | Anthropic upstream. |
+| `OPENAI_UPSTREAM` | `https://api.openai.com` | Any OpenAI-compatible endpoint — OpenRouter, Groq, Together, etc. Paste the provider's documented `base_url` as-is, with or without a trailing `/v1`; both work (`https://openrouter.ai/api/v1` and `https://openrouter.ai/api` are equivalent here). |
+| `ANONPROXY_API_TOKEN` | *(empty)* | Require `X-Anonproxy-Token` on the engine API. **Empty means the engine API and `/audit` are unauthenticated** — fine on an isolated laptop bound to `127.0.0.1`, but set this if the proxy is ever reachable by anything else (a shared box, a tunneled VPS). |
+| `ANONPROXY_STRICT` | `false` | Fail closed (502) instead of forwarding a request unredacted when its body can't be parsed as JSON. Off by default so odd/legacy clients keep working. |
+
+## Detection backends
+
+Detection is a configurable chain set by `ANONPROXY_DETECTORS` (the wizard asks
+too). The regex floor is always on and always first; everything else is
+optional, lazily loaded, and skipped with a warning if its dependency or model
+isn't present — so the default never breaks. Pick the trade-off your colleague
+wants:
+
+| Backend | Catches | Cost / setup |
+|---|---|---|
+| `regex` *(always on)* | IPs, CIDRs, hashes, JWTs, cloud keys, MACs, FQDNs, emails, labelled creds, payment cards (Luhn), Bearer/Basic auth, SSNs, and **any cookie value in a `Set-Cookie:`/`Cookie:` header — by structure, not by a fixed name list**, so a CMS's own custom session-cookie name (WordPress's `wordpress_logged_in_<hash>`, say) is caught the same as `PHPSESSID`. Also resilient to URL-encoded delimiters (`%7C` etc.) that would otherwise glue an alnum byte onto the next token and defeat a boundary-anchored match — a real, previously-silent gap on form-urlencoded bodies and cookie headers. | none — deterministic floor |
+| `ollama` *(default)* | bare hostnames, org/project/person names, unlabelled creds in prose | local Ollama + a model (`--model`) |
+| `gliner2` ⭐ *(recommended for general PII)* | 42 PII types, best span-level F1 on SPY (beats the others), <100ms — but its fixed taxonomy has **no hostname or organization label**, so it does not replace `ollama` for pentest-specific redaction | `pip install "anonproxy[gliner2]"`, CPU-friendly |
+| `openai-privacy-filter` | ~96–97% F1 PII (Apache-2.0, OpenAI) | `pip install "anonproxy[openai-pii]"` (torch) |
+| `gliner` | zero-shot person/org/username/hostname/email — the older urchade model, but because it's zero-shot you can *ask* it for `hostname`/`organization`, which `gliner2` can't be asked for | `pip install "anonproxy[gliner]"`, CPU-friendly |
+| `piiranha` | high-accuracy passwords/emails/usernames (6 languages) | `pip install "anonproxy[piiranha]"` (torch) |
+| `anonymizer-slm` | purpose-built PII detect+replace (Eternis Qwen3 fine-tune) | import GGUF via `models/anonymizer-slm.Modelfile` |
+
+All four transformer backends (`gliner`, `gliner2`, `piiranha`, `openai-privacy-filter`)
+chunk large inputs internally — a transformer has a fixed context window, and
+handing one a whole multi-hundred-KB HTTP response either truncates coverage
+silently or exhausts memory. Chunking bounds memory while still scanning the
+full input.
+
+> **Best recall for general PII:** `gliner2` (Fastino's GLiNER2-PII, May 2026)
+> currently tops the SPY PII benchmark, runs in-process (no external service to
+> keep alive), and surfaces failures cleanly instead of swallowing them.
+> **It does not catch bare hostnames or client/org names** — that's the #1
+> redaction need on a pentest engagement, and nothing in its 42-label PII
+> taxonomy covers it. Don't drop `ollama` (or `gliner`) in favor of `gliner2`
+> alone; stack them instead: **`ANONPROXY_DETECTORS=regex,gliner2,ollama`**.
+> (When pulling `openai/privacy-filter`, use exactly that org — typosquats have
+> appeared.)
+>
+> **Web-app testing:** the regex floor now covers payment cards, session cookies,
+> Bearer/Basic auth and SSNs, so a lot of HTTP traffic is handled deterministically.
+> But **names and addresses in request/response bodies are not regex-detectable** —
+> for those add a PII model. Always confirm on real traffic with
+> `python -m anonproxy verify`.
+
+```bash
+# default, out of the box
+ANONPROXY_DETECTORS=regex,ollama python -m anonproxy serve
+
+# recommended: fast general-PII coverage (gliner2) + hostnames/orgs/names (ollama)
+ANONPROXY_DETECTORS=regex,gliner2,ollama python -m anonproxy serve
+
+# no external service at all — zero-shot gliner covers hostname/org itself
+ANONPROXY_DETECTORS=regex,gliner,gliner2 python -m anonproxy serve
+
+# stack several — order is just declaration order; regex always wins for
+# structured types so a hash stays a hash
+ANONPROXY_DETECTORS=regex,piiranha,ollama python -m anonproxy serve
+```
+
+The regex floor always wins type classification for structured data (so a hash
+is never mis-typed as an org name), contextual backends add recall, and the
+consistency rescan re-catches anything seen once. `python -m anonproxy verify`
+prints the active chain and confirms coverage.
+
+### Hostnames & the scope seed
+
+The regex floor catches **fully-qualified** hostnames (`dc01.acme.local`,
+`shop.acme.com`, `portal.acme.dev`) via a broad TLD list. It deliberately does
+**not** guess **bare** hostnames (`DC01`, `WEB-PRD-03`) — there's no safe pattern
+that separates them from ordinary words.
+
+Two ways to cover bare names:
+
+1. **Scope seed (recommended, deterministic).** Tell Anonproxy your engagement
+   scope and every occurrence is anonymized, no model needed:
+
+   ```bash
+   # inline
+   python -m anonproxy serve --engagement acme-2026 \
+     --scope "acme.com,portal.acme.com,DC01,WEB-PRD-03,Acme Corp"
+   # or a file (one term per line, optional value=TYPE)
+   python -m anonproxy serve --scope-file acme-2026-scope.txt
+   ```
+
+   The wizard writes a starter `<engagement>-scope.txt` (with examples) for you
+   and wires it into `.env` — just keep editing it as scope grows. The `--scope` /
+   `--scope-file` flags work on `serve`, `verify`, and `anon` too; or set
+   `ANONPROXY_SCOPE` / `ANONPROXY_SCOPE_FILE`.
+
+   Scope terms run as part of the floor (always on, even regex-only), match whole
+   tokens only (so `acme` won't touch `acmespeak`), and get the usual consistent,
+   reversible surrogates.
+
+2. **A contextual backend** (`ollama` or `gliner` — not `gliner2`, which doesn't
+   request hostname/org labels) infers hostnames/org names it wasn't told about
+   — good for catching scope you forgot to list.
+
+> **Very short bare names** (`db`, `sql`, `dc1`): a contextual single token
+> under 4 characters is dropped as probable noise, not a hostname. Put short
+> names on the scope list (deterministic, always caught), or lower
+> `ANONPROXY_CONTEXTUAL_MIN_LEN`.
+
+Use both: seed what you know, let the model catch the rest.
+
+## Verify coverage before an engagement
+
+```bash
+python -m anonproxy verify                  # uses your local Ollama if it's running
+python -m anonproxy verify --no-llm          # regex floor only
+python -m anonproxy verify --show-mappings   # also print anonymized output + original→surrogate table
+ANONPROXY_DETECTORS=regex,gliner2 python -m anonproxy verify   # test a specific chain
+```
+
+`--show-mappings` (alias `--audit`) is the audit view for a verify run — it shows
+exactly what the LLM would receive and every swap it made. (The `/audit` web
+dashboard is for *live proxy* traffic; verify runs in a throwaway vault so it
+won't clutter a real engagement.)
+
+Runs realistic nmap / secretsdump / netexec / config / HTTP outputs through the
+full pipeline and reports any **leaks** (sensitive strings that survived) and
+**round-trip** failures. It prints the active detector chain, and whichever
+contextual backends are up are used automatically — so this also confirms your
+local model works. Secrets only a contextual backend can catch (bare hostnames,
+unlabelled creds) are shown as *needs-contextual* in regex-only mode rather than
+counted as leaks. It also runs an **adversarial "repeat the context verbatim"
+probe** that asserts no regex-layer secret could appear in what was sent
+upstream, and a **tool-call payload probe** that checks an Anthropic `tool_use`
+block / OpenAI `tool_calls[].function.arguments` echoed back in conversation
+history doesn't leak a real value either — that path bypassed anonymization
+entirely until this check existed. Exit code is non-zero if anything real leaks.
+
+Check what's actually active any time:
+
+```bash
+curl -s http://127.0.0.1:8099/anonproxy/health | python -m json.tool
+# -> detectors[]: name, available, model/effective_model, detail
+```
+
+If a configured Ollama model isn't pulled, the detector auto-falls back to an
+installed one and says so in `health` and `verify` (so "Ollama is running but no
+model" can't silently degrade you to regex-only). If Ollama dies or times out
+mid-engagement, the next request re-checks reachability instead of trusting a
+stale "available" — `health` will flip to `available: false` with the reason,
+rather than staying green while quietly returning zero detections.
+
+## Tests
+
+```bash
+python3 -m pytest -q                    # 120 tests: round-trip, streaming, proxy, audit, verify, detectors, tool-calls, vault, profiles, llm_detector, webapp, scope, config, polish
+python3 scripts/benchmark_roundtrip.py  # naive vs tolerant pass-rate table
+```
+
+## Techniques & prior art
+
+Anonproxy implements the reversible-anonymization techniques that fit a pentest
+LLM pipeline, and deliberately skips the ones that don't:
+
+| Technique | In Anonproxy |
+|---|---|
+| Format-preserving masking | ✅ surrogates are valid instances of their type |
+| Pseudonymization via consistent keyed-hash substitution | ✅ HMAC keyed on the engagement id + vault |
+| Tolerant, streaming-safe restoration | ✅ the core improvement over match/replace |
+| Generalization (ranges) / nulling (redaction) | ➖ not applicable to pentest infra data |
+| Synthetic data generation | ➖ surrogates are synthetic but kept structure-faithful |
+
+**One method to keep in mind — linkability.** Consistent surrogates (the same
+input always maps to the same fake) make the AI's reasoning coherent across a
+session, but that same consistency means a provider could, in principle,
+*correlate* requests over time even without the real values (see RAT-Bench and
+"localized adversarial anonymization", 2026). For pentest work the consistency is
+usually worth it (you want the model to track "the same host" across turns). If
+you need stronger anonymity against correlation, rotate the engagement id per
+session — reversibility still works (it's vault-backed), only the surrogate
+values change.
+
+Inspired by [DontFeedTheAI](https://github.com/zeroc00I/DontFeedTheAI) and informed by:
+
+- Fastino, [*GLiNER2-PII*](https://huggingface.co/fastino/gliner2-privacy-filter-PII-multi) — current SOTA PII span extraction; the recommended `gliner2` backend.
+- OpenAI, [*Privacy Filter*](https://huggingface.co/openai/privacy-filter) — open-weight local PII classifier; the `openai-privacy-filter` backend.
+- Eternis, [*Anonymizer SLM series*](https://huggingface.co/blog/pratyushrt/anonymizerslm) — purpose-built Qwen3 fine-tune for surgical PII detect+replace; the optional `anonymizer-slm` backend.
+- Red-Gate Simple Talk, [*How to anonymize PII in LLM pipelines*](https://www.red-gate.com/simple-talk/data-security-privacy-compliance/how-to-anonymize-pii-in-llm-pipelines-5-key-techniques-explained/) — the five-technique taxonomy above and the adversarial "repeat the context" leak test now in `verify`.
+- Earlier/optional models: [GLiNER](https://huggingface.co/urchade/gliner_multi_pii-v1), [Piiranha](https://huggingface.co/iiiorg/piiranha-v1-detect-personal-information).
+
+## Scope & limits
+
+This is a **risk-reduction layer, not a privacy guarantee** (same honest framing
+as the original). It does not defend against query-pattern correlation, prompt
+injection in tool output, or compromise of the local host. It only anonymizes
+text — pasted screenshots or other non-text content bypass it entirely. It
+assumes a single proxy process; the vault's in-memory cache has no cross-process
+invalidation, so don't run it behind multiple workers (`serve` always starts
+one). It is not a substitute for reading what your NDA and engagement contract
+allow before using any cloud AI on client data. Verify coverage per engagement
+with the `/audit` page, `export`, and the test suite.
+
+## License
+
+MIT.