agent-sanitizer 2.0.0

package/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Alexander Turner
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

package/README.md

@@ -0,0 +1,217 @@
+# `agent-sanitizer`
+
+Most prompt-injection tools run a classifier _over_ the text and hope it
+generalizes. This library targets a narrower, verifiable claim: the
+specific byte-level channels—invisible Unicode, ANSI escapes, human-hidden
+HTML, confusable glyphs, exfil-shaped URLs—that let an attacker smuggle a
+payload the operator can't see but the model still reads. Every layer is a
+deterministic transform you can unit-test with equality assertions.
+
+```sh
+npm install agent-sanitizer
+```
+
+## Quick start
+
+```js
+import { sanitize } from "agent-sanitizer";
+
+// Layer 1 (invisible chars + ANSI), zero heavy deps:
+const { cleaned, found, warnings } = await sanitize(untrustedText);
+
+// Opt into the HTML layers for web ingress (lazy-loads ~200 ms of deps):
+const result = await sanitize(pageSource, { html: true });
+```
+
+`sanitize` never throws and never silently drops content—any change comes with
+at least one `warnings` entry. `found` names the neutralized category codes
+(e.g. `["cf-format", "hidden-html"]`); `cleaned` is the safe text, with
+placeholders where hidden HTML was spliced out.
+
+## Entry points
+
+Split into subpaths so the heavy HTML dependency stays opt-in. The **seam**
+column marks entry points that inject the agent-specific concern (a callback
+you supply) rather than baking it in; `—` means the function is a pure
+transform with no such hook, and `fs (direct)` means it does its own file I/O
+(Node's filesystem, not an agent harness) instead of taking a callback.
+
+| #   | Import          | Purpose                                                                                                                                                    | Seam                        |
+| --- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
+| 1   | `/invisible`    | Strip zero-width, bidi, variation-selector and tag chars + ANSI/SGR escapes. Preserves ZWNJ/ZWJ for Arabic/Indic/emoji. Zero deps.                         | —                           |
+| 2   | `/html`         | Splice out instructions hidden in comments, `display:none`, off-screen, white-on-white, `hidden`. Leaves a placeholder.                                    | —                           |
+| 3   | `/html`         | Detect exfil-shaped URLs (payloads in query/path, embedded creds, `data:`/`javascript:`, off-origin redirects). Reports only.                              | —                           |
+| 4   | `/confusables`  | Fold look-alike glyphs in tool-call input (paths, commands) to ASCII, closing a cross-script deny-rule bypass.                                             | `scan`                      |
+| 5   | `/instructions` | Scan/auto-clean `CLAUDE.md`, `AGENTS.md`, `SKILL.md`, etc., decoding Unicode-tag + zero-width-binary payloads.                                             | `fs` (direct)               |
+| 6   | `/prompt`       | Classify a prompt pass / SGR-note / block on payload-capable invisible/ANSI content.                                                                       | —                           |
+| 7   | `/output`       | Run Layers 1–4 over structured tool output, preserving shape. The Layer-5 slot takes a delete-only filter.                                                 | `redact`, `filterInjection` |
+| 8   | `/rehydrate`    | Re-anchor a model Edit composed from the _sanitized_ view back onto real bytes; deny anything ambiguous or secret-exposing.                                | `io`                        |
+| —   | `/view-map`     | Pure offset/text machinery mapping a file's on-disk bytes ↔ the sanitized view (Layer-1 deletions, Layer-4 redactions). No I/O — consumed by `/rehydrate`. | —                           |
+
+See [`THREAT-MODEL.md`](./THREAT-MODEL.md) for per-vector detail.
+
+### `found` codes
+
+`found` (from `sanitize`/`stripInvisibleWithReport`) is a stable, machine-readable
+contract—branch on these codes, not on `warnings` prose, which can be reworded
+without notice.
+
+| Code                  | Meaning                                                                                                 |
+| --------------------- | ------------------------------------------------------------------------------------------------------- |
+| `cf-format`           | Unicode format chars (`Cf`): zero-width space/joiner, bidi overrides, tag chars                         |
+| `variation-selectors` | Variation selectors (U+FE00–FE0F, U+E0100–E01EF)                                                        |
+| `blank-fillers`       | Blank-rendering fillers not covered by `Cf` (Hangul fillers, Braille blank, zero-width combining marks) |
+| `ansi`                | ANSI/SGR escapes and other terminal control sequences                                                   |
+| `lone-surrogates`     | Unpaired UTF-16 surrogates                                                                              |
+| `html-comments`       | HTML comments spliced out by Layer 2                                                                    |
+| `hidden-html`         | Elements hidden via CSS/attribute (`display:none`, `hidden`, etc.) spliced out by Layer 2               |
+| `exfil-urls`          | Exfil-shaped URLs detected by Layer 3 (reported, not removed)                                           |
+
+### `FILTER_WARNING` codes (Layer 5)
+
+The Layer-5 `filterInjection` seam is deliberately thin: the injected filter may
+only ask for **verbatim span deletions** (`removeSpans`) and may only warn with a
+**closed enum code** (`warning`), never free text. Because the filter runs on
+attacker-influenced content and its `warning` would otherwise be concatenated
+straight into the model-facing context **without** re-passing Layer 1, a
+compromised or prompt-injected filter emitting arbitrary text would defeat the
+"a compromised filter can only remove bytes, never inject" contract. So the
+**library owns the message** for each code, and a filter returning any value
+outside this enum makes `sanitizeText` **throw** (fail loud). Branch on the code,
+like `found`:
+
+| `FILTER_WARNING` code | Meaning                                                                                 |
+| --------------------- | --------------------------------------------------------------------------------------- |
+| `spans-removed`       | The filter removed one or more verbatim spans it flagged as prompt injection            |
+| `filter-flagged`      | The filter flagged the output as a possible injection without deleting (content intact) |
+| `filter-error`        | The filter reported a non-fatal internal error while scanning (a fatal filter throws)   |
+
+## How this compares
+
+The "sanitize untrusted LLM input" space mostly splits into two camps: ML
+classifiers that score a prompt's _intent_ (Lakera Guard, Meta's Prompt
+Guard, Rebuff, NeMo Guardrails' input rails), and PII-focused redactors
+(Microsoft Presidio). Neither targets the byte-level hiding channel this
+library covers, and that gap is exactly where invisible-Unicode and
+hidden-HTML payloads live—content a semantic classifier never "sees" as
+suspicious because it renders as blank space or doesn't render at all.
+
+|                               | `agent-sanitizer`                                                                                                        | Semantic guard/classifier (Lakera, Prompt Guard, Rebuff, NeMo rails)                                       | PII redactor (Presidio)                                      |
+| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
+| **What it catches**           | Payload-capable invisible chars, ANSI/SGR, hidden HTML, confusable glyphs, exfil-shaped URLs                             | Malicious _intent_—jailbreaks, injected instructions, off-topic asks                                       | Names, emails, SSNs, and other PII spans                     |
+| **How it decides**            | Deterministic parsing/regex over real tokenizer output—no model call                                                     | ML/LLM classification—probabilistic, needs a threshold and retuning as attacks shift                       | NER + pattern matching                                       |
+| **Failure mode**              | Fails open on ambiguous input (see [`THREAT-MODEL.md`](./THREAT-MODEL.md)); false negative over false positive by design | False positives silently mangle or block legitimate prompts; false negatives are invisible until exploited | Under/over-redaction depending on locale and entity coverage |
+| **Latency / infra**           | Pure JS, mostly zero-dep (`/html` lazy-loads ~200 ms once)                                                               | Network round-trip to a hosted model, or a local model to host yourself                                    | Local, but heavier NLP pipeline                              |
+| **Determinism / testability** | Exact-equality unit tests, no flakiness across runs                                                                      | Same input can classify differently across model versions                                                  | Deterministic per rule, but rule coverage varies             |
+| **Reversibility**             | `/rehydrate` re-anchors a model's edit from the sanitized view back onto real bytes, denying anything ambiguous          | N/A—classifiers only pass/block, they don't rewrite-and-reverse                                            | N/A                                                          |
+| **Non-JS support**            | Same verdicts via a bundled CLI/worker—Python client included, no reimplementation                                       | Usually a hosted API (language-agnostic) or Python-only SDK                                                | Python-first (spaCy-based)                                   |
+
+In practice these are complementary, not competing: run a semantic guard for
+intent, Presidio for PII you must not leak, and this library for the hidden
+channel both of those are blind to. If you already run a classifier and are
+still getting bitten by zero-width payloads or `display:none` instructions
+riding along in RAG context, that's the gap this library closes.
+
+### Examples
+
+```js
+import { stripInvisibleWithReport } from "agent-sanitizer/invisible";
+const { cleaned, found } = stripInvisibleWithReport(text); // found: ["variation-selectors"]
+
+import { sanitizeHtml, detectExfil, checkExfilUrl } from "agent-sanitizer/html";
+sanitizeHtml(pageSource); // { text, removed, warned } | null — text may be unchanged if only reportable (not strippable) tags were found
+detectExfil(pageSource); // [{ isImage, reason, target }] or null
+checkExfilUrl(oneUrl); // reason string or null
+```
+
+The agent-pipeline entry points take plain arguments and inject their
+agent-specific seam:
+
+```js
+import { normalizeConfusables } from "agent-sanitizer/confusables";
+normalizeConfusables(
+  "Bash",
+  { command: "/аpt update" },
+  { scan: (t) => myHomoglyphEngine.scan(t) }, // -> { findings: [{ index, char, latinEquivalent }] }
+); // null, or { updatedInput, normalized }
+
+import { scanInstructionFiles, cleanFile } from "agent-sanitizer/instructions";
+const findings = scanInstructionFiles(["CLAUDE.md", "**/SKILL.md"], {
+  cwd: projectDir,
+});
+for (const { file } of findings) cleanFile(`${projectDir}/${file}`);
+
+import { classifyPrompt } from "agent-sanitizer/prompt";
+classifyPrompt(submittedPrompt); // { action: "pass" | "note" | "block", reason? }
+
+import { sanitizeText } from "agent-sanitizer/output";
+await sanitizeText(toolText, {
+  html: isWebPage,
+  exfilScan: isUntrustedIngress,
+  redact: async (t) => myRedactor.redact(t), // -> { text, found, note? } | null
+  filterInjection: (t) => mySemanticFilter(t), // -> { removeSpans, warning } | null
+  //   removeSpans: verbatim spans to delete (delete-only — never replacement text)
+  //   warning:     a FILTER_WARNING enum CODE, never free text (see below)
+});
+
+import { rehydrateRedacted } from "agent-sanitizer/rehydrate";
+await rehydrateRedacted("Edit", toolInput, {
+  readFile: (p) => fs.readFileSync(p, "utf8"),
+  redactMap: (t) => myRedactor.map(t), // -> { text, pairs } | { unmappable }
+  redact: (t) => myRedactor.redact(t), // -> string | null
+}); // { updatedInput, context } | { deny } | null — a deny never exposes a secret
+```
+
+## Limits
+
+The CLI (and the worker that backs the Python client) rejects any single request
+larger than `AGENT_SANITIZER_MAX_INPUT_BYTES` UTF-8 bytes — **default 10 MiB** —
+with a structured error instead of buffering an unbounded payload. Raise or lower
+it by setting that environment variable in the calling process.
+
+## Security
+
+Found a vulnerability? See [`SECURITY.md`](./SECURITY.md) for the private
+disclosure channel — it ships in the npm tarball, so it is also available offline
+from an installed copy. [`THREAT-MODEL.md`](./THREAT-MODEL.md) covers what each
+layer does and does not defend against.
+
+## Non-JS pipelines (Python, etc.)
+
+The JS is the **single source of truth**—non-JS callers drive the same verdicts
+through the bundled CLI, so there’s
+no second implementation to drift. An `op` field selects the entry point
+(default `sanitize`); the self-contained ones—`sanitizeText`, `classifyPrompt`,
+`scanInstructionFiles`, `cleanFile`—are all bridged. Entry points with an
+injected JS callback have no wire form and stay JS-only. The bridged
+`sanitizeText` runs Layers 1–3 only (invisible/ANSI strip, HTML hidden-content
+splice, exfil detection): it never redacts secrets (Layer 4) or runs injection
+filtering (Layer 5), and the wire protocol's `sgrNote` (Python's
+`TextResult.sgr_note`) is always `false`, since the bridge never wires
+`sgrCarveOut`.
+
+```sh
+echo '{"text":"a​b"}' | npx sanitize-cli           # default op: sanitize
+echo '{"op":"classifyPrompt","text":"…"}' | npx sanitize-cli
+sanitize-cli --worker                              # newline-delimited, one response/line
+```
+
+The [`python/`](./python) client wraps every bridged op (`sanitize`,
+`sanitize_text`, `classify_prompt`, `scan_instruction_files`, `clean_file`). The
+wheel ships a self-contained, single-file build of the CLI (`src/` and its npm
+dependencies bundled into one `.mjs` at release time), so a `pip install` plus
+Node.js (>=22) on `PATH` works with no separate JavaScript checkout to clone.
+`AGENT_SANITIZER_CLI` is only an override escape hatch (e.g. to point at a
+custom/dev build)—a normal install never needs to set it. The first
+`html=True` call starts a shared worker, so the ~200 ms HTML module-load is
+paid **once per process**; Layer-1 calls stay one-shot. `persist=True/False`
+forces the mode and `shutdown_worker()` (also an `atexit` hook) stops it.
+
+```python
+from agent_sanitizer import sanitize, Sanitizer
+
+sanitize(untrusted_text)          # Layer 1, one-shot
+sanitize(page_source, html=True)  # HTML layers, warm worker reused
+with Sanitizer() as s:            # own the worker’s lifetime
+    s.sanitize(page, html=True)
+```

package/SECURITY.md

@@ -0,0 +1,31 @@
+# Security Policy
+
+`agent-sanitizer` is a security library: it sits between untrusted text
+and a model, so a vulnerability here can defeat the very protection a downstream
+pipeline is relying on. Please treat reports accordingly.
+
+## Reporting a vulnerability
+
+**Do not open a public GitHub issue for a security vulnerability.** A public
+report tips off attackers before downstream users can update.
+
+Instead, report privately through one of:
+
+- GitHub's [private vulnerability reporting](https://github.com/AlexanderMattTurner/agent-sanitizer/security/advisories/new)
+  ("Report a vulnerability" under the **Security** tab), or
+- email the maintainer at `security@turntrout.com`.
+
+Please include enough to reproduce: the input that triggers it, the entry point
+(`/invisible`, `/html`, `/confusables`, `/rehydrate`, etc.), what the sanitizer
+did versus what it should have done, and the package version.
+
+A bypass that lets payload-capable content reach the model unflagged, a splice
+that exposes a secret, or a crash on adversarial input are all in scope. Please
+don't include real credentials in a report—a credential-shaped placeholder is
+enough.
+
+## What to expect
+
+The maintainer will acknowledge your report and work with you on a fix and a
+coordinated disclosure timeline before any public detail is published. Fixes
+ship in a patched release with the advisory.

package/THREAT-MODEL.md

@@ -0,0 +1,173 @@
+# Threat model
+
+`agent-sanitizer` defends the boundary where untrusted text enters an
+agent-driven pipeline (agent tool output, RAG retrieval, fetched web pages). It is
+a detect/neutralize layer, not an enforcement boundary: it makes hidden content
+visible-or-gone and surfaces exfil-shaped URLs, so the model and the operator
+see the same thing. Egress controls remain your enforcement layer.
+
+The three layers are independent; use only the ones your ingress needs.
+
+## Layer 1—invisible characters & ANSI (zero-dependency)
+
+**What it removes**
+
+| Category                   | Examples                                                                                                       | Why it’s a payload channel                                    |
+| -------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
+| Format chars (`Cf`)        | zero-width space/joiner, bidi overrides, Unicode **tag** chars                                                 | render blank but reach the model as bytes; tags smuggle ASCII |
+| Variation selectors        | U+FE00–FE0F, U+E0100–E01EF                                                                                     | not `Cf`; a run encodes a hidden payload                      |
+| Blank-rendering fillers    | Hangul fillers U+115F/1160/3164/FFA0, Braille blank U+2800, zero-width combining marks (`Mn`) U+034F/17B4/17B5 | render blank, not `Cf`, so a naive `\p{Cf}` strip misses them |
+| Soft hyphen / interior BOM | U+00AD, interior U+FEFF                                                                                        | either can encode hidden instructions                         |
+| ANSI / SGR escapes         | `ESC[…m`, cursor moves, OSC                                                                                    | repaint or hide what an operator reads in a terminal          |
+
+**What it preserves**
+
+- A **single leading BOM** (a legitimate marker); interior BOMs are stripped.
+- **ZWNJ (U+200C) / ZWJ (U+200D)** in genuine linguistic context: between two
+  letters of a script whose orthography requires them (Arabic, Devanagari,
+  Bengali, Gurmukhi, Gujarati, Oriya, Tamil, Telugu, Kannada, Malayalam,
+  Sinhala) or inside an emoji ZWJ sequence. The carve-out fires only when **both**
+  neighbors clearly belong to the context, and it is disabled once the total
+  invisible count crosses a scatter floor—over-stripping beats under-stripping.
+
+**Reassembly hardening.** Stripping an invisible char can reconstitute an ANSI
+escape its split had hidden, and removing one ANSI sequence can reconstitute
+another. Layer 1 strips ANSI to a fixed point and then sweeps any residual raw
+control introducer—7-bit ESC (U+001B) or 8-bit C1 CSI (U+009B)—outright, so the
+result carries no raw ANSI introducer for _any_ input and the operation is
+idempotent. OSC strings (titles, clickable-hyperlink URLs) are consumed as a
+whole, for every terminator form—ST (`ESC\` or 8-bit C1 ST U+009C) and the
+legacy BEL—and for the 8-bit C1 OSC introducer (U+009D); an _unterminated_ OSC
+introducer is dropped through end-of-string (fail-closed), so no OSC body
+survives to carry a payload.
+
+## Layer 2—hidden HTML (remark/rehype)
+
+For web/HTML ingress, splice out exactly what a human viewing the rendered page
+cannot see:
+
+- `<!-- HTML comments -->`
+- elements hidden by inline style: `display:none`, `visibility:hidden`,
+  `content-visibility:hidden`, `opacity:0`, `filter:opacity(0)`, off-screen
+  positioning, zero/negative sizes, `text-indent` off-screen, collapsing
+  `clip`/`clip-path`/`transform:scale(0)`, white-on-white / transparent text,
+  `overflow:hidden` with a zero dimension
+- elements hidden by attribute: `hidden`, `aria-hidden="true"`
+
+Spliced ranges are replaced with a placeholder; **every byte outside a spliced
+range is preserved verbatim** (no re-serialization). Unclosed hidden markup
+extends to the end of the fragment—fail-closed for truncated input.
+
+Scripting/resource tags (`script`, `style`, `object`, `embed`, `iframe`, `svg`,
+`math`) and `data:` URI resources are **reported, not removed**: their bodies are
+page source the model may legitimately need to inspect.
+
+## Layer 3—exfil URLs (detection only)
+
+Report—never rewrite—URLs in markdown links/images/definitions and HTML
+attributes (`src`/`href`/`background`/`srcset`/`ping`, form `action`/`formaction`,
+`meta refresh`) that are shaped to carry data off-origin:
+
+- suspicious query/fragment parameters (long base64/hex blobs, credential-shaped
+  tokens), tuned to skip request-signing / pagination / analytics parameters
+  that are legitimately long (`X-Amz-*`, SAS, `cursor`, `utm_*`, `gclid`, …)
+- oversized or active-content (`text/html`, `image/svg+xml`, JS) `data:` URIs
+- embedded `user:password@host` credentials
+- unusually long query strings or fragments
+- encoded-data blobs smuggled in a path segment (a beacon that avoids the query)
+- off-origin form actions and `meta refresh` redirects
+- `javascript:` / `vbscript:` targets
+
+Each threat carries a `reason` and the destination `target` (never the
+payload-bearing query/fragment), suitable for a warning shown to the operator.
+
+## Confusable folding (tool input)
+
+`./confusables` folds look-alike glyphs in tool-call **input** fields (paths,
+commands) to their ASCII canon. A denied path/command spelled in homoglyphs (a
+Cyrillic `а` for ASCII `a`) would not match an ASCII deny rule; folding closes
+that cross-script bypass (CVE-2025-54794 class). Folding is per-character and
+context-free, so it also catches an **isolated** confusable with no ASCII anchor
+that a context-sensitive canonicaliser would leave alone. The homoglyph engine
+is **injected** (`{ scan }`)—the package owns no glyph map. An all-ASCII field
+never invokes the scanner. This narrows a steganographic channel; it is not an
+enforcement boundary (distinct code points would not match a deny rule anyway).
+
+## Instruction-file scanning
+
+`./instructions` scans the markdown that loads as model context (`CLAUDE.md`,
+`AGENTS.md`, `SKILL.md`, any `.claude` markdown—a **caller-supplied glob
+set**), which bypasses a tool-output sanitizer entirely. It flags long invisible
+runs, decodes the two common smuggling encodings (Unicode **tag** characters →
+ASCII, zero-width **binary**), and catches scattered payloads below the long-run
+threshold. `cleanFile` strips the payload in place (Layer-1 strip), failing loud
+if a contaminated file cannot be rewritten.
+
+## User-prompt verdict
+
+`./prompt` classifies a submitted prompt as **pass / pass-with-note / block** on
+payload-capable invisible Unicode and ANSI. A prompt-submission channel usually
+cannot rewrite the prompt in place, so the only neutralization is to block.
+One carve-out: a prompt whose only escape content is display-only SGR color
+passes with a note (pasting colored terminal output is the common case, and SGR
+cannot move the cursor, erase, or carry an OSC payload). The SGR-only test
+gates on both the 7-bit ESC (`U+001B`) introducer and the whole 8-bit C1
+control block (U+0080–U+009F)—not just the CSI byte (`U+009B`)—so a
+C1-introduced cursor-move, erase, or OSC/DCS/SOS/PM/APC string is never
+mistaken for benign color.
+
+## Tool-output pipeline & Layer 5
+
+`./output` runs Layers 1–4 over structured tool output, **preserving its shape**
+(a harness that gets a shape-mismatched value silently shows the raw output).
+Layer 4 (secret redaction) is an **injected** redactor and is the one
+fail-closed path: a redactor that throws makes the pipeline rethrow, so the
+caller suppresses the output rather than emit an unvetted value. Layer 5 is a
+deliberately thin, safe slot: the injected filter returns **verbatim spans to
+delete** (never replacement text), so even a compromised filter can only remove
+legitimate content—it can never inject bytes into the model’s view. A live
+second-LLM injection filter is the caller’s to wire behind that contract.
+
+The same "never inject" property governs the filter’s `warning`: it is a
+**closed enum code** (`FILTER_WARNING`: `spans-removed` / `filter-flagged` /
+`filter-error`), and the **library**—not the filter—owns the human-readable
+string each code maps into `warnings`. This closes a seam the delete-only span
+contract left open: `warnings` is concatenated into the model-facing context
+**without** re-running Layer 1, so a compromised or prompt-injected filter that
+could return free-text `warning` would smuggle attacker bytes straight to the
+model. A filter that returns any value outside the enum makes the pipeline
+**throw** (fail loud), exactly as an unrunnable Layer-4 redactor does—no
+filter-supplied byte (span or warning) ever reaches the model’s view.
+
+## Edit-repair / rehydration
+
+Sanitizing the model’s view of a file makes that view diverge from disk: Layer 1
+deletes invisible/ANSI runs, and Layer 4 replaces secrets with `[REDACTED…]`
+placeholders. An Edit whose `old_string` was copied from that view then fails
+exact-match, and a whole-file Write would persist the placeholder over the real
+secret—so a sanitizer _without_ rehydration silently breaks editing.
+`./rehydrate` (offset machinery in `./view-map`) re-derives the sanitized view,
+locates the model’s `old_string` in it, and maps it span-exact back to the
+on-disk bytes across both placeholder expansion and stripped invisible runs,
+substituting placeholders in `new_string` with the real secrets. Two invariants
+are load-bearing and **fail closed**:
+
+- **Never mis-anchor.** Greedy deletion alignment is ambiguous when a stripped
+  run abuts kept text it resembles; any call that does not re-clean back to the
+  span’s view, matches multiple view locations, or cuts through a placeholder is
+  **denied** with an instructive reason rather than edited at a guessed anchor.
+- **Never expose a secret this call rehydrated.** Before rewriting, the
+  would-be post-edit content is re-sanitized (Layer 1 + the injected
+  redactor); if any secret THIS EDIT resolved from a placeholder would survive
+  in the model’s next view (e.g. an edit whose `old_string`/`new_string`
+  carries a placeholder and relabels a field the redactor no longer
+  recognizes), the call is **denied**. This check only runs when the edit
+  touches a placeholder — a relabel that never names one (e.g. `password=` →
+  `notes=` with no `[REDACTED…]` in either string) is an accepted scope gap,
+  not covered here: simulating full-file exposure on every relabel-adjacent
+  edit would re-run redaction on every `Edit` call and risk false denials on a
+  legitimate relabel in a large file. The secret flows disk → tool input only;
+  the model’s next view is sanitized again.
+
+File access and the redactor are injected via `io`; the package performs no I/O
+of its own and bundles no secret engine.