agent-sanitizer 2.19.2 → 2.19.3

package/README.md

@@ -94,6 +94,11 @@ owns each message, and any value outside the enum makes `sanitizeText` **throw**
 | `filter-flagged`      | The filter flagged the output as a possible injection without deleting (content intact) |
 | `filter-error`        | The filter reported a non-fatal internal error while scanning (a fatal filter throws)   |
 
+Every span is matched against the **original** text and the deletions applied in
+a single ordered pass, so the bytes a filter can remove are exactly the bytes its
+spans matched in the input — an earlier deletion can never manufacture a match
+for a later span (overlapping spans resolve first-match-wins).
+
 ## What installing entails
 
 Installing the plugin puts four hooks on every session, and this is what they

package/THREAT-MODEL.md

@@ -169,8 +169,12 @@ fail-closed path: a redactor that throws makes the pipeline rethrow, so the
 caller suppresses the output rather than emit an unvetted value. Layer 5 is a
 deliberately thin, safe slot: the injected filter returns **verbatim spans to
 delete** (never replacement text), so even a compromised filter can only remove
-legitimate content—it can never inject bytes into the model’s view. A live
-second-LLM injection filter is the caller’s to wire behind that contract.
+legitimate content—it can never inject bytes into the model’s view. That removal
+is bounded to the spans the filter actually named: every span is matched against
+the **original** text and the deletions applied in a single ordered pass, so an
+earlier deletion cannot join two kept regions into a match for a later span and
+erase text neither span occurred in. A live second-LLM injection filter is the
+caller’s to wire behind that contract.
 
 The same "never inject" property governs the filter’s `warning`: it is a
 **closed enum code** (`FILTER_WARNING`: `spans-removed` / `filter-flagged` /

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-sanitizer",
-  "version": "2.19.2",
+  "version": "2.19.3",
   "description": "Defend an agent against hidden-content injection: strip payload-capable invisible Unicode and ANSI, splice out human-invisible HTML, and flag data-exfil URLs in untrusted text before any model sees it.",
   "type": "module",
   "repository": {

package/src/output.mjs

@@ -27,6 +27,7 @@
 import { CATEGORY, describeStripped, isSgrOnly } from "./invisible.mjs";
 import { HTML_TAG_PRESENT, MD_LINK_HINT } from "./gates.mjs";
 import { applyLayer1, LONE_SURROGATE_RE } from "./layer1.mjs";
+import { orderedMatches, spliceOrdered } from "./view-map.mjs";
 
 /**
  * Closed enum of LIBRARY-OWNED Layer-5 warning codes — the ONLY warning values
@@ -206,22 +207,38 @@ export function describeWarned(warned) {
 /**
  * Delete each verbatim span in `spans` from `text`. The secure Layer-5
  * primitive: a filter can only ask for deletions, so this can never inject
- * bytes. Returns the new text and how many distinct span-occurrences were
- * removed (0 when no span was present).
+ * bytes. Returns the new text and how many span occurrences were removed (0
+ * when no span was present).
+ *
+ * Every occurrence is located in the ORIGINAL `text` and the deletions applied
+ * in one ordered pass ({@link spliceOrdered}), so every removed byte lies inside
+ * a match some span had in the INPUT. Deleting span-by-span with a
+ * chained `split`/`join` would not hold that line: an earlier deletion joins the
+ * bytes on either side of it and can CREATE a match for a later span that never
+ * occurred in the input — `deleteVerbatimSpans("PRE-XX-POST", ["-XX-", "PREPOST"])`
+ * then deletes the whole document. That would widen the Layer-5 seam's blast
+ * radius (see the module doc) from "a compromised filter can at most remove the
+ * content it named" to "it can remove content it never named".
+ *
+ * Overlapping spans are resolved first-match-wins, so `removed` counts the
+ * occurrences actually spliced out, never a double-count of the same bytes.
  * @param {string} text
  * @param {string[]} spans
  * @returns {{ text: string, removed: number }}
  */
 export function deleteVerbatimSpans(text, spans) {
-  let out = text;
-  let removed = 0;
-  for (const span of spans) {
-    if (!span) continue;
-    const parts = out.split(span);
-    removed += parts.length - 1;
-    out = parts.join("");
-  }
-  return { text: out, removed };
+  // Keep only non-empty STRING spans. The filter is untrusted JS, not a
+  // type-checked caller, so the array can hold anything: `indexOf(123)` would
+  // silently match the literal text "123" (deleting content the filter never
+  // named), and `occurrences` steps by `needle.length` — `undefined` for a
+  // number — making `indexOf(needle, NaN)` clamp back to the same index and
+  // loop forever. Fail open on a malformed entry rather than mangle bytes or
+  // hang the pipeline.
+  const usable = spans.filter(
+    (span) => typeof span === "string" && span !== "",
+  );
+  const spliced = spliceOrdered(text, orderedMatches(text, usable), () => "");
+  return { text: spliced.text, removed: spliced.spans.length };
 }
 
 /**

package/src/rehydrate.mjs

@@ -51,6 +51,7 @@ import {
   occurrences,
   overlapAwareCount,
   orderedMatches,
+  spliceOrdered,
   alignDeletions,
   resolveSpan,
   rehydrateNewString,
@@ -415,9 +416,9 @@ async function rehydrateWrite(ti, view, io, hint) {
     };
 
   // Resolve each of this file's placeholder texts to its single secret first,
-  // then splice in ONE ordered pass (R6). A chained `out.split(ph).join(secret)`
-  // per placeholder is unsound: an inserted secret whose bytes contain a later
-  // placeholder text would be re-matched and corrupted by the next split.
+  // then splice in ONE ordered pass (R6) via the shared `spliceOrdered` — see
+  // its doc for why a chained `out.split(ph).join(secret)` per placeholder is
+  // unsound.
   const valueByPh = new Map();
   for (const phText of texts) {
     const produced = view.pairs.filter((pair) => pair.placeholder === phText);
@@ -438,23 +439,15 @@ async function rehydrateWrite(ti, view, io, hint) {
       };
     valueByPh.set(phText, values[0]);
   }
-  const matches = orderedMatches(ti.content, texts);
-  let out = "";
-  let last = 0;
-  // Byte ranges in `out` occupied by the substituted secret values. A hint
-  // occurrence inside one of these is a pathological secret whose bytes contain
-  // the hint prefix, NOT a placeholder the model pasted — so it is excluded from
-  // the foreign-placeholder scan below.
-  const secretSpans = [];
-  for (const match of matches) {
-    const secret = valueByPh.get(match.text);
-    out += ti.content.slice(last, match.index);
-    const secretStart = out.length;
-    out += secret;
-    secretSpans.push({ start: secretStart, end: out.length });
-    last = match.index + match.text.length;
-  }
-  out += ti.content.slice(last);
+  // `secretSpans`: byte ranges in `out` occupied by the substituted secret
+  // values. A hint occurrence inside one of these is a pathological secret whose
+  // bytes contain the hint prefix, NOT a placeholder the model pasted — so it is
+  // excluded from the foreign-placeholder scan below.
+  const { text: out, spans: secretSpans } = spliceOrdered(
+    ti.content,
+    orderedMatches(ti.content, texts),
+    (match) => valueByPh.get(match.text),
+  );
   const secrets = [...valueByPh.values()];
 
   // R3: the new content may mix a valid same-file placeholder (substituted

package/src/view-map.mjs

@@ -223,9 +223,15 @@ export function resolveSpan(
 }
 
 /**
- * All occurrences of any needle in `text`, ordered by position. Placeholder
- * texts never substring-overlap one another (each ends in "]" right after its
- * distinguishing label), so the sorted matches are non-overlapping.
+ * All occurrences of any needle in `text`, ordered by position. Every index is
+ * computed against the ORIGINAL `text`, so the caller can splice them in one
+ * pass ({@link spliceOrdered}). Redaction placeholder texts never
+ * substring-overlap one another (each ends in "]" right after its
+ * distinguishing label), so for that caller the sorted matches are also
+ * non-overlapping; needles from an untrusted source (a Layer-5 filter's
+ * removeSpans) can overlap, which spliceOrdered resolves first-match-wins.
+ * Distinct needles matching at the SAME index keep `needles` order (Array#sort
+ * is stable), so first-match-wins is deterministic.
  * @param {string} text
  * @param {string[]} needles
  * @returns {{text: string, index: number}[]}
@@ -238,6 +244,47 @@ export function orderedMatches(text, needles) {
   return out.sort((left, right) => left.index - right.index);
 }
 
+/**
+ * Replace every match in `matches` with `replacementFor(match, i)` in a SINGLE
+ * ordered pass over `text`. THE splice primitive for this codebase — the sole
+ * sound way to substitute several needles at once.
+ *
+ * A chained `text.split(needle).join(value)` per needle is unsound in both
+ * directions, which is why no caller may hand-roll one:
+ *   - substitution: an inserted value whose bytes contain a LATER needle is
+ *     re-matched by the next split and corrupted (or partially exposed);
+ *   - deletion: an earlier deletion joins the bytes on either side of it and
+ *     can CREATE a later needle's match, deleting text that needle never
+ *     matched in the input ("PRE-XX-POST" minus "-XX-" yields "PREPOST").
+ * Because every index in `matches` is measured against the original `text`,
+ * this pass only ever touches bytes the caller actually matched.
+ *
+ * Overlapping matches are resolved first-match-wins: a match starting before
+ * the previous one ended is skipped, never spliced at a shifted offset.
+ * `i` is the match's index in `matches` (stable across skips) so a caller
+ * pairing matches positionally with its own array stays aligned.
+ * @param {string} text
+ * @param {{text: string, index: number}[]} matches ordered by index, indices into `text`
+ * @param {(match: {text: string, index: number}, i: number) => string} replacementFor
+ * @returns {{text: string, spans: {start: number, end: number}[]}} spliced text
+ *   and the [start, end) range each replacement occupies in it
+ */
+export function spliceOrdered(text, matches, replacementFor) {
+  let out = "";
+  let last = 0;
+  /** @type {{start: number, end: number}[]} */
+  const spans = [];
+  matches.forEach((match, i) => {
+    if (match.index < last) return;
+    out += text.slice(last, match.index);
+    const start = out.length;
+    out += replacementFor(match, i);
+    spans.push({ start, end: out.length });
+    last = match.index + match.text.length;
+  });
+  return { text: out + text.slice(last), spans };
+}
+
 /**
  * On-disk [start, end) span of every redaction pair, mapped from its view
  * offset through placeholder expansion (view → cleaned) and stripped invisible
@@ -308,24 +355,16 @@ export function rehydrateNewString(oldS, newS, spanPairs, filePairs) {
     newSeq.length === spanPairs.length &&
     newSeq.every((match, i) => match.text === spanPairs[i].placeholder)
   ) {
-    let out = "";
-    let last = 0;
-    newSeq.forEach((match, i) => {
-      out += newS.slice(last, match.index) + spanPairs[i].original;
-      last = match.index + match.text.length;
-    });
     return {
-      text: out + newS.slice(last),
+      text: spliceOrdered(newS, newSeq, (_match, i) => spanPairs[i].original)
+        .text,
       secrets: spanPairs.map((pair) => pair.original),
     };
   }
 
   // Each placeholder text must name exactly one secret; resolve that mapping
-  // first, then splice in a SINGLE ordered pass. A chained
-  // `out.split(ph).join(secret)` per placeholder is unsound: an inserted secret
-  // whose bytes happen to contain a later placeholder text would be re-matched
-  // and corrupted (or partially exposed) by the next split. One pass over the
-  // ordered match positions only ever touches the original new_string bytes.
+  // first, then splice in a SINGLE ordered pass (see spliceOrdered for why a
+  // chained `out.split(ph).join(secret)` per placeholder is unsound).
   const valueByPh = new Map();
   for (const phText of new Set(newSeq.map((match) => match.text))) {
     const values = [
@@ -344,11 +383,9 @@ export function rehydrateNewString(oldS, newS, spanPairs, filePairs) {
       };
     valueByPh.set(phText, values[0]);
   }
-  let out = "";
-  let last = 0;
-  for (const match of newSeq) {
-    out += newS.slice(last, match.index) + valueByPh.get(match.text);
-    last = match.index + match.text.length;
-  }
-  return { text: out + newS.slice(last), secrets: [...valueByPh.values()] };
+  return {
+    text: spliceOrdered(newS, newSeq, (match) => valueByPh.get(match.text))
+      .text,
+    secrets: [...valueByPh.values()],
+  };
 }

package/types/output.d.mts

@@ -26,8 +26,21 @@ export function describeWarned(warned: {
 /**
  * Delete each verbatim span in `spans` from `text`. The secure Layer-5
  * primitive: a filter can only ask for deletions, so this can never inject
- * bytes. Returns the new text and how many distinct span-occurrences were
- * removed (0 when no span was present).
+ * bytes. Returns the new text and how many span occurrences were removed (0
+ * when no span was present).
+ *
+ * Every occurrence is located in the ORIGINAL `text` and the deletions applied
+ * in one ordered pass ({@link spliceOrdered}), so every removed byte lies inside
+ * a match some span had in the INPUT. Deleting span-by-span with a
+ * chained `split`/`join` would not hold that line: an earlier deletion joins the
+ * bytes on either side of it and can CREATE a match for a later span that never
+ * occurred in the input — `deleteVerbatimSpans("PRE-XX-POST", ["-XX-", "PREPOST"])`
+ * then deletes the whole document. That would widen the Layer-5 seam's blast
+ * radius (see the module doc) from "a compromised filter can at most remove the
+ * content it named" to "it can remove content it never named".
+ *
+ * Overlapping spans are resolved first-match-wins, so `removed` counts the
+ * occurrences actually spliced out, never a double-count of the same bytes.
  * @param {string} text
  * @param {string[]} spans
  * @returns {{ text: string, removed: number }}

package/types/view-map.d.mts

@@ -106,9 +106,15 @@ export function resolveSpan(content: string, cleaned: string, view: {
     }[];
 } | null;
 /**
- * All occurrences of any needle in `text`, ordered by position. Placeholder
- * texts never substring-overlap one another (each ends in "]" right after its
- * distinguishing label), so the sorted matches are non-overlapping.
+ * All occurrences of any needle in `text`, ordered by position. Every index is
+ * computed against the ORIGINAL `text`, so the caller can splice them in one
+ * pass ({@link spliceOrdered}). Redaction placeholder texts never
+ * substring-overlap one another (each ends in "]" right after its
+ * distinguishing label), so for that caller the sorted matches are also
+ * non-overlapping; needles from an untrusted source (a Layer-5 filter's
+ * removeSpans) can overlap, which spliceOrdered resolves first-match-wins.
+ * Distinct needles matching at the SAME index keep `needles` order (Array#sort
+ * is stable), so first-match-wins is deterministic.
  * @param {string} text
  * @param {string[]} needles
  * @returns {{text: string, index: number}[]}
@@ -117,6 +123,44 @@ export function orderedMatches(text: string, needles: string[]): {
     text: string;
     index: number;
 }[];
+/**
+ * Replace every match in `matches` with `replacementFor(match, i)` in a SINGLE
+ * ordered pass over `text`. THE splice primitive for this codebase — the sole
+ * sound way to substitute several needles at once.
+ *
+ * A chained `text.split(needle).join(value)` per needle is unsound in both
+ * directions, which is why no caller may hand-roll one:
+ *   - substitution: an inserted value whose bytes contain a LATER needle is
+ *     re-matched by the next split and corrupted (or partially exposed);
+ *   - deletion: an earlier deletion joins the bytes on either side of it and
+ *     can CREATE a later needle's match, deleting text that needle never
+ *     matched in the input ("PRE-XX-POST" minus "-XX-" yields "PREPOST").
+ * Because every index in `matches` is measured against the original `text`,
+ * this pass only ever touches bytes the caller actually matched.
+ *
+ * Overlapping matches are resolved first-match-wins: a match starting before
+ * the previous one ended is skipped, never spliced at a shifted offset.
+ * `i` is the match's index in `matches` (stable across skips) so a caller
+ * pairing matches positionally with its own array stays aligned.
+ * @param {string} text
+ * @param {{text: string, index: number}[]} matches ordered by index, indices into `text`
+ * @param {(match: {text: string, index: number}, i: number) => string} replacementFor
+ * @returns {{text: string, spans: {start: number, end: number}[]}} spliced text
+ *   and the [start, end) range each replacement occupies in it
+ */
+export function spliceOrdered(text: string, matches: {
+    text: string;
+    index: number;
+}[], replacementFor: (match: {
+    text: string;
+    index: number;
+}, i: number) => string): {
+    text: string;
+    spans: {
+        start: number;
+        end: number;
+    }[];
+};
 /**
  * On-disk [start, end) span of every redaction pair, mapped from its view
  * offset through placeholder expansion (view → cleaned) and stripped invisible