agent-sanitizer 2.66.0 → 2.66.2

package/THREAT-MODEL.md

@@ -134,9 +134,13 @@ spliced, a `<script>` inside it reported, a URL inside it read by Layer 3. A
 declarative shadow root (`shadowrootmode="open"`/`"closed"`) is reported too,
 since one the parser fails to attach (a second root on one host, say) is an
 ordinary inert template; its content is laid out in its host's box, so a flex
-host blockifies it as it would a light-DOM child. Slot assignment is not
-modelled: a hidden `<slot>` splices only itself, not the light-DOM children
-the browser renders through it. A `template` inside SVG or MathML (or an HTML
+host blockifies it as it would a light-DOM child. An inherited hide on the host
+(`visibility:hidden`, say) is judged through the root the parser attaches (the
+first one, on a host `attachShadow` accepts), whose content inherits from the
+host and never from the template's own attributes; a slot given light-DOM
+children there renders them, so its fallback content is spliced. Slot
+assignment is otherwise not modelled: a hidden `<slot>` splices only itself,
+not the light-DOM children the browser renders through it. A `template` inside SVG or MathML (or an HTML
 `template.x`) is not reported: it has no template content, so its children are
 ordinary children and are scanned as such.
 
@@ -1020,38 +1024,44 @@ up to the constant factors a parse pays, and one sort where findings or splice
 ranges arrive in an order the caller does not promise (`foldConfusables`,
 `mergeRanges`). The bound that matters is that nothing is _quadratic_ — a sort
 of the findings in a 1 MB paste is milliseconds, a quadratic rescan of it is
-minutes — and quadratic is what the two gates below are calibrated to catch,
+minutes — and quadratic is what the two tiers below are built to catch,
 because neither can see what the other does.
 
-**Static — every pattern.** `tests/test_redos_js_static_guard.py` drives
-regexploit over an inventory that `scripts/extract-js-regexes.mjs` extracts by
-walking the TypeScript AST of every shipped `.mjs` (not by matching regex source
-with a regex), and fails on any pattern with super-linear backtracking. A
-`RegExp(…)` built at runtime must be statically resolvable or carry a written
-exemption, so a new dynamic pattern is a red build rather than a hole.
-`tests/secrets/test_redos_static_guard.py` is the twin over the Python engine,
-and `eslint-plugin-redos` reports the same class in the editor, before CI.
-
-**Empirical — everything the static gates cannot see.** Two shapes of quadratic
-carry no backtracking at all, so no pattern analyzer will ever report them:
-
-- an **unanchored** pattern retried at every start offset, each attempt linear
-  and none of them backtracking — `(?=.*[A-Z])(?=.*[0-9])` against a long value
-  carrying neither class, or a `[^>]*$` tail rescanned from every `<`;
-- work that is not a pattern at all — a string rebuilt once per finding, a
-  prefix re-sliced inside a loop, a re-walk of an already-walked node.
-
-`test/algorithmic-complexity.test.mjs` runs an entry point over an adversarial
-input at four sizes spanning 8x and fits how its CPU cost grows — an exponent,
-classified against the one pair of thresholds every growth gate shares
-(`test/helpers/cpu-timing.mjs`, twin `tests/_growth.py`) — with a
-known-quadratic specimen as the control that proves the harness can still see
-one. Each case also asserts, at the size it measures, that
-the input still reaches the code the case is about — a length gate answering
-first is how such a case quietly degrades into timing nothing.
-`tests/secrets/test_algorithmic_complexity.py` is the twin over the Python
-engine. `test/hook-latency.test.mjs` covers the three blocking hook paths
-instead, in cost per calibration pass, with one wall-clock ceiling.
+**Deterministic — every pattern, no clock.** `tests/test_redos_js_static_guard.py`
+drives regexploit over an inventory that `scripts/extract-js-regexes.mjs`
+extracts by walking the TypeScript AST of every shipped `.mjs` (not by matching
+regex source with a regex), and fails on any pattern with super-linear
+backtracking. A `RegExp(…)` built at runtime must be statically resolvable or
+carry a written exemption, so a new dynamic pattern is a red build rather than a
+hole. `tests/secrets/test_redos_static_guard.py` is the twin over the Python
+engine. In `pnpm lint`, `eslint-plugin-redos` (recheck) and
+`eslint-plugin-regexp`'s `no-super-linear-backtracking` and
+`no-super-linear-move` read every shipped regex literal — the last of them
+reports the shape that carries no backtracking at all, an **unanchored**
+pattern retried at every start offset (`(?=.*[A-Z])(?=.*[0-9])` against a long
+value carrying neither class). `test/regex-recheck.test.mjs` hands every
+pattern a shipped module actually compiles at runtime to recheck. Every recheck
+verdict runs with its wall-clock incubation and attack timeouts off, so it
+rests on step counts alone and no machine's speed can change it.
+
+**Budgets — everything that is not a pattern.** Work that is not a regex at
+all — a string rebuilt once per finding, a prefix re-sliced inside a loop, a
+re-walk of an already-walked node, a parser's own look-back — no pattern
+analyzer will ever report. `test/algorithmic-complexity.test.mjs` runs each
+entry point ONCE, after a warm-up, over an adversarial input sized so the
+linear implementation costs tens to hundreds of milliseconds, against an
+absolute CPU ceiling at least ten times that reading (`assertUnderBudget` in
+`test/helpers/budget.mjs`, twin `tests/_budget.py`). A quadratic at that size
+overshoots the ceiling by another order of magnitude, so no runner is slow or
+busy enough to blur the two; a constant-factor slowdown passes, by design. Each
+case also asserts, at the size it measures, that the input still reaches the
+code the case is about — a length gate answering first is how such a case
+quietly degrades into timing nothing. `tests/secrets/test_algorithmic_complexity.py`
+is the twin over the Python engine. `test/hook-latency.test.mjs` holds the
+three blocking hook paths to absolute ceilings per 256 KB call, read
+uninstrumented in CI. Every test verdict in the repo that reads a clock goes
+through that one helper (`test/real-clock-thresholds.test.mjs` and its Python
+twin enforce it).
 
 **Idioms new code is held to.** Each is the fix for a defect one of the gates
 caught:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-sanitizer",
-  "version": "2.66.0",
+  "version": "2.66.2",
   "description": "Defend an agent against hidden-content injection: strip payload-capable invisible Unicode and ANSI, splice out human-invisible HTML, and flag data-exfil URLs in untrusted text before any model sees it.",
   "type": "module",
   "repository": {
@@ -62,11 +62,13 @@
     "esbuild": "0.28.1",
     "eslint": "10.4.0",
     "eslint-plugin-redos": "4.5.0",
+    "eslint-plugin-regexp": "3.3.1",
     "fast-check": "4.8.0",
     "globals": "17.6.0",
     "lint-staged": "^17.0.5",
     "micromark-extension-gfm-autolink-literal": "2.1.0",
     "prettier": "^3.0.0",
+    "recheck": "4.5.0",
     "rehype-parse": "9.0.1",
     "remark-gfm": "4.0.1",
     "smol-toml": "^1.7.1",

package/src/html.mjs

@@ -1680,6 +1680,100 @@ function layoutContainer(parent) {
   }
 }
 
+// The elements `attachShadow` accepts as a host, besides a custom element.
+const SHADOW_HOST_TAGS = new Set([
+  "article",
+  "aside",
+  "blockquote",
+  "body",
+  "div",
+  "footer",
+  "h1",
+  "h2",
+  "h3",
+  "h4",
+  "h5",
+  "h6",
+  "header",
+  "main",
+  "nav",
+  "p",
+  "section",
+  "span",
+]);
+// Hyphenated names HTML reserves, so no custom element takes them.
+const RESERVED_CUSTOM_ELEMENT_NAMES = new Set([
+  "annotation-xml",
+  "color-profile",
+  "font-face",
+  "font-face-src",
+  "font-face-uri",
+  "font-face-format",
+  "font-face-name",
+  "missing-glyph",
+]);
+
+// True for an ASCII valid custom element name (the tokenizer only opens a tag
+// on a letter, lowercased). The grammar also admits non-ASCII names; reading
+// one as no host leaves its shadow content inert, which only ever splices more
+// of a hidden host.
+/** @param {string} name @returns {boolean} */
+function isCustomElementName(name) {
+  return (
+    name.includes("-") &&
+    !RESERVED_CUSTOM_ELEMENT_NAMES.has(name) &&
+    [...name].every(
+      (char) =>
+        (char >= "a" && char <= "z") ||
+        (char >= "0" && char <= "9") ||
+        "-._".includes(char),
+    )
+  );
+}
+
+// The declarative shadow root the parser attaches to `host`: its first template
+// child with a valid `shadowrootmode`, when `attachShadow` accepts the host.
+// Every other template stays in the tree as an ordinary, inert one.
+/** @param {any} host a hast element @returns {any} the attached template */
+function attachedShadowTemplate(host) {
+  if (!SHADOW_HOST_TAGS.has(host.tagName) && !isCustomElementName(host.tagName))
+    return undefined;
+  return host.children.find(
+    (/** @type {any} */ child) =>
+      child.content !== undefined && !isInertTemplate(child),
+  );
+}
+
+// The slot names `host`'s light children are assigned to: an element's `slot`
+// attribute, or the default slot for a text node or an element without one.
+/** @param {any} host @param {any} template @returns {Set<string>} */
+function assignedSlotNames(host, template) {
+  const names = new Set();
+  for (const child of host.children) {
+    if (child === template) continue;
+    if (child.type === "text") names.add("");
+    else if (child.type === "element")
+      names.add(String(child.properties.slot ?? ""));
+  }
+  return names;
+}
+
+// `node`'s children in render order, as runs no splice may span. A declarative
+// shadow root attached to `node` replaces its template with the content's
+// children, laid out in `node` and inheriting from it: the parser removes the
+// template, so its own attributes style nothing. That run carries the template.
+/** @param {any} node @returns {Array<[any[], any]>} */
+function renderedRuns(node) {
+  const template = attachedShadowTemplate(node);
+  if (!template) return [[node.children, undefined]];
+  const at = node.children.indexOf(template);
+  return [
+    [node.children.slice(0, at), undefined],
+    [template.content.children, template],
+    [node.children.slice(at + 1), undefined],
+  ];
+}
+
 // True when `node` lays out as a non-replaced INLINE box. A flex/grid layout
 // container (see {@link layoutContainer}) blockifies it.
 /** @param {any} node @param {any} [parent] @returns {boolean} */
@@ -1780,7 +1874,8 @@ function keepsHide(kind, cascade, inlineBox, color) {
   return keeps;
 }
 
-/** @typedef {{ kinds: Set<InheritedHide>, color: ColorState }} RegionState */
+// `slots`: the assigned slot names of the shadow tree a node sits in, if any.
+/** @typedef {{ kinds: Set<InheritedHide>, color: ColorState, slots: Set<string> | undefined }} RegionState */
 // `neutral`: nothing to see either way; `mixed`: hidden and visible content.
 /** @typedef {"hidden" | "neutral" | "visible" | "mixed"} RegionVerdict */
 
@@ -1795,11 +1890,17 @@ function regionVerdicts(root, kinds) {
   const stack = [];
   /** @param {any} node @param {RegionState} state */
   const pushChildren = (node, state) => {
-    for (const child of node.children) stack.push([child, node, state]);
+    for (const [children, template] of renderedRuns(node)) {
+      const runState = template
+        ? { ...state, slots: assignedSlotNames(node, template) }
+        : state;
+      for (const child of children) stack.push([child, node, runState]);
+    }
   };
   pushChildren(root, {
     kinds,
     color: colorState(elementCascade(root), null),
+    slots: undefined,
   });
   const order = [root];
   while (stack.length > 0) {
@@ -1821,7 +1922,11 @@ function regionVerdicts(root, kinds) {
       // Only an element's own context-free hides are ever added on the way
       // down, so a region never hides more than its elements would alone.
       const own = elementHides(node, parent);
-      if (own.has("box")) {
+      // A slot given light children renders them, never its fallback content.
+      const unrenderedFallback =
+        node.tagName === "slot" &&
+        state.slots?.has(String(node.properties.name ?? ""));
+      if (own.has("box") || unrenderedFallback) {
         verdicts.set(node, "hidden");
         continue;
       }
@@ -1834,7 +1939,7 @@ function regionVerdicts(root, kinds) {
       if (REPLACED_ELEMENTS.has(node.tagName))
         verdicts.set(node, inForce.has("visibility") ? "hidden" : "visible");
       else if (inForce.size === 0) verdicts.set(node, "visible");
-      else pushChildren(node, { kinds: inForce, color });
+      else pushChildren(node, { kinds: inForce, color, slots: state.slots });
     }
   }
   for (let i = order.length - 1; i >= 0; i--) {
@@ -1842,7 +1947,7 @@ function regionVerdicts(root, kinds) {
     if (verdicts.has(node)) continue;
     let hidden = false;
     let shown = false;
-    for (const child of node.children) {
+    for (const child of renderedRuns(node).flatMap(([children]) => children)) {
       const verdict = verdicts.get(child);
       if (verdict === "hidden") hidden = true;
       else if (verdict === "visible" || verdict === "mixed") shown = true;
@@ -1888,38 +1993,43 @@ function inheritedRegionSplice(root, kinds) {
       trailingComments = [];
       trailingElements = [];
     };
-    for (const child of node.children) {
-      const position = child.position;
-      if (
-        !position ||
-        position.start.offset < lastEnd ||
-        position.end.offset > node.position.end.offset
-      )
-        return null;
-      lastEnd = position.end.offset;
-      const verdict = verdicts.get(child);
-      if (verdict === "hidden") {
-        if (runStart === -1) {
-          closeRun(); // what precedes the run stays outside it
-          runStart = position.start.offset;
+    for (const [children, template] of renderedRuns(node)) {
+      closeRun();
+      // The template the parser removed is left standing, not spliced.
+      if (template) mixed.push(template);
+      for (const child of children) {
+        const position = child.position;
+        if (
+          !position ||
+          position.start.offset < lastEnd ||
+          position.end.offset > node.position.end.offset
+        )
+          return null;
+        lastEnd = position.end.offset;
+        const verdict = verdicts.get(child);
+        if (verdict === "hidden") {
+          if (runStart === -1) {
+            closeRun(); // what precedes the run stays outside it
+            runStart = position.start.offset;
+          }
+          runEnd = position.end.offset;
+          trailingComments = [];
+          trailingElements = [];
+        } else if (verdict === "mixed") {
+          closeRun();
+          stack.push(child);
+        } else if (verdict === "visible") {
+          closeRun();
+          rescan.push([child, node]);
+        } else if (child.type === "comment") {
+          trailingComments.push({
+            start: position.start.offset,
+            end: position.end.offset,
+            kind: "comment",
+          });
+        } else if (child.type === "element") {
+          trailingElements.push([child, node]);
         }
-        runEnd = position.end.offset;
-        trailingComments = [];
-        trailingElements = [];
-      } else if (verdict === "mixed") {
-        closeRun();
-        stack.push(child);
-      } else if (verdict === "visible") {
-        closeRun();
-        rescan.push([child, node]);
-      } else if (child.type === "comment") {
-        trailingComments.push({
-          start: position.start.offset,
-          end: position.end.offset,
-          kind: "comment",
-        });
-      } else if (child.type === "element") {
-        trailingElements.push([child, node]);
       }
     }
     closeRun();

package/types/html.d.mts

@@ -195,6 +195,7 @@ export type ColorState = {
 export type RegionState = {
     kinds: Set<InheritedHide>;
     color: ColorState;
+    slots: Set<string> | undefined;
 };
 export type RegionVerdict = "hidden" | "neutral" | "visible" | "mixed";
 export type CollectedUrl = {