agent-sanitizer 2.64.2 → 2.65.0

package/THREAT-MODEL.md

@@ -122,8 +122,23 @@ range is preserved verbatim** (no re-serialization). Unclosed hidden markup
 extends to the end of the fragment—fail-closed for truncated input.
 
 Scripting/resource tags (`script`, `style`, `object`, `embed`, `iframe`, `svg`,
-`math`) and `data:` URI resources are **reported, not removed**: their bodies are
-page source the model may legitimately need to inspect.
+`math`, `template`) and `data:` URI resources are **reported, not removed**:
+their bodies are page source the model may legitimately need to inspect.
+
+A `<template>`'s content is a fragment the browser parses but never renders, so
+instructions placed there reach a model reading the source and no human reading
+the page. It is also where web components and client-side renderers keep their
+UI markup, so the template is reported rather than spliced, and its content is
+scanned like any other markup: a hidden element or comment inside it is
+spliced, a `<script>` inside it reported, a URL inside it read by Layer 3. A
+declarative shadow root (`shadowrootmode="open"`/`"closed"`) is reported too,
+since one the parser fails to attach (a second root on one host, say) is an
+ordinary inert template; its content is laid out in its host's box, so a flex
+host blockifies it as it would a light-DOM child. Slot assignment is not
+modelled: a hidden `<slot>` splices only itself, not the light-DOM children
+the browser renders through it. A `template` inside SVG or MathML (or an HTML
+`template.x`) is not reported: it has no template content, so its children are
+ordinary children and are scanned as such.
 
 HTML comments (`<!--…-->`, and the bogus `<!…>`/`<?…?>` forms) are spliced
 like hidden elements — a human viewing the rendered page never sees them. But
@@ -146,9 +161,12 @@ Report—never rewrite—URLs in markdown links/images/definitions, every HTML/S
 URL attribute in `src/url-attributes.mjs` (`src`, `href`, `xlink:href`,
 `srcset`/`imagesrcset`, `ping`, `poster`, `object data`, form
 `action`/`formaction`, `meta refresh`, …), CSS `url()` in `style` attributes and
-`<style>` elements, and the same again inside HTML comments, that are shaped to
-carry data off-origin. A URL's scheme is read by the WHATWG URL parser, and
-`srcset` and `meta refresh` by ports of the HTML Standard's own parsing steps:
+`<style>` elements, and the same again inside HTML comments and `<template>`
+content, that are shaped to carry data off-origin. A URL inside a comment or an
+inert template is reported as not auto-fetched: nothing there loads until a
+reader or a script acts on it. A URL's scheme is read by the WHATWG URL parser,
+and `srcset` and `meta refresh` by ports of the HTML Standard's own parsing
+steps:
 
 - suspicious query/fragment parameters (long base64/hex blobs, credential-shaped
   tokens), tuned to skip request-signing / pagination / analytics parameters

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-sanitizer",
-  "version": "2.64.2",
+  "version": "2.65.0",
   "description": "Defend an agent against hidden-content injection: strip payload-capable invisible Unicode and ANSI, splice out human-invisible HTML, and flag data-exfil URLs in untrusted text before any model sees it.",
   "type": "module",
   "repository": {

package/src/html-tree-adapter.mjs

@@ -174,8 +174,8 @@ export const MAX_HTML_DEPTH = 4096;
  * the name up to its first `#` or `.`, lowercased.
  *
  * WHY IT MATTERS — the converter then recurses into the parse5 node's `content`,
- * which only an HTML-namespace `<template>` has. A `template` inside SVG or
- * MathML, or an HTML `template.x` tag, has none, the converter throws,
+ * which only an HTML-namespace `<template>` has. An `<svg><template>`, a
+ * `<math><template>` or an HTML `<template.x>` has none, the converter throws,
  * and the whole document would be withheld over a benign element.
  */
 const CONVERTS_AS_TEMPLATE = /^template(?:[#.]|$)/i;
@@ -189,6 +189,15 @@ const CONVERTS_AS_TEMPLATE = /^template(?:[#.]|$)/i;
  * changes it (lowercasing, the SVG case map). The element then converts the way
  * any element without template content does: its children as `children`, where
  * every scan of the hast tree reads them, which is also where the DOM has them.
+ *
+ * WHY A MASK, and not an empty parse5 `content` handed to the element so the
+ * converter has a fragment to recurse into: hastscript's `h` and `s` (the
+ * converter builds every element with one of them) move the children of ANY
+ * element they name `template` into the hast node's `content` and leave its
+ * `children` empty, and the converter then overwrites that `content` with its
+ * conversion of the parse5 fragment. Handed an empty one, the element's
+ * children would vanish with no error, and every Layer 2/3 scan in
+ * `./html.mjs` would go blind to them.
  */
 const TEMPLATE_MASK = " ";
 

package/src/html.mjs

@@ -1502,7 +1502,13 @@ export function isHiddenStyle(styleStr) {
 // Scripting / resource-loading tags whose PRESENCE is reported to the model
 // but whose content is preserved: their bodies are page source the model may
 // legitimately need to inspect (how a page's scripts work, its styles, its
-// SVGs), so unlike hidden elements they are never removed.
+// SVGs), so unlike hidden elements they are never removed. An HTML `template`
+// is here because the browser parses its content into a fragment it never
+// renders — a reader of the source sees it, a reader of the page does not —
+// while that content is also where a component's UI markup legitimately lives.
+// A declarative shadow root is reported too: whether the parser attaches one
+// depends on its host's name and on the host having no earlier root, and a
+// root that fails to attach is an ordinary inert template.
 export const REPORTED_TAGS = new Set([
   "script",
   "style",
@@ -1511,8 +1517,34 @@ export const REPORTED_TAGS = new Set([
   "iframe",
   "svg",
   "math",
+  "template",
 ]);
 
+/**
+ * True for a template whose content the page never renders: every one but a
+ * declarative shadow root (`shadowrootmode` of `open` or `closed`), whose
+ * content the browser's parser attaches to the parent as a shadow tree.
+ * @param {any} node a hast element with template content (a `content` root)
+ * @returns {boolean}
+ */
+function isInertTemplate(node) {
+  const mode = String(node.properties.shadowRootMode ?? "").toLowerCase();
+  return mode !== "open" && mode !== "closed";
+}
+
+/**
+ * True when Layer 2 reports `node`'s presence (see {@link REPORTED_TAGS}).
+ * Only an HTML `template` has template content; one inside SVG or MathML, or a
+ * `template.x`, is an ordinary element whose children are scanned as children.
+ * @param {any} node a hast element
+ * @returns {boolean}
+ */
+function isReportedElement(node) {
+  return node.tagName === "template"
+    ? node.content !== undefined
+    : REPORTED_TAGS.has(node.tagName);
+}
+
 // HTML void elements: they never carry content and never emit a closing tag, so
 // a hidden one (<img hidden>, <input hidden>, …) must be spliced as a single node
 // — opening a balance region for it would run to the container's end (no close
@@ -1629,8 +1661,27 @@ function elementCascade(node) {
   return cascade;
 }
 
-// True when `node` lays out as a non-replaced INLINE box. A flex/grid `parent`
-// blockifies it; a `display:contents` parent defers to its own parent.
+// The template whose content each hast `content` root is, recorded by `walk`.
+const contentOwners = new WeakMap();
+
+// The box `parent`'s children are laid out in: a `display:contents` parent
+// defers to its own parent, and a declarative shadow root's content to the
+// template's parent, the shadow host whose box its children render in.
+/** @param {any} parent @returns {any} */
+function layoutContainer(parent) {
+  let container = parent;
+  for (;;) {
+    const template = contentOwners.get(container);
+    if (template && !isInertTemplate(template))
+      container = parents.get(template);
+    else if (elementCascade(container).textOf("display") === "contents")
+      container = parents.get(container);
+    else return container;
+  }
+}
+
+// True when `node` lays out as a non-replaced INLINE box. A flex/grid layout
+// container (see {@link layoutContainer}) blockifies it.
 /** @param {any} node @param {any} [parent] @returns {boolean} */
 function isInlineBox(node, parent) {
   if (!INLINE_ELEMENTS.has(node.tagName)) return false;
@@ -1642,10 +1693,7 @@ function isInlineBox(node, parent) {
     BLOCKIFYING_VALUES.has(textOf("float"))
   )
     return false;
-  let container = parent;
-  while (elementCascade(container).textOf("display") === "contents")
-    container = parents.get(container);
-  return !elementCascade(container)
+  return !elementCascade(layoutContainer(parent))
     .textOf("display")
     .split(" ")
     .some((token) => ITEM_BLOCKIFYING_DISPLAYS.has(token));
@@ -1893,6 +1941,13 @@ function hasDataSrc(el) {
  * the `unist-util-visit` contract, which is what the call sites here are
  * written against.
  *
+ * One addition to that contract: an HTML `template`'s content (hast keeps it
+ * as a `content` root beside the element's empty `children`) is walked after
+ * its children, with the template as its parent, and `SKIP` leaves it
+ * unvisited too. Rendered or not (see {@link isInertTemplate}), that markup is
+ * in the source a model reads, so a scan that stopped at `children` would pass
+ * any hidden element, script or URL placed in it unseen.
+ *
  * Spelled out rather than imported because `unist-util-visit` allocates a fresh
  * ancestors array and a closure per node, and on a document that is one long
  * flat sibling list — 32k `<p>` elements per megabyte of ordinary HTML — that
@@ -1921,6 +1976,12 @@ function walk(tree, test, visitor, treeParent = undefined) {
         : undefined;
     if (result === EXIT) return;
     if (result === SKIP) continue;
+    if (node.type === "element" && node.content !== undefined) {
+      contentOwners.set(node.content, node);
+      nodes.push(node.content);
+      indices.push(undefined);
+      parents.push(node);
+    }
     const children = node.children;
     if (children === undefined) continue;
     for (let i = children.length - 1; i >= 0; i--) {
@@ -2163,7 +2224,7 @@ function scanFragmentTree(html, tree) {
   const warned = newWarned();
   /** @param {any} node */
   const countElement = (node) => {
-    if (REPORTED_TAGS.has(node.tagName)) countTag(warned, node.tagName);
+    if (isReportedElement(node)) countTag(warned, node.tagName);
     if (hasDataSrc(node)) warned.dataSrc += 1;
   };
   // Subtrees still to scan, each with its parent: the tree itself, then the
@@ -2485,7 +2546,11 @@ function scanInlineChildren(node, text, ranges, warned) {
           // A raw-text open tag starts an opaque region (a self-closing `/>`
           // does not apply to these in HTML — they always open).
           if (RAW_TEXT_ELEMENTS.has(el.tagName)) rawText = el.tagName;
-          if (REPORTED_TAGS.has(el.tagName)) countTag(warned, el.tagName);
+          // `el` is one tag parsed alone, so a `<template>` inside an inline
+          // `<svg>`/`<math>` reads as HTML here and is reported, where the
+          // source branch, which sees the namespace, does not: this errs toward
+          // reporting.
+          if (isReportedElement(el)) countTag(warned, el.tagName);
           if (hasDataSrc(el)) warned.dataSrc += 1;
         }
       }
@@ -3628,20 +3693,40 @@ function documentUrls(text, parseMd, parseHtml) {
   });
   /** @type {string[]} */
   const comments = [];
-  walk(parseHtml(text), null, (/** @type {any} */ node) => {
-    if (node.type === "comment") comments.push(node.value);
-    else if (node.type === "element") urls.push(...elementUrls(node));
-  });
+  // Nodes inside an inert template's content, where nothing loads, submits or
+  // redirects until a script clones it into the page.
+  const inert = new Set();
+  walk(
+    parseHtml(text),
+    null,
+    (/** @type {any} */ node, _index, /** @type {any} */ parent) => {
+      if (
+        inert.has(parent) ||
+        (parent?.content === node && isInertTemplate(parent))
+      )
+        inert.add(node);
+      if (node.type === "comment") comments.push(node.value);
+      else if (node.type === "element")
+        for (const url of elementUrls(node))
+          urls.push(
+            inert.has(node)
+              ? { ...url, autoFetched: false, context: "resource" }
+              : url,
+          );
+    },
+  );
   return { urls, comments };
 }
 
 // Every URL a Layer-3 detector reads: markdown links/images/definitions and
-// the URL attributes of every element, also inside comments. Both detectors
-// below consume this, so a node type is in scope for all of them or for none;
-// `parseMarkdown` memoizes its tree, so back-to-back detectors share the parse.
-// A comment is invisible on the page but not to a model reading the source:
-// its text is read one level deep with its markup openers defused, and what it
-// names is not fetched and has no off-origin rule.
+// the URL attributes of every element, also inside comments and template
+// content. Both detectors below consume this, so a node type is in scope for
+// all of them or for none; `parseMarkdown` memoizes its tree, so back-to-back
+// detectors share the parse. A comment is invisible on the page but not to a
+// model reading the source: its text is read one level deep with its markup
+// openers defused, and what it names is not fetched and has no off-origin
+// rule. An inert template's content gets the same two answers: nothing in it
+// is fetched, submitted or followed until a script clones it into the page.
 /** @param {string} text @returns {CollectedUrl[]} */
 function collectUrls(text) {
   const { urls, comments } = documentUrls(text, parseMarkdown, parseFragment);