scenescout 3.11.0 → 3.11.1

package/CHANGELOG.md

@@ -1,5 +1,13 @@
 # scenescout
 
+## 3.11.1
+
+### Patch Changes
+
+- 847c149: Two different defects on one element now stay two findings. Finding dedup compares what is wrong as well as where: identical evidence, a shared quoted label or a reworded title merges two findings only when they are one kind of defect (the same category, or neighbouring categories of one family such as `page-error` and `console-error`). `visual`, `ux-polish`, `a11y` and `missing-testid` are each their own kind, so a link clipped out of view and the same link styled like body text are no longer merged. When `scout_finding` merges a filing it now names the category of the finding it joined and which categories the filing could have merged with.
+- aca7a28: When a lane report is folded, a judged defect on a failing request now counts as filed when a finding names that request with its path written as a template: `GET /api/things/{id} 500` (or `:id`, or `*`) covers a lane's `GET /api/things/7 500`. A template stands only for one id segment (a number, a UUID or a long hex id), never for a word such as `me` or `export`. A defect naming several failing requests counts as filed only when every one of them is. On a request that did not fail, the lane's evidence, once the paths are aligned, must be the finding's, a restatement of part of it, or the finding's evidence followed only by the status the call should have returned, as in `… 200 as role=viewer (expected 403)`.
+- 2b8ac9a: The write policy now judges writes a page sends as it is being left. In Chromium, a `navigator.sendBeacon` or `fetch(..., { keepalive: true })` sent on `pagehide`, `visibilitychange` or `unload` was never intercepted and reached the server in every mode, observe included; the engine now catches it at the browser level and applies the same rules, so under observe (and under read-only, for a destructive write) it is refused and reported with the other refused writes, and a mode that allows it still sends it. Such a write is recognised by its headers, since the page that sent it is gone: one out of the app whose Origin is another site's, or `null`, is treated as an embed's and refused outside destructive, and the exception for a captcha on the app's sign-in page is decided on the Referer, the page it was sent from. Browsers send only the origin as the Referer of a request to another site by default, so that exception seldom applies and such a captcha write is refused; a sign-in the tester completes while the page is open is not affected. In Chromium, a write a 307 or 308 redirect carries on to a new address is now judged there as well. In every browser, the pages the engine closes (at the end of a session, on a re-attach, and a popup of another site) are left for `about:blank` first, so what they send on the way out meets the policy too: in Firefox and WebKit such a write had also gone out unseen.
+
 ## 3.11.0
 
 ### Minor Changes

package/dist/browsers.js

@@ -182,14 +182,58 @@ export function beaconResourceType(engine) {
     return engine === "chromium" ? "ping" : "beacon";
 }
 /**
- * Whether a beacon a page sends as it is being left (sendBeacon, or a
- * keepalive fetch, on `pagehide`) escapes interception. In Chromium it is
- * never routed, so the write policy does not see it and it reaches the server
- * in every mode; Firefox and WebKit route it, and the policy refuses it like
- * any other. A limit of the network-layer policy (ADR 2), reported in ADR 12;
- * the check smoke suite asserts both directions per engine.
- */
-export function unloadBeaconsEscapePolicy(engine) {
+ * Where the write policy meets a write a page sends as it is being left (a
+ * sendBeacon or keepalive fetch on `pagehide`, `visibilitychange` or
+ * `unload`, or one still leaving as a page is closed).
+ *
+ * - `route`: Firefox and WebKit hand it to the context's route handler like
+ *   any other request.
+ * - `browser-fetch`: Chromium never routes it (it arrives with no network id,
+ *   or after the page's session has gone, and the driver sends it on), so on
+ *   its own it reached the server in every mode. The engine also pauses
+ *   requests at the browser level with the DevTools Fetch domain and judges
+ *   there, by the same rules, every write the route handler did not
+ *   (unload.ts). Only in modes where the policy can refuse: destructive
+ *   installs neither.
+ *
+ * Either way the write is judged and a refusal is reported like any other
+ * (ADR 2); the unload smoke suite asserts it per engine.
+ */
+export function unloadWriteInterception(engine) {
+    return engine === "chromium" ? "browser-fetch" : "route";
+}
+/**
+ * Whether a write a page sends as it is being left may be lost after the
+ * write policy lets it through. WebKit cancels such a request when the route
+ * handler answers after the page has gone, so a mode that allows the write
+ * does not promise it arrives; nothing is sent that the policy refused. Firefox
+ * sends it on, and so does Chromium's browser-level interception. In
+ * destructive mode nothing is intercepted and every engine sends it.
+ */
+export function allowedUnloadWritesMayBeLost(engine) {
+    return engine === "webkit";
+}
+/**
+ * Whether the writes another site's frame sends on `pagehide` may never be
+ * issued when the top page is left. Firefox can tear such a frame down before
+ * its requests reach the route handler, so there is nothing to refuse or to
+ * report, and nothing is sent. It depends on timing: seen in CI and under load
+ * locally, not on an idle machine. Chromium's browser-level interception and
+ * WebKit's route handler met them every time in the same runs. The writes the
+ * top page itself sends are not affected.
+ */
+export function frameUnloadWritesMayGoUnissued(engine) {
+    return engine === "firefox";
+}
+/**
+ * Whether a write carried on by a redirect (a 307 or 308 keeps the method and
+ * the body) is judged at its new address. The route handler sees only the
+ * first request of a redirect in every engine. In Chromium the browser-level
+ * interception (`unloadWriteInterception`) sees each hop and judges it like any
+ * write; Firefox and WebKit send the later hops on unjudged, a known limit of
+ * the policy there (ADR 2).
+ */
+export function writeRedirectHopsJudged(engine) {
     return engine === "chromium";
 }
 /**

package/dist/engine/browser.js

@@ -15,7 +15,7 @@ import { describeStep, FLOW_AFTER_LAST_STEP_MS, FLOW_STEP_TIMEOUT_MS, isAction,
 import { framePath, RECORD_MAX_FRAMES } from "./replay.js";
 import { keepWatchingUrl, normalizePace, SETTLE_TICK_MS, shouldKeepWaiting } from "./settle.js";
 import { buildRequestScript, formatReplay, replaySignature, requestHeaders, resolveMethod, resolveRequestUrl, toReplayResult } from "./request.js";
-import { defaultEngine, focusAdvanceKey, REMOVE_SHARED_WORKER_SCRIPT, screencastSupport, serviceWorkerPolicy, sharedWorkersAllowed, } from "../browsers.js";
+import { defaultEngine, focusAdvanceKey, REMOVE_SHARED_WORKER_SCRIPT, screencastSupport, serviceWorkerPolicy, sharedWorkersAllowed, unloadWriteInterception, } from "../browsers.js";
 import { revealedLines } from "./hover.js";
 import { FORMS_INVENTORY_SCRIPT, FORM_PROBE_BODY, formProbeExpression, formIdentity, formStatus, FORMS_READ_FAILED, FORMS_SUBMIT_UNMATCHED, isEmptySubmit, sameControl, isNavigationTeardown, submits, tracksForm, } from "./forms.js";
 import { explainLaunchFailure, isMissingBrowser } from "./launch.js";
@@ -26,6 +26,7 @@ import { answersWithRefusal, destructiveRefusal, isDestructive, isDestructiveWir
 import { scanProject } from "../scan.js";
 import { analyzeDesign, DESIGN_COLLECT_SCRIPT } from "./design.js";
 import { acceptMatches, generatedUpload } from "./fixtures.js";
+import { bodyDigest, headerOf, isReadMethod, judgeUnseenWrite, pausedRequestBytes, RoutedWrites, UnseenRefusals, unseenWriteSource, } from "./unload.js";
 const SETTLE_MS = 400;
 /**
  * Request URL → pathname, falling back to the raw string for anything
@@ -504,6 +505,19 @@ export class BrowserEngine {
     engineName = "chromium";
     /** Non-GET requests fired since the last action — surfaces silent state mutation in read-only runs (timestamped for attribution). */
     mutationRequests = [];
+    /**
+     * Chromium only (browsers.ts `unloadWriteInterception`): the writes the
+     * route handler already let through, so the browser-level interception that
+     * catches what it never sees does not judge them twice. That session lives
+     * as long as the browser, so a page closed at the end still meets it.
+     */
+    routedWrites = new RoutedWrites();
+    /** Writes refused at the browser level, so the driver's own report of them (a redirect hop failing) is known as the policy's doing. */
+    unseenRefusals = new UnseenRefusals();
+    /** Whether the write policy stopped this request, at the route handler or at the browser level. */
+    refusedByAnyPolicy(req) {
+        return this.refusedByPolicy.has(req) || this.unseenRefusals.has(req.method(), req.url());
+    }
     /**
      * Every request answered since the last action, with its status. The HTTP
      * oracle sees each 4xx as it happens and reports it on its own; this ledger
@@ -580,7 +594,7 @@ export class BrowserEngine {
             url: req.url(),
             status,
             resourceType: req.resourceType(),
-            blockedByPolicy: this.refusedByPolicy.has(req),
+            blockedByPolicy: this.refusedByAnyPolicy(req),
         });
     }
     /**
@@ -700,7 +714,7 @@ export class BrowserEngine {
             this.projectDirNote = ` (its real path could not be resolved: ${err instanceof Error ? err.message : String(err)} — a symlinked project path may be wrongly refused)`;
         }
         this.oracles = new OracleMonitor();
-        this.oracles.setPolicyRefusalCheck((req) => this.refusedByPolicy.has(req));
+        this.oracles.setPolicyRefusalCheck((req) => this.refusedByAnyPolicy(req));
         // A request an embed sends to the app is the app's to answer: only one headed outside the app is the embed's.
         this.oracles.setEmbedAttribution((req) => {
             let site = null;
@@ -934,8 +948,10 @@ export class BrowserEngine {
                     return refuse(`sent from a page of ${offApp}, outside the app`);
                 // Auth/session flows must work in every mode — but never a destructive
                 // one, and in observe only the requests a login itself needs.
-                if (isAuthExempt(this.mode, method, pathname, destructiveWire))
+                if (isAuthExempt(this.mode, method, pathname, destructiveWire)) {
+                    this.routedWrites.note(this.mode, method, url, bodyDigest(req.postDataBuffer()));
                     return route.continue();
+                }
                 let owned = this.isOwnedResource(pathname);
                 // A single UI action commonly fires create-then-immediately-save
                 // (POST gets an id, PUT saves content under it) faster than the
@@ -969,10 +985,22 @@ export class BrowserEngine {
                             .finally(() => this.pendingCreations.delete(task));
                         this.pendingCreations.add(task);
                     }
+                    this.routedWrites.note(this.mode, method, url, bodyDigest(req.postDataBuffer()));
                     return route.continue();
                 }
                 return refuse();
             });
+            // Chromium never routes a write a page sends as it is being left; it is judged at the browser level instead.
+            if (unloadWriteInterception(this.engineName) === "browser-fetch" && this.browser) {
+                try {
+                    await this.interceptUnseenWrites(this.browser);
+                }
+                catch (err) {
+                    // Fail closed: without it, a page being left would write past the policy.
+                    await this.close();
+                    throw new Error(`Could not start the write policy's browser-level interception: ${err instanceof Error ? err.message.split("\n")[0] : String(err)}`);
+                }
+            }
         }
         // Popups / target=_blank: adopt same-origin pages as the active page (with
         // oracles attached); close foreign-origin popups so exploration cannot
@@ -999,7 +1027,7 @@ export class BrowserEngine {
                     this.lastSnap = null;
                 }
                 else {
-                    void newPage.close().catch(() => { });
+                    void BrowserEngine.leaveAndClose(newPage);
                 }
             })
                 .catch(() => { });
@@ -1818,6 +1846,121 @@ export class BrowserEngine {
             return null;
         return foreign;
     }
+    /**
+     * Pause, at the browser level, every request the context's route handler
+     * lets through or never sees (unload.ts says why), and judge the writes it
+     * never saw. Everything else is sent on at once.
+     */
+    async interceptUnseenWrites(browser) {
+        this.routedWrites.clear();
+        const session = await browser.newBrowserCDPSession();
+        session.on("Fetch.requestPaused", (event) => void this.judgeUnseenRequest(session, event));
+        await session.send("Fetch.enable", { patterns: [{ urlPattern: "*" }] });
+    }
+    async judgeUnseenRequest(session, event) {
+        const { url, method, headers } = event.request;
+        // Every paused request gets exactly one answer, whatever throws on the way: a request left paused hangs its page.
+        let answered = false;
+        const answer = async (allow) => {
+            answered = true;
+            await (allow
+                ? session.send("Fetch.continueRequest", { requestId: event.requestId })
+                : session.send("Fetch.failRequest", { requestId: event.requestId, errorReason: "BlockedByClient" })).catch(() => {
+                /* the request or the browser is already gone */
+            });
+        };
+        try {
+            // Reads, and everything in destructive, go on. A later hop of a redirect is judged like any write: the route handler never sees one.
+            if (isReadMethod(method) || this.mode === "destructive")
+                return await answer(true);
+            const bytes = pausedRequestBytes(event.request);
+            if (this.routedWrites.claim(this.mode, method, url, bodyDigest(bytes)))
+                return await answer(true);
+            let verdict;
+            let pathname = url;
+            try {
+                pathname = pathnameOf(url);
+                const { foreign, offApp } = unseenWriteSource({
+                    appUrl: this.baseUrl,
+                    mode: this.mode,
+                    url,
+                    originHeader: headerOf(headers, "origin"),
+                    referer: headerOf(headers, "referer"),
+                    currentPageUrl: this.page?.url(),
+                    trustedEmbeds: this.trustedEmbeds,
+                    movedByEmbed: this.embedMoves.movedTo,
+                });
+                verdict = judgeUnseenWrite({
+                    mode: this.mode,
+                    method,
+                    pathname,
+                    destructiveWire: isDestructiveWire(pathname, bytes?.toString("utf8")),
+                    foreign,
+                    offApp,
+                    owned: this.isOwnedResource(pathname),
+                });
+            }
+            catch (err) {
+                // Fail closed: a write that cannot be judged is refused, and reported as refused below.
+                console.error(`[scenescout] write policy: could not judge ${method} at the browser level, refused: ${err instanceof Error ? err.message : String(err)}`);
+                verdict = { allow: false };
+            }
+            // Known as refused before the browser fails it, so the driver's report of the failure finds it.
+            if (!verdict.allow)
+                this.unseenRefusals.note(method, url);
+            await answer(verdict.allow);
+            if (verdict.allow) {
+                // Reported like any other write that went out, as the request event does for a routed one.
+                if (!BENIGN_MUTATION_RE.test(url) && this.mutationRequests.length < 20)
+                    this.mutationRequests.push({ at: Date.now(), sig: `${method} ${url.slice(0, 120)}` });
+                return;
+            }
+            // Refused and reported as any refusal is. Dropped rather than answered: the page that sent it is going or gone.
+            if (this.blockedRequests.length < 20)
+                this.blockedRequests.push({
+                    at: Date.now(),
+                    sig: `${method} ${url.slice(0, 140)}`,
+                    answered: false,
+                    why: verdict.why,
+                    type: (event.resourceType ?? "other").toLowerCase(),
+                });
+            this.logAction({ action: "write-policy:blocked", target: `${method} ${pathname}${verdict.why ? ` (${verdict.why})` : ""}`, url: this.page?.url() ?? "" });
+            this.oracles.notePolicyBlock();
+        }
+        catch (err) {
+            console.error(`[scenescout] write policy: browser-level interception failed on ${method}: ${err instanceof Error ? err.message : String(err)}`);
+        }
+        finally {
+            if (!answered) {
+                // Fail closed, and still say so: a refusal nobody hears about reads as a write that went out.
+                try {
+                    this.unseenRefusals.note(method, url);
+                }
+                catch {
+                    /* the error above is already logged */
+                }
+                await answer(false);
+                try {
+                    if (this.blockedRequests.length < 20)
+                        this.blockedRequests.push({
+                            at: Date.now(),
+                            sig: `${method} ${url.slice(0, 140)}`,
+                            answered: false,
+                            type: (event.resourceType ?? "other").toLowerCase(),
+                        });
+                    this.logAction({
+                        action: "write-policy:blocked",
+                        target: `${method} ${url.slice(0, 140)} (not judged: an error in the policy)`,
+                        url: this.page?.url() ?? "",
+                    });
+                    this.oracles.notePolicyBlock();
+                }
+                catch (err) {
+                    console.error(`[scenescout] write policy: could not report a refusal: ${err instanceof Error ? err.message : String(err)}`);
+                }
+            }
+        }
+    }
     /** Report (and clear) write-policy blocks since the last action. */
     drainBlocked() {
         this.lastActionBlocked = this.blockedRequests.length;
@@ -1868,7 +2011,7 @@ export class BrowserEngine {
         // to see the DUPLICATES that the reporting dedup below intentionally hides.
         this.lastActionMutationSigs = this.mutationRequests.map((e) => e.sig);
         const fresh = this.mutationRequests
-            .filter((entry) => !this.refusedByPolicy.has(entry.req))
+            .filter((entry) => !entry.req || !this.refusedByAnyPolicy(entry.req))
             .filter((entry) => {
             const key = entry.sig.split("?")[0];
             if (this.reportedMutationSigs.has(key))
@@ -3614,6 +3757,23 @@ export class BrowserEngine {
             }
         }
     }
+    /**
+     * Leave a page for about:blank. A write a page sends as it is closed (a
+     * sendBeacon or keepalive fetch on pagehide) is not routed in Firefox or
+     * WebKit, so it reached the server in every mode; one it sends as it
+     * navigates is routed and judged like any other. (Chromium routes neither;
+     * its browser-level interception catches both.)
+     */
+    static async leave(page) {
+        if (page.isClosed() || page.url() === "about:blank")
+            return;
+        await page.goto("about:blank", { timeout: 3000 }).catch(() => { });
+    }
+    /** Close a page the engine will not drive, after leaving it (`leave`), so what it sends on its way out meets the policy. */
+    static async leaveAndClose(page) {
+        await BrowserEngine.leave(page);
+        await page.close().catch(() => { });
+    }
     async close() {
         // Marks the end of the time this session held a browser, so the pace
         // section can say how long it was held with nothing happening. Only when a
@@ -3621,6 +3781,10 @@ export class BrowserEngine {
         // is not an event.
         if (this.page)
             this.logAction({ action: "close", url: this.page.isClosed() ? "" : this.page.url() });
+        // Every page is left before it is closed, while the write policy still holds, so what a page sends on its way out
+        // is judged and its refusal logged before the flush below.
+        const pages = this.context?.pages() ?? [];
+        await BrowserEngine.settleWithin(Promise.allSettled(pages.map((p) => BrowserEngine.leave(p))), 5000);
         // Pending debounced coverage writes must land before the process can exit.
         try {
             this.memory?.flush();
@@ -3636,13 +3800,16 @@ export class BrowserEngine {
         // If teardown overruns the cap, the leftover process is reaped by the
         // orphan cleaner on the next attach (or server start).
         await BrowserEngine.settleWithin((async () => {
-            await this.page?.close().catch(() => { });
+            for (const p of this.context?.pages() ?? [])
+                await p.close().catch(() => { });
             await this.context?.close().catch(() => { });
             await this.browser?.close().catch(() => { });
         })(), 8000);
         this.page = null;
         this.context = null;
         this.browser = null;
+        this.routedWrites.clear();
+        this.unseenRefusals.clear();
         this.refs.clear();
         this.snapshotUrl = "";
         this.currentFingerprint = "";

package/dist/engine/calibration.js

@@ -20,6 +20,7 @@
  *
  * Pure, so every rule here is table-tested.
  */
+import { requestsPair, statedRequests, templatedPathsMatch } from "./lane.js";
 import { failingSignatures } from "./memory.js";
 /**
  * Upper edge of each confidence bucket. Five is enough to see a shape and few
@@ -226,45 +227,129 @@ export const MAX_UNFILED_NAMED = 10;
  * nobody — and it is cheapest to catch at the moment the planner folds the
  * report, while the lane's session is still open.
  *
- * Matched on the store's failing-endpoint signature where the evidence has
- * one, and on the evidence text otherwise. The text match is what the
- * calibration join refuses, because there a miss is scored against the lane;
- * here a miss only asks someone to check, which costs a look and nothing more.
+ * The check exists to catch unfiled defects, so a false "filed" costs more
+ * than a false alarm, which only asks someone to look. A decision counts as
+ * filed by one of three rules, tried in order:
+ *
+ * 1. Failing signatures. When the decision's evidence names one or more
+ *    failing requests (the store's `failingSignatures`), it is filed only when
+ *    EVERY one of them is covered by a finding's signature: same method, same
+ *    status, same route, where a path-template segment (`{id}`, `:id`, `*`) on
+ *    either side stands for one id segment (see `templatedPathsMatch`). So a
+ *    finding filed as `GET /api/things/{id} 500` covers `GET /api/things/7 500`,
+ *    and a decision naming two failing requests is not filed by a finding for
+ *    one of them.
+ * 2. A request that did not fail. When the decision names a request with a
+ *    2xx or 3xx status, or none, and a finding names the same request
+ *    (template-aware, same status), the decision is filed only when its
+ *    evidence, with its path written as the finding's, is the finding's
+ *    evidence exactly, restates part of it, or is the finding's evidence
+ *    followed only by that request's other status (`… 200 as role=viewer
+ *    (expected 403)` against `… 200 as role=viewer`). Word overlap is not enough
+ *    there: a 200 names a resource, not a defect, and two different defects
+ *    on one endpoint read alike. A failing signature in the decision does not
+ *    block this when it names the same request (`POST /api/things/7/archive
+ *    200 (expected 403)` reads as a 403 to the store); one naming any other
+ *    request does, and sends the decision to rule 1's verdict.
+ * 3. Text. Evidence naming no failing request is matched on its text: the
+ *    same evidence, a restatement of part of it, or shared identifiers and
+ *    near-identical wording. The calibration join refuses a text match,
+ *    because there a miss is scored against the lane.
  */
 export function unfiledDefects(decisions, findings) {
-    const keys = new Set();
+    const keys = [];
     const filed = [];
     for (const f of findings) {
         const text = `${f.evidence ?? ""} ${f.title}`;
+        const requests = statedRequests(f.evidence ?? "");
+        // The store's signatures, and the finding's own failing requests as
+        // written: the store reads no `*` in a path, so `GET /api/things/* 500`
+        // has no signature of its own.
         if (f.evidence)
             for (const key of joinKeys(f.evidence))
-                keys.add(key);
-        filed.push({ ids: identifiers(text), words: words(text), evidenceWords: words(f.evidence ?? ""), evidence: squash(f.evidence ?? "") });
+                keys.push(...statedRequests(key));
+        keys.push(...requests.filter((q) => q.status !== null && /^[45]/.test(q.status)));
+        filed.push({
+            ids: identifiers(text),
+            words: words(text),
+            evidenceWords: words(f.evidence ?? ""),
+            evidence: squash(f.evidence ?? ""),
+            raw: f.evidence ?? "",
+            requests,
+        });
     }
     const out = [];
     for (const d of decisions) {
         if (d.verdict !== "defect")
             continue;
         if (d.evidence) {
-            const joined = [...joinKeys(d.evidence)];
-            if (joined.some((k) => keys.has(k)))
+            const evidence = d.evidence;
+            const joined = [...joinKeys(evidence)];
+            const failing = joined.flatMap((sig) => statedRequests(sig));
+            if (failing.length > 0 && failing.every((sig) => keys.some((k) => requestsPair(sig, k))))
+                continue;
+            if (filed.some((f) => restatesRequest(evidence, failing, f)))
                 continue;
             // A failing-endpoint signature is the store's own identity for a bug. When
-            // the decision has one and no finding shares it, nothing else is a match.
+            // the decision has one and no finding covers it, nothing else is a match.
             if (joined.length > 0) {
                 out.push(`${d.observation} — ${d.evidence}`);
                 continue;
             }
-            const ids = identifiers(d.evidence);
-            const w = words(d.evidence);
-            const text = squash(d.evidence);
-            if (filed.some((f) => (text !== "" && f.evidence === text) || restates(text, f) || covers(ids, w, f)))
+            if (filed.some((f) => matchesText(evidence, f)))
                 continue;
         }
         out.push(d.evidence ? `${d.observation} — ${d.evidence}` : d.observation);
     }
     return out;
 }
+/** Rule 3: the same evidence, a restatement of part of it, or shared identifiers and wording. */
+function matchesText(reported, f) {
+    const text = squash(reported);
+    return (text !== "" && f.evidence === text) || restates(text, f) || covers(identifiers(reported), words(reported), f);
+}
+/**
+ * Rule 2 of unfiledDefects: the decision names a request that did not fail,
+ * the finding names the same request, and the decision's evidence — its path
+ * written as the finding's, so a template and the id it stands for are not a
+ * difference — is the finding's evidence exactly, restates part of it, or
+ * adds only the status the request should have returned.
+ * `failing` is the decision's failing signatures; each must name this same
+ * request, or the pair is refused.
+ */
+function restatesRequest(evidence, failing, f) {
+    for (const r of statedRequests(evidence)) {
+        if (r.status !== null && /^[45]/.test(r.status))
+            continue;
+        const paired = f.requests.find((q) => requestsPair(r, q));
+        if (!paired)
+            continue;
+        if (!failing.every((sig) => sig.method === r.method && templatedPathsMatch(sig.segments, r.segments)))
+            continue;
+        const aligned = squash(evidence.slice(0, r.start) + f.raw.slice(paired.start, paired.end) + evidence.slice(r.end));
+        if (aligned !== "" && (f.evidence === aligned || restates(aligned, f) || addsOnlyExpectedStatus(aligned, f.evidence, r.status)))
+            return true;
+    }
+    return false;
+}
+/**
+ * The status a lane adds after the call it quotes, to say what the call
+ * should have returned: "(expected 403)", "(reject 403)", "reject 403". One
+ * status, an optional verb, optional brackets, and nothing else.
+ */
+const EXPECTED_STATUS_RE = /^[\s,;]*\(?\s*(?:(?:expected|expect|rejected|reject)\s*:?\s*)?([1-5]\d{2})\s*\)?[\s.,;]*$/;
+/**
+ * Whether the reported evidence is the filed evidence followed by nothing but
+ * the status the request should have returned, which differs from the one it
+ * did. Anything else after it — another request, another role, another
+ * claim — is something the finding does not say, so it is not covered.
+ */
+function addsOnlyExpectedStatus(reported, filed, status) {
+    if (filed === "" || !reported.startsWith(filed))
+        return false;
+    const expected = EXPECTED_STATUS_RE.exec(reported.slice(filed.length))?.[1];
+    return expected !== undefined && expected !== status;
+}
 /**
  * Shortest reported evidence that counts as restating a filing it is part of.
  */

package/dist/engine/fingerprint.js

@@ -4,6 +4,17 @@ const HEX_RE = /^[0-9a-f]{12,}$/i;
 const NUM_RE = /^\d+$/;
 /** Query params that name a UI state (a different screen), not transient data. */
 const UI_STATE_PARAM_RE = /^(tab|view|mode|step|panel|section)$/i;
+/**
+ * Whether a path segment is an id that route identity collapses to `:id`: a
+ * UUID or long hex string anywhere, a number from the second real segment on.
+ * `index` counts as `path.split("/")` does, so the first real segment of a
+ * leading-slash path is 1 (see the note on numeric first segments below).
+ */
+export function isIdSegment(seg, index) {
+    if (UUID_RE.test(seg) || HEX_RE.test(seg))
+        return true;
+    return NUM_RE.test(seg) && index > 1;
+}
 /**
  * Normalize a URL into a route-class identity:
  * - path ids collapse (/orders/123 → /orders/:id)
@@ -38,13 +49,7 @@ export function normalizePath(rawUrl) {
     }
     // segments[0] is always "" for a leading-slash path, so the first REAL
     // segment is index 1 — that is the one a bare numeric page occupies.
-    const segments = path.split("/").map((seg, i) => {
-        if (UUID_RE.test(seg) || HEX_RE.test(seg))
-            return ":id";
-        if (NUM_RE.test(seg) && i > 1)
-            return ":id";
-        return seg;
-    });
+    const segments = path.split("/").map((seg, i) => (isIdSegment(seg, i) ? ":id" : seg));
     let normalized = segments.join("/") || "/";
     if (normalized === "")
         normalized = "/";

package/dist/engine/lane.js

@@ -23,6 +23,7 @@
  * Pure logic, no browser, so every rule here is table-tested.
  */
 import { z } from "zod";
+import { isIdSegment } from "./fingerprint.js";
 import { FINDING_CATEGORIES } from "./memory.js";
 /** Upper bound on the decisions and routes arrays, so a report is always small enough to hold whole. */
 export const LANE_MAX_ITEMS = 255;
@@ -271,6 +272,86 @@ export function summarizeLaneReport(r) {
         parts.push(`blocked by ${r.blocked_by}`);
     return `${r.lane}: ${parts.join(", ")}`;
 }
+/**
+ * Method, optional origin, path (template characters included), optional query
+ * and fragment, then a status only when it comes next: after an arrow or a
+ * colon, or after spaces, and nothing else. A status further along
+ * ("... (expected 403)") is a different claim about the request and is not
+ * read as its status.
+ *
+ * The separator is two alternatives rather than `\s*(?:->|:)?\s*`: with the
+ * arrow optional, two adjacent `\s*` can split a run of spaces every way
+ * there is, and evidence ending in a long run of whitespace took quadratic
+ * time to reject.
+ */
+const STATED_REQUEST_RE = /\b(GET|POST|PUT|PATCH|DELETE)\s+(?:https?:\/\/[^/\s]+)?(\/[A-Za-z0-9/_.:{}$*~%-]*)(?:\?[^\s#]*)?(?:#\S*)?(?:(?:\s*(?:->|→|=>|:)\s*|\s+)([1-5]\d{2})\b)?/gi;
+/** The requests a piece of evidence names, in order. */
+export function statedRequests(text) {
+    const out = [];
+    for (const m of text.matchAll(STATED_REQUEST_RE)) {
+        const raw = m[2].replace(/[.:,]+$/, "");
+        const start = (m.index ?? 0) + m[0].indexOf(m[2]);
+        out.push({ method: m[1].toUpperCase(), segments: pathSegments(raw), status: m[3] ?? null, start, end: start + raw.length });
+    }
+    return out;
+}
+function pathSegments(path) {
+    const trimmed = path.replace(/\/+$/, "");
+    if (trimmed === "")
+        return [];
+    return trimmed
+        .slice(1)
+        .split("/")
+        .map((seg) => {
+        try {
+            return decodeURIComponent(seg).toLowerCase();
+        }
+        catch {
+            return seg.toLowerCase();
+        }
+    });
+}
+/**
+ * A path-template segment: `{name}`, `:name` or `*`. Whole segments only —
+ * `order-{id}` is a literal, and so is `**`, which some routers read as "any
+ * number of segments": a template here stands for exactly one.
+ */
+export function isTemplateSegment(seg) {
+    return seg === "*" || /^\{[^/{}]+\}$/.test(seg) || /^:[a-z_][a-z0-9_-]*$/i.test(seg);
+}
+/**
+ * Whether two paths name the same route. A template segment on either side
+ * stands for ONE segment that is an id by route identity's own rule
+ * (`isIdSegment`: a number after the first segment, a UUID, a long hex
+ * string), or for another template. Never for a word: `/users/{id}` is not
+ * `/users/me`, `/things/:id` is not `/things/export`, and a path of three stars does
+ * not stand for every three-segment path. Same number of segments, every literal
+ * equal (case-insensitive), so a template never absorbs a slash or an empty
+ * segment.
+ */
+export function templatedPathsMatch(a, b) {
+    if (a.length !== b.length)
+        return false;
+    // +1: segments start at the first real one; isIdSegment counts as split("/") does.
+    const standsFor = (seg, i) => isTemplateSegment(seg) || isIdSegment(seg, i + 1);
+    return a.every((x, i) => {
+        const y = b[i];
+        if (isTemplateSegment(x))
+            return standsFor(y, i);
+        if (isTemplateSegment(y))
+            return standsFor(x, i);
+        return x === y;
+    });
+}
+/**
+ * Whether two stated requests are the same call: same method, same route with
+ * a template read as an id (see templatedPathsMatch), and the same status — both stated and equal, or
+ * both unstated. A different status is a different outcome of the call, and
+ * pairing a 200 with a 403 would file one claim under its opposite.
+ */
+export function requestsPair(a, b) {
+    return a.method === b.method && a.status === b.status && templatedPathsMatch(a.segments, b.segments);
+}
 /**
  * Which sessions are lanes of a parallel run, and whether each one's report
  * has been folded, so a lane's session is not closed before its decisions

package/dist/engine/memory.js

@@ -451,6 +451,14 @@ export function requestsDisagree(a, b) {
  * refused save is data-loss to one and data-inconsistency to another. Across
  * families — a layout defect and a data defect — a shared quoted string names
  * a place on the page, not a bug.
+ *
+ * The presentation kinds are each a family of their own. They are not
+ * alternative labels for one observation: one element is routinely wrong in
+ * several independent ways at once — clipped out of view (visual), styled like
+ * body text (ux-polish), unnamed to a screen reader (a11y), missing its test id
+ * — and the findings about it quote the same control name and name the same
+ * test id. As one family, a link "styled like body text" was merged into the
+ * finding that the same link was clipped, and the second finding was lost.
  */
 const CATEGORY_FAMILY = {
     "data-inconsistency": "data",
@@ -460,10 +468,10 @@ const CATEGORY_FAMILY = {
     network: "failure",
     "console-error": "failure",
     "page-error": "failure",
-    visual: "presentation",
-    "ux-polish": "presentation",
-    a11y: "presentation",
-    "missing-testid": "presentation",
+    visual: "visual",
+    "ux-polish": "ux-polish",
+    a11y: "a11y",
+    "missing-testid": "missing-testid",
     "ux-confusing": "flow",
     "dead-end": "flow",
     security: "security",
@@ -485,7 +493,30 @@ export function sameFamily(a, b) {
     const fa = CATEGORY_FAMILY[a];
     return fa !== undefined && fa === CATEGORY_FAMILY[b];
 }
+/**
+ * Whether two findings may be one defect by KIND — what is wrong, as opposed
+ * to where. The same category, or two categories of one family. Every
+ * route-scoped rule below asks this first: evidence, a quoted literal and a
+ * title all say where a defect is as often as what it is, so none of them may
+ * merge a finding of one kind into a finding of another.
+ */
+function sameKind(a, b) {
+    return (!!a && a === b) || sameFamily(a, b);
+}
+/**
+ * The categories a finding of `category` can merge with, in list order —
+ * so scout_finding can tell a lane whose filing was merged which kinds would
+ * have been kept apart. An unknown category merges only with itself.
+ */
+export function mergeableCategories(category) {
+    const known = FINDING_CATEGORIES.filter((c) => sameKind(category, c));
+    return known.length > 0 ? known : [category];
+}
 function sameFinding(a, b) {
+    // What is wrong before where: two kinds of defect are two findings, however
+    // alike their evidence, quoted text or titles.
+    if (!sameKind(a.category, b.category))
+        return false;
     const aEv = a.evidence?.toLowerCase().replace(/\s+/g, " ").trim();
     const bEv = b.evidence?.toLowerCase().replace(/\s+/g, " ").trim();
     if (aEv && bEv && aEv === bEv)
@@ -506,7 +537,9 @@ function sameFinding(a, b) {
     // literal with the data defect describing what the button does — in its
     // detail or its own title. Merged, the layout defect was filed and then lost
     // from the report on most runs of a benchmark. Within a family, one bug
-    // filed twice under neighbouring categories still merges.
+    // filed twice under neighbouring categories still merges. Stricter than the
+    // kind check above in one way: an unknown category read from an older file
+    // is never merged on a quoted string, even with itself.
     //
     // And never when both findings' evidence names requests with no endpoint in
     // common. A quoted control name bridges two findings about that control

package/dist/engine/unload.js

@@ -0,0 +1,201 @@
+/**
+ * Writes the context's route handler never sees, and how the engine judges
+ * them anyway.
+ *
+ * In Chromium, a request a page sends while it is being left (a
+ * `navigator.sendBeacon` or a `fetch(..., { keepalive: true })` on
+ * `pagehide`, `visibilitychange` or `unload`, or one still leaving as a page
+ * is closed) is never handed to the context's route handler: it arrives with
+ * no network id, or after the page's own DevTools session has gone, and the
+ * driver lets it through unseen. Left alone it reached the server in every
+ * mode, observe included. The engine therefore also intercepts at the
+ * browser level (the DevTools Fetch domain on the browser target, which does
+ * pause these requests) and judges there every write the route handler did
+ * not already judge, by the same rules (ADR 2). Which engines need this is
+ * `unloadWriteInterception` in `browsers.ts`.
+ *
+ * Everything here is pure, so the rules are table-tested in policy-test; the
+ * wiring to the browser lives in `browser.ts`.
+ */
+import { createHash } from "node:crypto";
+import { allowsForeignWriteOnSignIn, allowsWrite, foreignWrite, isAuthExempt, offAppPageWrite, trustsForeignWrite } from "./policy.js";
+/** Whether a method sends nothing the write policy judges. */
+export function isReadMethod(method) {
+    return method === "GET" || method === "HEAD" || method === "OPTIONS";
+}
+/**
+ * Who sent a write the route handler never saw, as far as its headers tell:
+ * the embed it is charged to (`foreign`) and the off-app page it came from
+ * (`offApp`), the two inputs `judgeUnseenWrite` takes from the sender.
+ *
+ * The sending frame may be gone, so its URL is not known. The page it was
+ * sent from is its Referer; where there is none, the page the session drives
+ * stands in for `offApp` only.
+ *
+ * - `foreign`: policy.ts `foreignWrite` with no frame chain, so a write out of
+ *   the app whose Origin header names another site is that site's embed's. A
+ *   sign-in provider's page reached as the whole page, beaconing to itself as
+ *   it is left, is therefore refused outside destructive; the route handler,
+ *   which knows the frame, would let it through. The cautious direction. An
+ *   `Origin: null` write out of the app is taken to be an embed's too, whether
+ *   or not the page embeds anything, since the page can no longer be asked.
+ * - The sign-in exception (a captcha on the app's own sign-in page) is judged
+ *   against the Referer, the page the request was sent from, never against
+ *   whatever page the session has moved to by the time the request is seen.
+ *   With no Referer it does not apply.
+ * - Trusted embeds (safe-write only) are judged by the Origin header alone.
+ */
+export function unseenWriteSource(input) {
+    const { appUrl, mode, url, originHeader, referer } = input;
+    let foreign = foreignWrite(appUrl, { url, frameChain: [], frameUrl: null, originHeader, pageHasForeignFrame: true });
+    if (foreign && referer && allowsForeignWriteOnSignIn(mode, referer, appUrl))
+        foreign = null;
+    if (foreign && trustsForeignWrite(mode, input.trustedEmbeds, appUrl, { frameChain: [], originHeader, unadoptedPageUrl: null }))
+        foreign = null;
+    const offApp = offAppPageWrite(appUrl, referer ?? input.currentPageUrl, url, input.movedByEmbed);
+    return { foreign, offApp };
+}
+/**
+ * The route handler's verdict on a write, from what a browser-level
+ * interception can know about it. Same rules in the same order: a write
+ * another site's frame sends outside the app is refused; so is one sent from
+ * a page an embed moved off the app, a sign-in excepted; a sign-in passes;
+ * then the mode decides. `foreign` and `offApp` come from
+ * `unseenWriteSource`, since the sending frame may no longer exist.
+ * The one step it does not take is safe-write's short wait for a creation
+ * still in flight: a page being left has nothing more to wait for.
+ */
+export function judgeUnseenWrite(input) {
+    const { mode, method, pathname, destructiveWire } = input;
+    if (mode === "destructive" || isReadMethod(method))
+        return { allow: true };
+    if (input.foreign)
+        return { allow: false, why: `sent from a frame of ${input.foreign}` };
+    const authExempt = isAuthExempt(mode, method, pathname, destructiveWire);
+    if (input.offApp && !authExempt)
+        return { allow: false, why: `sent from a page of ${input.offApp}, outside the app` };
+    if (authExempt)
+        return { allow: true };
+    return allowsWrite(mode, method, destructiveWire, input.owned) ? { allow: true } : { allow: false };
+}
+/**
+ * How long a write the route handler let through stays on the ledger. The
+ * browser-level pause follows the route handler's `continue` within
+ * milliseconds; an entry that outlives this is a request that never reached
+ * the network (answered by a service worker, cancelled by the page).
+ */
+export const ROUTED_WRITE_TTL_MS = 5000;
+/** Entries past this many are dropped oldest first: a page that posts in a loop cannot grow the ledger without bound. */
+const MAX_ROUTED_WRITES = 200;
+/** A request body's identity for the ledger: the SHA-256 of its bytes, or "" for no body. */
+export function bodyDigest(body) {
+    return body && body.length > 0 ? createHash("sha256").update(body).digest("hex") : "";
+}
+/**
+ * The writes the route handler has already judged and let through, so the
+ * browser-level interception, which sees them again, lets them pass without
+ * judging them twice. The key is the mode, the method, the URL and the body's
+ * digest (`bodyDigest`), so a write let through can only excuse the very same
+ * request: never one with another body to the same URL, such as a delete
+ * command sent where a harmless save was allowed a moment before, and never
+ * one sent after a flow switched to a stricter rule. A write is claimed once:
+ * two identical requests, one routed and one not, leave the second to be
+ * judged. Requests the route handler refused never reach the browser level,
+ * so they are not recorded.
+ */
+export class RoutedWrites {
+    ttlMs;
+    entries = [];
+    constructor(ttlMs = ROUTED_WRITE_TTL_MS) {
+        this.ttlMs = ttlMs;
+    }
+    static key(mode, method, url, digest) {
+        return `${mode} ${method} ${url.split("#")[0]} ${digest}`;
+    }
+    prune(now) {
+        this.entries = this.entries.filter((e) => now - e.at <= this.ttlMs);
+        if (this.entries.length > MAX_ROUTED_WRITES)
+            this.entries = this.entries.slice(-MAX_ROUTED_WRITES);
+    }
+    note(mode, method, url, digest, now = Date.now()) {
+        this.prune(now);
+        this.entries.push({ key: RoutedWrites.key(mode, method, url, digest), at: now });
+    }
+    /** Whether the route handler let this write through a moment ago; each note is claimed at most once. */
+    claim(mode, method, url, digest, now = Date.now()) {
+        this.prune(now);
+        const key = RoutedWrites.key(mode, method, url, digest);
+        const i = this.entries.findIndex((e) => e.key === key);
+        if (i < 0)
+            return false;
+        this.entries.splice(i, 1);
+        return true;
+    }
+    clear() {
+        this.entries = [];
+    }
+}
+/** How long a write refused at the browser level stays recognisable to the driver's request events. */
+export const UNSEEN_REFUSAL_TTL_MS = 10000;
+/**
+ * Writes refused at the browser level, which the driver may still report as
+ * requests of its own: a 307 or 308 hop is a request Playwright knows, and it
+ * then sees it fail (ERR_BLOCKED_BY_CLIENT). Without this the refusal was also
+ * filed as the app's failed request and listed as a possible mutation. Keyed
+ * by method and URL (fragment dropped), since the driver's request object is
+ * not the paused one; kept briefly, and not consumed, since several listeners
+ * ask about the same request.
+ */
+export class UnseenRefusals {
+    ttlMs;
+    entries = [];
+    constructor(ttlMs = UNSEEN_REFUSAL_TTL_MS) {
+        this.ttlMs = ttlMs;
+    }
+    static key(method, url) {
+        return `${method} ${url.split("#")[0]}`;
+    }
+    prune(now) {
+        this.entries = this.entries.filter((e) => now - e.at <= this.ttlMs);
+        if (this.entries.length > MAX_ROUTED_WRITES)
+            this.entries = this.entries.slice(-MAX_ROUTED_WRITES);
+    }
+    note(method, url, now = Date.now()) {
+        this.prune(now);
+        this.entries.push({ key: UnseenRefusals.key(method, url), at: now });
+    }
+    has(method, url, now = Date.now()) {
+        this.prune(now);
+        const key = UnseenRefusals.key(method, url);
+        return this.entries.some((e) => e.key === key);
+    }
+    clear() {
+        this.entries = [];
+    }
+}
+/**
+ * The body of a request paused by the DevTools Fetch domain, as bytes: the
+ * base64 entries decoded and joined, else the plain `postData`. Undefined when
+ * the protocol sent neither (a body too large to include). Its digest then
+ * matches no routed write, so the request is judged, on its URL alone.
+ */
+export function pausedRequestBytes(request) {
+    const entries = request.postDataEntries?.filter((e) => typeof e.bytes === "string");
+    if (entries && entries.length > 0)
+        return Buffer.concat(entries.map((e) => Buffer.from(e.bytes, "base64")));
+    return request.postData === undefined ? undefined : Buffer.from(request.postData, "utf8");
+}
+/** The same body as text, for the policy's body rules. */
+export function pausedRequestBody(request) {
+    return pausedRequestBytes(request)?.toString("utf8");
+}
+/** A header's value from a DevTools header map, whose names keep the case they were sent with. */
+export function headerOf(headers, name) {
+    if (!headers)
+        return undefined;
+    const wanted = name.toLowerCase();
+    for (const [k, v] of Object.entries(headers))
+        if (k.toLowerCase() === wanted)
+            return v;
+    return undefined;
+}

package/dist/mcp-server.js

@@ -34,7 +34,7 @@ import { ErrorCode, GetPromptRequestSchema, ListPromptsRequestSchema, McpError }
 import { z } from "zod";
 import { BrowserEngine } from "./engine/browser.js";
 import { reapOrphanBrowsers } from "./engine/reaper.js";
-import { FINDING_CATEGORIES, MemoryStore, redactSecrets } from "./engine/memory.js";
+import { FINDING_CATEGORIES, MemoryStore, mergeableCategories, redactSecrets } from "./engine/memory.js";
 import { decodedEntitiesNote, LANE_NAME_MAX, LaneLedger, laneCloseGuard, laneReportInstruction, parseLaneReport, summarizeLaneReport } from "./engine/lane.js";
 import { MAX_UNFILED_NAMED, unfiledDefects } from "./engine/calibration.js";
 import { SessionQueue, withWatchdog } from "./engine/dispatch.js";
@@ -1168,7 +1168,7 @@ server.registerTool("scout_finding", {
             ? `Finding recorded: [${finding.severity}] ${finding.title} (id ${finding.id})`
             : finding.regressedAt
                 ? `⟳ REOPENED as a REGRESSION: finding ${finding.id} was previously resolved but the evidence reproduces again (seen in ${finding.runs} runs). Worth calling out to the user.`
-                : `Not recorded as new: merged into existing finding ${finding.id} — [${finding.severity}] ${finding.title}${finding.evidence ? ` (evidence: ${finding.evidence.slice(0, 160)})` : " (no evidence)"}, seen in ${finding.runs} runs. If yours is a different bug, file it again with evidence naming the request that failed for you (method and path): two findings are kept apart when both name requests and none is shared.`, session);
+                : `Not recorded as new: merged into existing finding ${finding.id} — [${finding.severity}] ${finding.title}${finding.evidence ? ` (evidence: ${finding.evidence.slice(0, 160)})` : " (no evidence)"}, filed as ${finding.category}, seen in ${finding.runs} runs. If yours is a different bug, file it again: under the category that says what is wrong if it is another kind of defect (a finding filed as ${category} merges only with one filed as ${mergeableCategories(category).join(" or ")}), or with evidence naming the request that failed for you (method and path) — two findings are kept apart when both name requests and none is shared.`, session);
     }
     catch (err) {
         return errorText(err);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "scenescout",
-  "version": "3.11.0",
+  "version": "3.11.1",
   "description": "SceneScout — exploratory UI testing for AI coding agents. An MCP server that gives any agent (Claude Code, Cursor, VS Code Copilot, Codex, Gemini CLI and others) a structured view of a running web app, always-on oracles, a network-level write policy, memory across runs and a gap-checked report.",
   "license": "MIT",
   "author": "brunoboto96",