scenescout 3.17.0 → 3.18.0

package/CHANGELOG.md

@@ -1,5 +1,22 @@
 # scenescout
 
+## 3.18.0
+
+### Minor Changes
+
+- 1a6ce12: Ask the start-of-run questions as one form where the client can show one. A new `scout_intake` tool checks whether the client declared MCP elicitation in form mode and, if it did, asks the address, whether and how to sign in, what to check and whether the site holds real data in a single form, then returns the `scout_login` and `scout_attach` calls the answers choose. With no form support, or when the person declines or closes the form, it returns the questions for the agent to ask in chat, as before. The form never asks for a password or a code: signing in stays in the window `scout_login` opens. The skill and the `explore` prompt call `scout_intake` before setup.
+
+### Patch Changes
+
+- 96bee5c: `scenescout check` and the first look no longer report a feed, a plain-text file, XML, JSON, a PDF or an image as a dead end. A route's response content type now decides whether it is a page: one served as anything but `text/html` or `application/xhtml+xml` is listed under "Not pages" in the report and as `resources` in `check.json`, is not checked against the page rules, and does not count towards the routes checked or `--max-routes`. One that answers 4xx or 5xx is still reported as the route's error.
+- 952fbf3: A check or first look with no signed-in session no longer reports each route that redirects to sign-in as an `auth-redirect` issue. It lists them once as a coverage gap ("N routes need sign-in; give a role to cover them"), under "Needs sign-in" in the report and as `needsSignIn` in `check.json`. With `--storage-state`, a redirect to sign-in is still reported as a lost session.
+- 6073a0d: A link or button with no text is now named by a descendant's `aria-label` (an icon element inside the link) and by an image's `title` when its alt text is empty, as the browser names it, so `unnamed-control` no longer reports those controls. Coverage recorded for them under their earlier keys carries over.
+- cd0639b: Text drawn over a positioned `<img>`, `<picture>`, `<video>` or `<canvas>` is no longer reported as a contrast failure measured against the page background (often as 1.00:1). Like text over a CSS background image, it has no single background colour, so the design audit and `scenescout check` leave it unmeasured.
+- 680e4f5: The focus-indicator rule no longer reports an iframe as a control with no visible focus indicator. A Tab onto a frame moves focus into the frame's document, so the design audit and `scenescout check` now follow focus into a same-origin frame and measure the control focused there, reported by its own name. A press that leaves focus inside another site's frame is skipped.
+- fe16d4c: A page that locks scrolling behind a modal opened inside a full-viewport frame is no longer reported as a leaked scroll lock. The overlay probe now counts a visible iframe that covers the viewport, is pinned itself or through a fixed ancestor, and is neither faded out nor click-through as an open overlay, provided that, when the frame is same-origin, a dialog or a dimming backdrop is showing inside it. A frame left mounted after its modal closed does not justify the lock, so that page is still reported at high severity.
+- 96b74aa: The design audit's `image-aspect` rule no longer reports an image that keeps its proportions through `object-fit: cover`, `contain`, `scale-down` or `none`, whether set inline or by a class. Only an image stretched to its box under `fill`, the default, is reported as distorted.
+- 7fc6013: The gap ledger's list of pages whose POST observe refused now names the page that sent the POST. A page whose script posted as it loaded could be listed as the page the session came from, because the browser had not yet reported the new page; the request's Referer now decides, with the session's page as the fallback.
+
 ## 3.17.0
 
 ### Minor Changes

package/README.md

@@ -205,7 +205,7 @@ The agent scans the project (if there is one), attaches read-only, explores, and
 
 **Common flags** — `--level minimal|medium|extensive` · `--url <app>` · `--role <name\|path>` (who to explore as: a login saved with `scenescout login`, a storage state found by the scan, or a path to a Playwright storage-state JSON) · `--focus <text>` (a ticket or a sentence to check) · `--observe` / `--read-only` / `--safe-write` / `--allow-destructive`.
 
-**No flags at all** (`/scenescout` on its own) and the agent asks four plain questions instead: the address, whether and how you sign in, what to check (tickets or a description), and whether the site holds real data. Real data, or not being sure, means nothing but `GET` requests leave the page; you are never asked to pick a mode. Any flag skips the questions. See [Plain questions instead of flags](docs/guide/Ways-to-use-it.md#plain-questions-instead-of-flags).
+**No flags at all** (`/scenescout` on its own) and the agent asks four plain questions instead: the address, whether and how you sign in, what to check (tickets or a description), and whether the site holds real data. Where your client can show a form, `scout_intake` asks them as one form, never for a password; otherwise the agent asks in chat. Real data, or not being sure, means nothing but `GET` requests leave the page; you are never asked to pick a mode. Any flag skips the questions. See [Plain questions instead of flags](docs/guide/Ways-to-use-it.md#plain-questions-instead-of-flags).
 
 ### 🔑 Signing in as a role
 
@@ -329,11 +329,11 @@ Snapshots are cheap: re-snapshotting a route returns only *what changed*, with s
 
 ## 🧰 The toolbox
 
-34 deterministic tools. The agent picks; you rarely call these by hand.
+35 deterministic tools. The agent picks; you rarely call these by hand.
 
 | Phase | Tools | What they do |
 |---|---|---|
-| **Set up** | `scout_playbook` `scout_scan` `scout_attach` `scout_session` | Hand the testing method to an agent that has no skill loaded; discover routes; launch a browser in a write-mode; keep several authenticated roles alive at once |
+| **Set up** | `scout_playbook` `scout_intake` `scout_scan` `scout_attach` `scout_session` | Hand the testing method to an agent that has no skill loaded; ask the start-of-run questions as one form where the client can show one; discover routes; launch a browser in a write-mode; keep several authenticated roles alive at once |
 | **Explore** | `scout_crawl` `scout_coverage` | Sweep every route in one call; ask what's still untested |
 | **Look** | `scout_snapshot` `scout_hover` `scout_screenshot` `scout_capture` | Read the structured scene (diffed); reveal tooltips/hover cards; capture pixels only when needed; save one element as a PNG to show someone |
 | **Ask the server** | `scout_request` `scout_network` | Call the app's own API as this session, with the UI bypassed — the check that turns a hidden button into a proven refusal; list the requests the page itself made since it loaded |
@@ -761,6 +761,7 @@ test-app/           fixtures for the real-browser smoke tests
 skills/scenescout/   the testing method (SKILL.md): a skill in Claude Code, served by the server everywhere else
 docs/how-it-works.md  what happens at each stage, in diagrams
 docs/benchmark.md   measuring whether a change made runs better
+docs/validation.md  scorecards from runs against public open-source apps
 docs/adr/           why it's built this way
 ```
 
@@ -774,6 +775,8 @@ docs/adr/           why it's built this way
 
 **[Measuring whether a change helped](docs/benchmark.md)** — the demo app's answer key, the scorecard (recall, precision, judged-not-filed, severity, calibration), and the results log of every run, including what did not help.
 
+**[Validation on public open-source apps](docs/validation.md)** — runs against three well-known open-source web apps, with a scorecard for each: issues by severity, how many were real and how many were false positives, and the engine problems the runs exposed.
+
 The load-bearing choices are recorded as ADRs — read the relevant one before changing a rule it covers:
 
 - [1 · Completion is an enforced contract, not a claim](docs/adr/0001-completion-is-a-contract-not-a-vibe.md)

package/dist/check-run.js

@@ -11,11 +11,12 @@ import os from "node:os";
 import path from "node:path";
 import { BASELINE_CAPTURE, baselineFiles, baselineMeta, defaultBaselinesDir, elementTarget, isVisualPicture, judgeBaseline, missingTargetsMessage, parseBaselineTargets, readStoredBaseline, TARGETS_FILE, VISUAL_DIRNAME, visualFiles, } from "./engine/baseline.js";
 import { BrowserEngine } from "./engine/browser.js";
-import { checkFindings, redactBaselineRun, MAX_DISCOVERY_ROUNDS, redactFlowRuns, redactRoute, redactRoutes, settingsOf, unmeasuredReason, withoutOwnResponse, } from "./engine/check.js";
+import { checkFindings, redactBaselineRun, MAX_DISCOVERY_ROUNDS, redactFlowRuns, redactRoute, redactRoutes, settingsOf, splitResources, unmeasuredReason, withoutOwnResponse, } from "./engine/check.js";
 import { loadFlows, resolveFlowsDir } from "./engine/flow.js";
 import { MemoryStore, MEMORY_DIRNAME, writeSelfIgnore } from "./engine/memory.js";
 import { decodePng, encodePng } from "./engine/png.js";
 import { firstLineOf } from "./engine/limits.js";
+import { isNonPageResource } from "./engine/crawl.js";
 import { checkRetestPlan, retestResults, wellFormedFindings } from "./engine/verify.js";
 /** The baselines folder a check uses: the one --baselines names, else the project's own. */
 export function baselinesDirOf(options) {
@@ -107,6 +108,8 @@ export async function runCheck(options, log = () => { }, inputs = { flows: [], f
         if (authFailed)
             throw new Error(authFailed.replace(/ Continuing now tests a logged-out app\.$/, "").replace(/re-attach/, "run the check again"));
         const routes = [];
+        // A feed or a file the crawl followed is not a page, so it does not use up --max-routes.
+        const pagesSoFar = () => routes.filter((r) => !isNonPageResource(r)).length;
         const crawl = async (paths, limit, deadline) => {
             await engine.crawl(paths, { inspect: true, limit, deadline });
             routes.push(...engine.lastCrawlHealth);
@@ -118,15 +121,15 @@ export async function runCheck(options, log = () => { }, inputs = { flows: [], f
             // The start page first, whatever else is known: it is the one route the user named. The
             // time budget does not apply to it, so a run always measures at least the page it was given.
             await crawl([`${start.pathname}${start.search}${start.hash}`], 1);
-            for (let round = 0; round < MAX_DISCOVERY_ROUNDS && routes.length < options.maxRoutes; round++) {
+            for (let round = 0; round < MAX_DISCOVERY_ROUNDS && pagesSoFar() < options.maxRoutes; round++) {
                 if (engine.crawlableRoutes().length === 0 || pastDeadline())
                     break;
-                await crawl(undefined, options.maxRoutes - routes.length, deadline);
-                log(`  ${routes.length} route(s) checked`);
+                await crawl(undefined, options.maxRoutes - pagesSoFar(), deadline);
+                log(`  ${pagesSoFar()} route(s) checked`);
             }
         }
         // Out of time with routes still to visit; a cap of routes reached first is --max-routes's to report.
-        const timeLimitReached = pastDeadline() && routes.length < options.maxRoutes && engine.crawlableRoutes().length > 0;
+        const timeLimitReached = pastDeadline() && pagesSoFar() < options.maxRoutes && engine.crawlableRoutes().length > 0;
         // Pages of open findings the crawl did not load exactly: loaded now, so each re-test has its own measurement.
         // Kept apart from `routes`: they are measured only to re-test, never checked against the page rules, and do not count
         // towards --max-routes. With --paths the check stays on the paths it was given.
@@ -172,18 +175,22 @@ export async function runCheck(options, log = () => { }, inputs = { flows: [], f
                 }))),
             }
             : null;
-        const measured = redactRoutes(routes.map(withoutOwnResponse));
+        const { pages: measured, resources } = splitResources(redactRoutes(routes.map(withoutOwnResponse)));
         const flows = redactFlowRuns(flowRuns);
         const pictured = baselines ? redactBaselineRun(baselines) : null;
-        const { issues, worthALook } = checkFindings(measured, start.origin, options.ignore, flows, pictured, options.ignorePaths);
+        // Signed in only when given a session: with none, a route that sends the browser to sign-in needs one, and is not a lost session.
+        const signedIn = options.storageStatePath !== undefined;
+        const { issues, worthALook, needsSignIn } = checkFindings(measured, start.origin, options.ignore, flows, pictured, options.ignorePaths, signedIn);
         return {
             url: redactRoute(options.url),
             generatedAt: new Date().toISOString(),
             mode: options.mode,
             failOn: options.failOn,
             routes: measured,
+            resources,
             issues,
             worthALook,
+            ...(signedIn ? {} : { needsSignIn }),
             // Routes that failed to load are issues already; "not visited" is only what --max-routes (or the time budget) left out.
             unvisited: options.paths ? [] : engine.crawlableRoutes().map(redactRoute),
             ...(options.timeBudgetMs !== undefined ? { timeBudget: { ms: options.timeBudgetMs, reached: timeLimitReached } } : {}),

package/dist/engine/browser.js

@@ -1,8 +1,8 @@
 import { chromium, firefox, webkit, } from "playwright";
 import fs from "node:fs";
 import path from "node:path";
-import { elementKey, fingerprintState, isNonPageRoute, keyAliases, normalizePath, ordinalKeys, refsSurviveUrlChange, routeBase, } from "./fingerprint.js";
-import { AUTH_LOSS_PREFIX, JOURNEY_END, JOURNEY_START, MemoryStore, reachedRoutes, redactSecrets, TASK_SET } from "./memory.js";
+import { elementKey, fingerprintState, isNonPageRoute, keyAliasSteps, normalizePath, ordinalKeys, refsSurviveUrlChange, routeBase, } from "./fingerprint.js";
+import { AUTH_LOSS_PREFIX, JOURNEY_END, JOURNEY_START, MemoryStore, NAME_RULE, reachedRoutes, redactSecrets, TASK_SET, } from "./memory.js";
 import { normalizeTask } from "./task.js";
 import { ACTED_CONTROL_SRC, CLAIM_SCAN_SCRIPT, findContradictions, INFRASTRUCTURE_WRITE_RE, isBackgroundRequest, OPEN_DIALOGS_SCRIPT, quietAnswer, saidSince, } from "./claims.js";
 import { POSTMESSAGE_BINDING, describeTokenPost, postMessageCaptureScript, tokenHits, tokenPostKey } from "./postmessage.js";
@@ -17,7 +17,7 @@ import { formatJourney, journeyTime, measureJourney } from "./journey.js";
 import { describeStep, FLOW_AFTER_LAST_STEP_MS, isAction, matchRequest, parseTarget, splitRefusals, TARGET_HELP, urlMatches, } from "./flow.js";
 import { framePath, RECORD_MAX_FRAMES } from "./replay.js";
 import { InFlightRequests, keepWatchingUrl, normalizePace, SETTLE_TICK_MS, shouldKeepWaiting } from "./settle.js";
-import { crawledRoute, crawlLine, mainStateFlag } from "./crawl.js";
+import { crawledRoute, crawlLine, isFileMediaType, isNonPageResource, mainStateFlag, mediaTypeOf } from "./crawl.js";
 import { authToRemember, BODY_FETCH_MAX, buildRequestScript, formatPageRequests, formatReplay, PageRequests, replaySignature, requestHeaders, resolveMethod, resolveRequestUrl, resolveTarget, staleCredentialNote, toReplayResult, wantsView, } from "./request.js";
 import { defaultEngine, focusAdvanceKey, REMOVE_SHARED_WORKER_SCRIPT, screencastSupport, serviceWorkerPolicy, sharedWorkersAllowed, unloadWriteInterception, closeWaitsForLeavingWrites, } from "../browsers.js";
 import { revealedLines } from "./hover.js";
@@ -28,7 +28,7 @@ import { performScroll, probeFocusIndicators, probeOverlays, scrollContainer } f
 import { boundedTeardown } from "./teardown.js";
 import { BROWSER_MARKER, reapOrphanBrowsers } from "./reaper.js";
 import { planUploadOptions, resolveDiskUpload } from "./uploads.js";
-import { answersWithRefusal, destructiveRefusal, destructiveLabelOf, pickIsDestructive, isDestructive, isDestructiveWire, blockSignature, BlockNotices, dialogNote, dialogResponse, ESCAPE_REFUSAL, allowsWrite, policyRefusal, foreignFrameOrigin, foreignWrite, withForeignFrameSandbox, offAppPageWrite, embedOfRequest, hostileForEmbed, trustedEmbedOrigins, MAX_TRUSTED_EMBEDS, MAX_READ_POSTS, READ_POSTS_ENV, readPostAllowed, matchReadPost, readPostEntries, readPostsSetting, trustsForeignWrite, embedProbeRefusal, EmbedMoveTracker, sandboxedRedirectPage, allowsForeignWriteOnSignIn, isAuthExempt, WriteRule, } from "./policy.js";
+import { answersWithRefusal, destructiveRefusal, destructiveLabelOf, pickIsDestructive, isDestructive, isDestructiveWire, blockSignature, BlockNotices, dialogNote, dialogResponse, ESCAPE_REFUSAL, allowsWrite, policyRefusal, foreignFrameOrigin, foreignWrite, withForeignFrameSandbox, offAppPageWrite, embedOfRequest, hostileForEmbed, trustedEmbedOrigins, MAX_TRUSTED_EMBEDS, MAX_READ_POSTS, READ_POSTS_ENV, readPostAllowed, matchReadPost, refusedPostPage, readPostEntries, readPostsSetting, trustsForeignWrite, embedProbeRefusal, EmbedMoveTracker, sandboxedRedirectPage, allowsForeignWriteOnSignIn, isAuthExempt, WriteRule, } from "./policy.js";
 import { scanProject } from "../scan.js";
 import { analyzeDesign, DESIGN_COLLECT_SCRIPT } from "./design.js";
 import { acceptMatches, generatedUpload } from "./fixtures.js";
@@ -1406,7 +1406,7 @@ export class BrowserEngine {
                 const refuse = (why) => {
                     const answered = answersWithRefusal(req.resourceType());
                     if (rule === "observe" && method === "POST" && !why && answered && !BENIGN_MUTATION_RE.test(url))
-                        this.noteObserveRefusedPost(url, req.postData());
+                        this.noteObserveRefusedPost(url, req.postData(), this.sentByMainFrame(req) ? req.headers()["referer"] : undefined);
                     this.noteBlocked({ at: Date.now(), sig: `${method} ${url.slice(0, 140)}`, answered, why, type: req.resourceType() });
                     this.logAction({ action: "write-policy:blocked", target: `${method} ${pathname}${why ? ` (${why})` : ""}`, url: this.page?.url() ?? "" });
                     this.refusedByPolicy.add(req);
@@ -1826,16 +1826,16 @@ export class BrowserEngine {
         // A live region listed for what it says is known by its role, not its
         // text, so a new message reads as the same region saying something else.
         const baseKeyOf = (el, name, tag) => frameElementKey(el.liveOnly ? `live:${el.role}` : elementKey({ ...el, name }), tag);
+        // Each element's base key under every name rule, rule 1 first, so each step between two rules gets its own aliases.
         const bases = all.map(({ raw: el, tag }) => ({
-            base: baseKeyOf(el, el.name, tag),
-            prior: baseKeyOf(el, el.priorName ?? el.name, tag),
+            byRule: Array.from({ length: NAME_RULE }, (_, i) => baseKeyOf(el, i < NAME_RULE - 1 ? (el.priorNames?.[i] ?? el.name) : el.name, tag)),
             tracked: !el.liveOnly,
         }));
-        const keys = ordinalKeys(bases.map((b) => b.base));
-        const aliases = keyAliases(bases);
+        const keys = ordinalKeys(bases.map((b) => b.byRule[NAME_RULE - 1]));
+        const aliases = keyAliasSteps(bases);
         const built = all.map(({ raw: el, frame, tag }, i) => {
             const key = keys[i];
-            const { ownText: _ownText, centre: _centre, cut: _cut, priorName: _priorName, ...listed } = el;
+            const { ownText: _ownText, centre: _centre, cut: _cut, priorNames: _priorNames, ...listed } = el;
             const full = {
                 ...listed,
                 ...(tag ? { frame: tag } : {}),
@@ -2332,18 +2332,29 @@ export class BrowserEngine {
     }
     /**
      * Remember in project memory, for the gap ledger, a script's POST that
-     * observe refused on the page the session is on. Not one that looks
-     * destructive, nor one to an endpoint already named as a read (its body was
-     * a mutation): naming it would change nothing.
+     * observe refused, charged to the page that sent it (policy.ts
+     * refusedPostPage). Not one that looks destructive, nor one to an endpoint
+     * already named as a read (its body was a mutation): naming it would change
+     * nothing.
      */
-    noteObserveRefusedPost(url, body) {
-        const pageUrl = this.page?.url();
+    noteObserveRefusedPost(url, body, referer) {
+        const pageUrl = refusedPostPage(this.baseUrl, referer, this.page?.url());
         if (!pageUrl || !this.memory || isDestructiveWire(pathnameOf(url), body) || matchReadPost(this.readPosts, this.baseUrl, url))
             return;
         const endpoint = this.refusedPostEndpoint(url);
         if (endpoint)
             this.memory.noteObserveRefusedPost(normalizePath(pageUrl), endpoint);
     }
+    /** Whether the session's page sent `req` from its top document, not from a frame or a service worker. */
+    sentByMainFrame(req) {
+        try {
+            return !!this.page && req.frame() === this.page.mainFrame();
+        }
+        catch {
+            // A service worker's request has no frame.
+            return false;
+        }
+    }
     /** A POST to this endpoint went out, so the pages observe refused it on are no longer a gap for it. */
     clearRefusedPost(url) {
         const endpoint = this.refusedPostEndpoint(url);
@@ -4332,7 +4343,9 @@ export class BrowserEngine {
     /** The full route contract: scanned filesystem routes ∪ link-discovered route classes. */
     allKnownRoutes() {
         const discovered = this.memory ? Object.keys(this.memory.discoveredRoutes) : [];
-        return [...new Set([...this.knownRoutes, ...discovered])].filter((r) => !isNonPageRoute(r));
+        // A route that answered with a feed or a file is not a page, whatever its path looks like.
+        const resources = new Set(Object.keys(this.memory?.resourceRoutes ?? {}).map(normalizePath));
+        return [...new Set([...this.knownRoutes, ...discovered])].filter((r) => !isNonPageRoute(r) && !resources.has(normalizePath(r)));
     }
     /**
      * Contract routes neither visited nor attempted by THIS role.
@@ -4450,29 +4463,89 @@ export class BrowserEngine {
             this.actionStartedAt = Date.now();
             this.oracles.drain(false); // discard pre-route leftovers WITHOUT marking their signatures as reported
             let status = "ERR";
+            // The main document's response, as the browser received it. Kept apart from goto's own result: a browser that
+            // treats a PDF, a feed or an attachment as a download throws from goto, and this is then the only record of it.
+            const answered = { last: null };
+            const asked = new URL(url).href;
+            const onResponse = (r) => {
+                if (r.frame() !== page.mainFrame() || !r.request().isNavigationRequest())
+                    return;
+                // A late event from the page before is not this route's: its redirect chain must start at the URL asked for.
+                let first = r.request();
+                for (let from = first.redirectedFrom(); from; from = from.redirectedFrom())
+                    first = from;
+                if (first.url() !== asked)
+                    return;
+                answered.last = { url: r.url(), status: r.status(), contentType: mediaTypeOf(r.headers()["content-type"]) };
+            };
+            page.on("response", onResponse);
+            let resource = null;
             try {
                 const resp = await page.goto(url, { waitUntil: "domcontentloaded", timeout: this.limits.crawlNavMs });
                 status = resp?.status() ?? "no-response";
+                const contentType = mediaTypeOf(resp?.headers()["content-type"]);
+                if (resp && isNonPageResource({ status: resp.status(), contentType }))
+                    resource = { url: page.url(), status: resp.status(), contentType: contentType };
             }
             catch (err) {
-                const explained = explainTimeout(err, "nav", this.limits.crawlNavMs);
-                const reason = explained instanceof Error ? explained.message.split("\n")[0] : String(explained);
-                // Not retried by later crawls in this process (a check would otherwise try it
-                // on every discovery round). Deliberately not written to memory: one outage
-                // must not count a route as covered in every later run's gap ledger.
-                if (!opts.measureOnly)
-                    this.loadFailedRoutes.add(normalizePath(url));
-                await this.errorPageCommitted(page);
-                summary.push(`${path} — LOAD FAILED`);
-                // The page a re-attach went back to never loaded, so nothing says whether it worked.
-                if (this.authLoss.reattaching)
-                    this.authLoss.abortReattach(`the page it went back to, ${path}, did not load (${reason})`);
-                problems.push(`${path}: ${reason}`);
+                // A browser that took the answer as a download failed the navigation, but the route did answer, with its status:
+                // a feed that is missing is the route's 404, not a page that did not load.
+                const seen = answered.last;
+                if (seen && isFileMediaType(seen.contentType))
+                    resource = { url: seen.url, status: seen.status, contentType: seen.contentType };
+                else {
+                    const explained = explainTimeout(err, "nav", this.limits.crawlNavMs);
+                    const reason = explained instanceof Error ? explained.message.split("\n")[0] : String(explained);
+                    // Not retried by later crawls in this process (a check would otherwise try it
+                    // on every discovery round). Deliberately not written to memory: one outage
+                    // must not count a route as covered in every later run's gap ledger.
+                    if (!opts.measureOnly)
+                        this.loadFailedRoutes.add(normalizePath(url));
+                    await this.errorPageCommitted(page);
+                    summary.push(`${path} — LOAD FAILED`);
+                    // The page a re-attach went back to never loaded, so nothing says whether it worked.
+                    if (this.authLoss.reattaching)
+                        this.authLoss.abortReattach(`the page it went back to, ${path}, did not load (${reason})`);
+                    problems.push(`${path}: ${reason}`);
+                    this.crawlHealth.push({
+                        path,
+                        url,
+                        status: null,
+                        loadError: reason,
+                        loginRedirect: false,
+                        elements: 0,
+                        unnamed: [],
+                        placeholderOnly: [],
+                        violations: [],
+                        geometry: [],
+                        brokenImages: [],
+                        design: [],
+                    });
+                    continue;
+                }
+            }
+            finally {
+                page.off("response", onResponse);
+            }
+            // A feed, a file or data: recorded as what it is and left there. It has no controls to collect or audit, and one
+            // that answered leaves the route contract, so it is neither a dead end nor a gap.
+            if (resource) {
+                if (!opts.measureOnly) {
+                    // Only a file that answered leaves the contract; a failing one is attempted with its status, as a failing page is.
+                    if (isNonPageResource(resource))
+                        memory.markResource(normalizePath(url), resource.contentType);
+                    else
+                        memory.markAttempted(normalizePath(url), `status:${resource.status}`, this.role);
+                }
+                this.oracles.drain(false);
+                summary.push(`${path} — ${resource.status} · not a page (${resource.contentType})`);
+                if (!isNonPageResource(resource))
+                    problems.push(`${path} → HTTP ${resource.status} (${resource.contentType})`);
                 this.crawlHealth.push({
                     path,
-                    url,
-                    status: null,
-                    loadError: reason,
+                    url: resource.url,
+                    status: resource.status,
+                    contentType: resource.contentType,
                     loginRedirect: false,
                     elements: 0,
                     unnamed: [],
@@ -4494,6 +4567,9 @@ export class BrowserEngine {
                 memory.visitState(fp, finalUrl, route, trackedElements(elements).map((el) => el.key), inertKeys(elements), this.sessionKey, aliases);
                 for (const f of forms)
                     memory.recordForm(fp, f.key, f.guarded, this.sessionKey);
+                // A route remembered as a feed or a file that now answers as a page (a route class whose ids serve both, or
+                // one that served a plain-text maintenance reply once) is a page again, so it rejoins the contract.
+                memory.clearResource(normalizePath(url));
                 memory.recordRoleAccess(this.role, route, "reached");
             }
             // If we landed somewhere else (auth wall, canonical redirect), the

package/dist/engine/check.js

@@ -13,6 +13,7 @@
 import { createHash } from "node:crypto";
 import { BROWSER_ENGINES } from "../browsers.js";
 import { BASELINE_MODES, baselineEvidence, baselineFingerprint, countStatuses, describeChange, parseThreshold, VISUAL_RULE, } from "./baseline.js";
+import { isNonPageResource } from "./crawl.js";
 import { flowStepEvidence } from "./flow.js";
 import { parseLimitFlag } from "./limits.js";
 import { redactRoute, redactSecrets } from "./memory.js";
@@ -20,6 +21,23 @@ import { SARIF_ANCHOR_FALLBACKS, checkSarifAnchor, sarifFileFor, sarifLocation }
 import { httpErrorDetail, httpStatusOf } from "./oracles.js";
 export const CHECK_SEVERITIES = ["high", "medium", "low"];
 export const FAIL_ON = [...CHECK_SEVERITIES, "never"];
+/**
+ * The routes that answered as pages, and those that answered as something
+ * else. Only the pages are checked against the page rules and counted as
+ * routes; the rest are listed apart, so a feed or a licence file is neither a
+ * dead end nor a page the check claims to have measured.
+ */
+export function splitResources(routes) {
+    const pages = [];
+    const resources = [];
+    for (const r of routes) {
+        if (isNonPageResource(r) && r.status !== null && r.contentType)
+            resources.push({ path: r.path, status: r.status, contentType: r.contentType });
+        else
+            pages.push(r);
+    }
+    return { pages, resources };
+}
 export const CHECK_RULES = {
     "route-load-failed": { severity: "high", title: "Page did not load", help: "Navigating to the route failed or timed out." },
     "route-server-error": { severity: "high", title: "Page answered with a server error", help: "The document itself returned HTTP 5xx." },
@@ -45,7 +63,11 @@ export const CHECK_RULES = {
         title: "Credential posted to any origin",
         help: 'The page called postMessage with targetOrigin "*" and a token-shaped value in the message, so whatever origin the receiving window holds can read it. The value is never reported: its path in the message, its shape and its first four characters are.',
     },
-    "auth-redirect": { severity: "medium", title: "Sent to sign-in", help: "The route redirected to a sign-in page; the session is missing or expired." },
+    "auth-redirect": {
+        severity: "medium",
+        title: "Sent to sign-in",
+        help: "The route redirected to a sign-in page although the check was given a signed-in session, so the session is missing or expired. With no session given, such a route is listed as needing sign-in instead.",
+    },
     "dead-end": { severity: "medium", title: "Dead end", help: "The page has no controls at all: no navigation and no way back." },
     "blocking-overlay": {
         severity: "high",
@@ -81,7 +103,11 @@ export const CHECK_RULES = {
         title: "Field labelled only by its placeholder",
         help: "The field has no label, aria-label, aria-labelledby or title. Its placeholder is not a label: it disappears as soon as the user types, and some assistive technology does not announce it.",
     },
-    contrast: { severity: "low", title: "Text contrast below WCAG", help: "Text needs 4.5:1 (3:1 when large) against its background." },
+    contrast: {
+        severity: "low",
+        title: "Text contrast below WCAG",
+        help: "Text needs 4.5:1 (3:1 when large) against its background. Text over an image, gradient, video or canvas has no single background colour and is not measured.",
+    },
     "focus-indicator": { severity: "low", title: "No visible focus indicator", help: "Tabbing to the control changes nothing on screen." },
     "horizontal-scroll": { severity: "medium", title: "Page scrolls sideways", help: "Content is wider than the viewport." },
     "tiny-target": {
@@ -214,8 +240,15 @@ export function geometryRule(line) {
  * `--ignore-path` drops a fact on the route it names, or one rule there,
  * before the fact is filed. A fact also seen on a route that is not exempted
  * stays, on that route alone.
+ *
+ * A route that sends the browser to sign-in is an `auth-redirect` issue only
+ * when the check was `signedIn`: then a session it was given was lost. With
+ * no session, sending a signed-out visitor to sign-in is the route working,
+ * so it goes to `needsSignIn` instead, a coverage gap rather than a defect.
+ * The default is signed in, so a caller that does not say never hides one.
  */
-export function checkFindings(routes, origin, ignore = [], flows = [], baselines = null, ignorePaths = []) {
+export function checkFindings(routes, origin, ignore = [], flows = [], baselines = null, ignorePaths = [], signedIn = true) {
+    const needsSignIn = [];
     const byKey = new Map();
     const looks = new Map();
     /** A route (and, when named, one rule on it) that --ignore-path exempts. */
@@ -254,6 +287,10 @@ export function checkFindings(routes, origin, ignore = [], flows = [], baselines
     };
     for (const r of routes) {
         const route = r.path;
+        // A feed, a text file or an image is not a page: it has no controls to judge. Only one that answered without an
+        // error is one (crawl.ts isNonPageResource), so a feed that fails reaches the route's own error below.
+        if (isNonPageResource(r))
+            continue;
         if (r.loadError !== undefined) {
             add("route-load-failed", `${route}: ${r.loadError}`, route);
             continue;
@@ -262,7 +299,12 @@ export function checkFindings(routes, origin, ignore = [], flows = [], baselines
             add("route-server-error", `${route} → HTTP ${r.status}`, route);
         else if (r.status !== null && r.status >= 400)
             add("route-client-error", `${route} → HTTP ${r.status}`, route);
-        if (r.loginRedirect)
+        if (r.loginRedirect && !signedIn) {
+            // An --ignore-path exemption takes the route off the list too; --ignore auth-redirect does not, since this is coverage, not that rule.
+            if (!exempt("auth-redirect", route) && !needsSignIn.includes(route))
+                needsSignIn.push(route);
+        }
+        else if (r.loginRedirect)
             add("auth-redirect", `${route} → ${withoutOrigin(r.url, origin) || "/"}`, route);
         else if (r.elements === 0 && (r.status === null || r.status < 400))
             add("dead-end", `${route}: 0 controls`, route);
@@ -310,11 +352,12 @@ export function checkFindings(routes, origin, ignore = [], flows = [], baselines
     return {
         issues: [...byKey.values(), ...visual].sort((a, b) => SEVERITY_RANK[a.severity] - SEVERITY_RANK[b.severity] || a.rule.localeCompare(b.rule)),
         worthALook: [...looks.values()].sort((a, b) => a.rule.localeCompare(b.rule)),
+        needsSignIn,
     };
 }
 /** The defect tier of `checkFindings`: what counts, and what the gate reads. */
-export function issuesFromRoutes(routes, origin, ignore = [], flows = [], ignorePaths = []) {
-    return checkFindings(routes, origin, ignore, flows, null, ignorePaths).issues;
+export function issuesFromRoutes(routes, origin, ignore = [], flows = [], ignorePaths = [], signedIn = true) {
+    return checkFindings(routes, origin, ignore, flows, null, ignorePaths, signedIn).issues;
 }
 /**
  * Why a check has nothing to give a verdict on, or null when it measured the
@@ -916,6 +959,42 @@ export function routesTable(result) {
     }
     return lines;
 }
+/** The routes that answered with a feed, a file or data, as their own table; nothing when there were none. */
+export function resourcesTable(resources) {
+    if (!resources || resources.length === 0)
+        return [];
+    const lines = [
+        "",
+        "## Not pages",
+        "",
+        "Answered with a feed, a file or data, so not checked as pages.",
+        "",
+        "| Route | Status | Content type |",
+        "|---|---|---|",
+    ];
+    for (const r of resources)
+        lines.push(`| ${cell(r.path)} | ${r.status} | ${cell(r.contentType)} |`);
+    return lines;
+}
+/** The coverage gap a check with no session leaves, in one sentence: how many routes (or pages) need sign-in, and `cover`, what covers them. */
+export function needsSignInText(count, cover, noun = "route") {
+    return `${count} ${noun}${count === 1 ? " needs" : "s need"} sign-in; give a role to cover them: ${cover}.`;
+}
+/** The routes that sent a check with no session to sign-in, as their own section; nothing when there were none. */
+export function needsSignInSection(result) {
+    const routes = result.needsSignIn ?? [];
+    if (routes.length === 0)
+        return [];
+    const cover = `pass ${code("--storage-state")} with a signed-in session, such as the profile ${code(`scenescout login ${result.url} --role <name>`)} saves`;
+    return [
+        "",
+        `## Needs sign-in (${routes.length})`,
+        "",
+        `${needsSignInText(routes.length, cover)} The check had no session, so sending it to sign-in is these routes working, not a defect: what is behind them was not measured.`,
+        "",
+        `${routes.slice(0, 20).map(code).join(", ")}${routes.length > 20 ? " …" : ""}`,
+    ];
+}
 /** The routes known and not visited, as one line; nothing when every known route was visited. */
 export function unvisitedLine(unvisited, why) {
     if (unvisited.length === 0)
@@ -942,12 +1021,15 @@ export function formatCheck(result) {
             : `${couldNotRun > 0 ? "failed" : "**FAILED**"} — ${failingText} · ${counts.high} high · ${counts.medium} medium · ${counts.low} low`) +
         (unaudited > 0 ? ` · design not measured on ${unaudited} route(s)` : "") +
         (uncompared > 0 ? ` · ${uncompared} visual target(s) not compared: no baseline yet` : "") +
-        (result.worthALook.length > 0 ? ` · ${result.worthALook.length} worth a look, never gated` : ""));
+        (result.worthALook.length > 0 ? ` · ${result.worthALook.length} worth a look, never gated` : "") +
+        (result.needsSignIn?.length ? ` · ${result.needsSignIn.length} route(s) need sign-in, not covered` : ""));
     // Right under the verdict: what a green check was allowed to do is part of what it means.
     lines.push("", `Settings — ${describeSettings(result)}`);
     lines.push(...issueSections(result.issues));
     lines.push(...worthALookSection(result.worthALook));
+    lines.push(...needsSignInSection(result));
     lines.push(...routesTable(result));
+    lines.push(...resourcesTable(result.resources));
     lines.push(...unvisitedLine(result.unvisited, "over --max-routes"));
     if (result.flows.length > 0 || result.skippedFlows.length > 0) {
         lines.push("", `## Flows (${result.flows.length})`, "");
@@ -1023,6 +1105,7 @@ export function toSummaryJson(result, toolVersion) {
             controls: r.elements,
             ...(r.auditError !== undefined ? { designNotMeasured: r.auditError } : {}),
         })),
+        resources: result.resources ?? [],
         unvisited: result.unvisited,
         ...(result.timeBudget ? { timeBudget: result.timeBudget } : {}),
         ignored: result.ignored,
@@ -1042,5 +1125,7 @@ export function toSummaryJson(result, toolVersion) {
         issues: result.issues,
         // Apart from `issues` and `counts`, which the gate reads: none of these is counted or gated.
         worthALook: result.worthALook,
+        // Routes a check with no session was sent to sign-in from: a coverage gap. Absent when the check had a session.
+        ...(result.needsSignIn ? { needsSignIn: result.needsSignIn } : {}),
     };
 }