scenescout 3.17.0 → 3.19.0

package/CHANGELOG.md

@@ -1,5 +1,42 @@
 # scenescout
 
+## 3.19.0
+
+### Minor Changes
+
+- fb7a3e5: An optional Claude Code mod, `scenescout-mod`, is listed in the plugin marketplace beside `scenescout`. In the Claude Code CLI and the desktop Code tab, `/scenescout-pane` opens a pane that refreshes every 2.5 seconds from the server's `scout_status_poll` tool: each session with its task and objective, open findings by severity, coverage, and a link to the live view. Its `lane_model` setting, empty by default, sets the model SceneScout lane agents start on. The `scenescout` plugin, skill and server are unchanged and work the same without it.
+- 1a2db2f: `scenescout export --from <file>` exports a `check.json` from `scenescout check` or a `ci.json` from `scenescout ci`, so the unattended runs can file GitHub or Jira issues without an interactive session's memory. Their issues get the same body, label and marker as memory findings: a check issue is keyed on its fingerprint and a ci finding keeps its memory id, so a re-export files nothing twice. The project's memory stays the default source.
+- 498dbdf: A saved flow can take a `type` or `select` value from the environment with `${env:NAME}`, so a one-time code or a password stays in the CI's secret store rather than in the flow file. A variable that is not set stops the check before it starts, naming the flow and the variable, and every substituted value of four characters or more is masked as `[$NAME]` in everything the check writes and prints.
+- ea8fbfa: Saved flows can assert an element's state with a new step, `expect-element`: a target and a state of `visible`, `hidden`, `enabled`, `disabled`, `checked` or `unchecked`. `hidden` also holds when nothing matches, so a flow can say a dialog closed or a receipt was never shown, and a failure says what is true instead (`testid=receipt is visible, expected hidden`).
+- c07941d: Saved flows have a `repeat` step: run a few click, type, select or press steps until an `expect-text`, `expect-element` or `expect-url` step holds, at most `max` times. It is for gates a fixed script cannot walk, such as paging through a document until its Continue button is enabled. The condition is checked first, so a page already in that state runs nothing, and a failure names what never held and how many rounds were tried.
+- 54257fc: A saved flow can name the `role` it runs as. `scenescout check` replays it in a browser of its own, signed in with the profile `scenescout login --role <name>` saved in the project, while flows naming no role keep the check's own session. Flows run in file-name order, so a journey that passes between people (one submits, another approves) is a sequence of flows. A role with no saved profile stops the check before it starts, with exit 2 and the command that saves one.
+- d439d28: Saved flows have an `upload` step: attach a small valid file generated on the spot (`pdf`, `png`, `txt`, `csv` or `json`, or the kind the input's `accept` asks for) to a file input, the control that opens its chooser, or the page's only file input. A journey that starts from an attached file can now be saved and replayed by `scenescout check`. Submitting the upload is a write, so it reaches the server only under `--flow-writes allow`.
+
+### Patch Changes
+
+- 36c6e11: `scenescout check` no longer passes when the only page it measured was a start page showing at most one control and linking nowhere, which is what an app measured before it finished drawing looks like. Without `--paths`, that check now exits 2 saying only the start page was measured, rather than going green having checked nothing.
+- 87fa5bf: The contrast rule no longer reports text whose own colour is fully transparent, such as the selectable text layer a PDF viewer lays over the rendered page. That text is not painted, so its 1.00:1 ratio is not what anyone sees, and how many of those spans were measured depended on whether the viewer had finished loading. Translucent text that is painted, a faint watermark say, is still measured.
+- 8857bab: `scenescout export` no longer files a finding twice when an export runs straight after another, or after a create whose result was uncertain, while the tracker's issue listing has not caught up. Each filed issue is recorded in `.scenescout/exported.json` and read back by number on the next export; GitHub's newest issues are also read without the label filter; and in Jira a finding whose create may have been made is held back for 15 minutes unless its issue is found.
+- 82f775c: A saved flow step on a control that is shown but stays disabled (or, for `type`, read-only) until the action limit runs out now fails saying so first, `testid=save is visible but disabled after 5s`, followed by the limit hint. It used to report only an action timeout, which never said the control was there but disabled.
+- 830b663: `scenescout check --baseline` now takes each picture again, a frame later, until two in a row are the same, so a baseline is no longer of a moment the page was still drawing: the last frame of an animation it had just stopped, or a script still filling something in after the load. A page that never holds still within the action limit keeps its last picture, and the report says so on that target's line.
+
+## 3.18.0
+
+### Minor Changes
+
+- 1a6ce12: Ask the start-of-run questions as one form where the client can show one. A new `scout_intake` tool checks whether the client declared MCP elicitation in form mode and, if it did, asks the address, whether and how to sign in, what to check and whether the site holds real data in a single form, then returns the `scout_login` and `scout_attach` calls the answers choose. With no form support, or when the person declines or closes the form, it returns the questions for the agent to ask in chat, as before. The form never asks for a password or a code: signing in stays in the window `scout_login` opens. The skill and the `explore` prompt call `scout_intake` before setup.
+
+### Patch Changes
+
+- 96bee5c: `scenescout check` and the first look no longer report a feed, a plain-text file, XML, JSON, a PDF or an image as a dead end. A route's response content type now decides whether it is a page: one served as anything but `text/html` or `application/xhtml+xml` is listed under "Not pages" in the report and as `resources` in `check.json`, is not checked against the page rules, and does not count towards the routes checked or `--max-routes`. One that answers 4xx or 5xx is still reported as the route's error.
+- 952fbf3: A check or first look with no signed-in session no longer reports each route that redirects to sign-in as an `auth-redirect` issue. It lists them once as a coverage gap ("N routes need sign-in; give a role to cover them"), under "Needs sign-in" in the report and as `needsSignIn` in `check.json`. With `--storage-state`, a redirect to sign-in is still reported as a lost session.
+- 6073a0d: A link or button with no text is now named by a descendant's `aria-label` (an icon element inside the link) and by an image's `title` when its alt text is empty, as the browser names it, so `unnamed-control` no longer reports those controls. Coverage recorded for them under their earlier keys carries over.
+- cd0639b: Text drawn over a positioned `<img>`, `<picture>`, `<video>` or `<canvas>` is no longer reported as a contrast failure measured against the page background (often as 1.00:1). Like text over a CSS background image, it has no single background colour, so the design audit and `scenescout check` leave it unmeasured.
+- 680e4f5: The focus-indicator rule no longer reports an iframe as a control with no visible focus indicator. A Tab onto a frame moves focus into the frame's document, so the design audit and `scenescout check` now follow focus into a same-origin frame and measure the control focused there, reported by its own name. A press that leaves focus inside another site's frame is skipped.
+- fe16d4c: A page that locks scrolling behind a modal opened inside a full-viewport frame is no longer reported as a leaked scroll lock. The overlay probe now counts a visible iframe that covers the viewport, is pinned itself or through a fixed ancestor, and is neither faded out nor click-through as an open overlay, provided that, when the frame is same-origin, a dialog or a dimming backdrop is showing inside it. A frame left mounted after its modal closed does not justify the lock, so that page is still reported at high severity.
+- 96b74aa: The design audit's `image-aspect` rule no longer reports an image that keeps its proportions through `object-fit: cover`, `contain`, `scale-down` or `none`, whether set inline or by a class. Only an image stretched to its box under `fill`, the default, is reported as distorted.
+- 7fc6013: The gap ledger's list of pages whose POST observe refused now names the page that sent the POST. A page whose script posted as it loaded could be listed as the page the session came from, because the browser had not yet reported the new page; the request's Referer now decides, with the session's page as the fallback.
+
 ## 3.17.0
 
 ### Minor Changes

package/README.md

@@ -154,6 +154,8 @@ Either way it downloads the browser and registers the server with the client you
 
 Then start a new chat to use SceneScout; the test browser downloads on first use (to have it ready beforehand: `npx -y scenescout install --browser-only`). The command becomes `/scenescout:scenescout`. A plugin's skill comes from this repository and its server from the latest npm release, so right after a release lands here the two can differ for a short while; `/plugin marketplace update scenescout-marketplace` brings the skill up to date.
 
+An optional second plugin, `scenescout-mod@scenescout-marketplace`, adds a run pane (`/scenescout-pane`) and a setting for the model lane agents run on, in the Claude Code CLI and the desktop Code tab. It is a mod: unsandboxed JavaScript that runs inside Claude Code, so it is opt-in. [The Claude Code mod](docs/guide/Ways-to-use-it.md#the-claude-code-mod).
+
 **Using Claude Desktop?** Install the extension: download `scenescout-X.Y.Z.mcpb` from the [latest release](https://github.com/brunoboto96/SceneScout/releases/latest) and open it (or Settings > Extensions > Advanced settings > Install Extension). It works as soon as it is installed, with no terminal step: the test browser downloads on first use. Start a new chat and ask *"Use SceneScout to test http://localhost:3000"*. [More in the guide](docs/guide/Start-here.md#as-a-claude-desktop-extension).
 
 **A client that is not in that list?** [Add the server to its config by hand](#-other-mcp-clients); the test browser downloads on first use.
@@ -205,7 +207,7 @@ The agent scans the project (if there is one), attaches read-only, explores, and
 
 **Common flags** — `--level minimal|medium|extensive` · `--url <app>` · `--role <name\|path>` (who to explore as: a login saved with `scenescout login`, a storage state found by the scan, or a path to a Playwright storage-state JSON) · `--focus <text>` (a ticket or a sentence to check) · `--observe` / `--read-only` / `--safe-write` / `--allow-destructive`.
 
-**No flags at all** (`/scenescout` on its own) and the agent asks four plain questions instead: the address, whether and how you sign in, what to check (tickets or a description), and whether the site holds real data. Real data, or not being sure, means nothing but `GET` requests leave the page; you are never asked to pick a mode. Any flag skips the questions. See [Plain questions instead of flags](docs/guide/Ways-to-use-it.md#plain-questions-instead-of-flags).
+**No flags at all** (`/scenescout` on its own) and the agent asks four plain questions instead: the address, whether and how you sign in, what to check (tickets or a description), and whether the site holds real data. Where your client can show a form, `scout_intake` asks them as one form, never for a password; otherwise the agent asks in chat. Real data, or not being sure, means nothing but `GET` requests leave the page; you are never asked to pick a mode. Any flag skips the questions. See [Plain questions instead of flags](docs/guide/Ways-to-use-it.md#plain-questions-instead-of-flags).
 
 ### 🔑 Signing in as a role
 
@@ -329,11 +331,11 @@ Snapshots are cheap: re-snapshotting a route returns only *what changed*, with s
 
 ## 🧰 The toolbox
 
-34 deterministic tools. The agent picks; you rarely call these by hand.
+35 deterministic tools. The agent picks; you rarely call these by hand.
 
 | Phase | Tools | What they do |
 |---|---|---|
-| **Set up** | `scout_playbook` `scout_scan` `scout_attach` `scout_session` | Hand the testing method to an agent that has no skill loaded; discover routes; launch a browser in a write-mode; keep several authenticated roles alive at once |
+| **Set up** | `scout_playbook` `scout_intake` `scout_scan` `scout_attach` `scout_session` | Hand the testing method to an agent that has no skill loaded; ask the start-of-run questions as one form where the client can show one; discover routes; launch a browser in a write-mode; keep several authenticated roles alive at once |
 | **Explore** | `scout_crawl` `scout_coverage` | Sweep every route in one call; ask what's still untested |
 | **Look** | `scout_snapshot` `scout_hover` `scout_screenshot` `scout_capture` | Read the structured scene (diffed); reveal tooltips/hover cards; capture pixels only when needed; save one element as a PNG to show someone |
 | **Ask the server** | `scout_request` `scout_network` | Call the app's own API as this session, with the UI bypassed — the check that turns a hidden button into a proven refusal; list the requests the page itself made since it loaded |
@@ -457,7 +459,7 @@ The defaults are what an unconfigured check does, for a first try or an AI agent
 
 `scenescout check --help` lists every option. Why the defaults are what they are: [ADR 11](docs/adr/0011-a-gate-is-deterministic-and-fails-only-on-what-it-can-prove.md).
 
-It also replays the flows saved in `.scenescout/flows/*.json`, with no model: the steps `scout_run_plan` takes (navigate, click, type, select, press) plus `expect-text`, `expect-url` and `expect-request`. A flow whose step breaks fails the gate, naming the flow and the step. And it re-tests the open findings earlier runs left in the project's memory that a page load can reproduce, reporting each as still reproducing or possibly fixed; by default a finding filed high that still reproduces fails the gate. [docs/ci.md](docs/ci.md#saved-flows) has the flow format; [ADR 12](docs/adr/0012-a-check-replays-saved-flows-and-reports-re-tests.md) says why it works this way.
+It also replays the flows saved in `.scenescout/flows/*.json`, with no model: the steps `scout_run_plan` takes (navigate, click, type, select, press, and upload, which attaches a generated file) plus `expect-text`, `expect-element` (a control is visible, hidden, enabled, disabled, checked or unchecked), `expect-url` and `expect-request`, and `repeat` to run actions until a condition holds (page through a document until Continue is enabled). A flow can name the `role` it runs as, signed in with a profile `scenescout login --role` saved, so a journey that passes between people (one submits, another approves) is a sequence of flows. Values can come from the environment, `${env:NAME}`, so a code or a password stays in the CI's secret store and is masked in everything the check writes. A flow whose step breaks fails the gate, naming the flow and the step. And it re-tests the open findings earlier runs left in the project's memory that a page load can reproduce, reporting each as still reproducing or possibly fixed; by default a finding filed high that still reproduces fails the gate. [docs/ci.md](docs/ci.md#saved-flows) has the flow format; [ADR 12](docs/adr/0012-a-check-replays-saved-flows-and-reports-re-tests.md) says why it works this way.
 
 With `--baseline compare` it also holds pages and elements to approved pictures: list them in a `targets.json`, take the baselines once with `--baseline update`, and a later check that finds one changed past `--baseline-threshold` fails the gate with the share of pixels changed and a diff picture beside the report. Baselines are kept per browser, in a git-ignored folder unless `--baselines` names one the project commits, and only `--baseline update` ever writes one. [The guide](docs/guide/Ways-to-use-it.md#visual-baselines) has the details; [ADR 19](docs/adr/0019-a-visual-baseline-changes-only-when-asked.md) says why.
 
@@ -761,6 +763,7 @@ test-app/           fixtures for the real-browser smoke tests
 skills/scenescout/   the testing method (SKILL.md): a skill in Claude Code, served by the server everywhere else
 docs/how-it-works.md  what happens at each stage, in diagrams
 docs/benchmark.md   measuring whether a change made runs better
+docs/validation.md  scorecards from runs against public open-source apps
 docs/adr/           why it's built this way
 ```
 
@@ -774,6 +777,8 @@ docs/adr/           why it's built this way
 
 **[Measuring whether a change helped](docs/benchmark.md)** — the demo app's answer key, the scorecard (recall, precision, judged-not-filed, severity, calibration), and the results log of every run, including what did not help.
 
+**[Validation on public open-source apps](docs/validation.md)** — runs against three well-known open-source web apps, with a scorecard for each: issues by severity, how many were real and how many were false positives, and the engine problems the runs exposed.
+
 The load-bearing choices are recorded as ADRs — read the relevant one before changing a rule it covers:
 
 - [1 · Completion is an enforced contract, not a claim](docs/adr/0001-completion-is-a-contract-not-a-vibe.md)

package/dist/check-run.js

@@ -11,11 +11,13 @@ import os from "node:os";
 import path from "node:path";
 import { BASELINE_CAPTURE, baselineFiles, baselineMeta, defaultBaselinesDir, elementTarget, isVisualPicture, judgeBaseline, missingTargetsMessage, parseBaselineTargets, readStoredBaseline, TARGETS_FILE, VISUAL_DIRNAME, visualFiles, } from "./engine/baseline.js";
 import { BrowserEngine } from "./engine/browser.js";
-import { checkFindings, redactBaselineRun, MAX_DISCOVERY_ROUNDS, redactFlowRuns, redactRoute, redactRoutes, settingsOf, unmeasuredReason, withoutOwnResponse, } from "./engine/check.js";
-import { loadFlows, resolveFlowsDir } from "./engine/flow.js";
+import { checkFindings, redactBaselineRun, MAX_DISCOVERY_ROUNDS, redactFlowRuns, redactRoute, redactRoutes, settingsOf, splitResources, unmeasuredReason, withoutOwnResponse, } from "./engine/check.js";
+import { flowRoles, loadFlows, maskEnvValues, resolveFlowEnv, resolveFlowsDir } from "./engine/flow.js";
+import { resolveAttachAuth } from "./engine/profiles.js";
 import { MemoryStore, MEMORY_DIRNAME, writeSelfIgnore } from "./engine/memory.js";
 import { decodePng, encodePng } from "./engine/png.js";
 import { firstLineOf } from "./engine/limits.js";
+import { isNonPageResource } from "./engine/crawl.js";
 import { checkRetestPlan, retestResults, wellFormedFindings } from "./engine/verify.js";
 /** The baselines folder a check uses: the one --baselines names, else the project's own. */
 export function baselinesDirOf(options) {
@@ -80,7 +82,23 @@ export async function runCheck(options, log = () => { }, inputs = { flows: [], f
     // Also on an exit the finally below never reaches, such as Ctrl+C, which the browser's driver answers with process.exit.
     const removeScratch = () => fs.rmSync(scratch, { recursive: true, force: true });
     process.once("exit", removeScratch);
+    // A flow that names a role whose sign-in is not saved would only fail after the crawl: refuse before anything starts,
+    // naming the command that saves it.
+    for (const role of flowRoles(inputs.flows))
+        resolveAttachAuth({ projectDir: options.projectDir, url: options.url, role });
+    // ${env:NAME} in a flow's values, from the environment; an unset name stops the check before anything starts.
+    const envValues = new Map();
+    const runFlows = inputs.flows.map((flow) => {
+        const resolved = resolveFlowEnv(flow, process.env);
+        if (!resolved.ok)
+            throw new Error(`flow "${flow.name}" (${flow.file}) takes ${resolved.missing.map((n) => `\${env:${n}}`).join(", ")} from the environment, which is not set`);
+        for (const [name, value] of resolved.values)
+            envValues.set(name, value);
+        return resolved.flow;
+    });
     const engine = new BrowserEngine();
+    /** One browser per role a flow runs as, each attached once, on first use, and closed with the check's own. */
+    const roleEngines = new Map();
     const start = new URL(options.url);
     // From the start of the run, browser launch included: the budget is wall-clock time a person waits.
     const deadline = options.timeBudgetMs !== undefined ? Date.now() + options.timeBudgetMs : undefined;
@@ -107,6 +125,8 @@ export async function runCheck(options, log = () => { }, inputs = { flows: [], f
         if (authFailed)
             throw new Error(authFailed.replace(/ Continuing now tests a logged-out app\.$/, "").replace(/re-attach/, "run the check again"));
         const routes = [];
+        // A feed or a file the crawl followed is not a page, so it does not use up --max-routes.
+        const pagesSoFar = () => routes.filter((r) => !isNonPageResource(r)).length;
         const crawl = async (paths, limit, deadline) => {
             await engine.crawl(paths, { inspect: true, limit, deadline });
             routes.push(...engine.lastCrawlHealth);
@@ -118,15 +138,15 @@ export async function runCheck(options, log = () => { }, inputs = { flows: [], f
             // The start page first, whatever else is known: it is the one route the user named. The
             // time budget does not apply to it, so a run always measures at least the page it was given.
             await crawl([`${start.pathname}${start.search}${start.hash}`], 1);
-            for (let round = 0; round < MAX_DISCOVERY_ROUNDS && routes.length < options.maxRoutes; round++) {
+            for (let round = 0; round < MAX_DISCOVERY_ROUNDS && pagesSoFar() < options.maxRoutes; round++) {
                 if (engine.crawlableRoutes().length === 0 || pastDeadline())
                     break;
-                await crawl(undefined, options.maxRoutes - routes.length, deadline);
-                log(`  ${routes.length} route(s) checked`);
+                await crawl(undefined, options.maxRoutes - pagesSoFar(), deadline);
+                log(`  ${pagesSoFar()} route(s) checked`);
             }
         }
         // Out of time with routes still to visit; a cap of routes reached first is --max-routes's to report.
-        const timeLimitReached = pastDeadline() && routes.length < options.maxRoutes && engine.crawlableRoutes().length > 0;
+        const timeLimitReached = pastDeadline() && pagesSoFar() < options.maxRoutes && engine.crawlableRoutes().length > 0;
         // Pages of open findings the crawl did not load exactly: loaded now, so each re-test has its own measurement.
         // Kept apart from `routes`: they are measured only to re-test, never checked against the page rules, and do not count
         // towards --max-routes. With --paths the check stays on the paths it was given.
@@ -144,15 +164,45 @@ export async function runCheck(options, log = () => { }, inputs = { flows: [], f
         // reached only sign-in pages: the check has no verdict then, and an update would write sign-in pages as baselines.
         if (options.baseline !== "off" && !inputs.baselineTargets)
             throw new Error(`--baseline ${options.baseline} was given no targets: read them with readCheckInputs`);
-        const baselines = options.baseline !== "off" && inputs.baselineTargets && unmeasuredReason(routes, !options.paths) === null
+        const baselines = options.baseline !== "off" && inputs.baselineTargets && unmeasuredReason(routes, !options.paths, options.paths ? [] : engine.crawlableRoutes()) === null
             ? await takeBaselines(engine, options, options.baseline, inputs.baselineTargets, log)
             : null;
         const flowRuns = [];
-        for (const flow of inputs.flows) {
+        // A flow that names a role runs in that role's own browser, so the check's session (and what its crawl found) is
+        // left as it was.
+        const engineFor = async (role) => {
+            if (role === undefined)
+                return engine;
+            const known = roleEngines.get(role);
+            if (known)
+                return known;
+            const roleEngine = new BrowserEngine();
+            roleEngines.set(role, roleEngine);
+            const roleDir = path.join(scratch, `role-${role}`);
+            fs.mkdirSync(roleDir, { recursive: true });
+            const attachedAs = await roleEngine.attach({
+                url: start.origin,
+                projectDir: options.projectDir,
+                memoryStore: new MemoryStore(roleDir),
+                mode: options.mode,
+                role,
+                ...(options.browser ? { browser: options.browser } : {}),
+                actionTimeoutMs: options.actionTimeoutMs,
+                navTimeoutMs: options.navTimeoutMs,
+                objective: `Deterministic check: replay the saved flows that run as ${role}`,
+                task: `Replaying flows as ${role}`,
+            });
+            const lost = attachedAs.split("\n").find((line) => line.startsWith("⚠ AUTH FAILED"));
+            if (lost)
+                throw new Error(`the saved sign-in for role "${role}" no longer signs in: ${lost.replace(/ Continuing now tests a logged-out app\.$/, "")}`);
+            return roleEngine;
+        };
+        for (const flow of runFlows) {
+            const runner = await engineFor(flow.role);
             // --flow-writes never: observe's rule, whatever --mode lets the crawl do.
-            const replay = await engine.replayFlow(flow.steps, options.flowWrites === "never" ? "observe" : options.mode);
-            flowRuns.push({ name: flow.name, file: flow.file, steps: flow.steps.length, ...replay });
-            log(`  flow ${flow.name}: ${replay.outcome.status}${replay.outcome.status === "passed" ? "" : ` at step ${replay.outcome.step}`}`);
+            const replay = await runner.replayFlow(flow.steps, options.flowWrites === "never" ? "observe" : options.mode);
+            flowRuns.push({ name: flow.name, file: flow.file, steps: flow.steps.length, ...(flow.role === undefined ? {} : { role: flow.role }), ...replay });
+            log(`  flow ${flow.name}${flow.role === undefined ? "" : ` (as ${flow.role})`}: ${replay.outcome.status}${replay.outcome.status === "passed" ? "" : ` at step ${replay.outcome.step}`}`);
             // --on-refused-step stop: nothing after a refused step runs, and the check ends without a verdict.
             if (replay.outcome.status === "refused" && options.onRefusedStep === "stop")
                 break;
@@ -172,18 +222,22 @@ export async function runCheck(options, log = () => { }, inputs = { flows: [], f
                 }))),
             }
             : null;
-        const measured = redactRoutes(routes.map(withoutOwnResponse));
+        const { pages: measured, resources } = splitResources(redactRoutes(routes.map(withoutOwnResponse)));
         const flows = redactFlowRuns(flowRuns);
         const pictured = baselines ? redactBaselineRun(baselines) : null;
-        const { issues, worthALook } = checkFindings(measured, start.origin, options.ignore, flows, pictured, options.ignorePaths);
-        return {
+        // Signed in only when given a session: with none, a route that sends the browser to sign-in needs one, and is not a lost session.
+        const signedIn = options.storageStatePath !== undefined;
+        const { issues, worthALook, needsSignIn } = checkFindings(measured, start.origin, options.ignore, flows, pictured, options.ignorePaths, signedIn);
+        const result = {
             url: redactRoute(options.url),
             generatedAt: new Date().toISOString(),
             mode: options.mode,
             failOn: options.failOn,
             routes: measured,
+            resources,
             issues,
             worthALook,
+            ...(signedIn ? {} : { needsSignIn }),
             // Routes that failed to load are issues already; "not visited" is only what --max-routes (or the time budget) left out.
             unvisited: options.paths ? [] : engine.crawlableRoutes().map(redactRoute),
             ...(options.timeBudgetMs !== undefined ? { timeBudget: { ms: options.timeBudgetMs, reached: timeLimitReached } } : {}),
@@ -195,8 +249,12 @@ export async function runCheck(options, log = () => { }, inputs = { flows: [], f
             settings: settingsOf(options),
             baselines: pictured,
         };
+        // A value taken from the environment (a code, a password) is never written, wherever the page echoed it.
+        return envValues.size > 0 ? JSON.parse(maskEnvValues(JSON.stringify(result), envValues)) : result;
     }
     finally {
+        for (const [role, roleEngine] of roleEngines)
+            await roleEngine.close().catch((err) => log(`closing the browser for role ${role} failed: ${err instanceof Error ? err.message : String(err)}`));
         await engine.close().catch((err) => log(`closing the browser failed: ${err instanceof Error ? err.message : String(err)}`));
         process.off("exit", removeScratch);
         removeScratch();
@@ -308,7 +366,12 @@ async function takeBaselines(engine, options, mode, targets, log) {
                 stored,
                 now: { capture, platform: process.platform, image },
             });
-            const result = { ...own, ...verdict, ...(shot.cut ? { partial: shot.cut } : {}) };
+            const result = {
+                ...own,
+                ...verdict,
+                ...(shot.cut ? { partial: shot.cut } : {}),
+                ...(shot.unsteady ? { unsteady: shot.unsteady } : {}),
+            };
             if (verdict.status === "updated") {
                 fs.mkdirSync(path.dirname(at(dir, files.png)), { recursive: true });
                 fs.writeFileSync(at(dir, files.png), shot.png);

package/dist/cli.js

@@ -162,8 +162,9 @@ Usage:
                                     from SCENESCOUT_LOGIN_<FLAG>, e.g. SCENESCOUT_LOGIN_SUCCESS_URL;
                                     --timeout seconds (default 60))
   scenescout export --to github|jira
-                                    File the project's open findings (from .scenescout/memory.json) as issues,
-                                    each once: a finding whose marker is already on an issue is skipped. A dry run
+                                    File the project's open findings (from .scenescout/memory.json, or from a
+                                    check.json or ci.json given with --from file) as issues, each once: a finding
+                                    whose marker is already on an issue is skipped. A dry run
                                     that lists what it would file unless --yes is given. Credentials come from the
                                     environment only: GH_TOKEN or GITHUB_TOKEN; JIRA_EMAIL and JIRA_API_TOKEN.
                                     (--repo owner/name for GitHub (GITHUB_API_URL for GitHub Enterprise Server);
@@ -181,7 +182,8 @@ Usage:
                                       in Jira (default severity: high… / High, Medium, Low); --labels a,b;
                                      --screenshots on|off (default on: the finding's picture and the run's frames,
                                       attached in Jira, named on GitHub);
-                                     --project dir (default: here); --dry-run; --yes)
+                                     --project dir (default: here; its .scenescout folder keeps the record of
+                                      filed issues); --from check.json|ci.json; --dry-run; --yes)
                                     Exit code: 0 done (findings over the cap wait for the next export), 2 could not
                                     export (it lists what it filed before it stopped), or a screenshot was not
                                     attached or a ticket not linked.
@@ -587,7 +589,7 @@ async function check(args) {
         console.error(`scenescout check: could not run: ${err instanceof Error ? err.message : String(err)}`);
         process.exit(EXIT.error);
     }
-    const unmeasured = unmeasuredReason(result.routes, !options.paths);
+    const unmeasured = unmeasuredReason(result.routes, !options.paths, result.unvisited);
     if (unmeasured) {
         console.error(`scenescout check: could not measure ${options.url}: ${unmeasured}`);
         process.exit(EXIT.error);

package/dist/engine/baseline.js

@@ -74,6 +74,49 @@ export const STOP_ANIMATIONS_SCRIPT = `(() => {
   }
   return true;
 })()`;
+/**
+ * Page-side source that resolves once the page has drawn a frame after this
+ * call (two animation frames: the first starts one, the second runs once it
+ * is done). A page whose frames are not running at all resolves after
+ * NEXT_FRAME_WAIT_MS instead; the browser side bounds the wait by the same
+ * time, for a page too busy to run either.
+ */
+export const NEXT_FRAME_WAIT_MS = 1000;
+export const NEXT_FRAME_SCRIPT = `new Promise((resolve) => {
+  requestAnimationFrame(() => requestAnimationFrame(() => resolve(true)));
+  setTimeout(() => resolve(false), ${NEXT_FRAME_WAIT_MS});
+})`;
+/**
+ * A picture of a page that has stopped changing: pictures are taken, a frame
+ * apart, until two in a row are the same, and that one is kept. Stopping CSS
+ * animations does not make a page still. A frame drawn before the stop took
+ * effect, or a script that draws for a moment after the load (a count-up, an
+ * entrance), each make the first picture depend on how fast the machine is,
+ * and a baseline taken on a fast run then fails on a slow one. No new
+ * picture is started once `budgetMs` has passed (the one in progress still
+ * finishes): a page that never holds still keeps its last picture, and
+ * `steady` is false so the result can say so. `take` and `nextFrame` are the
+ * browser's (BrowserEngine.captureForBaseline); the loop is table-tested.
+ */
+export async function steadyPicture(take, nextFrame, budgetMs, now = Date.now) {
+    const until = now() + budgetMs;
+    let last = await take();
+    let takes = 1;
+    for (;;) {
+        await nextFrame();
+        const png = await take();
+        takes += 1;
+        if (png.equals(last))
+            return { png, steady: true, takes };
+        last = png;
+        if (now() >= until)
+            return { png, steady: false, takes };
+    }
+}
+/** What a result says when its picture never held still (steadyPicture): a comparison of it may not repeat. */
+export function unsteadyNote(budgetMs) {
+    return `the picture was still changing after about ${budgetMs / 1000}s, so the last one taken is used: something on the page keeps moving, and a comparison of it may not repeat`;
+}
 const ELEMENT_HELP = `${PAGE_ELEMENT} or ${TARGET_HELP}`;
 /** What an element names: null for the page itself, else the target a saved flow would use. Undefined when it is neither. */
 export function elementTarget(element) {