scenescout 3.8.0 → 3.10.0

package/CHANGELOG.md

@@ -1,5 +1,28 @@
 # scenescout
 
+## 3.10.0
+
+### Minor Changes
+
+- 8901a03: `scenescout check` is now a GitHub Action: `uses: brunoboto96/SceneScout@v3.10.0` (or the release you are on) with a `url` installs SceneScout and the browser (cached between runs), runs the check, puts the report on the job summary, keeps `report.md`, `check.json` and `check.sarif` as an artifact, and can upload the SARIF to code scanning (`upload-sarif: true`, which needs `security-events: write`). Its inputs are the CLI's options by name, its outputs are the verdict and the counts by severity, and the step fails with the CLI's exit code: 1 when the gate fails, 2 with a "could not run" annotation when there is no verdict. `docs/ci.md` has a complete workflow and the same check on GitLab CI, CircleCI and plain shell. Each release also points the major tag (`v3`) at itself where the repository settings allow it.
+- 9b98a50: New `scenescout check <url>`: a deterministic sweep that needs no model, so it can gate a pull request. It visits the start page, the project's scanned routes and every same-origin link it finds, and measures each one: HTTP and page errors, layout geometry (covered, clipped and overlapping controls, blocking overlays), broken images, controls with no name, contrast, focus indicators and pages with no way out. It writes `report.md`, `check.sarif` and `check.json`, adds the report to the GitHub Actions job summary, and exits 0 on a pass, 1 when the gate fails (`--fail-on high` by default) and 2 when it could not run. It never writes to the app.
+- ccf5802: `scout_coverage` lists the forms seen this run that no session has submitted empty: a `<form>` with a native submit control (a button linked by `form="id"` included) and at least one text field, until a click on that submit control, or Enter in one of its text inputs, goes while every text field is blank. A form is known by its `id`, else its `name`, else its `action` and method, and only when it has none of those by its submit control. Forms with no text field (only checkboxes, radios or selects), search boxes, forms whose submit is disabled while they are blank, and fields with no `<form>` around them are not listed. The gap ledger is unchanged: like the dropdown options, this is a coverage prompt, not a gate on `extensive`.
+
+### Patch Changes
+
+- 4f883c7: The console line Chromium and WebKit print for a failed load ("Failed to load resource: …") now goes where its request went: when another site's frame sent the request outside the app, the line is attributed to that embed at medium severity, instead of being filed as a high-severity console error of the app. The same line for a request the app sent stays the app's, other console and page errors are still not attributed by frame, and Firefox prints no such line.
+- 8b18c82: `scout_close` no longer closes a lane of a parallel run whose report has not been accepted yet, since folding the report needs the lane's session attached. A session counts as a lane once `scout_lane_brief` or `scout_lane_report` has named it (a brief lane sharing the name of a session already live does not count), until the run's last session closes; a refused report does not count as folded. The refusal says to fold the lane with `scout_lane_report`, and `scout_close {all: true}` names every such lane, offers the other sessions to close by name, and closes nothing. Pass `force: true` to close anyway, which loses that lane's decisions. A session no lane tool has named closes as before.
+- 42f6a55: `scout_lane_report` decodes the HTML character references a relay adds when it escapes a lane's reply (`&lt;`, `&gt;`, `&amp;`, `&quot;`, `&#39;`, `&apos;` and numeric ones), once, in every string of the report, so evidence such as `-&gt;` matches the finding filed as `->` again. A reply that holds a literal `<` or `>` was not escaped on the way and is left as it is, so a lane reporting a page that double-escapes its text keeps its evidence. Length caps apply to the decoded text, so a route that only went over its cap by being escaped is no longer refused, and the fold says how many references it decoded.
+- d487207: A text field whose only label is its placeholder is no longer treated as labelled. The snapshot still shows the placeholder as the field's name and now flags it `no label: placeholder only`, the crawl counts it as unnamed, and `scenescout check` files it under a new medium rule, `placeholder-only-label`. A field named by nothing but its `name` attribute or type is flagged `no label` and filed as `unnamed-control`. A `<label>` (by `for` or wrapping the field), `aria-label`, `aria-labelledby` or `title` counts as a label. Element keys in project memory are unchanged.
+
+## 3.9.0
+
+### Minor Changes
+
+- e2cf5bb: Snapshots list the controls inside the page's frames, each marked with its frame, and every action works on them by ref. In a frame of another site, container text is masked and typed markup, fuzzing-length values, repeated-click probes and uploads are refused in every mode; ordinary clicks and typing are allowed, and the write policy still refuses the writes such a frame sends outside the app.
+- 5a48f6c: What happens inside another site's frame is attributed to it: failing requests and error responses it sent outside the app are labelled with its origin, kept at medium severity at most and grouped in their own report section (a request it sent to the app stays the app's), and its controls are counted apart from the app's coverage and gap ledger.
+- fa71eb8: `scout_attach` takes `trustedEmbeds`, a list of origins the user trusts (a provider in test mode, say): in safe-write mode only, the writes their frames send outside the app go out, provided every other site involved (each frame up to the page, and the Origin header) is trusted. Anything that is not a plain http(s) origin is refused at attach, trust is ignored in the other modes, and the report names the list.
+
 ## 3.8.0
 
 ### Minor Changes

package/README.md

@@ -344,6 +344,52 @@ A `🛡 WRITE-POLICY blocked` notice is the safety net doing its job, not an app
 
 ---
 
+## 🚦 In CI: a deterministic check
+
+An exploratory run is driven by a model, so two runs never find exactly the same things. That suits a report, but not a gate. `scenescout check` is the part that needs no model. It visits the start URL, the project's scanned routes and every same-origin link it finds, and measures each page:
+
+- HTTP and page errors
+- layout geometry (covered, clipped and overlapping controls, blocking overlays)
+- broken images
+- controls with no name, and fields whose only label is a placeholder
+- contrast and focus
+- pages with no way out
+
+```bash
+npx scenescout check http://127.0.0.1:3000 --fail-on high
+```
+
+| Exit code | Meaning |
+|---|---|
+| 0 | Passed the gate |
+| 1 | Failed it: something at the `--fail-on` severity or worse |
+| 2 | Could not run: a bad argument, an app that never answered, or only the sign-in page reached |
+
+It writes `report.md`, `check.sarif` (for code-scanning dashboards) and `check.json` to `.scenescout/check/`, and on GitHub Actions it also puts the report on the job's summary page.
+
+On GitHub Actions, this repository is also an action that installs everything and keeps the results:
+
+```yaml
+- uses: brunoboto96/SceneScout@v3.10.0
+  with:
+    url: http://127.0.0.1:3000
+```
+
+[docs/ci.md](docs/ci.md) has a complete workflow (start the app, wait for it, check it), the action's inputs and outputs, code-scanning upload, and the same check on GitLab CI, CircleCI or any shell.
+
+It never writes to the app (`--mode observe` or `read-only`), and by default it fails only on facts that mean a page is broken: a page that did not load, an uncaught exception, a 5xx, a failure shown as success. Other options:
+
+- `--fail-on medium` or `low` makes the gate stricter.
+- `--ignore <rule>` drops a rule.
+- `--paths /a,/b` checks only those pages.
+- `--storage-state <file>` checks while signed in.
+
+`scenescout check --help` lists every option. Why the defaults are what they are: [ADR 11](docs/adr/0011-a-gate-is-deterministic-and-fails-only-on-what-it-can-prove.md).
+
+The check fills no forms and follows no flows. That is the exploratory run's job, and its findings belong in a report, not a gate.
+
+---
+
 ## 🩺 Troubleshooting
 
 Run `npx -y scenescout doctor` first — it checks every setup item below (everything but the last row, which is about your app) and prints the fix.
@@ -549,7 +595,8 @@ npx -y scenescout watch <path>      # the same, live in your browser, with each
 src/
   mcp-server.ts     the 29 tools + per-session dispatch
   scan.ts           project discovery (framework, routes, auth)
-  cli.ts            scan · serve · install · doctor · status
+  cli.ts            scan · serve · install · doctor · check · status
+  check-run.ts      drives a check: attach, crawl every route, collect what was measured
   installer.ts      setup logic (skill link, MCP registration, diagnostics)
   engine/
     browser.ts      the engine class: attach, snapshot, actions, crawl, plans
@@ -571,9 +618,10 @@ src/
     design.ts       the design audit + page scoring
     memory.ts       cross-run storage + finding dedup
     report.ts       the gap ledger + report generation
+    check.ts        the check's rules, gate, report and SARIF
     replay.ts       the run as one page: steps, tasks, frames under each finding
     …               collector · dispatch · fixtures · authloss · reaper
-scripts/            the 23 test suites (smoke/ holds the real-browser ones)
+scripts/            the 24 test suites (smoke/ holds the real-browser ones)
 test-app/           fixtures for the real-browser smoke tests
 skills/scenescout/   the testing method (SKILL.md): a skill in Claude Code, served by the server everywhere else
 docs/how-it-works.md  what happens at each stage, in diagrams
@@ -603,6 +651,7 @@ The load-bearing choices are recorded as ADRs — read the relevant one before c
 - [8 · Recording is opt-in, and a recorded run is one self-contained page](docs/adr/0008-a-recorded-run-is-evidence-and-must-be-asked-for.md)
 - [9 · A refused write is answered, not dropped](docs/adr/0009-a-refused-write-is-answered-not-dropped.md)
 - [10 · A lane's confidence is checked, not trusted](docs/adr/0010-a-confidence-is-checked-not-trusted.md)
+- [11 · A gate is deterministic, and fails only on what it can prove](docs/adr/0011-a-gate-is-deterministic-and-fails-only-on-what-it-can-prove.md)
 
 ---
 
@@ -614,7 +663,7 @@ Working on SceneScout itself is the only reason to clone it:
 git clone https://github.com/brunoboto96/SceneScout.git scenescout && cd scenescout
 npm install        # installs dependencies and builds
 npm run setup      # same as `scenescout install`, but registers THIS checkout (the skill is linked, so edits are live)
-npm test           # build + 23 suites: 21 pure-logic suites (scan, oracle, policy, … bench, hygiene),
+npm test           # build + 24 suites: 22 pure-logic suites (scan, oracle, policy, … bench, hygiene),
                    #                     then smoke and mcp-check (the server over stdio), both with real browsers
 npm run bench -- --all   # re-score every archived benchmark run against the current answer key
 npm run demo       # regenerate examples/ from the demo app

package/dist/browsers.js

@@ -160,6 +160,18 @@ export function screencastSupport(engine) {
 export function focusAdvanceKey(engine, platform) {
     return engine === "webkit" && platform === "darwin" ? "Alt+Tab" : "Tab";
 }
+/**
+ * Whether the browser prints a console error of its own for a subresource that
+ * failed: "Failed to load resource: the server responded with a status of
+ * 500 …" or "Failed to load resource: net::ERR_…". Chromium and WebKit do, and
+ * give the resource's address as the message's location; Firefox prints
+ * nothing. Where it is printed, the console oracle charges the line to
+ * whoever sent the request (oracles.ts `failedLoadEchoOf`), so one failing
+ * request inside an embed is not also filed as the app's console error.
+ */
+export function echoesFailedLoads(engine) {
+    return engine !== "firefox";
+}
 /**
  * Run in every page before its own scripts: takes shared workers away.
  *

package/dist/check-run.js

@@ -0,0 +1,76 @@
+/**
+ * Runs `scenescout check`: attach, crawl every route the engine can find,
+ * measure each one, and write the verdict. The rules live in engine/check.ts;
+ * this file only drives the browser and the files.
+ */
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import { BrowserEngine } from "./engine/browser.js";
+import { issuesFromRoutes, redactRoute, redactRoutes, withoutOwnResponse } from "./engine/check.js";
+import { MemoryStore, MEMORY_DIRNAME } from "./engine/memory.js";
+/** Link discovery rounds: each crawl reveals the routes its pages link to. Past a few, a site is paginating rather than revealing. */
+const MAX_ROUNDS = 6;
+export async function runCheck(options, log = () => { }) {
+    // A throwaway memory: a check is one run, and a store shared with earlier
+    // exploratory runs would count their visits as this check's and skip those routes.
+    const scratch = fs.mkdtempSync(path.join(os.tmpdir(), "scenescout-check-"));
+    const engine = new BrowserEngine();
+    const start = new URL(options.url);
+    try {
+        // Attached at the origin: the engine joins every crawled path onto the URL
+        // it attached to, so attaching to a start page with a path would double it.
+        const attached = await engine.attach({
+            url: start.origin,
+            projectDir: options.projectDir,
+            memoryStore: new MemoryStore(scratch),
+            mode: options.mode,
+            storageStatePath: options.storageStatePath,
+            ...(options.browser ? { browser: options.browser } : {}),
+            objective: "Deterministic check: visit every route and measure it",
+            task: "Checking every route",
+        });
+        // A saved session that no longer signs in would check the sign-in page and call it the app.
+        const authFailed = attached.split("\n").find((line) => line.startsWith("⚠ AUTH FAILED"));
+        if (authFailed)
+            throw new Error(authFailed.replace(/ Continuing now tests a logged-out app\.$/, "").replace(/re-attach/, "run the check again"));
+        const routes = [];
+        const crawl = async (paths, limit) => {
+            await engine.crawl(paths, { inspect: true, limit });
+            routes.push(...engine.lastCrawlHealth);
+        };
+        if (options.paths) {
+            await crawl(options.paths.slice(0, options.maxRoutes), options.maxRoutes);
+        }
+        else {
+            // The start page first, whatever else is known: it is the one route the user named.
+            await crawl([`${start.pathname}${start.search}${start.hash}`], 1);
+            for (let round = 0; round < MAX_ROUNDS && routes.length < options.maxRoutes; round++) {
+                if (engine.crawlableRoutes().length === 0)
+                    break;
+                await crawl(undefined, options.maxRoutes - routes.length);
+                log(`  ${routes.length} route(s) checked`);
+            }
+        }
+        const measured = redactRoutes(routes.map(withoutOwnResponse));
+        return {
+            url: redactRoute(options.url),
+            generatedAt: new Date().toISOString(),
+            mode: options.mode,
+            failOn: options.failOn,
+            routes: measured,
+            issues: issuesFromRoutes(measured, start.origin, options.ignore),
+            // Routes that failed to load are issues already; "not visited" is only what --max-routes left out.
+            unvisited: options.paths ? [] : engine.crawlableRoutes().map(redactRoute),
+            ignored: options.ignore,
+        };
+    }
+    finally {
+        await engine.close().catch((err) => log(`closing the browser failed: ${err instanceof Error ? err.message : String(err)}`));
+        fs.rmSync(scratch, { recursive: true, force: true });
+    }
+}
+/** Where a check writes when not told: beside the project's other SceneScout output, which is already ignored by git. */
+export function defaultCheckDir(projectDir) {
+    return path.join(projectDir, MEMORY_DIRNAME, "check");
+}

package/dist/cli.js

@@ -6,6 +6,7 @@
  *   scenescout serve                Run the MCP server on stdio
  *   scenescout install              Install the skill, download the browser, register the MCP server
  *   scenescout doctor               Check every piece of the setup and say how to fix what is missing
+ *   scenescout check <url>          Visit every route, measure it, and pass or fail (no model involved)
  */
 import { spawnSync } from "node:child_process";
 import fs from "node:fs";
@@ -16,13 +17,20 @@ import { fileURLToPath } from "node:url";
 import { APPROX_DISK_MB, BROWSER_ENGINES, browserPresence, defaultAttachNote, defaultEngine, launchTarget, parseBrowserSelection, playwrightInstallArgs, } from "./browsers.js";
 import { CLIENT_LABELS, firstMessageHint, manualFor, parseClients, registerWithClient, vscodeBinary } from "./clients.js";
 import { CLI_NAME, diagnose, ensureCommand, findOnUserPath, installSkill, isEphemeralRoot, launchCommand, manualRegisterCommand, planCommand, registerMcp, resolveClaudeDir, spawnRunner, } from "./installer.js";
-import { LEGACY_MEMORY_DIRNAME, MEMORY_DIRNAME } from "./engine/memory.js";
+import { defaultCheckDir, runCheck } from "./check-run.js";
+import { EXIT, formatCheck, parseCheckArgs, summarise, toSarif, toSummaryJson, unmeasuredReason } from "./engine/check.js";
+import { LEGACY_MEMORY_DIRNAME, MEMORY_DIRNAME, writeSelfIgnore } from "./engine/memory.js";
 import { formatStatus, liveEngines, liveTokenFileName, localClock, LIVE_TOKEN_FILE, pidAlive, watchTarget, wholeSessions, } from "./engine/live.js";
 import { formatScan, scanProject } from "./scan.js";
 const here = path.dirname(fileURLToPath(import.meta.url));
 const packageRoot = path.resolve(here, "..");
+/** This package's version, as published. */
+function packageVersion() {
+    return JSON.parse(fs.readFileSync(path.join(packageRoot, "package.json"), "utf8")).version;
+}
 function usage(exitCode = 1) {
-    console.log(`SceneScout — AI exploratory UI testing engine (MCP)
+    // The version beside the name: an old global install is otherwise easy to mistake for this one.
+    console.log(`SceneScout ${packageVersion()} — AI exploratory UI testing engine (MCP)
 
 Usage:
   scenescout scan <projectPath>     Discover framework, routes, auth states
@@ -39,6 +47,14 @@ Usage:
   scenescout doctor                 Check the setup and print the fix for anything missing
                                     (--engine: only node, the build and the browser — for plugin
                                      installs and other MCP clients)
+  scenescout check <url>            Visit every route, measure each one, and pass or fail — no model involved,
+                                    so it can gate a pull request. Writes report.md, check.sarif and check.json.
+                                    (--fail-on high|medium|low|never (default high); --mode observe|read-only;
+                                     --max-routes N (default 50); --paths /a,/b to check only those;
+                                     --ignore rule,rule; --storage-state file to check signed in;
+                                     --project dir (default: here); --out dir (default: .scenescout/check);
+                                     --browser chromium|firefox|webkit)
+                                    Exit code: 0 passed, 1 failed the gate, 2 could not run.
   scenescout status [projectPath]   What is the engine doing right now? (every session + recent actions)
   scenescout watch [projectPath]    Open the live view in a browser: what each session is doing, a thumbnail
                                     of its page, and a live stream you can switch on per session
@@ -385,8 +401,7 @@ async function install(flags) {
     }
     else {
         const onPath = () => findOnUserPath({ names: process.platform === "win32" ? [`${CLI_NAME}.cmd`] : [CLI_NAME], pathValue: process.env.PATH ?? "" });
-        const pkg = JSON.parse(fs.readFileSync(path.join(packageRoot, "package.json"), "utf8"));
-        const done = ensureCommand(planCommand({ packageRoot, nodePath: process.execPath, version: pkg.version, resolved: onPath(), platform: process.platform }), spawnRunner);
+        const done = ensureCommand(planCommand({ packageRoot, nodePath: process.execPath, version: packageVersion(), resolved: onPath(), platform: process.platform }), spawnRunner);
         if (done.status === "present") {
             cli = CLI_NAME;
             console.log(`✓ \`${CLI_NAME}\` command already on PATH: ${done.at}`);
@@ -445,6 +460,53 @@ async function doctor(flags) {
         ? "\nAll good. Ask your agent:  Use SceneScout to test http://localhost:3000"
         : "\nAll good. In any project, run:  /scenescout   (or ask: Use SceneScout to test http://localhost:3000)");
 }
+/** `scenescout check`: exit 0 passed, 1 failed the gate, 2 could not run. */
+async function check(args) {
+    if (args.includes("--help") || args.includes("-h"))
+        usage(0);
+    const parsed = parseCheckArgs(args, process.cwd());
+    if (!parsed.ok) {
+        console.error(`scenescout check: ${parsed.error}`);
+        process.exit(EXIT.error);
+    }
+    const options = parsed.options;
+    const outDir = options.outDir ?? defaultCheckDir(options.projectDir);
+    let result;
+    try {
+        console.log(`Checking ${options.url} …`);
+        result = await runCheck(options, (line) => console.log(line));
+    }
+    catch (err) {
+        console.error(`scenescout check: could not run: ${err instanceof Error ? err.message : String(err)}`);
+        process.exit(EXIT.error);
+    }
+    const unmeasured = unmeasuredReason(result.routes, !options.paths);
+    if (unmeasured) {
+        console.error(`scenescout check: could not measure ${options.url}: ${unmeasured}`);
+        process.exit(EXIT.error);
+    }
+    const markdown = formatCheck(result);
+    try {
+        const version = packageVersion();
+        if (!options.outDir)
+            writeSelfIgnore(path.dirname(outDir));
+        fs.mkdirSync(outDir, { recursive: true });
+        fs.writeFileSync(path.join(outDir, "report.md"), markdown);
+        fs.writeFileSync(path.join(outDir, "check.sarif"), JSON.stringify(toSarif(result, version), null, 2) + "\n");
+        fs.writeFileSync(path.join(outDir, "check.json"), JSON.stringify(toSummaryJson(result, version), null, 2) + "\n");
+        // On GitHub Actions the verdict also goes on the run's summary page.
+        if (process.env.GITHUB_STEP_SUMMARY)
+            fs.appendFileSync(process.env.GITHUB_STEP_SUMMARY, markdown);
+    }
+    catch (err) {
+        // A verdict nobody can read is not a pass or a fail: exit as "could not run", never as the gate's 1.
+        console.error(`scenescout check: could not write the results to ${outDir}: ${err instanceof Error ? err.message : String(err)}`);
+        process.exit(EXIT.error);
+    }
+    console.log("\n" + markdown);
+    console.log(`Wrote report.md, check.sarif and check.json to ${outDir}`);
+    process.exit(summarise(result).passed ? EXIT.pass : EXIT.gateFailed);
+}
 const [, , command, ...args] = process.argv;
 // A CLI's failure mode should be a sentence, not a stack trace. `scan` on a
 // path that does not exist and `status` on a half-written status.json both
@@ -461,8 +523,7 @@ try {
             usage(0);
         case "--version":
         case "-v": {
-            const pkg = JSON.parse(fs.readFileSync(path.join(packageRoot, "package.json"), "utf8"));
-            console.log(pkg.version);
+            console.log(packageVersion());
             break;
         }
         case "scan": {
@@ -484,6 +545,10 @@ try {
             await doctor(args);
             break;
         }
+        case "check": {
+            await check(args);
+            break;
+        }
         case "status": {
             status(path.resolve(args[0] ?? process.cwd()));
             break;