scenescout 3.11.1 → 3.13.0

package/CHANGELOG.md

@@ -1,5 +1,36 @@
 # scenescout
 
+## 3.13.0
+
+### Minor Changes
+
+- a6bc077: Add a QA review started from a pull-request comment. An account the repository allows (by default its owners) comments `/scenescout qa` on a pull request, optionally with a preview URL and a focus, and an unattended `scenescout ci` run explores that pull request's deployed preview and posts its results as a reply, with a link to the full report. The job that holds the model's key checks out nothing and runs SceneScout from an exact release tag, so the pull request's code never runs beside the key; pull requests from forks are refused unless the repository allows them. The workflow to copy is `examples/workflows/scenescout-qa.yml`, and the new `brunoboto96/SceneScout/qa` action runs its keyless steps.
+- 7280e29: Add `scenescout ci <url>`, an exploratory run with no person present: a model reached through the Anthropic Messages API or the OpenAI Responses API drives the scout_* tools by the SceneScout method and the run ends in the ordinary report. It reports and never gates: it exits 0 when the run ran, whatever it found, and 2 when it could not run.
+  
+  - The key is read from `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` only and is redacted from everything the run prints or writes; with both set, `--provider` chooses. `--model`, `--effort` (default `low`) and `--base-url` override the defaults (`claude-sonnet-5`, `gpt-6-luna`).
+  - The run stops at the first of 40 turns, 1,500,000 tokens or 20 minutes (`--max-turns`, `--max-tokens`, `--max-minutes`), still writes the report, and says which cap ended it.
+  - It runs in `read-only` mode by default (`--mode observe|read-only|safe-write`, or `destructive` together with `--allow-destructive`) at level `medium` (`--level`).
+  - It writes `report.md`, `report.html`, `summary.md` (also on the GitHub job summary), `ci.json` and `ci.sarif`, with a usage line of turns, tokens, time and an estimated cost. `--price-in`, `--price-cached-in` and `--price-out` set the prices the estimate uses, for any model.
+  - A second GitHub Action, `brunoboto96/SceneScout/ci`, runs it; its inputs are the command's options.
+
+### Patch Changes
+
+- d392192: `scenescout check` no longer lets a write out under the crawl's rule when a saved flow's page sent it under the flow's: a beacon a page sent as the flow left it, heard of a moment after the hand-back, could reach the server under `--flow-writes never` about one run in five in Chromium, and a page a step had opened a tab from stayed open and sent its leaving beacon when the session closed. Every page in the context is now left before the crawl's rule comes back, and the flow's rule keeps judging for five seconds after.
+
+## 3.12.0
+
+### Minor Changes
+
+- 967e98c: Add a "worth a look" tier for observations that are defects only under a convention of the project the run cannot see, such as a spacing scale, link styling in navigation, or test ids on every control. SceneScout reports each one with the convention that would decide it, and never counts it as a defect.
+  
+  - Lane reports accept the verdict `worth_a_look`, which must name its `convention`. A `convention` on any other verdict is ignored, and the fold says so. Lane calibration and the benchmark's key calibration leave it unscored and count it under its own reason.
+  - `scout_finding` takes an optional `convention`, which files the finding in this tier. The report lists these findings under "Worth a look", below the findings, as "a defect only if your project uses …", and leaves them out of every defect total. Filing the same thing again as a defect promotes it, at the defect's severity, and the reply says so. The benchmark sets such findings aside from recall and precision, and the fold lists a lane's judged defect as unfiled when it was filed only as worth a look.
+  - `scenescout check` reports two new rules, `off-grid-spacing` and `indistinct-link`, in this tier. They have no severity and never fail the gate at any `--fail-on`. SARIF reports them at level `note`. The report and the job summary list them in their own section, `check.json` puts them under `worthALook`, separate from `issues`, and the GitHub Action publishes their number as the `worth-a-look` output. Every existing rule is unchanged.
+
+### Patch Changes
+
+- c40892f: `scout_lane_report` keeps the routes each lane's accepted report lists in the project's memory (`laneRoutes` in `.scenescout/memory.json`), with query strings and fragments removed so no token in an address is stored, and `npm run bench -- --archive` carries them into the run archive. The benchmark then decides whether a lane's "not a defect" is a remark about another lane's page by where the matched defect is, not by how the verdict is worded: a defect none of whose pages the lane covered is set aside as another lane's and listed on the scorecard, and one on the lane's own page, or one every lane can reach, is scored. Answer-key entries can name further pages with `alsoOn` and mark a defect every lane can reach with `everyPage`. Archives made before routes were kept, and lanes with a route that names no page, are scored by wording, as before.
+
 ## 3.11.1
 
 ### Patch Changes

package/README.md

@@ -332,6 +332,7 @@ A `🛡 WRITE-POLICY blocked` notice is the safety net doing its job, not an app
 `.scenescout/report.md` — a deduplicated, worst-first report with:
 
 - 🐛 **Findings** with repro traces and generated Playwright regression-test skeletons.
+- 🔎 **Worth a look** — observations that are defects only under a convention of your project the run cannot see (a spacing scale, link styling in navigation, test ids on every control), each naming that convention. Listed below the findings and not counted as defects ([ADR 13](docs/adr/0013-a-convention-is-the-projects-to-decide.md)).
 - 💯 **Page scores** (0–100: a11y · craft · consistency · task-clarity), ranked worst-first, with stale scores from old runs marked as such.
 - 👥 **A role capability matrix** — what each role could and couldn't reach.
 - 🧾 **A gap ledger** — everything *not* done, so the report is honest about its own coverage.
@@ -355,6 +356,8 @@ An exploratory run is driven by a model, so two runs never find exactly the same
 - contrast and focus
 - pages with no way out
 
+Some of what it measures is a defect only under a convention the check cannot see: paddings off a 4px grid, and links styled like body text. Those are listed under **Worth a look**, each with the convention that would make it a defect. They are never counted and never fail the gate, at any `--fail-on`; SARIF reports them at level `note` ([ADR 13](docs/adr/0013-a-convention-is-the-projects-to-decide.md)).
+
 ```bash
 npx scenescout check http://127.0.0.1:3000 --fail-on high
 ```
@@ -380,7 +383,7 @@ On GitHub Actions, this repository is also an action that installs everything an
 With the default settings its saved flows send no HTTP write (they replay under observe's rule), and its crawl runs under `--mode observe` or `read-only`; `--flow-writes allow` lets flows write as `--mode` allows. By default it fails only on facts that mean a page is broken: a page that did not load, an uncaught exception, a 5xx, a failure shown as success. Other options:
 
 - `--fail-on medium` or `low` makes the gate stricter.
-- `--ignore <rule>` drops a rule.
+- `--ignore <rule>` drops a rule, worth-a-look rules included.
 - `--paths /a,/b` checks only those pages.
 - `--storage-state <file>` checks while signed in.
 - `--flows <dir>` or `off` chooses which saved flows to replay; `--retest off` skips re-testing open findings.
@@ -396,6 +399,25 @@ It also replays the flows saved in `.scenescout/flows/*.json`, with no model: th
 
 Beyond those flows it explores nothing and fills no forms. That is the exploratory run's job, and its findings belong in a report, not a gate.
 
+## 🤖 In CI: an unattended exploratory run
+
+`scenescout ci` runs the exploratory side in a CI job, with no person and no coding agent. A model reached through its API drives the same `scout_*` tools by the same method, and the run ends in the ordinary report:
+
+```bash
+export OPENAI_API_KEY=…          # or ANTHROPIC_API_KEY; read from the environment only
+npx scenescout ci http://127.0.0.1:3000
+```
+
+- **It reports and never gates.** Exit 0 when the run ran, whatever it found; exit 2 when it could not run (no key, a key the API refused, an app that never answered). Two runs of the same app find different things, so a finding is something to read, never a reason to fail a build. `scenescout check` is the gate.
+- **Providers:** the Anthropic Messages API (default model `claude-sonnet-5`) or the OpenAI Responses API (default `gpt-6-luna`), chosen by which key is set; with both set, `--provider` decides. `--model` and `--effort` (default `low`) override; `--base-url` points at another endpoint that implements the same API.
+- **Caps:** at most 40 model turns, 1,500,000 tokens and 20 minutes (`--max-turns`, `--max-tokens`, `--max-minutes`). The first cap reached ends the exploration; the report is still written, and says which cap ended it.
+- **Mode:** `read-only` by default; `--mode observe` sends no form at all, `--mode safe-write` lets the run create records and change only the ones it created. `--mode destructive` runs only with `--allow-destructive` as well.
+- **Output**, in `.scenescout/ci/` (or `--out`): `report.md` and `report.html` (the report an agent's run writes), `summary.md` (also appended to the GitHub job summary), `ci.json` and `ci.sarif`, with a usage line: turns, tokens, time and an estimated cost where the model's price is known (`--price-in`, `--price-out` give one for any model).
+
+There is a GitHub Action for it (`uses: brunoboto96/SceneScout/ci@…`). [docs/ci.md](docs/ci.md#an-unattended-exploratory-run) has the workflow and every option; [ADR 14](docs/adr/0014-an-unattended-run-reports-and-never-gates.md) says why it works this way.
+
+On a pull request, an allowed account can comment `/scenescout qa` to run it against that pull request's deployed preview and get the results as a reply. The job that holds the key checks out nothing and runs SceneScout from an exact release tag, so the pull request's code never runs beside the key. [docs/ci.md](docs/ci.md#a-qa-review-from-a-pull-request-comment) has the workflow to copy and what a project configures; [ADR 15](docs/adr/0015-a-qa-comment-tests-a-preview-and-never-runs-the-pull-requests-code.md) says why.
+
 ---
 
 ## 🩺 Troubleshooting
@@ -603,8 +625,9 @@ npx -y scenescout watch <path>      # the same, live in your browser, with each
 src/
   mcp-server.ts     the 29 tools + per-session dispatch
   scan.ts           project discovery (framework, routes, auth)
-  cli.ts            scan · serve · install · doctor · check · status
+  cli.ts            scan · serve · install · doctor · check · ci · status
   check-run.ts      drives a check: attach, crawl every route, collect what was measured
+  ci-run.ts         drives a CI run: the MCP server as a child, the model's API, the agent loop
   installer.ts      setup logic (skill link, MCP registration, diagnostics)
   engine/
     browser.ts      the engine class: attach, snapshot, actions, crawl, plans
@@ -627,9 +650,11 @@ src/
     memory.ts       cross-run storage + finding dedup
     report.ts       the gap ledger + report generation
     check.ts        the check's rules, gate, report and SARIF
+    ci.ts           a CI run's options, provider choice, caps, key redaction, tools and files
+    provider.ts     the Anthropic and OpenAI message shapes, and retries
     replay.ts       the run as one page: steps, tasks, frames under each finding
     …               collector · dispatch · fixtures · authloss · reaper
-scripts/            the 24 test suites (smoke/ holds the real-browser ones)
+scripts/            the test suites (smoke/ holds the real-browser ones)
 test-app/           fixtures for the real-browser smoke tests
 skills/scenescout/   the testing method (SKILL.md): a skill in Claude Code, served by the server everywhere else
 docs/how-it-works.md  what happens at each stage, in diagrams
@@ -661,6 +686,7 @@ The load-bearing choices are recorded as ADRs — read the relevant one before c
 - [10 · A lane's confidence is checked, not trusted](docs/adr/0010-a-confidence-is-checked-not-trusted.md)
 - [11 · A gate is deterministic, and fails only on what it can prove](docs/adr/0011-a-gate-is-deterministic-and-fails-only-on-what-it-can-prove.md)
 - [12 · A check replays saved flows and re-tests open findings, within settings whose defaults do the least harm](docs/adr/0012-a-check-replays-saved-flows-and-reports-re-tests.md)
+- [13 · What depends on a project's convention is the project's to decide](docs/adr/0013-a-convention-is-the-projects-to-decide.md)
 
 ---
 

package/dist/check-run.js

@@ -9,7 +9,7 @@ import fs from "node:fs";
 import os from "node:os";
 import path from "node:path";
 import { BrowserEngine } from "./engine/browser.js";
-import { issuesFromRoutes, redactFlowRuns, redactRoute, redactRoutes, settingsOf, withoutOwnResponse, } from "./engine/check.js";
+import { checkFindings, redactFlowRuns, redactRoute, redactRoutes, settingsOf, withoutOwnResponse, } from "./engine/check.js";
 import { loadFlows, resolveFlowsDir } from "./engine/flow.js";
 import { MemoryStore, MEMORY_DIRNAME } from "./engine/memory.js";
 import { checkRetestPlan, retestResults, wellFormedFindings } from "./engine/verify.js";
@@ -123,13 +123,15 @@ export async function runCheck(options, log = () => { }, inputs = { flows: [], f
             : null;
         const measured = redactRoutes(routes.map(withoutOwnResponse));
         const flows = redactFlowRuns(flowRuns);
+        const { issues, worthALook } = checkFindings(measured, start.origin, options.ignore, flows);
         return {
             url: redactRoute(options.url),
             generatedAt: new Date().toISOString(),
             mode: options.mode,
             failOn: options.failOn,
             routes: measured,
-            issues: issuesFromRoutes(measured, start.origin, options.ignore, flows),
+            issues,
+            worthALook,
             // Routes that failed to load are issues already; "not visited" is only what --max-routes left out.
             unvisited: options.paths ? [] : engine.crawlableRoutes().map(redactRoute),
             ignored: options.ignore,

package/dist/ci-run.js

@@ -0,0 +1,366 @@
+/**
+ * Runs `scenescout ci`: starts the SceneScout MCP server as a child process,
+ * attaches it to the app, lets a model drive the scout_* tools until it is
+ * done or a cap ends the run, then has the report written and writes the CI
+ * files. The rules (options, caps, redaction, which tools, the files) are in
+ * engine/ci.ts and the message shapes in engine/provider.ts; this file only
+ * moves bytes between the model, the server and the disk.
+ *
+ * The server runs in its own process, over the same stdio protocol every
+ * coding agent uses, rather than in this one: the model then gets exactly the
+ * tools, schemas, watchdog and write policy an agent gets, and the server,
+ * which holds module-wide state, needs no second way to start. Its
+ * environment has no API key in it (childEnv).
+ */
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { Client } from "@modelcontextprotocol/sdk/client/index.js";
+import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
+import { addUsage, wallLeftMs, guardToolArgs, capReached, childEnv, CI_DIRNAME, ciExitCode, ciKickoff, ciSarif, ciSummaryJson, ciSummaryMarkdown, ciSystemPrompt, ciToolArgs, ciTools, describeStop, findingsThisRun, NO_USAGE, readFindings, redactKeys, toolResultText, usageLine, } from "./engine/ci.js";
+import { MEMORY_DIRNAME, writeSelfIgnore } from "./engine/memory.js";
+import { AnthropicConversation, backoffMs, errorMessage, MalformedReply, OpenAIConversation, retryable, retryAfterMs, } from "./engine/provider.js";
+import { loadPlaybook } from "./playbook.js";
+const here = path.dirname(fileURLToPath(import.meta.url));
+/** From dist/ or, under tsx, from src/: the package root is one up either way. */
+const packageRoot = path.resolve(here, "..");
+const serverPath = path.join(packageRoot, "dist", "mcp-server.js");
+/** The longest one attempt at a model call may take. An attempt that runs longer, with wall time left, is retried like a server error. */
+export const MODEL_CALL_MS = 180_000;
+/** Retries of one model call after its first attempt. */
+export const MAX_RETRIES = 3;
+/**
+ * The whole of what happens after the exploration ends: writing the report
+ * (a second, forced attempt when the level's contract is unmet) and closing
+ * the browser share this one budget, so a run ends at most this long after
+ * its time cap, plus the few seconds the files take.
+ */
+export const FINISH_MS = 180_000;
+/** Attaching before the exploration, which counts towards the time cap. */
+const ATTACH_MS = 120_000;
+/** Tool calls run from one model reply; any beyond are answered as not run. Bounds what one turn can spend. Set here only. */
+export const MAX_TOOL_CALLS_PER_TURN = 16;
+/** A model call failed for good: after its retries, or on a status no retry can change (a refused key, a bad request, a redirect). */
+export class ProviderError extends Error {
+}
+/** The time cap arrived during a model call, its retries or the wait between them. */
+export class OutOfTime extends Error {
+}
+const attempts = (n) => `${n} attempt${n === 1 ? "" : "s"}`;
+/** Whether a thrown fetch error is a refused redirect (undici reports it as a TypeError whose cause says so). */
+function isRedirectError(err) {
+    const cause = err instanceof Error ? err.cause : undefined;
+    return [err, cause].some((e) => e instanceof Error && /redirect/i.test(e.message));
+}
+/** The real client: one conversation, one key, bounded retries with backoff and jitter. */
+export class HttpModelClient {
+    conversation;
+    key;
+    deps;
+    constructor(conversation, key, deps = {}) {
+        this.conversation = conversation;
+        this.key = key;
+        this.deps = deps;
+    }
+    async next(wallLeftMs) {
+        const now = this.deps.now ?? Date.now;
+        const sleep = this.deps.sleep ?? ((ms) => new Promise((r) => setTimeout(r, ms)));
+        const attemptMs = this.deps.attemptMs ?? MODEL_CALL_MS;
+        const wallDeadline = now() + wallLeftMs;
+        let last = "";
+        let made = 0;
+        for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
+            const left = wallDeadline - now();
+            if (left <= 0)
+                throw new OutOfTime(`the time cap was reached${made > 0 ? ` after ${attempts(made)} (${last})` : ""}`);
+            const req = this.conversation.request(this.key);
+            let status = 0;
+            let body = "";
+            let retryAfter;
+            made += 1;
+            try {
+                const res = await (this.deps.fetch ?? fetch)(req.url, {
+                    method: "POST",
+                    headers: req.headers,
+                    body: JSON.stringify(req.body),
+                    // Never followed: a redirect would carry the request, and a key sent
+                    // as x-api-key (which fetch does not strip across origins) with it.
+                    redirect: "error",
+                    signal: AbortSignal.timeout(Math.min(attemptMs, left)),
+                });
+                status = res.status;
+                body = await res.text();
+                retryAfter = retryAfterMs(res.headers.get("retry-after"), now());
+            }
+            catch (err) {
+                if (isRedirectError(err))
+                    throw new ProviderError("the API answered with a redirect, which is not followed");
+                const timedOut = err instanceof Error && (err.name === "TimeoutError" || err.name === "AbortError");
+                if (timedOut && wallDeadline - now() <= 0)
+                    throw new OutOfTime(`the time cap was reached during a model call (${attempts(made)})`);
+                last = timedOut
+                    ? `the model call took longer than ${Math.round(attemptMs / 1000)}s`
+                    : `the request failed (${err instanceof Error ? err.message : String(err)})`;
+            }
+            if (status >= 300 && status < 400)
+                throw new ProviderError(`HTTP ${status}: the API answered with a redirect, which is not followed`);
+            if (status >= 200 && status < 300) {
+                try {
+                    return this.conversation.accept(JSON.parse(body));
+                }
+                catch (err) {
+                    // A reply that is not what the API promises is retried like a server error, never run.
+                    last = err instanceof MalformedReply ? `a malformed reply: ${err.message}` : "a reply that is not JSON";
+                }
+            }
+            else if (status !== 0) {
+                last = errorMessage(status, body);
+                if (!retryable(status))
+                    throw new ProviderError(last);
+            }
+            if (attempt < MAX_RETRIES) {
+                const wait = backoffMs(attempt + 1, this.deps.random ?? Math.random, { retryAfterMs: retryAfter });
+                // Waiting past the time cap would only end the run there: end it now, as a cap and not a failure.
+                if (now() + wait >= wallDeadline)
+                    throw new OutOfTime(`the time cap was reached after ${attempts(made)} (${last})`);
+                await sleep(wait);
+            }
+        }
+        throw new ProviderError(`${last} (after ${attempts(made)})`);
+    }
+    addResults(results) {
+        this.conversation.addResults(results);
+    }
+}
+export function httpClient(resolved, key, system, tools, kickoff) {
+    const o = { baseUrl: resolved.baseUrl, model: resolved.model, effort: resolved.effort, system, tools };
+    const conversation = resolved.provider === "anthropic" ? new AnthropicConversation(o, kickoff) : new OpenAIConversation(o, kickoff);
+    return new HttpModelClient(conversation, key);
+}
+async function startServer(log) {
+    if (!fs.existsSync(serverPath))
+        throw new Error(`${serverPath} is missing: run \`npm run build\` first`);
+    const transport = new StdioClientTransport({ command: process.execPath, args: [serverPath], env: childEnv(process.env), stderr: "pipe" });
+    transport.stderr?.on("data", (chunk) => {
+        for (const line of chunk.toString("utf8").split(/\r?\n/))
+            if (line.trim())
+                log(`  [server] ${line}`);
+    });
+    const client = new Client({ name: "scenescout-ci", version: "1" });
+    await client.connect(transport);
+    return {
+        tools: async () => (await client.listTools()).tools,
+        call: async (name, args, timeoutMs) => {
+            const r = (await client.callTool({ name, arguments: args }, undefined, { timeout: timeoutMs }));
+            return { text: toolResultText(r.content), isError: r.isError === true };
+        },
+        close: async () => {
+            await client.close();
+        },
+    };
+}
+function readMemoryFindings(projectDir) {
+    try {
+        return readFindings(JSON.parse(fs.readFileSync(path.join(projectDir, MEMORY_DIRNAME, "memory.json"), "utf8")));
+    }
+    catch {
+        // No memory yet (a first run), or one that does not parse: nothing earlier to tell this run's findings from.
+        return [];
+    }
+}
+/**
+ * The agent loop. Before each model call the caps are checked; the call runs
+ * with at most the time that is left; each tool call it asks for is run in
+ * order, on the one session, and none runs past the time cap: a call reached
+ * after it is answered as not run, as is any beyond MAX_TOOL_CALLS_PER_TURN.
+ * A call the loop cannot run (a tool it was not given, arguments that are not
+ * JSON, a scan of another directory) goes back to the model as an error
+ * result, so a malformed reply costs a turn, never the run.
+ */
+export async function agentLoop(o) {
+    const now = o.now ?? Date.now;
+    const spend = { turns: 0, usage: { ...NO_USAGE }, startedAt: o.startedAt ?? now() };
+    const allowed = new Set(o.tools.map((t) => t.name));
+    for (;;) {
+        const cap = capReached(spend, o.caps, now());
+        if (cap)
+            return { stop: cap, spend };
+        let turn;
+        try {
+            turn = await o.client.next(wallLeftMs(spend, o.caps, now()));
+        }
+        catch (err) {
+            if (err instanceof OutOfTime)
+                return { stop: "time", spend };
+            return { stop: "provider-error", stopDetail: err instanceof Error ? err.message : String(err), spend };
+        }
+        spend.turns += 1;
+        spend.usage = addUsage(spend.usage, turn.usage);
+        if (turn.resume)
+            continue;
+        if (turn.calls.length === 0) {
+            if (turn.text.trim())
+                o.log(`  model: ${turn.text.trim().slice(0, 600)}`);
+            return { stop: "done", ...(turn.note ? { stopDetail: turn.note } : {}), spend };
+        }
+        o.log(`  turn ${spend.turns}: ${turn.calls.map((c) => c.name || "(unnamed)").join(", ")} — ${(spend.usage.input + spend.usage.output).toLocaleString("en-US")} tokens so far`);
+        const results = [];
+        for (const [i, call] of turn.calls.entries()) {
+            if (i >= MAX_TOOL_CALLS_PER_TURN) {
+                results.push({ id: call.id, isError: true, text: `${call.name} was not run: at most ${MAX_TOOL_CALLS_PER_TURN} tool calls are run from one reply.` });
+                continue;
+            }
+            if (!allowed.has(call.name)) {
+                results.push({ id: call.id, isError: true, text: `There is no tool named "${call.name}" in this run. The tools are: ${[...allowed].join(", ")}.` });
+                continue;
+            }
+            if (call.argsError) {
+                results.push({ id: call.id, isError: true, text: `${call.name} was not run: ${call.argsError}. Send the arguments as one JSON object.` });
+                continue;
+            }
+            const args = ciToolArgs(call.input);
+            const guarded = args.ok ? guardToolArgs(call.name, args.args, o.projectDir) : args;
+            if (!guarded.ok) {
+                results.push({ id: call.id, isError: true, text: `${call.name} was not run: ${guarded.error}.` });
+                continue;
+            }
+            const left = wallLeftMs(spend, o.caps, now());
+            if (left <= 0) {
+                results.push({ id: call.id, isError: true, text: `${call.name} was not run: the time cap was reached.` });
+                continue;
+            }
+            try {
+                const r = await o.host.call(call.name, guarded.args, Math.min(600_000, left));
+                results.push({ id: call.id, isError: r.isError, text: r.text });
+            }
+            catch (err) {
+                results.push({ id: call.id, isError: true, text: `${call.name} failed: ${err instanceof Error ? err.message : String(err)}` });
+            }
+        }
+        o.client.addResults(results);
+    }
+}
+/**
+ * The whole run. `makeClient` is how the model is reached: the HTTP client in
+ * the CLI, a scripted one in the tests. Everything logged or written passes
+ * through the key redaction first.
+ */
+export async function runCi(options, resolved, deps) {
+    const secrets = deps.secrets ?? [];
+    const log = (line) => (deps.log ?? (() => { }))(redactKeys(line, secrets));
+    const now = deps.now ?? Date.now;
+    const startedAt = now();
+    const outDir = options.outDir ?? path.join(options.projectDir, MEMORY_DIRNAME, CI_DIRNAME);
+    const before = readMemoryFindings(options.projectDir);
+    let outcome = { stop: "could-not-start", spend: { turns: 0, usage: { ...NO_USAGE }, startedAt } };
+    let contractMet = false;
+    let reportWritten = false;
+    let host = null;
+    // Set when the exploration ends (or never starts): the report and the close share FINISH_MS from then.
+    let finishBy = 0;
+    const finishLeft = () => Math.max(1_000, finishBy - now());
+    try {
+        host = await startServer(log);
+        const attached = await host.call("scout_attach", {
+            url: options.url,
+            projectPath: options.projectDir,
+            mode: options.mode,
+            objective: `CI run: explore at level ${options.level}${options.focus ? `, focusing on ${options.focus}` : ""}`.slice(0, 300),
+            task: "Starting the CI run",
+            ...(options.storageStatePath ? { storageStatePath: options.storageStatePath } : {}),
+            ...(options.browser ? { browser: options.browser } : {}),
+        }, Math.min(ATTACH_MS, options.caps.wallMs));
+        const authFailed = attached.text.split("\n").find((l) => l.startsWith("⚠ AUTH FAILED"));
+        if (attached.isError || /^ERROR:/.test(attached.text) || authFailed) {
+            outcome = { ...outcome, stopDetail: (authFailed ?? attached.text).replace(/^ERROR:\s*/, "").slice(0, 400) };
+        }
+        else {
+            log(`Attached to ${options.url} in ${options.mode} mode.`);
+            const tools = ciTools(await host.tools());
+            const system = ciSystemPrompt(loadPlaybook(packageRoot), options);
+            const kickoff = ciKickoff(options);
+            outcome = await agentLoop({
+                client: deps.makeClient(system, tools, kickoff),
+                host,
+                tools,
+                caps: options.caps,
+                log,
+                now,
+                startedAt,
+                projectDir: options.projectDir,
+            });
+            log(`Run ended: ${describeStop(outcome.stop, options.caps, outcome.stopDetail)}.`);
+            finishBy = now() + FINISH_MS;
+            // The report is written whatever ended the run. A forced report still prints its gaps.
+            let report = await host.call("scout_report", { level: options.level }, finishLeft());
+            contractMet = !report.isError && !/NOT GENERATED/.test(report.text);
+            if (!contractMet && !report.isError)
+                report = await host.call("scout_report", { level: options.level, force: true }, finishLeft());
+            reportWritten = !report.isError && !/^ERROR:/.test(report.text) && fs.existsSync(path.join(options.projectDir, MEMORY_DIRNAME, "report.md"));
+            if (!reportWritten)
+                log(`The report could not be generated: ${report.text.slice(0, 400)}`);
+        }
+    }
+    catch (err) {
+        const message = err instanceof Error ? err.message : String(err);
+        if (outcome.stop === "could-not-start")
+            outcome = { ...outcome, stopDetail: message };
+        else
+            log(`The run could not finish cleanly: ${message}`);
+    }
+    finally {
+        if (host) {
+            if (finishBy === 0)
+                finishBy = now() + FINISH_MS;
+            await host
+                .call("scout_close", { all: true }, finishLeft())
+                .catch((err) => log(`Closing the browser failed: ${err instanceof Error ? err.message : String(err)}`));
+            await host.close().catch(() => { });
+        }
+    }
+    const endedAt = now();
+    const result = {
+        url: options.url,
+        provider: resolved.provider,
+        model: resolved.model,
+        effort: resolved.effort,
+        mode: options.mode,
+        level: options.level,
+        caps: options.caps,
+        ...(options.price ? { price: options.price } : {}),
+        stop: outcome.stop,
+        ...(outcome.stopDetail ? { stopDetail: outcome.stopDetail } : {}),
+        contractMet,
+        spend: outcome.spend,
+        endedAt,
+        findings: findingsThisRun(before, readMemoryFindings(options.projectDir)),
+    };
+    const written = [];
+    try {
+        if (!options.outDir)
+            writeSelfIgnore(path.dirname(outDir));
+        fs.mkdirSync(outDir, { recursive: true });
+        const write = (name, content) => {
+            fs.writeFileSync(path.join(outDir, name), redactKeys(content, secrets));
+            written.push(name);
+        };
+        if (reportWritten) {
+            write("report.md", fs.readFileSync(path.join(options.projectDir, MEMORY_DIRNAME, "report.md"), "utf8"));
+            const html = path.join(options.projectDir, MEMORY_DIRNAME, "report.html");
+            if (fs.existsSync(html))
+                write("report.html", fs.readFileSync(html, "utf8"));
+        }
+        const summary = ciSummaryMarkdown(result, secrets);
+        write("summary.md", summary);
+        write("ci.json", JSON.stringify(ciSummaryJson(result, deps.version, secrets), null, 2) + "\n");
+        write("ci.sarif", JSON.stringify(ciSarif(result, deps.version, secrets), null, 2) + "\n");
+        if (process.env.GITHUB_STEP_SUMMARY)
+            fs.appendFileSync(process.env.GITHUB_STEP_SUMMARY, redactKeys(summary, secrets));
+    }
+    catch (err) {
+        log(`Could not write the results to ${outDir}: ${err instanceof Error ? err.message : String(err)}`);
+        reportWritten = false;
+    }
+    log(`Usage: ${usageLine(result.spend, result.model, endedAt, result.price)}`);
+    return { result, exitCode: ciExitCode(result.stop, reportWritten), written };
+}

package/dist/cli.js

@@ -7,6 +7,7 @@
  *   scenescout install              Install the skill, download the browser, register the MCP server
  *   scenescout doctor               Check every piece of the setup and say how to fix what is missing
  *   scenescout check <url>          Visit every route, measure it, and pass or fail (no model involved)
+ *   scenescout ci <url>             An exploratory run driven by a model's API, unattended, that reports
  */
 import { spawnSync } from "node:child_process";
 import fs from "node:fs";
@@ -18,6 +19,8 @@ import { APPROX_DISK_MB, BROWSER_ENGINES, browserPresence, defaultAttachNote, de
 import { CLIENT_LABELS, firstMessageHint, manualFor, parseClients, registerWithClient, vscodeBinary } from "./clients.js";
 import { CLI_NAME, diagnose, ensureCommand, findOnUserPath, installSkill, isEphemeralRoot, launchCommand, manualRegisterCommand, planCommand, registerMcp, resolveClaudeDir, spawnRunner, } from "./installer.js";
 import { defaultCheckDir, readCheckInputs, runCheck } from "./check-run.js";
+import { httpClient, runCi } from "./ci-run.js";
+import { detectProvider, EXIT_CI, KEY_ENV, parseCiArgs, redactKeys, secretValues } from "./engine/ci.js";
 import { EXIT, exitCodeOf, formatCheck, parseCheckArgs, refusedFlowReason, toSarif, toSummaryJson, unmeasuredReason } from "./engine/check.js";
 import { LEGACY_MEMORY_DIRNAME, MEMORY_DIRNAME, writeSelfIgnore } from "./engine/memory.js";
 import { formatStatus, liveEngines, liveTokenFileName, localClock, LIVE_TOKEN_FILE, pidAlive, watchTarget, wholeSessions, } from "./engine/live.js";
@@ -64,6 +67,25 @@ Usage:
                                      --gate-retests never|high|all: which still-reproducing findings fail the gate
                                       (default high: those filed high))
                                     Exit code: 0 passed, 1 failed the gate, 2 could not run.
+  scenescout ci <url>               An exploratory run with no person present: a model reached through its API
+                                    drives the tools by the SceneScout method and the run ends in the report.
+                                    It reports and never gates. The key is read from ANTHROPIC_API_KEY or
+                                    OPENAI_API_KEY only. Writes report.md, summary.md, ci.json and ci.sarif.
+                                    (--provider anthropic|openai: needed only when both keys are set;
+                                     --model id (default claude-sonnet-5 / gpt-6-luna); --effort none|low|medium|
+                                      high|xhigh|max (default low; none is OpenAI only); --base-url https://…/v1 for
+                                      another endpoint that implements the same API;
+                                     --max-turns N (default 40); --max-tokens N (default 1500000);
+                                     --max-minutes N (default 20): the run stops at the first cap reached and still
+                                      writes the report;
+                                     --price-in, --price-cached-in, --price-out: US dollars per million tokens,
+                                      over the built-in prices, for the cost estimate of any model;
+                                     --mode observe|read-only|safe-write|destructive (default read-only;
+                                      destructive only with --allow-destructive as well); --level minimal|medium|
+                                      extensive (default medium); --focus "an area or flow";
+                                     --storage-state file; --browser chromium|firefox|webkit;
+                                     --project dir (default: here); --out dir (default: .scenescout/ci))
+                                    Exit code: 0 the run ran (findings never change it), 2 could not run.
   scenescout status [projectPath]   What is the engine doing right now? (every session + recent actions)
   scenescout watch [projectPath]    Open the live view in a browser: what each session is doing, a thumbnail
                                     of its page, and a live stream you can switch on per session
@@ -533,6 +555,49 @@ async function check(args) {
         console.error(`scenescout check: could not run a saved flow: ${refused}`);
     process.exit(exitCodeOf(result));
 }
+/** `scenescout ci`: exit 0 when the run ran, 2 when it could not. Findings never change the exit code. */
+async function ci(args) {
+    if (args.includes("--help") || args.includes("-h"))
+        usage(0);
+    const secrets = secretValues(process.env);
+    const say = (line) => console.log(redactKeys(line, secrets));
+    const fail = (message) => {
+        console.error(redactKeys(`scenescout ci: ${message}`, secrets));
+        process.exit(EXIT_CI.couldNotRun);
+    };
+    const parsed = parseCiArgs(args, process.cwd());
+    if (!parsed.ok)
+        return fail(parsed.error);
+    const options = parsed.options;
+    const provider = detectProvider(process.env, options);
+    if (!provider.ok)
+        return fail(provider.error);
+    const resolved = provider.resolved;
+    const key = (process.env[KEY_ENV[resolved.provider]] ?? "").trim();
+    say(`Exploring ${options.url} with ${resolved.provider} ${resolved.model} (effort ${resolved.effort}), ${options.mode} mode, level ${options.level} …`);
+    let run;
+    try {
+        run = await runCi(options, resolved, {
+            makeClient: (system, tools, kickoff) => httpClient(resolved, key, system, tools, kickoff),
+            log: say,
+            secrets,
+            version: packageVersion(),
+        });
+    }
+    catch (err) {
+        return fail(`could not run: ${err instanceof Error ? err.message : String(err)}`);
+    }
+    const { result, exitCode, written } = run;
+    if (written.length > 0)
+        say(`Wrote ${written.join(", ")} to ${options.outDir ?? path.join(options.projectDir, MEMORY_DIRNAME, "ci")}`);
+    if (exitCode !== EXIT_CI.completed) {
+        const why = result.stop === "could-not-start" || result.stop === "provider-error"
+            ? `${result.stop === "provider-error" ? "the model's API failed" : "could not start"}${result.stopDetail ? `: ${result.stopDetail}` : ""}`
+            : "the report could not be written";
+        fail(`could not run: ${why}`);
+    }
+    process.exit(exitCode);
+}
 const [, , command, ...args] = process.argv;
 // A CLI's failure mode should be a sentence, not a stack trace. `scan` on a
 // path that does not exist and `status` on a half-written status.json both
@@ -575,6 +640,10 @@ try {
             await check(args);
             break;
         }
+        case "ci": {
+            await ci(args);
+            break;
+        }
         case "status": {
             status(path.resolve(args[0] ?? process.cwd()));
             break;