scenescout 1.2.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +38 -7
- package/dist/browsers.js +11 -0
- package/dist/cli.js +144 -34
- package/dist/engine/browser.js +128 -4
- package/dist/engine/live-page.js +644 -0
- package/dist/engine/live.js +549 -0
- package/dist/engine/memory.js +3 -0
- package/dist/engine/report.js +8 -2
- package/dist/installer.js +82 -4
- package/dist/mcp-server.js +216 -26
- package/package.json +3 -2
- package/skills/scenescout/SKILL.md +2 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,18 @@
|
|
|
1
1
|
# scenescout
|
|
2
2
|
|
|
3
|
+
## 1.3.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- a7c978a: `scenescout install` now puts the `scenescout` command on your PATH. Until now neither an `npx` run nor a source checkout left it there, so `scenescout status`, `scenescout doctor` and the other commands the tool itself tells you to run answered "command not found".
|
|
8
|
+
|
|
9
|
+
Run through `npx`, install does `npm install -g` of the version you ran. From a checkout it does `npm link`, so the command always runs what you last built, and From a checkout it runs `npm link`, taking the name over from any other copy the way install already takes over the MCP registration. Run through `npx`, a command that is already there is left alone. On Windows the step prints the command to run by hand. If npm refuses, the step prints the command to run by hand and does not fail the setup. `--no-command` skips it.
|
|
10
|
+
- 195cb59: Watch a run live. `scout_attach` now returns a `Live view:` address, which the agent passes on to you, and `scenescout watch <project>` opens the same page from a terminal. It shows one card per session: the tool it is running, how long it has been there, the page it is on, a thumbnail of that page, a rolling feed of what it just did (each action, its target and how it turned out, read from the same action log a finding's repro trace uses), and a live stream you can switch on per session or for all of them. Opening a card's close-up shows a longer stretch of that feed beside the session's brief: the task the agent gave it at `scout_attach {task}`, and the goal of the journey it is on right now. Actions of one journey share a tint in the feed, and pointing at a group shows the goal those actions served. The Report button shows the run's report as it stands, rendered from the current state without writing it, so it can be read while the run is still going. It works for headless runs, and a session whose call is still running past its own tool's watchdog budget is marked as stuck.
|
|
11
|
+
|
|
12
|
+
`status.json` now describes every session instead of the last one to write, and `scenescout status` prints a line for each.
|
|
13
|
+
|
|
14
|
+
The live view is served on `127.0.0.1` only, behind a per-process token, answers GET and nothing else, and writes no frame to disk ([ADR 7](docs/adr/0007-the-live-view-is-local-read-only-and-leaves-nothing-behind.md)). A stream runs only while someone is watching it. Set `SCENESCOUT_LIVE=off` to keep the engine from opening the port. The engine now also shuts down, closing its browsers and removing the token file, when its client closes the connection instead of sending a signal.
|
|
15
|
+
|
|
3
16
|
## 1.2.0
|
|
4
17
|
|
|
5
18
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -35,7 +35,7 @@ This is a real run against the small demo app bundled in this repository. The ap
|
|
|
35
35
|
|
|
36
36
|
<p align="center"><img src="examples/screenshots/dashboard-annotated.png" alt="The demo app's dashboard with two defects outlined in red: 1, a yellow badge covering the All orders button; 2, the weekly chart image failing to load" width="760" /></p>
|
|
37
37
|
|
|
38
|
-
**The broken chart is the demo app's bug, not this page's** — it is one of the
|
|
38
|
+
**The broken chart is the demo app's bug, not this page's** — it is one of the twelve findings SceneScout filed, next to the badge sitting on a button. The red callouts were added for this README; the [unmarked screenshots](examples/screenshots/) are the ones the engine took.
|
|
39
39
|
|
|
40
40
|
An excerpt of the report it wrote — [read the whole thing](examples/report.md):
|
|
41
41
|
|
|
@@ -46,13 +46,16 @@ An excerpt of the report it wrote — [read the whole thing](examples/report.md)
|
|
|
46
46
|
> **🔴 [HIGH] Filtering orders by Archived fails, and the page shows an empty table instead of an error**
|
|
47
47
|
> Evidence: `GET /api/orders?status=archived → HTTP 500`
|
|
48
48
|
>
|
|
49
|
+
> **🔴 [HIGH] A clerk can approve an order by calling the endpoint the page hides from them**
|
|
50
|
+
> Evidence: `POST /api/orders/1037/approve 200 as clerk; POST /api/orders/1038/reject 403 as clerk` — the button was hidden, the server did not agree.
|
|
51
|
+
>
|
|
49
52
|
> **🟠 [MEDIUM] The "New: bulk import" badge sits on top of the All orders button** *(callout 1)*
|
|
50
53
|
> Evidence: `"All orders" overlaps "New: bulk import" (81%)` — measured from layout boxes, no screenshot needed.
|
|
51
54
|
>
|
|
52
55
|
> **🟡 [LOW] The dashboard chart image is missing** *(callout 2)*
|
|
53
56
|
> Evidence: `GET /img/weekly-chart.png → HTTP 404`
|
|
54
57
|
>
|
|
55
|
-
> **Gap ledger — what was NOT tested:**
|
|
58
|
+
> **Gap ledger — what was NOT tested:** 9/12 visited routes never design-audited · single-role run, so permission boundaries are untested
|
|
56
59
|
|
|
57
60
|
Every finding comes with a repro trace and a Playwright regression-test skeleton. To try it yourself, clone this repository, run `npm run demo:serve`, then `/scenescout --url http://127.0.0.1:4173` — see [demo-app/](demo-app/). Its README lists every seeded defect and which oracle catches it.
|
|
58
61
|
|
|
@@ -136,9 +139,10 @@ Then download the browser once with `npx -y scenescout install --browser-only`.
|
|
|
136
139
|
|
|
137
140
|
1. puts the `/scenescout` skill into `~/.claude/skills/` (or `$CLAUDE_CONFIG_DIR/skills/`) — a `scenescout` folder it didn't create is moved aside to a `.backup-…` copy, never deleted,
|
|
138
141
|
2. downloads the browser SceneScout drives (skipped if you already have it). By default that is Chromium, as two builds: the full browser for headed runs and the headless shell every other run uses. [Choose something else](#-choosing-browsers) with `--browsers`,
|
|
139
|
-
3. registers the MCP server with Claude Code at user scope. Run through `npx`, the launcher is `npx -y scenescout serve`, with the absolute path of `npx` where one sits beside node, so it works under nvm/fnm. From a clone or a global install it is the absolute node path plus that install's `dist/mcp-server.js
|
|
142
|
+
3. registers the MCP server with Claude Code at user scope. Run through `npx`, the launcher is `npx -y scenescout serve`, with the absolute path of `npx` where one sits beside node, so it works under nvm/fnm. From a clone or a global install it is the absolute node path plus that install's `dist/mcp-server.js`,
|
|
143
|
+
4. puts the `scenescout` command on your PATH, so `scenescout status`, `scenescout watch` and `scenescout doctor` work from any terminal. Run through `npx`, that is `npm install -g` of the version you just ran; from a clone it is `npm link`, so the command always runs what you last built. If npm refuses (a system-wide node usually needs `sudo` for this), the step prints the command to run by hand and the rest of the setup still counts as done: `npx -y scenescout <command>` works without it.
|
|
140
144
|
|
|
141
|
-
Re-run it any time: after moving the folder or switching node versions it refreshes the stored paths. It exits non-zero if
|
|
145
|
+
Re-run it any time: after moving the folder or switching node versions it refreshes the stored paths. It exits non-zero if a step the tool depends on failed, so it is safe to chain. Opt out of a step with `--no-register`, `--skip-browser` or `--no-command`.
|
|
142
146
|
|
|
143
147
|
If `claude` isn't on the PATH of the shell you ran it from, it prints the registration command instead of running it:
|
|
144
148
|
|
|
@@ -179,6 +183,32 @@ The agent scans the project (if there is one), attaches read-only, explores, and
|
|
|
179
183
|
|
|
180
184
|
---
|
|
181
185
|
|
|
186
|
+
## 📺 Watching a run live
|
|
187
|
+
|
|
188
|
+
When a session attaches, the engine starts a small live view and hands the agent its address on a `Live view:` line, which the agent passes on to you. From a terminal, `scenescout watch` opens the same page. There is one card per session:
|
|
189
|
+
|
|
190
|
+
<p align="center"><img src="examples/screenshots/live-view.png" alt="The live view during a run of six parallel agents against the demo app: one card per session, each with its role, the tool it is running and for how long, the page it is on, a thumbnail, and a feed of the actions it just took" width="880" /></p>
|
|
191
|
+
|
|
192
|
+
- **What it is doing:** the tool it is running and for how long, the page it is on, and a thumbnail of that page. This works for headless runs too, which have no window to look at.
|
|
193
|
+
- **What it just did:** a rolling feed of its actions, each with its target and how it turned out, with failures in red. It is the same trail a finding's repro trace uses. The engine never sees the agent's reasoning, so this is what the session *did*, not what it thought.
|
|
194
|
+
- **Stuck, not slow:** a call still running past its own tool's watchdog budget turns the card red, so a wedged session is visible without asking. A crawl legitimately runs for minutes; it is judged against the crawl's budget, not a click's.
|
|
195
|
+
- **Live stream:** switch it on for one card, or for all of them. Click a thumbnail for a close-up.
|
|
196
|
+
- **The report, as it stands:** the Report button in the top bar shows the same document `scout_report` writes at the end, rendered from the run's current state, so findings can be read while the agents are still working.
|
|
197
|
+
- **What it is for:** the close-up puts the feed beside the session's brief: the task the agent gave it when it attached (`scout_attach {task}`), and the goal of the journey it is on right now (`scout_journey`). Actions of one journey share a tint in the feed; point at a group and the brief shows the goal those actions served.
|
|
198
|
+
|
|
199
|
+
<p align="center"><img src="examples/screenshots/live-view-closeup.png" alt="A close-up of one session: its page streaming live, the feed of its recent actions grouped by journey, and beside it the task and current objective" width="880" /></p>
|
|
200
|
+
|
|
201
|
+
<p align="center"><img src="examples/screenshots/live-view-report.png" alt="The report opened from the live view's top bar while the run is still going: summary table, gap ledger and the findings filed so far" width="880" /></p>
|
|
202
|
+
|
|
203
|
+
The view is served on `127.0.0.1` only, behind a token that changes every time the engine starts. It answers `GET` and nothing else, so a viewer can watch a run but not act in it, and no frame is ever written to disk ([ADR 7](docs/adr/0007-the-live-view-is-local-read-only-and-leaves-nothing-behind.md)). A stream runs only while someone is watching it. `SCENESCOUT_LIVE=off` keeps the port closed.
|
|
204
|
+
|
|
205
|
+
**Try it with parallel agents.** The demo app has three roles and several separate areas, so a run can be split between agents. Start it with `npm run demo:serve`, then ask your agent to explore it with several agents in parallel, one role and one area each. The pictures above come from a run of six. Two things keep a parallel run efficient:
|
|
206
|
+
|
|
207
|
+
- **Each agent opens its own session when it starts and closes it when it is done.** An agent waiting for its turn then holds no browser. Opening every session up front leaves browsers idling while the machine runs out of memory for the agents that are working.
|
|
208
|
+
- **Run about as many agents at once as your machine has cores, less two.** Each one drives a real browser.
|
|
209
|
+
|
|
210
|
+
---
|
|
211
|
+
|
|
182
212
|
## 🔄 How a run works
|
|
183
213
|
|
|
184
214
|
One curiosity loop, repeated — breadth first, then judgment where it matters:
|
|
@@ -457,7 +487,8 @@ The CLI is also useful on its own:
|
|
|
457
487
|
|
|
458
488
|
```bash
|
|
459
489
|
npx -y scenescout scan <path> # project discovery: framework, routes, saved logins
|
|
460
|
-
npx -y scenescout status <path> # what a running engine is doing right now
|
|
490
|
+
npx -y scenescout status <path> # what every session of a running engine is doing right now
|
|
491
|
+
npx -y scenescout watch <path> # the same, live in your browser, with each session's page
|
|
461
492
|
```
|
|
462
493
|
|
|
463
494
|
---
|
|
@@ -519,7 +550,7 @@ npm test # build + 12 suites: scan, oracle, policy, fixture, dispatch,
|
|
|
519
550
|
npm run demo # regenerate examples/ from the demo app
|
|
520
551
|
```
|
|
521
552
|
|
|
522
|
-
Contributing? Start with [VISION.md](VISION.md) (what is in scope) and [CONTRIBUTING.md](CONTRIBUTING.md) (how changes land), then see [
|
|
553
|
+
Contributing? Start with [VISION.md](VISION.md) (what is in scope) and [CONTRIBUTING.md](CONTRIBUTING.md) (how changes land), then see [AGENTS.md](AGENTS.md) for the house rules — chiefly: bug fixes need a regression test at the cheapest layer that can fail, keep the repo project-agnostic (ADR 6), and `npm test` must pass.
|
|
523
554
|
|
|
524
555
|
## 🔐 Security
|
|
525
556
|
|
|
@@ -545,6 +576,6 @@ Found a way past the write policy, or another security problem? Please report it
|
|
|
545
576
|
- **A trustworthy gap ledger.** Entries must be actionable (a search box or wizard sub-step isn't "form filled but never submitted"); API/download URLs never enter the route contract.
|
|
546
577
|
- **Honest reporting.** Shared chrome counted once, stale scores marked, role matrix compares only roles that actually attempted a route.
|
|
547
578
|
- **Cross-run written knowledge.** `scout_note` curates `.scenescout/ASSUMPTIONS.md` — app model, personas, constraints, risks — in prose.
|
|
548
|
-
- **Daemon-grade robustness.** Per-tool watchdogs, orphaned-browser reaping, bounded teardown, live status via `scenescout status <project
|
|
579
|
+
- **Daemon-grade robustness.** Per-tool watchdogs, orphaned-browser reaping, bounded teardown, live status via `scenescout status <project>`, and a live view of every session's page: the agent gives you its address when it attaches, or run `scenescout watch <project>` (loopback only, read-only, nothing written to disk: [ADR 7](docs/adr/0007-the-live-view-is-local-read-only-and-leaves-nothing-behind.md)).
|
|
549
580
|
|
|
550
581
|
</details>
|
package/dist/browsers.js
CHANGED
|
@@ -140,6 +140,17 @@ export function installCommandFor(target, fromCheckout) {
|
|
|
140
140
|
export function serviceWorkerPolicy(engine) {
|
|
141
141
|
return engine === "chromium" ? "allow" : "block";
|
|
142
142
|
}
|
|
143
|
+
/**
|
|
144
|
+
* How the live view gets frames of a page.
|
|
145
|
+
*
|
|
146
|
+
* Chromium can push a frame on every repaint through the DevTools protocol,
|
|
147
|
+
* which costs nothing while the page sits still. Firefox and WebKit have no
|
|
148
|
+
* such channel in the driver, so a stream there is a screenshot taken on a
|
|
149
|
+
* timer for as long as somebody is watching.
|
|
150
|
+
*/
|
|
151
|
+
export function screencastSupport(engine) {
|
|
152
|
+
return engine === "chromium" ? "cdp" : "poll";
|
|
153
|
+
}
|
|
143
154
|
/**
|
|
144
155
|
* The key that moves keyboard focus to the next control, links and buttons
|
|
145
156
|
* included. WebKit on macOS follows Safari: plain Tab stops only at text
|
package/dist/cli.js
CHANGED
|
@@ -15,8 +15,9 @@ import path from "node:path";
|
|
|
15
15
|
import { fileURLToPath } from "node:url";
|
|
16
16
|
import { APPROX_DISK_MB, BROWSER_ENGINES, browserPresence, defaultAttachNote, defaultEngine, launchTarget, parseBrowserSelection, playwrightInstallArgs, } from "./browsers.js";
|
|
17
17
|
import { CLIENT_LABELS, firstMessageHint, manualFor, parseClients, registerWithClient, vscodeBinary } from "./clients.js";
|
|
18
|
-
import { diagnose, installSkill, launchCommand, manualRegisterCommand, registerMcp, resolveClaudeDir, spawnRunner } from "./installer.js";
|
|
18
|
+
import { CLI_NAME, diagnose, ensureCommand, findOnUserPath, installSkill, isEphemeralRoot, launchCommand, manualRegisterCommand, planCommand, registerMcp, resolveClaudeDir, spawnRunner, } from "./installer.js";
|
|
19
19
|
import { LEGACY_MEMORY_DIRNAME, MEMORY_DIRNAME } from "./engine/memory.js";
|
|
20
|
+
import { localClock, formatSessionLine, LIVE_TOKEN_FILE, watchTarget, wholeSessions } from "./engine/live.js";
|
|
20
21
|
import { formatScan, scanProject } from "./scan.js";
|
|
21
22
|
const here = path.dirname(fileURLToPath(import.meta.url));
|
|
22
23
|
const packageRoot = path.resolve(here, "..");
|
|
@@ -27,7 +28,8 @@ Usage:
|
|
|
27
28
|
scenescout scan <projectPath> Discover framework, routes, auth states
|
|
28
29
|
scenescout serve Run the MCP server (stdio)
|
|
29
30
|
scenescout install One-step setup: skill + Chromium + MCP registration
|
|
30
|
-
|
|
31
|
+
It also puts the \`scenescout\` command on your PATH.
|
|
32
|
+
(--skip-browser, --no-register, --no-command to opt out of a step;
|
|
31
33
|
--browser-only when the skill and server came from a plugin;
|
|
32
34
|
--browsers <list> to choose what to download: chromium (default),
|
|
33
35
|
chromium-headless-shell, firefox, webkit, all — comma-separated)
|
|
@@ -37,52 +39,87 @@ Usage:
|
|
|
37
39
|
scenescout doctor Check the setup and print the fix for anything missing
|
|
38
40
|
(--engine: only node, the build and the browser — for plugin
|
|
39
41
|
installs and other MCP clients)
|
|
40
|
-
scenescout status [projectPath] What is the engine doing right now? (
|
|
42
|
+
scenescout status [projectPath] What is the engine doing right now? (every session + recent actions)
|
|
43
|
+
scenescout watch [projectPath] Open the live view in a browser: what each session is doing, a thumbnail
|
|
44
|
+
of its page, and a live stream you can switch on per session
|
|
45
|
+
(--no-open to print the address only)
|
|
41
46
|
`);
|
|
42
47
|
process.exit(exitCode);
|
|
43
48
|
}
|
|
44
|
-
/**
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
+
/**
|
|
50
|
+
* A project last touched before the rename (or one a pre-rename engine is using
|
|
51
|
+
* right now) still keeps its status under the legacy directory.
|
|
52
|
+
*/
|
|
53
|
+
function statusDir(projectPath) {
|
|
54
|
+
return ([MEMORY_DIRNAME, LEGACY_MEMORY_DIRNAME]
|
|
49
55
|
.map((name) => path.join(projectPath, name))
|
|
50
|
-
.find((candidate) => fs.existsSync(path.join(candidate, "status.json"))) ?? path.join(projectPath, MEMORY_DIRNAME);
|
|
56
|
+
.find((candidate) => fs.existsSync(path.join(candidate, "status.json"))) ?? path.join(projectPath, MEMORY_DIRNAME));
|
|
57
|
+
}
|
|
58
|
+
/** null when there is no file; "unreadable" when there is one and it does not parse. */
|
|
59
|
+
function readStatusFile(dir) {
|
|
51
60
|
const statusPath = path.join(dir, "status.json");
|
|
52
|
-
if (!fs.existsSync(statusPath))
|
|
53
|
-
|
|
54
|
-
return;
|
|
55
|
-
}
|
|
56
|
-
let st;
|
|
61
|
+
if (!fs.existsSync(statusPath))
|
|
62
|
+
return null;
|
|
57
63
|
try {
|
|
58
|
-
|
|
64
|
+
return JSON.parse(fs.readFileSync(statusPath, "utf8"));
|
|
59
65
|
}
|
|
60
66
|
catch {
|
|
61
67
|
// status.json is written fire-and-forget on every tool call, so a process
|
|
62
68
|
// killed mid-write leaves a truncated file. That is a diagnosable state,
|
|
63
69
|
// not a reason for the diagnostic tool itself to crash.
|
|
64
|
-
|
|
70
|
+
return "unreadable";
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
function pidAlive(pid) {
|
|
74
|
+
if (!pid)
|
|
75
|
+
return false;
|
|
76
|
+
try {
|
|
77
|
+
process.kill(pid, 0);
|
|
78
|
+
return true;
|
|
79
|
+
}
|
|
80
|
+
catch (err) {
|
|
81
|
+
// EPERM means the process EXISTS but belongs to another user — only
|
|
82
|
+
// ESRCH actually means "no such process". Treating both as dead reported
|
|
83
|
+
// a live engine as stale.
|
|
84
|
+
return err?.code === "EPERM";
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
/** Realtime observability: read the status file + recent action log the running engine maintains. */
|
|
88
|
+
function status(projectPath) {
|
|
89
|
+
const dir = statusDir(projectPath);
|
|
90
|
+
const statusPath = path.join(dir, "status.json");
|
|
91
|
+
const st = readStatusFile(dir);
|
|
92
|
+
if (st === null) {
|
|
93
|
+
console.log(`No status file at ${statusPath} — no SceneScout engine has attached to this project (or it predates v0.8).`);
|
|
65
94
|
return;
|
|
66
95
|
}
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
process.kill(st.pid, 0);
|
|
71
|
-
alive = true;
|
|
72
|
-
}
|
|
73
|
-
catch (err) {
|
|
74
|
-
// EPERM means the process EXISTS but belongs to another user — only
|
|
75
|
-
// ESRCH actually means "no such process". Treating both as dead reported
|
|
76
|
-
// a live engine as stale.
|
|
77
|
-
alive = err?.code === "EPERM";
|
|
78
|
-
}
|
|
96
|
+
if (st === "unreadable") {
|
|
97
|
+
console.log(`Status file at ${statusPath} is unreadable or truncated — the engine was probably killed mid-write. Re-attach to refresh it.`);
|
|
98
|
+
return;
|
|
79
99
|
}
|
|
100
|
+
const alive = pidAlive(st.pid);
|
|
80
101
|
const age = st.at ? Math.round((Date.now() - new Date(st.at).getTime()) / 1000) : null;
|
|
81
102
|
console.log(`Engine pid ${st.pid ?? "?"} — ${alive ? "ALIVE" : "not running (stale status)"}`);
|
|
82
103
|
console.log(`${st.phase === "running" ? "⏳ running" : "· idle after"}: ${st.tool ?? "?"}${age !== null ? ` (as of ${age}s ago)` : ""}`);
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
104
|
+
// The file is written by another process and can be caught mid-write, so
|
|
105
|
+
// only entries whole enough to describe are described.
|
|
106
|
+
const sessions = wholeSessions(st.detail);
|
|
107
|
+
if (sessions.length > 0) {
|
|
108
|
+
// One line per session. The single "Session:" line below it is all an
|
|
109
|
+
// engine from before the live view can offer.
|
|
110
|
+
console.log(`Sessions (${sessions.length}):`);
|
|
111
|
+
for (const entry of sessions)
|
|
112
|
+
console.log(` ${formatSessionLine(entry, Date.now())}`);
|
|
113
|
+
if (alive && st.live?.port)
|
|
114
|
+
console.log("Live view: scenescout watch");
|
|
115
|
+
if (alive && st.live?.error)
|
|
116
|
+
console.log(`Live view unavailable: ${st.live.error}`);
|
|
117
|
+
}
|
|
118
|
+
else {
|
|
119
|
+
console.log(`Session: ${st.session ?? "?"} (${st.role ?? "?"})${st.sessions && st.sessions.length > 1 ? ` · all sessions: ${st.sessions.join(", ")}` : ""}`);
|
|
120
|
+
if (st.url)
|
|
121
|
+
console.log(`URL: ${st.url}`);
|
|
122
|
+
}
|
|
86
123
|
// Recent actions from the newest session log — the "what has it been doing" trail.
|
|
87
124
|
const logs = fs.existsSync(dir)
|
|
88
125
|
? fs
|
|
@@ -108,7 +145,7 @@ function status(projectPath) {
|
|
|
108
145
|
for (const line of lines) {
|
|
109
146
|
try {
|
|
110
147
|
const e = JSON.parse(line);
|
|
111
|
-
console.log(` ${e.at
|
|
148
|
+
console.log(` ${localClock(e.at)} ${e.action}${e.target ? ` ${e.target}` : ""} @ ${e.url}`);
|
|
112
149
|
}
|
|
113
150
|
catch {
|
|
114
151
|
/* skip malformed line */
|
|
@@ -116,6 +153,43 @@ function status(projectPath) {
|
|
|
116
153
|
}
|
|
117
154
|
}
|
|
118
155
|
}
|
|
156
|
+
/** Open the engine's live view. The engine serves it; this only finds the address and hands it to a browser. */
|
|
157
|
+
function watch(projectPath, open) {
|
|
158
|
+
const dir = statusDir(projectPath);
|
|
159
|
+
const st = readStatusFile(dir);
|
|
160
|
+
let token = null;
|
|
161
|
+
try {
|
|
162
|
+
token = fs.readFileSync(path.join(dir, LIVE_TOKEN_FILE), "utf8");
|
|
163
|
+
}
|
|
164
|
+
catch {
|
|
165
|
+
// watchTarget explains a missing token in context.
|
|
166
|
+
}
|
|
167
|
+
const target = watchTarget({ status: st, alive: st !== null && st !== "unreadable" && pidAlive(st.pid), token });
|
|
168
|
+
if ("problem" in target) {
|
|
169
|
+
console.log(target.problem);
|
|
170
|
+
process.exitCode = 1;
|
|
171
|
+
return;
|
|
172
|
+
}
|
|
173
|
+
console.log(`Live view: ${target.url}`);
|
|
174
|
+
console.log("It is served on this machine only, and the address holds its access token: treat it like a password.");
|
|
175
|
+
if (!open)
|
|
176
|
+
return;
|
|
177
|
+
const { command, args } = browserOpener(target.url);
|
|
178
|
+
const result = spawnSync(command, args, { stdio: "ignore" });
|
|
179
|
+
if (result.error || result.status !== 0)
|
|
180
|
+
console.log("Could not open a browser from here. Open the address above yourself.");
|
|
181
|
+
}
|
|
182
|
+
/** The platform's own "open this URL" command. */
|
|
183
|
+
function browserOpener(url) {
|
|
184
|
+
switch (process.platform) {
|
|
185
|
+
case "darwin":
|
|
186
|
+
return { command: "open", args: [url] };
|
|
187
|
+
case "win32":
|
|
188
|
+
return { command: "cmd", args: ["/c", "start", "", url] };
|
|
189
|
+
default:
|
|
190
|
+
return { command: "xdg-open", args: [url] };
|
|
191
|
+
}
|
|
192
|
+
}
|
|
119
193
|
/** Which browser builds are on disk, going by the paths Playwright reports for the version we depend on. */
|
|
120
194
|
async function presentBrowsers() {
|
|
121
195
|
const executables = { chromium: null, firefox: null, webkit: null };
|
|
@@ -303,8 +377,39 @@ async function install(flags) {
|
|
|
303
377
|
}
|
|
304
378
|
}
|
|
305
379
|
}
|
|
380
|
+
// `scenescout status`, `watch` and `doctor` are typed by a person, and neither
|
|
381
|
+
// a checkout nor an npx run leaves the command on PATH. Not having it costs
|
|
382
|
+
// convenience, never a working setup, so this step reports and does not fail.
|
|
383
|
+
let cli = isEphemeralRoot(packageRoot) ? `npx -y ${CLI_NAME}` : `node ${path.join(packageRoot, "dist", "cli.js")}`;
|
|
384
|
+
if (browserOnly) {
|
|
385
|
+
// a plugin install has no package of its own to put on PATH
|
|
386
|
+
}
|
|
387
|
+
else if (flags.includes("--no-command")) {
|
|
388
|
+
console.log(`· Putting \`${CLI_NAME}\` on PATH skipped (--no-command). Until then the command is: ${cli}`);
|
|
389
|
+
}
|
|
390
|
+
else {
|
|
391
|
+
const onPath = () => findOnUserPath({ names: process.platform === "win32" ? [`${CLI_NAME}.cmd`] : [CLI_NAME], pathValue: process.env.PATH ?? "" });
|
|
392
|
+
const pkg = JSON.parse(fs.readFileSync(path.join(packageRoot, "package.json"), "utf8"));
|
|
393
|
+
const done = ensureCommand(planCommand({ packageRoot, nodePath: process.execPath, version: pkg.version, resolved: onPath(), platform: process.platform }), spawnRunner);
|
|
394
|
+
if (done.status === "present") {
|
|
395
|
+
cli = CLI_NAME;
|
|
396
|
+
console.log(`✓ \`${CLI_NAME}\` command already on PATH: ${done.at}`);
|
|
397
|
+
}
|
|
398
|
+
else if (done.status === "installed") {
|
|
399
|
+
const at = onPath();
|
|
400
|
+
if (at)
|
|
401
|
+
cli = CLI_NAME;
|
|
402
|
+
const what = done.how === "link" ? "linked to this checkout, so it runs whatever was last built" : "installed globally";
|
|
403
|
+
console.log(at
|
|
404
|
+
? `✓ \`${CLI_NAME}\` command ${what}: ${at}${done.replaced ? ` (it replaces ${done.replaced})` : ""}`
|
|
405
|
+
: `· \`${CLI_NAME}\` was ${what}, but npm's global bin directory is not on this shell's PATH. Add it (\`npm prefix -g\` names it; the commands are in its bin folder), or use: ${cli}`);
|
|
406
|
+
}
|
|
407
|
+
else {
|
|
408
|
+
console.log(`· \`${CLI_NAME}\` was not put on PATH (${done.detail}). To do it by hand: ${done.manual}\n Until then the command is: ${cli}`);
|
|
409
|
+
}
|
|
410
|
+
}
|
|
306
411
|
if (failed) {
|
|
307
|
-
console.log(`\nSetup is incomplete — fix the lines marked ✗ or · above, then run:
|
|
412
|
+
console.log(`\nSetup is incomplete — fix the lines marked ✗ or · above, then run: ${cli} doctor${forClaude ? "" : " --engine"}`);
|
|
308
413
|
process.exitCode = 1;
|
|
309
414
|
return;
|
|
310
415
|
}
|
|
@@ -317,7 +422,7 @@ async function install(flags) {
|
|
|
317
422
|
// Telling someone to restart a client nothing was registered with sends them looking for a server that is not there.
|
|
318
423
|
if (others.length > 0 && !flags.includes("--no-register"))
|
|
319
424
|
console.log(`\n${firstMessageHint(others)}`);
|
|
320
|
-
console.log(`Something off? Run:
|
|
425
|
+
console.log(`Something off? Run: ${cli} doctor${forClaude ? "" : " --engine"}`);
|
|
321
426
|
}
|
|
322
427
|
async function doctor(flags) {
|
|
323
428
|
const checks = diagnose({
|
|
@@ -387,6 +492,11 @@ try {
|
|
|
387
492
|
status(path.resolve(args[0] ?? process.cwd()));
|
|
388
493
|
break;
|
|
389
494
|
}
|
|
495
|
+
case "watch": {
|
|
496
|
+
const positional = args.filter((a) => !a.startsWith("--"));
|
|
497
|
+
watch(path.resolve(positional[0] ?? process.cwd()), !args.includes("--no-open"));
|
|
498
|
+
break;
|
|
499
|
+
}
|
|
390
500
|
default:
|
|
391
501
|
usage();
|
|
392
502
|
}
|
package/dist/engine/browser.js
CHANGED
|
@@ -2,13 +2,13 @@ import { chromium, firefox, webkit } from "playwright";
|
|
|
2
2
|
import fs from "node:fs";
|
|
3
3
|
import path from "node:path";
|
|
4
4
|
import { elementKey, fingerprintState, isNonPageRoute, normalizePath } from "./fingerprint.js";
|
|
5
|
-
import { AUTH_LOSS_PREFIX, MemoryStore } from "./memory.js";
|
|
5
|
+
import { AUTH_LOSS_PREFIX, JOURNEY_END, JOURNEY_START, MemoryStore } from "./memory.js";
|
|
6
6
|
import { AuthLossTracker } from "./authloss.js";
|
|
7
7
|
import { COLLECT_INTERACTABLES_SCRIPT, VISIBLE_SRC, geometryIssues, BROKEN_IMAGES_SCRIPT, brokenImageIssues, } from "./collector.js";
|
|
8
8
|
import { OracleMonitor, formatViolations } from "./oracles.js";
|
|
9
9
|
import { extractCreatedIds, isOwnedResource, normalizeId } from "./ownership.js";
|
|
10
10
|
import { formatJourney, measureJourney } from "./journey.js";
|
|
11
|
-
import { defaultEngine, focusAdvanceKey, REMOVE_SHARED_WORKER_SCRIPT, serviceWorkerPolicy, sharedWorkersAllowed } from "../browsers.js";
|
|
11
|
+
import { defaultEngine, focusAdvanceKey, REMOVE_SHARED_WORKER_SCRIPT, screencastSupport, serviceWorkerPolicy, sharedWorkersAllowed, } from "../browsers.js";
|
|
12
12
|
import { revealedLines } from "./hover.js";
|
|
13
13
|
import { explainLaunchFailure, isMissingBrowser } from "./launch.js";
|
|
14
14
|
import { ACTION_TIMEOUT_MS, performScroll, probeFocusIndicators, probeOverlays, scrollContainer } from "./probes.js";
|
|
@@ -98,6 +98,8 @@ function actionabilityDiagnostic(message) {
|
|
|
98
98
|
* actions by ref, runs oracles after every action, and records everything in
|
|
99
99
|
* the persistent memory store. Contains no LLM calls — the MCP client is the brain.
|
|
100
100
|
*/
|
|
101
|
+
/** A screencast whose page has been gone for this many 500 ms ticks ends and says so; a re-attach takes fewer. */
|
|
102
|
+
const SCREENCAST_PAGELESS_TICKS = 20;
|
|
101
103
|
export class BrowserEngine {
|
|
102
104
|
browser = null;
|
|
103
105
|
context = null;
|
|
@@ -186,6 +188,8 @@ export class BrowserEngine {
|
|
|
186
188
|
designAuditCount = 0;
|
|
187
189
|
/** Active task-efficiency measurement (scout_journey), if any. */
|
|
188
190
|
journey = null;
|
|
191
|
+
/** The session's task, from scout_attach. Empty when the agent gave none. */
|
|
192
|
+
task = "";
|
|
189
193
|
/**
|
|
190
194
|
* Begin measuring a user JOURNEY — the interaction cost of completing one
|
|
191
195
|
* real task ("create an order", "approve a document"). E2E suites assert
|
|
@@ -200,7 +204,7 @@ export class BrowserEngine {
|
|
|
200
204
|
fromLog: this.memory?.actionLog.length ?? 0,
|
|
201
205
|
startUrl: page.url(),
|
|
202
206
|
};
|
|
203
|
-
this.logAction({ action:
|
|
207
|
+
this.logAction({ action: JOURNEY_START, target: goal, url: page.url() });
|
|
204
208
|
return `JOURNEY STARTED — "${goal}"\nFrom: ${page.url()}\nNow perform the task the way a first-time user would (click through the UI; don't jump straight to a known deep URL, or the measurement is meaningless). Call scout_journey {action:"end"} when the task is complete or you conclude it can't be.`;
|
|
205
209
|
}
|
|
206
210
|
/** Close the journey and report its interaction cost + friction signals. */
|
|
@@ -225,7 +229,7 @@ export class BrowserEngine {
|
|
|
225
229
|
catch {
|
|
226
230
|
/* fact recording is best-effort */
|
|
227
231
|
}
|
|
228
|
-
this.logAction({ action:
|
|
232
|
+
this.logAction({ action: JOURNEY_END, target: j.goal, url: page.url(), result: completed ? "completed" : "abandoned" });
|
|
229
233
|
return formatJourney({ goal: j.goal, completed, seconds, note }, measured);
|
|
230
234
|
}
|
|
231
235
|
/** Whether the browser window is visible — headed hover results carry a physical-cursor caveat. */
|
|
@@ -258,6 +262,7 @@ export class BrowserEngine {
|
|
|
258
262
|
throw new Error(`storageStatePath does not exist: ${opts.storageStatePath}`);
|
|
259
263
|
}
|
|
260
264
|
this.mode = opts.mode ?? "read-only";
|
|
265
|
+
this.task = (opts.task ?? "").trim().replace(/\s+/g, " ").slice(0, 300);
|
|
261
266
|
this.headed = opts.headed ?? false;
|
|
262
267
|
this.blockedRequests = [];
|
|
263
268
|
this.pendingCreations = new Set();
|
|
@@ -1909,6 +1914,125 @@ export class BrowserEngine {
|
|
|
1909
1914
|
this.logAction({ action: "screenshot", url: page.url() });
|
|
1910
1915
|
return { base64: buf.toString("base64"), mimeType: "image/jpeg" };
|
|
1911
1916
|
}
|
|
1917
|
+
/**
|
|
1918
|
+
* What the live view shows next to a session's name. The task is what the
|
|
1919
|
+
* agent said the session is for; the objective is the goal of the journey it
|
|
1920
|
+
* is on right now (scout_journey). Both are the agent's own words — the
|
|
1921
|
+
* engine sees tool calls, never the reasoning behind them. Neither is
|
|
1922
|
+
* redacted here: the caller that writes them anywhere does that.
|
|
1923
|
+
*/
|
|
1924
|
+
get liveDescription() {
|
|
1925
|
+
return {
|
|
1926
|
+
mode: this.mode,
|
|
1927
|
+
browser: this.engineName,
|
|
1928
|
+
headed: this.headed,
|
|
1929
|
+
...(this.task ? { task: this.task } : {}),
|
|
1930
|
+
...(this.journey ? { objective: this.journey.goal, objectiveSince: new Date(this.journey.startedAt).toISOString() } : {}),
|
|
1931
|
+
};
|
|
1932
|
+
}
|
|
1933
|
+
/**
|
|
1934
|
+
* A frame for somebody WATCHING the run, as opposed to scout_screenshot,
|
|
1935
|
+
* which is the agent looking. It is not logged: the action log is the repro
|
|
1936
|
+
* trace attached to findings, and a person glancing at the dashboard is not
|
|
1937
|
+
* a step anyone should replay. It is also bounded, because the moment a
|
|
1938
|
+
* viewer most wants a picture is when the renderer has wedged.
|
|
1939
|
+
*/
|
|
1940
|
+
async liveShot(timeoutMs = 3000) {
|
|
1941
|
+
const page = this.page;
|
|
1942
|
+
if (!page || page.isClosed())
|
|
1943
|
+
return null;
|
|
1944
|
+
let timer;
|
|
1945
|
+
// The driver's own timeout covers a slow capture; the race covers a
|
|
1946
|
+
// renderer that never answers the protocol at all.
|
|
1947
|
+
return Promise.race([
|
|
1948
|
+
page.screenshot({ type: "jpeg", quality: 55, fullPage: false, timeout: timeoutMs }).catch(() => null),
|
|
1949
|
+
new Promise((resolve) => {
|
|
1950
|
+
timer = setTimeout(() => resolve(null), timeoutMs + 500);
|
|
1951
|
+
}),
|
|
1952
|
+
]).finally(() => clearTimeout(timer));
|
|
1953
|
+
}
|
|
1954
|
+
/**
|
|
1955
|
+
* Push frames of this session's page until the returned function is called.
|
|
1956
|
+
* The stream follows the session rather than one tab: adopting a popup
|
|
1957
|
+
* replaces `this.page`, and a stream left on the old tab would show a page
|
|
1958
|
+
* the session is no longer driving.
|
|
1959
|
+
*/
|
|
1960
|
+
async startScreencast(onFrame, onEnd = () => { }) {
|
|
1961
|
+
if (!this.page || this.page.isClosed())
|
|
1962
|
+
return null;
|
|
1963
|
+
let stopped = false;
|
|
1964
|
+
let bound = null;
|
|
1965
|
+
let release = null;
|
|
1966
|
+
// Ticks in a row with no page to take frames from: a re-attach passes
|
|
1967
|
+
// through a few, a closed session never comes back.
|
|
1968
|
+
let pageless = 0;
|
|
1969
|
+
const bind = async (page) => {
|
|
1970
|
+
if (screencastSupport(this.engineName) === "cdp") {
|
|
1971
|
+
const cdp = await page.context().newCDPSession(page);
|
|
1972
|
+
try {
|
|
1973
|
+
cdp.on("Page.screencastFrame", (frame) => {
|
|
1974
|
+
if (!stopped)
|
|
1975
|
+
onFrame(Buffer.from(frame.data, "base64"));
|
|
1976
|
+
void cdp.send("Page.screencastFrameAck", { sessionId: frame.sessionId }).catch(() => { });
|
|
1977
|
+
});
|
|
1978
|
+
await cdp.send("Page.startScreencast", { format: "jpeg", quality: 55, maxWidth: 1280, maxHeight: 900, everyNthFrame: 2 });
|
|
1979
|
+
}
|
|
1980
|
+
catch (err) {
|
|
1981
|
+
// Not bound: the timer tries this page again on its next tick.
|
|
1982
|
+
await cdp.detach().catch(() => { });
|
|
1983
|
+
throw err;
|
|
1984
|
+
}
|
|
1985
|
+
release = async () => {
|
|
1986
|
+
await cdp.send("Page.stopScreencast").catch(() => { });
|
|
1987
|
+
await cdp.detach().catch(() => { });
|
|
1988
|
+
};
|
|
1989
|
+
}
|
|
1990
|
+
else {
|
|
1991
|
+
release = null;
|
|
1992
|
+
}
|
|
1993
|
+
bound = page;
|
|
1994
|
+
};
|
|
1995
|
+
const stop = async () => {
|
|
1996
|
+
stopped = true;
|
|
1997
|
+
clearInterval(timer);
|
|
1998
|
+
const current = release;
|
|
1999
|
+
release = null;
|
|
2000
|
+
await current?.();
|
|
2001
|
+
};
|
|
2002
|
+
await bind(this.page).catch(() => { });
|
|
2003
|
+
// One timer does both jobs: it notices a replaced tab, and where the
|
|
2004
|
+
// browser cannot push frames it is also what takes them.
|
|
2005
|
+
const timer = setInterval(() => {
|
|
2006
|
+
if (stopped)
|
|
2007
|
+
return;
|
|
2008
|
+
const page = this.page;
|
|
2009
|
+
if (!page || page.isClosed()) {
|
|
2010
|
+
pageless += 1;
|
|
2011
|
+
if (pageless >= SCREENCAST_PAGELESS_TICKS)
|
|
2012
|
+
void stop().finally(onEnd);
|
|
2013
|
+
return;
|
|
2014
|
+
}
|
|
2015
|
+
pageless = 0;
|
|
2016
|
+
if (page !== bound) {
|
|
2017
|
+
const previous = release;
|
|
2018
|
+
release = null;
|
|
2019
|
+
void (async () => {
|
|
2020
|
+
await previous?.();
|
|
2021
|
+
if (!stopped)
|
|
2022
|
+
await bind(page).catch(() => { });
|
|
2023
|
+
})();
|
|
2024
|
+
return;
|
|
2025
|
+
}
|
|
2026
|
+
if (screencastSupport(this.engineName) === "poll") {
|
|
2027
|
+
void this.liveShot(1500).then((jpeg) => {
|
|
2028
|
+
if (jpeg && !stopped)
|
|
2029
|
+
onFrame(jpeg);
|
|
2030
|
+
});
|
|
2031
|
+
}
|
|
2032
|
+
}, 500);
|
|
2033
|
+
timer.unref();
|
|
2034
|
+
return stop;
|
|
2035
|
+
}
|
|
1912
2036
|
get currentState() {
|
|
1913
2037
|
return this.currentFingerprint;
|
|
1914
2038
|
}
|