scenescout 3.0.1 → 3.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +26 -0
- package/README.md +50 -8
- package/dist/engine/browser.js +61 -1
- package/dist/engine/live-page.js +200 -10
- package/dist/engine/live.js +40 -2
- package/dist/engine/replay.js +348 -0
- package/dist/engine/report.js +101 -1
- package/dist/mcp-server.js +84 -8
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,31 @@
|
|
|
1
1
|
# scenescout
|
|
2
2
|
|
|
3
|
+
## 3.1.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- 52d97d8: Record a run, and read the whole thing back afterwards.
|
|
8
|
+
|
|
9
|
+
`scout_attach {record: true}` keeps a frame of the page after every action, and
|
|
10
|
+
`scout_report` then writes `report.html` beside `report.md`: the report, the
|
|
11
|
+
screenshots taken around each finding, and every session's trail in the blocks
|
|
12
|
+
its tasks made — one self-contained page that opens from the file system with
|
|
13
|
+
nothing running. Recording is off unless asked for, because the frames are
|
|
14
|
+
pictures of the app under test and no redaction can read a picture
|
|
15
|
+
([ADR 8](docs/adr/0008-a-recorded-run-is-evidence-and-must-be-asked-for.md)).
|
|
16
|
+
|
|
17
|
+
The live view serves the same document at its own address and goes there when
|
|
18
|
+
the run ends, so the report survives a refresh instead of dying with the board.
|
|
19
|
+
The close-up gains a timeline: a tick per action, coloured by task, that plays
|
|
20
|
+
a recorded run back while it is still going. A finding's screenshots also hang
|
|
21
|
+
under it in the live report panel, which no longer resets itself while it is
|
|
22
|
+
being read.
|
|
23
|
+
|
|
24
|
+
Fixes: a listener left on a control that had been replaced threw on load and
|
|
25
|
+
left the board blank; a viewer arriving after the last browser closed saw the
|
|
26
|
+
empty state instead of the finished run; a finding's evidence could be drawn
|
|
27
|
+
from a different session than the one that filed it.
|
|
28
|
+
|
|
3
29
|
## 3.0.1
|
|
4
30
|
|
|
5
31
|
### Patch Changes
|
package/README.md
CHANGED
|
@@ -187,28 +187,67 @@ The agent scans the project (if there is one), attaches read-only, explores, and
|
|
|
187
187
|
|
|
188
188
|
When a session attaches, the engine starts a small live view and hands the agent its address on a `Live view:` line, which the agent passes on to you. From a terminal, `scenescout watch` opens the same page. There is one card per session:
|
|
189
189
|
|
|
190
|
-
<p align="center"><img src="examples/screenshots/live-view.png" alt="The live view during a run of
|
|
190
|
+
<p align="center"><img src="examples/screenshots/live-view.png" alt="The live view during a run of three parallel agents against the demo app: one card per session, each with its role and objective, the task it is on, the tool it is running, the page it is on, a live thumbnail, and a feed of the actions it just took, tinted one colour per task" width="880" /></p>
|
|
191
191
|
|
|
192
192
|
- **What it is doing:** the tool it is running and for how long, the page it is on, and a thumbnail of that page. This works for headless runs too, which have no window to look at.
|
|
193
193
|
- **What it just did:** a rolling feed of its actions, each with its target and how it turned out, with failures in red. It is the same trail a finding's repro trace uses. The engine never sees the agent's reasoning, so this is what the session *did*, not what it thought.
|
|
194
194
|
- **Stuck, not slow:** a call still running past its own tool's watchdog budget turns the card red, so a wedged session is visible without asking. A crawl legitimately runs for minutes; it is judged against the crawl's budget, not a click's.
|
|
195
195
|
- **Live stream:** switch it on for one card, or for all of them. Click a thumbnail for a close-up.
|
|
196
196
|
- **The report, as it stands:** the Report button in the top bar shows the same document `scout_report` writes at the end, rendered from the run's current state, so findings can be read while the agents are still working.
|
|
197
|
-
- **What it is for:** the close-up puts the feed beside the session's brief
|
|
197
|
+
- **What it is for:** the close-up puts the feed beside the session's brief — the objective it was given when it attached (`scout_attach {objective}`), and underneath it the task it is on right now (`scout_task`), which the engine requires before any tool will act. Each task tints its own block of actions, so a change of task is a change of colour; point at a block and the brief names the task those actions served.
|
|
198
|
+
- **Scrub it back:** under the page is a tick per action, coloured by task. Click one to see the frame from that moment, and `Back to live` to return. On a run that was not recorded the ticks still read the trail; they just have no picture behind them.
|
|
198
199
|
|
|
199
|
-
<p align="center"><img src="examples/screenshots/live-view-closeup.png" alt="A close-up of one session:
|
|
200
|
+
<p align="center"><img src="examples/screenshots/live-view-closeup.png" alt="A close-up of one session: a frame from a step picked out of the timeline, the timeline itself as a tick per action coloured by task, the feed of the session's actions in the same colours, and beside it the objective and the task it is on" width="880" /></p>
|
|
200
201
|
|
|
201
|
-
<p align="center"><img src="examples/screenshots/live-view-report.png" alt="The report opened from the live view's top bar while the run is still going: summary table, gap ledger and the findings filed so far" width="880" /></p>
|
|
202
|
+
<p align="center"><img src="examples/screenshots/live-view-report.png" alt="The report opened from the live view's top bar while the run is still going: summary table, gap ledger, and the findings filed so far, each with an accordion of the screenshots taken around it" width="880" /></p>
|
|
202
203
|
|
|
203
|
-
The view is served on `127.0.0.1` only, behind a token that changes every time the engine starts. It answers `GET` and nothing else, so a viewer can watch a run but not act in it, and no frame is
|
|
204
|
+
The view is served on `127.0.0.1` only, behind a token that changes every time the engine starts. It answers `GET` and nothing else, so a viewer can watch a run but not act in it, and no frame it shows is written to disk ([ADR 7](docs/adr/0007-the-live-view-is-local-read-only-and-leaves-nothing-behind.md)) unless the run was recorded, which is asked for and off by default ([ADR 8](docs/adr/0008-a-recorded-run-is-evidence-and-must-be-asked-for.md)). A stream runs only while someone is watching it. `SCENESCOUT_LIVE=off` keeps the port closed.
|
|
204
205
|
|
|
205
|
-
**Try it with parallel agents.** The demo app has three roles and several separate areas, so a run can be split between agents. Start it with `npm run demo:serve`, then ask your agent to explore it with several agents in parallel, one role and one area each. The pictures above come from a run of
|
|
206
|
+
**Try it with parallel agents.** The demo app has three roles and several separate areas, so a run can be split between agents. Start it with `npm run demo:serve`, then ask your agent to explore it with several agents in parallel, one role and one area each. The pictures above come from a run of three. Two things keep a parallel run efficient:
|
|
206
207
|
|
|
207
208
|
- **Each agent opens its own session when it starts and closes it when it is done.** An agent waiting for its turn then holds no browser. Opening every session up front leaves browsers idling while the machine runs out of memory for the agents that are working.
|
|
208
209
|
- **Run about as many agents at once as your machine has cores, less two.** Each one drives a real browser.
|
|
209
210
|
|
|
210
211
|
---
|
|
211
212
|
|
|
213
|
+
## 🎬 Recording a run, and reading it back
|
|
214
|
+
|
|
215
|
+
A report says what happened. For QA work that is not always enough — the point
|
|
216
|
+
is often to *show* what was checked, not to assert it. Ask for a recorded run
|
|
217
|
+
and the engine keeps a frame of the page after every action:
|
|
218
|
+
|
|
219
|
+
```
|
|
220
|
+
Use SceneScout to test http://localhost:3000, record the run
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
or, on the tool directly, `scout_attach {record: true}`.
|
|
224
|
+
|
|
225
|
+
Then `scout_report` writes two files side by side in `.scenescout/`:
|
|
226
|
+
`report.md` as always, and `report.html` — the whole run as one self-contained
|
|
227
|
+
page. It opens from the file system with nothing running, needs no network, and
|
|
228
|
+
holds:
|
|
229
|
+
|
|
230
|
+
- **The report**, rendered from the same Markdown.
|
|
231
|
+
- **The screenshots around each finding**, in an accordion under it, from the
|
|
232
|
+
session that filed it.
|
|
233
|
+
- **Every session's trail**, in the blocks its tasks made, each step with the
|
|
234
|
+
page as it was at that moment.
|
|
235
|
+
|
|
236
|
+
<p align="center"><img src="examples/screenshots/run-page.png" alt="The saved copy of a run, opened from the file system with nothing running: a finding with its evidence accordion open, showing the four screenshots taken around it with the action and time under each" width="880" /></p>
|
|
237
|
+
|
|
238
|
+
The live view serves the same document at `run` while the engine is still up,
|
|
239
|
+
and sends you there when the run ends — so the address survives a refresh
|
|
240
|
+
instead of a panel over a dead board.
|
|
241
|
+
|
|
242
|
+
**What it costs.** Frames are pictures of the app under test, inside the tested
|
|
243
|
+
project's folder, and the secret redaction that protects everything else the
|
|
244
|
+
engine writes cannot read a picture. That is why it is off unless asked for,
|
|
245
|
+
capped per session, and written only under `.scenescout/`, which ignores itself
|
|
246
|
+
so `git add -A` in the tested project cannot pick the frames up. The reasoning is in
|
|
247
|
+
[ADR 8](docs/adr/0008-a-recorded-run-is-evidence-and-must-be-asked-for.md).
|
|
248
|
+
|
|
249
|
+
---
|
|
250
|
+
|
|
212
251
|
## 🔄 How a run works
|
|
213
252
|
|
|
214
253
|
One curiosity loop, repeated — breadth first, then judgment where it matters:
|
|
@@ -242,7 +281,7 @@ Snapshots are cheap: re-snapshotting a route returns only *what changed*, with s
|
|
|
242
281
|
| **Look** | `scout_snapshot` `scout_hover` `scout_screenshot` | Read the structured scene (diffed); reveal tooltips/hover cards; capture pixels only when needed |
|
|
243
282
|
| **Act** | `scout_click` `scout_type` `scout_select` `scout_upload` `scout_press` `scout_scroll` `scout_navigate` `scout_back` `scout_run_plan` | Drive the UI like a user; `scout_run_plan` batches a whole mechanical sequence into one call |
|
|
244
283
|
| **Assess** | `scout_design_audit` `scout_journey` | Score a page's craft/a11y/consistency; measure how hard a task is to complete |
|
|
245
|
-
| **Record** | `scout_note` `scout_finding` `scout_resolve` `scout_report` | Curate durable notes; file deduped findings; mark fixes;
|
|
284
|
+
| **Record** | `scout_note` `scout_finding` `scout_resolve` `scout_report` | Curate durable notes; file deduped findings; mark fixes; write the report, and on a recorded run the whole run as one page |
|
|
246
285
|
| **Close** | `scout_close` | Tear down one session or all |
|
|
247
286
|
|
|
248
287
|
A few that punch above their weight:
|
|
@@ -290,6 +329,8 @@ A `🛡 WRITE-POLICY blocked` notice is the safety net doing its job, not an app
|
|
|
290
329
|
- 👥 **A role capability matrix** — what each role could and couldn't reach.
|
|
291
330
|
- 🧾 **A gap ledger** — everything *not* done, so the report is honest about its own coverage.
|
|
292
331
|
|
|
332
|
+
`.scenescout/report.html` — the same report as one self-contained page, with every session's trail beside it, and on a [recorded run](#-recording-a-run-and-reading-it-back) the screenshots under each finding.
|
|
333
|
+
|
|
293
334
|
👀 Watch a run live: `node dist/cli.js status <project-path>`.
|
|
294
335
|
|
|
295
336
|
---
|
|
@@ -514,8 +555,9 @@ src/
|
|
|
514
555
|
design.ts the design audit + page scoring
|
|
515
556
|
memory.ts cross-run storage + finding dedup
|
|
516
557
|
report.ts the gap ledger + report generation
|
|
558
|
+
replay.ts the run as one page: steps, tasks, frames under each finding
|
|
517
559
|
… collector · dispatch · fixtures · authloss · reaper
|
|
518
|
-
scripts/ the
|
|
560
|
+
scripts/ the 14 test suites (smoke/ holds the real-browser ones)
|
|
519
561
|
test-app/ fixtures for the real-browser smoke tests
|
|
520
562
|
skills/scenescout/ the testing method (SKILL.md): a skill in Claude Code, served by the server everywhere else
|
|
521
563
|
docs/adr/ why it's built this way
|
package/dist/engine/browser.js
CHANGED
|
@@ -10,6 +10,7 @@ import { COLLECT_INTERACTABLES_SCRIPT, VISIBLE_SRC, geometryIssues, BROKEN_IMAGE
|
|
|
10
10
|
import { OracleMonitor, formatViolations } from "./oracles.js";
|
|
11
11
|
import { extractCreatedIds, isOwnedResource, normalizeId } from "./ownership.js";
|
|
12
12
|
import { formatJourney, measureJourney } from "./journey.js";
|
|
13
|
+
import { framePath, RECORD_MAX_FRAMES } from "./replay.js";
|
|
13
14
|
import { defaultEngine, focusAdvanceKey, REMOVE_SHARED_WORKER_SCRIPT, screencastSupport, serviceWorkerPolicy, sharedWorkersAllowed, } from "../browsers.js";
|
|
14
15
|
import { revealedLines } from "./hover.js";
|
|
15
16
|
import { explainLaunchFailure, isMissingBrowser } from "./launch.js";
|
|
@@ -102,6 +103,8 @@ function actionabilityDiagnostic(message) {
|
|
|
102
103
|
*/
|
|
103
104
|
/** A screencast whose page has been gone for this many 500 ms ticks ends and says so; a re-attach takes fewer. */
|
|
104
105
|
const SCREENCAST_PAGELESS_TICKS = 20;
|
|
106
|
+
/** A frame for the record is worth a moment, not a stall: the action has already happened. */
|
|
107
|
+
const RECORD_SHOT_TIMEOUT_MS = 2500;
|
|
105
108
|
export class BrowserEngine {
|
|
106
109
|
browser = null;
|
|
107
110
|
context = null;
|
|
@@ -206,6 +209,57 @@ export class BrowserEngine {
|
|
|
206
209
|
journey = null;
|
|
207
210
|
/** The session's objective: the whole remit the agent was given at scout_attach. Empty when none was given. */
|
|
208
211
|
sessionObjective = "";
|
|
212
|
+
/**
|
|
213
|
+
* Keep a frame of the page after each action, as evidence. Off by default:
|
|
214
|
+
* a recording is pictures of somebody's app sitting in their project
|
|
215
|
+
* folder, which is the rule ADR 7 otherwise holds ("no frame touches the
|
|
216
|
+
* disk"). QA work is what earns the exception — a report says what was
|
|
217
|
+
* checked, a recording shows it.
|
|
218
|
+
*/
|
|
219
|
+
recording = false;
|
|
220
|
+
framesKept = 0;
|
|
221
|
+
/** How many frames could not be written. The first one says so in the log; the rest are counted. */
|
|
222
|
+
framesFailed = 0;
|
|
223
|
+
/**
|
|
224
|
+
* The frame for the step just taken, as a path relative to the memory
|
|
225
|
+
* directory, or undefined when this run is not recorded. Failing to write
|
|
226
|
+
* one must never fail the action: evidence is worth having, not worth
|
|
227
|
+
* losing a run over.
|
|
228
|
+
*/
|
|
229
|
+
async recordFrame(action) {
|
|
230
|
+
const dir = this.memory?.dir;
|
|
231
|
+
if (!this.recording || !dir || this.framesKept >= RECORD_MAX_FRAMES)
|
|
232
|
+
return undefined;
|
|
233
|
+
const jpeg = await this.liveShot(RECORD_SHOT_TIMEOUT_MS);
|
|
234
|
+
if (!jpeg)
|
|
235
|
+
return undefined;
|
|
236
|
+
const rel = framePath(this.sessionKey, this.framesKept + 1, action);
|
|
237
|
+
try {
|
|
238
|
+
await fs.promises.mkdir(path.dirname(path.join(dir, rel)), { recursive: true });
|
|
239
|
+
await fs.promises.writeFile(path.join(dir, rel), jpeg);
|
|
240
|
+
}
|
|
241
|
+
catch (err) {
|
|
242
|
+
// A frame that cannot be written must not fail the action, but a run
|
|
243
|
+
// that keeps NOTHING — a read-only checkout, a full disk — would
|
|
244
|
+
// otherwise finish silently and produce a report with no evidence in
|
|
245
|
+
// it, which is the one thing recording exists to prevent. Said once,
|
|
246
|
+
// through the action log, so it reaches the live feed and the report.
|
|
247
|
+
this.framesFailed += 1;
|
|
248
|
+
if (this.framesFailed === 1) {
|
|
249
|
+
this.logAction({
|
|
250
|
+
action: "record:failed",
|
|
251
|
+
target: `no frame could be written under ${dir} (${err instanceof Error ? err.message : String(err)}); later failures are not repeated here`,
|
|
252
|
+
url: this.page?.url() ?? "",
|
|
253
|
+
});
|
|
254
|
+
}
|
|
255
|
+
return undefined;
|
|
256
|
+
}
|
|
257
|
+
this.framesKept += 1;
|
|
258
|
+
if (this.framesKept === RECORD_MAX_FRAMES) {
|
|
259
|
+
this.logAction({ action: "record:full", target: `${RECORD_MAX_FRAMES} frames kept; later steps have none`, url: this.page?.url() ?? "" });
|
|
260
|
+
}
|
|
261
|
+
return rel;
|
|
262
|
+
}
|
|
209
263
|
/**
|
|
210
264
|
* The batch of actions running right now. Required before a tool may act
|
|
211
265
|
* (task.ts), stated by the agent on the call or by a journey, and kept
|
|
@@ -352,6 +406,11 @@ export class BrowserEngine {
|
|
|
352
406
|
}
|
|
353
407
|
this.mode = opts.mode ?? "read-only";
|
|
354
408
|
this.sessionObjective = (opts.objective ?? "").trim().replace(/\s+/g, " ").slice(0, 300);
|
|
409
|
+
this.recording = opts.record === true;
|
|
410
|
+
// A re-attached engine starts a new recording: numbering from where the
|
|
411
|
+
// last one stopped would run into the cap with frames it never took.
|
|
412
|
+
this.framesKept = 0;
|
|
413
|
+
this.framesFailed = 0;
|
|
355
414
|
this.headed = opts.headed ?? false;
|
|
356
415
|
this.blockedRequests = [];
|
|
357
416
|
this.pendingCreations = new Set();
|
|
@@ -887,7 +946,8 @@ export class BrowserEngine {
|
|
|
887
946
|
blocked +
|
|
888
947
|
formatViolations(this.oracles.drain()));
|
|
889
948
|
}
|
|
890
|
-
this.
|
|
949
|
+
const frame = await this.recordFrame(action);
|
|
950
|
+
this.logAction({ action, target, url, ...(frame ? { frame } : {}) });
|
|
891
951
|
await this.scanForInjections();
|
|
892
952
|
const violations = this.oracles.drain();
|
|
893
953
|
const mutations = this.drainMutations() + this.drainBlocked() + this.drainCreated();
|
package/dist/engine/live-page.js
CHANGED
|
@@ -54,7 +54,8 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
54
54
|
#finished h2 { margin: 0 0 8px; font-size: 18px; }
|
|
55
55
|
#finished p { margin: 0 0 8px; color: var(--muted); }
|
|
56
56
|
#finished .where { font: 12px/1.6 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; overflow-wrap: anywhere; }
|
|
57
|
-
#finished
|
|
57
|
+
#finished .btn { display: inline-block; margin-top: 8px; padding: 8px 16px; font-weight: 600; text-decoration: none;
|
|
58
|
+
color: #fff; background: var(--accent); border-radius: 6px; }
|
|
58
59
|
main { display: grid; grid-template-columns: repeat(auto-fill, minmax(min(100%, 320px), 1fr)); gap: 12px; padding: 16px; }
|
|
59
60
|
.card { background: var(--panel); border: 1px solid var(--line); border-radius: 8px; overflow: hidden; display: flex; flex-direction: column; }
|
|
60
61
|
.card.stuck { border-color: var(--stuck); }
|
|
@@ -128,6 +129,18 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
128
129
|
#focus .bar .line { color: #cbd5e1; padding: 0; flex: 1 1 auto; }
|
|
129
130
|
#focus .bar .line b { color: #fff; }
|
|
130
131
|
#focus .stage { flex: 1 1 auto; min-height: 0; position: relative; }
|
|
132
|
+
#timeline { flex: 0 0 auto; display: flex; gap: 2px; overflow-x: auto; padding: 6px 0 2px; scrollbar-width: thin; }
|
|
133
|
+
#timeline button { flex: 0 0 auto; width: 16px; height: 26px; padding: 0; border: 0; border-radius: 3px; cursor: pointer;
|
|
134
|
+
background: rgba(230,233,238,.18); }
|
|
135
|
+
#timeline button.g0 { background: rgba(96,165,250,.75); } #timeline button.g1 { background: rgba(52,211,153,.75); }
|
|
136
|
+
#timeline button.g2 { background: rgba(251,191,36,.8); } #timeline button.g3 { background: rgba(244,114,182,.75); }
|
|
137
|
+
#timeline button.framed { height: 34px; }
|
|
138
|
+
#timeline .none { color: var(--muted); font-size: 12px; align-self: center; }
|
|
139
|
+
#timeline button.bad { outline: 2px solid var(--stuck); outline-offset: -2px; }
|
|
140
|
+
#timeline button[aria-pressed="true"] { outline: 2px solid #fff; outline-offset: -2px; }
|
|
141
|
+
#focus .scrub { display: flex; align-items: center; gap: 10px; color: #cbd5e1; font-size: 12px; }
|
|
142
|
+
#focus .scrub b { color: #fff; }
|
|
143
|
+
#focus .scrub button { padding: 3px 10px; }
|
|
131
144
|
#focus img { width: 100%; height: 100%; object-fit: contain; background: #000; border-radius: 6px; display: block; }
|
|
132
145
|
#focus .none { position: absolute; inset: 0; display: none; align-items: center; justify-content: center; padding: 16px;
|
|
133
146
|
color: #98a2b3; font-size: 13px; text-align: center; }
|
|
@@ -152,6 +165,18 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
152
165
|
#report .doc details { margin: 8px 0; }
|
|
153
166
|
#report .doc summary { cursor: pointer; color: var(--muted); }
|
|
154
167
|
#report .doc .unset { color: var(--muted); font-style: italic; }
|
|
168
|
+
#report .doc details.evidence { margin: 10px 0 16px; padding: 8px 12px; background: var(--panel); border: 1px solid var(--line); border-radius: 6px; }
|
|
169
|
+
#report .doc details.evidence[open] summary { margin-bottom: 10px; }
|
|
170
|
+
#report .doc details.evidence figure { margin: 0 0 12px; }
|
|
171
|
+
#report .doc details.evidence .shots { display: flex; flex-wrap: wrap; gap: 14px; }
|
|
172
|
+
#report .doc details.evidence figure { flex: 1 1 320px; max-width: 440px; }
|
|
173
|
+
#report .doc details.evidence img { display: block; width: 100%; max-height: 280px; object-fit: cover; object-position: top;
|
|
174
|
+
border: 1px solid var(--line); border-radius: 4px; background: var(--shade); }
|
|
175
|
+
#report .doc details.evidence .gone img { display: none; }
|
|
176
|
+
#report .doc details.evidence .gone-note { display: none; margin: 0 0 4px; padding: 12px; border: 1px dashed var(--line);
|
|
177
|
+
border-radius: 4px; color: var(--muted); font-size: 12px; }
|
|
178
|
+
#report .doc details.evidence .gone .gone-note { display: block; }
|
|
179
|
+
#report .doc details.evidence figcaption { margin-top: 4px; font: 11px/1.5 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; color: var(--muted); }
|
|
155
180
|
@media (prefers-reduced-motion: no-preference) { .badge.running { animation: pulse 1.6s ease-in-out infinite; } }
|
|
156
181
|
@keyframes pulse { 50% { opacity: .55; } }
|
|
157
182
|
</style>
|
|
@@ -173,7 +198,7 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
173
198
|
<h2>The run has finished</h2>
|
|
174
199
|
<p>Its browsers are closed, so there is nothing left to watch. What it found is in the report.</p>
|
|
175
200
|
<p class="where" id="finished-where" data-testid="live-finished-where"></p>
|
|
176
|
-
<
|
|
201
|
+
<p><a class="btn" href="run" id="finished-run" data-testid="live-finished-run">Open the whole run</a></p>
|
|
177
202
|
</div>
|
|
178
203
|
<main id="grid"></main>
|
|
179
204
|
<div id="report" role="dialog" aria-modal="true" aria-label="The run's report" data-testid="live-report-dialog">
|
|
@@ -196,6 +221,9 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
196
221
|
<img id="focus-img" alt="">
|
|
197
222
|
<span class="none">No frame available. The session's page may be closed, or not answering.</span>
|
|
198
223
|
</div>
|
|
224
|
+
<div class="scrub"><span id="scrub-where" data-testid="live-scrub-where">Live</span><span class="spacer"></span>
|
|
225
|
+
<button type="button" id="scrub-live" data-testid="live-scrub-live" hidden>Back to live</button></div>
|
|
226
|
+
<div id="timeline" data-testid="live-timeline" role="group" aria-label="The steps this session took"></div>
|
|
199
227
|
<div class="lower">
|
|
200
228
|
<div class="feed" id="focus-feed" data-testid="live-focus-feed"></div>
|
|
201
229
|
<aside class="brief" aria-label="What this session is doing" data-testid="live-focus-brief">
|
|
@@ -220,6 +248,10 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
220
248
|
var eventsKey = '';
|
|
221
249
|
var frames = {};
|
|
222
250
|
var hoverTask = null;
|
|
251
|
+
// A step the viewer picked from the timeline: the stage shows its frame
|
|
252
|
+
// instead of the live page, so a run can be watched back while it runs.
|
|
253
|
+
var scrubbed = null;
|
|
254
|
+
var timelineLines = [];
|
|
223
255
|
var reportOpen = false;
|
|
224
256
|
var reportTimer = null;
|
|
225
257
|
var reportProblem = null;
|
|
@@ -230,6 +262,10 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
230
262
|
var reportShown = false;
|
|
231
263
|
var reportFile = null;
|
|
232
264
|
var reportMarkdown = null;
|
|
265
|
+
// What the panel last rendered, so an unchanged report is left alone.
|
|
266
|
+
var reportKey = null;
|
|
267
|
+
// Per-finding frames from api/report: empty unless the run was recorded.
|
|
268
|
+
var reportEvidence = [];
|
|
233
269
|
var savedACopy = false;
|
|
234
270
|
// A result that reads as a failure is shown in red.
|
|
235
271
|
var BAD_RESULT = /error|fail|refus|block|violation|abandoned/i;
|
|
@@ -276,7 +312,7 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
276
312
|
frames[d.session] = src;
|
|
277
313
|
var card = cards[d.session];
|
|
278
314
|
if (card && card.live) card.img.src = src;
|
|
279
|
-
if (focused === d.session) document.getElementById('focus-img').src = src;
|
|
315
|
+
if (focused === d.session && !scrubbed) document.getElementById('focus-img').src = src;
|
|
280
316
|
});
|
|
281
317
|
events.addEventListener('unavailable', function (e) {
|
|
282
318
|
var d;
|
|
@@ -375,6 +411,35 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
375
411
|
}
|
|
376
412
|
if (last < text.length) node.appendChild(document.createTextNode(text.slice(last)));
|
|
377
413
|
}
|
|
414
|
+
/**
|
|
415
|
+
* The frames recorded around one finding, as a closed accordion. Nothing is
|
|
416
|
+
* shown when the run was not recorded: the report reads the same as before.
|
|
417
|
+
*/
|
|
418
|
+
function evidenceFor(id) {
|
|
419
|
+
var found = null;
|
|
420
|
+
reportEvidence.forEach(function (e) { if (e.id === id) found = e; });
|
|
421
|
+
if (!found || !found.frames.length) return null;
|
|
422
|
+
var box = el('details', 'evidence');
|
|
423
|
+
box.setAttribute('data-testid', 'live-report-evidence-' + id);
|
|
424
|
+
box.appendChild(el('summary', '', found.frames.length + (found.frames.length === 1 ? ' screenshot' : ' screenshots') + ' from around this finding'));
|
|
425
|
+
var shots = el('div', 'shots');
|
|
426
|
+
box.appendChild(shots);
|
|
427
|
+
found.frames.forEach(function (f) {
|
|
428
|
+
var fig = el('figure');
|
|
429
|
+
var img = el('img');
|
|
430
|
+
img.loading = 'lazy';
|
|
431
|
+
img.src = 'record/' + f.frame;
|
|
432
|
+
img.alt = f.action + ' ' + f.detail;
|
|
433
|
+
// A recording deleted since the run leaves a broken icon under a finding
|
|
434
|
+
// that still counts it as evidence. Say which it is.
|
|
435
|
+
img.addEventListener('error', function () { fig.classList.add('gone'); });
|
|
436
|
+
fig.appendChild(img);
|
|
437
|
+
fig.appendChild(el('figcaption', '', clock(f.at) + ' · ' + f.action + ' · ' + f.detail));
|
|
438
|
+
fig.appendChild(el('p', 'gone-note', 'This frame is no longer in ' + f.frame.replace(/\\/[^/]*$/, '') + '.'));
|
|
439
|
+
shots.appendChild(fig);
|
|
440
|
+
});
|
|
441
|
+
return box;
|
|
442
|
+
}
|
|
378
443
|
function renderMarkdown(root, md) {
|
|
379
444
|
root.textContent = '';
|
|
380
445
|
var lines = md.split('\\n');
|
|
@@ -390,14 +455,23 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
390
455
|
}
|
|
391
456
|
function list(tag, re) {
|
|
392
457
|
var box = el(tag);
|
|
458
|
+
var ids = [];
|
|
393
459
|
var m;
|
|
394
460
|
while (i < lines.length && (m = re.exec(lines[i]))) {
|
|
395
461
|
var li = el('li');
|
|
396
462
|
inline(li, m[1]);
|
|
397
463
|
box.appendChild(li);
|
|
464
|
+
// A finding names its id on its first bullet; the frames recorded
|
|
465
|
+
// around it hang under that list, so the proof sits with the claim.
|
|
466
|
+
var id = /^\\*\\*Id:\\*\\* \`([^\`]+)\`/.exec(m[1]);
|
|
467
|
+
if (id) ids.push(id[1]);
|
|
398
468
|
i += 1;
|
|
399
469
|
}
|
|
400
470
|
container.appendChild(box);
|
|
471
|
+
ids.forEach(function (id) {
|
|
472
|
+
var shots = evidenceFor(id);
|
|
473
|
+
if (shots) container.appendChild(shots);
|
|
474
|
+
});
|
|
401
475
|
}
|
|
402
476
|
while (i < lines.length) {
|
|
403
477
|
var line = lines[i];
|
|
@@ -482,8 +556,26 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
482
556
|
return;
|
|
483
557
|
}
|
|
484
558
|
reportMarkdown = d.markdown;
|
|
559
|
+
reportEvidence = d.evidence || [];
|
|
485
560
|
meta.textContent = (finished ? 'as the run left it at ' : 'as the run stands at ') + clock(d.at) + ' · ' + whereItIs();
|
|
561
|
+
// The report is re-read every few seconds while the run goes on.
|
|
562
|
+
// Re-rendering an unchanged document threw away what the reader was
|
|
563
|
+
// doing with it: an opened accordion shut itself, and the page jumped
|
|
564
|
+
// back to the top, every five seconds.
|
|
565
|
+
var key = d.markdown + '\\u0000' + JSON.stringify(reportEvidence);
|
|
566
|
+
if (key === reportKey) return;
|
|
567
|
+
var open = {};
|
|
568
|
+
var was = doc.querySelectorAll('details[open][data-testid]');
|
|
569
|
+
for (var k = 0; k < was.length; k += 1) open[was[k].getAttribute('data-testid')] = true;
|
|
570
|
+
var top = doc.parentNode ? doc.parentNode.scrollTop : 0;
|
|
571
|
+
reportKey = key;
|
|
486
572
|
renderMarkdown(doc, d.markdown);
|
|
573
|
+
// A finding whose evidence the reader had open stays open through a change.
|
|
574
|
+
Object.keys(open).forEach(function (id) {
|
|
575
|
+
var node = doc.querySelector('details[data-testid="' + id + '"]');
|
|
576
|
+
if (node) node.open = true;
|
|
577
|
+
});
|
|
578
|
+
if (doc.parentNode) doc.parentNode.scrollTop = top;
|
|
487
579
|
})
|
|
488
580
|
.catch(function (err) {
|
|
489
581
|
// The last rendering stays; with none, say why there is nothing to read.
|
|
@@ -531,6 +623,65 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
531
623
|
document.getElementById('report-open').focus();
|
|
532
624
|
}
|
|
533
625
|
|
|
626
|
+
// One tick per step, coloured by the task it served: a run can be scrolled
|
|
627
|
+
// back through while it is still going, and a step with a frame is taller.
|
|
628
|
+
function renderTimeline(lines) {
|
|
629
|
+
// Kept so a re-render (picking a step, say) draws the same run back. The
|
|
630
|
+
// status poll carries six lines and the close-up three hundred: redrawing
|
|
631
|
+
// from whichever was nearest to hand shrank the timeline to six ticks.
|
|
632
|
+
timelineLines = lines || timelineLines;
|
|
633
|
+
lines = timelineLines;
|
|
634
|
+
var node = document.getElementById('timeline');
|
|
635
|
+
var atEnd = node.scrollLeft + node.clientWidth >= node.scrollWidth - 8;
|
|
636
|
+
node.textContent = '';
|
|
637
|
+
var groups = 0;
|
|
638
|
+
var last = null;
|
|
639
|
+
(lines || []).forEach(function (line) {
|
|
640
|
+
var task = line.task || '';
|
|
641
|
+
if (task && task !== last) groups += 1;
|
|
642
|
+
last = task;
|
|
643
|
+
var tick = el('button');
|
|
644
|
+
tick.type = 'button';
|
|
645
|
+
tick.className = (task ? 'g' + ((groups - 1) % 4) : '') + (line.frame ? ' framed' : '') + (BAD_RESULT.test(line.result || '') ? ' bad' : '');
|
|
646
|
+
tick.title = clock(line.at) + ' ' + line.action + (line.target ? ' ' + line.target : '') + (task ? '\\n' + task : '');
|
|
647
|
+
tick.setAttribute('aria-label', tick.title);
|
|
648
|
+
tick.setAttribute('aria-pressed', scrubbed && scrubbed.at === line.at && scrubbed.action === line.action ? 'true' : 'false');
|
|
649
|
+
tick.addEventListener('click', function () { showStep(line); });
|
|
650
|
+
node.appendChild(tick);
|
|
651
|
+
});
|
|
652
|
+
if (atEnd) node.scrollLeft = node.scrollWidth;
|
|
653
|
+
}
|
|
654
|
+
|
|
655
|
+
// Show one step: its frame if the run was recorded, and what it was doing.
|
|
656
|
+
function showStep(line) {
|
|
657
|
+
var where = document.getElementById('scrub-where');
|
|
658
|
+
var back = document.getElementById('scrub-live');
|
|
659
|
+
scrubbed = line;
|
|
660
|
+
back.hidden = false;
|
|
661
|
+
if (!line.frame) {
|
|
662
|
+
where.textContent = clock(line.at) + ' ' + line.action + ' — this run kept no frame for that step';
|
|
663
|
+
// Leaving the live frame up under that caption is the most misleading
|
|
664
|
+
// state this page can reach: it reads as the page at that moment.
|
|
665
|
+
document.getElementById('focus-img').removeAttribute('src');
|
|
666
|
+
document.getElementById('focus-stage').classList.add('empty');
|
|
667
|
+
renderTimeline(null);
|
|
668
|
+
return;
|
|
669
|
+
}
|
|
670
|
+
document.getElementById('focus-stage').classList.remove('empty');
|
|
671
|
+
document.getElementById('focus-img').src = 'record/' + line.frame;
|
|
672
|
+
where.textContent = clock(line.at) + ' ' + line.action + (line.task ? ' · ' + line.task : '');
|
|
673
|
+
renderTimeline(null);
|
|
674
|
+
}
|
|
675
|
+
|
|
676
|
+
function backToLive() {
|
|
677
|
+
scrubbed = null;
|
|
678
|
+
document.getElementById('scrub-where').textContent = 'Live';
|
|
679
|
+
document.getElementById('scrub-live').hidden = true;
|
|
680
|
+
if (focused) document.getElementById('focus-img').src = frames[focused] || shotUrl(focused);
|
|
681
|
+
// Otherwise the step just left keeps its outline until the next full feed.
|
|
682
|
+
renderTimeline(null);
|
|
683
|
+
}
|
|
684
|
+
|
|
534
685
|
// Built with textContent only: an action's target is text from the app under test.
|
|
535
686
|
function renderFeed(node, lines, onGroup) {
|
|
536
687
|
// Measured before the node is emptied: an empty node always reads as scrolled to its end.
|
|
@@ -589,10 +740,17 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
589
740
|
|
|
590
741
|
function openFocus(name) {
|
|
591
742
|
focused = name;
|
|
743
|
+
scrubbed = null;
|
|
744
|
+
// The ticks belong to the session just left; showing them under this one's
|
|
745
|
+
// name, and playing its frames when one is clicked, is worse than none.
|
|
746
|
+
timelineLines = [];
|
|
747
|
+
renderTimeline([]);
|
|
592
748
|
document.getElementById('focus-stage').classList.remove('empty');
|
|
593
749
|
document.getElementById('focus-name').textContent = name;
|
|
594
750
|
document.getElementById('focus-img').alt = 'Live view of ' + name;
|
|
595
751
|
document.getElementById('focus-img').src = frames[name] || shotUrl(name);
|
|
752
|
+
document.getElementById('scrub-where').textContent = 'Live';
|
|
753
|
+
document.getElementById('scrub-live').hidden = true;
|
|
596
754
|
document.getElementById('focus').classList.add('open');
|
|
597
755
|
hoverTask = null;
|
|
598
756
|
renderFeed(document.getElementById('focus-feed'), (latest[name] || {}).feed, showTask);
|
|
@@ -612,8 +770,21 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
612
770
|
function loadFullFeed(name) {
|
|
613
771
|
fetch('api/activity?session=' + encodeURIComponent(name), { cache: 'no-store' })
|
|
614
772
|
.then(function (r) { return r.ok ? r.json() : null; })
|
|
615
|
-
.then(function (d) {
|
|
616
|
-
|
|
773
|
+
.then(function (d) {
|
|
774
|
+
if (!d || focused !== d.session) return;
|
|
775
|
+
renderFeed(document.getElementById('focus-feed'), d.feed, showTask);
|
|
776
|
+
renderTimeline(d.feed);
|
|
777
|
+
})
|
|
778
|
+
.catch(function () {
|
|
779
|
+
// The short feed from the last poll stays on screen, but the timeline
|
|
780
|
+
// is drawn from this call alone: an empty strip would read as a
|
|
781
|
+
// session that did nothing.
|
|
782
|
+
if (!timelineLines.length) {
|
|
783
|
+
var strip = document.getElementById('timeline');
|
|
784
|
+
strip.textContent = '';
|
|
785
|
+
strip.appendChild(el('span', 'none', 'The step list could not be loaded. Trying again.'));
|
|
786
|
+
}
|
|
787
|
+
});
|
|
617
788
|
}
|
|
618
789
|
|
|
619
790
|
function setBrief(id, text, unset) {
|
|
@@ -685,7 +856,10 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
685
856
|
});
|
|
686
857
|
syncEvents();
|
|
687
858
|
reportFile = snap.report || reportFile;
|
|
688
|
-
|
|
859
|
+
// A viewer who opens the board after the last browser closed never sees a
|
|
860
|
+
// session, but the engine only names a report file once one has attached:
|
|
861
|
+
// that is a finished run, not a board waiting for its first attach.
|
|
862
|
+
if (snap.sessions.length > 0 || snap.report) sawRun = true;
|
|
689
863
|
finished = sawRun && snap.sessions.length === 0;
|
|
690
864
|
document.getElementById('empty').style.display = snap.sessions.length || finished ? 'none' : 'block';
|
|
691
865
|
document.getElementById('finished').classList.toggle('open', finished);
|
|
@@ -693,10 +867,18 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
693
867
|
var where = document.getElementById('finished-where');
|
|
694
868
|
where.textContent = whereItIs();
|
|
695
869
|
where.className = 'where' + (reportFile && reportFile.written ? '' : ' unset');
|
|
696
|
-
// The moment somebody wants the
|
|
870
|
+
// The moment somebody wants the run is the moment it ends, and a panel
|
|
871
|
+
// over a dead board is lost on the next refresh. The run has its own
|
|
872
|
+
// address; go there, so the address IS the report from then on.
|
|
697
873
|
if (!reportShown) {
|
|
698
874
|
reportShown = true;
|
|
699
|
-
|
|
875
|
+
// The run has its own address, and going there means a refresh shows
|
|
876
|
+
// the report instead of a dead board. Ask whether it is there first:
|
|
877
|
+
// a run with no page to serve would otherwise land on a 404, and the
|
|
878
|
+
// panel below is the only other copy.
|
|
879
|
+
fetch('run', { cache: 'no-store' })
|
|
880
|
+
.then(function (r) { if (r.ok) location.href = 'run'; else openReport(); })
|
|
881
|
+
.catch(function () { openReport(); });
|
|
700
882
|
}
|
|
701
883
|
}
|
|
702
884
|
document.getElementById('engine').textContent = 'engine pid ' + snap.pid + ' · v' + snap.version;
|
|
@@ -719,14 +901,22 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
719
901
|
Object.keys(cards).forEach(function (name) { setLive(cards[name], streamAll); });
|
|
720
902
|
});
|
|
721
903
|
// A session with nothing to show answers 503; say so rather than leaving a broken-image icon.
|
|
722
|
-
document.getElementById('focus-img').addEventListener('error', function () {
|
|
904
|
+
document.getElementById('focus-img').addEventListener('error', function () {
|
|
905
|
+
document.getElementById('focus-stage').classList.add('empty');
|
|
906
|
+
// A live session with nothing to show and a recorded frame that has since
|
|
907
|
+
// been deleted look identical here; only the second is worth explaining.
|
|
908
|
+
if (scrubbed) {
|
|
909
|
+
document.getElementById('scrub-where').textContent =
|
|
910
|
+
clock(scrubbed.at) + ' ' + scrubbed.action + ' — its frame could not be loaded; the recording may have been removed';
|
|
911
|
+
}
|
|
912
|
+
});
|
|
723
913
|
document.getElementById('focus-img').addEventListener('load', function () { document.getElementById('focus-stage').classList.remove('empty'); });
|
|
724
914
|
document.getElementById('focus-close').addEventListener('click', closeFocus);
|
|
725
915
|
// A re-rendered feed replaces the group under the pointer without a mouseleave; leaving the feed itself still resets.
|
|
726
916
|
document.getElementById('focus-feed').addEventListener('mouseleave', function () { showTask(null); });
|
|
727
917
|
document.getElementById('focus').addEventListener('click', function (e) { if (e.target === this) closeFocus(); });
|
|
918
|
+
document.getElementById('scrub-live').addEventListener('click', backToLive);
|
|
728
919
|
document.getElementById('report-open').addEventListener('click', openReport);
|
|
729
|
-
document.getElementById('finished-report').addEventListener('click', openReport);
|
|
730
920
|
document.getElementById('report-save').addEventListener('click', saveACopy);
|
|
731
921
|
// Closing the tab on a finished run whose report was never written to disk
|
|
732
922
|
// throws the only copy away. The browser shows its own confirm/dismiss, and
|
package/dist/engine/live.js
CHANGED
|
@@ -80,6 +80,8 @@ export function feedForSession(log, session, limit, redact = (s) => s) {
|
|
|
80
80
|
line.target = e.target;
|
|
81
81
|
if (e.result !== undefined)
|
|
82
82
|
line.result = e.result;
|
|
83
|
+
if (e.frame !== undefined)
|
|
84
|
+
line.frame = e.frame;
|
|
83
85
|
if (task !== undefined)
|
|
84
86
|
line.task = task;
|
|
85
87
|
lines.push(line);
|
|
@@ -90,8 +92,13 @@ export function feedForSession(log, session, limit, redact = (s) => s) {
|
|
|
90
92
|
}
|
|
91
93
|
/** How many feed lines a status poll carries per session. Enough to read the last move at a glance; the close-up asks for more. */
|
|
92
94
|
export const FEED_LINES = 6;
|
|
93
|
-
/**
|
|
94
|
-
|
|
95
|
+
/**
|
|
96
|
+
* The cap the close-up gets. A long run's log is thousands of lines and none
|
|
97
|
+
* of it needs to reach the page — but this is also what the timeline is drawn
|
|
98
|
+
* from, and a timeline that only reaches back a minute cannot be scrubbed.
|
|
99
|
+
* The whole run, every step of it, is the page at `run`.
|
|
100
|
+
*/
|
|
101
|
+
export const FEED_LINES_MAX = 300;
|
|
95
102
|
/**
|
|
96
103
|
* How long a call may run before the session is judged wedged rather than
|
|
97
104
|
* busy, when the entry does not carry the tool's own watchdog budget (an
|
|
@@ -251,6 +258,13 @@ export function watchTarget(input) {
|
|
|
251
258
|
};
|
|
252
259
|
return { url: `http://127.0.0.1:${port}/${clean}/` };
|
|
253
260
|
}
|
|
261
|
+
/** What `run` says when there is no run to show: a page, because a person navigated here. */
|
|
262
|
+
export const NO_RUN_PAGE = '<!doctype html><html lang="en"><head><meta charset="utf-8"><title>No run to show</title>' +
|
|
263
|
+
"<style>body{margin:0;display:grid;place-items:center;min-height:100vh;background:#0e1116;color:#e6e9ee;" +
|
|
264
|
+
'font:16px/1.6 system-ui,-apple-system,"Segoe UI",sans-serif}main{max-width:34rem;padding:2rem;text-align:center}' +
|
|
265
|
+
"a{color:#7aa2ff}</style></head><body><main><h1>Nothing to show yet</h1>" +
|
|
266
|
+
"<p>This run has not produced a report. It may not have started, or its engine may have exited since you opened this page.</p>" +
|
|
267
|
+
'<p><a href="./">Back to the live board</a></p></main></body></html>';
|
|
254
268
|
const LIVE_CSP = "default-src 'none'; img-src 'self' data:; style-src 'unsafe-inline'; script-src 'unsafe-inline'; connect-src 'self'; base-uri 'none'; form-action 'none'";
|
|
255
269
|
const BOUNDARY = "scenescoutframe";
|
|
256
270
|
/** A thumbnail younger than this is served again instead of taking a new one. */
|
|
@@ -381,6 +395,30 @@ export class LiveServer {
|
|
|
381
395
|
return this.send(res, 404, "no run attached");
|
|
382
396
|
return this.send(res, 200, JSON.stringify(report), "application/json; charset=utf-8");
|
|
383
397
|
}
|
|
398
|
+
if (route === "run" && !name) {
|
|
399
|
+
const doc = this.provider.replay();
|
|
400
|
+
// Somebody clicked a button or refreshed a bookmark to get here, so an
|
|
401
|
+
// answer they cannot read is worse than no link at all.
|
|
402
|
+
if (!doc)
|
|
403
|
+
return this.send(res, 404, NO_RUN_PAGE, "text/html; charset=utf-8", { "Content-Security-Policy": LIVE_CSP });
|
|
404
|
+
return this.send(res, 200, doc, "text/html; charset=utf-8", { "Content-Security-Policy": LIVE_CSP });
|
|
405
|
+
}
|
|
406
|
+
if (route === "record" && name) {
|
|
407
|
+
// A recorded frame, by the path the feed gave. The provider decides what
|
|
408
|
+
// exists; nothing here builds a filesystem path from the request.
|
|
409
|
+
const rel = parts.slice(2).join("/");
|
|
410
|
+
let asked;
|
|
411
|
+
try {
|
|
412
|
+
asked = decodeURIComponent(rel);
|
|
413
|
+
}
|
|
414
|
+
catch {
|
|
415
|
+
return this.send(res, 404, "not found");
|
|
416
|
+
}
|
|
417
|
+
const jpeg = await this.provider.frame(asked);
|
|
418
|
+
if (!jpeg)
|
|
419
|
+
return this.send(res, 404, "no such frame");
|
|
420
|
+
return this.send(res, 200, jpeg, "image/jpeg", { "Content-Length": jpeg.length, "Cache-Control": "private, max-age=3600" });
|
|
421
|
+
}
|
|
384
422
|
if (route === "events" && !name) {
|
|
385
423
|
const wanted = this.param(req, "sessions");
|
|
386
424
|
if (wanted === null)
|
|
@@ -0,0 +1,348 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The run, as one file that outlives it.
|
|
3
|
+
*
|
|
4
|
+
* The live view dies with the engine: its address is a port in a process, so
|
|
5
|
+
* refreshing after a run is over gets nothing. This builds a self-contained
|
|
6
|
+
* HTML document instead — the report, and every session's steps grouped by
|
|
7
|
+
* the task they served — written next to report.md. It opens from the file
|
|
8
|
+
* system with no server, works offline, and can be handed to somebody who was
|
|
9
|
+
* never watching.
|
|
10
|
+
*
|
|
11
|
+
* Frames appear beside the steps only when the run was recorded
|
|
12
|
+
* (`scout_attach {record:true}`), which is off by default: a recording is
|
|
13
|
+
* pictures of somebody's app sitting in their project folder, and ADR 7's
|
|
14
|
+
* "no frame touches the disk" is the rule it deliberately relaxes. The
|
|
15
|
+
* document is built the same way either way; a step with no frame simply
|
|
16
|
+
* shows none.
|
|
17
|
+
*
|
|
18
|
+
* Everything here is pure string work — no browser, no filesystem — so the
|
|
19
|
+
* escaping and the grouping are table-tested.
|
|
20
|
+
*/
|
|
21
|
+
import path from "node:path";
|
|
22
|
+
/** Most frames one recorded session keeps. A long run is thousands of actions, and a project folder is not a video store. */
|
|
23
|
+
export const RECORD_MAX_FRAMES = 600;
|
|
24
|
+
/**
|
|
25
|
+
* Where a recorded frame is stored, relative to the memory directory — and
|
|
26
|
+
* the path the live view serves it at, so it is always written with forward
|
|
27
|
+
* slashes. The session name is the agent's and the action the tool's, so both
|
|
28
|
+
* are reduced to a plain file name here: a session called `../../etc` decides
|
|
29
|
+
* nothing about where the engine writes.
|
|
30
|
+
*/
|
|
31
|
+
export function framePath(session, index, action) {
|
|
32
|
+
const plain = (text, fallback) => text
|
|
33
|
+
.replace(/[^a-z0-9._-]+/gi, "-")
|
|
34
|
+
.replace(/^[.-]+/, "")
|
|
35
|
+
.slice(0, 60) || fallback;
|
|
36
|
+
return `recordings/${plain(session, "session")}/${String(index).padStart(4, "0")}-${plain(action, "step")}.jpg`;
|
|
37
|
+
}
|
|
38
|
+
/**
|
|
39
|
+
* The file a viewer's frame request names, or null when it is not one of this
|
|
40
|
+
* run's frames. The path comes from a browser and may be anything, so it is
|
|
41
|
+
* resolved and then required to still be under the recordings directory —
|
|
42
|
+
* `..`, an absolute path and a sibling directory whose name merely starts the
|
|
43
|
+
* same way are all refused. The one filesystem rule in the frame route, kept
|
|
44
|
+
* here so it can be table-tested rather than reached only through HTTP.
|
|
45
|
+
*/
|
|
46
|
+
export function resolveFrame(root, relPath) {
|
|
47
|
+
if (!relPath)
|
|
48
|
+
return null;
|
|
49
|
+
const inside = relPath.replace(/^recordings[\\/]/, "");
|
|
50
|
+
if (!inside || inside.includes("\0"))
|
|
51
|
+
return null;
|
|
52
|
+
// Both sides resolved: on Windows `path.resolve` prefixes the drive, so
|
|
53
|
+
// comparing its output against a root that has none rejects every frame.
|
|
54
|
+
const base = path.resolve(root);
|
|
55
|
+
const file = path.resolve(base, inside);
|
|
56
|
+
return file.startsWith(base + path.sep) ? file : null;
|
|
57
|
+
}
|
|
58
|
+
/** Text from the app under test reaches this document, so nothing is interpolated unescaped. */
|
|
59
|
+
export function escapeHtml(text) {
|
|
60
|
+
return text.replace(/[&<>"']/g, (c) => ({ "&": "&", "<": "<", ">": ">", '"': """, "'": "'" })[c]);
|
|
61
|
+
}
|
|
62
|
+
/**
|
|
63
|
+
* A session's steps in the blocks its tasks made. Consecutive steps that
|
|
64
|
+
* served the same task are one block, so the document reads the way the live
|
|
65
|
+
* feed did: a change of task is a change of block, and of colour.
|
|
66
|
+
*/
|
|
67
|
+
export function taskBlocks(steps) {
|
|
68
|
+
const blocks = [];
|
|
69
|
+
for (const step of steps) {
|
|
70
|
+
const task = step.task ?? null;
|
|
71
|
+
const last = blocks[blocks.length - 1];
|
|
72
|
+
if (!last || last.task !== task)
|
|
73
|
+
blocks.push({ task, steps: [step] });
|
|
74
|
+
else
|
|
75
|
+
last.steps.push(step);
|
|
76
|
+
}
|
|
77
|
+
return blocks;
|
|
78
|
+
}
|
|
79
|
+
/** The viewer's own clock is not this document's to assume; times are shown as they were logged. */
|
|
80
|
+
function clock(iso) {
|
|
81
|
+
const at = iso.length >= 19 ? iso.slice(11, 19) : iso;
|
|
82
|
+
return at;
|
|
83
|
+
}
|
|
84
|
+
/**
|
|
85
|
+
* When the document was written. The log is UTC and this page may be opened
|
|
86
|
+
* anywhere, so it says which clock it is quoting rather than implying the
|
|
87
|
+
* reader's own.
|
|
88
|
+
*/
|
|
89
|
+
function stamp(iso) {
|
|
90
|
+
return iso.length >= 16 ? `${iso.slice(0, 10)} ${iso.slice(11, 16)} UTC` : iso;
|
|
91
|
+
}
|
|
92
|
+
/**
|
|
93
|
+
* What an image that will not load does. This page is meant to be handed on,
|
|
94
|
+
* and a copy sent without its `recordings/` folder would otherwise show a row
|
|
95
|
+
* of broken icons under a finding that still counts them as evidence.
|
|
96
|
+
*/
|
|
97
|
+
const GONE = `onerror="this.parentNode.classList.add('gone')"`;
|
|
98
|
+
/** A result that reads as a failure, so a step that went wrong is visible without reading every line. */
|
|
99
|
+
const BAD_RESULT = /error|fail|refus|block|violation|abandoned/i;
|
|
100
|
+
function renderStep(step, framePrefix = "", savedAt = "") {
|
|
101
|
+
const detail = [step.target, step.result ? `→ ${step.result}` : ""].filter(Boolean).join(" ") || step.url || "";
|
|
102
|
+
const bad = BAD_RESULT.test(step.result ?? "") ? " bad" : "";
|
|
103
|
+
const frame = step.frame
|
|
104
|
+
? `<a class="frame" href="${escapeHtml(framePrefix + step.frame)}" target="_blank" rel="noreferrer"><img loading="lazy" ${GONE} src="${escapeHtml(framePrefix + step.frame)}" alt="What the page showed at this step"></a>`
|
|
105
|
+
: "";
|
|
106
|
+
const where = savedAt && step.frame ? `<p class="onDisk">${escapeHtml(savedAt + "/" + step.frame)}</p>` : "";
|
|
107
|
+
return (`<li class="step"><div class="row"><span class="t">${escapeHtml(clock(step.at))}</span>` +
|
|
108
|
+
`<span class="a${bad}">${escapeHtml(step.action)}</span>` +
|
|
109
|
+
`<span class="d" title="${escapeHtml([step.target, step.result, step.url].filter(Boolean).join(" · "))}">${escapeHtml(detail)}</span></div>${frame}${where}</li>`);
|
|
110
|
+
}
|
|
111
|
+
function renderSession(s, index, framePrefix = "", savedAt = "") {
|
|
112
|
+
const blocks = taskBlocks(s.steps);
|
|
113
|
+
const body = blocks
|
|
114
|
+
.map((b, i) => {
|
|
115
|
+
const head = b.task ? `<p class="task">${escapeHtml(b.task)}</p>` : `<p class="task none">No task stated for these</p>`;
|
|
116
|
+
return `<section class="block g${i % 4}">${head}<ol class="steps">${b.steps.map((step) => renderStep(step, framePrefix, savedAt)).join("")}</ol></section>`;
|
|
117
|
+
})
|
|
118
|
+
.join("");
|
|
119
|
+
const framed = s.steps.filter((x) => x.frame).length;
|
|
120
|
+
return (`<details class="session"${index === 0 ? " open" : ""}><summary><b>${escapeHtml(s.session)}</b> <span class="role">${escapeHtml(s.role)}</span>` +
|
|
121
|
+
`<span class="count">${s.steps.length} steps · ${blocks.length} task${blocks.length === 1 ? "" : "s"}${framed ? ` · ${framed} frames` : ""}</span></summary>` +
|
|
122
|
+
(s.objective ? `<p class="objective">${escapeHtml(s.objective)}</p>` : "") +
|
|
123
|
+
body +
|
|
124
|
+
`</details>`);
|
|
125
|
+
}
|
|
126
|
+
/**
|
|
127
|
+
* The steps that were on screen just before a finding was filed. The report's
|
|
128
|
+
* own repro trace says what happened; on a recorded run these show it.
|
|
129
|
+
*/
|
|
130
|
+
export function evidenceFor(steps, foundAt, most = 4) {
|
|
131
|
+
const before = steps.filter((x) => x.frame && x.at <= foundAt);
|
|
132
|
+
return before.slice(-most).map((x) => ({
|
|
133
|
+
at: x.at,
|
|
134
|
+
action: x.action,
|
|
135
|
+
detail: [x.target, x.result ? `→ ${x.result}` : ""].filter(Boolean).join(" ") || x.url || "",
|
|
136
|
+
frame: x.frame,
|
|
137
|
+
}));
|
|
138
|
+
}
|
|
139
|
+
function renderEvidence(e, framePrefix = "", savedAt = "") {
|
|
140
|
+
if (e.frames.length === 0)
|
|
141
|
+
return "";
|
|
142
|
+
const shots = e.frames
|
|
143
|
+
.map((f) => `<figure><img loading="lazy" ${GONE} src="${escapeHtml(framePrefix + f.frame)}" alt="The page when this step ran">` +
|
|
144
|
+
`<p class="gone-note">This frame is not beside this file. Frames live in the run's <code>recordings/</code> folder, which travels with it.</p>` +
|
|
145
|
+
`<figcaption>${escapeHtml(clock(f.at))} <b>${escapeHtml(f.action)}</b> ${escapeHtml(f.detail)}` +
|
|
146
|
+
(savedAt ? `<span class="onDisk">${escapeHtml(savedAt + "/" + f.frame)}</span>` : "") +
|
|
147
|
+
`</figcaption></figure>`)
|
|
148
|
+
.join("");
|
|
149
|
+
return `<details class="evidence"><summary>Evidence — the ${e.frames.length} step${e.frames.length === 1 ? "" : "s"} on screen before this was filed</summary><div class="shots">${shots}</div></details>`;
|
|
150
|
+
}
|
|
151
|
+
/**
|
|
152
|
+
* The report's markdown as elements. A deliberately small subset — headings,
|
|
153
|
+
* tables, lists, code fences, the report's own <details> repro blocks — built
|
|
154
|
+
* by escaping every piece of text, because finding titles and element names
|
|
155
|
+
* come from the app under test.
|
|
156
|
+
*/
|
|
157
|
+
export function renderMarkdown(md, evidence = [], framePrefix = "", savedAt = "") {
|
|
158
|
+
const out = [];
|
|
159
|
+
const lines = md.split("\n");
|
|
160
|
+
let i = 0;
|
|
161
|
+
let para = [];
|
|
162
|
+
const inline = (text) => escapeHtml(text)
|
|
163
|
+
.replace(/`([^`]+)`/g, "<code>$1</code>")
|
|
164
|
+
.replace(/\*\*([^*]+)\*\*/g, "<strong>$1</strong>");
|
|
165
|
+
const flush = () => {
|
|
166
|
+
if (para.length)
|
|
167
|
+
out.push(`<p>${inline(para.join(" "))}</p>`);
|
|
168
|
+
para = [];
|
|
169
|
+
};
|
|
170
|
+
while (i < lines.length) {
|
|
171
|
+
const line = lines[i];
|
|
172
|
+
let m;
|
|
173
|
+
if (/^```/.test(line)) {
|
|
174
|
+
flush();
|
|
175
|
+
const code = [];
|
|
176
|
+
i += 1;
|
|
177
|
+
while (i < lines.length && !/^```/.test(lines[i])) {
|
|
178
|
+
code.push(lines[i]);
|
|
179
|
+
i += 1;
|
|
180
|
+
}
|
|
181
|
+
i += 1;
|
|
182
|
+
out.push(`<pre><code>${escapeHtml(code.join("\n"))}</code></pre>`);
|
|
183
|
+
}
|
|
184
|
+
else if ((m = /^(#{1,6}) (.*)$/.exec(line))) {
|
|
185
|
+
flush();
|
|
186
|
+
const level = Math.min(6, m[1].length + 1);
|
|
187
|
+
out.push(`<h${level}>${inline(m[2])}</h${level}>`);
|
|
188
|
+
i += 1;
|
|
189
|
+
}
|
|
190
|
+
else if ((m = /^<details><summary>(.*)<\/summary>$/.exec(line))) {
|
|
191
|
+
flush();
|
|
192
|
+
out.push(`<details><summary>${inline(m[1])}</summary>`);
|
|
193
|
+
i += 1;
|
|
194
|
+
}
|
|
195
|
+
else if (/^<\/details>$/.test(line)) {
|
|
196
|
+
flush();
|
|
197
|
+
out.push(`</details>`);
|
|
198
|
+
i += 1;
|
|
199
|
+
}
|
|
200
|
+
else if (/^\|/.test(line)) {
|
|
201
|
+
flush();
|
|
202
|
+
const rows = [];
|
|
203
|
+
let first = true;
|
|
204
|
+
while (i < lines.length && /^\|/.test(lines[i])) {
|
|
205
|
+
const row = lines[i];
|
|
206
|
+
i += 1;
|
|
207
|
+
if (/^\|(\s*:?-+:?\s*\|)+\s*$/.test(row))
|
|
208
|
+
continue;
|
|
209
|
+
const cells = row
|
|
210
|
+
.replace(/^\||\|\s*$/g, "")
|
|
211
|
+
.split("|")
|
|
212
|
+
.map((c) => `<${first ? "th" : "td"}>${inline(c.trim())}</${first ? "th" : "td"}>`);
|
|
213
|
+
rows.push(`<tr>${cells.join("")}</tr>`);
|
|
214
|
+
first = false;
|
|
215
|
+
}
|
|
216
|
+
out.push(`<table>${rows.join("")}</table>`);
|
|
217
|
+
}
|
|
218
|
+
else if (/^\s*[-*] /.test(line)) {
|
|
219
|
+
flush();
|
|
220
|
+
const items = [];
|
|
221
|
+
// A finding's id sits in this list; its evidence goes straight under it,
|
|
222
|
+
// which is where a reader is when they ask "show me".
|
|
223
|
+
let found;
|
|
224
|
+
while (i < lines.length && (m = /^\s*[-*] (.*)$/.exec(lines[i]))) {
|
|
225
|
+
const id = /\*\*Id:\*\* `([0-9a-f]+)`/.exec(m[1])?.[1];
|
|
226
|
+
if (id)
|
|
227
|
+
found = evidence.find((e) => e.id === id);
|
|
228
|
+
items.push(`<li>${inline(m[1])}</li>`);
|
|
229
|
+
i += 1;
|
|
230
|
+
}
|
|
231
|
+
out.push(`<ul>${items.join("")}</ul>`);
|
|
232
|
+
if (found)
|
|
233
|
+
out.push(renderEvidence(found, framePrefix, savedAt));
|
|
234
|
+
}
|
|
235
|
+
else if (/^\d+\. /.test(line)) {
|
|
236
|
+
flush();
|
|
237
|
+
const items = [];
|
|
238
|
+
while (i < lines.length && (m = /^\d+\. (.*)$/.exec(lines[i]))) {
|
|
239
|
+
items.push(`<li>${inline(m[1])}</li>`);
|
|
240
|
+
i += 1;
|
|
241
|
+
}
|
|
242
|
+
out.push(`<ol>${items.join("")}</ol>`);
|
|
243
|
+
}
|
|
244
|
+
else if (line.trim() === "") {
|
|
245
|
+
flush();
|
|
246
|
+
i += 1;
|
|
247
|
+
}
|
|
248
|
+
else {
|
|
249
|
+
para.push(line);
|
|
250
|
+
i += 1;
|
|
251
|
+
}
|
|
252
|
+
}
|
|
253
|
+
flush();
|
|
254
|
+
return out.join("\n");
|
|
255
|
+
}
|
|
256
|
+
const STYLE = `
|
|
257
|
+
:root { color-scheme: light dark; --bg:#f6f7f9; --panel:#fff; --line:#d9dde3; --text:#15181d; --muted:#5d6673; --accent:#2563eb; --bad:#b91c1c; }
|
|
258
|
+
@media (prefers-color-scheme: dark) { :root { --bg:#0e1116; --panel:#161a21; --line:#2a303a; --text:#e6e9ee; --muted:#98a2b3; --accent:#7aa2ff; --bad:#fca5a5; } }
|
|
259
|
+
* { box-sizing: border-box; }
|
|
260
|
+
body { margin:0; background:var(--bg); color:var(--text); font:15px/1.55 system-ui,-apple-system,"Segoe UI",sans-serif; }
|
|
261
|
+
header { position:sticky; top:0; z-index:2; display:flex; flex-wrap:wrap; gap:8px 16px; align-items:baseline; padding:14px 20px; background:var(--panel); border-bottom:1px solid var(--line); }
|
|
262
|
+
header h1 { margin:0; font-size:17px; }
|
|
263
|
+
header .meta { color:var(--muted); font-size:13px; }
|
|
264
|
+
nav { margin-left:auto; display:flex; gap:12px; font-size:14px; }
|
|
265
|
+
nav a { color:var(--accent); }
|
|
266
|
+
main { max-width:1000px; margin:0 auto; padding:24px 20px 64px; }
|
|
267
|
+
h2 { font-size:21px; margin:32px 0 12px; padding-top:20px; border-top:1px solid var(--line); }
|
|
268
|
+
h3 { font-size:17px; margin:24px 0 8px; }
|
|
269
|
+
h4 { font-size:15px; margin:20px 0 6px; }
|
|
270
|
+
table { border-collapse:collapse; margin:10px 0 16px; font-size:13px; }
|
|
271
|
+
th, td { border:1px solid var(--line); padding:5px 9px; text-align:left; vertical-align:top; }
|
|
272
|
+
th { background:var(--panel); }
|
|
273
|
+
code { font:12px/1.4 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace; background:var(--panel); border:1px solid var(--line); border-radius:4px; padding:0 4px; }
|
|
274
|
+
pre { padding:12px; background:var(--panel); border:1px solid var(--line); border-radius:6px; overflow-x:auto; }
|
|
275
|
+
pre code { border:0; padding:0; background:none; }
|
|
276
|
+
details.session { background:var(--panel); border:1px solid var(--line); border-radius:8px; margin:12px 0; padding:10px 14px; }
|
|
277
|
+
details.session > summary { cursor:pointer; font-size:15px; display:flex; gap:10px; align-items:baseline; }
|
|
278
|
+
details.session .role { color:var(--muted); font-size:13px; }
|
|
279
|
+
details.session .count { margin-left:auto; color:var(--muted); font-size:12px; }
|
|
280
|
+
.objective { margin:8px 0 14px; color:var(--muted); }
|
|
281
|
+
.block { border-left:2px solid transparent; border-radius:4px; padding:6px 10px; margin:8px 0; }
|
|
282
|
+
.block.g0 { background:rgba(96,165,250,.22); border-color:rgba(96,165,250,.85); }
|
|
283
|
+
.block.g1 { background:rgba(52,211,153,.22); border-color:rgba(52,211,153,.85); }
|
|
284
|
+
.block.g2 { background:rgba(251,191,36,.24); border-color:rgba(251,191,36,.9); }
|
|
285
|
+
.block.g3 { background:rgba(244,114,182,.22); border-color:rgba(244,114,182,.85); }
|
|
286
|
+
@media (prefers-color-scheme: light) {
|
|
287
|
+
.block.g0 { background:rgba(37,99,235,.13); } .block.g1 { background:rgba(5,150,105,.13); }
|
|
288
|
+
.block.g2 { background:rgba(217,119,6,.15); } .block.g3 { background:rgba(219,39,119,.12); }
|
|
289
|
+
}
|
|
290
|
+
.task { margin:2px 0 8px; font-weight:600; }
|
|
291
|
+
.task.none { font-weight:400; color:var(--muted); font-style:italic; }
|
|
292
|
+
ol.steps { list-style:none; margin:0; padding:0; }
|
|
293
|
+
.step { margin:0 0 2px; }
|
|
294
|
+
.step .row { display:flex; gap:8px; font:12px/1.6 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace; }
|
|
295
|
+
.step .t { color:var(--muted); flex:0 0 auto; }
|
|
296
|
+
.step .a { font-weight:600; flex:0 0 auto; }
|
|
297
|
+
.step .a.bad { color:var(--bad); }
|
|
298
|
+
.step .d { color:var(--muted); overflow:hidden; text-overflow:ellipsis; white-space:nowrap; }
|
|
299
|
+
.step .frame { display:block; margin:4px 0 10px; }
|
|
300
|
+
.step .frame img { max-width:min(100%,720px); max-height:360px; object-fit:cover; object-position:top; border:1px solid var(--line); border-radius:6px; display:block; }
|
|
301
|
+
details.evidence { margin:6px 0 18px; padding:8px 12px; background:var(--panel); border:1px solid var(--line); border-radius:8px; }
|
|
302
|
+
details.evidence > summary { cursor:pointer; color:var(--muted); font-size:13px; }
|
|
303
|
+
details.evidence .shots { display:flex; flex-wrap:wrap; gap:14px; margin-top:12px; }
|
|
304
|
+
details.evidence figure { margin:0; max-width:min(100%,460px); }
|
|
305
|
+
details.evidence img { width:100%; max-height:300px; object-fit:cover; object-position:top; border:1px solid var(--line); border-radius:6px; display:block; background:var(--panel); }
|
|
306
|
+
.onDisk { display:block; margin-top:4px; color:var(--muted); font:11px/1.5 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace; overflow-wrap:anywhere; }
|
|
307
|
+
header .served { flex:1 1 100%; margin:6px 0 0; color:var(--muted); font-size:12px; }
|
|
308
|
+
header .served code { font:11px/1.5 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace; overflow-wrap:anywhere; }
|
|
309
|
+
.gone-note { display:none; margin:0; padding:14px; border:1px dashed var(--line); border-radius:6px; color:var(--muted); font-size:12px; }
|
|
310
|
+
figure.gone .gone-note, a.gone .gone-note { display:block; }
|
|
311
|
+
figure.gone img, a.gone img { display:none; }
|
|
312
|
+
a.frame.gone { display:block; max-width:min(100%,720px); }
|
|
313
|
+
details.evidence figcaption { margin-top:4px; color:var(--muted); font:11px/1.5 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace; overflow-wrap:anywhere; }
|
|
314
|
+
`;
|
|
315
|
+
/** The whole document: one file, no external assets, opens from the file system. */
|
|
316
|
+
export function buildReplayHtml(input) {
|
|
317
|
+
const framed = input.sessions.some((s) => s.steps.some((x) => x.frame));
|
|
318
|
+
const prefix = input.framePrefix ?? "";
|
|
319
|
+
const savedAt = input.savedAt ?? "";
|
|
320
|
+
const sessions = input.sessions.map((s, i) => renderSession(s, i, prefix, savedAt)).join("\n");
|
|
321
|
+
return `<!doctype html>
|
|
322
|
+
<html lang="en">
|
|
323
|
+
<head>
|
|
324
|
+
<meta charset="utf-8">
|
|
325
|
+
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
326
|
+
<title>SceneScout run — ${escapeHtml(input.project)}</title>
|
|
327
|
+
<style>${STYLE}</style>
|
|
328
|
+
</head>
|
|
329
|
+
<body>
|
|
330
|
+
<header>
|
|
331
|
+
<h1>SceneScout run</h1>
|
|
332
|
+
<span class="meta">${escapeHtml(input.project)} · written ${escapeHtml(stamp(input.at))}${input.version ? ` · v${escapeHtml(input.version)}` : ""}</span>
|
|
333
|
+
${savedAt ? `<p class="served">This page is served by the engine and goes when it does. The copy that stays is <code>${escapeHtml(savedAt)}/report.html</code>, beside the frames it shows.</p>` : ""}
|
|
334
|
+
<nav><a href="#report">Report</a><a href="#steps">Steps</a></nav>
|
|
335
|
+
</header>
|
|
336
|
+
<main>
|
|
337
|
+
<h2 id="report">Report</h2>
|
|
338
|
+
${renderMarkdown(input.markdown, input.evidence ?? [], prefix, savedAt)}
|
|
339
|
+
<h2 id="steps">What each session did</h2>
|
|
340
|
+
<p class="objective">Every action, in the blocks its tasks made. ${framed
|
|
341
|
+
? "Each step shows the page as it was; click a frame to open it full size."
|
|
342
|
+
: "This run was not recorded, so there are no frames — attach with record:true to keep them."}</p>
|
|
343
|
+
${sessions || "<p>No session recorded any action.</p>"}
|
|
344
|
+
</main>
|
|
345
|
+
</body>
|
|
346
|
+
</html>
|
|
347
|
+
`;
|
|
348
|
+
}
|
package/dist/engine/report.js
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
import fs from "node:fs";
|
|
2
2
|
import path from "node:path";
|
|
3
3
|
import { SHARED_CHROME_ROUTE } from "./memory.js";
|
|
4
|
+
import { feedForSession } from "./live.js";
|
|
5
|
+
import { buildReplayHtml, evidenceFor } from "./replay.js";
|
|
4
6
|
function playwrightSkeleton(f) {
|
|
5
7
|
const routeClass = f.state.split("#")[0].split("?")[0];
|
|
6
8
|
let gotoPath = routeClass;
|
|
@@ -56,6 +58,66 @@ function violationRollup(oracleLog) {
|
|
|
56
58
|
}
|
|
57
59
|
const SEVERITY_ORDER = { high: 0, medium: 1, low: 2 };
|
|
58
60
|
const SEVERITY_ICON = { high: "🔴", medium: "🟠", low: "🟡" };
|
|
61
|
+
/**
|
|
62
|
+
* What to call the project in a document meant to be handed on. The memory
|
|
63
|
+
* directory's absolute path names a person's home directory and often the
|
|
64
|
+
* machine it ran on; the project's own folder name says which project it was
|
|
65
|
+
* without any of that.
|
|
66
|
+
*/
|
|
67
|
+
function projectName(dir) {
|
|
68
|
+
const parent = path.dirname(dir);
|
|
69
|
+
return path.basename(parent) || path.basename(dir) || "project";
|
|
70
|
+
}
|
|
71
|
+
/** Every session that did anything, with its steps in order — what the HTML replays. */
|
|
72
|
+
function replaySessions(memory) {
|
|
73
|
+
const names = [...new Set(memory.actionLog.map((e) => e.session ?? "default"))];
|
|
74
|
+
return names.map((session) => ({
|
|
75
|
+
session,
|
|
76
|
+
// The role a session ran as is not on the log's entries; until it is, the
|
|
77
|
+
// session's own name is the honest label rather than a guess from coverage.
|
|
78
|
+
role: "",
|
|
79
|
+
// No redactor: the log is redacted as it is WRITTEN (memory.logAction
|
|
80
|
+
// covers url, target and result), so what is stored is already clean.
|
|
81
|
+
// Filtering again here would run a regex over every step of a long run to
|
|
82
|
+
// find what cannot be there. `replay-redaction` in contract-test pins it.
|
|
83
|
+
steps: feedForSession(memory.actionLog, session, Number.MAX_SAFE_INTEGER),
|
|
84
|
+
}));
|
|
85
|
+
}
|
|
86
|
+
/** The frames that were on screen while each finding was being found. */
|
|
87
|
+
export function findingEvidence(memory, sessions) {
|
|
88
|
+
const all = sessions.flatMap((s) => s.steps).sort((a, b) => a.at.localeCompare(b.at));
|
|
89
|
+
return memory.findings
|
|
90
|
+
.map((f) => {
|
|
91
|
+
// The session that filed it, where it said so: with three browsers
|
|
92
|
+
// running at once, the run's whole log interleaves them, and the steps
|
|
93
|
+
// before a finding would come from whichever lane acted last.
|
|
94
|
+
const own = f.session ? sessions.find((s) => s.session === f.session) : undefined;
|
|
95
|
+
return { id: f.id, frames: evidenceFor(own ? own.steps : all, f.foundAt) };
|
|
96
|
+
})
|
|
97
|
+
.filter((e) => e.frames.length > 0);
|
|
98
|
+
}
|
|
99
|
+
/**
|
|
100
|
+
* The whole run as one page, for the live view to serve at its own address.
|
|
101
|
+
* Frames go through the view's own route, since the browser is reading this
|
|
102
|
+
* over HTTP rather than from the folder the frames live in.
|
|
103
|
+
*/
|
|
104
|
+
export function replayDocument(memory, markdown, version = "") {
|
|
105
|
+
const sessions = replaySessions(memory);
|
|
106
|
+
return buildReplayHtml({
|
|
107
|
+
markdown,
|
|
108
|
+
sessions,
|
|
109
|
+
evidence: findingEvidence(memory, sessions),
|
|
110
|
+
project: projectName(memory.dir),
|
|
111
|
+
at: new Date().toISOString(),
|
|
112
|
+
version,
|
|
113
|
+
framePrefix: "record/",
|
|
114
|
+
savedAt: memory.dir,
|
|
115
|
+
});
|
|
116
|
+
}
|
|
117
|
+
/** What the live view shows under each finding on a recorded run: the frames it was found on. */
|
|
118
|
+
export function reportEvidence(memory) {
|
|
119
|
+
return findingEvidence(memory, replaySessions(memory));
|
|
120
|
+
}
|
|
59
121
|
/**
|
|
60
122
|
* Split an element key into words. Keys are `tid:some-test-id` or `role:name`,
|
|
61
123
|
* and testids come in every casing convention there is — a `\b`-anchored regex
|
|
@@ -492,13 +554,51 @@ export function generateReport(memory, oracleLog, extras, opts = {}) {
|
|
|
492
554
|
}
|
|
493
555
|
const markdown = lines.join("\n");
|
|
494
556
|
const outPath = path.join(memory.dir, "report.md");
|
|
495
|
-
|
|
557
|
+
const htmlPath = path.join(memory.dir, "report.html");
|
|
558
|
+
let htmlWritten = false;
|
|
559
|
+
let htmlProblem = "";
|
|
560
|
+
if (opts.write !== false) {
|
|
496
561
|
fs.writeFileSync(outPath, markdown);
|
|
562
|
+
// The same run as one file that outlives the engine: the live view's
|
|
563
|
+
// address is a port in a process, and refreshing after the run is over
|
|
564
|
+
// gets nothing. This opens from the file system, offline, forever.
|
|
565
|
+
try {
|
|
566
|
+
const sessions = replaySessions(memory);
|
|
567
|
+
fs.writeFileSync(htmlPath, buildReplayHtml({
|
|
568
|
+
markdown,
|
|
569
|
+
sessions,
|
|
570
|
+
evidence: findingEvidence(memory, sessions),
|
|
571
|
+
project: projectName(memory.dir),
|
|
572
|
+
at: new Date().toISOString(),
|
|
573
|
+
version: extras?.version ?? "",
|
|
574
|
+
}));
|
|
575
|
+
htmlWritten = true;
|
|
576
|
+
}
|
|
577
|
+
catch (err) {
|
|
578
|
+
// The markdown is the report of record and is already on disk, so this
|
|
579
|
+
// must not fail the report — but the summary then says what went wrong
|
|
580
|
+
// rather than naming a file that is not there.
|
|
581
|
+
htmlProblem = err instanceof Error ? err.message : String(err);
|
|
582
|
+
// An earlier run's page would otherwise sit beside a fresh report.md,
|
|
583
|
+
// carrying its own timestamp, looking like this run.
|
|
584
|
+
try {
|
|
585
|
+
fs.rmSync(htmlPath, { force: true });
|
|
586
|
+
}
|
|
587
|
+
catch {
|
|
588
|
+
htmlProblem += "; an older one may still be beside it";
|
|
589
|
+
}
|
|
590
|
+
}
|
|
591
|
+
}
|
|
497
592
|
// Bounded summary for the tool result: full reports have exceeded client
|
|
498
593
|
// token limits in real runs (66–72KB observed) — the wire gets the digest,
|
|
499
594
|
// the disk gets the document.
|
|
500
595
|
const summaryLines = [
|
|
501
596
|
`Report written to ${outPath}`,
|
|
597
|
+
...(htmlWritten
|
|
598
|
+
? [`The same run as one page, with every session's steps: ${htmlPath}`]
|
|
599
|
+
: htmlProblem
|
|
600
|
+
? [`The one-page version of this run could NOT be written (${htmlProblem}); ${outPath} is unaffected.`]
|
|
601
|
+
: []),
|
|
502
602
|
``,
|
|
503
603
|
`OPEN FINDINGS: ${open.length} (${open.filter((f) => f.severity === "high").length} high) — ${current.length} this session, ${historical.length} historical${resolved.length ? `, ${resolved.length} resolved` : ""}`,
|
|
504
604
|
...(extras && extras.routesTotal > 0
|
package/dist/mcp-server.js
CHANGED
|
@@ -38,7 +38,8 @@ import { MemoryStore, redactSecrets } from "./engine/memory.js";
|
|
|
38
38
|
import { SessionQueue, withWatchdog } from "./engine/dispatch.js";
|
|
39
39
|
import { FIXTURE_KINDS } from "./engine/fixtures.js";
|
|
40
40
|
import { feedForSession, LIVE_ENV, writeStatusFile, LIVE_TOKEN_FILE, LiveServer, StatusBoard, } from "./engine/live.js";
|
|
41
|
-
import { computeGaps, formatRouteCoverage, generateReport } from "./engine/report.js";
|
|
41
|
+
import { computeGaps, formatRouteCoverage, generateReport, replayDocument, reportEvidence } from "./engine/report.js";
|
|
42
|
+
import { RECORD_MAX_FRAMES, resolveFrame } from "./engine/replay.js";
|
|
42
43
|
import { needsTask, taskRefusal, TASK_MAX } from "./engine/task.js";
|
|
43
44
|
import { EXPLORE_PROMPT_ARGUMENTS, explorePrompt, loadPlaybook, PLAYBOOK_PROMPT, PLAYBOOK_TOOL, SERVER_INSTRUCTIONS } from "./playbook.js";
|
|
44
45
|
import { formatScan, scanProject } from "./scan.js";
|
|
@@ -116,12 +117,24 @@ let lastRun = null;
|
|
|
116
117
|
function keepReport(eng) {
|
|
117
118
|
if (!eng.memory)
|
|
118
119
|
return;
|
|
120
|
+
// The markdown is the record and is kept first. The frames and the one-page
|
|
121
|
+
// version are extras, and building them in the same attempt meant a failure
|
|
122
|
+
// in either threw the report away with them — leaving a finished run with
|
|
123
|
+
// findings in it telling the viewer there was nothing to report.
|
|
119
124
|
try {
|
|
120
125
|
const { markdown } = generateReport(eng.memory, eng.oracleLog.all, reportExtras(eng), { write: false });
|
|
121
|
-
lastRun = { markdown: redactSecrets(markdown), at: new Date().toISOString(), dir: eng.memory.dir };
|
|
126
|
+
lastRun = { markdown: redactSecrets(markdown), at: new Date().toISOString(), dir: eng.memory.dir, evidence: [], replay: "" };
|
|
122
127
|
}
|
|
123
|
-
catch {
|
|
124
|
-
|
|
128
|
+
catch (err) {
|
|
129
|
+
console.error(`[scenescout] the report for ${eng.sessionKey} could not be kept: ${err instanceof Error ? err.message : String(err)}`);
|
|
130
|
+
return;
|
|
131
|
+
}
|
|
132
|
+
try {
|
|
133
|
+
lastRun.evidence = reportEvidence(eng.memory);
|
|
134
|
+
lastRun.replay = replayDocument(eng.memory, lastRun.markdown, PKG_VERSION);
|
|
135
|
+
}
|
|
136
|
+
catch (err) {
|
|
137
|
+
console.error(`[scenescout] the one-page version of this run could not be built: ${err instanceof Error ? err.message : String(err)}; the report itself is unaffected`);
|
|
125
138
|
}
|
|
126
139
|
}
|
|
127
140
|
/** Where this run's report belongs, and whether the agent has written it there yet. */
|
|
@@ -155,9 +168,58 @@ const liveProvider = {
|
|
|
155
168
|
report: () => {
|
|
156
169
|
const eng = (lastWriter && engines.get(lastWriter.session)) ?? engines.values().next().value;
|
|
157
170
|
if (!eng?.memory)
|
|
158
|
-
return lastRun ? { markdown: lastRun.markdown, at: lastRun.at } : null;
|
|
171
|
+
return lastRun ? { markdown: lastRun.markdown, at: lastRun.at, evidence: lastRun.evidence } : null;
|
|
159
172
|
const { markdown } = generateReport(eng.memory, eng.oracleLog.all, reportExtras(eng), { write: false });
|
|
160
|
-
return { markdown: redactSecrets(markdown), at: new Date().toISOString() };
|
|
173
|
+
return { markdown: redactSecrets(markdown), at: new Date().toISOString(), evidence: reportEvidence(eng.memory) };
|
|
174
|
+
},
|
|
175
|
+
/**
|
|
176
|
+
* The whole run as one page, served at its own address. The live view sends
|
|
177
|
+
* a finished run here: the address then IS the report, so refreshing works
|
|
178
|
+
* and there is nothing to lose by closing a panel.
|
|
179
|
+
*/
|
|
180
|
+
replay: () => {
|
|
181
|
+
const eng = (lastWriter && engines.get(lastWriter.session)) ?? engines.values().next().value;
|
|
182
|
+
if (!eng?.memory)
|
|
183
|
+
return lastRun?.replay || null;
|
|
184
|
+
try {
|
|
185
|
+
const { markdown } = generateReport(eng.memory, eng.oracleLog.all, reportExtras(eng), { write: false });
|
|
186
|
+
return replayDocument(eng.memory, redactSecrets(markdown), PKG_VERSION);
|
|
187
|
+
}
|
|
188
|
+
catch (err) {
|
|
189
|
+
// This address is one people refresh and bookmark, so a throw here would
|
|
190
|
+
// hand them a blank page. Say so, and fall back to the last rendering.
|
|
191
|
+
console.error(`[scenescout] the run's page could not be rendered: ${err instanceof Error ? err.message : String(err)}`);
|
|
192
|
+
return lastRun?.replay || null;
|
|
193
|
+
}
|
|
194
|
+
},
|
|
195
|
+
/**
|
|
196
|
+
* A recorded frame, read from the run's own recordings directory. The path
|
|
197
|
+
* comes from a viewer, so it is resolved and then required to still be
|
|
198
|
+
* inside that directory: nothing else in the project is reachable this way.
|
|
199
|
+
*/
|
|
200
|
+
frame: async (relPath) => {
|
|
201
|
+
// Sessions may hold different project directories, so the frame belongs to
|
|
202
|
+
// the session its own path names — not to whichever engine happens to be
|
|
203
|
+
// first. Falling back keeps a finished run's frames reachable.
|
|
204
|
+
const named = relPath.replace(/^recordings[\\/]/, "").split("/")[0];
|
|
205
|
+
const dir = engines.get(named)?.memory?.dir ?? engines.values().next().value?.memory?.dir ?? lastRun?.dir;
|
|
206
|
+
if (!dir)
|
|
207
|
+
return null;
|
|
208
|
+
const file = resolveFrame(path.join(dir, "recordings"), relPath);
|
|
209
|
+
if (!file)
|
|
210
|
+
return null;
|
|
211
|
+
try {
|
|
212
|
+
return await fs.promises.readFile(file);
|
|
213
|
+
}
|
|
214
|
+
catch (err) {
|
|
215
|
+
// A viewer can ask for anything, so a missing file is ordinary. A frame
|
|
216
|
+
// that exists and cannot be read is not, and would otherwise present as
|
|
217
|
+
// "that frame does not exist".
|
|
218
|
+
if (err.code !== "ENOENT") {
|
|
219
|
+
console.error(`[scenescout] a recorded frame could not be read (${file}): ${err instanceof Error ? err.message : String(err)}`);
|
|
220
|
+
}
|
|
221
|
+
return null;
|
|
222
|
+
}
|
|
161
223
|
},
|
|
162
224
|
// Both go around the session queue on purpose: a viewer must never wait
|
|
163
225
|
// behind the agent's calls, and a session that is stuck is the one most
|
|
@@ -177,6 +239,7 @@ function reportExtras(eng) {
|
|
|
177
239
|
unvisitedRoutes: unvisited,
|
|
178
240
|
mode: eng.mode,
|
|
179
241
|
policyAttributed: eng.oracleLog.policyAttributed,
|
|
242
|
+
version: PKG_VERSION,
|
|
180
243
|
};
|
|
181
244
|
}
|
|
182
245
|
/** Hand the live view's token to `scenescout watch` through a file only the owner can read. */
|
|
@@ -456,13 +519,18 @@ server.registerTool("scout_attach", {
|
|
|
456
519
|
'"Approve and reject orders as a manager"). It sits above the task, which is what the session is doing at any moment. ' +
|
|
457
520
|
"Shown to whoever is watching the run; worth setting whenever more than one session is live."),
|
|
458
521
|
task: z.string().max(300).optional().describe("Old name for `objective` (2.0). Prefer `objective`."),
|
|
522
|
+
record: z
|
|
523
|
+
.boolean()
|
|
524
|
+
.default(false)
|
|
525
|
+
.describe("Keep a frame of the page after every action, under .scenescout/recordings/, and show it beside that step in report.html. " +
|
|
526
|
+
"Off by default: a recording is pictures of the app under test sitting in the project folder. Turn it on for QA work, where the run is evidence and not only a report."),
|
|
459
527
|
session: z
|
|
460
528
|
.string()
|
|
461
529
|
.max(40)
|
|
462
530
|
.optional()
|
|
463
531
|
.describe("Session name for multi-role runs (e.g. 'admin', 'qa'). Creates/replaces that session's browser and makes it the default. Default: 'default'."),
|
|
464
532
|
},
|
|
465
|
-
}, serializedControl(async ({ url, projectPath, storageStatePath, mode, headed, browser, viewportWidth, viewportHeight, objective, task, session, }) => {
|
|
533
|
+
}, serializedControl(async ({ url, projectPath, storageStatePath, mode, headed, browser, viewportWidth, viewportHeight, objective, task, record, session, }) => {
|
|
466
534
|
try {
|
|
467
535
|
const target = session ?? activeName;
|
|
468
536
|
if (session) {
|
|
@@ -533,6 +601,7 @@ server.registerTool("scout_attach", {
|
|
|
533
601
|
viewport,
|
|
534
602
|
// `task` is what this was called in 2.0; it named the session's whole remit, which is the objective.
|
|
535
603
|
objective: objective ?? task,
|
|
604
|
+
record,
|
|
536
605
|
memoryStore: store,
|
|
537
606
|
});
|
|
538
607
|
eng.role = storageStatePath ? path.basename(storageStatePath).replace(/\.json$/i, "") : "anonymous";
|
|
@@ -543,7 +612,13 @@ server.registerTool("scout_attach", {
|
|
|
543
612
|
await ensureLive(eng.memory.dir);
|
|
544
613
|
writeStatus(target, "idle", "scout_attach");
|
|
545
614
|
}
|
|
546
|
-
|
|
615
|
+
// Recording writes pictures of the app under test into the project, so
|
|
616
|
+
// a run doing it says where they go rather than leaving the person to
|
|
617
|
+
// find a folder of screenshots later.
|
|
618
|
+
const recordNote = record && eng.memory?.dir
|
|
619
|
+
? `\n\n📸 RECORDING: a frame of the page after each action, under ${path.join(eng.memory.dir, "recordings", target)}/ (at most ${RECORD_MAX_FRAMES}). scout_report writes them into report.html beside report.md.`
|
|
620
|
+
: "";
|
|
621
|
+
return text(out + conflictNote + recordNote + (engines.size > 1 ? `\n${sessionLines()}` : "") + liveLine(), target);
|
|
547
622
|
}
|
|
548
623
|
catch (err) {
|
|
549
624
|
return errorText(err);
|
|
@@ -922,6 +997,7 @@ server.registerTool("scout_finding", {
|
|
|
922
997
|
evidence,
|
|
923
998
|
url: eng.currentUrl,
|
|
924
999
|
state: eng.currentState || "(unknown)",
|
|
1000
|
+
session: eng.sessionKey,
|
|
925
1001
|
});
|
|
926
1002
|
return text(isNew
|
|
927
1003
|
? `Finding recorded: [${finding.severity}] ${finding.title} (id ${finding.id})`
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "scenescout",
|
|
3
|
-
"version": "3.0
|
|
3
|
+
"version": "3.1.0",
|
|
4
4
|
"description": "SceneScout — exploratory UI testing for AI coding agents. An MCP server that gives any agent (Claude Code, Cursor, VS Code Copilot, Codex, Gemini CLI and others) a structured view of a running web app, always-on oracles, a network-level write policy, memory across runs and a gap-checked report.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "brunoboto96",
|