scenescout 1.4.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/dist/engine/browser.js +29 -1
- package/dist/engine/live-page.js +84 -4
- package/dist/engine/objective.js +49 -0
- package/dist/mcp-server.js +73 -7
- package/package.json +1 -1
- package/skills/scenescout/SKILL.md +6 -4
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# scenescout
|
|
2
2
|
|
|
3
|
+
## 2.0.0
|
|
4
|
+
|
|
5
|
+
### Major Changes
|
|
6
|
+
|
|
7
|
+
- 74a45d6: **A tool that acts on the app now needs an objective.** `scout_navigate`, `scout_back`, `scout_click`, `scout_type`, `scout_select`, `scout_press`, `scout_upload` and `scout_run_plan` take an `objective`: one short sentence naming what the current batch of actions is for. It stays set until a different one is passed, so a batch costs one sentence rather than one per call, and `scout_journey` still sets it (and outranks it) while a journey runs. A call that acts with none standing is refused with what to pass and why.
|
|
8
|
+
|
|
9
|
+
This is the breaking part: an agent that never states one now gets a refusal instead of a click. It exists because the objective was optional and therefore usually absent — someone watching a run saw sessions working through their app with nothing to say why, which is the one thing the live view is for. The objective now also appears on each card in the grid, not only in a session's close-up, and a session that has not said anything yet says so.
|
|
10
|
+
|
|
11
|
+
## 1.5.0
|
|
12
|
+
|
|
13
|
+
### Minor Changes
|
|
14
|
+
|
|
15
|
+
- 791b4d0: The live view now hands over the report when the run ends. Closing the last session used to take the report with it — the view served it from the live engine, so the moment the browsers went the page said there was nothing to report. The last rendering is now kept, and when the board empties the page says the run has finished, opens the report by itself, and names the file it belongs in: `saved at <path>` once `scout_report` has written it, or plainly that it is not on disk and this page holds the only copy. A **Save a copy** button downloads that copy through the viewer's own browser (nothing is asked of the engine, which still answers `GET` and nothing else), and closing the tab on a finished run whose report was never written asks for confirmation first.
|
|
16
|
+
|
|
3
17
|
## 1.4.0
|
|
4
18
|
|
|
5
19
|
### Minor Changes
|
package/dist/engine/browser.js
CHANGED
|
@@ -3,6 +3,7 @@ import fs from "node:fs";
|
|
|
3
3
|
import path from "node:path";
|
|
4
4
|
import { elementKey, fingerprintState, isNonPageRoute, normalizePath } from "./fingerprint.js";
|
|
5
5
|
import { AUTH_LOSS_PREFIX, JOURNEY_END, JOURNEY_START, MemoryStore } from "./memory.js";
|
|
6
|
+
import { normalizeObjective } from "./objective.js";
|
|
6
7
|
import { describeInjection, newInjections, probeQueries, probeScript, probeShape, rememberProbe } from "./injection.js";
|
|
7
8
|
import { AuthLossTracker } from "./authloss.js";
|
|
8
9
|
import { COLLECT_INTERACTABLES_SCRIPT, VISIBLE_SRC, geometryIssues, BROKEN_IMAGES_SCRIPT, brokenImageIssues, } from "./collector.js";
|
|
@@ -205,6 +206,27 @@ export class BrowserEngine {
|
|
|
205
206
|
journey = null;
|
|
206
207
|
/** The session's task, from scout_attach. Empty when the agent gave none. */
|
|
207
208
|
task = "";
|
|
209
|
+
/**
|
|
210
|
+
* What the batch of actions running right now is for. Required before a
|
|
211
|
+
* tool may act (objective.ts), stated by the agent on the call or by a
|
|
212
|
+
* journey, and kept until it is replaced — a batch costs one sentence.
|
|
213
|
+
*/
|
|
214
|
+
objective = null;
|
|
215
|
+
/** Set what this session is doing now. An empty value clears it. */
|
|
216
|
+
setObjective(text) {
|
|
217
|
+
const clean = normalizeObjective(text);
|
|
218
|
+
if (!clean) {
|
|
219
|
+
this.objective = null;
|
|
220
|
+
return;
|
|
221
|
+
}
|
|
222
|
+
if (this.objective?.text === clean)
|
|
223
|
+
return;
|
|
224
|
+
this.objective = { text: clean, since: Date.now() };
|
|
225
|
+
}
|
|
226
|
+
/** Whether anything is standing that the live view could show as the objective. */
|
|
227
|
+
get hasObjective() {
|
|
228
|
+
return this.journey !== null || this.objective !== null;
|
|
229
|
+
}
|
|
208
230
|
/**
|
|
209
231
|
* Begin measuring a user JOURNEY — the interaction cost of completing one
|
|
210
232
|
* real task ("create an order", "approve a document"). E2E suites assert
|
|
@@ -2000,7 +2022,13 @@ export class BrowserEngine {
|
|
|
2000
2022
|
browser: this.engineName,
|
|
2001
2023
|
headed: this.headed,
|
|
2002
2024
|
...(this.task ? { task: this.task } : {}),
|
|
2003
|
-
|
|
2025
|
+
// A journey is a whole user task being measured, so its goal outranks
|
|
2026
|
+
// the batch objective while it runs.
|
|
2027
|
+
...(this.journey
|
|
2028
|
+
? { objective: this.journey.goal, objectiveSince: new Date(this.journey.startedAt).toISOString() }
|
|
2029
|
+
: this.objective
|
|
2030
|
+
? { objective: this.objective.text, objectiveSince: new Date(this.objective.since).toISOString() }
|
|
2031
|
+
: {}),
|
|
2004
2032
|
};
|
|
2005
2033
|
}
|
|
2006
2034
|
/**
|
package/dist/engine/live-page.js
CHANGED
|
@@ -48,6 +48,13 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
48
48
|
button[aria-pressed="true"] { background: var(--accent); border-color: var(--accent); color: #fff; }
|
|
49
49
|
#banner { display: none; margin: 12px 16px 0; padding: 8px 12px; border-radius: 6px; background: var(--stuck-bg); color: var(--stuck); }
|
|
50
50
|
#empty { display: none; padding: 48px 16px; text-align: center; color: var(--muted); }
|
|
51
|
+
#finished { display: none; max-width: 640px; margin: 48px auto; padding: 24px; text-align: center;
|
|
52
|
+
background: var(--panel); border: 1px solid var(--line); border-radius: 8px; }
|
|
53
|
+
#finished.open { display: block; }
|
|
54
|
+
#finished h2 { margin: 0 0 8px; font-size: 18px; }
|
|
55
|
+
#finished p { margin: 0 0 8px; color: var(--muted); }
|
|
56
|
+
#finished .where { font: 12px/1.6 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; overflow-wrap: anywhere; }
|
|
57
|
+
#finished button { margin-top: 8px; padding: 7px 14px; font-weight: 600; }
|
|
51
58
|
main { display: grid; grid-template-columns: repeat(auto-fill, minmax(min(100%, 320px), 1fr)); gap: 12px; padding: 16px; }
|
|
52
59
|
.card { background: var(--panel); border: 1px solid var(--line); border-radius: 8px; overflow: hidden; display: flex; flex-direction: column; }
|
|
53
60
|
.card.stuck { border-color: var(--stuck); }
|
|
@@ -96,6 +103,9 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
96
103
|
#focus .brief .since { margin-top: -10px; font-size: 12px; color: #98a2b3; }
|
|
97
104
|
@media (max-width: 700px) { #focus .lower { flex-direction: column; height: 45vh; } }
|
|
98
105
|
.task { padding: 0 12px 2px; font-size: 12px; color: var(--muted); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
|
106
|
+
.doing { padding: 0 12px 4px; font-size: 12px; color: var(--text); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
|
107
|
+
.doing::before { content: "▸ "; color: var(--accent); }
|
|
108
|
+
.doing.unset { color: var(--stuck); }
|
|
99
109
|
#focus .feed .a { color: #e6e9ee; }
|
|
100
110
|
#focus .feed .t, #focus .feed .d, #focus .feed .none { color: #98a2b3; }
|
|
101
111
|
#focus .feed .bad { color: #fca5a5; }
|
|
@@ -143,12 +153,19 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
143
153
|
</header>
|
|
144
154
|
<div id="banner" role="alert" data-testid="live-unreachable-banner">The engine is not answering. It may have exited; this page will pick up again if it comes back.</div>
|
|
145
155
|
<div id="empty" data-testid="live-empty-state">No session is attached yet. Cards appear here as soon as one attaches.</div>
|
|
156
|
+
<div id="finished" data-testid="live-finished-state">
|
|
157
|
+
<h2>The run has finished</h2>
|
|
158
|
+
<p>Its browsers are closed, so there is nothing left to watch. What it found is in the report.</p>
|
|
159
|
+
<p class="where" id="finished-where" data-testid="live-finished-where"></p>
|
|
160
|
+
<button type="button" id="finished-report" data-testid="live-finished-report">Read the report</button>
|
|
161
|
+
</div>
|
|
146
162
|
<main id="grid"></main>
|
|
147
163
|
<div id="report" role="dialog" aria-modal="true" aria-label="The run's report" data-testid="live-report-dialog">
|
|
148
164
|
<div class="bar">
|
|
149
165
|
<strong>Report</strong>
|
|
150
166
|
<span class="meta" id="report-meta" data-testid="live-report-meta"></span>
|
|
151
167
|
<span class="spacer"></span>
|
|
168
|
+
<button type="button" id="report-save" data-testid="live-report-save">Save a copy</button>
|
|
152
169
|
<button type="button" id="report-close" data-testid="live-report-close">Close</button>
|
|
153
170
|
</div>
|
|
154
171
|
<div class="doc" id="report-doc" data-testid="live-report-doc"></div>
|
|
@@ -187,6 +204,14 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
187
204
|
var reportOpen = false;
|
|
188
205
|
var reportTimer = null;
|
|
189
206
|
var reportProblem = null;
|
|
207
|
+
// The run had sessions and has none now: its browsers are gone, and this
|
|
208
|
+
// page holds the only rendering of the report unless it was written to disk.
|
|
209
|
+
var sawRun = false;
|
|
210
|
+
var finished = false;
|
|
211
|
+
var reportShown = false;
|
|
212
|
+
var reportFile = null;
|
|
213
|
+
var reportMarkdown = null;
|
|
214
|
+
var savedACopy = false;
|
|
190
215
|
// A result that reads as a failure is shown in red.
|
|
191
216
|
var BAD_RESULT = /error|fail|refus|block|violation|abandoned/i;
|
|
192
217
|
|
|
@@ -282,6 +307,8 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
282
307
|
top.appendChild(nameEl); top.appendChild(role); top.appendChild(badge);
|
|
283
308
|
var task = el('div', 'task');
|
|
284
309
|
task.setAttribute('data-testid', 'live-card-task-' + name);
|
|
310
|
+
var doing = el('div', 'doing');
|
|
311
|
+
doing.setAttribute('data-testid', 'live-card-objective-' + name);
|
|
285
312
|
var tool = el('div', 'line');
|
|
286
313
|
var url = el('div', 'line');
|
|
287
314
|
var shot = el('button', 'shot');
|
|
@@ -304,9 +331,9 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
304
331
|
toggle.setAttribute('data-testid', 'live-card-toggle-' + name);
|
|
305
332
|
var spec = el('span', 'spec');
|
|
306
333
|
foot.appendChild(toggle); foot.appendChild(spec);
|
|
307
|
-
root.appendChild(top); root.appendChild(task); root.appendChild(tool); root.appendChild(url); root.appendChild(shot); root.appendChild(feed); root.appendChild(foot);
|
|
334
|
+
root.appendChild(top); root.appendChild(task); root.appendChild(doing); root.appendChild(tool); root.appendChild(url); root.appendChild(shot); root.appendChild(feed); root.appendChild(foot);
|
|
308
335
|
|
|
309
|
-
var card = { name: name, root: root, role: role, badge: badge, task: task, tool: tool, url: url, shot: shot, img: img, feed: feed, toggle: toggle, spec: spec, live: false };
|
|
336
|
+
var card = { name: name, root: root, role: role, badge: badge, task: task, doing: doing, tool: tool, url: url, shot: shot, img: img, feed: feed, toggle: toggle, spec: spec, live: false };
|
|
310
337
|
toggle.addEventListener('click', function () { setLive(card, !card.live); });
|
|
311
338
|
shot.addEventListener('click', function () { openFocus(name); });
|
|
312
339
|
img.src = shotUrl(name);
|
|
@@ -435,7 +462,8 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
435
462
|
meta.textContent = '';
|
|
436
463
|
return;
|
|
437
464
|
}
|
|
438
|
-
|
|
465
|
+
reportMarkdown = d.markdown;
|
|
466
|
+
meta.textContent = (finished ? 'as the run left it at ' : 'as the run stands at ') + clock(d.at) + ' · ' + whereItIs();
|
|
439
467
|
renderMarkdown(doc, d.markdown);
|
|
440
468
|
})
|
|
441
469
|
.catch(function (err) {
|
|
@@ -448,6 +476,28 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
448
476
|
}
|
|
449
477
|
});
|
|
450
478
|
}
|
|
479
|
+
/** Where the report's file is, and whether the agent has written it there. */
|
|
480
|
+
function whereItIs() {
|
|
481
|
+
if (!reportFile) return 'scout_report writes this document to .scenescout/report.md at the end';
|
|
482
|
+
if (reportFile.written) return 'saved at ' + reportFile.path;
|
|
483
|
+
return 'NOT saved: ' + reportFile.path + ' does not exist — the agent has not run scout_report, so this page holds the only copy';
|
|
484
|
+
}
|
|
485
|
+
|
|
486
|
+
// Saving is the viewer's own browser writing a file the page already has;
|
|
487
|
+
// nothing is sent to the engine, which only ever answers GET (ADR 7).
|
|
488
|
+
function saveACopy() {
|
|
489
|
+
if (!reportMarkdown) return;
|
|
490
|
+
var url = URL.createObjectURL(new Blob([reportMarkdown], { type: 'text/markdown' }));
|
|
491
|
+
var a = document.createElement('a');
|
|
492
|
+
a.href = url;
|
|
493
|
+
a.download = 'scenescout-report.md';
|
|
494
|
+
document.body.appendChild(a);
|
|
495
|
+
a.click();
|
|
496
|
+
a.remove();
|
|
497
|
+
setTimeout(function () { URL.revokeObjectURL(url); }, 10000);
|
|
498
|
+
savedACopy = true;
|
|
499
|
+
}
|
|
500
|
+
|
|
451
501
|
function openReport() {
|
|
452
502
|
reportOpen = true;
|
|
453
503
|
document.getElementById('report').classList.add('open');
|
|
@@ -503,6 +553,12 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
503
553
|
card.task.textContent = s.task || '';
|
|
504
554
|
card.task.title = s.task || '';
|
|
505
555
|
card.task.hidden = !s.task;
|
|
556
|
+
// What it is doing right now, in the agent's words. A session that acts
|
|
557
|
+
// without saying is refused by the engine, so a blank one here is a
|
|
558
|
+
// session that has not acted yet — say that rather than showing nothing.
|
|
559
|
+
card.doing.textContent = s.objective || 'not said yet';
|
|
560
|
+
card.doing.title = s.objective || '';
|
|
561
|
+
card.doing.className = 'doing' + (s.objective ? '' : ' unset');
|
|
506
562
|
card.badge.className = 'badge ' + s.state;
|
|
507
563
|
card.badge.textContent = d.badge;
|
|
508
564
|
card.tool.textContent = d.tool;
|
|
@@ -608,7 +664,21 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
608
664
|
delete frames[name];
|
|
609
665
|
});
|
|
610
666
|
syncEvents();
|
|
611
|
-
|
|
667
|
+
reportFile = snap.report || reportFile;
|
|
668
|
+
if (snap.sessions.length > 0) sawRun = true;
|
|
669
|
+
finished = sawRun && snap.sessions.length === 0;
|
|
670
|
+
document.getElementById('empty').style.display = snap.sessions.length || finished ? 'none' : 'block';
|
|
671
|
+
document.getElementById('finished').classList.toggle('open', finished);
|
|
672
|
+
if (finished) {
|
|
673
|
+
var where = document.getElementById('finished-where');
|
|
674
|
+
where.textContent = whereItIs();
|
|
675
|
+
where.className = 'where' + (reportFile && reportFile.written ? '' : ' unset');
|
|
676
|
+
// The moment somebody wants the report is the moment the run ends: show it.
|
|
677
|
+
if (!reportShown) {
|
|
678
|
+
reportShown = true;
|
|
679
|
+
if (!reportOpen) openReport();
|
|
680
|
+
}
|
|
681
|
+
}
|
|
612
682
|
document.getElementById('engine').textContent = 'engine pid ' + snap.pid + ' · v' + snap.version;
|
|
613
683
|
document.getElementById('counts').textContent = snap.sessions.length + ' session' + (snap.sessions.length === 1 ? '' : 's') +
|
|
614
684
|
' · ' + counts.running + ' running · ' + counts.idle + ' idle' + (counts.stuck ? ' · ' + counts.stuck + ' stuck' : '');
|
|
@@ -633,6 +703,16 @@ export const LIVE_PAGE = `<!doctype html>
|
|
|
633
703
|
document.getElementById('focus-feed').addEventListener('mouseleave', function () { showObjective(null); });
|
|
634
704
|
document.getElementById('focus').addEventListener('click', function (e) { if (e.target === this) closeFocus(); });
|
|
635
705
|
document.getElementById('report-open').addEventListener('click', openReport);
|
|
706
|
+
document.getElementById('finished-report').addEventListener('click', openReport);
|
|
707
|
+
document.getElementById('report-save').addEventListener('click', saveACopy);
|
|
708
|
+
// Closing the tab on a finished run whose report was never written to disk
|
|
709
|
+
// throws the only copy away. The browser shows its own confirm/dismiss, and
|
|
710
|
+
// only when the person has interacted with the page at least once.
|
|
711
|
+
window.addEventListener('beforeunload', function (e) {
|
|
712
|
+
if (!finished || savedACopy || (reportFile && reportFile.written)) return;
|
|
713
|
+
e.preventDefault();
|
|
714
|
+
e.returnValue = '';
|
|
715
|
+
});
|
|
636
716
|
document.getElementById('report-close').addEventListener('click', closeReport);
|
|
637
717
|
document.addEventListener('keydown', function (e) {
|
|
638
718
|
if (e.key !== 'Escape') return;
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The objective: one short sentence saying what the batch of actions a session
|
|
3
|
+
* is performing right now is FOR.
|
|
4
|
+
*
|
|
5
|
+
* The engine sees tool calls, never the reasoning behind them, so the live
|
|
6
|
+
* view can only show an objective the agent states. Left optional it was
|
|
7
|
+
* usually blank — a person watching saw a session click through an app with
|
|
8
|
+
* nothing to say why. So a tool that acts on the app requires one: passed on
|
|
9
|
+
* the call, or already standing from an earlier call or a journey. It stays
|
|
10
|
+
* set until it is replaced, so a batch costs one sentence, not one per click.
|
|
11
|
+
*
|
|
12
|
+
* Reading the page (snapshot, hover, scroll, coverage, the audits) needs none:
|
|
13
|
+
* orienting is what an agent does before it can say what it is about to do.
|
|
14
|
+
*/
|
|
15
|
+
/** Longer than this is a paragraph, not an objective; the live view shows one line. */
|
|
16
|
+
export const OBJECTIVE_MAX = 120;
|
|
17
|
+
/**
|
|
18
|
+
* The tools that act on the app under test. Each changes what the person
|
|
19
|
+
* watching is looking at, so each has to be able to say why.
|
|
20
|
+
*/
|
|
21
|
+
export const NEEDS_OBJECTIVE = new Set([
|
|
22
|
+
"scout_navigate",
|
|
23
|
+
"scout_back",
|
|
24
|
+
"scout_click",
|
|
25
|
+
"scout_type",
|
|
26
|
+
"scout_select",
|
|
27
|
+
"scout_press",
|
|
28
|
+
"scout_upload",
|
|
29
|
+
"scout_run_plan",
|
|
30
|
+
]);
|
|
31
|
+
export function needsObjective(tool) {
|
|
32
|
+
return NEEDS_OBJECTIVE.has(tool);
|
|
33
|
+
}
|
|
34
|
+
/** One line, whitespace collapsed, bounded. Empty input clears the objective. */
|
|
35
|
+
export function normalizeObjective(text) {
|
|
36
|
+
return (text ?? "").replace(/\s+/g, " ").trim().slice(0, OBJECTIVE_MAX);
|
|
37
|
+
}
|
|
38
|
+
/**
|
|
39
|
+
* What a session is told when it acts with no objective standing. It names the
|
|
40
|
+
* parameter, gives the shape of a good one, and says the rule it satisfies —
|
|
41
|
+
* an agent that reads this once should not need it again.
|
|
42
|
+
*/
|
|
43
|
+
export function objectiveRefusal(tool) {
|
|
44
|
+
return (`${tool} needs an objective: this session has none, so the live view cannot say what it is doing.\n` +
|
|
45
|
+
`Pass objective:"…" on this call — one short sentence about the batch of actions you are performing, ` +
|
|
46
|
+
`in the words you would use to tell a colleague ("Sign in as QA_Team and check where it lands", "Fill the deviation form with invalid dates"). ` +
|
|
47
|
+
`It stays set until you pass a different one, so a batch costs one sentence, not one per call. ` +
|
|
48
|
+
`scout_journey {action:"start", goal:"…"} sets it too, and is what to use when you are measuring a whole user task.`);
|
|
49
|
+
}
|
package/dist/mcp-server.js
CHANGED
|
@@ -37,8 +37,9 @@ import { reapOrphanBrowsers } from "./engine/reaper.js";
|
|
|
37
37
|
import { MemoryStore, redactSecrets } from "./engine/memory.js";
|
|
38
38
|
import { SessionQueue, withWatchdog } from "./engine/dispatch.js";
|
|
39
39
|
import { FIXTURE_KINDS } from "./engine/fixtures.js";
|
|
40
|
-
import { feedForSession, LIVE_ENV, writeStatusFile, LIVE_TOKEN_FILE, LiveServer, StatusBoard } from "./engine/live.js";
|
|
40
|
+
import { feedForSession, LIVE_ENV, writeStatusFile, LIVE_TOKEN_FILE, LiveServer, StatusBoard, } from "./engine/live.js";
|
|
41
41
|
import { computeGaps, formatRouteCoverage, generateReport } from "./engine/report.js";
|
|
42
|
+
import { needsObjective, objectiveRefusal, OBJECTIVE_MAX } from "./engine/objective.js";
|
|
42
43
|
import { EXPLORE_PROMPT_ARGUMENTS, explorePrompt, loadPlaybook, PLAYBOOK_PROMPT, PLAYBOOK_TOOL, SERVER_INSTRUCTIONS } from "./playbook.js";
|
|
43
44
|
import { formatScan, scanProject } from "./scan.js";
|
|
44
45
|
/** Live sessions: each name owns an independent BrowserEngine (browser + auth). */
|
|
@@ -105,8 +106,44 @@ const liveTokenWrites = new Map();
|
|
|
105
106
|
let liveTokenWarned = false;
|
|
106
107
|
/** Why the live view could not start, when it could not: told to the agent and written to status.json. */
|
|
107
108
|
let liveError = null;
|
|
109
|
+
/**
|
|
110
|
+
* The last run's report, kept after its sessions close. The engines are gone
|
|
111
|
+
* by then, so this is the only way the live view can still show what the run
|
|
112
|
+
* found — which is the moment somebody most wants to read it.
|
|
113
|
+
*/
|
|
114
|
+
let lastRun = null;
|
|
115
|
+
/** Render the report for a session that is about to close, so the live view keeps it. */
|
|
116
|
+
function keepReport(eng) {
|
|
117
|
+
if (!eng.memory)
|
|
118
|
+
return;
|
|
119
|
+
try {
|
|
120
|
+
const { markdown } = generateReport(eng.memory, eng.oracleLog.all, reportExtras(eng), { write: false });
|
|
121
|
+
lastRun = { markdown: redactSecrets(markdown), at: new Date().toISOString(), dir: eng.memory.dir };
|
|
122
|
+
}
|
|
123
|
+
catch {
|
|
124
|
+
// Best-effort: a report that cannot be rendered must not fail a close.
|
|
125
|
+
}
|
|
126
|
+
}
|
|
127
|
+
/** Where this run's report belongs, and whether the agent has written it there yet. */
|
|
128
|
+
function reportFile(dir) {
|
|
129
|
+
if (!dir)
|
|
130
|
+
return undefined;
|
|
131
|
+
const file = path.join(dir, "report.md");
|
|
132
|
+
try {
|
|
133
|
+
return { path: file, written: fs.existsSync(file) };
|
|
134
|
+
}
|
|
135
|
+
catch {
|
|
136
|
+
return { path: file, written: false };
|
|
137
|
+
}
|
|
138
|
+
}
|
|
108
139
|
const liveProvider = {
|
|
109
|
-
snapshot: () => ({
|
|
140
|
+
snapshot: () => ({
|
|
141
|
+
pid: process.pid,
|
|
142
|
+
version: PKG_VERSION,
|
|
143
|
+
at: new Date().toISOString(),
|
|
144
|
+
sessions: board.list(),
|
|
145
|
+
report: reportFile(engines.values().next().value?.memory?.dir ?? lastRun?.dir),
|
|
146
|
+
}),
|
|
110
147
|
// The engine holds no reasoning — it never sees one — so the feed is what the
|
|
111
148
|
// session DID: the action log, which is the same trail a finding's repro uses.
|
|
112
149
|
activity: (session, limit) => feedForSession(engines.get(session)?.memory?.actionLog ?? [], session, limit, redactSecrets),
|
|
@@ -118,7 +155,7 @@ const liveProvider = {
|
|
|
118
155
|
report: () => {
|
|
119
156
|
const eng = (lastWriter && engines.get(lastWriter.session)) ?? engines.values().next().value;
|
|
120
157
|
if (!eng?.memory)
|
|
121
|
-
return null;
|
|
158
|
+
return lastRun ? { markdown: lastRun.markdown, at: lastRun.at } : null;
|
|
122
159
|
const { markdown } = generateReport(eng.memory, eng.oracleLog.all, reportExtras(eng), { write: false });
|
|
123
160
|
return { markdown: redactSecrets(markdown), at: new Date().toISOString() };
|
|
124
161
|
},
|
|
@@ -273,6 +310,16 @@ const sessionQueue = new SessionQueue();
|
|
|
273
310
|
function serializedPerSession(label, fn, timeoutMs = 60_000) {
|
|
274
311
|
return (args) => {
|
|
275
312
|
const session = args.session ?? activeName;
|
|
313
|
+
// Every acting tool passes through here, so the objective is required in
|
|
314
|
+
// one place rather than eight. A call that states one sets it for the
|
|
315
|
+
// batch; a call that acts with none standing is told what to pass.
|
|
316
|
+
const eng = engines.get(session);
|
|
317
|
+
if (eng) {
|
|
318
|
+
if (args.objective !== undefined)
|
|
319
|
+
eng.setObjective(args.objective);
|
|
320
|
+
if (needsObjective(label) && !eng.hasObjective)
|
|
321
|
+
return Promise.resolve(text(objectiveRefusal(label), session));
|
|
322
|
+
}
|
|
276
323
|
const exec = async () => {
|
|
277
324
|
writeStatus(session, "running", label, timeoutMs);
|
|
278
325
|
try {
|
|
@@ -312,6 +359,18 @@ const sessionParam = z
|
|
|
312
359
|
.max(40)
|
|
313
360
|
.optional()
|
|
314
361
|
.describe("Target this session directly instead of the active one — pass it explicitly when dispatching to MULTIPLE sessions in one turn (e.g. two scout_click calls with different `session`), which then run CONCURRENTLY rather than queueing. Omit for single-session sequential use.");
|
|
362
|
+
/**
|
|
363
|
+
* What the batch of actions this call belongs to is for. Required by the tools
|
|
364
|
+
* that act (objective.ts) unless one is already standing; shown to whoever is
|
|
365
|
+
* watching the run, beside the session's task.
|
|
366
|
+
*/
|
|
367
|
+
const objectiveParam = z
|
|
368
|
+
.string()
|
|
369
|
+
.max(OBJECTIVE_MAX)
|
|
370
|
+
.optional()
|
|
371
|
+
.describe("One short sentence naming what this batch of actions is for, in the words you would use to tell a colleague " +
|
|
372
|
+
'("Sign in as QA_Team and check where it lands"). It stays set until you pass a different one, and is shown live to the person watching. ' +
|
|
373
|
+
"Required on the tools that act unless a journey or an earlier call already set one.");
|
|
315
374
|
// The method, for every client that has no skill loader. It is read per call,
|
|
316
375
|
// not cached: a source checkout's skill file can change under a running server.
|
|
317
376
|
server.registerTool(PLAYBOOK_TOOL, {
|
|
@@ -549,6 +608,7 @@ server.registerTool("scout_run_plan", {
|
|
|
549
608
|
}))
|
|
550
609
|
.min(1)
|
|
551
610
|
.max(20),
|
|
611
|
+
objective: objectiveParam,
|
|
552
612
|
session: sessionParam,
|
|
553
613
|
},
|
|
554
614
|
}, serializedPerSession("scout_run_plan", async ({ steps }, session) => {
|
|
@@ -564,6 +624,7 @@ server.registerTool("scout_click", {
|
|
|
564
624
|
inputSchema: {
|
|
565
625
|
ref: z.string().describe("Element ref, e.g. e12"),
|
|
566
626
|
clicks: z.number().int().min(1).max(3).default(1).describe("1 = normal; 2-3 = rapid repeated clicks (double-submit probe)"),
|
|
627
|
+
objective: objectiveParam,
|
|
567
628
|
session: sessionParam,
|
|
568
629
|
},
|
|
569
630
|
}, serializedPerSession("scout_click", async ({ ref, clicks }, session) => {
|
|
@@ -584,6 +645,7 @@ server.registerTool("scout_type", {
|
|
|
584
645
|
value: z.string().optional().describe("Alias for `textValue`."),
|
|
585
646
|
pressEnter: z.boolean().default(false).describe("Press Enter after typing"),
|
|
586
647
|
replace: z.boolean().default(false).describe("Clear the field before typing instead of appending to existing content"),
|
|
648
|
+
objective: objectiveParam,
|
|
587
649
|
session: sessionParam,
|
|
588
650
|
},
|
|
589
651
|
}, serializedPerSession("scout_type", async ({ ref, textValue, value, pressEnter, replace }, session) => {
|
|
@@ -618,6 +680,7 @@ server.registerTool("scout_upload", {
|
|
|
618
680
|
.optional()
|
|
619
681
|
.describe("Generated fixture kind; default: inferred from the input's accept attribute (pdf when there is none, or none we can generate)"),
|
|
620
682
|
name: z.string().min(1).max(512).optional().describe("Filename override (default scenescout-fixture.<kind>, or the disk file's own name)"),
|
|
683
|
+
objective: objectiveParam,
|
|
621
684
|
session: sessionParam,
|
|
622
685
|
},
|
|
623
686
|
}, serializedPerSession("scout_upload", async ({ ref, filePath, fixture, name }, session) => {
|
|
@@ -641,7 +704,7 @@ server.registerTool("scout_hover", {
|
|
|
641
704
|
}));
|
|
642
705
|
server.registerTool("scout_select", {
|
|
643
706
|
description: "Select an option in a <select> by ref.",
|
|
644
|
-
inputSchema: { ref: z.string(), value: z.string().describe("Option value or label"), session: sessionParam },
|
|
707
|
+
inputSchema: { ref: z.string(), value: z.string().describe("Option value or label"), objective: objectiveParam, session: sessionParam },
|
|
645
708
|
}, serializedPerSession("scout_select", async ({ ref, value }, session) => {
|
|
646
709
|
try {
|
|
647
710
|
return text(await engineFor(session).select(ref, value), session);
|
|
@@ -652,7 +715,7 @@ server.registerTool("scout_select", {
|
|
|
652
715
|
}));
|
|
653
716
|
server.registerTool("scout_navigate", {
|
|
654
717
|
description: "Navigate to a URL or a path relative to the attached base URL (e.g. '/orders'). Also supports 'back' via scout_back.",
|
|
655
|
-
inputSchema: { target: z.string().describe("Absolute URL or path like /settings"), session: sessionParam },
|
|
718
|
+
inputSchema: { target: z.string().describe("Absolute URL or path like /settings"), objective: objectiveParam, session: sessionParam },
|
|
656
719
|
}, serializedPerSession("scout_navigate", async ({ target }, session) => {
|
|
657
720
|
try {
|
|
658
721
|
return text(await engineFor(session).navigate(target), session);
|
|
@@ -663,7 +726,7 @@ server.registerTool("scout_navigate", {
|
|
|
663
726
|
}));
|
|
664
727
|
server.registerTool("scout_back", {
|
|
665
728
|
description: "Go back in browser history (tests back-button resilience).",
|
|
666
|
-
inputSchema: { session: sessionParam },
|
|
729
|
+
inputSchema: { objective: objectiveParam, session: sessionParam },
|
|
667
730
|
}, serializedPerSession("scout_back", async (_args, session) => {
|
|
668
731
|
try {
|
|
669
732
|
return text(await engineFor(session).goBack(), session);
|
|
@@ -693,7 +756,7 @@ server.registerTool("scout_scroll", {
|
|
|
693
756
|
}));
|
|
694
757
|
server.registerTool("scout_press", {
|
|
695
758
|
description: "Press a keyboard key (e.g. Escape, Tab, Enter) — useful for closing modals and testing keyboard navigation.",
|
|
696
|
-
inputSchema: { key: z.string(), session: sessionParam },
|
|
759
|
+
inputSchema: { key: z.string(), objective: objectiveParam, session: sessionParam },
|
|
697
760
|
}, serializedPerSession("scout_press", async ({ key }, session) => {
|
|
698
761
|
try {
|
|
699
762
|
return text(await engineFor(session).press(key), session);
|
|
@@ -976,6 +1039,8 @@ server.registerTool("scout_close", {
|
|
|
976
1039
|
for (const e of engines.values())
|
|
977
1040
|
if (e.memory?.dir)
|
|
978
1041
|
dirs.add(e.memory.dir);
|
|
1042
|
+
for (const e of engines.values())
|
|
1043
|
+
keepReport(e);
|
|
979
1044
|
for (const name of engines.keys())
|
|
980
1045
|
live?.dropSession(name);
|
|
981
1046
|
await Promise.allSettled([...engines.values()].map((e) => e.close()));
|
|
@@ -991,6 +1056,7 @@ server.registerTool("scout_close", {
|
|
|
991
1056
|
const eng = engines.get(name);
|
|
992
1057
|
if (!eng)
|
|
993
1058
|
return text(`No live session "${name}".`, name);
|
|
1059
|
+
keepReport(eng);
|
|
994
1060
|
live?.dropSession(name);
|
|
995
1061
|
await eng.close();
|
|
996
1062
|
const saveError = eng.memory?.lastSaveError;
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "scenescout",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "2.0.0",
|
|
4
4
|
"description": "SceneScout — exploratory UI testing for AI coding agents. An MCP server that gives any agent (Claude Code, Cursor, VS Code Copilot, Codex, Gemini CLI and others) a structured view of a running web app, always-on oracles, a network-level write policy, memory across runs and a gap-checked report.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "brunoboto96",
|
|
@@ -37,10 +37,12 @@ You are the brain of an exploratory UI tester. The SceneScout MCP server gives y
|
|
|
37
37
|
4. **Snapshot economics:** `scout_snapshot` after landing somewhere new; re-snapshots of the same route return *diffs* with stable refs — "No element changes" costs you almost nothing. `scout_screenshot` ONLY for suspected pixel-native issues (a canvas, a rendering glitch); geometry problems (overlap, off-screen, a covered control) are already in the snapshot as GEOMETRY issues, and images that failed to load are listed under BROKEN IMAGES — file those, quoting the line.
|
|
38
38
|
5. **Native-user behaviours.** `scout_type {ref, textValue}` (or its alias `value`, matching `scout_select` and a plan step) APPENDS when a field already has content (menu clicks often insert @-mention chips or commands into composers — appending preserves them; the result reports what was already there); pass `replace=true` only to deliberately clear, and `pressEnter=true` to submit from the field the way a user would. Before concluding a badge, icon, or "N errors" indicator *does nothing*, `scout_hover` it — tooltips and hover cards are invisible to snapshots and clicks, and hover output includes what appeared. In HEADED mode (`scout_attach {headed:true}`, which the user asks for when they want to watch) the user's physical mouse competes with the synthetic pointer: if a hover reveals nothing and the finding matters, ask the user to move their mouse off the browser window and retry before filing. **Scroll long pages with `scout_scroll`** — the design audit and snapshot measure at the current scroll position, so judge deep sections by scrolling then re-auditing; it refuses to scroll where a real user couldn't and reports SCROLL LOCKED (the leaked modal scroll-lock that silently amputates everything below the fold — snapshots also flag it passively as an OVERLAY line), and scrolling triggers lazy-loaded content whose failures surface as fresh oracle violations. Elements fully clipped inside an overflow-hidden container are flagged UNREACHABLE in GEOMETRY issues — no amount of scrolling reveals them; that's a high-value layout bug, distinct from merely below-the-fold content. **A page can hold SEVERAL independent scroll regions** and plain `scout_scroll` moves the largest one, so a sidebar nav beside a taller main pane never budges: pass `scout_scroll {target:"testid=…"}` to scroll one region. Never report a nav item, tab or list row as missing/truncated until you have scrolled ITS container — content scrolled out of a secondary pane looks exactly like content that was cut off.
|
|
39
39
|
6. **The rest of the input vocabulary.** `scout_select` sets a `<select>` option by value or visible label — use it rather than clicking a native dropdown open, which does not render as page DOM. `scout_press` sends a real key to the focused element (`Escape` to dismiss a modal, `Tab` to walk focus order, `Enter` to submit from a field); it is also how the keyboard-only pass at `extensive` is performed, and it vets the focused control first so a destructive action cannot be triggered blind in read-only mode. **`scout_upload {ref}` attaches a file the way a user does** — `ref` is a visible `<input type=file>` (snapshots list these with role `file`; `scout_type` on one redirects here) OR the button/label/dropzone that opens the file chooser (the chooser is intercepted and answered — that is how the hidden input behind a styled "Choose file" control is reached); omit `ref` when the page has exactly one file input, hidden or not (snapshots disclose hidden ones on a FILE INPUTS line). Nothing needs to exist on disk: a small VALID fixture is generated in memory, its kind inferred from the input's `accept` attribute or chosen with `fixture` (`pdf`, `png`, `txt`, `csv`, `json`); `filePath` uploads a real file but must live inside the attached project (fenced, like navigation is fenced to the origin); `name` overrides the filename. The result flags a file that violates `accept` (a mismatch the app then ACCEPTS is a validation finding), warns when the app cleared the input after selection, and says whether a state-changing request fired on selection — if none did, either click the form's submit or read the next snapshot for a client-side rejection. Plans take `{action:"upload", target, value:"pdf"}` steps (`target` required). When the input or its trigger was addressed by `ref`, the gap ledger counts an attached-but-unsent file as filled-never-submitted; the ref-less path has no listed element to mark.
|
|
40
|
-
7. **
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
40
|
+
7. **Say what you are doing: `objective` is required before a tool acts.** Every tool that changes the app or the page — `scout_navigate`, `scout_back`, `scout_click`, `scout_type`, `scout_select`, `scout_press`, `scout_upload`, `scout_run_plan` — takes an `objective`: one short sentence naming what this batch of actions is FOR, in the words you would use to tell a colleague ("Sign in as QA_Team and check where it lands", "Fill the deviation form with invalid dates", "Walk the approval queue as a manager"). It STAYS SET until you pass a different one, so a batch costs one sentence, not one per call — pass a fresh one whenever you move on to something else, and keep it true rather than letting it describe what you finished ten calls ago. Acting with none standing is refused: the person watching the live view would otherwise see a session clicking through their app with nothing to say why. `scout_journey {action:"start", goal:…}` sets it too and outranks it while it runs — use a journey when you are MEASURING a whole user task, the parameter for everything else.
|
|
41
|
+
|
|
42
|
+
8. **Design-connoisseur pass without pixels: `scout_design_audit`.** Run it once per representative page (dashboard, a form, a detail view, a data table). Its output has two tiers: **⚠ measurable defects** (WCAG contrast, tiny targets, clipped text, aspect-distorted images, horizontal overflow, keyboard tab stops with no visible focus indicator — sampled with real Tab presses) and **→ craft suggestions** (line measure and line-height rhythm, spacing-grid adherence, typography entropy, gray census and accent-hue count, pure-#000 body text, elevation/control consistency, heading structure, indistinguishable links, AI-slop tells like gradient text/glassmorphism/side-stripe borders/identical card grids), closing with a SYSTEM SUMMARY of design-system coherence. Judge every line with product context (dense tables legitimately have small targets; a chart page legitimately uses many hues). File ⚠ defects as `visual`/`a11y`, and genuine → opportunities as `ux-polish` findings **quoting the concrete numbers** — "~142 characters per line (65–75 ideal)" beats "text feels wide". Every audit ends with a **PAGE SCORE** (0–100 overall + a11y/craft/consistency/task-clarity subscores) persisted per route — the report ranks pages worst-first, so re-runs show whether pages got better or worse. Separately, every `scout_snapshot` runs an **overlay/modal probe** automatically: an empty dialog over a grayed page, a backdrop with no dialog, a far-off-centre dialog leaving a blank band, or a dialog extending unreachably below the viewport appear as OVERLAY lines in GEOMETRY issues — treat these as high-value findings (the user is visually stuck). This is where "how could this page be better" gets answered, not just "is it broken".
|
|
43
|
+
9. **Measure task EASE with `scout_journey`, not just correctness.** Wrap each module's primary task (`{action:"start", goal:"Create an order"}` → do it → `{action:"end", completed:…}`). Navigate by CLICKING like a first-time user — typing a known deep URL shortcuts the very thing being measured (a route you can only reach by editing the address bar is itself a finding). The result gives interaction count, distinct screens, the path taken, and BACKTRACKS — returning to a screen already left is the clearest evidence the next step wasn't discoverable. An abandoned journey (`completed:false`) is a high-severity finding: the task is blocked or undiscoverable, which no passing e2e suite would ever reveal.
|
|
44
|
+
10. **Walk the auth surface too — anonymously.** Attach a second session WITHOUT a storage-state file (a fresh logged-out profile) and exercise signup, login failure states, and forgot/reset-password **as far as they physically go**. The mailbox wall is expected — reaching "check your email" IS the success condition; everything before it is what you're testing: does submit actually fire (a dead signup button is a high finding), are errors specific and actionable, can the user resend or recover from a typo, does the flow dead-end. Use plausible synthetic identities only (invent `qa-<runid>@example.com`-style addresses, never a real person's), submit each form valid AND invalid, and judge the feedback. Two classic findings live here: a forgot-password that answers "no account with that email" is an **account-enumeration leak** (file as security; "if an account exists, we sent a link" is the correct shape), and a signup that accepts the form then lands on a blank or logged-out page with no guidance is a **journey dead-end**. Signup creates a record, so what this pass may do depends on the mode. In `observe`, fill and submit the auth forms for their CLIENT-SIDE behaviour only: the engine blocks signup, password change and reset, and lets only a login itself go out. Disclose the server-side half as a gap. Actually creating an account needs the user's explicit okay and safe-write mode; the engine tracks the created account like any other creation.
|
|
45
|
+
11. **`scout_coverage` decides what's next** — it lists unvisited routes and unexercised elements. Trust it over your memory. Prefer reaching routes by clicking real navigation; fall back to direct URLs for coverage completeness and re-verification, and say which you used when it affects the finding (see the provenance rule below).
|
|
44
46
|
|
|
45
47
|
## Levels (completion contracts — the engine ENFORCES them via `scout_report {level}`)
|
|
46
48
|
|