scenescout 3.22.0 → 3.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/README.md +1 -0
- package/dist/ci-run.js +259 -36
- package/dist/cli.js +1 -1
- package/dist/engine/brief.js +35 -3
- package/dist/engine/ci-lanes.js +28 -12
- package/dist/engine/ci.js +39 -1
- package/dist/engine/from-run.js +649 -0
- package/dist/engine/memory.js +123 -0
- package/dist/engine/report.js +26 -0
- package/dist/mcp-server.js +63 -5
- package/package.json +1 -1
- package/skills/scenescout/SKILL.md +2 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
|
1
1
|
# scenescout
|
|
2
2
|
|
|
3
|
+
## 3.23.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- ed24973: Start a run from an earlier run's record, opt-in. Every run that writes its report now keeps a record of what it worked on and left (routes in order, each session's steps, and on each route the controls never exercised, forms never submitted and options never chosen), in the project's memory and in `ci.json` under `record`. `scenescout ci --from-run <ci.json or project directory>` (or `SCENESCOUT_FROM_RUN`) with `--from-run-mode continue` (the default) takes first the routes that run never worked on, then the ones it left work on, told exactly what to do first on each, then the rest, reaching a page the earlier run got to by acting on another page through the same steps; a run takes on only its budget's worth of pages (`--from-run-turns-per-page`, default 7 turns a page) and the next continued run takes the pages after them; `--from-run-mode replay` follows its routes and steps in order, one lane per session it had. `scout_lane_brief` takes `fromRun` and `fromRunMode`, the ci action takes `from-run` and `from-run-mode`, and the run it started from is named in the report, `summary.md` and `ci.json` (`fromRun`). Without it, runs behave as before.
|
|
8
|
+
|
|
3
9
|
## 3.22.0
|
|
4
10
|
|
|
5
11
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -485,6 +485,7 @@ npx scenescout ci http://127.0.0.1:3000
|
|
|
485
485
|
- **Caps:** at most 80 model turns, 3,000,000 tokens and 20 minutes (`--max-turns`, `--max-tokens`, `--max-minutes`), shared by two model loops that each explore their own part of the app (`--lanes`, default 2). The first cap reached ends the exploration; the report is still written, and says which cap ended it. On the benchmark's demo app a run at these defaults cost about $0.03 on `gpt-6-luna` ([docs/benchmark.md](docs/benchmark.md#choosing-the-defaults-issue-419)).
|
|
486
486
|
- **Mode:** `read-only` by default; `--mode observe` sends no form at all, `--mode safe-write` lets the run create records and change only the ones it created. `--mode destructive` runs only with `--allow-destructive` as well.
|
|
487
487
|
- **Duplicates:** when the dedup rule keeps a filed finding apart, the run's model is asked at its lowest effort whether it is one already open on the same page, and merges it on a "same", keeping the filing's title, category, severity and evidence under that finding. The two findings' titles, categories and evidence, and the page's path, are sent; `--dedup rule` turns it off ([ADR 17](docs/adr/0017-a-model-judges-only-the-merges-the-rule-misses.md)).
|
|
488
|
+
- **Starting from an earlier run (opt-in):** every run that writes its report leaves a record in `ci.json` and the project's memory: the routes it worked on, its steps, and what it left on each route. `--from-run <ci.json or project directory>` continues where that run left off (routes it never worked on first, then the ones it left work on, with exactly which forms, options and controls to take first), and `--from-run-mode replay` follows its routes and steps in order. `scout_lane_brief {fromRun, fromRunMode}` does the same for a parallel run. On the demo app a chain of continued runs found 8 defects against 7 for fresh runs at the default budget, and 4, 3 and 4 against 4 at a budget cut to stand in for a larger app, within the noise, so it stays off unless asked for ([the measurement](docs/benchmark.md#starting-from-an-earlier-run-issue-418)).
|
|
488
489
|
- **Output**, in `.scenescout/ci/` (or `--out`): `report.md` and `report.html` (the report an agent's run writes), `summary.md` (also appended to the GitHub job summary), `ci.json` and `ci.sarif`, with a usage line: turns, tokens, time and an estimated cost where the model's price is known (`--price-in`, `--price-out` give one for any model).
|
|
489
490
|
|
|
490
491
|
There is a GitHub Action for it (`uses: brunoboto96/SceneScout/ci@…`). [docs/ci.md](docs/ci.md#an-unattended-exploratory-run) has the workflow and every option; [ADR 14](docs/adr/0014-an-unattended-run-reports-and-never-gates.md) says why it works this way.
|
package/dist/ci-run.js
CHANGED
|
@@ -22,9 +22,10 @@ import { CreateMessageRequestSchema, ErrorCode, McpError } from "@modelcontextpr
|
|
|
22
22
|
import { DEDUP_JUDGE_CAPABILITY, durationText, JUDGE_CALL_MS, JUDGE_MAX_OUTPUT_TOKENS, JUDGE_SYSTEM, JUDGE_TOOL, judgeKickoffOf, samplingResultOf, } from "./engine/dedup.js";
|
|
23
23
|
import { CAPTURE_MARGIN, parseCaptureResult, rebaseUrl, SHOT_FILES, SHOTS_DIRNAME } from "./engine/capture.js";
|
|
24
24
|
import { addUsage, attachFailure, budgetSpend, wallLeftMs, guardToolArgs, CAPTURE_TOOLS, childEnv, ciCaptureKickoff, ciCaptureSystemPrompt, CI_DIRNAME, ciExitCode, ciKickoff, ciSarif, ciSummaryJson, ciSummaryMarkdown, ciSystemPrompt, ciToolArgs, ciTools, describeStop, findingsThisRun, newBudget, NO_USAGE, readFindings, redactKeys, settleTurn, takeTurn, toolResultText, usageLine, } from "./engine/ci.js";
|
|
25
|
-
import { ciLaneKickoff, ciLaneSystemPrompt, crawlFoundNothing, crawlNotes, LANE_TOOLS, mergeLaneStops, PLAN_CRAWL_ROUNDS, planCiLanes, PLANNER_SESSION, } from "./engine/ci-lanes.js";
|
|
25
|
+
import { ciLaneKickoff, ciLaneSystemPrompt, crawlFoundNothing, crawlNotes, LANE_TOOLS, mergeLaneStops, PLAN_CRAWL_ROUNDS, planCiLanes, plannedRoutes, planReplayLanes, PLANNER_SESSION, } from "./engine/ci-lanes.js";
|
|
26
26
|
import { resolveTimeLimits } from "./engine/limits.js";
|
|
27
|
-
import { MEMORY_DIRNAME, writeSelfIgnore } from "./engine/memory.js";
|
|
27
|
+
import { loadRunRecord, MEMORY_DIRNAME, noteFromRunOnDisk, readRunRecordsOnDisk, writeSelfIgnore } from "./engine/memory.js";
|
|
28
|
+
import { carryForward, continueLines, continueExhausted, continuePlan, DEFAULT_TURNS_PER_PAGE, pageCap, takenPages, fromRunLine, isPattern, pathLine, prefixOutcome, prefixPlan, replayLines, replayPlan, } from "./engine/from-run.js";
|
|
28
29
|
import { sarifFilesFor } from "./engine/sarif.js";
|
|
29
30
|
import { decodePng, diffImages, encodePng } from "./engine/png.js";
|
|
30
31
|
import { AnthropicConversation, backoffMs, errorMessage, MalformedReply, OpenAIConversation, retryable, retryAfterMs, } from "./engine/provider.js";
|
|
@@ -434,43 +435,119 @@ function takesSession(t) {
|
|
|
434
435
|
}
|
|
435
436
|
const messageOf = (err) => (err instanceof Error ? err.message : String(err));
|
|
436
437
|
/**
|
|
437
|
-
*
|
|
438
|
-
*
|
|
439
|
-
*
|
|
440
|
-
*
|
|
441
|
-
*
|
|
442
|
-
*
|
|
443
|
-
* runs the agent loop there with its own conversation, draws its turns from
|
|
444
|
-
* the run's one budget, and is closed when it ends. Their findings are already
|
|
445
|
-
* one: every session files into the project's one memory, whose dedup folds a
|
|
446
|
-
* defect two lanes filed. `outcome` is absent when there was nothing to split,
|
|
447
|
-
* and the run explores in one loop instead.
|
|
438
|
+
* Take the earlier run's path to a page (from-run.ts prefixPlan) in one
|
|
439
|
+
* scout_run_plan call, which the write policy governs like any other step.
|
|
440
|
+
* Nothing happens when the record never reached the page or opened it by its
|
|
441
|
+
* address. When a step cannot be repeated (the record cannot say it, or the
|
|
442
|
+
* page changed), the session navigates to the page directly instead, if it is
|
|
443
|
+
* not a pattern. Returns the note for the record and the line the model is told.
|
|
448
444
|
*/
|
|
449
|
-
async function
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
445
|
+
async function reachByPath(o) {
|
|
446
|
+
if (!o.route)
|
|
447
|
+
return { lines: [] };
|
|
448
|
+
const plan = prefixPlan(o.record, o.route);
|
|
449
|
+
if (!plan || ("steps" in plan && plan.direct))
|
|
450
|
+
return { lines: [] };
|
|
451
|
+
let why;
|
|
452
|
+
let steps = 0;
|
|
453
|
+
if ("cannot" in plan)
|
|
454
|
+
why = `its path cannot be repeated: ${plan.cannot}`;
|
|
455
|
+
else {
|
|
456
|
+
steps = plan.steps.length;
|
|
457
|
+
const r = await o.host
|
|
458
|
+
.call("scout_run_plan", { session: o.session, steps: plan.steps, task: `Taking the earlier run's path to ${o.route}` }, Math.min(240_000, o.timeLeft()))
|
|
459
|
+
.catch((err) => ({ text: `ERROR: ${messageOf(err)}`, isError: true }));
|
|
460
|
+
const out = prefixOutcome(r.text.replace(/^\[session [^\]\n]*\]\n/, ""), steps, o.route);
|
|
461
|
+
if (out.ok) {
|
|
462
|
+
o.say(`reached ${o.route} by the earlier run's path (${steps} step(s)).`);
|
|
463
|
+
return {
|
|
464
|
+
note: { session: o.session, target: o.route, steps, outcome: "replayed" },
|
|
465
|
+
lines: [`Your browser reached ${o.route} the way the earlier run did: ${pathLine(plan.steps)}. Start there.`],
|
|
466
|
+
};
|
|
467
|
+
}
|
|
468
|
+
why = out.why;
|
|
469
|
+
}
|
|
470
|
+
o.say(`could not take the earlier run's path to ${o.route} (${why}); navigating to it instead.`);
|
|
471
|
+
const direct = isPattern(o.route)
|
|
472
|
+
? undefined
|
|
473
|
+
: await o.host
|
|
474
|
+
.call("scout_navigate", { session: o.session, target: o.route, task: `Opening ${o.route}` }, Math.min(120_000, o.timeLeft()))
|
|
475
|
+
.catch((err) => ({ text: `ERROR: ${messageOf(err)}`, isError: true }));
|
|
476
|
+
const opened = !!direct && !direct.isError && !/^(ERROR|REFUSED):/.test(direct.text.replace(/^\[session [^\]\n]*\]\n/, ""));
|
|
477
|
+
return {
|
|
478
|
+
note: { session: o.session, target: o.route, steps, outcome: opened ? "navigated" : "unreached", why: why.slice(0, 300) },
|
|
479
|
+
lines: [
|
|
480
|
+
opened
|
|
481
|
+
? `The earlier run's path to ${o.route} could not be repeated (${why}), so your browser was sent there directly.`
|
|
482
|
+
: `The earlier run's path to ${o.route} could not be repeated (${why}); reach it from the page you are on.`,
|
|
483
|
+
],
|
|
484
|
+
};
|
|
485
|
+
}
|
|
486
|
+
/** The planning crawl's notes with these routes added, each with no note: routes a record knew that the crawl did not list. */
|
|
487
|
+
function withRoutes(notes, routes) {
|
|
488
|
+
const out = new Map(notes);
|
|
489
|
+
for (const r of routes)
|
|
490
|
+
if (!out.has(r))
|
|
491
|
+
out.set(r, []);
|
|
492
|
+
return out;
|
|
493
|
+
}
|
|
494
|
+
/**
|
|
495
|
+
* The record this run's report left in the project's memory (from-run.ts), for
|
|
496
|
+
* ci.json: the newest written since the run started. A continued run's carries
|
|
497
|
+
* the record it continued, so a chain of runs each given the last one's
|
|
498
|
+
* ci.json accumulates what the chain covered. Never fatal: without one, ci.json
|
|
499
|
+
* has no record, and the log says why.
|
|
500
|
+
*/
|
|
501
|
+
function thisRunsRecord(o) {
|
|
502
|
+
let own;
|
|
503
|
+
try {
|
|
504
|
+
own = readRunRecordsOnDisk(o.projectDir)
|
|
505
|
+
.filter((r) => r.at >= o.since)
|
|
506
|
+
.at(-1);
|
|
507
|
+
}
|
|
508
|
+
catch (err) {
|
|
509
|
+
o.log(`The run's record could not be read from the project's memory, so ci.json holds none: ${messageOf(err)}`);
|
|
510
|
+
return undefined;
|
|
511
|
+
}
|
|
512
|
+
if (!own) {
|
|
513
|
+
o.log("The report left no run record in the project's memory, so ci.json holds none.");
|
|
514
|
+
return undefined;
|
|
515
|
+
}
|
|
516
|
+
const withPrefixes = {
|
|
517
|
+
...own,
|
|
518
|
+
...(o.prefixes && o.prefixes.length > 0 ? { prefixes: [...o.prefixes] } : {}),
|
|
519
|
+
...(o.continuedFresh ? { continuedFresh: true } : {}),
|
|
520
|
+
...(o.assigned && o.assigned.length > 0 ? { assigned: [...new Set(o.assigned)] } : {}),
|
|
521
|
+
};
|
|
522
|
+
return o.continued ? carryForward(o.continued, withPrefixes) : withPrefixes;
|
|
523
|
+
}
|
|
524
|
+
/**
|
|
525
|
+
* The planning crawl a run split into lanes, or a continued run, starts with: a
|
|
526
|
+
* snapshot from the planner's session (attaching harvests no links; a snapshot
|
|
527
|
+
* of the page it landed on does), then a crawl repeated while it finds routes,
|
|
528
|
+
* up to PLAN_CRAWL_ROUNDS. No model call: only its time counts.
|
|
529
|
+
*/
|
|
530
|
+
async function planningCrawl(o) {
|
|
453
531
|
// What went wrong while planning, so a plan left with nothing to split says why rather than blaming the app.
|
|
454
532
|
let planningFailed;
|
|
455
533
|
/** One planner call: its text, or undefined after saying why it failed. */
|
|
456
534
|
const plannerCall = async (tool, maxMs, what) => {
|
|
457
535
|
try {
|
|
458
|
-
const r = await host.call(tool, { session: PLANNER_SESSION }, Math.min(maxMs, timeLeft()));
|
|
536
|
+
const r = await o.host.call(tool, { session: PLANNER_SESSION }, Math.min(maxMs, o.timeLeft()));
|
|
459
537
|
if (r.isError)
|
|
460
538
|
throw new Error(r.text.replace(/^ERROR:\s*/, ""));
|
|
461
539
|
return r.text;
|
|
462
540
|
}
|
|
463
541
|
catch (err) {
|
|
464
542
|
planningFailed = `the planning ${what} failed: ${messageOf(err).slice(0, 300)}`;
|
|
465
|
-
log(
|
|
543
|
+
o.log(`${o.label}: ${planningFailed}.`);
|
|
466
544
|
return undefined;
|
|
467
545
|
}
|
|
468
546
|
};
|
|
469
|
-
|
|
470
|
-
if (timeLeft() > 0)
|
|
547
|
+
if (o.timeLeft() > 0)
|
|
471
548
|
await plannerCall("scout_snapshot", 120_000, "snapshot");
|
|
472
549
|
const notes = new Map();
|
|
473
|
-
for (let round = 0; round < PLAN_CRAWL_ROUNDS && timeLeft() > 0; round += 1) {
|
|
550
|
+
for (let round = 0; round < PLAN_CRAWL_ROUNDS && o.timeLeft() > 0; round += 1) {
|
|
474
551
|
const text = await plannerCall("scout_crawl", 600_000, "crawl");
|
|
475
552
|
if (text === undefined)
|
|
476
553
|
break;
|
|
@@ -481,14 +558,24 @@ async function exploreInLanes(o) {
|
|
|
481
558
|
if (crawlFoundNothing(text) || notes.size === known)
|
|
482
559
|
break;
|
|
483
560
|
}
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
561
|
+
return { notes, ...(planningFailed ? { planningFailed } : {}) };
|
|
562
|
+
}
|
|
563
|
+
/**
|
|
564
|
+
* A run split into lanes (--lanes): plan, then run every lane at once, then
|
|
565
|
+
* fold what they did. The plan is a snapshot and a crawl from the planner's
|
|
566
|
+
* session (no model call: only their time counts), the crawl repeated while it
|
|
567
|
+
* finds routes, and the split brief.ts makes of them. Each lane attaches its
|
|
568
|
+
* own session on the run's target URL, as the planner did (the engine resolves
|
|
569
|
+
* every path against the URL a session attached with), opens its first route,
|
|
570
|
+
* runs the agent loop there with its own conversation, draws its turns from
|
|
571
|
+
* the run's one budget, and is closed when it ends. Their findings are already
|
|
572
|
+
* one: every session files into the project's one memory, whose dedup folds a
|
|
573
|
+
* defect two lanes filed. `outcome` is absent when there was nothing to split,
|
|
574
|
+
* and the run explores in one loop instead.
|
|
575
|
+
*/
|
|
576
|
+
async function exploreInLanes(o) {
|
|
577
|
+
const { host, options, log, now, budget, plan } = o;
|
|
578
|
+
const timeLeft = () => wallLeftMs(budgetSpend(budget), options.caps, now());
|
|
492
579
|
if (plan.oneLoop) {
|
|
493
580
|
log(`Lanes: ${plan.oneLoop}. Exploring in one loop.`);
|
|
494
581
|
return { lanes: { asked: options.lanes, sessions: [], oneLoop: plan.oneLoop } };
|
|
@@ -539,7 +626,9 @@ async function exploreInLanes(o) {
|
|
|
539
626
|
on = lane.url;
|
|
540
627
|
}
|
|
541
628
|
say(`attached on ${new URL(on).pathname}, owning ${lane.modules.join(", ")}.`);
|
|
542
|
-
|
|
629
|
+
// A continued run: the earlier run's path to this lane's first page, when it got there by acting on another.
|
|
630
|
+
const reached = o.reach && timeLeft() > 0 ? await o.reach(lane.session, lane.routes[0], say) : [];
|
|
631
|
+
return await exploreLane(lane, on, say, { ...result, attached: true }, reached);
|
|
543
632
|
}
|
|
544
633
|
catch (err) {
|
|
545
634
|
// Not a cap and not the model's API: the lane itself broke. Reported as the lane's, and the run's (mergeLaneStops), never dropped.
|
|
@@ -548,7 +637,7 @@ async function exploreInLanes(o) {
|
|
|
548
637
|
return { ...result, attached, stop: "could-not-start", stopDetail: why };
|
|
549
638
|
}
|
|
550
639
|
};
|
|
551
|
-
const exploreLane = async (lane, on, say, result) => {
|
|
640
|
+
const exploreLane = async (lane, on, say, result, reached = []) => {
|
|
552
641
|
const outcome = await agentLoop({
|
|
553
642
|
client: o.makeClient(system, tools, ciLaneKickoff({
|
|
554
643
|
lane: { ...lane, url: on },
|
|
@@ -559,6 +648,7 @@ async function exploreInLanes(o) {
|
|
|
559
648
|
level: options.level,
|
|
560
649
|
focus: options.focus,
|
|
561
650
|
caps: options.caps,
|
|
651
|
+
...(o.laneLines || reached.length > 0 ? { fromRun: [...(o.laneLines?.(lane) ?? []), ...reached] } : {}),
|
|
562
652
|
})),
|
|
563
653
|
host,
|
|
564
654
|
tools,
|
|
@@ -625,6 +715,11 @@ export async function runCi(options, resolved, deps) {
|
|
|
625
715
|
let reportWritten = false;
|
|
626
716
|
let capture;
|
|
627
717
|
let lanes;
|
|
718
|
+
let runFromRun;
|
|
719
|
+
let runRecord;
|
|
720
|
+
let earlier;
|
|
721
|
+
// The real clock, as the server's: this run's record is the one its report wrote after this.
|
|
722
|
+
const startedIso = new Date().toISOString();
|
|
628
723
|
let host = null;
|
|
629
724
|
// Set when the exploration ends (or never starts): the report and the close share FINISH_MS from then.
|
|
630
725
|
let finishBy = 0;
|
|
@@ -636,6 +731,17 @@ export async function runCi(options, resolved, deps) {
|
|
|
636
731
|
const judge = judgeAsk
|
|
637
732
|
? judgeHandler({ ask: judgeAsk, model: resolved.model, spend: budget, caps: options.caps, calls: judgeCalls, secrets, now })
|
|
638
733
|
: undefined;
|
|
734
|
+
// The earlier run's record, read before anything starts: a run asked to continue or replay one must not quietly start fresh.
|
|
735
|
+
if (options.fromRun && !options.show) {
|
|
736
|
+
try {
|
|
737
|
+
earlier = loadRunRecord(options.fromRun.path);
|
|
738
|
+
}
|
|
739
|
+
catch (err) {
|
|
740
|
+
throw new Error(`--from-run: ${messageOf(err)}`);
|
|
741
|
+
}
|
|
742
|
+
if (options.fromRun.mode === "replay" && replayPlan(earlier).length === 0)
|
|
743
|
+
throw new Error(`--from-run: the run recorded in ${options.fromRun.path} took no steps, so there is nothing to replay`);
|
|
744
|
+
}
|
|
639
745
|
host = await (deps.startHost ?? startServer)(log, judge);
|
|
640
746
|
// What every session of the run attaches with: the planner's here, and each lane's when the run is split.
|
|
641
747
|
const attachArgs = (a) => ({
|
|
@@ -666,10 +772,67 @@ export async function runCi(options, resolved, deps) {
|
|
|
666
772
|
const listed = await host.tools();
|
|
667
773
|
const toolHost = host;
|
|
668
774
|
let captured = null;
|
|
669
|
-
|
|
775
|
+
// Named as it was given: a resolved path would put a local directory in the report and ci.json.
|
|
776
|
+
const from = options.fromRun ? (options.fromRun.given ?? options.fromRun.path) : "";
|
|
777
|
+
const replay = earlier && options.fromRun?.mode === "replay" ? replayPlan(earlier) : undefined;
|
|
778
|
+
const continuing = earlier && options.fromRun?.mode === "continue" ? earlier : undefined;
|
|
779
|
+
// A run split into lanes, or a continued one, crawls first: the lanes are split, and a continued run's routes ordered, from what it finds.
|
|
780
|
+
// A replay does not: its routes and steps are the record's.
|
|
781
|
+
const timeLeft = () => wallLeftMs(budgetSpend(budget), options.caps, now());
|
|
782
|
+
const plans = !options.show && !replay && (options.lanes > 1 || !!continuing);
|
|
783
|
+
const planned = plans ? await planningCrawl({ host, log, timeLeft, label: options.lanes > 1 ? "Lanes" : "From run" }) : undefined;
|
|
784
|
+
const planItems = continuing && planned ? continuePlan(continuing, plannedRoutes(options.url, planned.notes)) : undefined;
|
|
785
|
+
// A record with no work left anywhere: explore as a fresh run (the stable split, the landing page, no path), and say so.
|
|
786
|
+
const continuedFresh = !!planItems && continueExhausted(planItems);
|
|
787
|
+
const items = continuedFresh ? undefined : planItems;
|
|
788
|
+
// How many pages a continued run takes on, from its budget (from-run.ts pageCap): each lane's share of the turns, or the run's.
|
|
789
|
+
const perPage = options.fromRun?.turnsPerPage ?? DEFAULT_TURNS_PER_PAGE;
|
|
790
|
+
const cap = items ? pageCap(options.caps.turns, perPage) : undefined;
|
|
791
|
+
const laneCap = items ? pageCap(Math.floor(options.caps.turns / Math.max(1, options.lanes)), perPage) : undefined;
|
|
792
|
+
// The pages this run was given, kept in its record so the next continued run takes the next ones.
|
|
793
|
+
const assigned = [];
|
|
794
|
+
if (options.fromRun && earlier && !options.show) {
|
|
795
|
+
runFromRun = { mode: options.fromRun.mode, source: from, runId: earlier.runId, recordAt: earlier.at };
|
|
796
|
+
log(`From run: ${fromRunLine(runFromRun)}.`);
|
|
797
|
+
if (items)
|
|
798
|
+
log(`From run: ${items.filter((i) => i.tier === 1).length} route(s) it never worked on, ${items.filter((i) => i.tier === 2).length} with work left, ${items.filter((i) => i.tier === 3).length} worked through.`);
|
|
799
|
+
if (continuedFresh)
|
|
800
|
+
log("From run: the earlier run left no recorded work on any route, so this run explores as a fresh one.");
|
|
801
|
+
// Noted in the project's memory for the report, which the server writes. Never fatal: the run goes on.
|
|
802
|
+
try {
|
|
803
|
+
noteFromRunOnDisk(options.projectDir, { at: new Date().toISOString(), ...runFromRun });
|
|
804
|
+
}
|
|
805
|
+
catch (err) {
|
|
806
|
+
log(`From run: the project's memory could not be told, so the report will not name the earlier run: ${messageOf(err)}`);
|
|
807
|
+
}
|
|
808
|
+
}
|
|
809
|
+
// How a continued run's lanes (or its one loop) reached their first page by the earlier run's path, for the record.
|
|
810
|
+
const prefixes = [];
|
|
811
|
+
const reach = items
|
|
812
|
+
? async (session, route, say) => {
|
|
813
|
+
const r = await reachByPath({ host: toolHost, session, record: continuing, route, timeLeft, say });
|
|
814
|
+
if (r.note)
|
|
815
|
+
prefixes.push(r.note);
|
|
816
|
+
return r.lines;
|
|
817
|
+
}
|
|
818
|
+
: undefined;
|
|
819
|
+
const oneLoop = async () => {
|
|
670
820
|
const tools = ciTools(listed, options.show ? CAPTURE_TOOLS : undefined);
|
|
671
821
|
const system = options.show ? ciCaptureSystemPrompt() : ciSystemPrompt(loadPlaybook(packageRoot), options);
|
|
672
|
-
const
|
|
822
|
+
const reached = reach && items && !options.show ? await reach(PLANNER_SESSION, items[0]?.route, (l) => log(`From run: ${l}`)) : [];
|
|
823
|
+
if (items && cap !== undefined) {
|
|
824
|
+
const mine = takenPages(items, cap);
|
|
825
|
+
assigned.push(...mine);
|
|
826
|
+
log(`From run: takes on ${mine.join(", ")} (${cap} page(s) at ${perPage} turns a page).`);
|
|
827
|
+
}
|
|
828
|
+
const fromRun = planItems
|
|
829
|
+
? [...continueLines(planItems, { from, ...(cap !== undefined ? { cap } : {}) }), ...reached]
|
|
830
|
+
: replay && replay.length > 0
|
|
831
|
+
? replayLines(replay[0], { from })
|
|
832
|
+
: undefined;
|
|
833
|
+
const kickoff = options.show
|
|
834
|
+
? ciCaptureKickoff({ url: options.url, show: options.show })
|
|
835
|
+
: ciKickoff({ ...options, ...(fromRun && fromRun.length > 0 ? { fromRunLines: fromRun } : {}) });
|
|
673
836
|
return agentLoop({
|
|
674
837
|
client: deps.makeClient(system, tools, kickoff),
|
|
675
838
|
host: toolHost,
|
|
@@ -685,8 +848,56 @@ export async function runCi(options, resolved, deps) {
|
|
|
685
848
|
},
|
|
686
849
|
});
|
|
687
850
|
};
|
|
688
|
-
|
|
689
|
-
|
|
851
|
+
// A replay has as many lanes as the recorded run had sessions; otherwise the planning crawl is split as asked.
|
|
852
|
+
const lanePlan = options.show
|
|
853
|
+
? undefined
|
|
854
|
+
: replay
|
|
855
|
+
? planReplayLanes({ target: options.url, lanes: replay, ...(options.focus ? { focus: options.focus } : {}) })
|
|
856
|
+
: options.lanes > 1 && planned
|
|
857
|
+
? planCiLanes({
|
|
858
|
+
target: options.url,
|
|
859
|
+
// A continued run splits every route it orders, the record's own included, so none is left out of every lane.
|
|
860
|
+
notes: items
|
|
861
|
+
? withRoutes(planned.notes, items.map((i) => i.route))
|
|
862
|
+
: planned.notes,
|
|
863
|
+
count: options.lanes,
|
|
864
|
+
focus: options.focus,
|
|
865
|
+
mode: options.mode,
|
|
866
|
+
...(planned.planningFailed ? { planningFailed: planned.planningFailed } : {}),
|
|
867
|
+
...(items ? { order: items.map((i) => i.route) } : {}),
|
|
868
|
+
})
|
|
869
|
+
: undefined;
|
|
870
|
+
if (replay && replay.length > 1 && replay.length !== options.lanes)
|
|
871
|
+
log(`From run: replaying the recorded run's ${replay.length} sessions as ${replay.length} lanes.`);
|
|
872
|
+
if (lanePlan && (lanePlan.lanes.length > 0 || options.lanes > 1)) {
|
|
873
|
+
// A replayed lane is told its own session's steps: the plan keeps the recorded sessions' order.
|
|
874
|
+
const replayed = new Map(replay ? lanePlan.lanes.map((l, i) => [l.session, replay[i]]) : []);
|
|
875
|
+
const laneLines = planItems
|
|
876
|
+
? (lane) => {
|
|
877
|
+
if (items && laneCap !== undefined) {
|
|
878
|
+
const mine = takenPages(items, laneCap, lane.routes);
|
|
879
|
+
assigned.push(...mine);
|
|
880
|
+
log(`From run: lane ${lane.session} takes on ${mine.join(", ") || "no page with work left"} (${laneCap} page(s) at ${perPage} turns a page).`);
|
|
881
|
+
}
|
|
882
|
+
return continueLines(planItems, { from, only: lane.routes, ...(laneCap !== undefined ? { cap: laneCap } : {}) });
|
|
883
|
+
}
|
|
884
|
+
: replay
|
|
885
|
+
? (lane) => replayLines(replayed.get(lane.session), { from })
|
|
886
|
+
: undefined;
|
|
887
|
+
const split = await exploreInLanes({
|
|
888
|
+
host,
|
|
889
|
+
listed,
|
|
890
|
+
options,
|
|
891
|
+
makeClient: deps.makeClient,
|
|
892
|
+
log,
|
|
893
|
+
now,
|
|
894
|
+
budget,
|
|
895
|
+
attachArgs,
|
|
896
|
+
attachMs,
|
|
897
|
+
plan: lanePlan,
|
|
898
|
+
...(laneLines ? { laneLines } : {}),
|
|
899
|
+
...(reach ? { reach } : {}),
|
|
900
|
+
});
|
|
690
901
|
lanes = split.lanes;
|
|
691
902
|
outcome = split.outcome ?? (await oneLoop());
|
|
692
903
|
}
|
|
@@ -709,6 +920,16 @@ export async function runCi(options, resolved, deps) {
|
|
|
709
920
|
reportWritten = !report.isError && !/^ERROR:/.test(report.text) && fs.existsSync(path.join(options.projectDir, MEMORY_DIRNAME, "report.md"));
|
|
710
921
|
if (!reportWritten)
|
|
711
922
|
log(`The report could not be generated: ${report.text.slice(0, 400)}`);
|
|
923
|
+
else
|
|
924
|
+
runRecord = thisRunsRecord({
|
|
925
|
+
projectDir: options.projectDir,
|
|
926
|
+
since: startedIso,
|
|
927
|
+
continued: earlier && options.fromRun?.mode === "continue" ? earlier : undefined,
|
|
928
|
+
prefixes,
|
|
929
|
+
continuedFresh,
|
|
930
|
+
assigned,
|
|
931
|
+
log,
|
|
932
|
+
});
|
|
712
933
|
}
|
|
713
934
|
}
|
|
714
935
|
}
|
|
@@ -748,6 +969,8 @@ export async function runCi(options, resolved, deps) {
|
|
|
748
969
|
findings: findingsThisRun(before, readMemoryFindings(options.projectDir)),
|
|
749
970
|
...(capture ? { capture } : {}),
|
|
750
971
|
...(lanes ? { lanes } : {}),
|
|
972
|
+
...(runFromRun ? { fromRun: runFromRun } : {}),
|
|
973
|
+
...(runRecord ? { record: runRecord } : {}),
|
|
751
974
|
...(options.show
|
|
752
975
|
? {}
|
|
753
976
|
: {
|
package/dist/cli.js
CHANGED
|
@@ -777,7 +777,7 @@ async function ci(args) {
|
|
|
777
777
|
console.error(redactKeys(`scenescout ci: ${message}`, secrets));
|
|
778
778
|
process.exit(EXIT_CI.couldNotRun);
|
|
779
779
|
};
|
|
780
|
-
const parsed = parseCiArgs(args, process.cwd());
|
|
780
|
+
const parsed = parseCiArgs(args, process.cwd(), process.env);
|
|
781
781
|
if (!parsed.ok)
|
|
782
782
|
return fail(parsed.error);
|
|
783
783
|
const options = parsed.options;
|
package/dist/engine/brief.js
CHANGED
|
@@ -49,7 +49,7 @@ export function moduleOf(route) {
|
|
|
49
49
|
* to the point, is stable — the same routes produce the same split every time,
|
|
50
50
|
* so a re-run of a lane can be given the same brief.
|
|
51
51
|
*/
|
|
52
|
-
export function splitRoutes(routes, laneCount) {
|
|
52
|
+
export function splitRoutes(routes, laneCount, order) {
|
|
53
53
|
const lanes = Math.max(1, Math.min(Math.floor(laneCount) || 1, MAX_LANES));
|
|
54
54
|
const byModule = new Map();
|
|
55
55
|
for (const route of routes) {
|
|
@@ -63,9 +63,16 @@ export function splitRoutes(routes, laneCount) {
|
|
|
63
63
|
// Biggest first, then by name, and each module's own routes sorted: nothing
|
|
64
64
|
// about the split may depend on the order the routes were discovered in, or
|
|
65
65
|
// a lane that has to be re-run cannot be handed the same brief.
|
|
66
|
-
const
|
|
66
|
+
const sorted = [...byModule.entries()]
|
|
67
67
|
.map(([name, list]) => [name, [...list].sort((a, b) => a.localeCompare(b))])
|
|
68
68
|
.sort((a, b) => b[1].length - a[1].length || a[0].localeCompare(b[0]));
|
|
69
|
+
// Given an order (a run continuing an earlier one, from-run.ts), a route's
|
|
70
|
+
// place in it decides: the modules are dealt by their earliest route, and
|
|
71
|
+
// each lane takes its routes in that order, so every lane starts on what the
|
|
72
|
+
// order puts first. Routes it does not name go after, in the stable order.
|
|
73
|
+
const rank = order ? new Map(order.map((r, i) => [r, i])) : null;
|
|
74
|
+
const at = (r) => rank?.get(r) ?? Number.MAX_SAFE_INTEGER;
|
|
75
|
+
const modules = rank ? [...sorted].sort((a, b) => Math.min(...a[1].map(at)) - Math.min(...b[1].map(at))) : sorted;
|
|
69
76
|
const out = Array.from({ length: lanes }, () => ({ modules: [], routes: [] }));
|
|
70
77
|
for (const [name, list] of modules) {
|
|
71
78
|
let smallest = 0;
|
|
@@ -75,6 +82,9 @@ export function splitRoutes(routes, laneCount) {
|
|
|
75
82
|
out[smallest].modules.push(name);
|
|
76
83
|
out[smallest].routes.push(...list);
|
|
77
84
|
}
|
|
85
|
+
if (rank)
|
|
86
|
+
for (const lane of out)
|
|
87
|
+
lane.routes = [...lane.routes].sort((a, b) => at(a) - at(b));
|
|
78
88
|
// A lane with nothing to do is a browser held open for no reason.
|
|
79
89
|
return out.filter((lane) => lane.routes.length > 0);
|
|
80
90
|
}
|
|
@@ -93,7 +103,7 @@ function signInArgument(opts) {
|
|
|
93
103
|
}
|
|
94
104
|
/** The lanes to run, each with the objective to attach with. */
|
|
95
105
|
export function planLanes(routes, laneCount, opts = {}) {
|
|
96
|
-
return splitRoutes(routes, laneCount).map((lane, i) => ({
|
|
106
|
+
return splitRoutes(routes, laneCount, opts.order).map((lane, i) => ({
|
|
97
107
|
lane: laneName(lane.modules, i),
|
|
98
108
|
objective: laneObjective(lane.modules, opts.goal),
|
|
99
109
|
modules: lane.modules,
|
|
@@ -101,6 +111,23 @@ export function planLanes(routes, laneCount, opts = {}) {
|
|
|
101
111
|
landing: landingOf(lane.routes),
|
|
102
112
|
}));
|
|
103
113
|
}
|
|
114
|
+
/**
|
|
115
|
+
* The lanes of a replay (from-run.ts replayPlan): one per session of the
|
|
116
|
+
* recorded run, with its routes in the order it took them, not a new split.
|
|
117
|
+
* Named as a split's lanes are, by what they own, and made unique.
|
|
118
|
+
*/
|
|
119
|
+
export function replayBriefs(lanes, goal) {
|
|
120
|
+
const taken = new Set();
|
|
121
|
+
return lanes.map((l, i) => {
|
|
122
|
+
const modules = [...new Set(l.routes.map(moduleOf))];
|
|
123
|
+
const base = laneName(modules, i);
|
|
124
|
+
let lane = base;
|
|
125
|
+
for (let k = 2; taken.has(lane); k += 1)
|
|
126
|
+
lane = `${base.slice(0, LANE_NAME_MAX - `-${k}`.length)}-${k}`;
|
|
127
|
+
taken.add(lane);
|
|
128
|
+
return { lane, objective: laneObjective(modules, goal), modules, routes: [...l.routes], landing: landingOf(l.routes) };
|
|
129
|
+
});
|
|
130
|
+
}
|
|
104
131
|
/**
|
|
105
132
|
* The first of a lane's routes a browser can open as it stands. A route the
|
|
106
133
|
* engine normalised (`/orders/:id`) is a pattern, not an address; a lane with
|
|
@@ -143,6 +170,9 @@ export function formatBriefs(briefs, opts = {}) {
|
|
|
143
170
|
const mode = opts.mode ?? "read-only";
|
|
144
171
|
const lines = [
|
|
145
172
|
`LANE PLAN — ${briefs.length} lane(s) over ${briefs.reduce((n, b) => n + b.routes.length, 0)} route(s).`,
|
|
173
|
+
...(opts.fromRunNote
|
|
174
|
+
? [`This run ${opts.fromRunNote}. Each lane's routes are listed in the order to take them, and what to do first is under each lane: pass it on.`]
|
|
175
|
+
: []),
|
|
146
176
|
``,
|
|
147
177
|
`Give each lane its own agent. Every lane attaches with its own session name, so the browsers run genuinely in parallel, and lands on its own first route rather than the home page:`,
|
|
148
178
|
` scout_attach { session: "<lane>", url: "<origin><landing>", projectPath, mode: "${mode}"${signInArgument(opts)}, objective: "<objective>" }`,
|
|
@@ -158,6 +188,8 @@ export function formatBriefs(briefs, opts = {}) {
|
|
|
158
188
|
lines.push(`owns: ${b.modules.join(", ")} (${b.routes.length} route(s))`);
|
|
159
189
|
lines.push(`landing: ${b.landing}`);
|
|
160
190
|
lines.push(`routes: ${b.routes.slice(0, 20).join(", ")}${b.routes.length > 20 ? ` … and ${b.routes.length - 20} more` : ""}`);
|
|
191
|
+
for (const l of opts.laneLines?.(b) ?? [])
|
|
192
|
+
lines.push(l);
|
|
161
193
|
lines.push(``);
|
|
162
194
|
}
|
|
163
195
|
return lines.join("\n");
|
package/dist/engine/ci-lanes.js
CHANGED
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
* and how the lanes' endings become the run's. The loop that runs them is
|
|
9
9
|
* src/ci-run.ts; the shared budget is engine/ci.ts's. Why: ADR 20.
|
|
10
10
|
*/
|
|
11
|
-
import { LANE_RULES, planLanes } from "./brief.js";
|
|
11
|
+
import { LANE_RULES, planLanes, replayBriefs } from "./brief.js";
|
|
12
12
|
import { CI_TOOLS } from "./ci.js";
|
|
13
13
|
/** The session the run attaches first. It plans, holds no lane, and writes the report once every lane is done. */
|
|
14
14
|
export const PLANNER_SESSION = "default";
|
|
@@ -88,6 +88,10 @@ export function crawlNotes(text) {
|
|
|
88
88
|
export function crawlFoundNothing(text) {
|
|
89
89
|
return /^(?:\[session [^\]]*\]\s*)?(Nothing to crawl|Nothing new to crawl|No routes to crawl yet)/m.test(text);
|
|
90
90
|
}
|
|
91
|
+
/** The routes a run has to plan with: the page it attached on, then every route the planning crawl listed. */
|
|
92
|
+
export function plannedRoutes(target, notes) {
|
|
93
|
+
return [...new Set([routeOf(target), ...notes.keys()])];
|
|
94
|
+
}
|
|
91
95
|
/** The path a target URL opens, as a route: the planning crawl never lists the page the run attached on. */
|
|
92
96
|
function routeOf(url) {
|
|
93
97
|
try {
|
|
@@ -119,12 +123,12 @@ export function laneSessions(names) {
|
|
|
119
123
|
* run has nothing to split, and explores in one loop, saying why.
|
|
120
124
|
*/
|
|
121
125
|
export function planCiLanes(o) {
|
|
122
|
-
const routes =
|
|
126
|
+
const routes = plannedRoutes(o.target, o.notes);
|
|
123
127
|
if (o.count < 2)
|
|
124
128
|
return { lanes: [], oneLoop: "one lane was asked for" };
|
|
125
129
|
if (o.notes.size === 0 && o.planningFailed)
|
|
126
130
|
return { lanes: [], oneLoop: `${o.planningFailed}, so there was nothing to split` };
|
|
127
|
-
const briefs = planLanes(routes, o.count, { goal: o.focus, mode: o.mode });
|
|
131
|
+
const briefs = planLanes(routes, o.count, { goal: o.focus, mode: o.mode, ...(o.order ? { order: o.order } : {}) });
|
|
128
132
|
if (briefs.length < 2) {
|
|
129
133
|
const where = briefs[0]?.modules[0];
|
|
130
134
|
return {
|
|
@@ -132,16 +136,27 @@ export function planCiLanes(o) {
|
|
|
132
136
|
oneLoop: `the planning crawl found ${routes.length} route(s), all in ${where ? `one module (${where})` : "no module"}, so there was nothing to split`,
|
|
133
137
|
};
|
|
134
138
|
}
|
|
139
|
+
return { lanes: asCiLanes(briefs, o.target, o.notes) };
|
|
140
|
+
}
|
|
141
|
+
function asCiLanes(briefs, target, notes) {
|
|
135
142
|
const sessions = laneSessions(briefs.map((b) => b.lane));
|
|
136
|
-
return {
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
143
|
+
return briefs.map((b, i) => ({
|
|
144
|
+
...b,
|
|
145
|
+
session: sessions[i],
|
|
146
|
+
// On the target's origin whatever the route says: a path written `//host/x` must not become another host.
|
|
147
|
+
url: `${new URL(target).origin}${b.landing.startsWith("/") ? "" : "/"}${b.landing}`,
|
|
148
|
+
crawl: b.routes.flatMap((r) => notes.get(r) ?? []).slice(0, LIST_MAX),
|
|
149
|
+
}));
|
|
150
|
+
}
|
|
151
|
+
/**
|
|
152
|
+
* A replayed run's lanes (from-run.ts replayPlan): one per session the
|
|
153
|
+
* recorded run had, each with its routes in recorded order, however many lanes
|
|
154
|
+
* were asked for. With fewer than two the run replays in one loop.
|
|
155
|
+
*/
|
|
156
|
+
export function planReplayLanes(o) {
|
|
157
|
+
if (o.lanes.length < 2)
|
|
158
|
+
return { lanes: [], oneLoop: `the recorded run had ${o.lanes.length === 1 ? "one session" : "no steps"}, so it is replayed in one loop` };
|
|
159
|
+
return { lanes: asCiLanes(replayBriefs(o.lanes, o.focus), o.target, new Map()) };
|
|
145
160
|
}
|
|
146
161
|
// ── what a lane is told ─────────────────────────────────────────────────────
|
|
147
162
|
/**
|
|
@@ -171,6 +186,7 @@ export function ciLaneKickoff(o) {
|
|
|
171
186
|
`Target: ${o.url}`,
|
|
172
187
|
`Your lane: ${lane.objective}`,
|
|
173
188
|
`Your routes (${lane.routes.length}): ${lane.routes.slice(0, LIST_MAX).join(", ")}${lane.routes.length > LIST_MAX ? ` … and ${lane.routes.length - LIST_MAX} more` : ""}`,
|
|
189
|
+
...(o.fromRun ?? []),
|
|
174
190
|
`Your browser is on ${routeOf(lane.url)}.`,
|
|
175
191
|
lane.crawl.length > 0 ? `What the planning crawl saw on your routes:\n${lane.crawl.map((l) => ` ${l.trim()}`).join("\n")}` : "",
|
|
176
192
|
`Project directory (for scout_scan): ${o.projectDir}`,
|