pi-agent-browser-native 0.6.11 → 0.6.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -0
- package/README.md +1 -1
- package/dist/extensions/agent-browser/lib/input-modes/job.js +7 -7
- package/docs/ARCHITECTURE.md +1 -1
- package/docs/COMMAND_REFERENCE.md +2 -2
- package/docs/RELEASE.md +3 -1
- package/docs/SUPPORT_MATRIX.md +3 -1
- package/docs/TOOL_CONTRACT.md +15 -5
- package/docs/platform-smoke.md +9 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,17 @@
|
|
|
2
2
|
|
|
3
3
|
## Unreleased
|
|
4
4
|
|
|
5
|
+
## 0.6.12 - 2026-09-13
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
|
|
9
|
+
- Make QA non-pass when final page-error rows match a nonempty post-clear baseline. Report “page-error check could not be verified” instead of ignoring matches as unchanged; preserve separate novel-error counts, clean passes, and explicit `checkErrors: false` checks.
|
|
10
|
+
|
|
11
|
+
### Validation
|
|
12
|
+
|
|
13
|
+
- Keep real-browser fixture probes on the correct owned daemon and use matching private socket directories; preserve the cold first-snapshot and 12-second recording checks.
|
|
14
|
+
- Add a native hosted-Windows PR gate using the existing packed-Pi and browser suites. Document direct local macOS qualification without Remote Login; neither alternative is a Crabbox SSH/Parallels pass.
|
|
15
|
+
|
|
5
16
|
## 0.6.11 - 2026-09-12
|
|
6
17
|
|
|
7
18
|
### Added
|
package/README.md
CHANGED
|
@@ -540,7 +540,7 @@ After either path, use `qa: { "attached": true, ... }` for a current-session smo
|
|
|
540
540
|
|
|
541
541
|
### Lightweight QA preset
|
|
542
542
|
|
|
543
|
-
For a quick smoke/QA pass, use top-level `qa`. It compiles to the same batch path as `job` and uses `batch --bail` so failed readiness/text/selector assertions stop before slower diagnostics can burn the wrapper watchdog. The URL form clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks optional expected text or selector, inspects fresh network requests, console messages, and page errors when preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped output.
|
|
543
|
+
For a quick smoke/QA pass, use top-level `qa`. It compiles to the same batch path as `job` and uses `batch --bail` so failed readiness/text/selector assertions stop before slower diagnostics can burn the wrapper watchdog. The URL form clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks optional expected text or selector, inspects fresh network requests, console messages, and page errors when preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped output. Final page-error rows matching a nonempty post-clear baseline make QA non-pass: old residue and identical new errors cannot be distinguished, especially when the native buffer rolls over. `qaPreset.passed` is `false` with `failureCategory: "qa-failure"` and the reason “page-error check could not be verified”—not a claim that those rows prove new application errors. Definitely new rows, including matching errors after an empty baseline, still count as page errors. Clean checks with no ambiguous or new rows can pass; explicit `checkErrors: false` skips this check for a more limited smoke test. Expected text is checked with bounded visible-text `wait --fn … --timeout 5000` predicates after the requested load state so dense pages can pass on visible headings/copy and missing text becomes crisp QA evidence. The attached form (`qa: { "attached": true }`) runs checks against the current managed session, such as an attached Electron app, rejects `url`, and deliberately preserves existing diagnostics instead of clearing evidence; its diagnostic reads default off so stale buffers do not fail a current-page smoke unless `checkNetwork`, `checkConsole`, or `checkErrors` is explicitly `true`. `loadState` defaults to `"domcontentloaded"`; set it to `"load"` or `"networkidle"` only when the stricter state is useful and the site is not expected to keep background requests alive. For URL-opening QA, `checkNetwork`, `checkConsole`, and `checkErrors` default to true; set one to `false` to skip that diagnostic read. Network failures are classified by likely impact and failed rows are listed first in network previews: actionable document/script/API-style failures still fail QA, while some low-impact browser icon asset misses (for example certain `favicon` or `apple-touch-icon` paths when upstream marks the row failed and resource metadata looks image-like) surface only as warnings instead of failing an otherwise healthy smoke check (`details.qaPreset.warnings`, with human-readable `details.qaPreset.summary` when the preset still passes). Exact predicates live in [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#qa) and `classifyNetworkRequestFailure` in `extensions/agent-browser/lib/results/network.ts`.
|
|
544
544
|
|
|
545
545
|
```json
|
|
546
546
|
{
|
|
@@ -257,7 +257,7 @@ export function buildQaCompactPassText(options) {
|
|
|
257
257
|
lines.push(`Page: ${pageParts.join(" — ")}`);
|
|
258
258
|
lines.push(`Checks run: ${describeQaChecksRun(options.checks)} (${options.batchStepCount} batch step${options.batchStepCount === 1 ? "" : "s"})`);
|
|
259
259
|
if (options.checks.diagnosticsResetAtStart && (options.checks.checkNetwork || options.checks.checkConsole || options.checks.checkErrors)) {
|
|
260
|
-
lines.push("Diagnostic isolation: URL QA clears enabled
|
|
260
|
+
lines.push("Diagnostic isolation: URL QA requests clears of enabled diagnostic buffers before opening the target.");
|
|
261
261
|
}
|
|
262
262
|
if (options.checks.attached && !options.checks.diagnosticsResetAtStart && (options.checks.checkNetwork || options.checks.checkConsole || options.checks.checkErrors)) {
|
|
263
263
|
lines.push("Attached diagnostics: existing upstream session console/network/error buffers were preserved; rows may include events from before qa.attached started.");
|
|
@@ -368,17 +368,17 @@ function subtractQaBaselineErrors(errors, baselineErrors) {
|
|
|
368
368
|
const signature = qaErrorSignature(error);
|
|
369
369
|
baselineCounts.set(signature, (baselineCounts.get(signature) ?? 0) + 1);
|
|
370
370
|
}
|
|
371
|
-
let
|
|
371
|
+
let matchedCount = 0;
|
|
372
372
|
const novelErrors = errors.filter((error) => {
|
|
373
373
|
const signature = qaErrorSignature(error);
|
|
374
374
|
const count = baselineCounts.get(signature) ?? 0;
|
|
375
375
|
if (count === 0)
|
|
376
376
|
return true;
|
|
377
377
|
baselineCounts.set(signature, count - 1);
|
|
378
|
-
|
|
378
|
+
matchedCount += 1;
|
|
379
379
|
return false;
|
|
380
380
|
});
|
|
381
|
-
return {
|
|
381
|
+
return { matchedCount, novelErrors };
|
|
382
382
|
}
|
|
383
383
|
function isDiagnosticResetCommand(item) {
|
|
384
384
|
const command = item.command;
|
|
@@ -422,11 +422,11 @@ export function analyzeQaPresetResults(data, compiled) {
|
|
|
422
422
|
continue;
|
|
423
423
|
}
|
|
424
424
|
if (commandName === "errors" && Array.isArray(result?.errors) && result.errors.length > 0) {
|
|
425
|
-
const {
|
|
425
|
+
const { matchedCount, novelErrors } = subtractQaBaselineErrors(result.errors, baselineErrors);
|
|
426
426
|
if (novelErrors.length > 0)
|
|
427
427
|
failedChecks.push(`${novelErrors.length} page error(s)`);
|
|
428
|
-
if (
|
|
429
|
-
|
|
428
|
+
if (matchedCount > 0)
|
|
429
|
+
failedChecks.push(`page-error check could not be verified (${matchedCount} row(s) matched the post-clear baseline; old residue and identical new errors are indistinguishable)`);
|
|
430
430
|
}
|
|
431
431
|
if (commandName === "console" && Array.isArray(result?.messages)) {
|
|
432
432
|
const errorCount = result.messages.filter((message) => isRecord(message) && /error/i.test(String(message.type ?? message.level ?? ""))).length;
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -45,7 +45,7 @@ The extension should:
|
|
|
45
45
|
- accept an optional top-level `script` string as a mutually exclusive one-shot orchestration mode for loops, conditional page branches, and multi-page aggregation. Source runs in a separate permissioned Node child with a constrained VM context; only null-prototype `browser({ args, stdin?, timeoutMs? })` and `emit(value)` task functions cross a bounded JSON-lines IPC bridge. The parent serializes at most one inner call at a time through the same full ordinary tool executor, clears ambient upstream launch/proxy controls across helpers and cleanup, caps calls/source/post-redaction output/time, injects one unique restore-disabled wrapper-owned session, writes a strict Pi custom-entry cleanup lease before first launch, closes in `finally`, aborts and awaits active-script cleanup on branch change/shutdown, and recovers exact non-closed active-branch leases afterward. Script requires Pi session persistence and exposes no profile/attachment/session-control, host API, named recipe, import, or persistent workflow-state surface (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#script)).
|
|
46
46
|
- accept an optional native `semanticAction` object as a mutually exclusive alternative to `args` on a single tool call (and to `script`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, and `electron` on the same call), compile locator actions into upstream `find` argv, direct selector/ref click/check/fill into upstream command argv, and native dropdown selection into upstream `select <selector> <value...>` argv (with optional `semanticAction.session` expanding to a leading `--session <name>` before the compiled command when targeting a named upstream browser instead of the managed default), and echo the compiled shape in `details.compiledSemanticAction` for observability (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#semanticaction))
|
|
47
47
|
- accept an optional native `job` object (mutually exclusive with `script`, `args`, `semanticAction`, `qa`, `sourceLookup`, `networkSourceLookup`, and `electron` on the same call) with a small fixed step vocabulary that compiles only to existing upstream `batch` argv rows, generates the JSON batch stdin string internally, defaults to fail-fast `batch --bail` unless `failFast:false` is explicit, supports semantic locator fields on constrained `click`/`fill` steps by reusing the top-level semantic-action compiler, supports human-paced `type` by expanding to a bounded set of existing focus/keyboard/wait/press rows while compacting model-visible batch text, and echoes `details.compiledJob` for observability (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#job))
|
|
48
|
-
- accept an optional native `qa` object (mutually exclusive with `script`, `args`, `semanticAction`, `job`, `sourceLookup`, `networkSourceLookup`, and `electron` on the same call) that compiles to the same fail-fast `batch --bail` path as `job`, runs a fixed diagnostic smoke sequence with bounded visible-text predicates for `expectedText`, preserves existing diagnostics for `qa.attached` while
|
|
48
|
+
- accept an optional native `qa` object (mutually exclusive with `script`, `args`, `semanticAction`, `job`, `sourceLookup`, `networkSourceLookup`, and `electron` on the same call) that compiles to the same fail-fast `batch --bail` path as `job`, runs a fixed diagnostic smoke sequence with bounded visible-text predicates for `expectedText`, preserves existing diagnostics for `qa.attached` while requesting clears only for URL-opening QA, and echoes `details.compiledQaPreset` plus structured `details.qaPreset` pass/fail evidence. Native clearing remains upstream-owned; the wrapper makes matched final rows from a nonempty post-clear error baseline non-pass through the existing `failedChecks` / `qa-failure` path because old residue and identical new errors are indistinguishable. Definitely new errors are counted separately, clean evidence can pass, and explicit `checkErrors: false` omits the check without changing browser lifecycle (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#qa))
|
|
49
49
|
- accept an optional native `sourceLookup` object (mutually exclusive with `script`, `args`, `semanticAction`, `job`, `qa`, `networkSourceLookup`, and `electron` on the same call) that compiles to the same `batch` path, gathers evidence-backed local source *candidates* for a selector/fiber/component name, and echoes `details.compiledSourceLookup` plus structured `details.sourceLookup` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sourcelookup)); unlike `qa`, it never applies a second pass/fail layer that marks the tool failed when upstream already reported batch success—failed upstream steps still fail the invocation normally, and `details.sourceLookup` may still be present for partial evidence
|
|
50
50
|
- accept an optional native `networkSourceLookup` object (mutually exclusive with `script`, `args`, `semanticAction`, `job`, `qa`, `sourceLookup`, and `electron` on the same call) that compiles to the same `batch` path, correlates failed network requests with initiator metadata and bounded workspace URL literals, and echoes `details.compiledNetworkSourceLookup` plus structured `details.networkSourceLookup` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#networksourcelookup)); like `sourceLookup`, it never flips a successful upstream batch to failed solely because no source candidates were found
|
|
51
51
|
- accept an optional native `electron` object (mutually exclusive with `script`, `args`, `semanticAction`, `job`, `qa`, `sourceLookup`, and `networkSourceLookup` on the same call) for bounded desktop Electron lifecycle: `list` scans the host for install candidates, `launch` creates a wrapper-owned isolated profile plus OS-chosen remote-debugging port, then attaches through upstream `connect` with `sessionMode: "fresh"`, cancellation prevents host spawn or interrupts readiness polling with process/profile cleanup, and `status` / `cleanup` / `probe` operate only on wrapper-tracked launches; host-side spawn and CDP discovery live in `extensions/agent-browser/lib/electron/discovery.ts`, `launch.ts`, and `cleanup.ts`, while compilation, transcript restore for `launchId` records, handoff probes, and merged `details.electron*` fields live under `extensions/agent-browser/lib/orchestration/electron-host/` and `extensions/agent-browser/lib/orchestration/browser-run/` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron))
|
|
@@ -439,9 +439,9 @@ On app pages that expose a native dropdown, add a `select` step such as `{ "acti
|
|
|
439
439
|
|
|
440
440
|
Use raw `args: ["batch"]` with `stdin` when you need arbitrary upstream commands, flags, or batch failure policies outside the constrained schema. Do not pass `stdin` with `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron`; those modes generate or manage their own input.
|
|
441
441
|
|
|
442
|
-
For quick smoke/QA checks, use top-level `qa`. It clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks expected text/selector, then inspects fresh network requests, console messages, and page errors only if preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped diagnostic output.
|
|
442
|
+
For quick smoke/QA checks, use top-level `qa`. It clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks expected text/selector, then inspects fresh network requests, console messages, and page errors only if preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped diagnostic output. If final page-error rows match a nonempty post-clear baseline, QA is non-pass with “page-error check could not be verified”: identical new errors can replace old rows in the native FIFO buffer without changing the result. Those matches are not proof of new application errors. Definitely new rows, including errors after an empty baseline, count separately as page errors; a clean final buffer can pass. Explicit `checkErrors: false` omits this check for a more limited smoke test. The preset compiles to `batch --bail` so a missing text/selector assertion fails crisply instead of letting slower diagnostics burn the wrapper watchdog. Expected text compiles to bounded visible-text `wait --fn … --timeout 5000` predicates after load so dense pages can pass on visible headings/copy without dumping `body` text; missing text reports a crisp QA failure. The readiness wait defaults to `loadState: "domcontentloaded"`; set `loadState` to `"load"` or `"networkidle"` only when that stricter state is useful and the site is not expected to keep background requests alive. QA network diagnostics classify failed requests by likely impact and list failed rows first in the network preview: actionable document/script/API-style failures fail the preset, while common low-impact browser icon misses such as `favicon.ico` are surfaced as warnings (`qaPreset.warnings`) so they do not fail an otherwise healthy page. Successful QA with no failed checks returns compact model-visible prose (page URL/title when known, checks run, optional screenshot verification) while keeping the full step matrix in `details.qaPreset` and `details.batchSteps`. Failed QA presets report `details.resultCategory: "failure"`, `failureCategory: "qa-failure"`, show compact failure reasons with the full matrix in `details.batchSteps`, and real Pi sessions treat the diagnostic as a failed tool result. Prose output also gets a model-visible result-category line including `Pi tool isError: true`; caller-requested `--json` output keeps the JSON string parseable and relies on the patched `isError` plus `details` fields.
|
|
443
443
|
|
|
444
|
-
The same classification drives plain `network requests` presentation: when any row counts as failed (HTTP status ≥ 400, `failed: true`, or a string `error`), model-facing text starts with a line like `Network failure summary: 0 actionable, 1 benign low-impact (1 total).`, and each preview line can end with an impact tag such as `[benign: low-impact browser icon asset]` or `[actionable: document, script, API, or non-benign request failure]`. When safe request IDs are present, `details.nextActions` adds bounded read-only follow-ups such as `network request <id>`, `networkSourceLookup` for actionable failed rows, `network requests --filter <path>`, `network requests --clear` before a repro, and `network har start`; prefer those payloads over rebuilding request-id commands from prose. For aggregate buffers, the wrapper accepts `network requests --current-page` / `--current-origin` to render only rows matching the active page origin, or `--current-url` for exact active document URL matching; it strips those wrapper-only flags before upstream spawn and reports counts in `details.networkRequestsPageFilter`. If the wrapper has seen a prior `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows add `details.networkRouteDiagnostics` plus executable route-mock follow-ups (`inspect-routed-network-request` and `start-network-har-capture-for-route-mock`) so agents do not mistake an unfulfilled mock for a fulfilled mock; same-origin/CORS fixture retry guidance stays in visible prose. `network requests` also hides `data:image` screenshot/artifact noise from the compact preview by default while preserving raw rows in `details.data.requests`. Rules live in `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `extensions/agent-browser/lib/results/network.ts`; QA aggregation is `analyzeQaPresetResults` in `extensions/agent-browser/
|
|
444
|
+
The same classification drives plain `network requests` presentation: when any row counts as failed (HTTP status ≥ 400, `failed: true`, or a string `error`), model-facing text starts with a line like `Network failure summary: 0 actionable, 1 benign low-impact (1 total).`, and each preview line can end with an impact tag such as `[benign: low-impact browser icon asset]` or `[actionable: document, script, API, or non-benign request failure]`. When safe request IDs are present, `details.nextActions` adds bounded read-only follow-ups such as `network request <id>`, `networkSourceLookup` for actionable failed rows, `network requests --filter <path>`, `network requests --clear` before a repro, and `network har start`; prefer those payloads over rebuilding request-id commands from prose. For aggregate buffers, the wrapper accepts `network requests --current-page` / `--current-origin` to render only rows matching the active page origin, or `--current-url` for exact active document URL matching; it strips those wrapper-only flags before upstream spawn and reports counts in `details.networkRequestsPageFilter`. If the wrapper has seen a prior `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows add `details.networkRouteDiagnostics` plus executable route-mock follow-ups (`inspect-routed-network-request` and `start-network-har-capture-for-route-mock`) so agents do not mistake an unfulfilled mock for a fulfilled mock; same-origin/CORS fixture retry guidance stays in visible prose. `network requests` also hides `data:image` screenshot/artifact noise from the compact preview by default while preserving raw rows in `details.data.requests`. Rules live in `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `extensions/agent-browser/lib/results/network.ts`; QA aggregation is `analyzeQaPresetResults` in `extensions/agent-browser/lib/input-modes/job.ts`.
|
|
445
445
|
|
|
446
446
|
```json
|
|
447
447
|
{ "qa": { "url": "https://example.com", "expectedText": "Example Domain", "screenshotPath": ".dogfood/qa-example.png" } }
|
package/docs/RELEASE.md
CHANGED
|
@@ -47,6 +47,8 @@ npm run verify -- pre-pr
|
|
|
47
47
|
3. `npm run verify -- package-pi`, which first validates package contents via `npm pack --json --dry-run` and then smoke-loads the packed package in Pi isolation
|
|
48
48
|
4. `npm run smoke:platform:doctor` and the full Crabbox matrix from [`platform-smoke.md`](platform-smoke.md): macOS SSH, Ubuntu local-container, and native Windows Parallels targets running fast target-local `platform-build` plus `browser-dogfood-smoke`
|
|
49
49
|
|
|
50
|
+
The 0.6.12 release uses the documented [local macOS and hosted native-Windows alternatives](platform-smoke.md#alternate-native-transports) instead of SSH and Parallels. Run and inspect those target-local suites separately alongside the remaining release checks. The default composed gate is unchanged; alternate evidence must not be labeled a passing Crabbox matrix.
|
|
51
|
+
|
|
50
52
|
`npm publish` runs npm’s `prepublishOnly` script from `package.json`, which executes the same `npm run verify -- release` gate and then `npm pack --dry-run`. That concatenated gate is everything in the default `npm run verify` step (generated playbook drift, clean `dist/` build, TypeScript, the unit/fake suite, generated command-reference blocks, and live upstream command-reference sampling against the targeted `agent-browser` on `PATH`), the configured-source lifecycle harness, the packaged Pi smoke in `package-pi`, and the release-blocking Crabbox platform matrix. Using `npm publish --ignore-scripts` skips that contract intentionally.
|
|
51
53
|
|
|
52
54
|
`prepublishOnly` intentionally does **not** run the standalone host-only `npm run verify -- startup-profile`, `npm run verify -- real-upstream`, `npm run verify -- dogfood` modes; those remain separate `npm run verify` modes in [`scripts/project.mjs`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/scripts/project.mjs). The platform matrix includes its own fast target-local build/package gate and browser dogfood suite, and is automated through the `release` slice.
|
|
@@ -289,7 +291,7 @@ For a root-check change, run the identical final test against both the pre-fix a
|
|
|
289
291
|
|
|
290
292
|
- **Focused and broad cases:** `test/agent-browser.real-upstream-contract.test.ts` keeps the broad command matrix in one 180-second case and separate 60-second cold-reopen, duplicate-name click-mutation, navigation-availability/tab-setup, and sessionless-plugin cases. The `real-upstream` facade's `contract suite matches` selection includes the broad matrix, cold reopen, click mutation, and navigation setup. The click case uses two native DOM buttons to prove a stale duplicate ordinal cannot contradict trusted target events, while exact XPath probes still detect missing events and native clicks remain dispatch-only evidence.
|
|
291
293
|
- **Output-shape locking:** Expected `details` / `data` keys per step live in `test/fixtures/agent-browser-real-output-shapes.json`, keyed by logical groups (`version`, `rootHelp`, `commandHelp`, `skillsList`, `skillsGetFull`, `skillsPath`, `open`, `eval`, `snapshot`, `coreCommand`, `coreSubcommand`, `coreFileArtifact`, `batch`, `pushstate`, `vitals`, `networkRoute`, `nonCoreStatus`, `nonCoreArtifact`, `diffScreenshotArtifact`, `streamControl`, `streamStatus`, `cookiesCurl`, `reactMissingRenderer`, `waitDownload`). Keep `targetVersion` in that file aligned with `scripts/agent-browser-capability-baseline.mjs`, and extend entries whenever the suite starts asserting on new presentation fields.
|
|
292
|
-
- **Isolation:** The harness
|
|
294
|
+
- **Isolation:** The harness uses a throwaway HOME and screenshot directory, and pairs `PI_AGENT_BROWSER_SOCKET_DIR` with `AGENT_BROWSER_SOCKET_DIR` in a short private socket directory. This keeps owned commands and native policy probes on the same daemon without exceeding Unix socket path limits. Raw click/FPS helpers retain owned-session context; cold-resume observers use the registered tool so they do not restart the browser they are measuring. The harness serves loopback fixtures, closes its sessions and removes only its own directories; it does not reuse your normal profile or socket locations.
|
|
293
295
|
- **React DevTools branch:** After the core matrix, the suite performs another `open` with `--enable react-devtools` and `sessionMode: "fresh"`, then expects `react tree` to fail with a missing-renderer style error on the same non-React contract page. The following download fixture + `wait --download` assertions run against whichever managed session is current after that fresh `open` (typically the React DevTools session), not the original pre-matrix session name.
|
|
294
296
|
|
|
295
297
|
**Troubleshooting**
|
package/docs/SUPPORT_MATRIX.md
CHANGED
|
@@ -60,7 +60,7 @@ Current summary:
|
|
|
60
60
|
| RQ-0142 | An optional host [`beforeExecute` callback](TOOL_CONTRACT.md#host-execution-hook) uses Pi sequential scheduling and the existing inner-call queue to save state before dispatch, retaining the outer tool-call ID and per-dispatch cancellation (`test/agent-browser.pi-pipeline.test.ts`). Top-level `script` provides bounded one-shot JavaScript loops, branches, and multi-page aggregation through the complete ordinary browser executor. A permissioned child exposes only null-prototype `browser` / `emit` task functions, receives no user-visible host capabilities, serializes at most 25 calls, and uses a unique restore-disabled wrapper session with a durable pre-spawn Pi lease, finally-close, active branch/shutdown abort-and-wait cleanup (including upstream-version preflight), and exact lease recovery. Script-owned helpers/cleanup case-insensitively clear ambient upstream launch/profile/restore/attachment and proxy controls and bypass user/project config; inner `--config` is rejected before dispatch while ordinary `args` retains it; final data is compact-serialized with a second post-redaction byte/depth guard; rehydrated spills reserve IPC-envelope headroom, inner summary/text are bounded, complete response envelopes are size-checked, and script-visible next actions are policy-compatible after isolated identity removal; malformed bridge calls and missing workers fail structurally; policy-rejection and dispatched-failure counters are disjoint. The collapsed Pi call row shows a bounded terminal-safe source preview with visible line-break markers and its expanded state shows the full terminal-safe source with JavaScript line terminators preserved and removed controls marked. It requires persisted Pi sessions, classifies uncaught source exceptions as `script-error`, confirms successful cleanup in compact prose, and deliberately has no named recipe/import/shared-state surface. The same dogfood pass compacted common/QA output, made failure `nextActions` model-visible and exact-namespace/session-stable (including explicit empty namespace overrides), limited close cleanup guidance to existing explicit artifacts, clarified raw batch stdin, normalized empty semantic names, and added bounded timeout/navigation recovery actions. | [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#script), [`docs/ARCHITECTURE.md`](ARCHITECTURE.md#one-shot-script-isolation), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#one-shot-code-mode), `test/agent-browser.script.test.ts` |
|
|
61
61
|
| RQ-0143 | Browser-backed calls require a stable upstream version at or above the configured 0.35.0 floor, report minimum/observed version mismatch before launch, and keep stateless/local diagnostics available; the recommended target and minimum are shared by runtime, docs verification, and package contents. | [`scripts/agent-browser-target.mjs`](../scripts/agent-browser-target.mjs), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0360-rebaseline), `test/agent-browser.upstream-version.test.ts` |
|
|
62
62
|
| RQ-0144 | Managed restore keys are scoped to both checkout generation and Pi transcript so upstream 0.33.2's newest-file-wins loading cannot let concurrent chats clobber or inherit cookies/storage. Fresh rotations, reload, restart, and `/resume` of one transcript retain continuity; after a confirmed inactive daemon, a branch-restored compatible owned session retains its pending reopen through daemon-starting non-page calls and transcript replay, then reopens the complete recorded URL (including its fragment) before current-page operations such as `get url` or `reload`, verifies the actual target and invalidates old refs. This reloads a URL, not forms, JavaScript memory, or history; live wrong-tab recovery never auto-navigates. Passive and explicit upstream configs pass through, while explicit config selection suppresses the wrapper's automatic restore injection. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.managed-session-restore.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
|
|
63
|
-
| RQ-0145 | Mega-dogfood correctness closes false-positive success paths: URL QA
|
|
63
|
+
| RQ-0145 | Mega-dogfood correctness closes false-positive success paths: URL QA makes matched post-clear page-error evidence non-pass as unverified, counts definitely new errors separately, and waits 150 ms for immediate post-load diagnostics, stale pre-existing artifacts and missing/stale recordings finalized by restart fail verification, aliased destinations (including dangling symlinks, full macOS/Windows Unicode folds, final short/long diff output flags, and same-call `outputPath` aliases) and active recording-path reuse through artifacts, path-bearing HAR stops, long/short or reordered explicit download waits, interspersed-global command forms, Electron cleanup output, or normalized output writes is rejected through an atomic process-wide, transcript-backed namespace/session reservation index independent of bounded manifest eviction; cross-branch close tombstones survive reload and each direct or ordered nested close retires exact identity at that lifecycle point while terminal closes clear attached/ref/page/route state, later lifecycle-proven browser launches—including failed post-close rows and `record stop`—can reactivate/rebuild it without stale pre-close `about:blank` recovery while bounded launch evidence survives failed-step presentation and transcript replay, explicitly non-launching diagnostics preserve the close, missing evidence stays conservatively active even on the first managed call, every successful direct/nested close clears wrapper trace/profiler ownership before later successful rows can rebuild it, and namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, only the newest pending recording path remains authoritative even across same-timestamp restart rows, current recording transitions replay directly without artifact-manifest compatibility inference, definitive direct/batch no-recording stop failures retire stale state at their ordered step, recording starts after nested close are rejected, pending rows coalesce into terminal batch outcomes without retaining an intermediate close-abandoned duplicate after a later saved stop, and any later same-session failure retains exact cleanup actions, same-page getter refs are refreshed, locator select resolves exact current native controls, no-movement scroll fails honestly, and script cleanup always attempts fail-closed close. | [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md), `test/agent-browser.extension-input-modes.test.ts`, `test/agent-browser.extension-ref-guards.test.ts`, `test/agent-browser.presentation.test.ts`, `test/agent-browser.presentation-artifacts-batch.test.ts`, `test/agent-browser.recording-reservations.test.ts`, `test/agent-browser.extension-validation.test.ts`, `test/agent-browser.script.test.ts` |
|
|
64
64
|
| RQ-0146 | Upstream `agent-browser 0.34.0` rebaseline adds sticky `--pin-tab` / `--no-pin-tab` (`AGENT_BROWSER_PIN_TAB`) for shared CDP/auto-connect sessions, CDP target ids as tab refs, and `tab_gone` recovery data (`data.targetId`, optional `data.lastUrl`). The wrapper parses those flags as optional global booleans rather than launch-scoped, classifies `tab_gone` as `failureCategory: "tab-gone"` with `list-tabs-after-tab-gone` / `open-tab-after-tab-gone`, and shows `targetId` in `tab list` presentation. | [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0340-rebaseline), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), `test/agent-browser.results.test.ts`, `test/agent-browser.runtime.test.ts` |
|
|
65
65
|
| RQ-0147 | Native follow-ups preserve profiled and other launch-configured browser pages by omitting wrapper compatibility launch arguments while the daemon is active; daemon inspection reapplies the retained compatibility user agent only when relaunch is required. Caller `--args`, config, file access, and local navigation pass through unchanged, and cancellation wins over policy validation before spawn. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.runtime.test.ts`, `test/agent-browser.process.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
|
|
66
66
|
| RQ-0148 | Upstream 0.35.0 adds Linux Chromium private CA trust and the `protected-vercel-deployments` skill. The wrapper parses CA set/clear globals, requires fresh managed launches, suppresses automatic managed restore for CA-enabled sessions, passes caller certificate paths upstream unchanged, and uses 0.35.0 as its runtime floor. | [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0350-rebaseline), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.runtime.test.ts`, `test/agent-browser.managed-session-restore.test.ts`, `test/agent-browser.page-target-validation.test.ts` |
|
|
@@ -69,6 +69,8 @@ Current summary:
|
|
|
69
69
|
| RQ-0151 | Upstream 0.36.0 adds experimental WebMCP page tools, default local-Chrome enablement with launch-scoped `--no-webmcp`, an opt-in MCP profile, and the `webmcp-gen` skill. The wrapper passes list/invoke/result/cancel through, preserves params/frame/detach/timeout arguments, derives watchdogs from effective direct/raw/stdin commands, keeps pending detached targets unverified, and restores batch target/ref state only after URL verification and a fresh snapshot, while keeping 0.35.0 as the stable floor. | [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0360-rebaseline), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#experimental-webmcp), `test/agent-browser.runtime.test.ts`, `test/agent-browser.wait-timeouts.test.ts`, `test/agent-browser.process.test.ts`, `test/agent-browser.page-target-validation.test.ts`, `test/agent-browser.extension-ref-guards.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
|
|
70
70
|
| RQ-0152 | Agent feedback closes recovery and evidence gaps without a new runtime layer: unknown-target timeouts return a visibly executable `get url` → `snapshot -i` fail-fast action while dialog recovery remains allowed; successful tab selection/close live-verifies URL/fresh titles and retains deliberate and post-close blank tabs; compacted direct or batched `outputPath` writes use only full command-redacted live wrapper spills and fail rather than save metadata; explicit URL reads/all-read batches avoid browser helpers and managed replacement, while proven native read confirmations preserve exact session routing and capability-gated browser independence; single session-info preflights distinguish daemon/browser/native/Pi ownership; native recording receipts expose capture versus encoded measurements and survive failure/timeout exports, with bounded identity-matched recovery and no file-only success; direct or batched first/fresh local headed launches expose an unverified `browserWindow` handoff while attachments do not; snapshot spills and SAML/OAuth URL values are redacted at model/persistence boundaries; unsupported direct/batch `scrollintoview text=...` fails before upstream false success with visible native recovery payloads while help passes through; and web search sends Exa primary/version guidance, removes exact normalized-URL duplicates without overfetch, and labels provider page dates without claiming crawl/version proof. | [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md), [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), `test/agent-browser.extension-errors-artifacts.test.ts`, `test/agent-browser.extension-tabs.test.ts`, `test/agent-browser.extension-validation.test.ts`, `test/agent-browser.page-target-validation.test.ts`, `test/agent-browser.runtime.test.ts`, `test/agent-browser.snapshot-presentation.test.ts`, `test/agent-browser.web-search.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
|
|
71
71
|
|
|
72
|
+
QA truthfulness (#121, RQ-0145) is covered by the registered-tool controls in `test/agent-browser.extension-input-modes.test.ts` and the strict `contract suite matches QA non-pass after same-URL error-buffer rollover` case in `test/agent-browser.real-upstream-contract.test.ts`. The real fixture fills the native error FIFO with identical rows, then verifies that a new document actually throws again at the same URL and that QA is always non-pass. Matched rows fail as an unverified check, not an application-error assertion. Empty/cleared and explicit `checkErrors: false` controls preserve valid passes; the wrapper does not fix upstream `errors --clear` or accept a warning-only pass as resolution.
|
|
73
|
+
|
|
72
74
|
Browser evidence (RQ-0152) is covered by `test/agent-browser.browser-evidence.test.ts`, `test/agent-browser.read-confirmation.test.ts`, `test/agent-browser.recording-receipts.test.ts` and `test/agent-browser.recording-recovery.test.ts`, alongside existing presentation, reservation/replay and lifecycle checks. They cover cold/shared/unknown/invalid/timeout reads, native-default and shared confirmation routing, capability absence and script identity boundaries, separate daemon/browser/Pi identity, sparse/legacy capture metrics, native failure retention, stale/same-path mismatches, one-query stop recovery, error exports and artifact aliases. Full live identity, receipt lookup and browser-independent native read/confirm handling require companion upstream fixes; missing fields remain unknown, and native receipts do not survive daemon exit. These source checks do not replace paired native/Pi runtime verification.
|
|
73
75
|
|
|
74
76
|
Contributor fixes #133/#152 remove unused prompt suffix entries without changing runtime guidance and diagnose bare `--no-sandbox` only in the command slot or navigation option positions. Native `--args` values and literal operands remain intact; batch checks use raw effective rows without treating row-local `--args` as a launch setting. `test/agent-browser.chromium-args.test.ts` covers pre-dispatch rejection, literal/flag-value controls, inspection, raw/stdin batch precedence and dispatcher outcomes. Existing prompt/grammar checks remain; source checks do not replace native-product gates.
|
package/docs/TOOL_CONTRACT.md
CHANGED
|
@@ -433,15 +433,15 @@ Because `job` still executes as upstream `batch` with generated stdin, the same
|
|
|
433
433
|
- type: object with either required `url` (normal URL-opening QA) or `attached: true` (current attached-session QA)
|
|
434
434
|
- optional; mutually exclusive with `script`, `args`, `semanticAction`, `job`, `sourceLookup`, `networkSourceLookup`, and `electron`
|
|
435
435
|
- lightweight preset built on the same batch compiler path as `job`, using `batch --bail` so missing readiness/text/selector assertions stop before slower diagnostics can burn the wrapper watchdog
|
|
436
|
-
- URL form: clears enabled network/console diagnostic buffers first and snapshots any page-error residue after `errors --clear` (
|
|
436
|
+
- URL form: clears enabled network/console diagnostic buffers first and snapshots any page-error residue after `errors --clear` (native clear can leave residue), then opens `url`, waits with `wait --load <state>` using the resolved `loadState`, adds a bounded 150 ms diagnostic settle when console or page-error checks are enabled so immediate post-load callbacks can report, optionally asserts `expectedText` (string or string array, compiled to bounded visible-text `wait --fn … --timeout 5000` predicates after load) and/or `expectedSelector` (each may be omitted for a load-plus-diagnostics-only smoke), then runs enabled diagnostics: `network requests`, `console`, and `errors` only if preceding batch steps pass. Successful reset-step rows are labeled as reset output. Final page-error rows that match a nonempty post-clear baseline make the check unverified and fail QA; identical rows do not prove that no new error occurred. Multiset subtraction still identifies definitely new rows, which count as page errors. Failed reset commands still fail the batch.
|
|
437
437
|
- URL QA uses its explicitly requested navigation without requiring the previous tab; diagnostic clears remain before `open`. This does not reopen remembered pages for ordinary reads or `qa.attached`.
|
|
438
438
|
- attached form: `qa: { attached: true, expectedText?, expectedSelector?, screenshotPath?, checkNetwork?, checkConsole?, checkErrors?, loadState? }` runs the same waits (including the bounded diagnostic settle when needed), optional assertions, diagnostics, and screenshot against the current attached managed session without opening a URL. It rejects `url` and cannot be used with `sessionMode: "fresh"`; attach first with `electron.launch` or raw `args: ["connect", "<port-or-url>"]`, then run `qa.attached`. Before spawning the diagnostic batch, the wrapper preflights the attached session: `get url` must succeed and return a non-empty page URL. Missing URLs and read failures fail fast with `failureCategory: "validation-error"`, `details.validationError`, and recovery `nextActions` such as `list-tabs-before-qa-attached` and `snapshot-before-qa-attached` instead of running the full QA batch. `file:`, custom-scheme, and other attached targets are accepted. Attached QA does **not** run `network requests --clear`, `console --clear`, or `errors --clear`; `details.compiledQaPreset.checks.diagnosticsResetAtStart` is `false`. Visible text warns that existing diagnostic buffers were preserved only when `checkNetwork`, `checkConsole`, or `checkErrors` is enabled, and those diagnostics may include events from before the QA check.
|
|
439
439
|
- `loadState` is optional and must be `domcontentloaded`, `load`, or `networkidle`; it defaults to `domcontentloaded` so analytics-heavy or long-polling pages do not hang routine QA. Use `networkidle` only when the site is expected to go fully quiet.
|
|
440
|
-
- `checkNetwork`, `checkConsole`, and `checkErrors` default to `true` for URL-opening QA; for `qa.attached` they default to `false` because preserved upstream buffers may predate the current check. Set a field to `true` on `qa.attached` to opt into preserved-buffer diagnostics.
|
|
440
|
+
- `checkNetwork`, `checkConsole`, and `checkErrors` default to `true` for URL-opening QA; for `qa.attached` they default to `false` because preserved upstream buffers may predate the current check. Set a field to `true` on `qa.attached` to opt into preserved-buffer diagnostics. Explicit `checkErrors: false` skips error clearing, baseline collection, and the final error read; an otherwise healthy limited check can pass.
|
|
441
441
|
- optional `screenshotPath` adds an evidence screenshot step
|
|
442
442
|
- reports `details.compiledQaPreset` with the compiled batch plan and resolved `loadState`, plus `details.qaPreset` with `{ passed, failedChecks, warnings, summary }`
|
|
443
|
-
- on success with no failed checks, model-visible prose collapses to a compact pass summary (current page URL/title when known, checks run, optional screenshot path plus artifact verification, pointer to `details.qaPreset` and `details.batchSteps` for the full matrix). Failed QA
|
|
444
|
-
- fails the native tool result with `failureCategory: "qa-failure"` when diagnostics report page errors, console error messages, actionable failed network requests, any batch step failure, or missing expected text after the requested load state. Expected-text QA checks use bounded visible-element predicates plus fail-fast batch behavior instead of reading all body text, so dense pages can pass on visible headings/copy and missing text reports `expected text not found: …` rather than burning the full wrapper watchdog. Benign classification (implementation: `classifyNetworkRequestFailure` → `isBenignAssetFailure` in `extensions/agent-browser/lib/results/network.ts`) applies only when the row is already treated as failed (`status >= 400`, `failed: true`, or a string `error`—see `isFailedNetworkRequest`), the URL path’s last segment matches the icon basename heuristic (`favicon` plus `.ico`/`.png`/`.svg`, or `apple-touch-icon` plus `.png`, each allowing an optional `[-.\w]*` stem suffix before the extension), **and** at least one of `status === 404`, `failed === true`, or `typeof request.error === "string"` holds (so a **status-only** failure such as `500` on that path with neither `failed` nor a string `error` stays actionable). It also requires the upstream `resourceType` / `mimeType` (whichever is present) to be absent or look image-like: `image`, `img`, `other`, or a value starting with `image/`. Those rows are counted in `qaPreset.warnings` (for example `N benign network request failure(s) ignored`) and omitted from the actionable failed-network tally; every other failed request stays actionable.
|
|
443
|
+
- on success with no failed checks, model-visible prose collapses to a compact pass summary (current page URL/title when known, checks run, optional screenshot path plus artifact verification, pointer to `details.qaPreset` and `details.batchSteps` for the full matrix). Failed QA uses a compact failure summary with reasons in `failedChecks`; the full diagnostic matrix remains in `details.batchSteps`.
|
|
444
|
+
- fails the native tool result with `failureCategory: "qa-failure"` when diagnostics report definitely new page errors, an unverified page-error check, console error messages, actionable failed network requests, any batch step failure, or missing expected text after the requested load state. Expected-text QA checks use bounded visible-element predicates plus fail-fast batch behavior instead of reading all body text, so dense pages can pass on visible headings/copy and missing text reports `expected text not found: …` rather than burning the full wrapper watchdog. Benign classification (implementation: `classifyNetworkRequestFailure` → `isBenignAssetFailure` in `extensions/agent-browser/lib/results/network.ts`) applies only when the row is already treated as failed (`status >= 400`, `failed: true`, or a string `error`—see `isFailedNetworkRequest`), the URL path’s last segment matches the icon basename heuristic (`favicon` plus `.ico`/`.png`/`.svg`, or `apple-touch-icon` plus `.png`, each allowing an optional `[-.\w]*` stem suffix before the extension), **and** at least one of `status === 404`, `failed === true`, or `typeof request.error === "string"` holds (so a **status-only** failure such as `500` on that path with neither `failed` nor a string `error` stays actionable). It also requires the upstream `resourceType` / `mimeType` (whichever is present) to be absent or look image-like: `image`, `img`, `other`, or a value starting with `image/`. Those rows are counted in `qaPreset.warnings` (for example `N benign network request failure(s) ignored`) and omitted from the actionable failed-network tally; every other failed request stays actionable.
|
|
445
445
|
|
|
446
446
|
Example:
|
|
447
447
|
|
|
@@ -449,6 +449,16 @@ Example:
|
|
|
449
449
|
{ "qa": { "url": "https://example.com", "expectedText": "Example Domain", "screenshotPath": ".dogfood/qa-example.png" } }
|
|
450
450
|
```
|
|
451
451
|
|
|
452
|
+
A nonempty post-clear baseline is not reliable event identity. A full native FIFO buffer can replace old errors with identical new errors while both reads remain identical. Matched final rows therefore add “page-error check could not be verified” to `failedChecks`, set `qaPreset.passed: false`, and take the existing `qa-failure` / Pi `isError: true` path. This is an evidence failure, not proof that the application emitted those matched rows during QA; even a genuinely clean page can be non-pass when residue cannot be cleared. The wrapper does not reset the browser, add event-correlation heuristics, or repair native clearing.
|
|
453
|
+
|
|
454
|
+
| Error evidence with `checkErrors` enabled | QA outcome when other checks pass |
|
|
455
|
+
| --- | --- |
|
|
456
|
+
| Empty post-clear baseline and no final errors | Pass |
|
|
457
|
+
| No final errors, even if the baseline was nonempty | Pass |
|
|
458
|
+
| Final rows not accounted for by the baseline | Non-pass: page errors |
|
|
459
|
+
| Final rows match the nonempty baseline | Non-pass: page-error check could not be verified |
|
|
460
|
+
| Both novel and matched final rows | Non-pass: page errors and an unverified check |
|
|
461
|
+
|
|
452
462
|
Use custom `job` or raw `batch` for QA flows that need custom commands, flags, auth setup, HAR capture, or project-specific assertions.
|
|
453
463
|
|
|
454
464
|
### `electron`
|
|
@@ -896,7 +906,7 @@ Additional structured fields can appear when relevant:
|
|
|
896
906
|
- `compiledSourceLookup` when the call used `sourceLookup`: `{ args: ["batch"], stdin, steps, query }` with the generated local-evidence plan and original query fields (`selector?`, `reactFiberId?`, `componentName?`, `includeDomHints?`, `maxWorkspaceFiles?`).
|
|
897
907
|
- `sourceLookup` when the call used `sourceLookup`: `{ status, candidates, limitations, summary, workspaceRoot?, electronContext? }`; wrapper-tracked packaged Electron no-candidate diagnostics may carry `workspaceRoot` plus `electronContext` and live Electron nextActions without marking the successful batch as a tool failure.
|
|
898
908
|
- `compiledNetworkSourceLookup` / `networkSourceLookup` when the call used `networkSourceLookup`: the generated batch plan plus bounded failed-request/candidate evidence as described above.
|
|
899
|
-
- `qaPreset` when the call used `qa`: `{ passed, failedChecks, warnings, summary }`. Network rows inside the `network requests` batch step use `summarizeNetworkFailures` / `classifyNetworkRequestFailure` in `network.ts`: actionable failures appear in `failedChecks` (and fail the tool when the upstream batch still succeeded); benign icon-classified failures appear only in `warnings` and in `summary` as `QA preset passed with warnings: …` when nothing else failed.
|
|
909
|
+
- `qaPreset` when the call used `qa`: `{ passed, failedChecks, warnings, summary }`. `failedChecks` includes “page-error check could not be verified” when final errors match a nonempty post-clear baseline; this sets `passed: false` without claiming those matched rows are new application errors. Definitely new rows are counted separately as page errors. Network rows inside the `network requests` batch step use `summarizeNetworkFailures` / `classifyNetworkRequestFailure` in `network.ts`: actionable failures appear in `failedChecks` (and fail the tool when the upstream batch still succeeded); benign icon-classified failures appear only in `warnings` and in `summary` as `QA preset passed with warnings: …` when nothing else failed.
|
|
900
910
|
- `networkRouteDiagnostics` after successful `network requests` when the wrapper has observed active `network route` patterns for that session and a matching request row is pending/no-status or carries CORS/preflight-looking error text. Each row includes `reason` (`"pending-routed-request"` or `"cors-likely-routed-request"`), `routePattern`, `mode`, optional `requestId`, optional `requestUrl`, and `summary`; visible text starts with `Network route diagnostics`, and `details.nextActions` prepends executable route-mock inspection/HAR follow-ups before generic request follow-ups.
|
|
901
911
|
- `compiledElectron` when the call used `electron`: redacted action plan for `list`, `launch`, `status`, `cleanup`, or `probe`.
|
|
902
912
|
- `electron` when the call used `electron`: action-specific lifecycle, discovery, probe, and cleanup data; see the `electron` section below.
|
package/docs/platform-smoke.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
`pi-agent-browser-native` uses a Crabbox-backed local platform smoke gate to prove the package on macOS, Ubuntu Linux, and native Windows before release.
|
|
4
4
|
|
|
5
|
-
This is a release-blocking gate. Missing
|
|
5
|
+
This is a release-blocking gate. Missing setup is not a skipped pass. When a maintainer approves an [alternate native transport](#alternate-native-transports), it must prove the same target-local suites; missing upstream `agent-browser` or browser dependencies still blocks that target.
|
|
6
6
|
|
|
7
7
|
## Required release gate
|
|
8
8
|
|
|
@@ -36,6 +36,14 @@ npm run verify -- platform-smoke run --target ubuntu --suite platform-build
|
|
|
36
36
|
| `ubuntu` | `local-container` | POSIX shell in a Docker-compatible local container | Required |
|
|
37
37
|
| `windows-native` | `parallels` | native Windows PowerShell over OpenSSH | Required |
|
|
38
38
|
|
|
39
|
+
## Alternate native transports
|
|
40
|
+
|
|
41
|
+
For the 0.6.12 release, local macOS execution replaces localhost SSH, and the [Native Windows workflow](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/windows-native.yml) runs the existing Windows suites on a GitHub-hosted Windows runner. These are native platform checks, not Crabbox SSH or Parallels passes. The default `release` and `prepublishOnly` commands still select the Crabbox matrix; record alternate suite evidence separately rather than reporting those commands as passed.
|
|
42
|
+
|
|
43
|
+
On macOS, run the unchanged commands returned by `buildPlatformBuildCommand('macos', 'pi-agent-browser-native', 24)` and `buildBrowserDogfoodCommand('macos', '0.37.0', true)` in [`scripts/platform-smoke/targets.mjs`](../scripts/platform-smoke/targets.mjs), serially in a clean private source copy. Use private HOME, npm and Pi settings, and headless browser profiles. No Remote Login or host security change is required.
|
|
44
|
+
|
|
45
|
+
Retain the exact source head/tree, package and tool versions, native OS/architecture, generated commands, stdout/stderr, every `PLATFORM_*` exit marker, actual Pi registration output, dogfood JSON and verified screenshot. Check the suite assertions below and clean up only the run's processes and temporary files. Hosted Windows uploads this evidence from one bounded job; it does not upload installed projects, dependencies or browser profiles. These alternatives do not waive Ubuntu or any non-platform release check.
|
|
46
|
+
|
|
39
47
|
## Required environment
|
|
40
48
|
|
|
41
49
|
Install Crabbox on the macOS maintainer host and keep it on `PATH`:
|
package/package.json
CHANGED