pi-agent-browser-native 0.6.5 → 0.6.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +30 -1
- package/README.md +20 -5
- package/dist/extensions/agent-browser/index.js +138 -75
- package/dist/extensions/agent-browser/lib/command-taxonomy.js +6 -5
- package/dist/extensions/agent-browser/lib/electron/cleanup.js +10 -1
- package/dist/extensions/agent-browser/lib/input-modes/params.js +20 -7
- package/dist/extensions/agent-browser/lib/launch-scoped-flags.js +0 -1
- package/dist/extensions/agent-browser/lib/managed-session-restore.js +13 -12
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.js +6 -25
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result.js +2 -3
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +10 -3
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-policy.js +4 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/wait-timeouts.js +1 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +104 -117
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +61 -35
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state.js +100 -135
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +3 -1
- package/dist/extensions/agent-browser/lib/orchestration/input-plan.js +3 -1
- package/dist/extensions/agent-browser/lib/page-target-validation.js +10 -10
- package/dist/extensions/agent-browser/lib/parsing.js +7 -0
- package/dist/extensions/agent-browser/lib/playbook.js +5 -8
- package/dist/extensions/agent-browser/lib/process-identity.js +10 -2
- package/dist/extensions/agent-browser/lib/process.js +23 -7
- package/dist/extensions/agent-browser/lib/recording-reservations.js +3 -1
- package/dist/extensions/agent-browser/lib/results/envelope.js +4 -1
- package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +45 -43
- package/dist/extensions/agent-browser/lib/results/recovery-actions.js +4 -4
- package/dist/extensions/agent-browser/lib/runtime.js +18 -2
- package/dist/extensions/agent-browser/lib/session-page-state.js +29 -10
- package/docs/ARCHITECTURE.md +7 -4
- package/docs/COMMAND_REFERENCE.md +22 -15
- package/docs/ELECTRON.md +6 -6
- package/docs/RELEASE.md +14 -5
- package/docs/REQUIREMENTS.md +1 -1
- package/docs/SUPPORT_MATRIX.md +13 -3
- package/docs/TOOL_CONTRACT.md +33 -22
- package/package.json +1 -1
|
@@ -333,9 +333,9 @@ For desktop, contenteditable, or host-controlled rich inputs, treat a semantic `
|
|
|
333
333
|
|
|
334
334
|
Do not assume Playwright selector dialects such as `text=Close` or `button:has-text('Close')` are supported wrapper syntax. In particular, current upstream can report successful `scrollintoview text=...` without moving the page, so the wrapper rejects that form before dispatch—directly or in an effective raw/stdin batch row—and shows executable `find text <label> hover` plus snapshot/ref recovery payloads in visible failure text and `details.nextActions`. `scrollintoview ... --help` and `-h` remain native help calls. Use `scrollintoview` with CSS, `xpath=...`, or a current `@e…` ref; use `find` for semantic text targets.
|
|
335
335
|
|
|
336
|
-
Treat `@
|
|
336
|
+
Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside `batch` stdin JSON, the wrapper also walks steps in order before spawn: steps whose first token can navigate or mutate set a latch; a later step whose first token is `snapshot` clears that latch for following rows; guarded steps that still mention `@e…` after an uncleared latch fail with the same `stale-ref` bucket without launching upstream. Same-snapshot form fills and native form-control steps are allowed before a click or submit step, so `fill`, `check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, `select` combobox refs, then a final submit `click` can run from one snapshot. Split dynamic or autosubmit forms with a fresh snapshot if a control interaction rerenders the targets. Follow the `refresh-interactive-refs` next action (it includes `--session <name>` when needed) and prefer stable `find` or `semanticAction` locators when navigation or rerendering is likely. Contract detail: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) (`refSnapshot`, `refSnapshotInvalidation`).
|
|
337
337
|
|
|
338
|
-
A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab
|
|
338
|
+
A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
|
|
339
339
|
|
|
340
340
|
Successful `snapshot -i` results can also surface `Possible overlay blockers` when their own refs already show dialog/alertdialog context plus close/dismiss controls, so agents can detect likely obstruction before clicking. When a **top-level** `@e…`/`ref=` click succeeds (not a `click` hidden inside a `batch`/`job` tool call—the unified command must be `click`), the upstream payload includes `data.clicked`, no `details.clickDispatch` diagnostic fired for the same result, and the wrapper sees `details.navigationSummary.url` unchanged after the same normalization it uses for ref guards (**`#fragment` ignored**), it may run one extra `snapshot -i` and surface `Possible overlay blockers` plus `details.overlayBlockers` (`candidates`, `summary`, and a `snapshot` map that can refresh `refSnapshot`) when that snapshot shows strong modal context (`dialog` / `alertdialog`) **and** up to three close/dismiss-like controls; page-wide words such as privacy, sign in, or banner alone do not trigger it. The URL check compares the session’s prior pinned tab target to `details.navigationSummary.url`. CSS selector clicks do not run this overlay probe. The diagnostic is skipped if the wrapper already applied tab-focus correction or about-blank recovery on that result. Appended `inspect-overlay-state` / `try-overlay-blocker-candidate-*` entries in `details.nextActions` preserve namespace/session context (`--namespace <namespace> --session <name>` when namespaced, otherwise `--session <name>` when the session is named), same as other session-scoped follow-ups. Treat `inspect-overlay-state` as the safe first follow-up; only use a `try-overlay-blocker-candidate-*` next action when the candidate is clearly the control you intend to close.
|
|
341
341
|
|
|
@@ -370,10 +370,12 @@ On tabbed or hidden-DOM pages, `get text <selector>` reads the upstream-selected
|
|
|
370
370
|
{ "args": ["batch"], "stdin": "[[\"open\",\"https://example.com\"],[\"snapshot\",\"-i\"]]" }
|
|
371
371
|
```
|
|
372
372
|
|
|
373
|
-
Use `batch --bail` when later steps should stop after the first failed command.
|
|
373
|
+
Use exact `batch --bail` when later steps should stop after the first failed command; omit it to continue after errors. `--bail=true` / `--bail=false` are unsupported: upstream treats them as raw command strings and ignores stdin. The wrapper returns shape guidance without running that ignored stdin. Tab recovery does not change caller flags, literal operands, or batch control flow; a failed wrapper tab selection stops before user commands. Both pinned and unpinned mixed failures retain per-step results and failure counts.
|
|
374
374
|
|
|
375
375
|
For short constrained flows, use top-level `job` instead of hand-writing `batch` stdin. Supported job steps are `open`, `click`, `fill`, `type`, `select`, `wait`, `assertText`, `assertUrl`, `waitForDownload`, `snapshot`, and `screenshot`. `open` can include `loadState: "domcontentloaded" | "load" | "networkidle"` to insert a `wait --load …` row immediately after navigation before the next click/read step. `click` and `fill` accept either a stable `selector` or the same semantic locator fields as top-level `semanticAction` (`locator`, plus `role`/`name` or `value` as appropriate) and compile locator steps to upstream `find` argv. `type` focuses an optional selector, sends text through upstream keyboard typing, can insert `wait` rows via `delayMs` for human-paced input, and can append a final `press` key such as `Enter`; delayed typing is capped at 200 characters per step, and generated per-character rows are compacted in model-visible batch text while remaining available in `details.batchSteps`. `select` requires `selector` plus `value` or `values`, and compiles to upstream `select <selector> <value...>`. By default the wrapper compiles steps to upstream `batch --bail` so a failed setup/fill/assertion step stops later mutating clicks; set `failFast: false` only when you explicitly need continue-after-error diagnostics and those later steps remain safe if an earlier navigation fails; otherwise keep fail-fast or split navigation from content. The wrapper records `details.compiledJob.steps[]` plus `details.compiledJob.failFast`. There is still no separate first-class catalog of reusable named browser recipes above `job`, the `qa` preset, and raw `batch`; see [`ARCHITECTURE.md`](ARCHITECTURE.md#no-reusable-recipe-layer-yet) for the closed `RQ-0068` decision and revisit bar.
|
|
376
376
|
|
|
377
|
+
`assertText` takes only `text`, not selector or locator fields. Clicks can stale subsequent `@refs`; split the job and take a fresh snapshot before using them.
|
|
378
|
+
|
|
377
379
|
**Job navigation is explicit.** A `click` step (or other navigation-prone interaction) does not prove the next page loaded. The wrapper does not auto-insert `assertUrl` or `assertText` after clicks inside `job`; add those steps yourself with the exact URL, a `*` / `**` glob-style URL pattern, or on-page text you expect, especially after forms, checkout, tabs, or submit buttons, before screenshots or later steps. Exact and glob-style `assertUrl` values compile to `wait --url` unchanged, including query strings and literal `?`; upstream `agent-browser 0.31.1` matches `*` / `**` patterns against the full active URL. Do not put a whole dynamic checkout into one long job: split around login, sorting/cart mutations, checkout navigation, and final evidence capture so refs and app state can be rechecked between phases.
|
|
378
380
|
|
|
379
381
|
```json
|
|
@@ -459,7 +461,9 @@ Typical lifecycle:
|
|
|
459
461
|
{ "electron": { "action": "cleanup", "launchId": "electron-…" } }
|
|
460
462
|
```
|
|
461
463
|
|
|
462
|
-
`electron.status` and `electron.cleanup` take either `launchId`, **`all: true`** (literal boolean) to walk every wrapper-tracked launch
|
|
464
|
+
`electron.status` and `electron.cleanup` take either `launchId`, **`all: true`** (literal boolean) to walk every active wrapper-tracked launch (including dead, failed, or partial records, but excluding cleaned records), or neither when exactly one active launch exists—never both `launchId` and `all`. They can target the current branch-visible launch plus still-owned off-branch launch records by `launchId`; default no-arg calls are intentionally ambiguous when more than one active launch is owned. `/reload` preserves the current branch-visible active Electron launch and its isolated temp `userDataDir` for continuity, and cleans off-branch owned Electron launches; if cleanup is partial and skips or fails profile removal, the generic temp sweep preserves that `userDataDir` across reload, quit, later temp cleanup, process exit, and stale temp-root pruning after restart. `electron.list` has no configurable timeout and rejects both top-level and nested `timeoutMs`. For `electron.launch`, nested `timeoutMs` sets host CDP readiness polling to a **15s** default and **120s** cap after target discovery; upstream attach and handoff use separate subprocess budgets. Optional `timeoutMs` on **`status`** applies to managed-session `get url`, then `get title` reads (localhost CDP probes stay on a short fixed fetch budget). On **`cleanup`**, it is applied separately to upstream `close` and the initial host process-exit wait, not to the entire teardown; debug-port checks have fixed fetch budgets and profile removal has no configurable deadline; when omitted it follows the implicit session close default (**5s** unless `PI_AGENT_BROWSER_IMPLICIT_SESSION_CLOSE_TIMEOUT_MS` overrides). A successful managed-session close step retires that wrapper-managed session even when host process/profile cleanup remains partial. On **`probe`**, it bounds each underlying upstream read subprocess—omit it to use the normal tool subprocess default, or raise it on slow desktops.
|
|
465
|
+
|
|
466
|
+
Explicit-ID `electron.status` labels a cleaned record as historical while measuring PID/port liveness independently. `details.electron.statuses[].userDataDirState` freshly reports the tracked profile path as `present`, `absent` (only ENOENT), or `unknown` (other native `lstat` errors); dangling symlinks are present. This is not an audit of all app residue and does not change stored launch records or cleanup ownership.
|
|
463
467
|
|
|
464
468
|
`launch.handoff` defaults to `"snapshot"`, which attaches through upstream `connect`, lists targets, and captures a current `snapshot -i` in one call. Snapshot handoff retries briefly when the first Electron snapshot has no refs; if it still reports no refs, run `snapshot -i` once more before assuming the app is blank. Use `handoff: "tabs"` as the safer diagnostic starting point when you only need target discovery and do not want to snapshot app content yet, or `handoff: "connect"` when you want to attach first and run your own follow-up commands. `targetType` defaults to `"page"`; use `"webview"` or `"any"` for apps that expose useful webviews. When a matching CDP target exposes a WebSocket URL, launch connects to that target; otherwise it falls back to the browser port.
|
|
465
469
|
|
|
@@ -533,15 +537,15 @@ Prefer `download <selector> <path>` when the target element itself is the downlo
|
|
|
533
537
|
|
|
534
538
|
For evidence-only screenshots, QA captures, or audit artifacts, save to an explicit path and branch on `details.artifactVerification` plus `details.artifacts` before reporting PASS/FAIL. Inline image attachments are optional convenience when size limits allow; do not require vision review unless the user asked for visual inspection.
|
|
535
539
|
|
|
536
|
-
Wrapper result rendering is metadata-first for saved files. An artifact-producing command fails as `artifact-missing` with artifact `status: "stale"` when the reported path's `mtimeMs` falls outside the command's bounded start/end window (with two seconds of filesystem precision tolerance), including a previous recording that `record restart` claims to finalize; clearly old or future-dated evidence is never accepted as a fresh capture. A batch, whether supplied through stdin arrays or argument command strings, must use distinct explicit artifact destinations; preflight canonicalizes existing path ancestry, compares existing file identities to catch hardlinks, and applies full Unicode plus platform case folding on macOS/Windows so aliases cannot satisfy another step's verification. The same preflight prevents `outputPath` from aliasing a same-call browser artifact, follows upstream's forward option consumption and final effective `-o` / `--output` for `diff screenshot`, and treats the optional path on `network har stop` as an artifact destination; upstream ignores positional paths on `network har start`. Artifact and lifecycle parsing first removes upstream global flags wherever they occur, so accepted forms such as `record --json start <path>` and `pdf --quiet <path>` cannot shift or bypass destination tracking. Screenshot destination parsing mirrors upstream's exact flag matching and `[selector] [path]` positional order: `--` is positional, `true` / `false` after screenshot-only `--full` / `-f` remain positional, extra positionals are ignored after the path slot, selector-prefixed (`.`, `#`, `@`) or uppercase-extension single arguments remain selectors, and lowercase image extensions or slash-bearing arguments are paths. The wrapper deliberately keeps its existing slash-bearing hidden-workspace path normalization (for example `.dogfood/run/foo.png`) before launch. `wait --download` is observational and may verify a download that completed just before the wait began, so it is exempt from the command-window mtime gate; an explicit wait destination, in long `--download <path>` or short `-d <path>` form (including after `--timeout`), still participates in active-recording reservation preflight; unsupported `--download=<path>` fails with split-argument guidance:
|
|
540
|
+
Wrapper result rendering is metadata-first for saved files. Image MIME types come from a bounded header read for PNG, JPEG, GIF and WebP, never from a filename suffix; missing, unreadable, unknown or truncated headers omit `mediaType`. This identifies a format, not full image validity. Inline screenshots use the same byte check and existing size limit, so a PNG saved as `.webm` still attaches as `image/png`; other artifact kinds are not auto-inlined. An artifact-producing command fails as `artifact-missing` with artifact `status: "stale"` when the reported path's `mtimeMs` falls outside the command's bounded start/end window (with two seconds of filesystem precision tolerance), including a previous recording that `record restart` claims to finalize; clearly old or future-dated evidence is never accepted as a fresh capture. A batch, whether supplied through stdin arrays or argument command strings, must use distinct explicit artifact destinations; preflight canonicalizes existing path ancestry, compares existing file identities to catch hardlinks, and applies full Unicode plus platform case folding on macOS/Windows so aliases cannot satisfy another step's verification. The same preflight prevents `outputPath` from aliasing a same-call browser artifact, follows upstream's forward option consumption and final effective `-o` / `--output` for `diff screenshot`, and treats the optional path on `network har stop` as an artifact destination; upstream ignores positional paths on `network har start`. Artifact and lifecycle parsing first removes upstream global flags wherever they occur, so accepted forms such as `record --json start <path>` and `pdf --quiet <path>` cannot shift or bypass destination tracking. Screenshot destination parsing mirrors upstream's exact flag matching and `[selector] [path]` positional order: `--` is positional, `true` / `false` after screenshot-only `--full` / `-f` remain positional, extra positionals are ignored after the path slot, selector-prefixed (`.`, `#`, `@`) or uppercase-extension single arguments remain selectors, and lowercase image extensions or slash-bearing arguments are paths. The wrapper deliberately keeps its existing slash-bearing hidden-workspace path normalization (for example `.dogfood/run/foo.png`) before launch. `wait --download` is observational and may verify a download that completed just before the wait began, so it is exempt from the command-window mtime gate; an explicit wait destination, in long `--download <path>` or short `-d <path>` form (including after `--timeout`), still participates in active-recording reservation preflight; unsupported `--download=<path>` fails with split-argument guidance:
|
|
537
541
|
- screenshots return a saved-path summary, visible artifact metadata, structured `details.artifacts` metadata, and an inline image attachment when safe; the visible block includes artifact type, requested path, absolute path, existence, size, cwd, session, and repair/copy status when applicable
|
|
538
542
|
- downloads, PDFs, `wait --download` files, `state save` state files, diff screenshot output images, traces, CPU profiles, completed WebM recordings from `record stop`, and path-bearing HAR captures return concise saved-path summaries plus structured `details.artifacts` metadata without inlining large files
|
|
539
|
-
- `record start <path>` and `record restart <path>` report `successCategory: "artifact-pending"` and that output will be written on `record stop`; `record start`
|
|
543
|
+
- `record start <path>` and `record restart <path>` report `successCategory: "artifact-pending"` and that output will be written on `record stop`; dispatched `record start` and URL-bearing `record restart` attempts append one `Page state:` warning on success or failure, advising a fresh snapshot because in-page DOM and JavaScript state may not carry over; explicit `--json` puts that warning in `warnings`. Only reached batch rows qualify, not preflight failures, missing binaries, help calls or unconfirmed planned rows — the wrapper invalidates the session’s prior ref snapshot (direct calls and batch steps alike, and even when the start fails with `Recording already active`, because upstream swaps the page before that check), so old `@e…` refs fail as `stale-ref` until a fresh `snapshot -i` succeeds; `record restart <path> <url>` navigates the current page and invalidates refs the same way, while a plain `record restart <path>` keeps the current page and refs; `details.artifacts` / `details.artifactVerification` mark that future file as `pending` with `recordingState: "openRecording"` and `willExistOnStop: true`, and `details.nextActions` includes exact `stop-pending-recording` args. When `record restart` finalizes a previous wrapper-known recording, that file must exist and fall within the command mtime window before the result includes `Previous recording saved: …`; a missing or stale prior file fails as `artifact-missing` while the new recording remains visible as pending and the prior manifest row is retired. Within one Pi extension process, an unbounded transcript-backed index reserves active recording destinations independently of the bounded artifact manifest. Artifact lifecycle calls and result `outputPath` writes serialize around that global check; reservations use canonical namespace/session identity, survive manifest eviction and branch replay, and retire after direct, ordered nested-batch, fresh-replacement, script, Electron, or shutdown close; the newest pending row per identity is authoritative. Legacy batch replay retires a pending manifest only when the ordered close lifecycle leaves recording closed; a later successful browser reactivation plus `record start` keeps the new pending reservation. Lexical, hardlink, existing/dangling symlink, full Unicode-fold, and macOS/Windows case aliases are rejected, so `record restart` must use a distinct new path. Do not place `record start` or `record restart` after `close` / `quit` / `exit` in one batch: wrapper preflight rejects it because upstream can report success without starting a recording; split the close and recording into separate calls. A definitive `No recording in progress` stop failure, whether direct or inside a batch, retires stale reservation state at that ordered step; a later successful batch recording row opens its new pending path normally. Any success or failure result that still contains pending recording output includes `stop-pending-recording`. The target may not exist until recording stops, and upstream needs `ffmpeg` on `PATH` at stop time to encode the WebM. If `ffmpeg` is missing after a successful `record start` / `record restart`, the wrapper appends `Recording dependency warning: ffmpeg not found on PATH` and sets `details.recordingDependencyWarning` without blocking the upstream command.
|
|
540
544
|
- `batch` keeps each step's artifacts in `details.batchSteps[].artifacts`; top-level `details.artifacts` and `details.artifactManifest` coalesce an earlier pending recording into the later saved, missing, or stale terminal result for the same namespace/session identity; a successful later close marks an unfinalized pending recording `missing` / `close-abandoned`, removes its stop action, and resets earlier ref/page/network-route batch state; a later successful `record stop` replaces that intermediate abandoned row with its verified saved artifact, and later rows—including failed rows—whose lifecycle reports a browser launch may rebuild state without triggering stale pre-close `about:blank` recovery; failed-step `batchSteps[]` retains only the bounded `lifecycle.effectiveLaunch.browserLaunched` boolean for replay, explicitly non-launching diagnostics leave the close terminal, missing lifecycle evidence remains conservatively active even on the first managed call, every successful close clears wrapper trace/profiler ownership before ordered later successful rows can rebuild it, namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, and any later same-session failure before recording stops keeps exact `stop-pending-recording` args alongside its normal recovery
|
|
541
545
|
|
|
542
546
|
`diff screenshot` follows the file-artifact path above for the **diff** image: model-visible text and `details.artifacts` focus on that output, while baseline paths stay out of the artifact summary block, and Pi does **not** auto-inline the diff the way it inlines trusted `screenshot` captures. `state load` may print the loaded path in prose but does not add a saved-file artifact entry the way `state save` does.
|
|
543
547
|
|
|
544
|
-
For screenshot paths under dot-directories such as `.dogfood/run/foo.png`, the wrapper normalizes the requested path to an absolute path before invoking upstream `agent-browser`, verifies the requested file exists, and repairs from an upstream temp screenshot when possible. For direct artifact commands and batch artifact steps (`download`, `pdf`, `screenshot`, `state save`, and `wait --download`), the wrapper creates missing parent directories before launch.
|
|
548
|
+
For screenshot paths under dot-directories such as `.dogfood/run/foo.png`, the wrapper normalizes the requested path to an absolute path before invoking upstream `agent-browser`, verifies the requested file exists, and repairs from an upstream temp screenshot when possible. For direct artifact commands and batch artifact steps (`download`, `pdf`, `screenshot`, `state save`, and `wait --download`), the wrapper creates missing parent directories before launch. A parent-directory failure returns `validation-error` with the attempted directory and `verify-artifact-path` guidance before the browser command runs. Use **absolute paths in raw batch artifact rows**: raw strings stay unchanged, and the daemon's working directory may differ from Pi's. Screenshot path normalization still applies only to direct calls and stdin rows. Known caller paths appear as `Requested path`; `Absolute path` is the resolved location checked on disk, and `Reported path` exposes a differing screenshot report (including a canonical `/private/tmp` alias) through the existing `tempPath` metadata. No extra canonicalization rewrites upstream arguments.
|
|
545
549
|
|
|
546
550
|
For annotated screenshots in `batch`, put `--annotate` in top-level args instead of inside the screenshot step:
|
|
547
551
|
|
|
@@ -664,7 +668,7 @@ Skill-source debugging note: upstream honors `AGENT_BROWSER_SKILLS_DIR` as an ov
|
|
|
664
668
|
| `click <sel>` | Click an element or `@ref`. |
|
|
665
669
|
| `click <sel> --new-tab` | Click a link/control while requesting a new tab. |
|
|
666
670
|
| `dblclick <sel>` | Double-click an element. |
|
|
667
|
-
| `type <sel> <text>` | Type into an element. |
|
|
671
|
+
| `type <sel> <text>` | Type into an element; both selector and text are required. For the focused element without a selector, use `keyboard type <text>`. |
|
|
668
672
|
| `fill <sel> <text>` | Clear and fill an element. |
|
|
669
673
|
| `press <key>` | Press a key such as `Enter`, `Tab`, or `Control+a`. `key <key>` is the upstream alias. |
|
|
670
674
|
| `key <key>` | Alias for `press <key>`. |
|
|
@@ -697,7 +701,7 @@ Skill-source debugging note: upstream honors `AGENT_BROWSER_SKILLS_DIR` as an ov
|
|
|
697
701
|
|
|
698
702
|
On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
|
|
699
703
|
|
|
700
|
-
Comboboxes vary by app. For native `<select>` controls, prefer raw `select <selector> <value...>`, direct `semanticAction: { action: "select", selector, value|values }`, active-session semantic role/name or label select, or a `job` `select` step instead of clicking option refs; native option refs can be non-boxed in CDP and fail before a real selection. A `click` or `semanticAction` role/name click may focus a searchable custom combobox without opening its option list. For explicit combobox-targeted actions such as `semanticAction` role `combobox`, the wrapper checks whether a combobox-like element is focused, has explicit `aria-expanded` state, and has no visible listbox/options open; this still applies when the semantic action first resolves to a current visible `@ref` before execution. When that happens it appends `Combobox diagnostic: focused combobox did not expose visible options`, exposes `details.comboboxFocus`, and adds exact `details.nextActions` for a fresh `snapshot -i`, `press ArrowDown`, and `press Enter`. Use those instead of assuming click alone expanded the control
|
|
704
|
+
Comboboxes vary by app. For native `<select>` controls, prefer raw `select <selector> <value...>`, direct `semanticAction: { action: "select", selector, value|values }`, active-session semantic role/name or label select, or a `job` `select` step instead of clicking option refs; native option refs can be non-boxed in CDP and fail before a real selection. A `click` or `semanticAction` role/name click may focus a searchable custom combobox without opening its option list. For explicit combobox-targeted actions such as `semanticAction` role `combobox`, the wrapper checks whether a combobox-like element is focused, has explicit `aria-expanded` state, and has no visible listbox/options open; this still applies when the semantic action first resolves to a current visible `@ref` before execution. When that happens it appends `Combobox diagnostic: focused combobox did not expose visible options`, exposes `details.comboboxFocus`, and adds exact `details.nextActions` for a fresh `snapshot -i`, `press ArrowDown`, and `press Enter`. Use those instead of assuming click alone expanded the control. To search the focused input, use `keyboard type <text>`; raw `type` requires both a selector and text. Reserve visible option refs for custom comboboxes after a fresh snapshot shows the intended option.
|
|
701
705
|
|
|
702
706
|
### Navigation
|
|
703
707
|
|
|
@@ -726,10 +730,12 @@ Comboboxes vary by app. For native `<select>` controls, prefer raw `select <sele
|
|
|
726
730
|
| `dialog accept [text]` | Accept an alert, confirm, or prompt dialog, optionally supplying prompt text. |
|
|
727
731
|
| `dialog dismiss` | Dismiss or cancel the current dialog. |
|
|
728
732
|
| `dialog status` | Check whether a dialog is pending. |
|
|
729
|
-
| `window new` | Open a new
|
|
733
|
+
| `window new` | Open and activate a new blank window. The wrapper keeps that intentional `about:blank` target rather than selecting the old tab. Old refs are invalid; take a fresh snapshot before further interaction. |
|
|
730
734
|
| `close` | Close the current browser session. |
|
|
731
735
|
| `close --all` | Close every session. |
|
|
732
736
|
|
|
737
|
+
A canceled cold-session reopen returns an aborted result with its exact session identity and the consumed reopen marker once the CLI starts. Reload does not repeat that navigation. Cancellation before the attempt leaves the remembered URL pending for the next current-page operation.
|
|
738
|
+
|
|
733
739
|
<!-- agent-browser-playbook:start inspection -->
|
|
734
740
|
<!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
|
|
735
741
|
Native inspection calls use the `agent_browser` tool shape, not shell-like direct-binary commands:
|
|
@@ -813,7 +819,7 @@ For dense pages, the wrapper also accepts `snapshot -i --search <text>` and `sna
|
|
|
813
819
|
| Mode | Purpose |
|
|
814
820
|
| --- | --- |
|
|
815
821
|
| `wait <selector>` | Wait for an element to appear. |
|
|
816
|
-
| `wait <ms>` | Wait for a fixed number of milliseconds
|
|
822
|
+
| `wait <ms>` | Wait for a fixed number of milliseconds; the duration is positional, not `--time <ms>`. The native Pi wrapper now forwards long waits and derives a subprocess watchdog from the explicit wait duration when the caller does not provide top-level `timeoutMs`. |
|
|
817
823
|
| `wait --url <pattern>` | Wait for the URL to match a pattern. On timeout the wrapper appends a `fresh-session-after-url-wait-timeout` next action (`sessionMode: "fresh"` + `open about:blank`, after the inspect action): if a preceding click or submit reported success but the page never navigated, upstream click dispatch may have silently missed, so replace about:blank with the target URL and replay the flow as one batch in a fresh session instead of retrying the wait. |
|
|
818
824
|
| `wait --load <state>` | Wait for load state: `load`, `domcontentloaded`, or `networkidle`. |
|
|
819
825
|
| `wait --fn <expression>` | Wait for a JavaScript expression to become truthy. |
|
|
@@ -829,7 +835,7 @@ Current upstream still does not parse `wait <selector> --state hidden` / `wait <
|
|
|
829
835
|
| --- | --- |
|
|
830
836
|
| `diff snapshot` | Compare current versus last snapshot. Use `diff snapshot --baseline <file> --selector <sel> --compact --depth <n>` when you need a saved baseline, scoped subtree, compact output, or depth bound. |
|
|
831
837
|
| `diff screenshot --baseline` | Compare current screenshot versus a baseline image. Use `diff screenshot --baseline <file> --output <file> --threshold <0-1> --selector <sel> --full` when you need a saved diff image, threshold tuning, element scope, or full-page capture. |
|
|
832
|
-
| `diff url <u1> <u2>` |
|
|
838
|
+
| `diff url <u1> <u2>` | Navigate to both pages and compare them, leaving the second destination active. The wrapper observes the final URL, including redirects to `about:blank`, and invalidates old refs without recovering the old tab; direct and reached batch rows use the same rule. If the URL cannot be observed, run `get url` before taking a fresh snapshot. Use `diff url <u1> <u2> --screenshot --wait-until <strategy> --selector <sel> --compact --depth <n>` when you need screenshot comparison, navigation wait control, or scoped/compact snapshot comparison. |
|
|
833
839
|
| `trace start`, `trace stop [path]` | Record a Chrome DevTools trace. |
|
|
834
840
|
| `profiler start|stop [path]` | Record a Chrome DevTools profile. |
|
|
835
841
|
| `record start <path> [url]` | Start WebM video recording; output is written on `record stop`. Requires `ffmpeg` on `PATH` for the final encode. |
|
|
@@ -882,7 +888,7 @@ Long-running or lifecycle commands should be explicitly paired with cleanup call
|
|
|
882
888
|
| `device list` | List available iOS simulators. Use with `-p ios` when exercising iOS provider flows. |
|
|
883
889
|
| `install` | Install browser binaries. |
|
|
884
890
|
| `install --with-deps` | Install browser binaries plus Linux system dependencies; exits nonzero when required libraries cannot be installed. |
|
|
885
|
-
| `upgrade` | Upgrade `agent-browser`
|
|
891
|
+
| `upgrade` | Upgrade `agent-browser` using its detected package manager. Native output is text, even with `--json`; the wrapper displays it with surrounding whitespace trimmed and normal redaction, and exposes it in `details.data`. Caller-requested `--json` stays a parseable result with the text in `data`. Nonzero exits, spawn failures, timeout and cancellation remain failures; failed upgrade stdout and stderr remain available as diagnostics. |
|
|
886
892
|
| `doctor [--fix]` | Diagnose install issues and optionally auto-clean stale files. Use `doctor --offline --quick` for a fast local-only check and `doctor --json` for structured output. |
|
|
887
893
|
| `plugin add <ref>` | Add a plugin from npm or GitHub (`<owner>/<repo>` or `@scope/<name>`); writes `agent-browser.json`. Flags such as `--name`, `--capability`, `--global`, and `--no-manifest` shape discovery. |
|
|
888
894
|
| `plugin [list]` | List configured plugins (default subcommand); `{ "plugins": [...] }` is a successful sessionless result. |
|
|
@@ -1004,7 +1010,7 @@ Browser default config is conservative: it adds agent guidance for signed-in/acc
|
|
|
1004
1010
|
|
|
1005
1011
|
- `--executable-path <path>`: custom Chromium-compatible browser executable, such as Brave/Edge/Arc/Vivaldi when upstream can launch that binary. Environment: `AGENT_BROWSER_EXECUTABLE_PATH`.
|
|
1006
1012
|
- `--extension <path>`: load browser extensions; repeatable. Environment: `AGENT_BROWSER_EXTENSIONS`.
|
|
1007
|
-
- `--args <args>`: browser launch args, comma or newline separated. Environment: `AGENT_BROWSER_ARGS`.
|
|
1013
|
+
- `--args <args>`: browser launch args, comma or newline separated. Chromium switches belong in this value, for example `{ "args": ["--args", "--no-sandbox", "open", "https://example.com"], "sessionMode": "fresh" }`. A bare `--no-sandbox` is diagnosed when it occupies the command slot (unknown command) or an `open` / `goto` / `navigate` option position (ignored by upstream); literal operands in other commands are left alone. For batches, put `--args` before `batch` in top-level `args`, not inside a row. Environment: `AGENT_BROWSER_ARGS`.
|
|
1008
1014
|
- `--user-agent <ua>`: custom user agent. Environment: `AGENT_BROWSER_USER_AGENT`.
|
|
1009
1015
|
- `--proxy <server>`: proxy server URL. Environments: `AGENT_BROWSER_PROXY`, `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`.
|
|
1010
1016
|
- `--proxy-bypass <hosts>`: proxy bypass hosts. Environments: `AGENT_BROWSER_PROXY_BYPASS`, `NO_PROXY`.
|
|
@@ -1070,7 +1076,8 @@ Other useful environment variables include `AGENT_BROWSER_DEFAULT_TIMEOUT`, `AGE
|
|
|
1070
1076
|
<!-- agent-browser-playbook:start wrapper-tab-recovery -->
|
|
1071
1077
|
<!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
|
|
1072
1078
|
- After open/goto/navigate calls with --profile, --restore, --session-name, or --state, agent_browser best-effort re-selects the tab whose URL matches the returned page when restored tabs steal focus during launch or reconnect.
|
|
1073
|
-
- After
|
|
1079
|
+
- After confirmed shutdown of an automatically restored managed session, the wrapper retains its complete recorded URL, including the fragment, until the first current-page operation (including get url and reload). Non-page calls such as tab list or read <url> may start a daemon without fulfilling that reopen. The wrapper uses native open once, verifies the observed tab, and discards old refs/frame scope; it does not restore unsaved forms, JavaScript memory, or history. Explicit navigation, caller-owned/attached sessions, and restore-disabled sessions are not auto-reopened.
|
|
1080
|
+
- For a still-live browser after tab drift or resume, the wrapper verifies/selects the intended tab before ref/semantic helpers and page commands; failed selection stops the call without navigating. Local commands, read <url>, URL a11y/vitals, diff url, window new, and explicit tab/navigation/connection/state recovery do not require the prior tab. Batch checks follow effective rows past non-page prefixes and stop at explicit context changes, preserving caller argv/stdin and continue-on-error behavior. Same-tab reselection is avoided because it clears refs. Use exact batch --bail for fail-fast, not --bail=<value>. Routine same-session calls skip tab-list preflights.
|
|
1074
1081
|
- For sessions with observed tab-drift risk, after a successful command on a known target tab, agent_browser also best-effort restores that intended tab if a restored/background tab steals focus after the command completes. Routine same-session commands skip this post-command tab-list probe.
|
|
1075
1082
|
- If a known session target unexpectedly reports about:blank, agent_browser best-effort re-selects the prior intended target when it still exists; if recovery fails, it records the observed about:blank target and reports exact recovery guidance instead of treating the prior page as active.
|
|
1076
1083
|
- If upstream reports tab_gone, the pinned bound tab is gone; use details.nextActions (tab list / tab new) instead of assuming another tab is yours.
|
package/docs/ELECTRON.md
CHANGED
|
@@ -153,7 +153,7 @@ Handoff selection (`handoff` field):
|
|
|
153
153
|
|
|
154
154
|
`targetType` defaults to `"page"`; use `"webview"` or `"any"` for apps whose useful UI is exposed as a webview target.
|
|
155
155
|
|
|
156
|
-
Optional `timeoutMs` on `electron.launch`
|
|
156
|
+
Optional `timeoutMs` on `electron.launch` sets the host readiness polling budget for `DevToolsActivePort` and CDP metadata. The clock starts after target discovery and policy checks; upstream attach and handoff have separate subprocess budgets. When omitted, the default is **15 seconds** with a hard maximum of **120 seconds**, matching `ELECTRON_LAUNCH_DEFAULT_TIMEOUT_MS` and `ELECTRON_LAUNCH_MAX_TIMEOUT_MS` in `extensions/agent-browser/lib/electron/launch.ts`. Pi cancellation is separate: an already-cancelled call never launches the app, while cancellation during readiness polling or URL/tab/snapshot handoff closes the managed session, stops the tracked process, removes its isolated profile, and returns `failureCategory: "aborted"` without waiting for the launch timeout.
|
|
157
157
|
|
|
158
158
|
Wrapper-owned launches **always** use an isolated temp profile and an OS-chosen port. If wrapper validation, managed-session policy, or the post-attach live-URL handoff guard fails after the host app starts, the wrapper immediately stops that process and removes the isolated profile; it retains a partial tracked record only when cleanup itself cannot finish. `--user-data-dir`, `--remote-debugging-port`, `--remote-debugging-address`, `--remote-debugging-pipe`, and bare `--` in `appArgs` are rejected. There is no caller-supplied port and no way to make `electron.launch` reuse the app's normal signed-in profile or attach to an already-running app — by design. Use the manual path described above when those are the actual requirements.
|
|
159
159
|
|
|
@@ -167,7 +167,7 @@ Read-only inspection of one or more tracked launches. Without `launchId` or `all
|
|
|
167
167
|
{ "electron": { "action": "status", "all": true } }
|
|
168
168
|
```
|
|
169
169
|
|
|
170
|
-
Reports `cleanupState`, debug-port and PID liveness,
|
|
170
|
+
Reports `cleanupState`, current debug-port and PID liveness, bounded CDP targets, and freshly measured `userDataDirState` under `details.electron.statuses`. An explicit `launchId` can inspect a **historical cleaned launch record**; default and `all: true` selection exclude cleaned records. Cleanup history does not determine current liveness. The tracked profile path is `present`, `absent` (only native `lstat` ENOENT), or `unknown` (other filesystem errors); a dangling symlink is present. This measures that path only, not all app residue, and is not stored in the launch record. Its managed-session title/URL reads hold the normal daemon-policy lock and owned restore context. Mismatch fields surface when the current managed session or tab no longer matches a live wrapper launch target — typically the cue to follow `reattach-electron-launch` before trusting old refs.
|
|
171
171
|
|
|
172
172
|
### `electron.probe` — compact state read
|
|
173
173
|
|
|
@@ -204,14 +204,14 @@ On Pi `quit`, active wrapper-owned Electron launches are best-effort cleaned. On
|
|
|
204
204
|
|
|
205
205
|
### `timeoutMs` by action (quick reference)
|
|
206
206
|
|
|
207
|
-
`electron.list`
|
|
207
|
+
`electron.list` has no configurable timeout: neither top-level `timeoutMs` nor nested `electron.timeoutMs` is accepted for its host scan. For every other action, nested `timeoutMs` applies to **different surfaces**, not an end-to-end action deadline. Authoritative rules and env overrides live under **Validation and defaults** in [`TOOL_CONTRACT.md#electron`](TOOL_CONTRACT.md#electron).
|
|
208
208
|
|
|
209
209
|
| Action | What `timeoutMs` covers when set | Typical default when omitted |
|
|
210
210
|
| --- | --- | --- |
|
|
211
211
|
| `launch` | Host-side wait for `DevToolsActivePort` and CDP readiness | **15 s**, hard-capped at **120 s** (`normalizeTimeoutMs` in `extensions/agent-browser/lib/electron/launch.ts`) |
|
|
212
|
-
| `status` |
|
|
213
|
-
| `cleanup` |
|
|
214
|
-
| `probe` | **Each** upstream read in the probe chain (`get url`, then `get title`, focused `eval --stdin`, `tab list`, `snapshot -i`) | Same
|
|
212
|
+
| `status` | Each optional managed-session `get url` / `get title` subprocess used for mismatch diagnostics | Normal wrapper subprocess budget (**35 s**, or `PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS`); localhost CDP probes use **1000 ms** each (`ELECTRON_CDP_FETCH_TIMEOUT_MS` in `extensions/agent-browser/lib/electron/cdp.ts`) |
|
|
213
|
+
| `cleanup` | Applied separately to managed-session `close` and the initial tracked-process exit wait; not a deadline for debug-port checks or profile removal | `PI_AGENT_BROWSER_IMPLICIT_SESSION_CLOSE_TIMEOUT_MS` when set, else **5000 ms** (`getImplicitSessionCloseTimeoutMs` in `extensions/agent-browser/lib/runtime.ts`, passed through `cleanupTrackedElectronHostLaunches` in `extensions/agent-browser/lib/orchestration/electron-host/index.ts`) |
|
|
214
|
+
| `probe` | **Each** upstream read in the probe chain (`get url`, then `get title`, focused `eval --stdin`, `tab list`, `snapshot -i`) | Same wrapper subprocess default (**35 s**, or `PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS`, from `getAgentBrowserProcessTimeoutMs` in `extensions/agent-browser/lib/process.ts`) |
|
|
215
215
|
|
|
216
216
|
## `qa.attached` — current-session smoke check
|
|
217
217
|
|
package/docs/RELEASE.md
CHANGED
|
@@ -96,9 +96,11 @@ Use this optional-but-recommended checklist when a release touches dashboard beh
|
|
|
96
96
|
Target:
|
|
97
97
|
|
|
98
98
|
```text
|
|
99
|
-
https://play.grafana.org/d/
|
|
99
|
+
https://play.grafana.org/d/nodes/linux-node-overview?var-datasource=grafanacloud-prom
|
|
100
100
|
```
|
|
101
101
|
|
|
102
|
+
Public panels may show No data. Verify the rendered dashboard and controls; this UI checklist does not certify metric ingestion.
|
|
103
|
+
|
|
102
104
|
Minimum pass:
|
|
103
105
|
|
|
104
106
|
1. Open the URL with the native `agent_browser` tool in a fresh session.
|
|
@@ -231,7 +233,7 @@ npm run verify -- lifecycle --model openai-codex/gpt-5.5:minimal --timeout-ms 60
|
|
|
231
233
|
|
|
232
234
|
On failure it retains transcripts/session artifacts; on success it performs best-effort cleanup. It does not replace occasional real-browser manual smoke testing.
|
|
233
235
|
|
|
234
|
-
**Lifecycle triage:** a timeout on sentinel `v2` after exact-session relaunch means the new compiled entrypoint did not load after process restart. A reload-step timeout or missing post-reload snapshot often means Pi rejected reload while the TUI still showed `Working…` (`Wait for the current response to finish before reloading`), even when the session JSONL already has a final assistant message. Re-run with `--keep-artifacts --verbose`, inspect the retained pane capture, and confirm the configured model follows tool prompts reliably. Slower models may need a higher `--timeout-ms` than the **180000 ms** default.
|
|
236
|
+
**Lifecycle triage:** page checks require the first completed tool result to report a successful expected command and observed `data.url` (open) or `data.origin` (snapshot); recovery text and remembered targets are not page evidence. A cold post-quit `tab-drift` remains a continuity failure: upstream restores storage, and the wrapper must reopen the recorded URL after confirming shutdown before the first current-page read. Verify both empty storage and origin storage at non-root URLs and hash-routed URLs, including after `tab list` or explicit HTTP reads start the daemon and in batches with non-page prefixes, plus fresh refs and main-frame scope. A blank page, an origin root, or a URL mentioned only in an error is not the remembered page. Explicit URL QA recovery does not satisfy the same-page snapshot requirement; URL reopening does not recover unsaved forms, JavaScript memory, or history. A timeout on sentinel `v2` after exact-session relaunch means the new compiled entrypoint did not load after process restart. A reload-step timeout or missing post-reload snapshot often means Pi rejected reload while the TUI still showed `Working…` (`Wait for the current response to finish before reloading`), even when the session JSONL already has a final assistant message. Re-run with `--keep-artifacts --verbose`, inspect the retained pane capture, and confirm the configured model follows tool prompts reliably. Slower models may need a higher `--timeout-ms` than the **180000 ms** default.
|
|
235
237
|
|
|
236
238
|
### Environment and automation pitfalls
|
|
237
239
|
|
|
@@ -269,14 +271,21 @@ This suite requires the installed stable `agent-browser --version` to meet the m
|
|
|
269
271
|
- **WebMCP target contract:** on the 0.36.0 target, `webmcp list`, `invoke` with params/frame selection, detached `result` / `cancel`, ref invalidation after page tools, and a separate fresh launch with `--no-webmcp` returning an empty list.
|
|
270
272
|
- **Managed session core and safe diagnostic matrix:** fresh `open` on the contract fixture, then implicit reuse across `eval --stdin`, `snapshot -i`, interaction commands (`click`, `dblclick`, `fill`, `type`, `type --clear --delay`, `focus`, `keyboard` with `type` / `inserttext`, `press`, `hover`, `check`, `uncheck`, `select`, failed `select` no-match, `upload`, `drag`, `mouse`, `scroll`, off-viewport click, `scrollintoview`, `wait` on selectors in the main frame and a selected iframe), extraction (`get` variants, `is` variants, `find label … fill` via native `<label>`, `aria-label`, and `aria-labelledby`, inline `eval`), file outputs (`screenshot`, `pdf`), navigation (`back`, `forward`, `reload`, `tab list`, another `open` to the same fixture), `batch` stdin, `pushstate`, `vitals … --json`, network route/requests/HAR, diff snapshot/screenshot/url, trace/profiler, console/errors/highlight, stream enable/status/disable, and `cookies set --curl`.
|
|
271
273
|
- **Managed restore correctness and persistence:** while the restore-enabled managed daemon is active, assert raw argument and stdin batches containing nested `connect` fail before upstream spawn; a new empty-transcript harness must also reject incompatible reuse of that live same-name daemon. Seed a cookie plus localStorage/sessionStorage, close the first managed browser while a conflicting parent namespace is set, verify the default-namespace daemon actually closed, create a new extension harness with the same cwd, reopen the fixture, and assert all three values restore before closing the second browser. On POSIX, separate isolated real-browser launches assert automatic restore stays disabled and no snapshot is written through either a symlinked `sessions` directory or a file symlink in `sessions/.tmp`; a relative `HOME`, untrusted writable HOME ancestry, and a non-Git cwd must fail closed. Verify a checkout rename preserves its generation identity but starts a fresh composite restore key (fail-closed, because the cwd-derived managed-session base name changes), a copied or path-replacement checkout gets a new key, and changing the Git-generation marker between planning and spawn prevents agent-browser from starting. Run two same-identity harnesses concurrently so a compatible launch publishes its daemon policy before a waiting incompatible launch re-inspects and fails without reaching its main spawn; also fail a fresh non-batch command after daemon creation and verify shutdown closes the retained identity.
|
|
274
|
+
- **Cold first-read continuity:** the focused `contract suite matches cold URL reopen after quit` case runs actual quit cleanup, waits for the exact old daemon to exit, reloads the saved branch, and requests `snapshot -i` before any explicit navigation. Empty storage and origin storage must both reach their remembered non-root URL, report fresh refs in the main frame, and retain storage without pretending to retain unsaved forms or JavaScript memory. Separate deterministic cold-resume/boundary tests cover first `get url` and history commands, non-page daemon-starting calls and batch prefixes, hash routes, pending-state transcript replay, live missing tabs versus explicit URL destinations, close retirement, and caller-owned/attached/restore-disabled boundaries. Those fake-upstream checks do not replace native-browser or real Pi lifecycle qualification of the complete candidate.
|
|
272
275
|
- **Failure shape:** `react tree` on a page opened with `--enable react-devtools` but without a React app (expects a clear missing-renderer error with session-bound `details`).
|
|
273
276
|
- **Async download:** `open` on the `/download` fixture, anchor-triggered export, then `wait --download <path>` metadata and wrapper artifact reporting for the requested path.
|
|
274
277
|
|
|
275
278
|
The default unit suite also runs `agentBrowserExtension passes through core command coverage fallback matrix` in [`test/agent-browser.extension-passthrough-validation.test.ts`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/test/agent-browser.extension-passthrough-validation.test.ts): a fake upstream records argv so explicit `--session connector connect 9222`, plus `download` with a selector and path, `get url`, `snapshot --compact`, and `tab new` / `tab t1` / `tab close` on implicit managed sessions, still prove `--json` and session ordering without a browser. A second fake-upstream matrix in that file (`agentBrowserExtension passes through non-core network debug diff stream dashboard and chat families`) pins representative `network`, `diff`, `trace` / `profiler` / `record`, `console` / `errors` / `highlight` / `inspect` / `clipboard`, `stream`, `dashboard`, and `chat` JSON shapes plus redacted `details.data` and argv echoes without a browser. A third matrix (`agentBrowserExtension passes through provider and specialized skill workflows`) asserts provider `open` argv shapes still receive `--json` plus implicit `--session` while read-only `skills get …` stays stateless (no managed session fields) and provider credential env vars are forwarded into the fake upstream log. Extend those matrices when adding passthrough coverage that should stay out of the slow real-upstream loop.
|
|
276
279
|
|
|
280
|
+
### Native Linux socket-root regression
|
|
281
|
+
|
|
282
|
+
`test/agent-browser.socket-namespace.test.ts` is opt-in (`PI_AGENT_BROWSER_SOCKET_NAMESPACE=1`). Run it with the installed `tsx` test runner inside a disposable bubblewrap user/mount namespace: a read-only mode-`0755` `/` whose owner is unmapped, and a current-user-owned mode-`0700` tmpfs at `/tmp`. Use a cleared environment with only the required executable PATH, private HOME and the opt-in variable. The test verifies those identities, exchanges data over a real Unix socket, dispatches native `agent-browser --version` through the shared subprocess wrapper, and inspects a disposable managed session with automatic restore enabled. It does not launch a browser or call a model.
|
|
283
|
+
|
|
284
|
+
For a root-check change, run the identical final test against both the pre-fix and rebuilt implementation: the old code must fail the socket validation assertion after the native socket exchange succeeds, and the new code must pass without filesystem mocks or runtime patch hooks. Also retain process/restore negatives for foreign or writable non-root ancestry, leaf symlinks, planted entries and unsafe alias destinations, plus existing root-owned sticky-mode coverage. Namespace setup failures are environment blockers, not test passes; do not silently widen container privileges or host policy. This qualified private-`/tmp` layout does not imply support for unmapped `/home` ancestry or replace the broader release gates.
|
|
285
|
+
|
|
277
286
|
### Real upstream suite mechanics, isolation, and troubleshooting
|
|
278
287
|
|
|
279
|
-
- **
|
|
288
|
+
- **Focused and broad cases:** `test/agent-browser.real-upstream-contract.test.ts` keeps the broad command matrix in one 180-second case and separate 60-second cold-reopen, duplicate-name click-mutation, and sessionless-plugin cases. The `real-upstream` facade's `contract suite matches` selection includes the broad matrix, cold reopen, and click mutation. The click case uses two native DOM buttons to prove a stale duplicate ordinal cannot contradict trusted target events, while exact XPath probes still detect missing events and native clicks remain dispatch-only evidence.
|
|
280
289
|
- **Output-shape locking:** Expected `details` / `data` keys per step live in `test/fixtures/agent-browser-real-output-shapes.json`, keyed by logical groups (`version`, `rootHelp`, `commandHelp`, `skillsList`, `skillsGetFull`, `skillsPath`, `open`, `eval`, `snapshot`, `coreCommand`, `coreSubcommand`, `coreFileArtifact`, `batch`, `pushstate`, `vitals`, `networkRoute`, `nonCoreStatus`, `nonCoreArtifact`, `diffScreenshotArtifact`, `streamControl`, `streamStatus`, `cookiesCurl`, `reactMissingRenderer`, `waitDownload`). Keep `targetVersion` in that file aligned with `scripts/agent-browser-capability-baseline.mjs`, and extend entries whenever the suite starts asserting on new presentation fields.
|
|
281
290
|
- **Isolation:** The harness allocates a throwaway directory under the system temp folder, points `HOME`, `AGENT_BROWSER_SOCKET_DIR`, and `AGENT_BROWSER_SCREENSHOT_DIR` at that tree, serves HTML fixtures from loopback (`startAgentBrowserContractFixtureServer` in `test/helpers/agent-browser-harness.ts`), and closes the managed session before deleting the temp tree. The main matrix does not reuse your normal profile or socket locations.
|
|
282
291
|
- **React DevTools branch:** After the core matrix, the suite performs another `open` with `--enable react-devtools` and `sessionMode: "fresh"`, then expects `react tree` to fail with a missing-renderer style error on the same non-React contract page. The following download fixture + `wait --download` assertions run against whichever managed session is current after that fresh `open` (typically the React DevTools session), not the original pre-matrix session name.
|
|
@@ -285,7 +294,7 @@ The default unit suite also runs `agentBrowserExtension passes through core comm
|
|
|
285
294
|
|
|
286
295
|
- **Version mismatch:** Install a stable `agent-browser` at or above the configured floor (prefer the recommended capability baseline), or follow the maintainer rebaselining sequence in `AGENTS.md` if you intentionally move either version.
|
|
287
296
|
- **Missing or extra `details` / `data` keys:** Update `test/fixtures/agent-browser-real-output-shapes.json` in the same change as the wrapper or presentation code that shifts those keys.
|
|
288
|
-
- **Timeouts:**
|
|
297
|
+
- **Timeouts:** The broad matrix has a 180-second bound; the focused cold-reopen, click-mutation, and plugin cases each have a 60-second bound. Repeated timeouts usually mean a hung browser, blocked loopback, or an environment preventing headful/headless launch—check upstream logs and local security tooling before loosening timeouts.
|
|
289
298
|
|
|
290
299
|
The upstream `agent-browser` `wait --download <path>` saveAs persistence limitation is tracked at [vercel-labs/agent-browser#1300](https://github.com/vercel-labs/agent-browser/issues/1300); until it is fixed, release validation must treat `details.savedFilePath` as upstream-reported metadata and use `details.artifacts[].exists` as the filesystem truth (the contract asserts the requested path is absent on disk while upstream still reports success). If the suite fails because JSON/detail keys drifted, update the wrapper behavior or refresh `test/fixtures/agent-browser-real-output-shapes.json` together with the presentation work that consumes those shapes.
|
|
291
300
|
|
|
@@ -299,7 +308,7 @@ Recommended configured-source lifecycle follow-up:
|
|
|
299
308
|
|
|
300
309
|
1. Open a page with the implicit managed session and confirm the title.
|
|
301
310
|
2. Run `/reload`, then ask for `snapshot -i` and confirm the same page is still active.
|
|
302
|
-
3. Exit `pi`, relaunch
|
|
311
|
+
3. Exit `pi`, confirm the old managed daemon stopped, relaunch against the same exact session id/path or use `/resume`, then ask for `snapshot -i` before any explicit navigation. Confirm the same recorded URL was reopened, its observed page is correct, and refs are fresh. Cover empty storage and origin storage at non-root paths; do not expect unsaved forms, JavaScript memory, or history to survive.
|
|
303
312
|
4. Open a large page that compacts its snapshot output and confirm `details.fullOutputPath` still exists after the restart/resume/exact-session flow.
|
|
304
313
|
5. Trigger an oversized non-snapshot output (for example a deliberately large `eval --stdin` result) and confirm the tool prints the actual spill file path directly in content instead of only referencing a details key.
|
|
305
314
|
6. Validate at least one direct file-download flow with `download <selector> <path>`.
|
package/docs/REQUIREMENTS.md
CHANGED
|
@@ -125,7 +125,7 @@ The design should comfortably support workflows such as:
|
|
|
125
125
|
- Keep bundled-skill coexistence simple; do not add extra moving parts unless observed behavior justifies them.
|
|
126
126
|
- Prefer narrow, evidence-backed compatibility mitigations over broad stealth layers when a specific upstream site starts rejecting the default headless launch fingerprint.
|
|
127
127
|
- Preserve the page that a profiled `open` just navigated to; if restored profile tabs steal focus during launch, the wrapper should best-effort switch back to the returned page URL before handing control back to the agent.
|
|
128
|
-
- Once a tab target is known for a session, later active-tab commands should best-effort
|
|
128
|
+
- Once a tab target is known for a session, later active-tab commands should best-effort verify and select that tab under the existing session queue before dispatch, preserving caller argv/stdin, when reconnect drift would otherwise land on a restored/background tab.
|
|
129
129
|
- If a restored/background tab steals focus after a successful command, the wrapper should best-effort restore the intended target tab again before handing control back.
|
|
130
130
|
- On local Unix launches, extension-generated session names should not fail just because the upstream default socket path is too long; the wrapper should choose a shorter socket directory when needed.
|
|
131
131
|
- Provider selection flags (`-p`, `--provider`) and provider device flags (`--device`) are launch-scoped like profile, CDP, persisted state, and upstream's `--webgpu` launch preset: if an extension-managed implicit session is already active, the planner must fail fast with the same recovery guidance as other startup-scoped flags instead of silently forwarding argv upstream would ignore; contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sessionmode) and session model in [`ARCHITECTURE.md`](ARCHITECTURE.md).
|
package/docs/SUPPORT_MATRIX.md
CHANGED
|
@@ -59,7 +59,7 @@ Current summary:
|
|
|
59
59
|
| RQ-0141 | Headed managed sessions avoid upstream 0.33.2 visible autosave targets by default and retain that daemon environment across helpers, still-owned off-current follow-ups, failed-replacement ownership, Electron cleanup, and reload/resume; slow daemon-policy inspection uses a 35-second budget; headed mode is launch-scoped; the effective launch interval persists across resume and changes in either direction require close plus a fresh daemon; and bare, review-only, fenced-reference, conditional, directly negated, or reference attachment paths do not become close-blocking output requirements while explicit contiguous output path lists, per-clause availability, duplicate precedence, and Markdown destinations are handled conservatively and linearly. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.managed-session-daemon-policy.test.ts`, `test/agent-browser.process.test.ts`, `test/agent-browser.prompt-policy.test.ts` |
|
|
60
60
|
| RQ-0142 | Top-level `script` provides bounded one-shot JavaScript loops, branches, and multi-page aggregation through the complete ordinary browser executor. A permissioned child exposes only null-prototype `browser` / `emit` task functions, receives no user-visible host capabilities, serializes at most 25 calls, and uses a unique restore-disabled wrapper session with a durable pre-spawn Pi lease, finally-close, active branch/shutdown abort-and-wait cleanup (including upstream-version preflight), and exact lease recovery. Script-owned helpers/cleanup case-insensitively clear ambient upstream launch/profile/restore/attachment and proxy controls; final data is compact-serialized with a second post-redaction byte/depth guard; rehydrated spills reserve IPC-envelope headroom, inner summary/text are bounded, complete response envelopes are size-checked, and script-visible next actions are policy-compatible after isolated identity removal; malformed bridge calls and missing workers fail structurally; policy-rejection and dispatched-failure counters are disjoint. The collapsed Pi call row shows a bounded terminal-safe source preview with visible line-break markers and its expanded state shows the full terminal-safe source with JavaScript line terminators preserved and removed controls marked. It requires persisted Pi sessions, classifies uncaught source exceptions as `script-error`, confirms successful cleanup in compact prose, and deliberately has no named recipe/import/shared-state surface. The same dogfood pass compacted common/QA output, made failure `nextActions` model-visible and exact-namespace/session-stable (including explicit empty namespace overrides), limited close cleanup guidance to existing explicit artifacts, clarified raw batch stdin, normalized empty semantic names, and added bounded timeout/navigation recovery actions. | [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#script), [`docs/ARCHITECTURE.md`](ARCHITECTURE.md#one-shot-script-isolation), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#one-shot-code-mode), `test/agent-browser.script.test.ts` |
|
|
61
61
|
| RQ-0143 | Browser-backed calls require a stable upstream version at or above the configured 0.35.0 floor, report minimum/observed version mismatch before launch, and keep stateless/local diagnostics available; the recommended target and minimum are shared by runtime, docs verification, and package contents. | [`scripts/agent-browser-target.mjs`](../scripts/agent-browser-target.mjs), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0360-rebaseline), `test/agent-browser.upstream-version.test.ts` |
|
|
62
|
-
| RQ-0144 | Managed restore keys are scoped to both checkout generation and Pi transcript so upstream 0.33.2's newest-file-wins loading cannot let concurrent chats clobber or inherit cookies/storage. Fresh rotations, reload, restart, and `/resume` of one transcript retain continuity;
|
|
62
|
+
| RQ-0144 | Managed restore keys are scoped to both checkout generation and Pi transcript so upstream 0.33.2's newest-file-wins loading cannot let concurrent chats clobber or inherit cookies/storage. Fresh rotations, reload, restart, and `/resume` of one transcript retain continuity; after a confirmed inactive daemon, a branch-restored compatible owned session retains its pending reopen through daemon-starting non-page calls and transcript replay, then reopens the complete recorded URL (including its fragment) before current-page operations such as `get url` or `reload`, verifies the actual target and invalidates old refs. This reloads a URL, not forms, JavaScript memory, or history; live wrong-tab recovery never auto-navigates. Passive and explicit upstream configs pass through, while explicit config selection suppresses the wrapper's automatic restore injection. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.managed-session-restore.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
|
|
63
63
|
| RQ-0145 | Mega-dogfood correctness closes false-positive success paths: URL QA subtracts only unchanged post-clear page-error residue and waits 150 ms for immediate post-load diagnostics, stale pre-existing artifacts and missing/stale recordings finalized by restart fail verification, aliased destinations (including dangling symlinks, full macOS/Windows Unicode folds, final short/long diff output flags, and same-call `outputPath` aliases) and active recording-path reuse through artifacts, path-bearing HAR stops, long/short or reordered explicit download waits, interspersed-global command forms, Electron cleanup output, or normalized output writes is rejected through an atomic process-wide, transcript-backed namespace/session reservation index independent of bounded manifest eviction; cross-branch close tombstones survive reload and each direct or ordered nested close retires exact identity at that lifecycle point while terminal closes clear attached/ref/page/route state, later lifecycle-proven browser launches—including failed post-close rows and `record stop`—can reactivate/rebuild it without stale pre-close `about:blank` recovery while bounded launch evidence survives failed-step presentation and transcript replay, explicitly non-launching diagnostics preserve the close, missing evidence stays conservatively active even on the first managed call, every successful direct/nested close clears wrapper trace/profiler ownership before later successful rows can rebuild it, and namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, only the newest pending recording path remains authoritative even across same-timestamp restart rows, current recording transitions replay directly without artifact-manifest compatibility inference, definitive direct/batch no-recording stop failures retire stale state at their ordered step, recording starts after nested close are rejected, pending rows coalesce into terminal batch outcomes without retaining an intermediate close-abandoned duplicate after a later saved stop, and any later same-session failure retains exact cleanup actions, same-page getter refs are refreshed, locator select resolves exact current native controls, no-movement scroll fails honestly, and script cleanup always attempts fail-closed close. | [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md), `test/agent-browser.extension-input-modes.test.ts`, `test/agent-browser.extension-ref-guards.test.ts`, `test/agent-browser.presentation.test.ts`, `test/agent-browser.presentation-artifacts-batch.test.ts`, `test/agent-browser.recording-reservations.test.ts`, `test/agent-browser.extension-validation.test.ts`, `test/agent-browser.script.test.ts` |
|
|
64
64
|
| RQ-0146 | Upstream `agent-browser 0.34.0` rebaseline adds sticky `--pin-tab` / `--no-pin-tab` (`AGENT_BROWSER_PIN_TAB`) for shared CDP/auto-connect sessions, CDP target ids as tab refs, and `tab_gone` recovery data (`data.targetId`, optional `data.lastUrl`). The wrapper parses those flags as optional global booleans rather than launch-scoped, classifies `tab_gone` as `failureCategory: "tab-gone"` with `list-tabs-after-tab-gone` / `open-tab-after-tab-gone`, and shows `targetId` in `tab list` presentation. | [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0340-rebaseline), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), `test/agent-browser.results.test.ts`, `test/agent-browser.runtime.test.ts` |
|
|
65
65
|
| RQ-0147 | Native follow-ups preserve profiled and other launch-configured browser pages by omitting wrapper compatibility launch arguments while the daemon is active; daemon inspection reapplies the retained compatibility user agent only when relaunch is required. Caller `--args`, config, file access, and local navigation pass through unchanged, and cancellation wins over policy validation before spawn. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.runtime.test.ts`, `test/agent-browser.process.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
|
|
@@ -69,6 +69,16 @@ Current summary:
|
|
|
69
69
|
| RQ-0151 | Upstream 0.36.0 adds experimental WebMCP page tools, default local-Chrome enablement with launch-scoped `--no-webmcp`, an opt-in MCP profile, and the `webmcp-gen` skill. The wrapper passes list/invoke/result/cancel through, preserves params/frame/detach/timeout arguments, derives watchdogs from effective direct/raw/stdin commands, keeps pending detached targets unverified, and restores batch target/ref state only after URL verification and a fresh snapshot, while keeping 0.35.0 as the stable floor. | [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0360-rebaseline), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#experimental-webmcp), `test/agent-browser.runtime.test.ts`, `test/agent-browser.wait-timeouts.test.ts`, `test/agent-browser.process.test.ts`, `test/agent-browser.page-target-validation.test.ts`, `test/agent-browser.extension-ref-guards.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
|
|
70
70
|
| RQ-0152 | Agent feedback closes recovery and evidence gaps without a new runtime layer: unknown-target timeouts return a visibly executable `get url` → `snapshot -i` fail-fast action while dialog recovery remains allowed; successful tab selection/close live-verifies URL/fresh titles and retains deliberate and post-close blank tabs; compacted direct or batched `outputPath` writes use only full command-redacted live wrapper spills and fail rather than save metadata; direct reads expose visible and structured CLI/browser/source lifecycle; direct or batched first/fresh local headed launches expose an unverified `browserWindow` handoff while attachments do not; snapshot spills and SAML/OAuth URL values are redacted at model/persistence boundaries; unsupported direct/batch `scrollintoview text=...` fails before upstream false success with visible native recovery payloads while help passes through; and web search sends Exa primary/version guidance, removes exact normalized-URL duplicates without overfetch, and labels provider page dates without claiming crawl/version proof. | [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md), [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), `test/agent-browser.extension-errors-artifacts.test.ts`, `test/agent-browser.extension-tabs.test.ts`, `test/agent-browser.extension-validation.test.ts`, `test/agent-browser.page-target-validation.test.ts`, `test/agent-browser.runtime.test.ts`, `test/agent-browser.snapshot-presentation.test.ts`, `test/agent-browser.web-search.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
|
|
71
71
|
|
|
72
|
+
Contributor fixes #133/#152 remove unused prompt suffix entries without changing runtime guidance and diagnose bare `--no-sandbox` only in the command slot or navigation option positions. Native `--args` values and literal operands remain intact; batch checks use raw effective rows without treating row-local `--args` as a launch setting. `test/agent-browser.chromium-args.test.ts` covers pre-dispatch rejection, literal/flag-value controls, inspection, raw/stdin batch precedence and dispatcher outcomes. Existing prompt/grammar checks remain; source checks do not replace native-product gates.
|
|
73
|
+
|
|
74
|
+
Electron diagnostics (RQ-0096, #128) keep list timeout rejection truthful, label explicit-ID cleaned records as historical without changing active selection or actions, and measure the tracked profile path with native `lstat` (`present` / ENOENT-only `absent` / `unknown`). Existing Electron discovery/lifecycle tests cover current liveness independently of cleanup history, native path errors and dangling symlinks, transcript replay, and unchanged cleanup ownership. Failed-launch stdout/stderr capture remains unresolved; #128 is partial, and these source checks do not replace native-app, Pi lifecycle, package, or live-site gates.
|
|
75
|
+
|
|
76
|
+
Artifact diagnostics (#124/#127) use a shared pre-dispatch mkdir-error boundary, preserve raw batch argv/precedence, recommend absolute raw artifact paths, recognize image headers rather than filename MIME guesses, retain known requested/reported paths, and warn once for dispatched recording page transitions on success or failure. `test/agent-browser.artifact-diagnostics.test.ts` covers registered filesystem failures, real image bytes and misleading suffixes, the inline bound, native macOS path aliases, recording/ref continuity and unreached-row negatives; `test/agent-browser.presentation-artifacts-batch.test.ts` retains artifact/persistence coverage. These source regressions do not qualify daemon-cwd differences or affected-filesystem timestamp behavior (#118), or replace native/Pi/package/live-site gates.
|
|
77
|
+
|
|
78
|
+
Cold URL reopen is covered by `test/agent-browser.cold-resume.test.ts`, the daemon/page-boundary regressions in `test/agent-browser.cold-boundaries.test.ts`, and the opt-in `contract suite matches cold URL reopen after quit` case in `test/agent-browser.real-upstream-contract.test.ts`. Deterministic boundary tests cover non-page prefixes, fragment routes, history commands, explicit destinations, transcript replay, unreached navigation and unchanged native batch/error order. `test/agent-browser.destination-cancel.test.ts` covers direct/native-batch new-window and URL-diff follow-ups, redirected/unknown targets (including blank redirects with a remembered duplicate tab), ordered ref invalidation, reached versus unreached rows, and cancellation before or after a cold-reopen attempt with replay and process cleanup. Release acceptance requires the first page operation after confirmed shutdown to observe the remembered non-root URL with both empty and origin storage, including after non-page calls and on hash-routed pages, before any explicit open, with refreshed refs. Focused native Mac evidence does not replace actual Ubuntu, real Pi lifecycle, checkout, live-site, or final release gates.
|
|
79
|
+
|
|
80
|
+
Batch/ref fidelity (#122/#123/#125/#126): tab recovery uses verified native selection under existing session serialization without reconstructing argv/stdin or changing caller batch control flow. Selector-only stale-ref checks cover `@eN`, `eN`, and `ref=eN`; literal operands and keyboard/mouse data remain native. `test/agent-browser.batch-fidelity.test.ts` runs paired pinned/unpinned real-upstream fixtures, including mixed failures, header/timeout flags, ignored stdin, current/stale refs, failed/wrong-target selection, sessionless commands, and explicit connection/state recovery with failed-batch negatives. It runs in `npm run verify -- real-upstream`; deterministic selector/shape checks also run in the default suite.
|
|
81
|
+
|
|
72
82
|
## Verification evidence
|
|
73
83
|
|
|
74
84
|
Re-run the gates below before each release; this table records what the closure audit exercised. The recommended 0.36.0 rebaseline passed the local pre-PR, real-upstream, and deterministic dogfood gates on 2026-09-01. The 0.35.0 runtime floor was last validated locally on 2026-08-26; the full platform matrix remains setup-blocked by disabled macOS SSH and missing Parallels `prlctl`.
|
|
@@ -98,7 +108,7 @@ Runtime floor note: package metadata keeps optional Pi core package peer ranges
|
|
|
98
108
|
| Core page, element, navigation, and extraction commands | 82 canonical tokens from baseline section `core-commands`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-page-and-element-commands). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-page-and-element-commands), [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md), README quick start. | Thin passthrough with wrapper-owned JSON/session planning, ref guidance, artifact verification, page-change summaries, click-dispatch diagnostics, deterministic document scroll before wheel/no-op diagnostics, shorthand compilers, and redaction. | Real-upstream core matrix (including snapshot-ref select) plus fake core matrix for passthrough, ordering, diagnostics, and compiler validation. | Supported. Upstream semantics remain upstream-owned. |
|
|
99
109
|
| Sessions, state, tabs, frames, dialogs, and windows | 28 canonical tokens from baseline section `state-tabs-frames-dialogs`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#session-state-frames-dialogs-windows-and-inspection-commands). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#session-state-frames-dialogs-windows-and-inspection-commands), stateful workflow notes, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). | Stateful summaries redact credentials while preserving restore identifiers and every session/state list row; explicit targets and paths pass through. Runtime retains state artifact verification, sessionless local command planning, automatic managed restore, tab target pinning, active-target refresh after tab close, and close alias cleanup. | Extension tab/ref tests, real-upstream stable-id/label tab lifecycle, runtime session/resume tests, presentation redaction tests, lifecycle harness. | Supported. External profile/auth state remains operator-owned. |
|
|
100
110
|
| Network, storage, artifacts, diagnostics, and performance | 57 canonical tokens from baseline section `network-storage-artifacts-diagnostics`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#page-state-finding-mouse-settings-network-and-storage). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#page-state-finding-mouse-settings-network-and-storage), diagnostic sections, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). | Thin passthrough plus compact diagnostics, route-mock warnings, useful-but-redacted storage output, stream idempotency normalization, artifact metadata, missing-ffmpeg warnings, sensitive-data redaction, timeout bounds, and cleanup-pair guidance. | Fake non-core matrix and safe real-upstream coverage for network/HAR, diff, trace/profiler, console/errors/highlight, stream, vitals, and React missing-renderer. | Supported. Environment-sensitive operations need suitable local/browser state. |
|
|
101
|
-
| Batch, auth, confirmations, setup, dashboard, devices, and AI commands | 36 canonical tokens from baseline section `batch-auth-setup-ai`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#batch-auth-confirmations-sessions-chat-dashboard-devices-and-setup). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#batch-auth-confirmations-sessions-chat-dashboard-devices-and-setup), README security notes, release docs. | Native-tool batch stdin, generated `job`/`qa`/lookup batch plans, auth/confirmation redaction, sessionless local auth/setup/dashboard/doctor/plugin planning, plugin list/show JSON envelope normalization, bare-`mcp` validation with `mcp --help` preserved, timeout/cleanup guidance. | Parser/runtime plugin and MCP unit coverage; fake-upstream plugin list/show and MCP help/blocking coverage; real-upstream plugin list shape probe; structured input-mode tests. | Supported. Interactive side-effecting setup/auth/chat remains upstream-owned. `plugin` is local/sessionless; `mcp` is external-client-only except help; `auth login --credential-provider` resolves credentials via a plugin; `install --with-deps` failures remain upstream-owned. |
|
|
111
|
+
| Batch, auth, confirmations, setup, dashboard, devices, and AI commands | 36 canonical tokens from baseline section `batch-auth-setup-ai`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#batch-auth-confirmations-sessions-chat-dashboard-devices-and-setup). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#batch-auth-confirmations-sessions-chat-dashboard-devices-and-setup), README security notes, release docs. | Native-tool batch stdin, generated `job`/`qa`/lookup batch plans, auth/confirmation redaction, sessionless local auth/setup/dashboard/doctor/plugin planning, upgrade-only native text normalization with error precedence and explicit-JSON results, plugin list/show JSON envelope normalization, bare-`mcp` validation with `mcp --help` preserved, timeout/cleanup guidance. | Parser/runtime plugin and MCP unit coverage; fake-upstream plugin list/show and MCP help/blocking coverage; registered upgrade text/success/nonzero/timeout/abort/JSON controls in `test/agent-browser.upgrade-output.test.ts`; real-upstream plugin list shape probe; structured input-mode tests. | Supported. Interactive side-effecting setup/auth/chat remains upstream-owned. `plugin` is local/sessionless; `mcp` is external-client-only except help; `auth login --credential-provider` resolves credentials via a plugin; `install --with-deps` failures remain upstream-owned. |
|
|
102
112
|
| Global flags, config, providers, policy, and environment | 152 canonical tokens from baseline section `options-and-env`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#important-global-flags-config-and-environment). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#important-global-flags-config-and-environment), README provider/setup notes, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sessionmode), architecture/runtime docs. | Runtime handles command discovery, value-flag prevalidation, launch-scoped flags, redacted echoes, fresh-session recovery hints, explicit sessions, provider/device launch-scoping, parent env forwarding with consistent managed idle-timeout/socket/operation-timeout overrides on top-level and helper subprocesses, subprocess completion, and package-owned Pi-scoped config for optional companion features. | Runtime tests for flags/planning/redaction/session behavior; process/ref tests for env consistency and stdio-linger completion; config/web-search/CLI tests; fake provider/specialized-skill matrix; package doctor. | Supported. Provider clouds, iOS/Appium, proxies, profiles, and credentials require external setup. |
|
|
103
113
|
|
|
104
114
|
## Follow-up decision after closure
|
|
@@ -109,6 +119,6 @@ The compact decision index lives at [`docs/support-notes.md`](https://github.com
|
|
|
109
119
|
| --- | --- | --- |
|
|
110
120
|
| Native structured input modes (`script`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, `semanticAction`) | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md), [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md) | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
|
|
111
121
|
| Electron lifecycle, manual CDP attach, desktop readiness, and tab/surface recovery | [`ELECTRON.md`](ELECTRON.md), [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron), [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#electron-desktop-apps) | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
|
|
112
|
-
| Ref lifecycle, click dispatch, selector recovery, rich inputs, and dense snapshots | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#selector-strategy), README pitfalls | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
|
|
122
|
+
| Ref lifecycle, click dispatch (unique role/name or XPath targets only; native duplicate-name mutation regression), selector recovery, rich inputs, and dense snapshots | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#selector-strategy), README pitfalls | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
|
|
113
123
|
| Diagnostics, artifacts, QA/network classification, timeout recovery, scroll/combobox/recording guidance | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md), [`RELEASE.md`](RELEASE.md) | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
|
|
114
124
|
| Package config and optional web search, including the default Exa mode and bounded filters | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#optional-companion-web-search), README optional package config, [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#optional-package-config-and-companion-web-search) | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
|