pi-agent-browser-native 0.9.4 → 0.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +29 -0
- package/README.md +58 -813
- package/dist/extensions/agent-browser/index.js +927 -883
- package/dist/extensions/agent-browser/lib/browser-execution-claims.js +5 -5
- package/dist/extensions/agent-browser/lib/config-policy.js +1 -1
- package/dist/extensions/agent-browser/lib/config-validation.js +21 -8
- package/dist/extensions/agent-browser/lib/config.js +1 -1
- package/dist/extensions/agent-browser/lib/electron/cdp.js +5 -1
- package/dist/extensions/agent-browser/lib/electron/cleanup.js +13 -47
- package/dist/extensions/agent-browser/lib/electron/discovery-evidence.js +1 -0
- package/dist/extensions/agent-browser/lib/electron/launch.js +12 -1
- package/dist/extensions/agent-browser/lib/fs-utils.js +1 -9
- package/dist/extensions/agent-browser/lib/input-modes/job.js +7 -4
- package/dist/extensions/agent-browser/lib/input-modes/lookup-network-evidence.js +15 -3
- package/dist/extensions/agent-browser/lib/input-modes/lookups.js +3 -2
- package/dist/extensions/agent-browser/lib/input-modes/params.js +23 -23
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/electron-observation.js +9 -8
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-electron.js +8 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/scroll-shims.js +2 -2
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +9 -0
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-electron.js +4 -2
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-refs.js +9 -11
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/branch.js +1 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-collection.js +5 -6
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-data.js +4 -2
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-presentation.js +20 -2
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/status.js +4 -2
- package/dist/extensions/agent-browser/lib/orchestration/execution-queue.js +1 -1
- package/dist/extensions/agent-browser/lib/results/presentation/skills.js +7 -2
- package/dist/extensions/agent-browser/lib/temp-artifact-budget.js +4 -1
- package/dist/extensions/agent-browser/lib/temp-root.js +22 -18
- package/dist/extensions/agent-browser/lib/temp.js +1 -1
- package/dist/extensions/agent-browser/lib/web-search-providers.js +0 -36
- package/dist/extensions/agent-browser/lib/web-search-request.js +20 -26
- package/dist/extensions/agent-browser/lib/web-search-schema.js +5 -5
- package/dist/extensions/agent-browser/lib/web-search.js +22 -4
- package/docs/ARCHITECTURE.md +3 -2
- package/docs/CODE_QUALITY.md +28 -3
- package/docs/COMMAND_REFERENCE.md +20 -20
- package/docs/ELECTRON.md +24 -2
- package/docs/REFERENCE.md +869 -0
- package/docs/RELEASE.md +9 -9
- package/docs/REQUIREMENTS.md +1 -1
- package/docs/SUPPORT_MATRIX.md +9 -8
- package/docs/TOOL_CONTRACT.md +13 -9
- package/docs/platform-smoke.md +3 -3
- package/package.json +3 -2
- package/scripts/config.mjs +23 -19
- package/scripts/doctor.mjs +2 -2
- package/scripts/platform-smoke/lease-evidence.mjs +2 -4
- package/scripts/platform-smoke/linux-image/Dockerfile +8 -2
- package/scripts/platform-smoke/targets.mjs +1 -5
package/docs/RELEASE.md
CHANGED
|
@@ -38,7 +38,7 @@ The 0.7 tool contract is breaking: direct argv stays compact, `script` moves to
|
|
|
38
38
|
|
|
39
39
|
Before activating the update, finish or inspect in-flight mutations/recordings and restart every cooperating Pi process onto the same package. `/reload` alone is not reliable for picking up rebuilt compiled code. Mixed versions do not share the new execution-ordering guarantee.
|
|
40
40
|
|
|
41
|
-
Rollback is package-level:
|
|
41
|
+
Rollback is package-level: restore the previously installed 0.6 package from your saved local copy, then fully restart participating Pi processes with their existing session files. Do not delete auth/profile/state files, artifacts, or transcripts. The older package cannot interpret all new inner-transition records; inspect the live URL and obtain fresh refs before further actions. Rollback does not reverse submitted browser actions, stop external clients, or restore JavaScript variables.
|
|
42
42
|
|
|
43
43
|
### Changed-path qualification
|
|
44
44
|
|
|
@@ -91,7 +91,7 @@ npm run build
|
|
|
91
91
|
npm run verify -- startup-profile --samples 3
|
|
92
92
|
```
|
|
93
93
|
|
|
94
|
-
The profiler first clean-builds `dist/`, then records direct package entrypoint import/factory timing in fresh Node processes and writes `.artifacts/startup-profile/latest.json` before reporting any budget failure. Failed reports retain every sample and a nonzero exit. Per-sample CPU, resource, hardware, load and artifact-identity receipts follow the [startup evidence contract](
|
|
94
|
+
The profiler first clean-builds `dist/`, then records direct package entrypoint import/factory timing in fresh Node processes and writes `.artifacts/startup-profile/latest.json` before reporting any budget failure. Failed reports retain every sample and a nonzero exit. Per-sample CPU, resource, hardware, load and artifact-identity receipts follow the [startup evidence contract](REFERENCE.md#proof-and-verification); their diagnostic collection does not move the existing wall-clock boundaries, prewarm the extension, or change the strict 250 ms maximum over three fresh regression samples. The report includes a safety block confirming it did not launch Pi, tmux, mise, npm, browsers, or `agent-browser`. Full Pi TUI ready-prompt profiling is intentionally excluded because repeated real Pi/tmux launches proved too invasive for routine verification on the operator machine.
|
|
95
95
|
|
|
96
96
|
For a deterministic host-only real-browser wrapper smoke without model choice in the loop, run:
|
|
97
97
|
|
|
@@ -110,7 +110,7 @@ The Crabbox gate is only green when suite assertions and artifact manifests unde
|
|
|
110
110
|
|
|
111
111
|
The deterministic dogfood mode clean-builds the compiled package, uses the extension harness and real `agent-browser` on `PATH` against a deterministic loopback HTTP fixture, then verifies persistent browser code aggregation, flat advanced QA/actions, native batch, screenshot artifact verification, and explicit session close. Use `npm run verify -- dogfood --keep-artifacts` or `--artifact-dir <path>` only while debugging, then delete retained screenshots. This smoke complements, but does not replace, human-readable interactive transcript evidence.
|
|
112
112
|
|
|
113
|
-
Every release also requires interactive `tmux`-driven Pi dogfood with the native `agent_browser` tool against real sites. For extension-focused release smokes, use `pi --approve --no-extensions --no-skills -e .` from the trusted checkout before publish so auto-loaded dogfood/QA skills cannot replace the bounded smoke workflow; omit `--approve` only when the smoke is explicitly testing Pi's Project Trust prompt. Run separate skill-enabled dogfood only when validating skill routing or report-generation behavior. Drive prompts with `tmux send-keys`, exercise at least one simple static site and one real documentation/product site, include advanced QA, code, or native batch when those surfaces changed, close every opened browser session, remove screenshots/temp artifacts, and record the outcome in the release notes or support-matrix evidence. Do not paste raw multi-line prompts into a tmux Pi pane: plain newlines submit separate queued user messages. For scripted smoke driving, collapse prompt files to one line before sending (`PROMPT=$(tr '\n' ' ' < /tmp/smoke-prompt.md); tmux send-keys -t "$SESSION":0.0 -l "$PROMPT"; tmux send-keys -t "$SESSION":0.0 Enter`). For manual multi-line editing, use Pi's external editor shortcut (`Ctrl+G`) or configure tmux extended keys so Pi can receive `Shift+Enter` for newlines; see the installed Pi `docs/tmux.md` guidance. Automated localhost, fake-upstream, and deterministic dogfood gates do not replace this human-readable live-site transcript evidence. When code changes, add a persisted-session `agent_browser_code` pass that aggregates at least two real pages, exercises a conditional branch, compares selected output with an ordinary-call baseline, verifies selected screenshot bytes reach the model, and checks browser/auth continuity on a later direct/code call with fresh JS globals. Review ordered custom transition entries and interrupted/replayed state, not only the outer result. Exercise handled failures, timeout/cancellation, whole-cell ordering, and child reaping; close only test-owned browsers afterward and retain Pi transcripts. Never use `--no-session` for code dogfood. When `agent_browser_web_search` or package config changed, add one key-free smoke proving the optional tool is absent without config, one fake/unit-backed smoke in the default suite, and one opt-in live Exa or Brave Search check with a real key while confirming the key does not appear in transcripts, stdout/stderr, config status, PR text, or artifacts. When `electron.*` surfaces, attached-session diagnostics, or `qa.attached` changed, add a local Electron pass: `electron.list` → `electron.launch` (expect isolated profile behavior) → `snapshot -i` or `electron.probe` / `qa.attached` → `electron.cleanup` with the returned `launchId`, verifying status/mismatch guidance if you simulate a dead renderer or stale refs. For dense-dashboard stress coverage, use the [public Grafana stress checklist](#public-grafana-stress-checklist) below; it is a maintainer workflow, not bundled product skill or recipe runtime.
|
|
113
|
+
Every release also requires interactive `tmux`-driven Pi dogfood with the native `agent_browser` tool against real sites. For extension-focused release smokes, use `pi --approve --no-extensions --no-skills -e .` from the trusted checkout before publish so auto-loaded dogfood/QA skills cannot replace the bounded smoke workflow; omit `--approve` only when the smoke is explicitly testing Pi's Project Trust prompt. Run separate skill-enabled dogfood only when validating skill routing or report-generation behavior. Drive prompts with `tmux send-keys`, exercise at least one simple static site and one real documentation/product site, include advanced QA, code, or native batch when those surfaces changed, close every opened browser session, remove screenshots/temp artifacts, and record the outcome in the release notes or support-matrix evidence. Do not paste raw multi-line prompts into a tmux Pi pane: plain newlines submit separate queued user messages. For scripted smoke driving, collapse prompt files to one line before sending (`PROMPT=$(tr '\n' ' ' < /tmp/smoke-prompt.md); tmux send-keys -t "$SESSION":0.0 -l "$PROMPT"; tmux send-keys -t "$SESSION":0.0 Enter`). For manual multi-line editing, use Pi's external editor shortcut (`Ctrl+G`) or configure tmux extended keys so Pi can receive `Shift+Enter` for newlines; see the installed Pi `docs/tmux.md` guidance. Automated localhost, fake-upstream, and deterministic dogfood gates do not replace this human-readable live-site transcript evidence. When code changes, add a persisted-session `agent_browser_code` pass that aggregates at least two real pages, exercises a conditional branch, compares selected output with an ordinary-call baseline, verifies selected screenshot bytes reach the model, and checks browser/auth continuity on a later direct/code call with fresh JS globals. Review ordered custom transition entries and interrupted/replayed state, not only the outer result. Exercise handled failures, timeout/cancellation, whole-cell ordering, and child reaping; close only test-owned browsers afterward and retain Pi transcripts. Never use `--no-session` for code dogfood. When `agent_browser_web_search` or package config changed, add one key-free smoke proving the optional tool is absent without config, one fake/unit-backed smoke in the default suite, and one opt-in live Exa or Brave Search check with a real key while confirming the key does not appear in transcripts, stdout/stderr, config status, PR text, or artifacts. When `electron.*` surfaces, attached-session diagnostics, or `qa.attached` changed, add a local Electron pass against a disposable fixture app, never the operator's installed desktop apps such as Slack or Obsidian: wrapper launches open real app windows on the operator's desktop even with isolated profiles. Install the stock `electron` npm package into a temp directory (run its `install.js` if npm skipped the binary download) next to a minimal app whose hidden window (`new BrowserWindow({ show: false })`, plus `app.setActivationPolicy('prohibited')` and `app.dock?.hide()` on macOS) loads a known page so nothing appears on the operator's desktop, then run `electron.list` (discovery only) → `electron.launch` with `executablePath` set to that package's Electron binary and `appArgs` set to the fixture app directory (expect isolated profile behavior) → `snapshot -i` or `electron.probe` / `qa.attached` → `electron.cleanup` with the returned `launchId`, verifying status/mismatch guidance if you simulate a dead renderer or stale refs. Delete the fixture afterwards. For dense-dashboard stress coverage, use the [public Grafana stress checklist](#public-grafana-stress-checklist) below; it is a maintainer workflow, not bundled product skill or recipe runtime.
|
|
114
114
|
|
|
115
115
|
When reviewing saved session JSONL after a failed smoke or QA reclassification, expect browser tool rows to carry `isError: true` whenever `details.resultCategory` is `failure`. The model-visible observation must also report failure. Canonical JSON stays parseable and unchanged. Final public results return `isError` directly, with a prose failure notice outside explicit JSON, so Pi's transcript matches the wrapper contract ([`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details)). Preserve a normal Pi session directory for those checks; avoiding `--no-session` keeps this evidence intact ([`AGENTS.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/AGENTS.md) preferred validation workflow).
|
|
116
116
|
|
|
@@ -188,7 +188,7 @@ Please gather enough evidence to support the smoke result:
|
|
|
188
188
|
Return a concise PASS/FAIL report with evidence and any tool or workflow issues you noticed. Do not create a dogfood-output report directory.
|
|
189
189
|
```
|
|
190
190
|
|
|
191
|
-
Evaluator expectations after the queued Sauce Demo fixes: the agent should independently choose efficient, safe browser operations; native add-to-cart clicks should mutate cart state without the agent authoring `eval`/DOM-click fallbacks (the wrapper may fail with `details.clickDispatch` when upstream reports click success but no trusted DOM event reached the target); same-snapshot form fills may be batched safely when the agent chooses that route; the selected sort order should be verified; checkout must stop before Finish and must not place the order; the agent must not attempt Finish or another likely final submit action because prompt stop-boundaries are agent responsibility rather than wrapper-enforced business-intent policy; screenshot and recording must use the requested paths or be explicitly reported unavailable, and close should be blocked with `details.promptGuard.reason: "requested-artifacts-missing-before-close"` until required screenshot paths are verified; `network requests` may show public-demo telemetry 401s; `console` may report offline-cache logs; `errors` should show no page errors; and the browser session plus temp artifacts should be cleaned up after evidence is recorded. A run that reaches `checkout-complete.html` or silently substitutes artifact paths is a workflow failure even if other store flow steps work.
|
|
191
|
+
Evaluator expectations after the queued Sauce Demo fixes: the agent should independently choose efficient, safe browser operations; native add-to-cart clicks should mutate cart state without the agent authoring `eval`/DOM-click fallbacks (the wrapper may fail with `details.clickDispatch` when upstream reports click success but no trusted DOM event reached the target; that probe covers direct `click` calls on uniquely named button-like refs and XPath selectors, so batch rows, CSS selectors, and Sauce's repeated "Add to cart" refs are reported as unverified instead); same-snapshot form fills may be batched safely when the agent chooses that route; the selected sort order should be verified; checkout must stop before Finish and must not place the order; the agent must not attempt Finish or another likely final submit action because prompt stop-boundaries are agent responsibility rather than wrapper-enforced business-intent policy; screenshot and recording must use the requested paths or be explicitly reported unavailable, and close should be blocked with `details.promptGuard.reason: "requested-artifacts-missing-before-close"` until required screenshot paths are verified; `network requests` may show public-demo telemetry 401s; `console` may report offline-cache logs; `errors` should show no page errors; and the browser session plus temp artifacts should be cleaned up after evidence is recorded. A run that reaches `checkout-complete.html` or silently substitutes artifact paths is a workflow failure even if other store flow steps work. Known upstream defect: agent-browser 0.38.1 and 0.39.0 can report Sauce add-to-cart click success while delivering no trusted DOM events when the click follows an idle gap of about a second. Before calling an empty cart a wrapper regression, run the bare CLI control: log in, select `lohi`, sleep 1 s, `click '#add-to-cart-sauce-labs-onesie'`, then read `.shopping_cart_badge`; if the bare control also leaves the cart empty, record the cart/checkout steps as blocked upstream.
|
|
192
192
|
|
|
193
193
|
## What package verification checks
|
|
194
194
|
|
|
@@ -275,7 +275,7 @@ These show up often in cloud dev boxes and scripted smokes; they are maintainer
|
|
|
275
275
|
| Topic | What to watch for | Mitigation |
|
|
276
276
|
| ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
277
277
|
| **Pi CLI vs repo devDependencies** | Global `pi` older than the minimum Pi runtime floor for the release can change TUI behavior, `/reload`, package installs, and tool routing during lifecycle or checkout smokes. | Run `npm run doctor` and align `pi` with the current audited baseline before release gates (`pi update` or install the matching version). The published peer range stays wildcard per Pi package docs, and the doctor enforces the minimum Pi runtime floor before package validation. |
|
|
278
|
-
| **npm lockfile (`packageManager`)** | `package.json` pins **npm@12.2.0** for the Node 24.21.0 runtime baseline. Older npm versions can rewrite optional-platform metadata in `package-lock.json`; private registry proxies can also leak organization URLs into `resolved` entries. |
|
|
278
|
+
| **npm lockfile (`packageManager`)** | `package.json` pins **npm@12.2.0** for the Node 24.21.0 runtime baseline. Older npm versions can rewrite optional-platform metadata in `package-lock.json`; private registry proxies can also leak organization URLs into `resolved` entries. | Use the npm version declared in `packageManager` when refreshing the lockfile; do not commit unrelated lockfile churn or WorkOS/private-registry URLs. |
|
|
279
279
|
| **`pi -p` / print mode** | Non-interactive `pi -p` may hang or emit no stdout for long real-browser smokes without a TTY. | Use **tmux**-driven interactive `pi` for release evidence and checkout smokes; reserve `-p` for short, non-browser checks. |
|
|
280
280
|
| **Real-browser cleanup** | `real-upstream`, Sauce Demo, and live-site runs can leave defunct Chrome/`agent-browser` children if a session aborts mid-flow. | Close only test-owned browsers via `agent_browser` / `agent-browser` `close`, stop this run's tmux sessions, and remove its disposable screenshots/HARs. Preserve unrelated resources and retained evidence. |
|
|
281
281
|
| **Automated prompt driving** | Grepping tmux pane text for words that also appear in the **user** prompt (`PASS`, `FAIL`, `checkout overview`, `Smoke result:`) can false-complete before the agent finishes. | Wait for pane idle (no `Working…`), `agent_browser close` / `Artifact lifecycle`, or JSONL tool results—not instruction phrases copied from the prompt. |
|
|
@@ -290,13 +290,13 @@ Manual validation remains useful for release confidence and installed-package ch
|
|
|
290
290
|
|
|
291
291
|
### Real upstream contract validation
|
|
292
292
|
|
|
293
|
-
The default `npm test` and `npm run verify` paths use
|
|
293
|
+
The default `npm test` and `npm run verify` paths use deterministic tests and fake binaries in two stages: `tsx --test "test/**/!(*.serial).test.ts"` at Node's default file concurrency, then `tsx --test --test-concurrency=1 "test/**/*.serial.test.ts"` for strict timing budgets and shared policy-collector fixtures. The browser harness isolates each process's socket root; build/pack fixtures use disposable source trees so parallel consumers retain immutable checkout `dist/`. For a focused single-file rerun, use `npx tsx --test test/<file>.test.ts`; `npm test -- test/<file>.test.ts` still runs the package script's full glob. When a change touches upstream command planning, result presentation, managed-session behavior, or the canonical capability baseline, also run the opt-in real-upstream contract suite:
|
|
294
294
|
|
|
295
295
|
```bash
|
|
296
296
|
npm run verify -- real-upstream
|
|
297
297
|
```
|
|
298
298
|
|
|
299
|
-
That npm script sets `PI_AGENT_BROWSER_REAL_UPSTREAM=1
|
|
299
|
+
That npm script sets `PI_AGENT_BROWSER_REAL_UPSTREAM=1`, runs the sessionless plugin probe first in its own fail-fast process, then all other declarations in the contract file (including standalone/compound/opaque native text), followed by native batch fidelity and the real overlay-click suite. Each stage uses `--test-force-exit`; a failed plugin probe stops the workflow before browser work. To run `test/agent-browser.real-upstream-contract.test.ts` directly (for example with `node --test` and `tsx`), set the same variable yourself; the suite is skipped when it is unset.
|
|
300
300
|
|
|
301
301
|
This suite requires the installed stable `agent-browser --version` to meet the minimum in `scripts/agent-browser-target.mjs`; the output-shape fixture and command-reference verifier remain aligned to the current recommended baseline. It serves fixture pages from localhost and checks stable `details`/`data` keys via `test/fixtures/agent-browser-real-output-shapes.json`. Coverage groups:
|
|
302
302
|
|
|
@@ -312,7 +312,7 @@ This suite requires the installed stable `agent-browser --version` to meet the m
|
|
|
312
312
|
- **Failure shape:** `react tree` on a page opened with `--enable react-devtools` but without a React app (expects a clear missing-renderer error with session-bound `details`).
|
|
313
313
|
- **Async download:** `open` on the `/download` fixture, anchor-triggered export, then `wait --download <path>` metadata and wrapper artifact reporting for the requested path.
|
|
314
314
|
|
|
315
|
-
The default unit suite also runs `agentBrowserExtension passes through core command coverage fallback matrix` in [`test/agent-browser.extension-passthrough-
|
|
315
|
+
The default unit suite also runs `agentBrowserExtension passes through core command coverage fallback matrix` in [`test/agent-browser.extension-passthrough-native-commands.test.ts`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/test/agent-browser.extension-passthrough-native-commands.test.ts): a fake upstream records argv so explicit `--session connector connect 9222`, plus `download` with a selector and path, `get url`, `snapshot --compact`, and `tab new` / `tab t1` / `tab close` on implicit managed sessions, still prove `--json` and session ordering without a browser. A second fake-upstream matrix in [`test/agent-browser.extension-passthrough-debug-providers.test.ts`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/test/agent-browser.extension-passthrough-debug-providers.test.ts) (`agentBrowserExtension passes through non-core network debug diff stream dashboard and chat families`) pins representative `network`, `diff`, `trace` / `profiler` / `record`, `console` / `errors` / `highlight` / `inspect` / `clipboard`, `stream`, `dashboard`, and `chat` JSON shapes plus redacted `details.data` and argv echoes without a browser. Its `agentBrowserExtension passes through provider workflows` matrix asserts provider `open` argv shapes still receive `--json` plus implicit `--session` and provider credential env vars are forwarded into the fake upstream log, while `agentBrowserExtension keeps skills inspection flows stateless and useful` in the native-commands file keeps read-only `skills get …` stateless (no managed session fields). Extend those matrices when adding passthrough coverage that should stay out of the slow real-upstream loop.
|
|
316
316
|
|
|
317
317
|
### Native lifecycle qualification
|
|
318
318
|
|
|
@@ -386,7 +386,7 @@ After publishing a release, validate the package-first path in isolation. `npm r
|
|
|
386
386
|
```bash
|
|
387
387
|
npm exec --package pi-agent-browser-native -- pi-agent-browser-doctor
|
|
388
388
|
npm run verify -- release
|
|
389
|
-
pi --no-extensions -e npm:pi-agent-browser-native
|
|
389
|
+
pi --no-extensions -e npm:pi-agent-browser-native
|
|
390
390
|
```
|
|
391
391
|
|
|
392
392
|
Then run the real-browser smoke prompt:
|
package/docs/REQUIREMENTS.md
CHANGED
|
@@ -48,7 +48,7 @@ Define the product requirements and constraints for `pi-agent-browser-native`.
|
|
|
48
48
|
### Install priority
|
|
49
49
|
|
|
50
50
|
- Prioritize the package install path first.
|
|
51
|
-
- User-facing install docs should lead with `pi install npm:pi-agent-browser-native`; ephemeral package trials and validation should use `pi --no-extensions -e npm:pi-agent-browser-native
|
|
51
|
+
- User-facing install docs should lead with `pi install npm:pi-agent-browser-native`; ephemeral package trials and validation should use `pi --no-extensions -e npm:pi-agent-browser-native` so configured checkout or global sources cannot duplicate `agent_browser`, adding `--approve` in automation when the current project is intentionally trusted.
|
|
52
52
|
- User-facing install docs should also include the GitHub source path `pi install https://github.com/fitchmultz/pi-agent-browser-native`.
|
|
53
53
|
- Provide a read-only package-level doctor command that checks upstream `agent-browser` PATH/version and duplicate Pi package/checkout sources before first use. It must not mutate Pi settings and must remain distinct from upstream `agent-browser doctor`.
|
|
54
54
|
- Keep the current local-checkout path documented as the practical pre-release and development flow.
|
package/docs/SUPPORT_MATRIX.md
CHANGED
|
@@ -39,7 +39,8 @@ The compact direct/code/loader surface replaces public multimode inputs while re
|
|
|
39
39
|
| RQ-0065 QA | Flat `agent_browser_qa` preserves visible assertions, attached-buffer scope, diagnostic residue ambiguity, actionable/benign classification, and actual pass/fail. Fail-fast receipts separate failed checks from later `notRunChecks`; incomplete/timeout evidence stays unverified and execution counts without receipts remain unknown. |
|
|
40
40
|
| RQ-0066 / RQ-0067 source | Flat source/network tools retain bounded host scans and candidates-only semantics even in browser-only hosts. |
|
|
41
41
|
| RQ-0068 recipes | No recipe registry, persistent JavaScript, second driver, or interpreter dependency. |
|
|
42
|
-
| RQ-
|
|
42
|
+
| RQ-0072 refs | Page-scoped ref checks scan effective raw/stdin batch plans through every snapshot. A snapshot clears the invalidation latch, but a later page-changing step sets it again even when no refs precede that snapshot; later guarded ref use fails before spawn unless another snapshot intervenes. Same-snapshot fill and role-checked form-control exceptions remain. [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), `test/agent-browser.extension-ref-guards.test.ts` own this contract. |
|
|
43
|
+
| RQ-0096 Electron | Flat lifecycle tool preserves app/process/profile ownership and cancellation; native browser/code operations reuse its returned session. CDP readiness/liveness requires a nonempty Browser/browser string. Restored cleanup requires the launch-time process start identity and fails closed without a match ([`ELECTRON.md`](ELECTRON.md)). |
|
|
43
44
|
| RQ-0142 code | Fresh permissioned Node child over persistent parent-selected browser; explicit JSON/current-cell image output, visible failures/counts, ordinary auth/config/environment, ordered Pi intent/completion transitions, child cancellation/reaping, no legacy public execution. |
|
|
44
45
|
| Hotpath boundaries | Unrelated tools/harmless Bash return before prompt policy. Latest raw user intent is seeded on start/tree, updated by `message_end` and refreshed after native content replacement. Captured admission uses a validated append-only prefix; one queued replay restores branch/lease state and folds per-session launch settings once. Per-row renderer memoization skips unchanged formatting. No broad config/trust/history cache, new wait/timer or budget increase. |
|
|
45
46
|
| Shared ordering | Actual socket/namespace/session lock spans helper/action or entire cell; sorted identity sets for replacement/cleanup and namespace barrier for direct close-all. Requires every participating Pi to run the updated package; no human/third-party/older-version guarantee or transaction. |
|
|
@@ -80,8 +81,8 @@ Current summary:
|
|
|
80
81
|
| RQ-0137 | Upstream `agent-browser 0.32.4` rebaseline adds HAR `--content` body modes, the `derive-client` skill, and fixed `find role` implicit-ARIA/name matching with locator-detail misses; wrapper classifier/argv/docs cover those surfaces. | [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0330-rebaseline) |
|
|
81
82
|
| RQ-0138 | Upstream `agent-browser 0.33.0` rebaseline adds `a11y` axe-core audits and discarded-tab revival on tab switch; wrapper documents/presents `a11y` and samples the new help surface. | [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0330-rebaseline) |
|
|
82
83
|
| RQ-0139 | Upstream `agent-browser 0.33.2` rebaseline documents daemon idle timeout, streaming controls, and tab recovery. Wrapper-owned implicit sessions add transcript/checkout-scoped automatic restore with current v4 set-valued ticket-claim daemon coordination, eight-candidate opportunistic cleanup with randomized first windows and in-process rotation, verified snapshot retention, and page-target verification after ambiguous transitions. Explicit sessions, state/restore paths, config, file access, launch environment, local pages, output paths, and close arguments pass through unchanged; session/state lists keep all rows and identifiers visible. | [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0332-rebaseline), [`docs/ARCHITECTURE.md`](ARCHITECTURE.md) |
|
|
83
|
-
| RQ-0140 | Successful CDP/auto-connect and Electron sessions preserve one attached browser across native-tool follow-ups and cleanup by omitting wrapper local-launch defaults, with environment support, branch replay, first-use content blocking, live URL gates against external drift, and close cleanup. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.extension-passthrough-
|
|
84
|
-
| RQ-0141 | Headed managed sessions avoid upstream 0.33.2 visible autosave targets by default and retain that daemon environment across helpers, still-owned off-current follow-ups, failed-replacement ownership, Electron cleanup, and reload/resume; slow daemon-policy inspection uses a 35-second budget; headed mode is launch-scoped; the effective launch interval persists across resume and changes in either direction require close plus a fresh daemon; and bare, review-only, fenced-reference, conditional, directly negated, or reference attachment paths do not become close-blocking output requirements while explicit contiguous output path lists, per-clause availability, duplicate precedence, and Markdown destinations are handled conservatively and linearly. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.managed-session-daemon-policy.test.ts`, `test/agent-browser.process.test.ts`, `test/agent-browser.prompt-policy.test.ts`
|
|
84
|
+
| RQ-0140 | Successful CDP/auto-connect and Electron sessions preserve one attached browser across native-tool follow-ups and cleanup by omitting wrapper local-launch defaults, with environment support, branch replay, first-use content blocking, live URL gates against external drift, and close cleanup. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.extension-passthrough-session-artifacts.test.ts` |
|
|
85
|
+
| RQ-0141 | Headed managed sessions avoid upstream 0.33.2 visible autosave targets by default and retain that daemon environment across helpers, still-owned off-current follow-ups, failed-replacement ownership, Electron cleanup, and reload/resume; slow daemon-policy inspection uses a 35-second budget; headed mode is launch-scoped; the effective launch interval persists across resume and changes in either direction require close plus a fresh daemon; and bare, review-only, fenced-reference, conditional, directly negated, or reference attachment paths do not become close-blocking output requirements while explicit contiguous output path lists, per-clause availability, duplicate precedence, and Markdown destinations are handled conservatively and linearly. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.managed-session-daemon-policy.test.ts`, `test/agent-browser.process.test.ts`, `test/agent-browser.prompt-policy.serial.test.ts` |
|
|
85
86
|
| RQ-0142 | Persistent `agent_browser_code` uses the retained permissioned child and the same native executor as direct calls. Browser/auth state persists; JS state resets. Selected JSON/images, handled-failure receipts/counts, bounded IPC/deadline, shared whole-cell ordering, and ordered Pi transition replay replace isolated finally-close execution. The optional host `beforeExecute` callback retains outer call identity and serial dispatch. Strict old lease readers exist only for upgrade cleanup. | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#agent_browser_code), [`ARCHITECTURE.md`](ARCHITECTURE.md#persistent-browser-code), `test/agent-browser.script.test.ts`, `test/agent-browser.tool-surface.test.ts`, `test/agent-browser.execution-lock.test.ts`, `test/agent-browser.execution-order.test.ts` |
|
|
86
87
|
| RQ-0143 | Browser-backed calls require a stable upstream version at or above the configured 0.35.0 floor, report minimum/observed version mismatch before launch, and keep stateless/local diagnostics available; the recommended target and minimum are shared by runtime, docs verification, and package contents. | [`scripts/agent-browser-target.mjs`](../scripts/agent-browser-target.mjs), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0360-rebaseline), `test/agent-browser.upstream-version.test.ts` |
|
|
87
88
|
| RQ-0144 | Managed restore keys are scoped to both checkout generation and Pi transcript so upstream 0.33.2's newest-file-wins loading cannot let concurrent chats clobber or inherit cookies/storage. Fresh rotations, reload, restart, and `/resume` of one transcript retain continuity; after a confirmed inactive daemon, a branch-restored compatible owned session retains its pending reopen through daemon-starting non-page calls and transcript replay, then reopens the complete recorded URL (including its fragment) before current-page operations such as `get url` or `reload`, verifies the actual target and invalidates old refs. This reloads a URL, not forms, JavaScript memory, or history; live wrong-tab recovery never auto-navigates. Passive and explicit upstream configs pass through, while explicit config selection suppresses the wrapper's automatic restore injection. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.managed-session-restore.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
|
|
@@ -101,11 +102,11 @@ QA truthfulness (#121, RQ-0145) is covered by the registered-tool controls in `t
|
|
|
101
102
|
|
|
102
103
|
Explicit native text preserves standalone and compound/raw-or-stdin batch output without changing caller input. `test/agent-browser.text-output.test.ts` covers opaque JSON/confirmation-like page strings, complete whitespace-preserving stdout and redacted subprocess/presentation spills, text-only output export success/failure notices, nonzero failure and strict malformed JSON; the `real native explicit text` case in `test/agent-browser.real-upstream-contract.test.ts` proves native URL reads, both diff orders, row boolean operands and last-wins outer selection. The same native case protects empty stdin-row skipping, later successful rows, Unicode results, bail behavior and original row positions in text errors; malformed shapes still fail before dispatch. Failed-exit stderr is primary while zero-exit malformed JSON remains strict, covered by the registered `failed-json`/`strict-json` cases. Text contains no trusted row, confirmation, lifecycle, page-target or artifact receipts; use JSON for those fields. See [the output contract](TOOL_CONTRACT.md#wrapper-json).
|
|
103
104
|
|
|
104
|
-
Browser evidence (RQ-0152) is covered by `test/agent-browser.browser-evidence.test.ts`, `test/agent-browser.read-confirmation.test.ts`, `test/agent-browser.recording-receipts.test.ts` and `test/agent-browser.recording-recovery.test.ts`, alongside existing presentation, reservation/replay and lifecycle checks. They cover cold/shared/unknown/invalid/timeout reads, native-default and shared confirmation routing, capability absence and script identity boundaries, separate daemon/browser/Pi identity, sparse/legacy capture metrics, native failure retention, stale/same-path mismatches, one-query stop recovery, error exports and artifact aliases. Full live identity, receipt lookup and browser-independent native read/confirm handling require companion upstream fixes; missing fields remain unknown, and native receipts do not survive daemon exit. These source checks do not replace paired native/Pi runtime verification.
|
|
105
|
+
Browser evidence (RQ-0152) is covered by `test/agent-browser.browser-evidence.test.ts`, `test/agent-browser.read-confirmation-routing.test.ts`, `test/agent-browser.read-confirmation-capture-replay.test.ts`, `test/agent-browser.read-confirmation-capture-actions.test.ts`, `test/agent-browser.recording-receipts.test.ts` and `test/agent-browser.recording-recovery.test.ts`, alongside existing presentation, reservation/replay and lifecycle checks. They cover cold/shared/unknown/invalid/timeout reads, native-default and shared confirmation routing, capability absence and script identity boundaries, separate daemon/browser/Pi identity, sparse/legacy capture metrics, native failure retention, stale/same-path mismatches, one-query stop recovery, error exports and artifact aliases. Full live identity, receipt lookup and browser-independent native read/confirm handling require companion upstream fixes; missing fields remain unknown, and native receipts do not survive daemon exit. These source checks do not replace paired native/Pi runtime verification.
|
|
105
106
|
|
|
106
107
|
Contributor fixes #133/#152 remove unused prompt suffix entries without changing runtime guidance and diagnose bare `--no-sandbox` only in the command slot or navigation option positions. Native `--args` values and literal operands remain intact; batch checks use raw effective rows without treating row-local `--args` as a launch setting. `test/agent-browser.chromium-args.test.ts` covers pre-dispatch rejection, literal/flag-value controls, inspection, raw/stdin batch precedence and dispatcher outcomes. Existing prompt/grammar checks remain; source checks do not replace native-product gates.
|
|
107
108
|
|
|
108
|
-
Electron diagnostics (RQ-0096, #128) keep list timeout rejection truthful, label explicit-ID cleaned records as historical without changing active selection or actions, and measure the tracked profile path with native `lstat` (`present` / ENOENT-only `absent` / `unknown`). Focused Electron discovery, host-lifecycle, and branch-ownership tests cover current liveness independently of cleanup history, native path errors and dangling symlinks, transcript replay, and unchanged cleanup ownership. Failed-startup diagnostics include redacted last-4096-byte stdout/stderr tails in visible text and structured details, using private profile-local regular files. Regression tests cover exact tails, empty output, capture/spawn errors and closed native file handles; a real spawned fixture with injected kill denial proves live output through temp cleanup, retained logs through native host-exit behavior, and persistence failure reporting. Tracked-child cleanup regressions require actual exit before profile removal and immediate executable deletion. Restored-PID cleanup
|
|
109
|
+
Electron diagnostics (RQ-0096, #128) keep list timeout rejection truthful, label explicit-ID cleaned records as historical without changing active selection or actions, and measure the tracked profile path with native `lstat` (`present` / ENOENT-only `absent` / `unknown`). Focused Electron discovery, host-lifecycle, and branch-ownership tests cover current liveness independently of cleanup history, native path errors and dangling symlinks, transcript replay, and unchanged cleanup ownership. Failed-startup diagnostics include redacted last-4096-byte stdout/stderr tails in visible text and structured details, using private profile-local regular files. Regression tests cover exact tails, empty output, capture/spawn errors and closed native file handles; a real spawned fixture with injected kill denial proves live output through temp cleanup, retained logs through native host-exit behavior, and persistence failure reporting. Tracked-child cleanup regressions require actual exit before profile removal and immediate executable deletion. Restored-PID cleanup requires the process start identity captured at launch and replayed from the transcript; missing, unreadable, or mismatched identities fail closed. `test/agent-browser.extension-electron-host-lifecycle.test.ts` covers matching-identity cleanup of a spaces/Unicode profile, refusal of missing or mismatched identities even when the live command line contains the exact profile operand, malformed restored identities, and shutdown after transcript restoration, plus the nonempty `Browser`/`browser` CDP readiness requirement with other metadata optional. The shared daemon policy verifies a restored Electron attachment's live browser endpoint and named upstream `get cdp-url` before ordinary calls, status, or probe reuse; `test/agent-browser.extension-electron-branch-ownership.test.ts` covers successful reuse and repeated rejection of replaced app, connection, and namespace identities without weakening generic restore-disabled rules. These checks do not replace native-app, Pi lifecycle, package, or live-site gates.
|
|
109
110
|
|
|
110
111
|
Artifact diagnostics (#124/#127) use a shared pre-dispatch mkdir-error boundary, preserve raw batch argv/precedence, recommend absolute raw artifact paths, recognize image headers rather than filename MIME guesses, retain known requested/reported paths, and warn once about conservative ref invalidation after dispatched recording starts and URL-bearing restarts, on success or failure, without claiming an observed page change. `test/agent-browser.artifact-diagnostics.test.ts` covers registered filesystem failures, real image bytes and misleading suffixes, the inline bound, native macOS path aliases, recording/ref continuity and unreached-row negatives; `test/agent-browser.presentation-artifacts-batch.test.ts` retains artifact/persistence coverage. These source regressions do not qualify daemon-cwd differences or affected-filesystem timestamp behavior (#118), or replace native/Pi/package/live-site gates.
|
|
111
112
|
|
|
@@ -115,7 +116,7 @@ Upstream 0.37 recording compatibility uses one command-local path/URL reader aro
|
|
|
115
116
|
|
|
116
117
|
Artifact argv and timeout recovery (#119, RQ-0076) use native operand positions, retain literal global-looking tokens inside batch rows, and select raw batch commands instead of ignored stdin. Timeout observations retain current page and artifact existence evidence while all unacknowledged planned rows stay unknown; neither a matching URL nor an existing file proves execution. Multi-step timeouts offer inspection without inferring a failed/pending boundary. Executable retry candidates for a single retryable step preserve the original row in a one-row batch. Interleaved `wait --download` / `-d` timeout pairs keep the next retained path and its original index. Runtime, artifact-diagnostics, and timeout tests cover extraction, reservation, preparation, and presentation; `test/agent-browser.batch-fidelity.test.ts` adds real native saved-file, missing wait-download, reservation, and watchdog evidence under `npm run verify -- real-upstream`. Getter batching amortizes one extra ref-freshness snapshot per top-level call or batch when the existing checks apply. Direct `session info` timeouts stay status-only (`retry-session-info`), preserving page/ref/reopen state; cold-boundary and result tests cover this.
|
|
117
118
|
|
|
118
|
-
Cold URL reopen is covered by `test/agent-browser.cold-resume.test.ts`, the daemon/page-boundary regressions in `test/agent-browser.cold-boundaries.test.ts`, and the opt-in `contract suite matches cold URL reopen after quit` case in `test/agent-browser.real-upstream-contract.test.ts`. Deterministic boundary tests cover non-page prefixes, fragment routes, history commands, explicit destinations, transcript replay, unreached navigation and unchanged native batch/error order. `test/agent-browser.destination-cancel.test.ts` covers direct/native-batch new-window and URL-diff follow-ups, redirected/unknown targets (including blank redirects with a remembered duplicate tab), ordered ref invalidation, reached versus unreached rows, and cancellation before or after a cold-reopen attempt with replay and process cleanup. Release acceptance requires the first page operation after confirmed shutdown to observe the remembered non-root URL with both empty and origin storage, including after non-page calls and on hash-routed pages, before any explicit open, with refreshed refs. Focused native Mac evidence does not replace actual Ubuntu, real Pi lifecycle, checkout, live-site, or final release gates.
|
|
119
|
+
Cold URL reopen is covered by `test/agent-browser.cold-resume.test.ts`, the daemon/page-boundary regressions in `test/agent-browser.cold-boundaries-session-policy.test.ts`, `test/agent-browser.cold-boundaries-page-reopen.test.ts` and `test/agent-browser.cold-boundaries-command-classification.test.ts`, and the opt-in `contract suite matches cold URL reopen after quit` case in `test/agent-browser.real-upstream-contract.test.ts`. Deterministic boundary tests cover non-page prefixes, fragment routes, history commands, explicit destinations, transcript replay, unreached navigation and unchanged native batch/error order. `test/agent-browser.destination-cancel.test.ts` covers direct/native-batch new-window and URL-diff follow-ups, redirected/unknown targets (including blank redirects with a remembered duplicate tab), ordered ref invalidation, reached versus unreached rows, and cancellation before or after a cold-reopen attempt with replay and process cleanup. Release acceptance requires the first page operation after confirmed shutdown to observe the remembered non-root URL with both empty and origin storage, including after non-page calls and on hash-routed pages, before any explicit open, with refreshed refs. Focused native Mac evidence does not replace actual Ubuntu, real Pi lifecycle, checkout, live-site, or final release gates.
|
|
119
120
|
|
|
120
121
|
Batch/ref fidelity (#122/#123/#125/#126): tab recovery uses verified native selection under existing session serialization without reconstructing argv/stdin or changing caller batch control flow. Selector-only stale-ref checks cover `@eN`, `eN`, and `ref=eN`; literal operands and keyboard/mouse data remain native. `test/agent-browser.batch-fidelity.test.ts` runs paired pinned/unpinned real-upstream fixtures, including mixed failures, header/timeout flags, ignored stdin, current/stale refs, failed/wrong-target selection, sessionless commands, and explicit connection/state recovery with failed-batch negatives. It runs in `npm run verify -- real-upstream`; deterministic selector/shape checks also run in the default suite.
|
|
121
122
|
|
|
@@ -125,7 +126,7 @@ Lightpanda support preserves native command access rather than promising Chrome
|
|
|
125
126
|
|
|
126
127
|
Confirmation-policy continuity uses existing canonical session page events, resolved before helpers. The focused native nextAction case proves exact denial, a useful confirmed audit followed immediately by title/body reads across resume, bare getter/PID continuity, and denied recording-restart sentinel/original-take preservation; the shared-defaults owner covers selected-string/empty/unset precedence, code/advanced routing, ordered pending-batch targets/replay with a gated URL helper, failed replacement cleanup/recovery, off-branch cleanup, close and identity isolation. Strict native guarded-control markers reuse the same transcript reducer with compact command/action provenance, not command replay; exact decisions avoid helper-slot replacement without claiming native ID validation or browser independence. Read-confirmation tests cover forged/foreign/first-nondecision controls, raw-argv-over-stdin decisions, later unknown-page checks, ordered confirmed batch effects, truthful requested/early-helper/post-command-helper pending copy, and completed click/tab/failed-inner/close reconciliation through real code intent/finish replay, cold-resume covers pending helper exposure/replay, and Electron host tests cover ambient/retained policy. Definite no-admission attempts restore prior policy while interrupted work retains intent. Pending close does not retire ownership or restore state. Native policy changes still restart normally; pending commands remain native-owned. Read-confirmation regressions also cover ordered confirmed snapshots, later pending/failed/invalidation rows, one-use freshness consumption through direct/code/batch resume, semantic role click/check/fill and unique role/label select after snapshot approval with unchanged payloads, absent/ambiguous select targets and drift, warm/no-resume direct and semantic select success on unchanged path/hash routes for caller-owned and freshly established wrapper-owned managed sessions versus refusal after a live-observed complete-URL change, including hash-only routes, without a gated-snapshot loop or changing ordinary fragment-insensitive ref/tab comparisons, plus live/resumed attachment retirement after confirmed close. Registered confirmation tests also cover direct/batch tab transport metadata, exact approval/denial, visible conditional continuation, malformed/mismatched extras and page/eval/HTTP provenance rejection; actual SDK/native proof uses the real authenticated-tab transport with a dummy loopback token. Presentation tests distinguish direct/nested pending snapshot controls from complete empty snapshots and preserve redacted recovery actions and explicit JSON. Click-dispatch owner tests and local native SDK proof cover timer cancellation, pending-safe marker/listener expiry without an extra helper, and native ref/XPath click dispatch when `evaluate` guards optional diagnostic probes. The patched recording-restart/navigation subcase has a separate test-only opt-in; stock upstream still runs the remaining confirmation checks. See [real upstream qualification](RELEASE.md#real-upstream-contract-validation).
|
|
127
128
|
|
|
128
|
-
Shared native defaults are covered by `test/agent-browser.shared-defaults.test.ts`: environment/file/argv precedence, caller-owned reuse across independent Pi contexts, quit ownership, native helper idle settings, unchanged implicit cleanup, and an opt-in real local-profile fixture proving shared state and code follows the selected ordinary browser rather than a disposable isolated session. Code tests cover the fresh child language boundary and persistent browser behavior; the local Electron handoff regression in `test/agent-browser.extension-electron-lifecycle.test.ts` keeps generated launch ownership despite a native session environment default; native config validation uses upstream rather than a second schema. See [shared browser defaults](
|
|
129
|
+
Shared native defaults are covered by `test/agent-browser.shared-defaults.test.ts`: environment/file/argv precedence, caller-owned reuse across independent Pi contexts, quit ownership, native helper idle settings, unchanged implicit cleanup, and an opt-in real local-profile fixture proving shared state and code follows the selected ordinary browser rather than a disposable isolated session. Code tests cover the fresh child language boundary and persistent browser behavior; the local Electron handoff regression in `test/agent-browser.extension-electron-host-lifecycle.test.ts` keeps generated launch ownership despite a native session environment default; native config validation uses upstream rather than a second schema. See [shared browser defaults](REFERENCE.md#shared-browser-defaults). These tests do not promise permanent third-party authentication.
|
|
129
130
|
|
|
130
131
|
Browser correctness regressions cover credential-bearing filtered network details/exports, native generated-Blob downloads with actual saved bytes, accessible-name click target identity and abstention, smooth CSS-container movement, and parseable early-result JSON. Native fixture coverage complements deterministic dispatch/cleanup and output-shape checks; release qualification remains separate.
|
|
131
132
|
|
|
@@ -169,7 +170,7 @@ Runtime floor note: package metadata requires Node 24.21.0 or newer, keeps optio
|
|
|
169
170
|
| Baseline section | Baseline items | Documentation | Runtime handling | Test coverage | Validation status |
|
|
170
171
|
| ---------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
171
172
|
| Built-in skills | 19 canonical tokens from baseline section `skills`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#built-in-skills). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#built-in-skills), generated baseline block, README proof section, release docs. | `needsManagedSession` keeps read-only skills inspection sessionless while preserving thin upstream passthrough; upstream `@agent-browser/sandbox` remains external package guidance, not a bundled wrapper dependency. | Runtime and extension-validation skills/provider matrix; real-upstream inspection/skills group. | Supported. |
|
|
172
|
-
| Core page, element, navigation, and extraction commands | Canonical tokens from baseline section `core-commands`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-page-and-element-commands). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-page-and-element-commands), [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md), README quick start. | Thin passthrough with wrapper-owned JSON/session planning, ref guidance, artifact verification, page-change summaries, click-dispatch diagnostics only after native temporary-marker target identity readback (otherwise abstaining), inspection-only recovery for upstream overlay-blocked clicks, deterministic document scroll before wheel/no-op diagnostics and instant CSS-container movement, shorthand compilers, and redaction.
|
|
173
|
+
| Core page, element, navigation, and extraction commands | Canonical tokens from baseline section `core-commands`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-page-and-element-commands). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-page-and-element-commands), [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md), README quick start. | Thin passthrough with wrapper-owned JSON/session planning, ref guidance, artifact verification, page-change summaries, click-dispatch diagnostics only after native temporary-marker target identity readback (otherwise abstaining), inspection-only recovery for upstream overlay-blocked clicks, deterministic document scroll before wheel/no-op diagnostics and instant CSS-container movement only for complete recognized shim shapes (surplus operands remain native), shorthand compilers, and redaction. | Real-upstream core matrix (including snapshot-ref select) plus fake core matrix for passthrough, ordering, diagnostics, and compiler validation. | Supported. Upstream semantics remain upstream-owned. |
|
|
173
174
|
| Sessions, state, tabs, frames, dialogs, and windows | 28 canonical tokens from baseline section `state-tabs-frames-dialogs`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#session-state-frames-dialogs-windows-and-inspection-commands). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#session-state-frames-dialogs-windows-and-inspection-commands), stateful workflow notes, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). | Stateful summaries redact credentials while preserving restore identifiers, every session/state list row, and ordinary `cookies set --curl` file operands in invocation echoes and failed batch diagnostics; explicit targets and paths pass through. Runtime retains state artifact verification, sessionless local command planning, automatic managed restore, native-CDP-ID tab recovery across duplicate URLs and transcript replay, reached failed-batch target invalidation/live verification, active-target refresh after tab close, and close alias cleanup. | Extension tab/ref tests, real-upstream stable-id/label tab lifecycle, runtime session/resume tests, presentation redaction tests, lifecycle harness. | Supported. External profile/auth state remains operator-owned. |
|
|
174
175
|
| Network, storage, artifacts, diagnostics, and performance | Canonical tokens from baseline section `network-storage-artifacts-diagnostics`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#page-state-finding-mouse-settings-network-and-storage). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#page-state-finding-mouse-settings-network-and-storage), diagnostic sections, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). | Thin passthrough plus native-owned click/download execution for all downloads (including loopback, Blob exports, and redirects), compact diagnostics with identical credential redaction for page/origin/URL-filtered details and exports, route-mock warnings, useful-but-redacted storage output, stream idempotency normalization, FPS/cursor/contact-sheet-aware recording destinations/pinning and contact-sheet image verification, truthful ref-refresh and missing-ffmpeg warnings, positive WebMCP navigation hints and catalog updates, sensitive-data redaction, timeout bounds, and cleanup-pair guidance. | Fake non-core matrix and safe real-upstream coverage for network/HAR, diff, trace/profiler, console/errors/highlight, stream, vitals, React missing-renderer, native FPS recording path/page checks and WebMCP navigation hints. | Supported. Environment-sensitive operations need suitable local/browser state. |
|
|
175
176
|
| Batch, auth, confirmations, setup, dashboard, devices, and AI commands | Canonical tokens from baseline section `batch-auth-setup-ai`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#batch-auth-confirmations-sessions-chat-dashboard-devices-and-setup). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#batch-auth-confirmations-sessions-chat-dashboard-devices-and-setup), README security notes, release docs. | Native-tool batch stdin, generated QA/lookup batch plans, auth/confirmation redaction, sessionless local auth/setup/dashboard/doctor/plugin planning, upgrade-only native text normalization with error precedence and explicit-JSON results, plugin list/show JSON envelope normalization, bare-`mcp` validation with `mcp --help` preserved, timeout/cleanup guidance. | Parser/runtime plugin and MCP unit coverage; fake-upstream plugin list/show and MCP help/blocking coverage; registered upgrade text/success/nonzero/timeout/abort/JSON controls in `test/agent-browser.upgrade-output.test.ts`; real-upstream plugin list shape probe; structured input-mode tests. | Supported. Interactive side-effecting setup/auth/chat remains upstream-owned. `plugin` is local/sessionless; `mcp` is external-client-only except help; `auth login --credential-provider` resolves credentials via a plugin; `install --with-deps` failures remain upstream-owned. |
|
package/docs/TOOL_CONTRACT.md
CHANGED
|
@@ -137,7 +137,7 @@ Result details:
|
|
|
137
137
|
}
|
|
138
138
|
```
|
|
139
139
|
|
|
140
|
-
For Exa, `details.searchType` is the effective requested type even when the provider response does not echo it. `requestId` is included when Exa returns one. Brave omits both fields. `pageDate` comes from Exa `publishedDate` (an estimated page creation date) or Brave `page_age`; Brave may separately return `age`. These provider fields are not crawl/retrieval age or proof that a result matches a requested version. For version-sensitive work, inspect the result, constrain one follow-up to the primary domain (`includeDomains` for Exa or `site:` in a Brave query), and read the primary page. Failures return structured results: local config/validation problems (invalid config, disabled tool, missing credential, blank query, Exa-only filters resolved to Brave) are `validation-error`; provider/network failures are `upstream-error`; both keep `isError` and redacted messages.
|
|
140
|
+
For Exa, `details.searchType` is the effective requested type even when the provider response does not echo it. `requestId` is included when Exa returns one. Brave omits both fields. `pageDate` comes from Exa `publishedDate` (an estimated page creation date) or Brave `page_age`; Brave may separately return `age`. These provider fields are not crawl/retrieval age or proof that a result matches a requested version. For version-sensitive work, inspect the result, constrain one follow-up to the primary domain (`includeDomains` for Exa or `site:` in a Brave query), and read the primary page. Failures return structured results: local config/validation problems (invalid config, disabled tool, missing credential, blank query, Exa-only filters resolved to Brave) are `validation-error`; provider/network failures are `upstream-error`; both keep `isError` and redacted messages. Malformed provider JSON reports the provider-specific invalid-JSON error without parser previews or parser causes. Malformed config files, in extension loading and the config CLI alike, report their scope/path and `Invalid JSON` plus the parser's numeric position/line/column when available, never source fragments; either payload may contain credentials. Provider HTTP failures never echo the response body or status text: they report the provider, `HTTP <status>`, and a fixed hint (credential/permission check for 401/403, retry later for 5xx, check the search parameters otherwise); HTTP 429 keeps the rate-limit guidance. Text content, `details`, and structured output are rendered from one redacted result object.
|
|
141
141
|
|
|
142
142
|
## Input mode chooser
|
|
143
143
|
|
|
@@ -300,7 +300,7 @@ Examples:
|
|
|
300
300
|
|
|
301
301
|
Relative file operands, `outputPath`, and source/network-source scans use one `pi-change-working-dir` owner reply captured before browser policy and queues; code calls inherit it. The integration uses the synchronous `pi-change-working-dir:resolve-execution-cwd` event with `{ sessionManager, result?: { cwd, error? } }`. Owner errors or invalid replies fail validation. No owner means native `ctx.cwd`; a known older owner identified through public tool/command package provenance requires an update and Pi restart. No private directory journal is read.
|
|
302
302
|
|
|
303
|
-
Browser/session/config ownership stays separate. Ordinary calls retain the native project or managed launch root, while an unnamed fresh launch and explicit native config resolve against the execution directory. `details.managedSessionCwd`, when present, records the managed launch root for transcript replay, including ordered code transitions. Pi package config/trust and artifact-store identity remain unchanged. A deleted launch directory returns `validation-error` before dispatch; restore it or explicitly choose fresh/config from the new directory. See [working-directory changes](
|
|
303
|
+
Browser/session/config ownership stays separate. Ordinary calls retain the native project or managed launch root, while an unnamed fresh launch and explicit native config resolve against the execution directory. `details.managedSessionCwd`, when present, records the managed launch root for transcript replay, including ordered code transitions. Pi package config/trust and artifact-store identity remain unchanged. A deleted launch directory returns `validation-error` before dispatch; restore it or explicitly choose fresh/config from the new directory. See [working-directory changes](REFERENCE.md#working-directory-changes).
|
|
304
304
|
|
|
305
305
|
### `agent_browser_code`
|
|
306
306
|
|
|
@@ -377,7 +377,7 @@ Enable `qa`. Flat input is either `{ url, ...checks }` or `{ attached: true, ...
|
|
|
377
377
|
{ "attached": true, "expectedText": "Explorer", "checkErrors": true }
|
|
378
378
|
```
|
|
379
379
|
|
|
380
|
-
QA compiles a fixed `batch --bail` diagnostic plan. URL QA clears enabled network/console buffers, captures page-error residue after native clear, opens the requested URL, waits for readiness, allows a bounded 150 ms diagnostic settle when needed, then performs visible-text/selector assertions, enabled diagnostic reads, and optional screenshot. Expected text uses bounded visible-element predicates with a 5-second native timeout. Missing assertions stop before later diagnostics. URL QA does not require the previous page target.
|
|
380
|
+
QA compiles a fixed `batch --bail` diagnostic plan. URL QA clears enabled network/console buffers, captures page-error residue after native clear, opens the requested URL, waits for readiness, allows a bounded 150 ms diagnostic settle when needed, then performs visible-text/selector assertions, enabled diagnostic reads, and optional screenshot. Expected text uses bounded visible-element predicates with a 5-second native timeout. Text nodes and control values under `display: none` or `opacity: 0` ancestors, or whose computed visibility is hidden, do not satisfy the assertion; `display: contents` wrappers and `visibility: visible` overrides still count. Missing assertions stop before later diagnostics. URL QA does not require the previous page target.
|
|
381
381
|
|
|
382
382
|
Attached QA checks the current browser without navigation or buffer clears. It first requires a live nonempty URL, including file/custom app URLs. Failed target verification returns exact tab/URL recovery instead of running the batch. Diagnostic reads default off because preserved buffers may predate this check; URL diagnostics default on. Explicit `checkErrors: false` skips the error-baseline check.
|
|
383
383
|
|
|
@@ -402,16 +402,20 @@ Enable `electron`. Inputs are flat `{ action, ...fields, outputPath? }`; caller
|
|
|
402
402
|
{ "action": "cleanup", "launchId": "electron-…" }
|
|
403
403
|
```
|
|
404
404
|
|
|
405
|
-
`handoff` defaults to `snapshot`; `tabs` verifies/discovers targets without an interactive snapshot, and `connect` stops after attach. `targetType` is `page` by default, or `webview`/`any`. Launch cannot reuse the normal signed-in app profile or an existing app; use host launch plus direct native `connect` for that. Windows launch requires a verifiable path rather than scan-based name resolution; this is not Windows qualification evidence.
|
|
405
|
+
`handoff` defaults to `snapshot`; `tabs` verifies/discovers targets without an interactive snapshot, and `connect` stops after attach. `targetType` is `page` by default, or `webview`/`any`. CDP readiness and debug-port liveness require `/json/version` to contain a nonempty, non-whitespace `Browser` or `browser` string; other version metadata is optional. Launch cannot reuse the normal signed-in app profile or an existing app; use host launch plus direct native `connect` for that. Windows launch requires a verifiable path rather than scan-based name resolution; this is not Windows qualification evidence.
|
|
406
406
|
|
|
407
407
|
`appArgs` cannot override user-data-dir/debug lifecycle flags or include bare `--`. Caller `allow`/`deny` substring policies match app names/IDs/paths, with deny winning. Likely-sensitive discovery annotations are advisory. Non-Electron executables fail validation.
|
|
408
408
|
|
|
409
|
+
Electron payload evidence accepts `app.asar`, `app/`, or the stock runtime's `default_app.asar`, alongside platform runtime resources (including Electron Framework on macOS). Stock npm Electron can launch an unpackaged app by passing its binary as `executablePath` and the app directory in `appArgs`; discovery and explicit paths share the same evidence check.
|
|
410
|
+
|
|
409
411
|
Timeouts are per action: launch readiness defaults to 15 seconds, capped at 120 seconds after discovery; status/probe use the normal 35-second subprocess budget per read (or configured override); cleanup applies its default 35-second managed-close budget separately to close and initial process-exit wait. Local CDP HTTP probes use one second. There is no combined end-to-end teardown deadline. Cancellation before spawn opens nothing; cancellation during launch cleans the new process/profile.
|
|
410
412
|
|
|
411
|
-
`details.electron` retains action-specific `apps`, `launch`, `statuses`, `targets`, `probe`, `cleanup`, and failure diagnostics; `compiledElectron` retains the action plan. `launchId` identifies host lifecycle; `sessionName` identifies browser commands. `userDataDirState` measures the tracked path as present, absent (ENOENT), or unknown; it does not audit all app residue. Failed startup output includes redacted last-4096-byte stdout/stderr tails; private profile-local files follow profile ownership, not a lifetime byte cap. Failed cleanup preserves live profiles. Reload preserves current branch-visible launches and cleans off-branch owned launches; quit cleans owned launches. Restored attachments must match live endpoint/PID/profile evidence before reuse.
|
|
413
|
+
`details.electron` retains action-specific `apps`, `launch`, `statuses`, `targets`, `probe`, `cleanup`, and failure diagnostics; `compiledElectron` retains the action plan. `launchId` identifies host lifecycle; `sessionName` identifies browser commands. `userDataDirState` measures the tracked path as present, absent (ENOENT), or unknown; it does not audit all app residue. Failed startup output includes redacted last-4096-byte stdout/stderr tails; private profile-local files follow profile ownership, not a lifetime byte cap. Failed cleanup preserves live profiles. Launch records the app process start identity. Before signaling a live restored PID without a tracked child handle, cleanup requires that recorded identity to match the live process; a missing (including records from older versions), unreadable, or mismatched identity leaves the process/profile intact. Reload preserves current branch-visible launches and cleans off-branch owned launches; quit cleans owned launches. Restored attachments must match live endpoint/PID/profile evidence before reuse.
|
|
412
414
|
|
|
413
415
|
Post-command process/target loss fails as `tab-drift`; fill verification and broad-shell-text diagnostics remain available. Recovery actions use `agent_browser_electron` with flat params for status/probe/cleanup and `agent_browser` for native tab/URL/ref operations. If inactive, enable `electron` first. Manually launched apps and explicit saved artifacts remain host-owned.
|
|
414
416
|
|
|
417
|
+
Status/probe mismatch checks and probe page-target state use complete URLs, not display previews. Managed-session/probe URL summaries remain capped at 300 characters and probe tab URLs at 240; shortened output does not itself imply drift. Probe `sessionTabTarget` and `refSnapshot` retain full target identity for follow-up commands and replay.
|
|
418
|
+
|
|
415
419
|
### `agent_browser_source`
|
|
416
420
|
|
|
417
421
|
Enable `source`. Flat fields: at least one of `selector`, `reactFiberId`, or `componentName`; optional `includeDomHints` (default true), `maxWorkspaceFiles` (default 2,000, maximum 5,000), `outputPath`, `timeoutMs`, and `sessionMode`.
|
|
@@ -432,7 +436,7 @@ Enable `network`. Flat fields: at least one of `requestId`, `filter`, or `url`;
|
|
|
432
436
|
{ "requestId": "req-1", "url": "/api/fail" }
|
|
433
437
|
```
|
|
434
438
|
|
|
435
|
-
A request ID runs native `network request`; filter/URL adds `network requests --filter`, with explicit filter winning. Failed HTTP/failed/error rows are correlated with initiator metadata and a bounded URL-literal search under the captured execution directory (up to eight needles and ten workspace matches). `networkSourceLookup` retains `{ status, failedRequests, candidates, limitations, summary }`; statuses are `failed-requests-found`, `no-failed-requests`, or `no-candidates`. URLs and credentials are redacted. Request URLs are diagnostic evidence and never replace the active page target. No candidates alone does not fail a successful batch or assign blame. HAR and full native diagnostics remain available through `args`.
|
|
439
|
+
A request ID runs native `network request`; filter/URL adds `network requests --filter`, with explicit filter winning for both the native query and failed-request/initiator analysis. Initiators of failed requests outside that filter/URL scope are ignored. Repeated request IDs count once (first matching failed row wins); anonymous rows and different IDs at the same URL remain distinct. Failed HTTP/failed/error rows are correlated with initiator metadata and a bounded URL-literal search under the captured execution directory (up to eight needles and ten workspace matches). `networkSourceLookup` retains `{ status, failedRequests, candidates, limitations, summary }`; statuses are `failed-requests-found`, `no-failed-requests`, or `no-candidates`. URLs and credentials are redacted. Request URLs are diagnostic evidence and never replace the active page target. No candidates alone does not fail a successful batch or assign blame. HAR and full native diagnostics remain available through `args`.
|
|
436
440
|
|
|
437
441
|
### `stdin`
|
|
438
442
|
|
|
@@ -499,7 +503,7 @@ Example:
|
|
|
499
503
|
|
|
500
504
|
### `sessionMode`
|
|
501
505
|
|
|
502
|
-
A native `session` default from user/project JSON or `AGENT_BROWSER_SESSION` selects a caller-owned browser for direct, code, action, QA, and source tools. Per-call flags still win. Like literal `--session`, this selection takes precedence over `fresh`; it reports `usedImplicitSession: false` and is not closed when Pi quits. Without a selected native session, attachment, or explicit fresh launch, auto calls select a stable root-Pi named session and root-specific native restore key. `PI_SUBAGENT_ROOT_SESSION_ID` carries the parent identity to descendants; ordinary new/fork/clone roots differ, while resume/cwd changes keep identity. Root sessions report `usedImplicitSession: false` and survive parent/child quit. Global/override package profile names with `policy: "always"` and executable defaults bootstrap only inactive automatic roots whose effective engine is Chrome. Lightpanda receives no wrapper-supplied Chrome profile/executable defaults; caller native args, environment and config remain authoritative. Active roots preserve native launch settings/current restore key, including matching root-name follow-ups; unrelated explicit browsers are unchanged. Profile paths and project-only profile config remain advisory. Native restore covers cookies and web storage, not newly created IndexedDB or memory-only authentication. See the
|
|
506
|
+
A native `session` default from user/project JSON or `AGENT_BROWSER_SESSION` selects a caller-owned browser for direct, code, action, QA, and source tools. Per-call flags still win. Like literal `--session`, this selection takes precedence over `fresh`; it reports `usedImplicitSession: false` and is not closed when Pi quits. Without a selected native session, attachment, or explicit fresh launch, auto calls select a stable root-Pi named session and root-specific native restore key. `PI_SUBAGENT_ROOT_SESSION_ID` carries the parent identity to descendants; ordinary new/fork/clone roots differ, while resume/cwd changes keep identity. Root sessions report `usedImplicitSession: false` and survive parent/child quit. Global/override package profile names with `policy: "always"` and executable defaults bootstrap only inactive automatic roots whose effective engine is Chrome. Lightpanda receives no wrapper-supplied Chrome profile/executable defaults; caller native args, environment and config remain authoritative. Active roots preserve native launch settings/current restore key, including matching root-name follow-ups; unrelated explicit browsers are unchanged. Profile paths and project-only profile config remain advisory. Native restore covers cookies and web storage, not newly created IndexedDB or memory-only authentication. See the detailed reference for communal bootstrap and persistent-profile fallback. Active older/fresh managed sessions retain the lifecycle below. `agent_browser_qa` with `attached: true` can inspect this shared current session using the same live target checks. See [shared browser defaults](REFERENCE.md#shared-browser-defaults).
|
|
503
507
|
|
|
504
508
|
- type: `"auto" | "fresh"`
|
|
505
509
|
- optional
|
|
@@ -644,7 +648,7 @@ Ref preflight details (command taxonomy in `extensions/agent-browser/lib/command
|
|
|
644
648
|
|
|
645
649
|
- **Spelling and operands:** `@eN`, `eN`, and `ref=eN` share the same stale-ref checks, including inside raw and stdin batches. Only upstream ref-resolving selector slots are considered; `get count` and `diff snapshot --selector` keep their CSS/XPath or CSS semantics, while literal text and key/mouse data are not refs.
|
|
646
650
|
- **URL alignment:** `refSnapshot.target.url` and the session’s current tab URL are compared via `targetsMatch` / `normalizeComparableUrl` in `extensions/agent-browser/index.ts`: values are trimmed, parsed as URLs when possible, compared **after dropping the `#fragment`**, and the query string remains significant. If either side lacks a `url`, `targetsMatch` treats the pair as matching so early-session calls are not blocked.
|
|
647
|
-
- **Batch
|
|
651
|
+
- **Batch ordering:** effective raw-argument or stdin `batch` steps are scanned in order. Any step whose first token satisfies `isRefInvalidatingBatchCommand` sets a latch that blocks later steps whose first token satisfies `isRefGuardedCommand` and that mention `@e…` refs, except for same-snapshot native form-control steps whose current snapshot role metadata identifies all refs as safe controls (`check`/`uncheck` or direct `click`/`tap` on checkbox or radio refs, and `select` on combobox refs). A step whose first token is `snapshot` clears that latch for subsequent steps (pre-spawn intent only; it does not wait for upstream success). The scan continues after every snapshot, even when no refs precede it: `snapshot -i`, a non-form `click @e1`, then `get value @e2` is rejected before spawn unless another snapshot separates the click from the getter. These predicates read explicit command capability flags from `command-taxonomy.ts`: navigation/mutation verbs such as `open` / `goto`, `reload`, non-form `click`, and related upstream commands have `invalidatesBatchRefs`, and `record start` steps (any outcome), `record restart` steps with a URL operand, plus WebMCP `invoke` / `result` / `cancel` steps also set the latch because upstream swaps or navigates the active page; same-snapshot `fill` rows and the role-checked native form-control rows stay guarded against missing/stale refs but do not set the latch, allowing ordinary form batches before a final click/submit step. Direct `click`/`tap @e…` is only treated as a safe form-control row when every ref in that step is a latest-snapshot checkbox or radio; other click/tap refs remain invalidating. Ref-guarded commands accept page-scoped refs for interaction (`click`, `fill`, `download`, `scrollintoview` / `scrollinto`, and others centralized in the command taxonomy). Changing either capability requires updating this contract, [`docs/SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md) `RQ-0072`/`RQ-0087` notes, README and command-reference pitfalls, and `test/agent-browser.extension-ref-guards.test.ts`.
|
|
648
652
|
|
|
649
653
|
**Presentation redaction (implementation map):** Successful non-`batch` tool calls and each successful `batchSteps[]` row run upstream `data` through `redactPresentationData` in `extensions/agent-browser/lib/results/presentation/diagnostics.ts`: `cookies` still walk objects/arrays and replace case-insensitive `value` keys with `"[REDACTED]"`; `storage` redacts values when the key or value looks credential-like (token, cookie, auth, secret, JWT, bearer/basic credential, high-entropy token-like string, or nested sensitive JSON) but keeps low-risk primitive QA values such as booleans, numbers, and short strings visible. Redacted storage entries add `valueRedacted` plus `valueRedactionReason` in `details.data`; diagnostic formatters mirror the same decision. Every other command’s payload is recursively scrubbed with the shared `redactSensitiveValue`, which redacts known sensitive key names and applies string-level sensitivity heuristics so network, diff, trace/profiler, stream, dashboard, chat, and other structured results do not echo bearer tokens, proxy credentials, or similar fields verbatim into `details.data`. Echoed `command` arrays in `details` and in batch roll-ups use `redactInvocationArgs` from `extensions/agent-browser/lib/runtime.ts` to mask trailing values for sensitive global flags (including `--body`, `--headers`, `--password`, and `--proxy`), preserve the special positional rules for `cookies set`, `storage local|session set`, and `set credentials` (native `cookies set --curl <file>` imports are file operands, so ordinary import paths remain visible in echoes and failed batch diagnostics), and scrub other argv tokens for URLs and inline secrets. Raw batch row strings are parsed and command-redacted for presentation and ordered intent/completion journals; execution argv remains unchanged. Failed batch steps additionally run `redactExactValues` on structured step errors so literals taken from that step’s argv (cookie value, storage set value, `--password` / `--password=` tokens) cannot reappear inside formatted error blobs. When the full batch is large enough to need its own aggregate spill, that spill reapplies these per-command data and argv redactors before persistence rather than using generic batch redaction.
|
|
650
654
|
|
|
@@ -755,7 +759,7 @@ Additional structured fields can appear when relevant:
|
|
|
755
759
|
- `networkRequestsPageFilter` after wrapper-side `network requests --current-page`, `--current-origin`, or `--current-url`. Shape: `{ cleanArgs, currentUrl, mode, matchedRows, totalRows }`; the visible rows and `details.data.requests` / `items` / `entries` are filtered while the active session page target is read with `get url`.
|
|
756
760
|
- `richInputRecovery` after a raw `find` or compiled `agent_browser_action` `fill` fails with `selector-not-found` and the same current-ref diagnostic finds exact editable `searchbox` / `textbox` candidates. Shape follows `RichInputRecoveryDiagnostic` in `extensions/agent-browser/lib/results/selector-recovery.ts`: `{ candidates, inputMethodHint, nextActionIds, summary, target }`, where each candidate has `ref`, `role`, `name`, `focusArgs`, `clickArgs`, and `reason`. Visible text appends `Rich input recovery`, and `details.nextActions` gains ids from `getAgentBrowserRichInputRecoveryNextActionIds`: `focus-current-editable-ref` / `click-current-editable-ref` (or numbered variants). These actions are bounded to focus/click/inspect-style recovery: they do not include the fill text, do not press `Enter`, and do not submit. After the right current editable ref is focused, use `keyboard type` for framework-controlled editors that require real key events. Use paste-like `keyboard inserttext` only with separate application-state verification, and submit only when explicitly required by the flow.
|
|
757
761
|
- unsupported `scrollintoview text=<label>` / `scrollinto text=<label>` fails before upstream dispatch, directly or inside an effective raw/stdin batch row, because current upstream can report success without moving the page; help forms pass through unchanged. Visible failure text and `details.nextActions` both return the session-scoped `scroll-semantic-text-target` (`find text <label> hover`) when hover side effects are acceptable, plus `refresh-refs-for-scroll-target` (`snapshot -i`) before `scrollintoview <@ref>`. CSS, `xpath=...`, and current `@e…` targets remain native pass-through.
|
|
758
|
-
- `scrollPage` when the wrapper moves `document.scrollingElement` directly for `scroll <up|down|left|right> [px|percent]` or `scroll to end|top`; it temporarily disables smooth scrolling so immediate before/after offsets are reliable, returns `{ request, result }`, and includes `exitCode: 0` on success. Directional document no-movement falls through to upstream wheel behavior so nested panes still work. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` before measuring movement so smooth-scroll CSS cannot cause a premature no-movement failure, and report `details.scrollContainer`. All scroll helper shims are skipped when startup-scoped flags are present so the requested browser/profile launches before any helper command.
|
|
762
|
+
- `scrollPage` when the wrapper moves `document.scrollingElement` directly for `scroll <up|down|left|right> [px|percent]` or `scroll to end|top`; it temporarily disables smooth scrolling so immediate before/after offsets are reliable, returns `{ request, result }`, and includes `exitCode: 0` on success. Directional document no-movement falls through to upstream wheel behavior so nested panes still work. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` before measuring movement so smooth-scroll CSS cannot cause a premature no-movement failure, and report `details.scrollContainer`. Shims recognize only complete command shapes: exactly three tokens for `scroll to end|top`, or three/four for CSS-container scroll. Surplus operands, such as `scroll to end --selector #panel`, pass through to native agent-browser without a shim discarding them. All scroll helper shims are skipped when startup-scoped flags are present so the requested browser/profile launches before any helper command.
|
|
759
763
|
- `scrollNoop` after a nominally successful large **top-level** upstream scroll fallback on an existing or fresh managed session when wrapper-side read-only probes before and after the command show no change in `window.scrollX` / `window.scrollY` and no change in the sampled prominent scrollable containers. The wrapper reclassifies this outcome as `failureCategory: "upstream-error"` rather than claiming the page scrolled. To avoid pre-launching a session without caller startup state, this probe is skipped for small pixel scrolls, calls that would create a managed session only for the probe, and invocations with startup-scoped flags such as `--profile`, `--state`, `--restore`, `--namespace`, `--session-name`, `--cdp`, providers, init scripts, or similar launch settings. Shape: `{ reason: "no-observed-scroll-position-change", message, before, after, recommendations }`; `before` / `after` include viewport dimensions, document scroll dimensions, and up to ten sampled container descriptors plus scroll offsets. Container descriptors use only sample index, tag name, and ARIA role; DOM ids/classes are intentionally not stored. This diagnostic is conservative evidence that the page-level scroll likely missed a nested pane, not proof that every app-specific region is unchanged. Visible text starts with `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, sets `details.data.scrolled` to `false` / `details.data.noMovement` to `true`, and `details.nextActions` gains `inspect-after-noop-scroll` (`snapshot -i`) plus `verify-noop-scroll-visually` (`screenshot`), session-prefixed when applicable.
|
|
760
764
|
- `comboboxFocus` after a successful explicit combobox-targeted `click` / `fill` / `find … click|fill` (for example `agent_browser_action` with role `combobox`, including when that semantic action resolves through a current visible `@ref` before execution) when a read-only probe sees the active element is combobox-like, `aria-expanded` is explicitly present (`false` or `true`), and no visible `listbox` / `option` / menu option elements are open. Shape: `{ reason: "focused-combobox-without-visible-options", message, activeElement, visibleListboxCount, visibleOptionCount, recommendations }`; `activeElement` includes bounded role/tag/expanded/hasPopup/name metadata with normal text redaction. Visible text appends `Combobox diagnostic: focused combobox did not expose visible options`, and `details.nextActions` gains `inspect-focused-combobox` (`snapshot -i`), `try-open-combobox-with-arrow` (`press ArrowDown`), and `try-open-combobox-with-enter` (`press Enter`), session-prefixed when applicable. The diagnostic is deliberately gated to explicit combobox-targeted calls to avoid extra probes or false positives on ordinary clicks/textboxes.
|
|
761
765
|
- `recordingDependencyWarning` after a successful `record start` or `record restart` when the wrapper cannot find an executable `ffmpeg` on the Pi process `PATH`. Shape: `{ reason: "ffmpeg-missing-for-recording", dependency: "ffmpeg", command, message, recommendations }`. Visible text appends `Recording dependency warning: ffmpeg not found on PATH`. This is a non-blocking warning after native success, not a preflight or encoding check. Native 0.37 validates ffmpeg at startup; older supported natives may defer failure. Treat the pending output as unverified, stop and check its result, then install ffmpeg before starting a new recording.
|
package/docs/platform-smoke.md
CHANGED
|
@@ -42,7 +42,7 @@ npm run verify -- platform-smoke run --target ubuntu --suite platform-build
|
|
|
42
42
|
|
|
43
43
|
With maintainer approval, local macOS execution can replace localhost SSH. Routine GitHub Actions has no Windows job; the separate Crabbox/Parallels release target remains as documented above. Record alternate native checks separately from Crabbox SSH or Parallels results. The default `release` and `prepublishOnly` commands still select the Crabbox matrix; record alternate suite evidence separately rather than reporting those commands as passed.
|
|
44
44
|
|
|
45
|
-
On macOS, run the unchanged commands returned by `buildPlatformBuildCommand('macos', 'pi-agent-browser-native', '24.21.0')` and `buildBrowserDogfoodCommand('macos', '0.38.1', true)` in [`scripts/platform-smoke/targets.mjs`](../scripts/platform-smoke/targets.mjs), serially in a clean private source copy. Use private HOME, npm and Pi settings, and headless browser profiles. No Remote Login or host security change is required.
|
|
45
|
+
On macOS, run the unchanged commands returned by `buildPlatformBuildCommand('macos', 'pi-agent-browser-native', '24.21.0')` and `buildBrowserDogfoodCommand('macos', '0.38.1', true)` in [`scripts/platform-smoke/targets.mjs`](../scripts/platform-smoke/targets.mjs), serially in a clean private source copy. Use private HOME, npm and Pi settings, and headless browser profiles, and clear inherited `PI_*` variables first: a shell started from Pi exports `PI_PACKAGE_DIR` and `PI_MANAGED_CLI`, which make the archived Pi CLI report the host package instead of the locked suite version. No Remote Login or host security change is required.
|
|
46
46
|
|
|
47
47
|
Retain the exact source head/tree, package and tool versions, native OS/architecture, generated commands, stdout/stderr, every `PLATFORM_*` exit marker, actual Pi registration output, dogfood JSON and verified screenshot. Check the suite assertions below and clean up only the run's processes and temporary files. These alternatives do not waive Ubuntu or any non-platform release check.
|
|
48
48
|
|
|
@@ -80,7 +80,7 @@ PLATFORM_SMOKE_WINDOWS_WORK_ROOT="C:\\crabbox\\pi-agent-browser-native"
|
|
|
80
80
|
PLATFORM_SMOKE_AUTH_ENV=""
|
|
81
81
|
```
|
|
82
82
|
|
|
83
|
-
The Ubuntu target image is derived from `node:24-bookworm`, installs the `agent-browser` version from [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs), installs Debian Chromium plus the upstream Linux WebGPU/Xvfb runtime packages (`libvulkan1`, `mesa-vulkan-drivers`, and `xvfb`) through apt, creates a non-root `circleci` user, and sets `AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium`. Rebuild it after upstream rebaselining, or override `PLATFORM_SMOKE_UBUNTU_IMAGE` with an equivalent prepared local image. Do not install `agent-browser` ad hoc inside the Ubuntu smoke command; a missing tool is image/template drift.
|
|
83
|
+
The Ubuntu target image is derived from `node:24-bookworm`, installs the `agent-browser` version from [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs), installs Debian Chromium plus the upstream Linux WebGPU/Xvfb runtime packages (`libvulkan1`, `mesa-vulkan-drivers`, and `xvfb`) through apt, copies the official Go 1.27.1 toolchain onto `PATH` so `platform-target` can build the [corrected quality checker](CODE_QUALITY.md), creates a non-root `circleci` user, and sets `AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium`. Rebuild it after upstream rebaselining, or override `PLATFORM_SMOKE_UBUNTU_IMAGE` with an equivalent prepared local image. Do not install `agent-browser` ad hoc inside the Ubuntu smoke command; a missing tool is image/template drift.
|
|
84
84
|
|
|
85
85
|
The configured upstream `agent-browser` baseline is imported from [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs). Target-local browser suites verify that exact `agent-browser` version before running. Bake the exact upstream CLI and browser runtime into the Windows template/snapshot for speed and reproducibility; missing or stale Windows `agent-browser` / browser readiness is a blocked setup, not something the smoke command repairs. The Windows browser suite checks the preinstalled browser cache and prewarms one short loopback HTTP URL before the extension harness runs.
|
|
86
86
|
|
|
@@ -89,7 +89,7 @@ The configured upstream `agent-browser` baseline is imported from [`scripts/agen
|
|
|
89
89
|
Crabbox does not install project runtime tools. The macOS host, Ubuntu image, and Windows template must already provide:
|
|
90
90
|
|
|
91
91
|
- Node/npm at or above the full Node version declared by `package.json` `engines.node` and used by [`platform-smoke.config.mjs`](../platform-smoke.config.mjs).
|
|
92
|
-
- Git and `
|
|
92
|
+
- Git, `tar`, and Go (the version `scripts/prepare-quality-checker.mjs` selects) for the `platform-target` corrected-checker build.
|
|
93
93
|
- Upstream `agent-browser` matching this wrapper’s capability baseline. The Ubuntu target gets it from [`scripts/platform-smoke/linux-image/Dockerfile`](../scripts/platform-smoke/linux-image/Dockerfile); the Windows template gets it from the shared `pi-extension-windows-template` snapshot named in [`platform-smoke.config.mjs`](../platform-smoke.config.mjs) (currently `crabbox-ready-ab-0.33.0`, a child of the shared `crabbox-ready` base used by other projects' linked clones).
|
|
94
94
|
- Browser/runtime dependencies needed by upstream `agent-browser`.
|
|
95
95
|
- Native PowerShell and OpenSSH Server on Windows.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-agent-browser-native",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.5",
|
|
4
4
|
"description": "pi extension that exposes agent-browser as a native tool for browser automation",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"author": "Mitch Fultz (https://github.com/fitchmultz)",
|
|
@@ -56,6 +56,7 @@
|
|
|
56
56
|
"docs/COMMAND_REFERENCE.md",
|
|
57
57
|
"docs/ELECTRON.md",
|
|
58
58
|
"docs/RELEASE.md",
|
|
59
|
+
"docs/REFERENCE.md",
|
|
59
60
|
"docs/SESSION_STORAGE.md",
|
|
60
61
|
"docs/platform-smoke.md",
|
|
61
62
|
"docs/REQUIREMENTS.md",
|
|
@@ -113,7 +114,7 @@
|
|
|
113
114
|
"smoke:platform:all": "npm run smoke:platform:doctor && node scripts/platform-smoke.mjs run --target macos,ubuntu,windows-native",
|
|
114
115
|
"check:compat": "node scripts/compat-host.mjs && npm run typecheck && npm test && node scripts/verify-package.mjs --smoke-pi",
|
|
115
116
|
"typecheck": "node ./scripts/project.mjs verify typecheck",
|
|
116
|
-
"test": "node ./scripts/build.mjs && tsx --test --test-reporter=tap --test-concurrency=1 \"test/**/*.test.ts\"",
|
|
117
|
+
"test": "node ./scripts/build.mjs && node ./scripts/prepare-quality-checker.mjs && tsx --test --test-reporter=tap \"test/**/!(*.serial).test.ts\" && tsx --test --test-reporter=tap --test-concurrency=1 \"test/**/*.serial.test.ts\"",
|
|
117
118
|
"verify": "node ./scripts/project.mjs verify",
|
|
118
119
|
"prepublishOnly": "npm run verify -- release && npm pack --dry-run",
|
|
119
120
|
"build": "node ./scripts/build.mjs",
|