pi-agent-browser-native 0.9.4 → 0.9.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/CHANGELOG.md +29 -0
  2. package/README.md +58 -813
  3. package/dist/extensions/agent-browser/index.js +927 -883
  4. package/dist/extensions/agent-browser/lib/browser-execution-claims.js +5 -5
  5. package/dist/extensions/agent-browser/lib/config-policy.js +1 -1
  6. package/dist/extensions/agent-browser/lib/config-validation.js +21 -8
  7. package/dist/extensions/agent-browser/lib/config.js +1 -1
  8. package/dist/extensions/agent-browser/lib/electron/cdp.js +5 -1
  9. package/dist/extensions/agent-browser/lib/electron/cleanup.js +13 -47
  10. package/dist/extensions/agent-browser/lib/electron/discovery-evidence.js +1 -0
  11. package/dist/extensions/agent-browser/lib/electron/launch.js +12 -1
  12. package/dist/extensions/agent-browser/lib/fs-utils.js +1 -9
  13. package/dist/extensions/agent-browser/lib/input-modes/job.js +7 -4
  14. package/dist/extensions/agent-browser/lib/input-modes/lookup-network-evidence.js +15 -3
  15. package/dist/extensions/agent-browser/lib/input-modes/lookups.js +3 -2
  16. package/dist/extensions/agent-browser/lib/input-modes/params.js +23 -23
  17. package/dist/extensions/agent-browser/lib/orchestration/browser-run/electron-observation.js +9 -8
  18. package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result-electron.js +8 -0
  19. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/scroll-shims.js +2 -2
  20. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +9 -0
  21. package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-electron.js +4 -2
  22. package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state-refs.js +9 -11
  23. package/dist/extensions/agent-browser/lib/orchestration/electron-host/branch.js +1 -0
  24. package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-collection.js +5 -6
  25. package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-data.js +4 -2
  26. package/dist/extensions/agent-browser/lib/orchestration/electron-host/probe-presentation.js +20 -2
  27. package/dist/extensions/agent-browser/lib/orchestration/electron-host/status.js +4 -2
  28. package/dist/extensions/agent-browser/lib/orchestration/execution-queue.js +1 -1
  29. package/dist/extensions/agent-browser/lib/results/presentation/skills.js +7 -2
  30. package/dist/extensions/agent-browser/lib/temp-artifact-budget.js +4 -1
  31. package/dist/extensions/agent-browser/lib/temp-root.js +22 -18
  32. package/dist/extensions/agent-browser/lib/temp.js +1 -1
  33. package/dist/extensions/agent-browser/lib/web-search-providers.js +0 -36
  34. package/dist/extensions/agent-browser/lib/web-search-request.js +20 -26
  35. package/dist/extensions/agent-browser/lib/web-search-schema.js +5 -5
  36. package/dist/extensions/agent-browser/lib/web-search.js +22 -4
  37. package/docs/ARCHITECTURE.md +3 -2
  38. package/docs/CODE_QUALITY.md +28 -3
  39. package/docs/COMMAND_REFERENCE.md +20 -20
  40. package/docs/ELECTRON.md +24 -2
  41. package/docs/REFERENCE.md +869 -0
  42. package/docs/RELEASE.md +9 -9
  43. package/docs/REQUIREMENTS.md +1 -1
  44. package/docs/SUPPORT_MATRIX.md +9 -8
  45. package/docs/TOOL_CONTRACT.md +13 -9
  46. package/docs/platform-smoke.md +3 -3
  47. package/package.json +3 -2
  48. package/scripts/config.mjs +23 -19
  49. package/scripts/doctor.mjs +2 -2
  50. package/scripts/platform-smoke/lease-evidence.mjs +2 -4
  51. package/scripts/platform-smoke/linux-image/Dockerfile +8 -2
  52. package/scripts/platform-smoke/targets.mjs +1 -5
@@ -32,6 +32,27 @@ Review fixes before committing; dangerous fixes and suggestion fixes are not par
32
32
  `lint:fix`. Run formatting after reviewed lint fixes and rerun acceptance. Failures
33
33
  are never converted to success by these commands.
34
34
 
35
+ ### Test scheduling
36
+
37
+ `npm test` builds once and prepares the corrected checker once, then runs
38
+ `tsx --test "test/**/!(*.serial).test.ts"` with Node's default file concurrency, followed by
39
+ `tsx --test --test-concurrency=1 "test/**/*.serial.test.ts"`.
40
+ The default/quality verification gates use the same two stages. File isolation
41
+ keeps environment patches process-local; the shared browser harness supplies a
42
+ private short socket directory unless `PI_AGENT_BROWSER_SOCKET_DIR` is already set.
43
+ The harness also clears inherited Pi/delegation identity variables; root-default
44
+ tests set their identities explicitly after import. No in-file concurrency is enabled.
45
+
46
+ The serial lane contains `test/agent-browser.startup.serial.test.ts` and
47
+ `test/agent-browser.prompt-policy.serial.test.ts` for wall-clock budgets, plus
48
+ `test/agent-browser.managed-session-policy-lock.serial.test.ts` for fixtures in
49
+ the shared policy collector directory. Their timing and global-GC contracts must
50
+ not compete with the parallel lane. Build/pack fixtures must use disposable source trees rather
51
+ than rewriting this checkout's `dist/` while other files consume it.
52
+ `platform-target` retains its explicit serial subset; `quality:probes` remains a
53
+ focused serial checker suite. The allowance-isolation mutation must fail in the
54
+ parallel stage with its named `code-quality.test.ts` diagnostic.
55
+
35
56
  ## Language and role scope
36
57
 
37
58
  The root `.oxlintrc.json` enables type-aware rules and compiler diagnostics. The
@@ -153,9 +174,13 @@ Non-generic TS/JSDoc array aliases retain exact declared identity without introd
153
174
  wrapper types. Built-in browser `URL` and Node's `node:url` `URL` have distinct
154
175
  origins; the filesystem `PathLike` contract requires the Node-qualified allowance.
155
176
 
156
- `test/code-quality.test.ts` exercises actual language/compiler scope, consumer
157
- resolution, test-role boundaries, framework diagnostics, declaration isolation,
158
- mutable intersections, suppression policy, and formatting idempotence. Probes
177
+ The real-CLI probes are split by contract: `test/code-quality.test.ts` covers
178
+ framework diagnostics and qualified registration isolation;
179
+ `test/code-quality-scope.test.ts` covers language/compiler scope, consumer resolution,
180
+ test-role boundaries, suppression policy, and formatting idempotence;
181
+ `test/code-quality-subtests.test.ts` covers subtest promise ownership and nested
182
+ registration; `test/code-quality-readonly.test.ts` covers declaration isolation and
183
+ mutable additions/intersections. `npm run quality:probes` runs all four files. Probes
159
184
  check expected rule identities; setup/configuration/parser failures are not valid
160
185
  negative evidence. Keep minimal unsuppressed checker reproductions when upgrading
161
186
  the backend and retire the patch only after the same controls pass.
@@ -348,7 +348,7 @@ For desktop, contenteditable, or host-controlled rich inputs, treat a semantic `
348
348
 
349
349
  Do not assume Playwright selector dialects such as `text=Close` or `button:has-text('Close')` are supported wrapper syntax. In particular, current upstream can report successful `scrollintoview text=...` without moving the page, so the wrapper rejects that form before dispatch—directly or in an effective raw/stdin batch row—and shows executable `find text <label> hover` plus snapshot/ref recovery payloads in visible failure text and `details.nextActions`. `scrollintoview ... --help` and `-h` remain native help calls. Use `scrollintoview` with CSS, `xpath=...`, or a current `@e…` ref; use `find` for semantic text targets.
350
350
 
351
- Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside `batch` stdin JSON, the wrapper also walks steps in order before spawn: steps whose first token can navigate or mutate set a latch; a later step whose first token is `snapshot` clears that latch for following rows; guarded steps that still mention `@e…` after an uncleared latch fail with the same `stale-ref` bucket without launching upstream. Same-snapshot form fills and native form-control steps are allowed before a click or submit step, so `fill`, `check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, `select` combobox refs, then a final submit `click` can run from one snapshot. Split dynamic or autosubmit forms with a fresh snapshot if a control interaction rerenders the targets. Follow the `refresh-interactive-refs` next action (it includes `--session <name>` when needed) and prefer stable `find` or `agent_browser_action` locators when navigation or rerendering is likely. Contract detail: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) (`refSnapshot`, `refSnapshotInvalidation`).
351
+ Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside effective raw-argument or stdin `batch` steps, the wrapper also walks the full plan in order before spawn: steps whose first token can navigate or mutate set a latch; a later step whose first token is `snapshot` clears that latch for following rows; guarded steps that still mention `@e…` after an uncleared latch fail with the same `stale-ref` bucket without launching upstream. This scan continues after a snapshot even when no refs appear before it: `snapshot -i`, a non-form `click @e1`, then `get value @e2` is rejected. Put another `snapshot -i` after the click or split the batch and inspect before continuing. Same-snapshot form fills and native form-control steps are allowed before a click or submit step, so `fill`, `check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, `select` combobox refs, then a final submit `click` can run from one snapshot. Split dynamic or autosubmit forms with a fresh snapshot if a control interaction rerenders the targets. Follow the `refresh-interactive-refs` next action (it includes `--session <name>` when needed) and prefer stable `find` or `agent_browser_action` locators when navigation or rerendering is likely. Contract detail: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) (`refSnapshot`, `refSnapshotInvalidation`).
352
352
 
353
353
  A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. For both ref and XPath candidates, the wrapper requires native `get attr <original-selector> <marker>` to read back a per-probe temporary DOM marker on that candidate. Missing or mismatched identity (including frame/XPath scope mismatches) cleans up the probe and lets the native click run without dispatch verification. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. Probe listeners and the window marker self-expire after the call's process-timeout budget plus the existing two-second cleanup grace if a pending decision blocks normal cleanup; collection cancels the timer. This uses no extra browser command while pending. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
354
354
 
@@ -410,7 +410,7 @@ Paced input uses explicit native rows, preserving the former job's behavior with
410
410
 
411
411
  For longer text, code can construct the rows and call native batch; keep the former 200-character pacing bound when porting a job. Add a final `press` only when submission is intended. Fixed waits are not postconditions. Timeout partial progress remains available through the same batch path. See [all job step equivalents](TOOL_CONTRACT.md#07-migration).
412
412
 
413
- For quick smoke/QA checks, enable `qa` and call `agent_browser_qa` with flat fields. It clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks expected text/selector, then inspects fresh network requests, console messages, and page errors only if preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped diagnostic output. If final page-error rows match a nonempty post-clear baseline, QA is non-pass with “page-error check could not be verified”: identical new errors can replace old rows in the native FIFO buffer without changing the result. Those matches are not proof of new application errors. Definitely new rows, including errors after an empty baseline, count separately as page errors; a clean final buffer can pass. Explicit `checkErrors: false` omits this check for a more limited smoke test. The preset compiles to `batch --bail` so a missing text/selector assertion fails crisply instead of letting slower diagnostics burn the wrapper watchdog. Expected text compiles to bounded visible-text `wait --fn … --timeout 5000` predicates after load so dense pages can pass on visible headings/copy without dumping `body` text; missing text reports a crisp QA failure. The readiness wait defaults to `loadState: "domcontentloaded"`; set `loadState` to `"load"` or `"networkidle"` only when that stricter state is useful and the site is not expected to keep background requests alive. QA network diagnostics classify failed requests by likely impact and list failed rows first in the network preview: actionable document/script/API-style failures fail the preset, while common low-impact browser icon misses such as `favicon.ico` are surfaced as warnings (`qaPreset.warnings`) so they do not fail an otherwise healthy page. Successful QA with no failed checks returns compact model-visible prose (page URL/title when known, checks run, optional screenshot verification) while keeping the full step matrix in `details.qaPreset` and `details.batchSteps`. Failed QA presets report `details.resultCategory: "failure"`, `failureCategory: "qa-failure"`, show compact failure reasons with the full matrix in `details.batchSteps`, and real Pi sessions treat the diagnostic as a failed tool result. Prose output also gets a model-visible result-category line including `Pi tool isError: true`; caller-requested `--json` output keeps the JSON string parseable and relies on the patched `isError` plus `details` fields.
413
+ For quick smoke/QA checks, enable `qa` and call `agent_browser_qa` with flat fields. It clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks expected text/selector, then inspects fresh network requests, console messages, and page errors only if preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped diagnostic output. If final page-error rows match a nonempty post-clear baseline, QA is non-pass with “page-error check could not be verified”: identical new errors can replace old rows in the native FIFO buffer without changing the result. Those matches are not proof of new application errors. Definitely new rows, including errors after an empty baseline, count separately as page errors; a clean final buffer can pass. Explicit `checkErrors: false` omits this check for a more limited smoke test. The preset compiles to `batch --bail` so a missing text/selector assertion fails crisply instead of letting slower diagnostics burn the wrapper watchdog. Expected text compiles to bounded visible-text `wait --fn … --timeout 5000` predicates after load so dense pages can pass on visible headings/copy without dumping `body` text; missing text reports a crisp QA failure. Text and control values under `display: none` or `opacity: 0` ancestors, or whose computed visibility is hidden, are excluded even when the control itself has a visible layout box; `display: contents` wrappers and `visibility: visible` overrides still count. The readiness wait defaults to `loadState: "domcontentloaded"`; set `loadState` to `"load"` or `"networkidle"` only when that stricter state is useful and the site is not expected to keep background requests alive. QA network diagnostics classify failed requests by likely impact and list failed rows first in the network preview: actionable document/script/API-style failures fail the preset, while common low-impact browser icon misses such as `favicon.ico` are surfaced as warnings (`qaPreset.warnings`) so they do not fail an otherwise healthy page. Successful QA with no failed checks returns compact model-visible prose (page URL/title when known, checks run, optional screenshot verification) while keeping the full step matrix in `details.qaPreset` and `details.batchSteps`. Failed QA presets report `details.resultCategory: "failure"`, `failureCategory: "qa-failure"`, show compact failure reasons with the full matrix in `details.batchSteps`, and real Pi sessions treat the diagnostic as a failed tool result. Prose output also gets a model-visible result-category line including `Pi tool isError: true`; caller-requested `--json` output keeps the JSON string parseable and relies on the patched `isError` plus `details` fields.
414
414
 
415
415
  For failed QA, inspect `details.qaPreset.failedChecks` separately from `notRunChecks`: a reported fail-fast error leaves later checks not run, rather than proving their assertions failed. Failure prose leads with the redacted cause and reports received execution coverage; missing timeout receipts mean an unknown executed count. Incomplete receipts without a reported failure are unverified, never proof of a pass or of unreached checks.
416
416
 
@@ -481,7 +481,7 @@ For local app debugging, enable `source`; `agent_browser_source` can gather cand
481
481
  { "selector": "#save", "reactFiberId": "2", "componentName": "SaveButton" }
482
482
  ```
483
483
 
484
- Enable `network` and call `agent_browser_network_source` for failed browser requests. `requestId` adds `network request <requestId>`; `filter` or `url` adds `network requests --filter …`, with explicit filter winning. Optional `namespace` and `session` select the native target; `namespace: ""` overrides an ambient namespace. HTTP status ≥ 400, `failed: true`, and error fields identify failed rows. Filter/URL limits correlation by substring. Workspace URL-literal search uses the captured execution directory and the source tool's bounds: 2,000 files by default, 5,000 maximum, eight needles and ten matches. Results retain `details.networkSourceLookup` with status, failed requests, candidate evidence, and limitations. They never assign definitive blame or replace the active page target and same-page refs.
484
+ Enable `network` and call `agent_browser_network_source` for failed browser requests. `requestId` adds `network request <requestId>`; `filter` or `url` adds `network requests --filter …`, with explicit filter winning. Optional `namespace` and `session` select the native target; `namespace: ""` overrides an ambient namespace. HTTP status ≥ 400, `failed: true`, and error fields identify failed rows. The same explicit-filter-wins query limits failed-request and initiator analysis by substring; unrelated failed-request initiators are ignored. Repeated request IDs count once, keeping the first matching failed row; anonymous rows and different IDs sharing one URL remain distinct. Workspace URL-literal search uses the captured execution directory and the source tool's bounds: 2,000 files by default, 5,000 maximum, eight needles and ten matches. Results retain `details.networkSourceLookup` with status, failed requests, candidate evidence, and limitations. They never assign definitive blame or replace the active page target and same-page refs.
485
485
 
486
486
  ```json
487
487
  { "requestId": "req-1", "url": "/api/fail" }
@@ -544,7 +544,7 @@ Wrapper result rendering is metadata-first for saved files. Image MIME types com
544
544
 
545
545
  `diff screenshot` follows the file-artifact path above for the **diff** image: model-visible text and `details.artifacts` focus on that output, while baseline paths stay out of the artifact summary block, and Pi does **not** auto-inline the diff the way it inlines trusted `screenshot` captures. `state load` may print the loaded path in prose but does not add a saved-file artifact entry the way `state save` does.
546
546
 
547
- For screenshot paths under dot-directories such as `.dogfood/run/foo.png`, the wrapper normalizes the requested path to an absolute path before invoking upstream `agent-browser`, verifies the requested file exists, and repairs from an upstream temp screenshot when possible. For direct artifact commands and batch artifact steps (`download`, `pdf`, `screenshot`, `state save`, and `wait --download`), the wrapper creates missing parent directories before launch. A parent-directory failure returns `validation-error` with the attempted directory and `verify-artifact-path` guidance before the browser command runs. Prefer **absolute paths in raw batch artifact rows** because the daemon's working directory may differ from Pi's. When `pi-change-working-dir` selects a different execution root, the wrapper binds relative file operands in direct, stdin, and raw batch rows to that invocation's root; upstream's raw-argument precedence over stdin is unchanged. Otherwise raw strings stay unchanged and screenshot path normalization applies only to direct calls and stdin rows. See [working-directory changes](../README.md#working-directory-changes). Known caller paths appear as `Requested path`; `Absolute path` is the resolved location checked on disk, and `Reported path` exposes a differing screenshot report (including a canonical `/private/tmp` alias) through the existing `tempPath` metadata. No extra canonicalization rewrites upstream arguments.
547
+ For screenshot paths under dot-directories such as `.dogfood/run/foo.png`, the wrapper normalizes the requested path to an absolute path before invoking upstream `agent-browser`, verifies the requested file exists, and repairs from an upstream temp screenshot when possible. For direct artifact commands and batch artifact steps (`download`, `pdf`, `screenshot`, `state save`, and `wait --download`), the wrapper creates missing parent directories before launch. A parent-directory failure returns `validation-error` with the attempted directory and `verify-artifact-path` guidance before the browser command runs. Prefer **absolute paths in raw batch artifact rows** because the daemon's working directory may differ from Pi's. When `pi-change-working-dir` selects a different execution root, the wrapper binds relative file operands in direct, stdin, and raw batch rows to that invocation's root; upstream's raw-argument precedence over stdin is unchanged. Otherwise raw strings stay unchanged and screenshot path normalization applies only to direct calls and stdin rows. See [working-directory changes](REFERENCE.md#working-directory-changes). Known caller paths appear as `Requested path`; `Absolute path` is the resolved location checked on disk, and `Reported path` exposes a differing screenshot report (including a canonical `/private/tmp` alias) through the existing `tempPath` metadata. No extra canonicalization rewrites upstream arguments.
548
548
 
549
549
  For annotated screenshots in `batch`, put `--annotate` in top-level args instead of inside the screenshot step:
550
550
 
@@ -663,7 +663,7 @@ The command groups below intentionally list more than the recommended workflow.
663
663
 
664
664
  ### Built-in skills
665
665
 
666
- Native-tool note: upstream skills are written for the standalone `agent-browser` CLI and may show bash/heredoc examples. In pi, convert those examples to `agent_browser` calls: pass CLI tokens in `args`, and pass heredoc/stdin bodies through the tool `stdin` field for `batch`, `eval --stdin`, or `auth save --password-stdin`.
666
+ Native-tool note: upstream skills are written for the standalone `agent-browser` CLI and may show bash/heredoc examples. In pi, convert those examples to `agent_browser` calls: pass CLI tokens in `args`, and pass heredoc/stdin bodies through the tool `stdin` field for `batch`, `eval --stdin`, or `auth save --password-stdin`. The wrapper's copyable skill calls preserve quoted empty and whitespace-only operands: `fill '#name' ""` becomes `args: ["fill", "#name", ""]`, not a missing value.
667
667
 
668
668
  Session note: `skills list`, `skills get …`, and `skills path …` are **stateless** in this wrapper. Even with default `sessionMode: "auto"`, the extension does not prepend the implicit managed `--session` for those commands, so reading bundled skills does not attach to or rotate the active browser session (same intent as plain-text `--help` / `--version` inspection). Other `skills` subcommands follow normal session rules until explicitly allowlisted in `extensions/agent-browser/lib/runtime.ts` alongside regression coverage in `test/agent-browser.runtime.test.ts`.
669
669
 
@@ -718,7 +718,7 @@ Skill-source debugging note: upstream honors `AGENT_BROWSER_SKILLS_DIR` as an ov
718
718
  - `tap <selector>` — Touch-oriented tap alias for iOS/provider workflows.
719
719
  - `swipe <direction> [distance]` — Touch-oriented swipe for iOS/provider workflows.
720
720
 
721
- On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` so smooth-scroll CSS cannot delay the measured movement, and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
721
+ On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` so smooth-scroll CSS cannot delay the measured movement, and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. These shims intercept only complete recognized shapes: `scroll to end|top` has no extra operands, and CSS-container scroll accepts only the selector, direction, and optional amount. Surplus operands such as `scroll to end --selector #panel` or `scroll #panel down 500 extra` pass through to native agent-browser unchanged rather than being silently dropped. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
722
722
 
723
723
  Comboboxes vary by app. For native `<select>` controls, prefer raw `select <selector> <value...>`, flat `agent_browser_action` select fields or active-session semantic role/name/label select instead of clicking option refs; native option refs can be non-boxed in CDP and fail before a real selection. A `click` or `agent_browser_action` role/name click may focus a searchable custom combobox without opening its option list. For explicit combobox-targeted actions such as `agent_browser_action` role `combobox`, the wrapper checks whether a combobox-like element is focused, has explicit `aria-expanded` state, and has no visible listbox/options open; this still applies when the semantic action first resolves to a current visible `@ref` before execution. When that happens it appends `Combobox diagnostic: focused combobox did not expose visible options`, exposes `details.comboboxFocus`, and adds exact `details.nextActions` for a fresh `snapshot -i`, `press ArrowDown`, and `press Enter`. Use those instead of assuming click alone expanded the control. To search the focused input, use `keyboard type <text>`; raw `type` requires both a selector and text. Reserve visible option refs for custom comboboxes after a fresh snapshot shows the intended option.
724
724
 
@@ -951,21 +951,21 @@ JSON
951
951
  `pi install` does not add package helper binaries to your shell `PATH`. Use direct JSON config edits, or run the helper only through `npm exec`:
952
952
 
953
953
  ```bash
954
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config paths
955
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config show
956
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config web-search set-env EXA_API_KEY --global
957
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config web-search set-env BRAVE_API_KEY --global
958
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config web-search set-env EXA_API_KEY --project
959
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config web-search prefer brave --global
960
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config web-search disable --global
961
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config web-search disable --project
962
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config web-search set-command "op read 'op://Private/Brave Search/API Key'" --provider brave --global
963
- printf '%s' "$EXA_API_KEY" | npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config web-search set-key --provider exa --stdin
964
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config browser profile set "Profile 1" --policy authenticated-only
965
- npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config browser executable set "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"
954
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config paths
955
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config show
956
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search set-env EXA_API_KEY --global
957
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search set-env BRAVE_API_KEY --global
958
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search set-env EXA_API_KEY --project
959
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search prefer brave --global
960
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search disable --global
961
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search disable --project
962
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search set-command "op read 'op://Private/Brave Search/API Key'" --provider brave --global
963
+ printf '%s' "$EXA_API_KEY" | npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config web-search set-key --provider exa --stdin
964
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config browser profile set "Profile 1" --policy authenticated-only
965
+ npm exec --yes --package pi-agent-browser-native -- pi-agent-browser-config browser executable set "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"
966
966
  ```
967
967
 
968
- The optional `agent_browser_web_search` tool is available when Exa or Brave credentials are visible from startup config or trusted session config and the runtime config has not set `webSearch.enabled` to `false`. It is a separate custom tool, not an `agent_browser` input mode, and does not launch a browser. Prefer it for current/live external web facts and URL discovery; use `agent_browser` for browser interaction, screenshots, authenticated/profile pages, and DOM inspection after you have a target URL. Prefer it over driving public search-engine forms such as Google with browser typing flows, which can redirect headless automation to anti-bot or CAPTCHA pages; do not attempt CAPTCHA bypass. Disable scope is explicit: `web-search disable --global` sets the normal user default, `web-search disable --project` disables it for one repo, and a `PI_AGENT_BROWSER_CONFIG` override containing `{ "version": 1, "webSearch": { "enabled": false } }` wins over both for a hard per-run disable. Loaded config may use plaintext, custom env aliases, interpolation literals, malformed-or-late-bound `$` values, and command-backed web-search keys; the resolved secret reaches the provider request while model-facing tool output and status text stay redacted. `web-search set-key`, `set-command`, and `clear` require `--provider`; `set-env` infers Exa/Brave from `EXA_API_KEY` or `BRAVE_API_KEY` unless you pass `--provider`.
968
+ The optional `agent_browser_web_search` tool is available when Exa or Brave credentials are visible from startup config or trusted session config and the runtime config has not set `webSearch.enabled` to `false`. It is a separate custom tool, not an `agent_browser` input mode, and does not launch a browser. Prefer it for current/live external web facts and URL discovery; use `agent_browser` for browser interaction, screenshots, authenticated/profile pages, and DOM inspection after you have a target URL. Prefer it over driving public search-engine forms such as Google with browser typing flows, which can redirect headless automation to anti-bot or CAPTCHA pages; do not attempt CAPTCHA bypass. Disable scope is explicit: `web-search disable --global` sets the normal user default, `web-search disable --project` disables it for one repo, and a `PI_AGENT_BROWSER_CONFIG` override containing `{ "version": 1, "webSearch": { "enabled": false } }` wins over both for a hard per-run disable. Loaded config may use plaintext, custom env aliases, interpolation literals, malformed-or-late-bound `$` values, and command-backed web-search keys; the resolved secret reaches the provider request while model-facing tool output and status text stay redacted. Malformed config files report their scope/path and `Invalid JSON`, plus the parser's numeric position/line/column when available; malformed provider JSON reports a provider-specific invalid-JSON error. Neither includes parser previews or parser causes that could echo partial credentials, and provider HTTP failures report only the status with a fixed hint. `web-search set-key`, `set-command`, and `clear` require `--provider`; `set-env` infers Exa/Brave from `EXA_API_KEY` or `BRAVE_API_KEY` unless you pass `--provider`.
969
969
 
970
970
  For Exa, effective search type precedence is per-call `searchType` → `webSearch.defaultSearchType` → `auto`, with regular `contents.highlights: true`. Typical latencies are `instant` ~250 ms, `fast` ~450 ms, `auto` ~1 second, `deep-lite` ~4 seconds, `deep` 4–15 seconds, and `deep-reasoning` 12–40 seconds. Prefer `deep-lite` for research before implementation, `deep` for hard multi-source work, and `deep-reasoning` only for exhaustive or still-thin research. Searches are serialized; do not launch several in parallel.
971
971
 
@@ -1003,7 +1003,7 @@ Example config:
1003
1003
  }
1004
1004
  ```
1005
1005
 
1006
- Browser defaults are scoped: global/override `defaultProfile` with `policy: "always"` and a Chrome profile name (not a path), plus `executablePath`, become native subprocess defaults only for automatic root browsers and their helpers when the effective engine is Chrome. Lightpanda omits these wrapper defaults, without changing caller-selected launch settings. Roots have distinct named sessions and restore keys; parents and descendants share one root through `PI_SUBAGENT_ROOT_SESSION_ID`. These defaults never apply to unrelated explicit sessions or Electron launches. Code selecting the ordinary root browser uses the same defaults. Profile paths, other policies, and project-only guidance remain advisory. Native restore keeps cookies/localStorage/sessionStorage, not new IndexedDB, service workers, or memory-only grants; verify app access after a real restart. Avoid global native profile changes or a shared writable profile path; see [shared browser defaults](../README.md#shared-browser-defaults). Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `profiles` and `doctor` when profile resolution fails, then use the reported Chrome profile directory name, a full profile/user-data directory path if upstream accepts one, or the configured `browser.executablePath` with top-level `sessionMode: "fresh"`.
1006
+ Browser defaults are scoped: global/override `defaultProfile` with `policy: "always"` and a Chrome profile name (not a path), plus `executablePath`, become native subprocess defaults only for automatic root browsers and their helpers when the effective engine is Chrome. Lightpanda omits these wrapper defaults, without changing caller-selected launch settings. Roots have distinct named sessions and restore keys; parents and descendants share one root through `PI_SUBAGENT_ROOT_SESSION_ID`. These defaults never apply to unrelated explicit sessions or Electron launches. Code selecting the ordinary root browser uses the same defaults. Profile paths, other policies, and project-only guidance remain advisory. Native restore keeps cookies/localStorage/sessionStorage, not new IndexedDB, service workers, or memory-only grants; verify app access after a real restart. Avoid global native profile changes or a shared writable profile path; see [shared browser defaults](REFERENCE.md#shared-browser-defaults). Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `profiles` and `doctor` when profile resolution fails, then use the reported Chrome profile directory name, a full profile/user-data directory path if upstream accepts one, or the configured `browser.executablePath` with top-level `sessionMode: "fresh"`.
1007
1007
 
1008
1008
  ## Important global flags, config, and environment
1009
1009
 
package/docs/ELECTRON.md CHANGED
@@ -131,7 +131,19 @@ Returns app metadata under `details.electron.apps`: `name`, optional `bundleId`/
131
131
 
132
132
  ### `electron.launch` — launch and attach
133
133
 
134
- Pass **exactly one** target: `appPath`, `appName`, `bundleId`, or `executablePath`. The wrapper resolves the target, verifies Electron framework evidence, applies optional caller-owned `allow` / `deny` policy, creates an isolated temp `userDataDir`, launches with `--remote-debugging-port=0` plus safe defaults, reads `DevToolsActivePort`, then attaches through upstream `connect` as a fresh managed session.
134
+ Pass **exactly one** target: `appPath`, `appName`, `bundleId`, or `executablePath`. The wrapper resolves the target, verifies Electron runtime and app payload evidence, applies optional caller-owned `allow` / `deny` policy, creates an isolated temp `userDataDir`, launches with `--remote-debugging-port=0` plus safe defaults, reads `DevToolsActivePort`, then attaches through upstream `connect` as a fresh managed session.
135
+
136
+ Payload evidence accepts `app.asar`, an `app/` directory, or the stock npm Electron runtime's `default_app.asar`. macOS also requires `Contents/Frameworks/Electron Framework.framework`; Linux/Windows require the existing Chromium resource evidence. These checks apply to discovery and explicit paths alike, and remain a correctness gate rather than a security boundary.
137
+
138
+ For an unpackaged development app, point `executablePath` at the stock Electron binary and put the app directory in `appArgs`:
139
+
140
+ ```json
141
+ {
142
+ "action": "launch",
143
+ "executablePath": "/path/to/node_modules/electron/dist/Electron.app/Contents/MacOS/Electron",
144
+ "appArgs": ["/path/to/fixture-app"]
145
+ }
146
+ ```
135
147
 
136
148
  ```json
137
149
  {
@@ -154,6 +166,8 @@ Handoff selection (`handoff` field):
154
166
 
155
167
  `targetType` defaults to `"page"`; use `"webview"` or `"any"` for apps whose useful UI is exposed as a webview target.
156
168
 
169
+ CDP readiness requires `/json/version` to contain a nonempty, non-whitespace `Browser` or `browser` string. An arbitrary JSON object is not readiness evidence; other version metadata is optional. The same rule determines debug-port liveness in status and cleanup.
170
+
157
171
  Optional `timeoutMs` on `electron.launch` sets the host readiness polling budget for `DevToolsActivePort` and CDP metadata. The clock starts after target discovery and policy checks; upstream attach and handoff have separate subprocess budgets. When omitted, the default is **15 seconds** with a hard maximum of **120 seconds**, matching `ELECTRON_LAUNCH_DEFAULT_TIMEOUT_MS` and `ELECTRON_LAUNCH_MAX_TIMEOUT_MS` in `extensions/agent-browser/lib/electron/launch.ts`. Pi cancellation is separate: an already-cancelled call never launches the app, while cancellation during readiness polling or URL/tab/snapshot handoff closes the managed session, stops the tracked process, removes its isolated profile, and returns `failureCategory: "aborted"` without waiting for the launch timeout.
158
172
 
159
173
  Wrapper-owned launches **always** use an isolated temp profile and an OS-chosen port. If wrapper validation, managed-session policy, or the post-attach live-URL handoff guard fails after the host app starts, the wrapper immediately stops that process and removes the isolated profile; it retains a partial tracked record only when cleanup itself cannot finish. `--user-data-dir`, `--remote-debugging-port`, `--remote-debugging-address`, `--remote-debugging-pipe`, and bare `--` in `appArgs` are rejected. There is no caller-supplied port and no way to make `electron.launch` reuse the app's normal signed-in profile or attach to an already-running app — by design. Use the manual path described above when those are the actual requirements.
@@ -170,6 +184,8 @@ Read-only inspection of one or more tracked launches. Without `launchId` or `all
170
184
 
171
185
  Reports `cleanupState`, current debug-port and PID liveness, bounded CDP targets, and freshly measured `userDataDirState` under `details.electron.statuses`. An explicit `launchId` can inspect a **historical cleaned launch record**; default and `all: true` selection exclude cleaned records. Cleanup history does not determine current liveness. The tracked profile path is `present`, `absent` (only native `lstat` ENOENT), or `unknown` (other filesystem errors); a dangling symlink is present. This measures that path only, not all app residue, and is not stored in the launch record. Its managed-session title/URL reads hold the normal daemon-policy lock and owned restore context. Mismatch fields surface when the current managed session or tab no longer matches a live wrapper launch target — typically the cue to follow `reattach-electron-launch` before trusting old refs.
172
186
 
187
+ Status and probe compare complete URLs before shortening them for display. Managed-session/probe URL summaries remain capped at 300 characters (probe tab URLs at 240); probe page-target/ref state retains the full URL.
188
+
173
189
  ### `electron.probe` — compact state read
174
190
 
175
191
  `probe` collapses what would otherwise be separate `get url` / `get title` / focused-element `eval` / `tab list` / `snapshot -i` calls into one bounded result. Use it instead of chaining those reads when you just need a quick "where are we?" check. The wrapper holds the managed-session daemon-policy lock for the probe and runs every underlying read with the session's owned restore decision, so probing cannot restart the daemon under a different restore key.
@@ -185,7 +201,7 @@ Output appears under `details.electron.probe`: `title`, `url`, `focusedElement`,
185
201
 
186
202
  ### `electron.cleanup` — wrapper-owned only
187
203
 
188
- Closes the tracked managed session, stops only the wrapper-tracked process, verifies that the debug port no longer serves `/json/version`, and removes the wrapper-created `userDataDir`. Cleanup waits for actual exit when it has a tracked child handle. For a live restored PID without that handle, it first checks the native process command line for the recorded profile: `Win32_Process.CommandLine` through Windows PowerShell on Windows, or `ps` on POSIX. If inspection fails or the profile does not match, cleanup leaves the process and profile intact.
204
+ Closes the tracked managed session, stops only the wrapper-tracked process, verifies that the debug port no longer serves `/json/version`, and removes the wrapper-created `userDataDir`. Cleanup waits for actual exit when it has a tracked child handle. For a live restored PID without that handle, it signals only when the process start identity recorded at launch matches the live process, which proves the PID was not reused. Records without an identity (for example from older versions), unreadable identities, and mismatches leave the process and profile intact; stop such a process manually.
189
205
 
190
206
  Cleanup partial failures fail the tool result with `failureCategory: "cleanup-failed"` and the `retry-electron-cleanup` next action references the same `launchId` so retries are bounded.
191
207
 
@@ -331,6 +347,11 @@ Single-instance Electron behavior is a common cause of `timeout` and `upstream-e
331
347
  - On macOS, apps installed outside `/Applications` and `~/Applications` are not scanned in v1. Pass `appPath` or `executablePath` explicitly.
332
348
  - Windows hosts report `platform: "unsupported"` from `electron.list`; always pass `executablePath` (or a resolvable `appPath`) for `launch`.
333
349
 
350
+ ### Launch rejects the target as non-Electron
351
+
352
+ - Verify both runtime resources and an app payload (`app.asar`, `app/`, or `default_app.asar`) exist beside the executable. On macOS, the Electron Framework is also required; Chromium paks alone do not establish an Electron target.
353
+ - The stock npm Electron runtime is supported without repackaging. Pass its binary as `executablePath` and your unpackaged app directory as an `appArgs` operand.
354
+
334
355
  ### Attach succeeds but `snapshot -i` returns no refs
335
356
 
336
357
  - Some Electron apps take a beat to render. The default `handoff: "snapshot"` already retries briefly; if it still reports no refs, run `tab list`, select the intended stable `t<N>` app tab, then run `snapshot -i` again.
@@ -356,6 +377,7 @@ Single-instance Electron behavior is a common cause of `timeout` and `upstream-e
356
377
 
357
378
  ### Mismatch between `status` and the active session
358
379
 
380
+ - A shortened URL ending in `...` is display-only; long URLs alone do not cause mismatch or reattach guidance. Status/probe compare full normalized URLs.
359
381
  - `electron.status` may report a live wrapper launch while the managed session has drifted to `about:blank`. Follow `reattach-electron-launch`, then refresh refs before reusing old `@e…` handles. For non-wrapper tab drift where `details.nextActions` names `select-intended-tab-after-drift`, use that stable `t<N>` action plus `snapshot-after-tab-recovery` before continuing.
360
382
 
361
383
  ## Code and shared-session coordination