pi-agent-browser-native 0.6.16 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/CHANGELOG.md +18 -1
  2. package/README.md +59 -165
  3. package/dist/extensions/agent-browser/index.js +634 -501
  4. package/dist/extensions/agent-browser/lib/browser-transcript.js +33 -0
  5. package/dist/extensions/agent-browser/lib/command-taxonomy.js +8 -4
  6. package/dist/extensions/agent-browser/lib/execution-cwd.js +66 -0
  7. package/dist/extensions/agent-browser/lib/input-modes/job.js +2 -213
  8. package/dist/extensions/agent-browser/lib/input-modes/lookups.js +1 -1
  9. package/dist/extensions/agent-browser/lib/input-modes/params.js +146 -142
  10. package/dist/extensions/agent-browser/lib/input-modes/script.js +64 -92
  11. package/dist/extensions/agent-browser/lib/managed-session-policy-lock.js +170 -24
  12. package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-paths.js +27 -20
  13. package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostics.js +14 -2
  14. package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result.js +26 -13
  15. package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +21 -1
  16. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +4 -2
  17. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +1 -3
  18. package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +41 -24
  19. package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery.js +3 -1
  20. package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-observation.js +79 -0
  21. package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +3 -4
  22. package/dist/extensions/agent-browser/lib/orchestration/input-plan.js +19 -52
  23. package/dist/extensions/agent-browser/lib/orchestration/native-session-defaults.js +3 -3
  24. package/dist/extensions/agent-browser/lib/orchestration/operation-paths.js +76 -0
  25. package/dist/extensions/agent-browser/lib/orchestration/output-file.js +3 -3
  26. package/dist/extensions/agent-browser/lib/orchestration/script-mode.js +91 -278
  27. package/dist/extensions/agent-browser/lib/pi-tool-rendering.js +9 -9
  28. package/dist/extensions/agent-browser/lib/playbook.js +46 -25
  29. package/dist/extensions/agent-browser/lib/process-identity.js +22 -6
  30. package/dist/extensions/agent-browser/lib/process.js +16 -13
  31. package/dist/extensions/agent-browser/lib/results/action-recommendations.js +2 -2
  32. package/dist/extensions/agent-browser/lib/results/next-actions.js +1 -2
  33. package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +25 -4
  34. package/dist/extensions/agent-browser/lib/results/presentation/batch.js +6 -2
  35. package/dist/extensions/agent-browser/lib/results/presentation/common.js +2 -0
  36. package/dist/extensions/agent-browser/lib/results/presentation/content.js +39 -0
  37. package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +3 -3
  38. package/dist/extensions/agent-browser/lib/results/presentation/large-output.js +41 -1
  39. package/dist/extensions/agent-browser/lib/results/presentation.js +16 -11
  40. package/dist/extensions/agent-browser/lib/runtime.js +18 -6
  41. package/dist/extensions/agent-browser/lib/session-page-state.js +3 -4
  42. package/dist/extensions/agent-browser/lib/tool-surface.js +131 -0
  43. package/dist/extensions/agent-browser/script-worker.js +4 -1
  44. package/docs/ARCHITECTURE.md +47 -61
  45. package/docs/COMMAND_REFERENCE.md +78 -137
  46. package/docs/ELECTRON.md +51 -49
  47. package/docs/RELEASE.md +31 -15
  48. package/docs/REQUIREMENTS.md +18 -14
  49. package/docs/SUPPORT_MATRIX.md +33 -15
  50. package/docs/TOOL_CONTRACT.md +218 -401
  51. package/package.json +4 -4
  52. package/scripts/doctor.mjs +2 -2
@@ -9,28 +9,19 @@ Related docs:
9
9
 
10
10
  ## Decision
11
11
 
12
- Build this as a **thin `pi` extension/package** that exposes `agent-browser` as one native tool while keeping upstream `agent-browser` as the source of truth.
12
+ Build this as a **thin `pi` extension/package** that exposes native commands and persistent-browser code while keeping upstream `agent-browser` as the source of truth.
13
13
 
14
14
  The package install path is the primary product path. Local checkout development should use explicit CLI loading for isolated smoke tests and configured-source plain `pi` runs for lifecycle validation, while package-manifest behavior and packaged contents matter more.
15
15
 
16
16
  ## Chosen shape
17
17
 
18
- ### One primary tool plus optional companion search
18
+ ### Compact tools and native discovery
19
19
 
20
- V1 exposes one native browser tool:
20
+ `agent_browser` accepts native argv/stdin, result output path, timeout, and session mode. `agent_browser_code` accepts strict JSON source plus optional parent-selected session/namespace, timeout, and output path. Both use one ordinary CLI executor. Use direct calls for one action, native batch for fixed sequences, and code for branches, loops, or aggregation; return to the model at judgment boundaries.
21
21
 
22
- - `agent_browser`
22
+ `agent_browser_tools` inventories/enables five flat specialized tools: action, QA, Electron, UI source, and network source. `lib/tool-surface.ts` registers them once and adds them through Pi's native active-tool API, preserving unrelated tools, explicit host selections, and native selected-tool history. It owns no parallel activation journal. QA verdicts, unique accessible select, host Electron lifecycle, and bounded source scans remain specialized capabilities rather than being reduced to raw-command approximations. The former public job DSL and isolated script route are removed.
23
23
 
24
- It may also expose one optional companion tool:
25
-
26
- - `agent_browser_web_search`, available when an Exa or Brave Search credential source is configured or resolvable from startup config or trusted session config and runtime `webSearch.enabled` is not false
27
-
28
- Why:
29
- - keeps browser automation centered on `agent_browser`
30
- - avoids colliding with generic `web_search`
31
- - keeps live search separate from browser state, screenshots, refs, and session lifecycle
32
- - supports requested Exa/Brave provider choice without turning search into an `agent_browser` input mode
33
- - keeps optional search invisible when it cannot run or when the user disables it
24
+ Optional `agent_browser_web_search` remains separately credential/config gated and independent of browser state.
34
25
 
35
26
  ### Direct subprocess execution
36
27
 
@@ -40,35 +31,32 @@ The extension should:
40
31
  - bound native process-identity queries at ten seconds on Windows, including PowerShell startup, and five seconds on POSIX. These probes preserve PID/start-time comparison and fail closed when identity is unavailable; their budget is separate from lock contention and browser command watchdogs
41
32
  - inject `--json`; caller-requested JSON also uses the existing `buildJsonVisibleContent` envelope for early snapshot/network filters, scroll results, and preparation failures, retaining metadata in `details` and keeping output-file notices out of visible JSON. Help/version stays native text
42
33
  - complete each upstream invocation when the direct `agent-browser` child exits even if Node delays `"close"`: piped stdio can stay referenced by longer-lived descendant processes, so `runAgentBrowserProcess` watches `exit` and `close` together, leaves stdio intact during a short post-`exit` grace so normal `close` can still win, destroys streams only when the post-`exit` fallback fires, and prefers `close` codes then wrapper timeout (`124`) over signal-shaped `exit` codes (`watchSpawnedChildCompletion` / `resolveSpawnedChildExitCode` in `extensions/agent-browser/lib/process.ts`) so the tool cannot hang after the CLI process has already terminated
43
- - support optional stdin only for `eval --stdin`, `batch`, `auth save --password-stdin`, and wrapper-generated `batch` stdin from top-level `job`, `qa`, `sourceLookup`, or `networkSourceLookup`, rejecting other command/stdin combinations before launch; top-level `electron` never accepts caller `stdin` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron))
34
+ - support optional stdin only for `eval --stdin`, `batch`, `auth save --password-stdin`, and internally generated QA/source batch plans, rejecting other command/stdin combinations before launch; Electron manages its own input
44
35
  - support optional top-level `outputPath` for successful browser results and failed/pending/recovered recording receipts. Recording exports retain success/error, original attempt, native data, artifacts and recovery provenance; they do not treat a written receipt as a successful video. Other successful results write `details.data` (or model-facing text when no structured data exists) to a caller-requested local file and reporting `details.outputFile`, without changing upstream argv semantics or overwriting a browser artifact when both destinations resolve to the same file. If presentation compacted direct data, a batch result row, or a whole batch, apply the relevant command-specific redactor before spilling and rehydrate each full pre-compaction value only from its matching live wrapper-manifest spill; fail without writing compact metadata when any required spill is unavailable or untrusted
45
- - support optional top-level `timeoutMs` as a per-call subprocess watchdog override for browser CLI input modes while keeping Electron-specific timeouts inside the `electron` object
46
- - accept an optional top-level `script` string as a mutually exclusive one-shot orchestration mode for loops, conditional page branches, and multi-page aggregation. Source runs in a separate permissioned Node child with a constrained VM context; only null-prototype `browser({ args, stdin?, timeoutMs? })` and `emit(value)` task functions cross a bounded JSON-lines IPC bridge. The parent serializes at most one inner call at a time through the same full ordinary tool executor, clears ambient upstream launch/proxy controls across helpers and cleanup, caps calls/source/post-redaction output/time, injects one unique restore-disabled wrapper-owned session, writes a strict Pi custom-entry cleanup lease before first launch, closes in `finally`, aborts and awaits active-script cleanup on branch change/shutdown, and recovers exact non-closed active-branch leases afterward. Script requires Pi session persistence and exposes no profile/attachment/session-control, host API, named recipe, import, or persistent workflow-state surface (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#script)).
47
- - accept an optional native `semanticAction` object as a mutually exclusive alternative to `args` on a single tool call (and to `script`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, and `electron` on the same call), compile locator actions into upstream `find` argv, direct selector/ref click/check/fill into upstream command argv, and native dropdown selection into upstream `select <selector> <value...>` argv (with optional `semanticAction.session` expanding to a leading `--session <name>` before the compiled command when targeting a named upstream browser instead of the managed default), and echo the compiled shape in `details.compiledSemanticAction` for observability (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#semanticaction))
48
- - accept an optional native `job` object (mutually exclusive with `script`, `args`, `semanticAction`, `qa`, `sourceLookup`, `networkSourceLookup`, and `electron` on the same call) with a small fixed step vocabulary that compiles only to existing upstream `batch` argv rows, generates the JSON batch stdin string internally, defaults to fail-fast `batch --bail` unless `failFast:false` is explicit, supports semantic locator fields on constrained `click`/`fill` steps by reusing the top-level semantic-action compiler, supports human-paced `type` by expanding to a bounded set of existing focus/keyboard/wait/press rows while compacting model-visible batch text, and echoes `details.compiledJob` for observability (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#job))
49
- - accept an optional native `qa` object (mutually exclusive with `script`, `args`, `semanticAction`, `job`, `sourceLookup`, `networkSourceLookup`, and `electron` on the same call) that compiles to the same fail-fast `batch --bail` path as `job`, runs a fixed diagnostic smoke sequence with bounded visible-text predicates for `expectedText`, preserves existing diagnostics for `qa.attached` while requesting clears only for URL-opening QA, and echoes `details.compiledQaPreset` plus structured `details.qaPreset` pass/fail evidence. Native clearing remains upstream-owned; the wrapper makes matched final rows from a nonempty post-clear error baseline non-pass through the existing `failedChecks` / `qa-failure` path because old residue and identical new errors are indistinguishable. Definitely new errors are counted separately, clean evidence can pass, and explicit `checkErrors: false` omits the check without changing browser lifecycle (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#qa))
50
- - accept an optional native `sourceLookup` object (mutually exclusive with `script`, `args`, `semanticAction`, `job`, `qa`, `networkSourceLookup`, and `electron` on the same call) that compiles to the same `batch` path, gathers evidence-backed local source *candidates* for a selector/fiber/component name, and echoes `details.compiledSourceLookup` plus structured `details.sourceLookup` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sourcelookup)); unlike `qa`, it never applies a second pass/fail layer that marks the tool failed when upstream already reported batch success—failed upstream steps still fail the invocation normally, and `details.sourceLookup` may still be present for partial evidence
51
- - accept an optional native `networkSourceLookup` object (mutually exclusive with `script`, `args`, `semanticAction`, `job`, `qa`, `sourceLookup`, and `electron` on the same call) that compiles to the same `batch` path, correlates failed network requests with initiator metadata and bounded workspace URL literals, and echoes `details.compiledNetworkSourceLookup` plus structured `details.networkSourceLookup` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#networksourcelookup)); like `sourceLookup`, it never flips a successful upstream batch to failed solely because no source candidates were found
52
- - accept an optional native `electron` object (mutually exclusive with `script`, `args`, `semanticAction`, `job`, `qa`, `sourceLookup`, and `networkSourceLookup` on the same call) for bounded desktop Electron lifecycle: `list` scans the host for install candidates, `launch` creates a wrapper-owned isolated profile plus OS-chosen remote-debugging port, then attaches through upstream `connect` with `sessionMode: "fresh"`, cancellation prevents host spawn or interrupts readiness polling with process/profile cleanup, and `status` / `cleanup` / `probe` operate only on wrapper-tracked launches; host-side spawn and CDP discovery live in `extensions/agent-browser/lib/electron/discovery.ts`, `launch.ts`, and `cleanup.ts`, while compilation, transcript restore for `launchId` records, handoff probes, and merged `details.electron*` fields live under `extensions/agent-browser/lib/orchestration/electron-host/` and `extensions/agent-browser/lib/orchestration/browser-run/` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron))
53
- - when a compiled `find` semantic action fails as `stale-ref`, optionally append a `retry-semantic-action-after-stale-ref` entry to `details.nextActions` after the usual `refresh-interactive-refs` snapshot step so agents can re-issue the same compiled `find` argv only when the failure implies the interaction did not run; `select` shorthands with stale `@refs` get refresh guidance only (contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#semanticaction))
54
- - when the same compiled path fails as `selector-not-found` for the bounded locator/action pairs documented there, optionally append `try-*-candidate` entries to `details.nextActions` and mirror them in visible text as `Agent-browser candidate fallbacks` so agents can retry role/name `find` variants without hand-rebuilding argv (`select` misses are intentionally excluded)
36
+ - support optional top-level `timeoutMs` as a per-call subprocess watchdog override for browser CLI input modes while Electron uses its flat action-specific timeout and code uses a whole-cell deadline
37
+ - preserve specialized action/QA/Electron/source outcomes through flat advanced tools that adapt into the ordinary executor. Semantic actions keep current-ref resolution and exact recovery; QA retains fail-fast visible assertions, preserved attached diagnostics, and truthful error classification; Electron retains process/profile ownership and cancellation; source tools retain bounded host scans even when other Pi file tools are unavailable.
38
+
39
+ ### Persistent browser code
40
+
41
+ `lib/input-modes/script.ts` and compiled `script-worker.js` retain the fresh permissioned Node child/VM and JSON-lines bridge. The internal filenames are not a public `script` input. `index.ts` selects the ordinary browser identity in the parent, holds its shared execution lock for the whole cell, serializes inner requests through the same executor, and reaps the child on completion, timeout, abort, branch change, or shutdown. The browser survives cell completion according to existing ownership. Native auth, profiles, state, connect, batch, and local commands retain ordinary parent-side configuration/environment; only changing the fixed session/namespace and namespace-wide close are excluded from an inner call.
42
+
43
+ The child has an empty environment, Node permission mode without host capability allowlists, a 64 MiB heap ceiling, and a VM with string/WebAssembly code generation disabled. Null-prototype `browser`, `emit`, and `emitImage` functions use JSON-only IPC; parent promises/objects/functions are not exposed. There are no imports or host filesystem/network/process/timer APIs. Node's VM alone is not a security boundary. The retained child isolation is distinct from the selected browser's persistent authentication and native capabilities; no QuickJS, broker, second driver, or provider payload injection is involved.
55
44
 
56
- ### One-shot script isolation
45
+ Limits are 25 attempted calls, 64 KiB source/final JSON, 1 MiB per IPC message, 8 MiB cumulative IPC, and a 120-second default/300-second maximum deadline. `browser()` returns a canonical observation. `emit` selects JSON; `emitImage` selects a cell-local verified image handle, with file identity rechecks and normal inline limits. `lib/orchestration/script-mode.ts` aggregates selected output, failures, and artifact receipts; its only legacy execution support reads exact pre-0.7 isolated-session leases for upgrade cleanup.
57
46
 
58
- `script` is orchestration around the native tool, not a second browser runtime. Its custom Pi call renderer keeps the approval boundary inspectable with a bounded terminal-safe preview whose line breaks render as `↵` and full terminal-safe source when expanded; JavaScript line terminators stay visible as newlines and removed controls become visible markers. The child never imports this extension or invokes `agent-browser`; it only emits bounded JSON call requests. The parent validates each request against script-specific policy, injects `--namespace "" --session piab-script-<uuid>`, and recursively uses the registered tool's ordinary executor. This preserves the same argv parsing, page-target validation, process lifecycle, presentation/redaction, spill/artifact verification, result categories, and timeout behavior instead of creating a weaker bare-process shortcut. The process isolation scope filters ambient `AGENT_BROWSER_*` and standard proxy variables during planning and spawn and selects an empty private temporary native config. This also bypasses HOME/project profile defaults during helpers and later lease cleanup. Inner `--config` is rejected before dispatch so it cannot override that empty config; ordinary top-level `args` still accepts native `--config`. The wrapper-owned namespace, timeout, and compatibility values are then applied.
47
+ ### Ordered transitions and shared execution
59
48
 
60
- Isolation is layered:
49
+ `lib/managed-session-policy-lock.ts` supplies cooperating-process execution coordination keyed by the **actual socket context**, canonical namespace, and session, separate from existing managed-daemon policy checks. The lock spans direct helper/action/state work and an entire code cell. Fresh replacement and multi-identity cleanup acquire sorted identity sets; namespace-wide direct close uses an exclusive namespace barrier. Existing local queues and branch-generation checks still protect in-process ownership/replay. Independent identities remain concurrent. Direct calls bound coordination waits using the existing configured/command-aware timeout, then retain their native subprocess and Electron action watchdogs. Code owns a whole-cell deadline, including local queue waits. Independently bounded cleanup can borrow an already-held covered claim after cancellation, but cannot acquire a new identity or release the outer claim.
61
50
 
62
- 1. schema/runtime bounds reject oversized/non-string source and invalid top-level mode combinations
63
- 2. a separate Node child receives an empty environment, a 64 MiB heap ceiling, permission mode, no host capability allowlists, and a VM context with string/WebAssembly code generation disabled. Node documents `node:vm` as not a security mechanism by itself; the permissioned child process and bounded IPC are the containment boundary, while the VM removes ordinary capability references. Re-run the constructor/stack-escape sandbox tests when raising the Node major floor.
64
- 3. the context receives JSON-created data only; the exposed `browser` / `emit` functions have null prototypes, and no parent promise/function/object is exposed to user source
65
- 4. JSON-lines IPC has per-message and cumulative byte limits, one in-flight browser call, 25 attempted calls, bounded inner summary/text, a serialized-envelope check that turns oversized inner results into handleable failures, compact final-output serialization with a second post-redaction byte/depth guard, and one outer deadline that clamps every inner timeout
66
- 5. inner `nextActions` are filtered back through script policy after the wrapper-owned isolated namespace/session prefix is removed, so source never receives a suggested browser payload that its next `browser()` call must reject
67
- 6. abort/timeout/branch change/shutdown sends SIGTERM then bounded SIGKILL, aborts any active inner call, waits for normal isolated-session cleanup, and reaps the child before branch restoration or shutdown cleanup proceeds
51
+ All participating Pi processes must restart with the updated package. The lock is not a browser transaction, rollback, or a guarantee against humans, third-party native clients, older extensions, or separately named sessions attached to one external Chrome. Separate tool calls still need workflow coordination.
68
52
 
69
- Browser isolation is separate from language isolation. A pre-spawn Pi custom entry records only the generated session name, exact generated close argv, launch marker, and cleanup state. That entry is durable but model-invisible. The generated identity has no restore key, cannot be selected by script source, never updates the implicit managed-session pointer, and is always closed. Exact pending records on the active transcript branch are retried after restart; malformed names, altered close argv, or missing launch markers are ignored. If Pi persistence is disabled, script fails before child/browser launch because crash recovery cannot be guaranteed.
53
+ `lib/browser-transcript.ts` appends ordered `agent-browser-transition` intent/completion custom entries and projects direct, advanced, and code transitions into existing replay reducers. Intent is written before inner dispatch; completion carries target/ref/attachment/close/ownership state rather than page output. Interrupted pending work remains unknown until inspected. Persistence failure after a command does not erase its side effects or permit blind retry. Recording reservations and Electron ownership keep their existing journals/reducers. A persisted Pi session is required; JavaScript variables are never persisted.
70
54
 
71
- Explicit URL reads remain available inside script; parent-injected isolated identities still constrain their confirmation routing. The browserless confirmation exemption requires both explicit-read provenance and native `capabilities.readRequiresConfirmation: true`, never page text or an ID from another session. It does not add child host APIs or weaken script launch/config restrictions.
55
+ ### Canonical observations and image geometry
56
+
57
+ `results/presentation/content.ts` projects decision-relevant fields once; `large-output.ts` renders model-visible JSON/prose after recovery, diagnostics, and requested output-file writes are assembled. Failed file exports retain partial code data/images and report a failed observation. `details` remains audit/replay metadata. Final text has a 16,000-character bound; a complete redacted observation spill retains data and exact recovery payloads that cannot fit inline. Native help/version stays text. Inner `modelVisible: false` calls keep data without full prose/spill/image encoding; selected JSON and images are emitted at the outer boundary. Handled failures remain visible with counts and recovery, even if source finishes successfully.
58
+
59
+ `orchestration/browser-run/screenshot-observation.ts` brackets standalone screenshots with CSS viewport/document/scroll/DPR/frame samples. Existing Pi image-dimension primitives measure saved pixels. Matching samples and dimensions can establish a crop mapping; changed, unmeasured batch, scrolled/ambiguous element, zoom, or frame-uncertain captures remain unknown. These samples are not an atomic capture guarantee. Native mouse input uses current viewport CSS pixels, not image pixels; full-page/element origin and any Pi resize must be accounted for separately. Unchanged conditional screenshots attach nothing. See [image observations](TOOL_CONTRACT.md#image-observations).
72
60
 
73
61
  ### Agent-first UX
74
62
 
@@ -93,10 +81,12 @@ Config layers merge in that order: global, project, override. The shared policy
93
81
 
94
82
  `agent_browser_web_search` availability is conditional. Startup registration uses global, override, and environment fallback config without reading project-local config before Pi trust context exists; trusted project config can register the companion tool on `session_start`, and every execution reloads the final session config so `webSearch.enabled: false` still prevents a request even if a startup credential made the tool visible. A global disable is the normal user default and can still be overridden by project config or `PI_AGENT_BROWSER_CONFIG`; a project disable applies to one repo; an explicit `PI_AGENT_BROWSER_CONFIG` file with `webSearch.enabled: false` is the highest-priority hard disable for that run. Literal and env-backed sources must resolve before they make the tool available; command-backed sources are considered configured without running the command until tool execution, so secret managers do not slow startup or prompt unexpectedly. The tool resolves the selected key lazily, chooses Exa or Brave from available credentials (preferring Exa by default unless `webSearch.preferredProvider` says otherwise), then follows one provider-agnostic execution path through provider adapters for request building, HTTP JSON fetch, response normalization, and provider-specific detail fields. Exa search-type precedence is per-call value, merged config default, then `auto`; bounded Exa filters stay on the Exa adapter and explicit Exa-only options fail before a Brave request. The tool calls Exa `/search` with highlights plus a fixed primary-source/version/date/distinct-result system prompt, or Brave Search, then removes later exact normalized-URL duplicates while retaining provider order. It does not guess across distinct paths/query URLs or overfetch replacements; provider page dates remain clues rather than claimed crawl/version evidence. Compact result details report any `duplicatesRemoved` without exposing keys.
95
83
 
96
- Browser default config is intentionally advisory. It can add prompt guidance for signed-in/account-specific tasks and alternate Chromium-compatible executables, but current releases do not auto-inject `--profile` or `--executable-path` into every launch. Loaded project config can provide the same guidance as global and override config; Pi/project trust decides whether that project layer is loaded. Automatic launch-default mutation would affect privacy, browser state, and host executable choice, so it needs a separate explicit design and test pass.
84
+ Global/override package profile names with `policy: "always"` and executable defaults bootstrap inactive automatic root browsers through scoped native environment. Active roots retain native settings; paths, other policies, and project-only values remain advisory. Direct and code calls selecting that root share the same configuration. Unrelated explicit sessions and Electron launches retain their separate native/owned rules.
97
85
 
98
86
  ### Prompt guidance budget
99
87
 
88
+ Use Pi-native `promptGuidelines`, tool declarations, and named `systemPromptOptions.sections` contributions. Do not return a replacement full system prompt from `before_agent_start`: it breaks additive declaration anchoring. Official Pi 0.87 and the fork use this same composition; optional fork checkpoint hooks do not make hosted programmatic tool calling, async provider lifecycle, image-detail overrides, or model-specific payload edits package dependencies. Prefix stability is not proof of a cache hit.
89
+
100
90
  Runtime `promptGuidelines` are a Tier A budget, not a full manual. They stay short enough to load on every `agent_browser`-aware turn and carry only high-impact rules: input-mode choice, the open → snapshot → ref loop, launch-scoped session handling, artifact verification, structured `nextActions`, extraction basics, and hard agent-responsibility boundaries: honor explicit stop boundaries, allow ordinary requested non-destructive submissions in authenticated unattended/auto-approved employee flows, and require explicit authorization for purchases, production-control, destructive/irreversible, or account/security/privacy changes.
101
91
 
102
92
  Tier B guidance lives in `SHARED_BROWSER_PLAYBOOK_GUIDELINES`, generated README/command-reference fragments, and targeted docs. When a workflow needs examples, caveats, or long command-family coverage, add it there instead of expanding always-on prompt text. If a Tier B rule prevents a repeated real failure, promote only the smallest durable sentence into Tier A and keep the generated-doc mirrors aligned.
@@ -105,15 +95,9 @@ Tier B guidance lives in `SHARED_BROWSER_PLAYBOOK_GUIDELINES`, generated README/
105
95
 
106
96
  Do **not** add reusable browser recipes as a first-class runtime surface yet.
107
97
 
108
- One-shot `script` does not change this decision. It accepts source only for the current tool call and has no names, registry, storage, imports, shared state, versioning, or invocation API. Do not layer those recipe capabilities on top of it without a new design pass.
98
+ Browser code accepts ad hoc source for the current call. It has no names, registry, imports, shared JavaScript state, or workflow versioning. Native batch and the preserved advanced tools cover fixed sequences and established specialized outcomes.
109
99
 
110
- Current evidence does not justify another source of truth for workflows:
111
- - unit/fake-upstream coverage and real-app dogfood exercise ad hoc `script`, `job`, `qa`, experimental lookups, and `electron` lifecycle without a separate accounting benchmark layer
112
- - repo-local dogfood evidence does not show repeated project-specific job recipes that need versioning or ownership
113
- - `qa` already covers the only repeated smoke-test shape with a stable top-level preset
114
- - docs and prompt guidance can carry examples without adding recipe state, migration rules, or another schema
115
-
116
- Revisit this only when dogfood data shows at least two repeated, failure-prone sequences that cannot be represented clearly by one-shot `script`, `job`, `qa`, top-level `electron`, or raw `batch`. If that happens, define ownership, versioning, schema boundaries, generated docs, and tests before adding executable recipes.
100
+ Revisit only when repeated dogfood demonstrates failure-prone sequences that cannot be clearly represented with code, native batch, or the advanced tools; define ownership and evidence before adding another runtime layer.
117
101
 
118
102
  ### Package layout versus local checkout development
119
103
 
@@ -136,15 +120,17 @@ The published package should exclude agent-only and internal planning materials
136
120
 
137
121
  ## Host execution hook
138
122
 
139
- The default extension factory optionally accepts `{ beforeExecute }`. `index.ts` awaits this host callback after input resolution and before non-script dispatch, forwarding the original outer Pi tool-call ID through recursive script calls and supplying each dispatch's signal. Configured hosts use Pi's native sequential tool scheduling; inner script calls retain the existing serial queue. No separate controller, persistence store, retry policy, or timeout is introduced. Ordinary installation, internal helper probes, cleanup, batch-row execution, and web search are unchanged. See [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#host-execution-hook) for cancellation and error behavior; `test/agent-browser.pi-pipeline.test.ts` exercises the registered factory through real Pi with a fake upstream executable.
123
+ The default extension factory optionally accepts `{ beforeExecute }`. `index.ts` awaits this host callback after input resolution and before ordinary dispatch, forwarding the original outer Pi tool-call ID through inner code calls and supplying each dispatch's signal. Configured hosts use Pi's native sequential tool scheduling; inner code calls retain the existing serial queue. No separate controller, persistence store, retry policy, or timeout is introduced. Ordinary installation, internal helper probes, cleanup, batch-row execution, and web search are unchanged. See [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#host-execution-hook) for cancellation and error behavior; `test/agent-browser.pi-pipeline.test.ts` exercises the registered factory through real Pi with a fake upstream executable.
140
124
 
141
125
  ## Session model
142
126
 
127
+ The optional `pi-change-working-dir` event protocol supplies one invocation execution cwd for file operands and workspace scans. It never mutates native `ctx.cwd`. Native config/CLI cwd stays at the project or managed launch anchor; explicit fresh/config choices use the invocation root. Managed launch roots persist in `details.managedSessionCwd`; selection runs inside the existing managed queue so an earlier fresh launch determines the next browser target without changing admitted file paths. Code calls retain the outer snapshot. Existing recordings and cached artifact stores keep their original ownership. If a launch root is deleted, validation reports that boundary rather than spawning under another project's config. Public tool/command source manifests identify legacy owners; no private directory journal or process-global cwd is involved.
128
+
143
129
  ### Default
144
130
 
145
131
  An explicit native `session` default (user/project JSON or `AGENT_BROWSER_SESSION`) selects a caller-owned browser across Pi sessions. Per-call session/namespace flags win. `lib/orchestration/native-session-defaults.ts` reads session/namespace and contributing launch defaults from native config paths and uses sessionless native `--config <path> --json session` to validate each contributing file and resolve its session. Native config continues to own launch/profile/storage settings; the adapter does not duplicate its schema. Explicit config and per-call idle settings are scoped to helpers through `lib/process-environment.ts`. No startup probe, persistent config cache, or new lifecycle service is added.
146
132
 
147
- When no native session, attachment, or explicit fresh launch is selected, `sessionMode: "auto"` uses `pi-root-<sha256(root Pi ID)[0:24]>` through the existing caller-owned named-session path. Descendants receive `PI_SUBAGENT_ROOT_SESSION_ID` from pi-subagents; root processes use `ctx.sessionManager.getSessionId()`. No cwd, human fork ancestry, run registry, or process-global assignment participates. The same name is the default native restore key. Existing native daemon inspection runs inside the existing identity queue: inactive roots receive bootstrap defaults, while active roots retain native launch settings and their current restore key. Scoped subprocess environment keeps helper settings consistent; no launch-settings journal is added. Root-name explicit follow-ups retain those defaults; unrelated explicit names do not. Global/override package `browser.defaultProfile` with `policy: "always"` supplies native Chrome profile names only, plus the configured executable. Paths and project-only profile settings stay advisory. Native config/environment and explicit launch settings win. The communal source is copied by native Chrome launch, never by the wrapper, and each root's restore writes remain separate. Native JSON restore omits IndexedDB, service workers, and page-memory credentials; application-level restart verification is required. Parent and child quit leave the root browser alive; existing per-session queues remain process-local, with no global broker. An active older/fresh managed session keeps its existing lifecycle. A configured session has the same precedence over `fresh` as a literal `--session`; script and Electron launch keep their separate wrapper-owned lifecycles. Electron launch suppresses the native session environment default only while allocating its generated session; ordinary browser follow-ups still honor the configured native default, so use the returned Electron session explicitly.
133
+ When no native session, attachment, or explicit fresh launch is selected, `sessionMode: "auto"` uses `pi-root-<sha256(root Pi ID)[0:24]>` through the existing caller-owned named-session path. Descendants receive `PI_SUBAGENT_ROOT_SESSION_ID` from pi-subagents; root processes use `ctx.sessionManager.getSessionId()`. No cwd, human fork ancestry, run registry, or process-global assignment participates. The same name is the default native restore key. Existing native daemon inspection runs inside the existing identity queue: inactive roots receive bootstrap defaults, while active roots retain native launch settings and their current restore key. Scoped subprocess environment keeps helper settings consistent; no launch-settings journal is added. Root-name explicit follow-ups retain those defaults; unrelated explicit names do not. Global/override package `browser.defaultProfile` with `policy: "always"` supplies native Chrome profile names only, plus the configured executable. Paths and project-only profile settings stay advisory. Native config/environment and explicit launch settings win. The communal source is copied by native Chrome launch, never by the wrapper, and each root's restore writes remain separate. Native JSON restore omits IndexedDB, service workers, and page-memory credentials; application-level restart verification is required. Parent and child quit leave the root browser alive; shared execution locks coordinate updated cooperating processes without a broker. An active older/fresh managed session keeps its existing lifecycle. A configured session has the same precedence over `fresh` as a literal `--session`; code follows the selected ordinary browser lifecycle; Electron launch retains its owned lifecycle. Electron launch suppresses the native session environment default only while allocating its generated session; ordinary browser follow-ups still honor the configured native default, so use the returned Electron session explicitly.
148
134
 
149
135
  Why:
150
136
  - works out of the box
@@ -189,15 +175,15 @@ Practical policy:
189
175
  - redact snapshot spill payloads before writing them, clean up process-private temp spill artifacts on shutdown, and keep persisted-session spill files in a private session-scoped artifact directory for `details.fullOutputPath` after reload/resume, with a 32 MiB per-session budget by default; `PI_AGENT_BROWSER_SESSION_ARTIFACT_MAX_BYTES=0` skips persistent-spill enumeration and eviction without changing temporary spill cleanup
190
176
  - keep explicit screenshots, downloads, PDFs, traces, HAR captures, and recordings written to caller-chosen paths on disk after a successful upstream close command (`close`, `quit`, or `exit`); before artifact-producing commands run, create missing parent directories for requested host paths, while native download always owns the click and download, including loopback links, generated Blob exports, and redirects; never substitute an anchor-href fetch. When the bounded `details.artifactManifest` has entries, successful close commands also surface `details.artifactCleanup` and a compact `Artifact lifecycle` note pointing to structured explicit paths so operators remove files with normal host tools—the native tool does not delete arbitrary user paths (`extensions/agent-browser/lib/orchestration/browser-run/diagnostics.ts`, `getArtifactCleanupGuidance`); contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), checklist `RQ-0079` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md)
191
177
  - reconstruct the current branch-visible extension-managed session, every transcript-proven still-active wrapper-owned managed identity, page-scoped refs, newest-revision aggregate artifact manifest, unbounded active-recording reservation events, and Electron launch records from the active transcript branch on `session_start` and `session_tree` so later default and explicit off-current calls keep following owned browsers after resume/reload or branch switching; restore also honors successful explicit `--session <wrapper-owned> close` rows, terminal nested-batch close outcomes even when aggregate artifact verification failed, and `electron.cleanup` managed-session steps so closed wrapper-owned sessions are not resurrected; a nested close invalidates the pre-close page target so a lifecycle-proven relaunch at `about:blank` is not treated as stale focus drift; explicit lifecycle evidence that a later diagnostic did not launch a browser preserves the terminal close, while any later row—including a failed row—whose lifecycle reports a browser launch keeps active/attached provenance; failed-step presentation persists only that bounded launch boolean so transcript replay reaches the same decision, missing lifecycle evidence remains conservatively active even on the first managed call, successful closes clear wrapper trace/profiler ownership before ordered later successful rows can rebuild it, namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, and recording starts after close are rejected before spawn
192
- - keep active recording destination reservations separate from the bounded metadata-only artifact manifest. The same journal retains native recording IDs and start windows; native receipts, including failures, flow through existing artifact metadata and verification. A stop timeout/no-recording result gets one two-second native `session info` query matching namespace/session, ID/path or effective batch start window. Recovery needs terminal native encoder success plus a matching verified file; filesystem presence alone is insufficient, old same-path receipts cannot verify a new take, and freshness starts at capture rather than status-query time. Original attempts and unrelated batch failures remain visible. No second ledger, recorder, sidecar or runtime ffprobe is added. The process-wide map is keyed by canonical namespace/session identity, rebuilt from append-only branch events, and retained for still-live process-owned recordings across branch switches. Shutdown/reload appends both terminal tombstones and still-live reservations onto the current branch so restart cannot resurrect a cross-branch close or lose a live-daemon reservation. One artifact lifecycle/output queue makes global destination preflight and reservation updates atomic across otherwise-concurrent caller-owned session queues. Every successful direct, ordered nested-batch, managed replacement, script, Electron, or shutdown close retires its exact identity at that lifecycle point; only the newest pending recording path remains authoritative across current transition replay (including same-timestamp restart rows), and recording starts after a nested close are rejected because upstream can falsely report success. Existing and dangling symlink ancestry, hardlink inode identity, full Unicode/platform case folding, and same-call `outputPath` comparison prevent alias reuse. One shared command-token projection mirrors upstream's full-argv global cleanup before artifact, recording, and presentation parsing; wait-download detection removes only the first timeout pair, follows upstream long/short mode precedence, and accepts both `--download` and `-d` wherever download mode wins; screenshot destinations use upstream's exact-flag, selector-prefix, case-sensitive extension, slash-path, and second-positional rules, while retaining the wrapper's intentional slash-bearing hidden-workspace path normalization. Recording path/URL consumers share a command-local reader that skips complete numeric `--fps` pairs, cursor/contact-sheet switches, and contact-sheet threshold values without rewriting argv or treating them as outer globals; native owns rate, format and extra-argument validation. Current recording transitions are replayed directly; artifact manifests are not treated as reservation events
193
- - keep process-owned cleanup registries for extension-managed sessions and wrapper-launched Electron records separate from the current branch-visible view; `session_tree` restore and wrapper-owned browser commands are serialized with managed-session work, while caller-owned explicit-session commands are serialized by process-local queues keyed to effective canonical namespace/session across prepare helpers (explicit namespace argv overrides inherited `AGENT_BROWSER_NAMESPACE`, including an explicit empty default) and main execution. macOS and Windows additionally normalize and case-fold namespace and session components to match case-insensitive daemon identity. Different caller-owned identities remain concurrent, except namespace-scoped `close --all` drains and exclusively barriers managed plus matching caller-owned work before clearing global namespace state; nested helpers never re-enter the outer queue, policy/route/artifact deltas merge across unrelated managed-state commits, and a separate branch-restore generation guard prevents stale completions from overwriting newer branch-visible state; aggregate artifact results use monotonic revisions so transcript replay cannot lose a concurrently completed entry. Branch switches still must not drop resources the current Pi process owns and must keep fresh-session allocation monotonic
178
+ - keep active recording destination reservations separate from the bounded metadata-only artifact manifest. The same journal retains native recording IDs and start windows; native receipts, including failures, flow through existing artifact metadata and verification. A stop timeout/no-recording result gets one two-second native `session info` query matching namespace/session, ID/path or effective batch start window. Recovery needs terminal native encoder success plus a matching verified file; filesystem presence alone is insufficient, old same-path receipts cannot verify a new take, and freshness starts at capture rather than status-query time. Original attempts and unrelated batch failures remain visible. No second ledger, recorder, sidecar or runtime ffprobe is added. The process-wide map is keyed by canonical namespace/session identity, rebuilt from append-only branch events, and retained for still-live process-owned recordings across branch switches. Shutdown/reload appends both terminal tombstones and still-live reservations onto the current branch so restart cannot resurrect a cross-branch close or lose a live-daemon reservation. One artifact lifecycle/output queue makes global destination preflight and reservation updates atomic across otherwise-concurrent caller-owned session queues. Every successful direct, ordered nested-batch, managed replacement, code, Electron, or shutdown close retires its exact identity at that lifecycle point; only the newest pending recording path remains authoritative across current transition replay (including same-timestamp restart rows), and recording starts after a nested close are rejected because upstream can falsely report success. Existing and dangling symlink ancestry, hardlink inode identity, full Unicode/platform case folding, and same-call `outputPath` comparison prevent alias reuse. One shared command-token projection mirrors upstream's full-argv global cleanup before artifact, recording, and presentation parsing; wait-download detection removes only the first timeout pair, follows upstream long/short mode precedence, and accepts both `--download` and `-d` wherever download mode wins; screenshot destinations use upstream's exact-flag, selector-prefix, case-sensitive extension, slash-path, and second-positional rules, while retaining the wrapper's intentional slash-bearing hidden-workspace path normalization. Recording path/URL consumers share a command-local reader that skips complete numeric `--fps` pairs, cursor/contact-sheet switches, and contact-sheet threshold values without rewriting argv or treating them as outer globals; native owns rate, format and extra-argument validation. Current recording transitions are replayed directly; artifact manifests are not treated as reservation events
179
+ - keep process-owned cleanup registries for extension-managed sessions and wrapper-launched Electron records separate from the current branch-visible view; `session_tree` restore and wrapper-owned browser commands are serialized with managed-session work, while caller-owned explicit-session commands use local queues and shared execution locks keyed to actual socket context and effective canonical namespace/session across prepare helpers (explicit namespace argv overrides inherited `AGENT_BROWSER_NAMESPACE`, including an explicit empty default) and main execution. macOS and Windows additionally normalize and case-fold namespace and session components to match case-insensitive daemon identity. Different caller-owned identities remain concurrent, except namespace-scoped `close --all` drains and exclusively barriers managed plus matching caller-owned work before clearing global namespace state; nested helpers never re-enter the outer queue, policy/route/artifact deltas merge across unrelated managed-state commits, and a separate branch-restore generation guard prevents stale completions from overwriting newer branch-visible state; aggregate artifact results use monotonic revisions so transcript replay cannot lose a concurrently completed entry. Branch switches still must not drop resources the current Pi process owns and must keep fresh-session allocation monotonic
194
180
  - record successful `connect`, `--cdp`, enabled `--auto-connect`, environment-configured CDP/auto-connect, and wrapper Electron attachment identities in branch-visible state. First-use and later content-bearing calls live-check `get url` because attached targets can drift outside Pi. Caller config, file access, launch arguments, and environment pass through unchanged; only wrapper-injected compatibility launch arguments are omitted on active attachments. A terminal successful close removes the marker; a close followed by a later step whose lifecycle reports a browser launch preserves it, while a non-launching diagnostic such as `stream status` leaves the close terminal
195
181
  - when a successful close targets the current extension-managed session, including an explicit `--session <current> close` or an `electron.cleanup` managed-session step, clear page/ref state, mark that session inactive, untrack cleanup ownership, and rotate the next default auto call to a fresh wrapper-generated session name rather than reusing the closed name
196
182
  - on non-quit shutdown such as `/reload`, close off-branch owned managed sessions and off-branch owned Electron launches before clearing process-local ownership, but preserve the current branch-visible active managed session and Electron launch plus that launch's isolated `userDataDir` so reload continuity still works from the active transcript branch. First reuse of a restored Electron attachment checks the recorded namespace/session, live PID/profile presence, and exact browser WebSocket identity against current CDP metadata, then uses upstream `get cdp-url` to verify that the named daemon still targets that browser or one of its current pages. The metadata probe cannot commit daemon provenance merely by spawning; only a matching result permits the existing policy to record it. Ordinary calls, status, and probe share this path under the existing lock. Generic restore-disabled sessions, mismatched connections, quit, and off-branch cleanup are unchanged
197
183
  - expose still-owned off-branch Electron launch records to `electron.status { launchId }`, `electron.status { all: true }`, `electron.probe { launchId }`, and `electron.cleanup`, while leaving default `electron.probe` scoped to the current managed session
198
184
  - if an unnamed fresh launch replaces an active extension-managed session, best-effort close the old managed session after the switch succeeds; `managedSessionOutcome.replacedSessionClosed` records whether that cleanup succeeded, and a failed close keeps the older identity wrapper-owned across transcript resume for explicit follow-up or cleanup
199
185
  - expose `details.browserWindow` and one visible login handoff only when a successful first/fresh local wrapper-managed headed result, including `batch`, is not an attachment and has upstream `lifecycle.effectiveLaunch.browserLaunched: true` and a `created`/`replaced` managed-session outcome. Keep `visibility: "unverified"`: this is launch evidence, never a claim about the user's OS desktop
200
- - leave explicit caller-provided `--session` choices alone unless the caller closes them explicitly, but before any DOM/content-bearing read or interaction against a caller-owned explicit session, live-probe that session with `get url` instead of trusting missing or stale transcript page state; hold the effective canonical namespace/session queue from that probe through semantic snapshot resolution and the main command so another same-instance call cannot change tabs in between. Non-bail batch analysis retains every possible page left by a failed transition up to a fixed bound and blocks later content only when the target is unverified; exceeding the bound also fails closed to exact `batch --bail` guidance. Nested `batch` steps remain unsupported, and raw batch command strings mirror upstream's ASCII-space tokenizer, including quote/backslash handling, rather than splitting on other Unicode whitespace.
186
+ - leave explicit caller-provided `--session` choices alone unless the caller closes them explicitly, but before any DOM/content-bearing read or interaction against a caller-owned explicit session, live-probe that session with `get url` instead of trusting missing or stale transcript page state; hold the effective canonical namespace/session queue from that probe through semantic snapshot resolution and the main command so another cooperating updated Pi call cannot change tabs in between. Non-bail batch analysis retains every possible page left by a failed transition up to a fixed bound and blocks later content only when the target is unverified; exceeding the bound also fails closed to exact `batch --bail` guidance. Nested `batch` steps remain unsupported, and raw batch command strings mirror upstream's ASCII-space tokenizer, including quote/backslash handling, rather than splitting on other Unicode whitespace.
201
187
  - after profiled `open` / `goto` / `navigate` calls, verify the active tab still matches the returned page URL and best-effort switch back when restored profile tabs steal focus
202
188
  - once the wrapper observes tab-drift risk for a session (profile restore correction, overlapping stale opens, or restored session state), later active-tab commands verify the intended tab under the existing session queue before semantic/ref helpers and user commands. Native selection runs only when the intended tab is not already active, because upstream selection clears refs and frame scope even on same-tab reselection. Missing targets, failed selection, and post-selection target mismatches fail before user commands. Caller argv/stdin and native `--pin-tab` / `--no-pin-tab` preferences remain unchanged. Local commands, live `get url`, explicit HTTP `read <url>` (including its flags), URL `a11y`/`vitals`/`web-vitals`, `diff url`, `window new`, URL-bearing recording commands, and explicit tab/navigation/`connect`/`state load` recovery do not require the old target; history back/forward/reload, `pushstate`, and page-content operations still do. The same classifier scans effective batch rows past non-page prefixes until a page dependency or explicit context change, without rewriting user rows or changing bail behavior. For `window new` and `diff url`, observe the resulting URL instead of retaining the old target or treating the requested second URL as redirect evidence. Fold only reached native batch rows when available, discard observations and refs from before those transitions, and let later successful snapshots rebuild refs even when another batch row fails. Retain an observed blank destination after either command instead of recovering the old page; if no final target is observed, use the existing unknown-target state. Caller batch arguments, stdin and bail behavior stay unchanged. Routine same-session commands avoid `tab list` preflights
203
189
  - for sessions with observed tab-drift risk, after a successful command on a known tab target, the wrapper may best-effort restore that same target again if restored/background tabs steal focus after the command returns; routine same-session commands skip this post-command `tab list` probe
@@ -205,7 +191,7 @@ Practical policy:
205
191
  - keep a per-session `refSnapshot` aligned with the last successful `snapshot` (including refs merged from a successful `batch` by taking the last successful `snapshot` step in batch result order): restore it from persisted tool `details` when reloading, resuming, or moving to a different Pi session-tree branch, store bounded ref role/name metadata from the same snapshot for wrapper-side current-ref diagnostics, drop it on successful close commands (`close`, `quit`, or `exit`), replace it with a persisted `page-transition` invalidation after any upstream-executed `record start` attempt (direct or batch; older supported natives can swap the page before their already-active check, so failed starts count; the warning and persisted summary explicitly describe conservative ref invalidation, not an observed page change), a `record restart` with a URL operand, `window new`, `diff url`, or WebMCP `invoke` / `result` / `cancel` (these page-provided tools can mutate, rerender, or navigate; when a spawned `batch` yields no parseable result rows, for example after a wrapper timeout, planned transition steps still record the invalidation), or after a failed non-batch transition command (`eval`, `back`, `forward`, `reload`, `connect`, `state load`, `tab` selection) whose live URL re-verification probe observed the page (a failed transition can still have mutated or replaced the document before throwing, so keeping the verified URL must not keep the prior refs; transcript replay preserves the persisted invalidation summary), and refuse page-scoped `@e…` argv before spawn when the active tab URL no longer matches the snapshot URL, when a ref id was never in that snapshot, when the snapshot state is invalidated, or when a `batch` step would reuse `@e…` on a guarded getter or mutation step after an earlier invalidating step (including `record start`, URL-bearing `record restart`, and WebMCP `invoke` / `result` / `cancel`) without a later `snapshot` step in the same plan; batch steps come from the source upstream actually executes (raw batch argument strings exclusively when any exist — filtering only the exact `--bail` token like upstream — stdin only otherwise, via `getUpstreamEffectiveBatchSteps` in `extensions/agent-browser/lib/orchestration/batch-stdin.ts`); tab recovery (which leaves user argv/stdin and continue-on-error control flow unchanged), artifact/recording preflight, batch screenshot path preparation (parent directories are created for effective raw rows too, without rewriting raw strings), and stale-ref echo args use that same selection so pinning and preflights cannot act on upstream-ignored stdin, while the pre-spawn state-policy validator deliberately keeps scanning parseable stdin alongside argv as a fail-closed content superset and treats stdin parse failures as fatal only when upstream would actually read stdin (its raw-token filter also uses the exact `--bail` token only). Same-snapshot `fill @e…` rows are guarded but do not themselves set that invalidation latch, so ordinary form fills can precede a click/submit row in one batch—see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) for the agent-visible contract and failure text; typed per-session tab/ref/pinning state lives in `extensions/agent-browser/lib/session-page-state.ts` and is updated from `extensions/agent-browser/index.ts` after each tool result
206
192
  - native 0.37 owns first-document setup inheritance for new tabs; the wrapper does not copy headers, user agents or other browser setup itself. The page-summary formatter displays native positive WebMCP availability while retaining raw details; it adds no discovery probe
207
193
  - when a direct or batched WebMCP call returns `status: "pending"`, or `result` / `cancel` fails while that target is unknown, keep its tab target unknown and discard same-call snapshot evidence instead of treating an immediate post-dispatch URL probe as stable; `webmcp result` / `cancel`, `get url`, and explicit navigation remain available while unknown; replace the generic blocked snapshot action with `verify-page-target-after-pending-webmcp` (`get url`), and let a completed `batch --bail` use that verification before `snapshot -i` to re-establish both target and refs
208
- - for top-level non-Electron direct `click` commands with an eligible target, install a bounded in-page target-specific event probe before upstream runs; if upstream reports success but no trusted pointer/mouse/click event reached the resolved target, fail the tool and report `details.clickDispatch` with explicit retry/inspect next actions (the wrapper does not replay clicks in-page). The probe covers `xpath=` targets and current `@e…` / `ref=` refs whose latest stored `refSnapshot.refs` role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`; it requires a unique role/name in the saved snapshot and the live candidates instead of taking a fresh pre-click snapshot that could recycle upstream refs. Ref and XPath candidates must additionally prove identity through a per-probe temporary DOM attribute read back by native `get attr <original-selector> <marker>`. Missing or mismatched identity, including XPath/frame scope mismatches, cleans up the probe and lets native click run without dispatch verification; candidate naming heuristics alone are never proof. Duplicate-name refs pass through without a probe: their old ordinal is not target identity. The probe is intentionally skipped for CSS selector clicks, unresolved `find … click` locators, and `batch`/`job`/`qa` click steps
194
+ - for top-level non-Electron direct `click` commands with an eligible target, install a bounded in-page target-specific event probe before upstream runs; if upstream reports success but no trusted pointer/mouse/click event reached the resolved target, fail the tool and report `details.clickDispatch` with explicit retry/inspect next actions (the wrapper does not replay clicks in-page). The probe covers `xpath=` targets and current `@e…` / `ref=` refs whose latest stored `refSnapshot.refs` role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`; it requires a unique role/name in the saved snapshot and the live candidates instead of taking a fresh pre-click snapshot that could recycle upstream refs. Ref and XPath candidates must additionally prove identity through a per-probe temporary DOM attribute read back by native `get attr <original-selector> <marker>`. Missing or mismatched identity, including XPath/frame scope mismatches, cleans up the probe and lets native click run without dispatch verification; candidate naming heuristics alone are never proof. Duplicate-name refs pass through without a probe: their old ordinal is not target identity. The probe is intentionally skipped for CSS selector clicks, unresolved `find … click` locators, and `batch`/`qa` click steps
209
195
  - derive narrow prompt guards only for concrete evidence invariants: explicitly requested screenshot/recording output paths block browser close until the artifact manifest verifies those paths, while bare inbound attachment paths remain inputs. The wrapper intentionally does not infer broad business/user intent from prompt text such as order/payment/post boundaries; agents must follow those instructions themselves. The artifact guard is bounded preflight policy (`details.promptGuard`, `failureCategory: "policy-blocked"`), not a reusable browser recipe layer
210
196
  - keep the existing page/selector scroll API; explicit CSS-container movement uses `Element.scrollBy` with `behavior: "instant"` before measuring offsets so smooth-scroll CSS cannot cause a premature no-movement failure
211
197
  - reject direct and effective batch `scrollintoview text=...` / `scrollinto text=...` before dispatch because current upstream can falsely report success without movement, while leaving help forms untouched; return only native recovery (`find text ... hover` or fresh snapshot/ref), leaving CSS, XPath, and current-ref behavior upstream-owned
@@ -217,7 +203,7 @@ This is primarily about ownership clarity and avoiding surprise, not adding a he
217
203
 
218
204
  ### Native checkpoint boundary
219
205
 
220
- The additive native `session_checkpoint` hook qualifies only the no-active-resource path. Pi already awaits tools, commands and lifecycle callbacks, including their returned execution queues. The hook checks the existing script controllers/completion promises, recording reservations and dirty retry state, cleanup leases, Electron records/children, attachments, trace owners and routes. It adds no work queue or state journal, and never invokes shutdown, stops recording, or closes a browser during capture.
206
+ The additive native `session_checkpoint` hook qualifies only the no-active-resource path. Pi already awaits tools, commands and lifecycle callbacks, including their returned execution queues. The hook checks the active code controllers/completion promises, recording reservations and dirty retry state, cleanup leases, Electron records/children, attachments, trace owners and routes. It adds no work queue or state journal, and never invokes shutdown, stops recording, or closes a browser during capture.
221
207
 
222
208
  Known native identities come from existing ownership and all native tool-result entries, including off-branch caller-owned/root sessions and the unnamed native default (launch flags can open it even for an explicit URL read). Read-only `session info` must prove each daemon inactive; active, missing-binary and unknown results remain blockers. These probes reuse the existing owned-session context for runtime-owned identities and transcript-proven wrapper-created identities, including off-branch browsers after an abnormal restart, so an ambient `AGENT_BROWSER_SOCKET_DIR` cannot redirect their inspection. Historical routing uses the existing managed-resource events and namespace/session identity, not a name pattern alone, and does not acquire cleanup ownership. For owned or historically owned targets, inspection also checks native ambient routing: the same namespace/name can identify different daemons in different socket roots. A later transcript close does not suppress the historical route's read-only probe, since it may have closed only the ambient caller. Explicit caller choices (even the exact historical name with `sessionMode: "fresh"`), unrelated namespaces and root identities retain native ambient routing. An unused extension needs no browser executable or profile scan. Independent hosts/group members must still coordinate their own writers and resources; a Pi receipt is not a cross-process browser lock.
223
209
 
@@ -232,7 +218,7 @@ With no detached work left, native ingress provides the hold and release; there
232
218
  Input resolution replaces URL-less `open` with native `get url` before planning, retaining requested argv separately. Effective raw/stdin batch rows use the existing native argument parser and raw-over-stdin precedence; only changed rows are rewritten. Native URL reads provide lazy launch and real lifecycle/URL output without navigating an existing page. Explicit navigation and help/version input are unchanged.
233
219
 
234
220
  `agent-browser` startup flags are sticky once a session is already running.
235
- The extension should surface that clearly and avoid hidden restart behavior in v1.
221
+ The extension should surface that clearly and avoid hidden restart behavior.
236
222
 
237
223
  That means explicit startup-scoping flags like `--allowed-domains`, `--auto-connect`, `--args`, `--user-agent`, `--cdp`, `--enable`, `--executable-path`, `--webgpu`, `--no-webmcp`, `--headed`, `--init-script`, `--device`, `--namespace`, `--profile`, `--provider`, `-p`, `--restore`, `--restore-save`, restore check flags, `--session-name`, and `--state` should remain explicit upstream argv choices instead of being wrapped in extra hidden restart or cloning logic. The one deliberate exception is the env-only managed-session `AGENT_BROWSER_RESTORE` key above, which does not inject `--restore` into argv and therefore does not trip launch-scoped `sessionMode: "fresh"` recovery.
238
224
 
@@ -252,11 +238,11 @@ Implementation detail lives in `extensions/agent-browser/lib/launch-scoped-flags
252
238
  - **`--headed`:** Treated as launch-scoped for both enabled and explicit `false` values so a visible-window choice cannot be silently ignored by an already-running managed session.
253
239
  - **`--allowed-domains`:** Treated as launch-scoped so it cannot silently reuse an active implicit browser. Upstream 0.32.0 owns request, worker, popup, and WebRTC containment plus incompatible-mode rejection; the wrapper passes the setting and result through unchanged.
254
240
 
255
- **Sessionless inspection and local commands:** Plain-text help/version probes and upstream commands that do not require a page skip implicit managed-session injection. This includes read-only skills, local auth/profile/setup commands, `session list`, syntactically local state lifecycle operations, explicit URL reads and all-read batches. The read classifier follows native option consumption; invalid read syntax remains native validation rather than triggering page work. State/session rows, restore identifiers, wrapper-prefixed session targets, caller-selected paths, upstream config, file access, launch arguments, and environment variables pass through unchanged. Browser-backed or context-dependent commands receive normal managed-session injection only when the caller did not choose an explicit session. `extensions/agent-browser/lib/page-target-validation.ts` owns only page-target correctness: after an ambiguous tab, attachment, history, script, or state-load transition, content reads require a live `get url` or explicit navigation so the wrapper cannot silently act on the wrong page. Command-shape allowlisting lives in `extensions/agent-browser/lib/command-policy.ts` (`needsManagedSession`), while `extensions/agent-browser/lib/runtime.ts` (`isPlainTextInspectionArgs`, `buildExecutionPlan`) applies that decision to execution planning.
241
+ **Sessionless inspection and local commands:** Plain-text help/version probes and upstream commands that do not require a page skip implicit managed-session injection. This includes read-only skills, local auth/profile/setup commands, `session list`, syntactically local state lifecycle operations, explicit URL reads and all-read batches. The read classifier follows native option consumption; invalid read syntax remains native validation rather than triggering page work. State/session rows, restore identifiers, wrapper-prefixed session targets, caller-selected paths, upstream config, file access, launch arguments, and environment variables pass through unchanged. Browser-backed or context-dependent commands receive normal managed-session injection only when the caller did not choose an explicit session. `extensions/agent-browser/lib/page-target-validation.ts` owns only page-target correctness: after an ambiguous tab, attachment, history, code, or state-load transition, content reads require a live `get url` or explicit navigation so the wrapper cannot silently act on the wrong page. Command-shape allowlisting lives in `extensions/agent-browser/lib/command-policy.ts` (`needsManagedSession`), while `extensions/agent-browser/lib/runtime.ts` (`isPlainTextInspectionArgs`, `buildExecutionPlan`) applies that decision to execution planning.
256
242
 
257
243
  A successful unnamed `sessionMode: "fresh"` launch should become the new extension-managed session so later default calls follow that browser instead of silently snapping back to the older managed session.
258
244
 
259
- When a managed implicit or fresh `--session` plan reaches process execution, `details.managedSessionOutcome` summarizes the managed-session transition: on **success**, statuses such as `created`, `replaced`, `unchanged`, or `closed` describe what became current (including successful close commands: `close`, `quit`, or `exit`); on **failure** (launch error, timeout, missing binary, failed close command, and similar), `preserved` vs `abandoned` captures whether a prior managed session stayed current or no managed session ended up active, plus related names and booleans. Post-launch failures after the fresh browser has already opened the target page—such as **`qa`** reclassification, a later failed `job`/batch step, or a fresh batch timeout with recovered current-page evidence—preserve that fresh session as current and set the visible recovery text to say the fresh launch became current instead of reverting the agent to the old session. Failing calls that used `sessionMode: "fresh"`, plus otherwise-successful replacements whose automatic old-session close failed, append a short `Managed session outcome: …` line to model-visible text so the next default `sessionMode: "auto"` hop and any retained cleanup ownership are obvious; `"auto"` failures may still populate the struct without that extra line. Implementation and field semantics live in `extensions/agent-browser/lib/session-page-state.ts` and `extensions/agent-browser/lib/orchestration/browser-run/session-state.ts`; agent contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details); checklist row `RQ-0077` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md).
245
+ When a managed implicit or fresh `--session` plan reaches process execution, `details.managedSessionOutcome` summarizes the managed-session transition: on **success**, statuses such as `created`, `replaced`, `unchanged`, or `closed` describe what became current (including successful close commands: `close`, `quit`, or `exit`); on **failure** (launch error, timeout, missing binary, failed close command, and similar), `preserved` vs `abandoned` captures whether a prior managed session stayed current or no managed session ended up active, plus related names and booleans. Post-launch failures after the fresh browser has already opened the target page—such as **`qa`** reclassification, a later failed native batch step, or a fresh batch timeout with recovered current-page evidence—preserve that fresh session as current and set the visible recovery text to say the fresh launch became current instead of reverting the agent to the old session. Failing calls that used `sessionMode: "fresh"`, plus otherwise-successful replacements whose automatic old-session close failed, append a short `Managed session outcome: …` line to model-visible text so the next default `sessionMode: "auto"` hop and any retained cleanup ownership are obvious; `"auto"` failures may still populate the struct without that extra line. Implementation and field semantics live in `extensions/agent-browser/lib/session-page-state.ts` and `extensions/agent-browser/lib/orchestration/browser-run/session-state.ts`; agent contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details); checklist row `RQ-0077` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md).
260
246
 
261
247
  ## Preferring the native tool
262
248
 
@@ -272,7 +258,7 @@ Upstream restore-state persistence remains upstream-owned. The wrapper passes an
272
258
 
273
259
  ### `pi-agent-browser-native` owns
274
260
 
275
- - tool registration and schema (including the optional `semanticAction` compilation path to upstream `find` or `select`)
261
+ - tool registration and schema (including the advanced `agent_browser_action` compilation path to upstream `find` or `select`)
276
262
  - subprocess execution and JSON parsing through `buildAgentBrowserProcessEnv` in `extensions/agent-browser/lib/process.ts`: copies the parent process environment so user-approved provider credentials and other runtime variables reach upstream, then applies wrapper overrides such as the managed socket directory and clamped default operation timeout
277
263
  - clear missing-binary errors
278
264
  - filtered `network requests --current-page` / `--current-origin` / `--current-url` preparation results use the same `redactPresentationData` credential redactor as ordinary diagnostics before populating `details.data` or writing `outputPath`
@@ -304,18 +290,18 @@ This mirrors the playbook contract pattern described in [`TOOL_CONTRACT.md`](TOO
304
290
 
305
291
  ## Not the right design
306
292
 
307
- V1 should avoid:
308
- - a wide family of bespoke browser tools
293
+ Avoid:
294
+ - duplicating native CLI semantics in bespoke tools
309
295
  - compatibility layers for old `agent-browser` versions
310
296
  - deep embedded SDK-style integration
311
297
  - embedding a human-browsable browser UI inside `pi`
312
298
  - a slash-command-heavy UX
313
299
 
314
- ## V1 priorities
300
+ ## Product priorities
315
301
 
316
302
  ### Must have
317
303
 
318
- - one native `agent_browser` tool
304
+ - compact direct/code tools and native advanced capability discovery
319
305
  - direct `--json` execution
320
306
  - optional stdin support
321
307
  - implicit-session convenience
@@ -332,7 +318,7 @@ V1 should avoid:
332
318
 
333
319
  The architecture should stay:
334
320
  - thin
335
- - latest-only
321
+ - aligned to the recommended upstream release and stable support floor
336
322
  - close to upstream
337
323
  - native where it matters
338
324
  - low-maintenance