pi-agent-browser-native 0.9.1 → 0.9.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +26 -0
- package/README.md +35 -8
- package/dist/extensions/agent-browser/index.js +39 -16
- package/dist/extensions/agent-browser/lib/batch-lifecycle.js +6 -3
- package/dist/extensions/agent-browser/lib/browser-transcript.js +2 -1
- package/dist/extensions/agent-browser/lib/launch-scoped-flags.js +4 -0
- package/dist/extensions/agent-browser/lib/native-output.js +2 -1
- package/dist/extensions/agent-browser/lib/orchestration/batch-stdin.js +2 -8
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.js +12 -2
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +62 -10
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-policy.js +18 -12
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +33 -24
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +111 -53
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-observation.js +6 -2
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state.js +32 -20
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +37 -32
- package/dist/extensions/agent-browser/lib/orchestration/native-session-defaults.js +30 -11
- package/dist/extensions/agent-browser/lib/orchestration/output-file.js +3 -1
- package/dist/extensions/agent-browser/lib/page-target-validation.js +10 -3
- package/dist/extensions/agent-browser/lib/pi-tool-rendering.js +2 -1
- package/dist/extensions/agent-browser/lib/playbook.js +7 -6
- package/dist/extensions/agent-browser/lib/process-environment.js +12 -3
- package/dist/extensions/agent-browser/lib/process.js +6 -6
- package/dist/extensions/agent-browser/lib/read-confirmation.js +56 -16
- package/dist/extensions/agent-browser/lib/results/confirmation.js +3 -0
- package/dist/extensions/agent-browser/lib/results/envelope.js +5 -1
- package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +10 -1
- package/dist/extensions/agent-browser/lib/results/presentation/common.js +1 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +34 -1
- package/dist/extensions/agent-browser/lib/results/presentation/registry.js +5 -1
- package/dist/extensions/agent-browser/lib/results/presentation.js +15 -2
- package/dist/extensions/agent-browser/lib/results/selector-recovery.js +5 -5
- package/dist/extensions/agent-browser/lib/runtime.js +5 -1
- package/dist/extensions/agent-browser/lib/session-page-state.js +35 -16
- package/docs/ARCHITECTURE.md +10 -9
- package/docs/COMMAND_REFERENCE.md +37 -13
- package/docs/RELEASE.md +6 -5
- package/docs/REQUIREMENTS.md +3 -1
- package/docs/SESSION_STORAGE.md +1 -1
- package/docs/SUPPORT_MATRIX.md +12 -3
- package/docs/TOOL_CONTRACT.md +29 -20
- package/package.json +4 -4
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -33,7 +33,7 @@ The extension should:
|
|
|
33
33
|
- resolve `agent-browser` from `PATH`
|
|
34
34
|
- invoke it with native Node `spawn` on POSIX and `cross-spawn` on Windows. For the recognized global npm CMD template, public `which`/`path-key` resolution selects the same launcher and Node invokes its exact adjacent native executable directly, preserving literal CR/LF. Recognition requires the stock template, package identity and executable; custom launchers, other layouts, ambiguous search environments and worker-thread calls retain `cross-spawn`. No later installation is substituted. Custom or unrecognized batch files can still truncate literal newlines. Caller commands and helper probes retain empty operands, literal doublequotes, command/subcommand adjacency and explicit default `--namespace ""`. Keep piped stdin/stdout/stderr and await full Windows process-tree termination with `taskkill /T /F` on timeout or abort before falling back to the direct child signal. Missing commands use the spawner's `ENOENT`
|
|
35
35
|
- bound native process-identity queries at ten seconds on Windows, including PowerShell startup, and five seconds on POSIX. These probes preserve PID/start-time comparison and fail closed when identity is unavailable; their budget is separate from lock contention and browser command watchdogs
|
|
36
|
-
- inject `--json
|
|
36
|
+
- inject `--json` only when no caller output mode is selected; explicit `--json false` retains opaque redacted native text and process outcomes without interpreting page text as trusted receipts or a session target. Optional booleans follow upstream last-wins semantics; structured helpers stay strict JSON. Nonzero exits with stderr retain the native error ahead of parse diagnostics, while zero-exit malformed machine output remains a strict failure. Empty stdin batch rows pass validation and native execution skips them without rewriting caller input. Caller-requested JSON also uses the existing `buildJsonVisibleContent` envelope for early snapshot/network filters, scroll results, and preparation failures, retaining metadata in `details` and keeping output-file notices out of visible JSON. Help/version stays native text
|
|
37
37
|
- complete each upstream invocation when the direct `agent-browser` child exits even if Node delays `"close"`: piped stdio can stay referenced by longer-lived descendant processes, so `runAgentBrowserProcess` watches `exit` and `close` together, leaves stdio intact during a short post-`exit` grace so normal `close` can still win, destroys streams only when the post-`exit` fallback fires, and prefers `close` codes then wrapper timeout (`124`) over signal-shaped `exit` codes (`watchSpawnedChildCompletion` / `resolveSpawnedChildExitCode` in `extensions/agent-browser/lib/process.ts`) so the tool cannot hang after the CLI process has already terminated
|
|
38
38
|
- support optional stdin only for `eval --stdin`, `batch`, `auth save --password-stdin`, and internally generated QA/source batch plans, rejecting other command/stdin combinations before launch; Electron manages its own input
|
|
39
39
|
- support optional top-level `outputPath` for successful browser results and failed/pending/recovered recording receipts. Recording exports retain success/error, original attempt, native data, artifacts and recovery provenance; they do not treat a written receipt as a successful video. Other successful results write `details.data` (or model-facing text when no structured data exists) to a caller-requested local file and reporting `details.outputFile`, without changing upstream argv semantics or overwriting a browser artifact when both destinations resolve to the same file. If presentation compacted direct data, a batch result row, or a whole batch, apply the relevant command-specific redactor before spilling and rehydrate each full pre-compaction value only from its matching live wrapper-manifest spill; fail without writing compact metadata when any required spill is unavailable or untrusted
|
|
@@ -64,7 +64,7 @@ All participating Pi processes must restart with the updated package. The lock i
|
|
|
64
64
|
|
|
65
65
|
Native tool declarations share Browser namespace metadata and `outputSchema`; `structuredContent` reuses the bounded redacted observation or its existing compact/spill object. Loader inventory and normalized provider search keep their existing shapes. Internal ownership/replay details are not exported wholesale. This interoperates with host-selected native codemode without auto-enabling it or replacing the Browser VM. A path/handle is not native image data; verified content image blocks remain separate. Final results return native `isError` and retain prose failure notices while explicit JSON stays parseable and unchanged. Per-row TUI memoization skips unchanged formatting/Text updates without caching browser state or observations across calls.
|
|
66
66
|
|
|
67
|
-
`orchestration/browser-run/screenshot-observation.ts` brackets standalone screenshots with CSS viewport/document/scroll/DPR/frame samples. Existing Pi image-dimension primitives measure saved pixels. Matching samples and dimensions can establish a crop mapping; changed, unmeasured batch, scrolled/ambiguous element, zoom, or frame-uncertain captures remain unknown. These samples are not an atomic capture guarantee. Native mouse input uses current viewport CSS pixels, not image pixels; full-page/element origin and any Pi resize must be accounted for separately. Unchanged conditional screenshots attach nothing. See [image observations](TOOL_CONTRACT.md#image-observations).
|
|
67
|
+
`orchestration/browser-run/screenshot-observation.ts` brackets standalone screenshots with CSS viewport/document/scroll/DPR/frame samples. Existing Pi image-dimension primitives measure saved pixels. Matching samples and dimensions can establish a crop mapping; changed, unmeasured batch, scrolled/ambiguous element, zoom, or frame-uncertain captures remain unknown. These samples are not an atomic capture guarantee. Native mouse input uses current viewport CSS pixels, not image pixels; full-page/element origin and any Pi resize must be accounted for separately. Unchanged conditional screenshots attach nothing. Native Lightpanda engine metadata labels verified images as `rendering: "text"` through shared attachment handling, including batch/code. The existing standalone samples also recognize the Lightpanda browser brand without another probe or engine cache. Text-rendered images retain unknown geometry and no crop/scale mapping, even when synthetic dimensions match. See [image observations](TOOL_CONTRACT.md#image-observations).
|
|
68
68
|
|
|
69
69
|
### Agent-first UX
|
|
70
70
|
|
|
@@ -89,7 +89,7 @@ Config layers merge in that order: global, project, override. The shared policy
|
|
|
89
89
|
|
|
90
90
|
`agent_browser_web_search` availability is conditional. Startup registration uses global, override, and environment fallback config without reading project-local config before Pi trust context exists; trusted project config can register the companion tool on `session_start`, and every execution reloads the final session config so `webSearch.enabled: false` still prevents a request even if a startup credential made the tool visible. A global disable is the normal user default and can still be overridden by project config or `PI_AGENT_BROWSER_CONFIG`; a project disable applies to one repo; an explicit `PI_AGENT_BROWSER_CONFIG` file with `webSearch.enabled: false` is the highest-priority hard disable for that run. Literal and env-backed sources must resolve before they make the tool available; command-backed sources are considered configured without running the command until tool execution, so secret managers do not slow startup or prompt unexpectedly. The tool resolves the selected key lazily, chooses Exa or Brave from available credentials (preferring Exa by default unless `webSearch.preferredProvider` says otherwise), then follows one provider-agnostic execution path through provider adapters for request building, HTTP JSON fetch, response normalization, and provider-specific detail fields. Exa search-type precedence is per-call value, merged config default, then `auto`; bounded Exa filters stay on the Exa adapter and explicit Exa-only options fail before a Brave request. The tool calls Exa `/search` with highlights plus a fixed primary-source/version/date/distinct-result system prompt, or Brave Search, then removes later exact normalized-URL duplicates while retaining provider order. It does not guess across distinct paths/query URLs or overfetch replacements; provider page dates remain clues rather than claimed crawl/version evidence. Compact result details report any `duplicatesRemoved` without exposing keys.
|
|
91
91
|
|
|
92
|
-
Global/override package profile names with `policy: "always"` and executable defaults bootstrap inactive automatic root browsers through scoped native environment. Active roots retain native settings; paths, other policies, and project-only values remain advisory. Direct and code calls selecting that root share the same configuration. Unrelated explicit sessions and Electron launches retain their separate native/owned rules.
|
|
92
|
+
Global/override package profile names with `policy: "always"` and executable defaults bootstrap inactive automatic root browsers through scoped native environment only when CLI → environment → native config resolves to Chrome. Lightpanda receives no wrapper Chrome defaults; explicit native settings are untouched. Active roots retain native settings; paths, other policies, and project-only values remain advisory. Direct and code calls selecting that root share the same configuration. Unrelated explicit sessions and Electron launches retain their separate native/owned rules.
|
|
93
93
|
|
|
94
94
|
### Prompt guidance budget
|
|
95
95
|
|
|
@@ -136,9 +136,9 @@ The optional `pi-change-working-dir` event protocol supplies one invocation exec
|
|
|
136
136
|
|
|
137
137
|
### Default
|
|
138
138
|
|
|
139
|
-
An explicit native `session` default (user/project JSON or `AGENT_BROWSER_SESSION`) selects a caller-owned browser across Pi sessions. Per-call session/namespace flags win. `lib/orchestration/native-session-defaults.ts` reads session/namespace and contributing launch defaults from native config paths and uses sessionless native `--config <path> --json session` to validate each contributing file and resolve its session. Native config continues to own launch/profile/storage settings; the adapter does not duplicate its schema. Explicit config
|
|
139
|
+
An explicit native `session` default (user/project JSON or `AGENT_BROWSER_SESSION`) selects a caller-owned browser across Pi sessions. Per-call session/namespace flags win. `lib/orchestration/native-session-defaults.ts` reads session/namespace and contributing launch defaults from native config paths and uses sessionless native `--config <path> --json session` to validate each contributing file and resolve its session. Native config continues to own launch/profile/storage settings; the adapter does not duplicate its schema. Explicit config, per-call idle settings, `--action-policy`, `--confirm-actions`, `--debug` and `--no-auto-dialog` are scoped to helpers through `lib/process-environment.ts`. Boolean selections also supply CLI operands to helpers lacking an explicit selection, because native ORs environment/config booleans and only CLI `false` overrides true defaults. Explicit debug false removes the debug environment variable because native daemon logging tests its presence; no-auto-dialog false retains native `0`. Caller argv stays intact and the environment scope ends with the call. The effective confirmation setting is retained in existing `SessionPageState` and canonical page events, resolved inside the identity queue before daemon inspection and helpers. First launch and new explicit `--confirm-actions` / `--config` selections use native argv → environment → config precedence; bare follow-ups retain selected strings, including explicit `""`, over ambient environment/discovered-config edits. Native unset is not retained and continues native default resolution; no config freezing or ambient-change detector is needed. Successful session/namespace close and successfully closed managed replacements clear it. Cleanup scopes every probe and close to the old identity's setting; failed close retains its page/policy continuity. Off-branch live owners keep their selected strings in the same page state until cleanup; an ordered post-close relaunch retains the new daemon's setting. Resume and branch restore use the same reducer. Native still owns policy changes and pending confirmations; there is no policy hot-reload, pending-command store, persistent config cache, or new lifecycle service. Electron launch and host probes retain the same selected confirmation setting. Begin records preserve interrupted intent; a definite no-dispatch/native CLI parse/config/local-inspection rejection restores the previous setting only when no earlier browser helper could have established the request. Other policy flags remain call-scoped.
|
|
140
140
|
|
|
141
|
-
When no native session, attachment, or explicit fresh launch is selected, `sessionMode: "auto"` uses `pi-root-<sha256(root Pi ID)[0:24]>` through the existing caller-owned named-session path. Descendants receive `PI_SUBAGENT_ROOT_SESSION_ID` from pi-subagents; root processes use `ctx.sessionManager.getSessionId()`. No cwd, human fork ancestry, run registry, or process-global assignment participates. The same name is the default native restore key. Existing native daemon inspection runs inside the existing identity queue: inactive roots receive bootstrap defaults, while active roots retain native launch settings and their current restore key. Scoped subprocess environment keeps helper settings consistent; no launch-settings journal is added. Root-name explicit follow-ups retain those defaults; unrelated explicit names do not. Global/override package `browser.defaultProfile` with `policy: "always"` supplies native Chrome profile names only, plus the configured executable. Paths and project-only profile settings stay advisory. Native config/environment and explicit launch settings win. The communal source is copied by native Chrome launch, never by the wrapper, and each root's restore writes remain separate. Native JSON restore omits IndexedDB, service workers, and page-memory credentials; application-level restart verification is required. Parent and child quit leave the root browser alive; shared execution locks coordinate updated cooperating processes without a broker. An active older/fresh managed session keeps its existing lifecycle. A configured session has the same precedence over `fresh` as a literal `--session`; code follows the selected ordinary browser lifecycle; Electron launch retains its owned lifecycle. Electron launch suppresses the native session environment default only while allocating its generated session; ordinary browser follow-ups still honor the configured native default, so use the returned Electron session explicitly.
|
|
141
|
+
When no native session, attachment, or explicit fresh launch is selected, `sessionMode: "auto"` uses `pi-root-<sha256(root Pi ID)[0:24]>` through the existing caller-owned named-session path. Descendants receive `PI_SUBAGENT_ROOT_SESSION_ID` from pi-subagents; root processes use `ctx.sessionManager.getSessionId()`. No cwd, human fork ancestry, run registry, or process-global assignment participates. The same name is the default native restore key. Existing native daemon inspection runs inside the existing identity queue: inactive roots receive bootstrap defaults, while active roots retain native launch settings and their current restore key. Scoped subprocess environment keeps helper settings consistent; no launch-settings journal is added. Root-name explicit follow-ups retain those defaults; unrelated explicit names do not. Global/override package `browser.defaultProfile` with `policy: "always"` supplies native Chrome profile names only, plus the configured executable, only when the effective engine is Chrome. Paths and project-only profile settings stay advisory. Native config/environment and explicit launch settings win. The communal source is copied by native Chrome launch, never by the wrapper, and each root's restore writes remain separate. Native JSON restore omits IndexedDB, service workers, and page-memory credentials; application-level restart verification is required. Parent and child quit leave the root browser alive; shared execution locks coordinate updated cooperating processes without a broker. An active older/fresh managed session keeps its existing lifecycle. A configured session has the same precedence over `fresh` as a literal `--session`; code follows the selected ordinary browser lifecycle; Electron launch retains its owned lifecycle. Electron launch suppresses the native session environment default only while allocating its generated session; ordinary browser follow-ups still honor the configured native default, so use the returned Electron session explicitly.
|
|
142
142
|
|
|
143
143
|
Why:
|
|
144
144
|
- works out of the box
|
|
@@ -151,7 +151,7 @@ Browser-independent calls to an already-owned session reuse its idle timeout, na
|
|
|
151
151
|
|
|
152
152
|
An explicit native `session info` call is the preflight: daemon activity/PID is separate from `runtime.browser` liveness, Chrome PID, exact profile, tabs and launched/attached ownership. Pi adds its own cleanup ownership from existing ownership records. Unknown native fields stay unknown; the wrapper does not infer the browser from daemon/config state or add a host process scan. Native `runtime.recording.current` / `last` receipts and protocol capabilities remain visible. Full fields require companion native support; receipts retained by the native daemon are not durable after it exits.
|
|
153
153
|
|
|
154
|
-
`lib/read-confirmation.ts` recognizes
|
|
154
|
+
`lib/read-confirmation.ts` recognizes strict native control envelopes with actual command provenance. Ordered pending/cleared `readConfirmation` markers in the existing `SessionPageState` and transcript preserve exact ID, canonical identity and compact originating command/native action across resume and branch replay; they never store argv or command blobs. For originating `tab new` commands, the same decoder and presentation validator also recognize paired `after_confirmation: ["open", <original URL>]` / nonempty `guidance` transport metadata, using the existing navigation operand reader and retaining strict control keys. It preserves conditional, redacted prose without persisting or executing continuation arguments; page/eval/HTTP data never acquires that provenance. Exact matching guarded decisions suppress only page/pinning helpers that could overwrite the native pending slot. For native batch this applies only when its first effective row is that matching decision, using native raw-argv-over-stdin selection; remaining page/ref checks stay intact. Completed browser-backed decisions still reconcile page/ref state, report inner failures and retire confirmed close normally; native ID validation is not implied. Explicit URL-read provenance remains separate: only `capabilities.readRequiresConfirmation: true` grants browser independence with native ID matching. Legacy URL reads keep normal checks; unrelated or forged/page-shaped observations gain no exemption. A per-call helper observer updates this same marker and stops later same-identity page helpers after a new pending control. Cold reopen exposes that helper decision without dispatching the original getter or replaying open. Confirmed full snapshots feed the existing ordered ref fold, while pending captures cannot replace completed refs. The existing cleared marker records one-use freshness only for a final successful complete confirmed capture; the next same-identity call consumes it in live state and replay. The semantic visible-ref resolver uses the same eligible capture for role click/check/fill and unique role/label select, preserving editable evidence and action payloads. This avoids gated semantic/freshness-snapshot retry loops without bypassing target, membership, invalidation or native generation checks. Pending snapshots use the common confirmation renderer rather than the empty-snapshot formatter. Required live URL verification replaces the call's remembered target before semantic resolution and direct-ref preflight, dropping old title/target identity when the complete URL changes. The existing stale-ref guard compares the complete approved URL, including fragments, only on one-use confirmed-capture calls; ordinary ref/tab URL comparisons remain fragment-insensitive. Caller-owned, attached and wrapper-owned managed sessions therefore reject refs from a different approved route without requiring resume, restore pinning or another gated snapshot. DOM-only renames on the same page after that sample are not resampled on this one use; ordinary checks resume afterward. Attachment replay honors realized compact close commands, including confirmed closes. There is no second confirmation store or wrapper authorization system. Click-dispatch probes keep their ordinary diagnostic window but self-clean their page listeners and marker after the per-call process timeout plus the existing two-second cleanup grace. Normal cleanup cancels that timer; expiry never needs a browser helper that could overwrite the pending slot.
|
|
155
155
|
|
|
156
156
|
### Native observation state
|
|
157
157
|
|
|
@@ -183,7 +183,7 @@ Practical policy:
|
|
|
183
183
|
- redact snapshot spill payloads before writing them, clean up process-private temp spill artifacts on shutdown, and keep persisted-session spill files in a private session-scoped artifact directory for `details.fullOutputPath` after reload/resume, with a 32 MiB per-session budget by default; `PI_AGENT_BROWSER_SESSION_ARTIFACT_MAX_BYTES=0` skips persistent-spill enumeration and eviction without changing temporary spill cleanup
|
|
184
184
|
- keep explicit screenshots, downloads, PDFs, traces, HAR captures, and recordings written to caller-chosen paths on disk after a successful upstream close command (`close`, `quit`, or `exit`); before artifact-producing commands run, create missing parent directories for requested host paths, while native download always owns the click and download, including loopback links, generated Blob exports, and redirects; never substitute an anchor-href fetch. When the internal recent artifact view has entries, successful close commands also surface `details.artifactCleanup` and a compact `Artifact lifecycle` note pointing to structured explicit paths so operators remove files with normal host tools—the native tool does not delete arbitrary user paths (`extensions/agent-browser/lib/orchestration/browser-run/diagnostics.ts`, `getArtifactCleanupGuidance`); contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), checklist `RQ-0079` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md)
|
|
185
185
|
- reconstruct the current branch-visible extension-managed session, every transcript-proven still-active wrapper-owned managed identity, page-scoped refs, journal-ordered artifact-row changes, unbounded active-recording reservation events, and Electron launch records from the active transcript branch on `session_start` and `session_tree` so later default and explicit off-current calls keep following owned browsers after resume/reload or branch switching; restore also honors successful explicit `--session <wrapper-owned> close` rows, terminal nested-batch close outcomes even when aggregate artifact verification failed, and `electron.cleanup` managed-session steps so closed wrapper-owned sessions are not resurrected; a nested close invalidates the pre-close page target so a lifecycle-proven relaunch at `about:blank` is not treated as stale focus drift; explicit lifecycle evidence that a later diagnostic did not launch a browser preserves the terminal close, while any later row—including a failed row—whose lifecycle reports a browser launch keeps active/attached provenance; failed-step presentation persists only that bounded launch boolean so transcript replay reaches the same decision, missing lifecycle evidence remains conservatively active even on the first managed call, successful closes clear wrapper trace/profiler ownership before ordered later successful rows can rebuild it, namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, and recording starts after close are rejected before spawn
|
|
186
|
-
- keep active recording destination reservations separate from the bounded metadata-only artifact manifest. The same journal retains native recording IDs and start windows; native receipts, including failures, flow through existing artifact metadata and verification. A stop timeout/no-recording result gets one two-second native `session info` query matching namespace/session, ID/path or effective batch start window. Recovery needs terminal native encoder success plus a matching verified file; filesystem presence alone is insufficient, old same-path receipts cannot verify a new take, and freshness starts at capture rather than status-query time. Original attempts and unrelated batch failures remain visible. No second ledger, recorder, sidecar or runtime ffprobe is added. The process-wide map is keyed by canonical namespace/session identity, rebuilt from append-only branch events, and retained for still-live process-owned recordings across branch switches. Shutdown/reload appends both terminal tombstones and still-live reservations onto the current branch so restart cannot resurrect a cross-branch close or lose a live-daemon reservation. One artifact lifecycle/output queue makes global destination preflight and reservation updates atomic across otherwise-concurrent caller-owned session queues. Every successful direct, ordered nested-batch, managed replacement, code, Electron, or shutdown close retires its exact identity at that lifecycle point; only the newest pending recording path remains authoritative across current transition replay (including same-timestamp restart rows), and recording starts after a nested close are rejected because upstream can falsely report success. Existing and dangling symlink ancestry, hardlink inode identity, full Unicode/platform case folding, and same-call `outputPath` comparison prevent alias reuse. One shared command-token projection mirrors upstream's full-argv global cleanup before artifact, recording, and presentation parsing; wait-download detection removes only the first timeout pair, follows upstream long/short mode precedence, and accepts both `--download` and `-d` wherever download mode wins; screenshot destinations use upstream's exact-flag, selector-prefix, case-sensitive extension, slash-path, and second-positional rules, while retaining the wrapper's intentional slash-bearing hidden-workspace path normalization. Recording path/URL consumers share a command-local reader that skips complete numeric `--fps` pairs, cursor/contact-sheet switches, and contact-sheet threshold values without rewriting argv or treating them as outer globals; native owns rate, format and extra-argument validation. Current recording transitions are replayed directly; artifact manifests are not treated as reservation events
|
|
186
|
+
- keep active recording destination reservations separate from the bounded metadata-only artifact manifest. The same journal retains native recording IDs and start windows; native receipts, including failures, flow through existing artifact metadata and verification. A restart without a terminal previous-take receipt retires the prior pending artifact only when native acknowledges `restarted: true`; rejected or pending restarts retain its reservation and stop action. A stop timeout/no-recording result gets one two-second native `session info` query matching namespace/session, ID/path or effective batch start window. Recovery needs terminal native encoder success plus a matching verified file; filesystem presence alone is insufficient, old same-path receipts cannot verify a new take, and freshness starts at capture rather than status-query time. Original attempts and unrelated batch failures remain visible. No second ledger, recorder, sidecar or runtime ffprobe is added. The process-wide map is keyed by canonical namespace/session identity, rebuilt from append-only branch events, and retained for still-live process-owned recordings across branch switches. Shutdown/reload appends both terminal tombstones and still-live reservations onto the current branch so restart cannot resurrect a cross-branch close or lose a live-daemon reservation. One artifact lifecycle/output queue makes global destination preflight and reservation updates atomic across otherwise-concurrent caller-owned session queues. Every successful direct, ordered nested-batch, managed replacement, code, Electron, or shutdown close retires its exact identity at that lifecycle point; only the newest pending recording path remains authoritative across current transition replay (including same-timestamp restart rows), and recording starts after a nested close are rejected because upstream can falsely report success. Existing and dangling symlink ancestry, hardlink inode identity, full Unicode/platform case folding, and same-call `outputPath` comparison prevent alias reuse. One shared command-token projection mirrors upstream's full-argv global cleanup before artifact, recording, and presentation parsing; wait-download detection removes only the first timeout pair, follows upstream long/short mode precedence, and accepts both `--download` and `-d` wherever download mode wins; screenshot destinations use upstream's exact-flag, selector-prefix, case-sensitive extension, slash-path, and second-positional rules, while retaining the wrapper's intentional slash-bearing hidden-workspace path normalization. Recording path/URL consumers share a command-local reader that skips complete numeric `--fps` pairs, cursor/contact-sheet switches, and contact-sheet threshold values without rewriting argv or treating them as outer globals; native owns rate, format and extra-argument validation. Current recording transitions are replayed directly; artifact manifests are not treated as reservation events
|
|
187
187
|
- keep process-owned cleanup registries for extension-managed sessions and wrapper-launched Electron records separate from the current branch-visible view; `session_tree` restore and wrapper-owned browser commands are serialized with managed-session work, while caller-owned explicit-session commands use local queues and shared execution locks keyed to actual socket context and effective canonical namespace/session across prepare helpers (explicit namespace argv overrides inherited `AGENT_BROWSER_NAMESPACE`, including an explicit empty default) and main execution. macOS and Windows additionally normalize and case-fold namespace and session components to match case-insensitive daemon identity. Different caller-owned identities remain concurrent, except namespace-scoped `close --all` drains and exclusively barriers managed plus matching caller-owned work before clearing global namespace state; nested helpers never re-enter the outer queue, policy/route/artifact deltas merge across unrelated managed-state commits, and a separate branch-restore generation guard prevents stale completions from overwriting newer branch-visible state; artifact-row changes use the existing artifact queue and journal order so replay retains concurrently completed entries. Branch switches still must not drop resources the current Pi process owns and must keep fresh-session allocation monotonic
|
|
188
188
|
- record successful `connect`, `--cdp`, enabled `--auto-connect`, environment-configured CDP/auto-connect, and wrapper Electron attachment identities in branch-visible state. First-use and later content-bearing calls live-check `get url` because attached targets can drift outside Pi. Caller config, file access, launch arguments, and environment pass through unchanged; only wrapper-injected compatibility launch arguments are omitted on active attachments. A terminal successful close removes the marker; a close followed by a later step whose lifecycle reports a browser launch preserves it, while a non-launching diagnostic such as `stream status` leaves the close terminal
|
|
189
189
|
- when a successful close targets the current extension-managed session, including an explicit `--session <current> close` or an `electron.cleanup` managed-session step, clear page/ref state, mark that session inactive, untrack cleanup ownership, and rotate the next default auto call to a fresh wrapper-generated session name rather than reusing the closed name
|
|
@@ -199,7 +199,7 @@ Practical policy:
|
|
|
199
199
|
- keep a per-session `refSnapshot` aligned with the last successful `snapshot` (including refs merged from a successful `batch` by taking the last successful `snapshot` step in batch result order): restore its once-stored ancestral definition when reloading, resuming, or moving to a different Pi session-tree branch, retain full membership and accessible metadata from that capture for wrapper-side diagnostics, drop it on successful close commands (`close`, `quit`, or `exit`), replace it with a persisted `page-transition` invalidation after any upstream-executed `record start` attempt (direct or batch; older supported natives can swap the page before their already-active check, so failed starts count; the warning and persisted summary explicitly describe conservative ref invalidation, not an observed page change), a `record restart` with a URL operand, `window new`, `diff url`, or WebMCP `invoke` / `result` / `cancel` (these page-provided tools can mutate, rerender, or navigate; when a spawned `batch` yields no parseable result rows, for example after a wrapper timeout, planned transition steps still record the invalidation), or after a failed transition command (`eval`, `back`, `forward`, `reload`, `connect`, `state load`, `tab` selection), including reached batch transitions and failed batch `open` rows whose live URL re-verification probe observed the page (a failed transition can still have mutated or replaced the document before throwing, so keeping the verified URL must not keep the prior refs; transcript replay preserves the persisted invalidation summary), and refuse page-scoped `@e…` argv before spawn when the active tab URL no longer matches the snapshot URL, when a ref id was never in that snapshot, when the snapshot state is invalidated, or when a `batch` step would reuse `@e…` on a guarded getter or mutation step after an earlier invalidating step (including `record start`, URL-bearing `record restart`, and WebMCP `invoke` / `result` / `cancel`) without a later `snapshot` step in the same plan; batch steps come from the source upstream actually executes (raw batch argument strings exclusively when any exist — filtering only the exact `--bail` token like upstream — stdin only otherwise, via `getUpstreamEffectiveBatchSteps` in `extensions/agent-browser/lib/orchestration/batch-stdin.ts`); tab recovery (which leaves user argv/stdin and continue-on-error control flow unchanged), artifact/recording preflight, batch screenshot path preparation (parent directories are created for effective raw rows too, without rewriting raw strings), and stale-ref echo args use that same selection so pinning and preflights cannot act on upstream-ignored stdin, while the pre-spawn state-policy validator deliberately keeps scanning parseable stdin alongside argv as a fail-closed content superset and treats stdin parse failures as fatal only when upstream would actually read stdin (its raw-token filter also uses the exact `--bail` token only). Same-snapshot `fill @e…` rows are guarded but do not themselves set that invalidation latch, so ordinary form fills can precede a click/submit row in one batch—see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) for the agent-visible contract and failure text; typed per-session tab/ref/pinning state lives in `extensions/agent-browser/lib/session-page-state.ts` and is updated from `extensions/agent-browser/index.ts` after each tool result
|
|
200
200
|
- native 0.37 owns first-document setup inheritance for new tabs; the wrapper does not copy headers, user agents or other browser setup itself. The page-summary formatter displays native positive WebMCP availability while retaining raw details; it adds no discovery probe
|
|
201
201
|
- when a direct or batched WebMCP call returns `status: "pending"`, or `result` / `cancel` fails while that target is unknown, keep its tab target unknown and discard same-call snapshot evidence instead of treating an immediate post-dispatch URL probe as stable; `webmcp result` / `cancel`, `get url`, and explicit navigation remain available while unknown; replace the generic blocked snapshot action with `verify-page-target-after-pending-webmcp` (`get url`), and let a completed `batch --bail` use that verification before `snapshot -i` to re-establish both target and refs
|
|
202
|
-
- for top-level non-Electron direct `click` commands with an eligible target, install a bounded in-page target-specific event probe before upstream runs; if upstream reports success but no trusted pointer/mouse/click event reached the resolved target, fail the tool and report `details.clickDispatch` with explicit retry/inspect next actions (the wrapper does not replay clicks in-page). The probe covers `xpath=` targets and current `@e…` / `ref=` refs whose latest stored `refSnapshot.refs` role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`; it requires a unique role/name in the saved snapshot and the live candidates instead of taking a fresh pre-click snapshot that could recycle upstream refs. Ref and XPath candidates must additionally prove identity through a per-probe temporary DOM attribute read back by native `get attr <original-selector> <marker>`. Missing or mismatched identity, including XPath/frame scope mismatches, cleans up the probe and lets native click run without dispatch verification; candidate naming heuristics alone are never proof. Duplicate-name refs pass through without a probe: their old ordinal is not target identity. The probe is intentionally skipped for CSS selector clicks, unresolved `find … click` locators, and `batch`/`qa` click steps
|
|
202
|
+
- for top-level non-Electron direct `click` commands with an eligible target, install a bounded in-page target-specific event probe before upstream runs; if upstream reports success but no trusted pointer/mouse/click event reached the resolved target, fail the tool and report `details.clickDispatch` with explicit retry/inspect next actions (the wrapper does not replay clicks in-page). The probe covers `xpath=` targets and current `@e…` / `ref=` refs whose latest stored `refSnapshot.refs` role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`; it requires a unique role/name in the saved snapshot and the live candidates instead of taking a fresh pre-click snapshot that could recycle upstream refs. Ref and XPath candidates must additionally prove identity through a per-probe temporary DOM attribute read back by native `get attr <original-selector> <marker>`. Missing or mismatched identity, including XPath/frame scope mismatches, cleans up the probe and lets native click run without dispatch verification; candidate naming heuristics alone are never proof. Duplicate-name refs pass through without a probe: their old ordinal is not target identity. The probe is intentionally skipped when the effective native confirmation list guards `evaluate`, avoiding an optional-eval approval loop without bypassing native click policy. Dynamic action-policy files stay native-owned. It is also skipped for CSS selector clicks, unresolved `find … click` locators, and `batch`/`qa` click steps
|
|
203
203
|
- skip unrelated tools and harmless Bash inspections before building prompt policy. Seed the latest raw user message on start/tree, update it on user `message_end`, and re-read its current content before parsing policy so native content replacement cannot leave stale intent. Capture that policy at queued admission without a whole-branch scan on every event. Derive narrow prompt guards only for concrete evidence invariants: explicitly requested screenshot/recording output paths block browser close until the artifact manifest verifies those paths, while bare inbound attachment paths remain inputs. The wrapper intentionally does not infer broad business/user intent from prompt text such as order/payment/post boundaries; agents must follow those instructions themselves. The artifact guard is bounded preflight policy (`details.promptGuard`, `failureCategory: "policy-blocked"`), not a reusable browser recipe layer
|
|
204
204
|
- keep the existing page/selector scroll API; explicit CSS-container movement uses `Element.scrollBy` with `behavior: "instant"` before measuring offsets so smooth-scroll CSS cannot cause a premature no-movement failure
|
|
205
205
|
- reject direct and effective batch `scrollintoview text=...` / `scrollinto text=...` before dispatch because current upstream can falsely report success without movement, while leaving help forms untouched; return only native recovery (`find text ... hover` or fresh snapshot/ref), leaving CSS, XPath, and current-ref behavior upstream-owned
|
|
@@ -224,7 +224,7 @@ Input resolution replaces URL-less `open` with native `get url` before planning,
|
|
|
224
224
|
`agent-browser` startup flags are sticky once a session is already running.
|
|
225
225
|
The extension should surface that clearly and avoid hidden restart behavior.
|
|
226
226
|
|
|
227
|
-
That means explicit startup-scoping flags like `--allowed-domains`, `--auto-connect`, `--args`, `--user-agent`, `--cdp`, `--enable`, `--executable-path`, `--webgpu`, `--no-webmcp`, `--headed`, `--init-script`, `--device`, `--namespace`, `--profile`, `--provider`, `-p`, `--restore`, `--restore-save`, restore check flags, `--session-name`, and `--state` should remain explicit upstream argv choices instead of being wrapped in extra hidden restart or cloning logic. The one deliberate exception is the env-only managed-session `AGENT_BROWSER_RESTORE` key above, which does not inject `--restore` into argv and therefore does not trip launch-scoped `sessionMode: "fresh"` recovery.
|
|
227
|
+
That means explicit startup-scoping flags like `--allowed-domains`, `--auto-connect`, `--args`, `--user-agent`, `--cdp`, `--enable`, `--engine`, `--executable-path`, `--webgpu`, `--no-webmcp`, `--headed`, `--init-script`, `--device`, `--namespace`, `--profile`, `--provider`, `-p`, `--restore`, `--restore-save`, restore check flags, `--session-name`, and `--state` should remain explicit upstream argv choices instead of being wrapped in extra hidden restart or cloning logic. The one deliberate exception is the env-only managed-session `AGENT_BROWSER_RESTORE` key above, which does not inject `--restore` into argv and therefore does not trip launch-scoped `sessionMode: "fresh"` recovery.
|
|
228
228
|
|
|
229
229
|
The wrapper may still apply narrow compatibility normalizations when observed behavior justifies them and the result remains thin, local, and opt-out. For example, OpenAI web properties and `dash.cloudflare.com` reject the default local `HeadlessChrome` user agent while the same flow works with a normal Chrome UA, so the extension injects a domain-specific fallback only when the caller did not already choose raw Chrome arguments, a custom user agent, headed mode, CDP, auto-connect, a provider-backed launch, or a non-Chrome engine through argv or matching upstream environment. Managed sessions retain the injected value as per-session wrapper state across helper calls and branch reload/resume. Active daemons omit both launch forms so upstream does not replace a launch-configured browser; a session proven inactive receives the retained compatibility launch values, including the same fixed compatibility value as a comma-safe Chrome launch argument covering tabs and SSO popups on supported native versions that do not inherit per-page CDP overrides. Wrapper-owned headed launches also default upstream periodic restore autosave off because agent-browser 0.33.2 collects non-current origins through visible temporary targets while holding the daemon state lock; save-on-close remains enabled, and an explicit `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` value opts in when the daemon launches. The effective interval is retained in owned-session state and transcript results; changing it in either direction on a running wrapper-owned headed daemon is rejected until close plus a fresh launch.
|
|
230
230
|
|
|
@@ -237,6 +237,7 @@ Implementation detail lives in `extensions/agent-browser/lib/launch-scoped-flags
|
|
|
237
237
|
- **Command discovery:** Leading argv is scanned with a value-taking allowlist so known global flags and documented command flags consume their values before the upstream command word is identified. Missing-value prevalidation is intentionally limited to upstream global value flags; command-scoped flags and literal text are left to upstream parsing so values like `fill #field --password` are not rejected by wrapper heuristics before the CLI sees them. Upstream 0.35.0 and newer accept `--restore=<key>` but reject other global `--flag=value` tokens during normal command execution, so the wrapper rejects those unsupported assignments before they can be mistaken for command or artifact operands. Plain help/version inspection preserves exact caller argv because upstream accepts those top-level shapes. Nested batch rows do not parse global flags, so their equals forms never receive the top-level inspection or restore exceptions. When upstream adds new global flags that take values ahead of the command, extend both the command-discovery and prevalidation allowlists; when it adds command-specific flags, extend only command discovery/redaction as needed. A smaller set of global boolean flags may be followed by an optional `true`/`false` literal; when present, that literal is consumed as the flag value before command discovery continues.
|
|
238
238
|
- **`--state` disambiguation:** Persisted browser `--state` before the command participates in launch-scoped validation and tab-correction hints. The same flag spelling after a `wait` command is excluded from startup-scoped detection so upstream help examples such as `wait @ref --state hidden` do not spuriously require `sessionMode: "fresh"` while an implicit session is active. As of the current upstream baseline, the parser still does not implement those `wait --state` examples as distinct wait modes, so agent-facing docs recommend `wait --fn` predicates for disappearance checks instead.
|
|
239
239
|
- **`--auto-connect`:** Treated as launch-scoped only when enabled (`--auto-connect` bare or `true`). `--auto-connect false` is ignored for startup-scoped blocking so disabled attach hints do not force a fresh launch.
|
|
240
|
+
- **`--engine`:** Treated as launch-scoped for Chrome and Lightpanda, using the same fresh-session recovery instead of letting native engine changes replace a running managed page. Caller-owned sessions remain native-owned. Chrome stays default; Lightpanda is opt-in with no bundle, second driver, emulation or automatic fallback. See [engine limits](COMMAND_REFERENCE.md#lightpanda-engine).
|
|
240
241
|
- **`--webgpu`:** Treated as launch-scoped for both enabled and explicit `false` values. Enabled WebGPU selects upstream's platform-specific local-launch preset; explicit false can override an environment/config default and still belongs to a fresh browser launch. Upstream rejects enabled WebGPU with CDP, auto-connect, or provider launches.
|
|
241
242
|
- **`--no-webmcp`:** Treated as launch-scoped for bare/`true` and explicit `false` values because it selects whether upstream 0.36.0 enables experimental WebMCP for a locally managed Chrome launch.
|
|
242
243
|
- **`--headed`:** Treated as launch-scoped for both enabled and explicit `false` values so a visible-window choice cannot be silently ignored by an already-running managed session.
|
|
@@ -194,7 +194,7 @@ The 0.27.3 rebaseline is an install-only compatibility update: upstream changed
|
|
|
194
194
|
|
|
195
195
|
Use `agent_browser` for one native command and `batch --bail` with JSON-array stdin for a fixed sequence. Use `agent_browser_code` for loops, branches, and aggregation over the same persistent browser. Return to the model when observation needs judgment. All native command families remain available; there is no public job DSL or isolated script input.
|
|
196
196
|
|
|
197
|
-
`agent_browser` accepts `{ args, stdin?, outputPath?, timeoutMs?, sessionMode? }`. Tokens exclude the binary and shell syntax. Stdin is accepted for native batch, `eval --stdin`, and `auth save --password-stdin`. Omit `--json` for prose; include
|
|
197
|
+
`agent_browser` accepts `{ args, stdin?, outputPath?, timeoutMs?, sessionMode? }`. Tokens exclude the binary and shell syntax. Stdin is accepted for native batch, `eval --stdin`, and `auth save --password-stdin`. Omit `--json` for prose backed by structured upstream JSON; include `--json` or `--json true` for a parseable canonical observation. Explicit `--json false` preserves native text for standalone commands and batch, including leading/trailing whitespace and line breaks in data, complete spills and exports, without trusted row/confirmation/page-target/artifact receipts. `outputPath` success/failure notices append prose without reparsing JSON-looking page text. Repeated flags follow upstream's last-wins semantics; batch-row flags remain native operands and do not select outer output mode. See [the output contract](TOOL_CONTRACT.md#wrapper-json). Help/version remains native text. Empty stdin batch rows (`[]`) pass through unchanged and native execution skips them; `[""]` still fails as an unknown command. `--bail` stops at that failure, while a non-bail batch retains later native results. A nonzero process exit that supplies stderr instead of JSON retains that native error and redaction, with parse diagnostics secondary. Zero-exit malformed JSON remains a strict failure.
|
|
198
198
|
|
|
199
199
|
```json
|
|
200
200
|
{ "args": ["open", "https://example.com"] }
|
|
@@ -340,7 +340,7 @@ Examples:
|
|
|
340
340
|
{ "args": ["snapshot", "-i"] }
|
|
341
341
|
```
|
|
342
342
|
|
|
343
|
-
The optional `agent_browser_action` tool preserves stable accessible click/check/fill and unique visible dropdown selection. Enable `action`, then pass flat locator/selector fields. Role/name or label select resolves one current visible native combobox/listbox;
|
|
343
|
+
The optional `agent_browser_action` tool preserves stable accessible click/check/fill and unique visible dropdown selection. Enable `action`, then pass flat locator/selector fields. Role/name or label select resolves one current visible native combobox/listbox from a fresh snapshot or the eligible next-call confirmed capture; missing or ambiguous select targets still fail. It does not assume native `find ... select`. A `session` field targets a named browser. `compiledSemanticAction` remains the audit metadata name; `effectiveArgs` identifies the executed ref/command. `uncheck` remains native argv. Selector failures retain exact ref/role candidates and session-aware recovery.
|
|
344
344
|
|
|
345
345
|
For desktop, contenteditable, or host-controlled rich inputs, treat a semantic `fill` miss or mismatch differently. Active-session role/name fills can execute through one exact current editable `combobox`, `searchbox`, or `textbox` ref before upstream `find` runs. If a later selector miss still finds an exact current editable ref (`searchbox` or `textbox`), `details.richInputRecovery` and visible `Rich input recovery` describe the candidate and append `focus-current-editable-ref*` / `click-current-editable-ref*` next actions. Those actions deliberately do **not** copy the fill text and never press `Enter` or submit. Direct `fill @ref <text>` on contenteditable refs may also append/prepend instead of replacing; when the latest snapshot proves the target is contenteditable, the wrapper verifies `get text` after a successful fill and appends `details.fillVerification` plus `inspect-after-fill-verification` / `verify-filled-value` if the visible text does not match. Use the safe ladder instead: refresh refs, choose the current editable `@ref`, focus or click it, then use `keyboard type` for framework-controlled editors that require real key events. `keyboard inserttext` is paste-like: it can change a DOM value without updating application state, so use it only when later application-state evidence proves the edit was accepted. Do not auto-submit unless the user flow explicitly calls for it.
|
|
346
346
|
|
|
@@ -348,7 +348,7 @@ Do not assume Playwright selector dialects such as `text=Close` or `button:has-t
|
|
|
348
348
|
|
|
349
349
|
Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside `batch` stdin JSON, the wrapper also walks steps in order before spawn: steps whose first token can navigate or mutate set a latch; a later step whose first token is `snapshot` clears that latch for following rows; guarded steps that still mention `@e…` after an uncleared latch fail with the same `stale-ref` bucket without launching upstream. Same-snapshot form fills and native form-control steps are allowed before a click or submit step, so `fill`, `check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, `select` combobox refs, then a final submit `click` can run from one snapshot. Split dynamic or autosubmit forms with a fresh snapshot if a control interaction rerenders the targets. Follow the `refresh-interactive-refs` next action (it includes `--session <name>` when needed) and prefer stable `find` or `agent_browser_action` locators when navigation or rerendering is likely. Contract detail: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) (`refSnapshot`, `refSnapshotInvalidation`).
|
|
350
350
|
|
|
351
|
-
A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. For both ref and XPath candidates, the wrapper requires native `get attr <original-selector> <marker>` to read back a per-probe temporary DOM marker on that candidate. Missing or mismatched identity (including frame/XPath scope mismatches) cleans up the probe and lets the native click run without dispatch verification. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
|
|
351
|
+
A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. For both ref and XPath candidates, the wrapper requires native `get attr <original-selector> <marker>` to read back a per-probe temporary DOM marker on that candidate. Missing or mismatched identity (including frame/XPath scope mismatches) cleans up the probe and lets the native click run without dispatch verification. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. Probe listeners and the window marker self-expire after the call's process-timeout budget plus the existing two-second cleanup grace if a pending decision blocks normal cleanup; collection cancels the timer. This uses no extra browser command while pending. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
|
|
352
352
|
|
|
353
353
|
Successful `snapshot -i` results can also surface `Possible overlay blockers` when their own refs already show dialog/alertdialog context plus close/dismiss controls, so agents can detect likely obstruction before clicking. When a **top-level** `@e…`/`ref=` click succeeds (not a `click` hidden inside a native batch tool call—the unified command must be `click`), the upstream payload includes `data.clicked`, no `details.clickDispatch` diagnostic fired for the same result, and the wrapper sees `details.navigationSummary.url` unchanged after the same normalization it uses for ref guards (**`#fragment` ignored**), it may run one extra `snapshot -i` and surface `Possible overlay blockers` plus `details.overlayBlockers` (`candidates`, `summary`, and a `snapshot` map that can refresh `refSnapshot`) when that snapshot shows strong modal context (`dialog` / `alertdialog`) **and** up to three close/dismiss-like controls; page-wide words such as privacy, sign in, or banner alone do not trigger it. The URL check compares the session’s prior pinned tab target to `details.navigationSummary.url`. CSS selector clicks do not run this overlay probe. The diagnostic is skipped if the wrapper already applied tab-focus correction or about-blank recovery on that result. Appended `inspect-overlay-state` / `try-overlay-blocker-candidate-*` entries in `details.nextActions` preserve namespace/session context (`--namespace <namespace> --session <name>` when namespaced, otherwise `--session <name>` when the session is named), same as other session-scoped follow-ups. Treat `inspect-overlay-state` as the safe first follow-up; only use a `try-overlay-blocker-candidate-*` next action when the candidate is clearly the control you intend to close. A click that upstream rejects because another element covers the target's click point (`is covered by` … `at its click point`) remains `failureCategory: "upstream-error"`, but receives the same session-aware `inspect-overlay-state` snapshot action. This includes direct `click`, `agent_browser_action` click, raw `find` clicks (including `nth` and omitted default-click actions), and failed native batch rows. That failed-click path does not retry the original click or synthesize a dismiss candidate before a fresh snapshot provides evidence; it is separate from silent input-dispatch failures.
|
|
354
354
|
|
|
@@ -516,14 +516,14 @@ The upstream screenshot aliases are `screenshot --full` for full-page capture an
|
|
|
516
516
|
|
|
517
517
|
Prefer `download <selector> <path>` when the target element itself is the downloadable link/control. Native download always owns the click and download, including loopback links, generated Blob exports, and redirects; the wrapper never substitutes a fetch of the anchor's `href`. Use `click` plus `wait --download [path]` when a previous action starts the download indirectly.
|
|
518
518
|
|
|
519
|
-
Screenshot `imageObservations` includes verified pixel dimensions and capture kind. Matching pre/post CSS viewport, scroll, DPR, and crop samples may establish a measured document-space mapping; otherwise geometry remains unknown, including unmeasured batches and uncertain scrolled/iframe element captures. Native mouse uses current viewport CSS pixels. Account separately for crop origin, current scroll/frame, and any Pi resize before using coordinates; prefer refs. See [image geometry](TOOL_CONTRACT.md#image-observations). Code only attaches images selected with `emitImage` from a current-cell handle.
|
|
519
|
+
Screenshot `imageObservations` includes verified pixel dimensions and capture kind. Positively identified Lightpanda images also carry `rendering: "text"`: their text representation is not graphical layout and never establishes CSS/mouse coordinates. Matching pre/post CSS viewport, scroll, DPR, and crop samples may establish a measured document-space mapping; otherwise geometry remains unknown, including unmeasured batches and uncertain scrolled/iframe element captures. Native mouse uses current viewport CSS pixels. Account separately for crop origin, current scroll/frame, and any Pi resize before using coordinates; prefer refs. See [image geometry](TOOL_CONTRACT.md#image-observations). Code only attaches images selected with `emitImage` from a current-cell handle.
|
|
520
520
|
|
|
521
521
|
For evidence-only screenshots, QA captures, or audit artifacts, save to an explicit path and branch on `details.artifactVerification` plus `details.artifacts` before reporting PASS/FAIL. Inline image attachments are optional convenience when size limits allow; do not require vision review unless the user asked for visual inspection.
|
|
522
522
|
|
|
523
523
|
Wrapper result rendering is metadata-first for saved files. Image MIME types come from a bounded header read for PNG, JPEG, GIF and WebP, never from a filename suffix; missing, unreadable, unknown or truncated headers omit `mediaType`. This identifies a format, not full image validity. Inline screenshots use the same byte check and existing size limit, so a PNG saved as `.webm` still attaches as `image/png`; other artifact kinds are not auto-inlined. An artifact-producing command fails as `artifact-missing` with artifact `status: "stale"` when the reported path's `mtimeMs` falls outside the command's bounded start/end window (with two seconds of filesystem precision tolerance), including a previous recording that `record restart` claims to finalize; clearly old or future-dated evidence is never accepted as a fresh capture. A batch, whether supplied through stdin arrays or argument command strings, must use distinct explicit artifact destinations; preflight canonicalizes existing path ancestry, compares existing file identities to catch hardlinks, and applies full Unicode plus platform case folding on macOS/Windows so aliases cannot satisfy another step's verification. The same preflight prevents `outputPath` from aliasing a same-call browser artifact, follows upstream's forward option consumption and final effective `-o` / `--output` for `diff screenshot`, and treats the optional path on `network har stop` as an artifact destination; upstream ignores positional paths on `network har start`. Outer CLI artifact parsing removes upstream global flags, so direct forms such as `record --json start <path>` and `pdf --quiet <path>` retain their native destinations. Native batch rows do not run that cleanup: `pdf --quick ignored.pdf` writes to the literal path `--quick`, and `download #link --quiet ignored.bin` writes to `--quiet`. Preflight, directory preparation, presentation, and timeout evidence use those same operands, not the ignored trailing tokens. Screenshot destination parsing mirrors upstream's exact flag matching and `[selector] [path]` positional order: `--` is positional, `true` / `false` after screenshot-only `--full` / `-f` remain positional, extra positionals are ignored after the path slot, selector-prefixed (`.`, `#`, `@`) or uppercase-extension single arguments remain selectors, and lowercase image extensions or slash-bearing arguments are paths. The wrapper deliberately keeps its existing slash-bearing hidden-workspace path normalization (for example `.dogfood/run/foo.png`) before launch. `wait --download` is observational and may verify a download that completed just before the wait began, so it is exempt from the command-window mtime gate; an explicit wait destination, in long `--download <path>` or short `-d <path>` form (including `wait --download --timeout 30000 capture.csv`), still participates in active-recording reservation preflight; the path is the next retained operand after the first timeout pair is removed; unsupported `--download=<path>` fails with split-argument guidance:
|
|
524
524
|
- screenshots return a saved-path summary, visible artifact metadata, structured `details.artifacts` metadata, and an inline image attachment when safe; the visible block includes artifact type, requested path, absolute path, existence, size, cwd, session, and repair/copy status when applicable
|
|
525
525
|
- downloads, PDFs, `wait --download` files, `state save` state files, diff screenshot output images, traces, CPU profiles, completed video recordings from `record stop`, and path-bearing HAR captures return concise saved-path summaries plus structured `details.artifacts` metadata without inlining large files
|
|
526
|
-
- `record start <path>` and `record restart <path>` report `successCategory: "artifact-pending"` and that output will be written on `record stop`; dispatched `record start` and URL-bearing `record restart` attempts append one `Page state:` warning on success or failure, describing conservative ref invalidation rather than an observed page change; explicit `--json` puts that warning in `warnings`. Only reached batch rows qualify, not preflight failures, missing binaries, help calls or unconfirmed planned rows — the wrapper invalidates the session’s prior ref snapshot (direct calls and batch steps alike, and even when the start fails with `Recording already active`, to protect older supported natives that can swap the page before that check), so old `@e…` refs fail as `stale-ref` until a fresh `snapshot -i` succeeds; `record restart <path> <url>` navigates the current page and invalidates refs the same way, while a restart without a URL, including FPS-only options, keeps the current page and refs; `details.artifacts` / `details.artifactVerification` mark that future file as `pending` with `recordingState: "openRecording"` and `willExistOnStop: true`, and `details.nextActions` includes exact `stop-pending-recording` args. When `record restart` returns a native `previousRecording`, that receipt's outcome and capture window control the previous artifact's verification;
|
|
526
|
+
- `record start <path>` and `record restart <path>` report `successCategory: "artifact-pending"` and that output will be written on `record stop`; dispatched `record start` and URL-bearing `record restart` attempts append one `Page state:` warning on success or failure, describing conservative ref invalidation rather than an observed page change; explicit `--json` puts that warning in `warnings`. Only reached batch rows qualify, not preflight failures, missing binaries, help calls or unconfirmed planned rows — the wrapper invalidates the session’s prior ref snapshot (direct calls and batch steps alike, and even when the start fails with `Recording already active`, to protect older supported natives that can swap the page before that check), so old `@e…` refs fail as `stale-ref` until a fresh `snapshot -i` succeeds; `record restart <path> <url>` navigates the current page and invalidates refs the same way, while a restart without a URL, including FPS-only options, keeps the current page and refs; `details.artifacts` / `details.artifactVerification` mark that future file as `pending` with `recordingState: "openRecording"` and `willExistOnStop: true`, and `details.nextActions` includes exact `stop-pending-recording` args. When `record restart` returns a native `previousRecording`, that receipt's outcome and capture window control the previous artifact's verification; without that receipt, only an acknowledged native `restarted: true` retires the previous take, whose file remains unverified, not saved; rejected or pending restarts keep the active take's reservation and stop action; a missing or stale prior file fails as `artifact-missing` while the new recording remains visible as pending and the prior manifest row is retired. Within one Pi extension process, an unbounded transcript-backed index reserves active recording destinations independently of the bounded artifact manifest. Artifact lifecycle calls and result `outputPath` writes serialize around that global check; reservations use canonical namespace/session identity, survive manifest eviction and branch replay, and retire after direct, ordered nested-batch, fresh-replacement, code, Electron, or shutdown close; the newest pending row per identity is authoritative. Legacy batch replay retires a pending manifest only when the ordered close lifecycle leaves recording closed; a later successful browser reactivation plus `record start` keeps the new pending reservation. Lexical, hardlink, existing/dangling symlink, full Unicode-fold, and macOS/Windows case aliases are rejected, so `record restart` must use a distinct new path. Do not place `record start` or `record restart` after `close` / `quit` / `exit` in one batch: wrapper preflight rejects it because upstream can report success without starting a recording; split the close and recording into separate calls. A `No recording in progress` stop failure checks the matching native receipt once and preserves checked file metadata instead of assuming the path is missing; a later successful batch recording row opens its new pending path normally. Recovery offers an exact status query, and a stop only when the matching take is still current and pending. The target remains unverified until recording stops. Native 0.37 checks `ffmpeg` before starting; older supported natives may defer failure. If a successful start/restart reports pending output without `ffmpeg`, the wrapper appends `Recording dependency warning: ffmpeg not found on PATH` and `details.recordingDependencyWarning`; stop, check the result, then install the dependency before starting a new recording.
|
|
527
527
|
- `batch` keeps each step's artifacts in `details.batchSteps[].artifacts`; top-level `details.artifacts` and `details.artifactManifest` coalesce an earlier pending recording into the later saved, missing, or stale terminal result for the same namespace/session identity; a successful later close retires an unfinalized recording as `close-abandoned`, uses `missing` only after a filesystem check proves absence (otherwise unverified), removes its stop action, and resets earlier ref/page/network-route batch state; a later successful `record stop` replaces that intermediate abandoned row with its verified saved artifact, and later rows—including failed rows—whose lifecycle reports a browser launch may rebuild state without triggering stale pre-close `about:blank` recovery; failed-step `batchSteps[]` retains only the bounded `lifecycle.effectiveLaunch.browserLaunched` boolean for replay, explicitly non-launching diagnostics leave the close terminal, missing lifecycle evidence remains conservatively active even on the first managed call, every successful close clears wrapper trace/profiler ownership before ordered later successful rows can rebuild it, namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, and any later same-session failure before recording stops keeps exact `stop-pending-recording` args alongside its normal recovery
|
|
528
528
|
|
|
529
529
|
`diff screenshot` follows the file-artifact path above for the **diff** image: model-visible text and `details.artifacts` focus on that output, while baseline paths stay out of the artifact summary block, and Pi does **not** auto-inline the diff the way it inlines trusted `screenshot` captures. `state load` may print the loaded path in prose but does not add a saved-file artifact entry the way `state save` does.
|
|
@@ -589,7 +589,9 @@ Use `tab list` and `tab <tab-id-or-label>` when a profile restore, pop-up, or cl
|
|
|
589
589
|
|
|
590
590
|
### Recover from guarded-action confirmations
|
|
591
591
|
|
|
592
|
-
|
|
592
|
+
Configure `--confirm-actions` on the session's first command. The extension retains that setting for the same canonical namespace/session across helpers, bare follow-ups, code and advanced calls, and transcript resume. Follow the exact returned confirm/deny action without repeating the policy. First launch and new explicit `--confirm-actions` / `--config` selections use native argv → environment → config precedence. Bare follow-ups retain an established string, including explicit `""`, over ambient environment/discovered-config edits; those defaults apply again after close or for a new identity. Native unset is not pinned and continues ordinary native default resolution. Explicit policy changes still use native restart behavior and can discard pending confirmations. Successful close, namespace-wide close, and successfully closed managed replacements retire the setting with that session. Replacement cleanup uses the old identity's setting; failed or pending close retains its continuity state. Electron launch and host probes use the selected setting too. A definite wrapper/native CLI parse/config/local-inspection rejection restores the previous setting unless an earlier browser helper could have established the request; interrupted work retains selected intent.
|
|
593
|
+
|
|
594
|
+
When upstream requires confirmation, the native tool result prints the pending confirmation id and both recovery calls. Confirming one action in a compound command can return another native confirmation; follow its returned action too. Pending navigation is intent, never an observed page target, including pending native batch rows. A completed compound confirmation reconciles its native resulting target before ordinary follow-ups. A tab-creation transport can add `after_confirmation: ["open", <requested URL>]` plus nonempty `guidance` when the URL matches the original `tab new` request, including a preceding `--label`. Follow that visible continuation only after approval successfully creates the blank/labeled tab, in the same session and new tab—never after denial. The extension preserves the exact decision and conditional guidance but does not execute the continuation. Confirmed full snapshots install returned refs in direct and batch row order. A final confirmed capture supplies the next same-identity call's freshness sample once, including role click/check/fill and unique accessible select resolution; an intervening call or later row consumes the opportunity. A pending snapshot is displayed as confirmation required, never as an empty capture. Use refs from that capture: membership, target and invalidation checks still apply. On this one use, live verification compares the complete capture URL, including fragments, and rejects refs when it changes—including wrapper-owned managed sessions and warm sessions without resume. Ordinary ref/tab URL comparisons remain fragment-insensitive. DOM-only renames on the same page after the sample are not resampled on this one use. Further ref actions use normal freshness checks. Optional click-dispatch eval probes are skipped when the effective native confirmation list guards `evaluate`; native click confirmation still applies. Use the same `agent_browser` tool; do not switch to bash.
|
|
593
595
|
|
|
594
596
|
```json
|
|
595
597
|
{ "args": ["--confirm-actions", "click", "click", "@danger"] }
|
|
@@ -602,7 +604,7 @@ If the result says `Pending confirmation id: c_8f3a1234`, choose one follow-up:
|
|
|
602
604
|
{ "args": ["deny", "c_8f3a1234"] }
|
|
603
605
|
```
|
|
604
606
|
|
|
605
|
-
|
|
607
|
+
Use the exact returned actions for a genuine native control: they preserve its ID and canonical namespace/session across resume/branch replay. Matching guarded decisions skip page/pinning helpers that could overwrite the native pending slot, then reconcile completed browser actions normally; this is not proof of native ID validation or browser independence. Only a matching first effective guarded decision gets that suppression in native batch; native raw-argument-over-stdin selection and later page/ref checks remain unchanged. Page content and unrelated/nonmatching decisions gain no exemption. Explicit URL reads remain separate: only `capabilities.readRequiresConfirmation: true` grants browser independence with native ID checking; legacy URL reads keep normal checks. Explicit caller identities and the code cell's fixed identity still win. A pending cold-reopen helper exposes its exact action without running the requested getter or replaying open. A failed inner confirmed action fails the tool; a completed confirmed close retires session state.
|
|
606
608
|
|
|
607
609
|
Confirmation context may be redacted when it contains credentials, tokens, cookies, or auth-bearing URLs. Replacements are marked `[REDACTED]` (URL-encoded in parsed URLs); ordinary technical phrases such as `bearer token` and `bearer authentication` stay intact outside credential fields and headers. URL scrubbing covers `code`, SAMLRequest, SAMLResponse, RelayState, `authorization_session_id`, and auth-context `state` / `nonce` while retaining ordinary non-auth query values and the spelling of URLs needing no redaction. Visible text, structured details, persisted spills, and `outputPath` exports use the same redaction; exact internal page-target URLs remain available to browser state logic. Use the id exactly as printed.
|
|
608
610
|
|
|
@@ -761,6 +763,8 @@ These calls return plain text and stay stateless: the extension does not inject
|
|
|
761
763
|
|
|
762
764
|
`cookies set --curl <file>` selects native file-import mode even when `--curl` follows other set operands. Ordinary import paths remain visible in invocation echoes and failed batch diagnostics; inline cookie values, sensitive flags, and secret-bearing path text still redact. File contents are not echoed by this exception.
|
|
763
765
|
|
|
766
|
+
With explicit `--json false`, cookie `name=value` (including empty names) and storage `key: value` (including empty and colon-containing keys) output use command-aware redaction before presentation and export; benign preferences remain visible. Standalone `storage local|session get <key>` and shorthand `storage local|session <key>` redact the whole sensitive value, including embedded newlines, even when invocation display redaction changes the supplied key. All-entry/raw-batch text has unescaped multiline values and no row provenance, so use structured JSON for unambiguous sensitive storage. See [the output contract](TOOL_CONTRACT.md#wrapper-json).
|
|
767
|
+
|
|
764
768
|
Privacy note: `cookies get` can expose real profile cookies. Do not run it against `--profile Default` or other authenticated profiles unless the user explicitly needs cookie inspection; prefer task-specific page actions and storage checks.
|
|
765
769
|
|
|
766
770
|
### WebMCP page tools
|
|
@@ -989,7 +993,7 @@ Example config:
|
|
|
989
993
|
}
|
|
990
994
|
```
|
|
991
995
|
|
|
992
|
-
Browser defaults are scoped: global/override `defaultProfile` with `policy: "always"` and a Chrome profile name (not a path), plus `executablePath`, become native subprocess defaults only for automatic root browsers and their helpers. Roots have distinct named sessions and restore keys; parents and descendants share one root through `PI_SUBAGENT_ROOT_SESSION_ID`. These defaults never apply to unrelated explicit sessions or Electron launches. Code selecting the ordinary root browser uses the same defaults. Profile paths, other policies, and project-only guidance remain advisory. Native restore keeps cookies/localStorage/sessionStorage, not new IndexedDB, service workers, or memory-only grants; verify app access after a real restart. Avoid global native profile changes or a shared writable profile path; see [shared browser defaults](../README.md#shared-browser-defaults). Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `profiles` and `doctor` when profile resolution fails, then use the reported Chrome profile directory name, a full profile/user-data directory path if upstream accepts one, or the configured `browser.executablePath` with top-level `sessionMode: "fresh"`.
|
|
996
|
+
Browser defaults are scoped: global/override `defaultProfile` with `policy: "always"` and a Chrome profile name (not a path), plus `executablePath`, become native subprocess defaults only for automatic root browsers and their helpers when the effective engine is Chrome. Lightpanda omits these wrapper defaults, without changing caller-selected launch settings. Roots have distinct named sessions and restore keys; parents and descendants share one root through `PI_SUBAGENT_ROOT_SESSION_ID`. These defaults never apply to unrelated explicit sessions or Electron launches. Code selecting the ordinary root browser uses the same defaults. Profile paths, other policies, and project-only guidance remain advisory. Native restore keeps cookies/localStorage/sessionStorage, not new IndexedDB, service workers, or memory-only grants; verify app access after a real restart. Avoid global native profile changes or a shared writable profile path; see [shared browser defaults](../README.md#shared-browser-defaults). Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `profiles` and `doctor` when profile resolution fails, then use the reported Chrome profile directory name, a full profile/user-data directory path if upstream accepts one, or the configured `browser.executablePath` with top-level `sessionMode: "fresh"`.
|
|
993
997
|
|
|
994
998
|
## Important global flags, config, and environment
|
|
995
999
|
|
|
@@ -1032,13 +1036,33 @@ The extension composes stock `--args --no-startup-window` for local Chrome start
|
|
|
1032
1036
|
- `--cdp <port|url>`: connect through Chrome DevTools Protocol. Use it, `--auto-connect`, or `connect <port|url>` once on a named/fresh session, verify with `get url`, then reuse that session without repeating the attach flag. After a successful attachment the wrapper avoids re-emitting its own compatibility launch argument; caller launch/config/file-access settings remain unchanged. Content-bearing first use is blocked until URL verification; later page reads/interactions live-check the URL because the attached browser can drift externally. `close` clears attachment state.
|
|
1033
1037
|
- `--color-scheme <scheme>`: `dark`, `light`, or `no-preference`. Environment: `AGENT_BROWSER_COLOR_SCHEME`.
|
|
1034
1038
|
- `--download-path <path>`: default browser download directory. Environment: `AGENT_BROWSER_DOWNLOAD_PATH`.
|
|
1035
|
-
- `--engine <name>`: browser engine, `chrome` by default or `lightpanda`. Environment: `AGENT_BROWSER_ENGINE`.
|
|
1039
|
+
- `--engine <name>`: browser engine, `chrome` by default or `lightpanda`. Environment: `AGENT_BROWSER_ENGINE`. Launch-scoped; use `sessionMode: "fresh"` when changing a managed browser. See [Lightpanda](#lightpanda-engine).
|
|
1036
1040
|
|
|
1037
1041
|
On Android/Termux, follow the README setup to install the packaged Linux-musl arm64 upstream binary, install Termux's `which`, and expose its launcher as `$PREFIX/bin/chromium`. Prefer that upstream system-browser discovery over ambient `AGENT_BROWSER_EXECUTABLE_PATH`: it survives isolated `HOME` values and works for ordinary native calls. Code retains the selected browser's normal launch configuration. Wrapper-generated Android managed identities use a compact 80-bit digest so ordinary namespaces and fresh rotations fit upstream's Unix socket path.
|
|
1038
1042
|
|
|
1039
|
-
- `--no-auto-dialog`: disable automatic dismissal of alert/beforeunload dialogs. Environment: `AGENT_BROWSER_NO_AUTO_DIALOG`.
|
|
1043
|
+
- `--no-auto-dialog`: disable automatic dismissal of alert/beforeunload dialogs. Environment: `AGENT_BROWSER_NO_AUTO_DIALOG`. Explicit `true`/`false` reaches all helpers in that call; keep the same selection on follow-ups to avoid native daemon replacement.
|
|
1040
1044
|
- `--idle-timeout <ms>`: native background browser lifecycle setting (also accepts `10s`, `3m`, `1h`). Caller-owned sessions retain native idle policy; an explicit flag is carried to every helper in that call. Keep it consistent between calls to avoid a native daemon restart. Only wrapper-owned sessions receive the implicit timeout and numeric mismatch check against `PI_AGENT_BROWSER_IMPLICIT_SESSION_IDLE_TIMEOUT_MS`.
|
|
1041
1045
|
|
|
1046
|
+
### Lightpanda engine
|
|
1047
|
+
|
|
1048
|
+
[Lightpanda 1.0](https://lightpanda.io/blog/posts/lightpanda-1-0) is an optional headless engine, installed separately from this package and `agent-browser`. The [native engine guide](https://github.com/vercel-labs/agent-browser/blob/v0.38.2/docs/src/app/engines/lightpanda/page.mdx) describes setup. Put its `lightpanda` executable on `PATH`; select it with `--engine lightpanda`, `AGENT_BROWSER_ENGINE=lightpanda`, or native config `engine`. Chrome remains the default.
|
|
1049
|
+
|
|
1050
|
+
```json
|
|
1051
|
+
{ "args": ["--engine", "lightpanda", "open", "https://example.com"], "sessionMode": "fresh" }
|
|
1052
|
+
{ "args": ["snapshot", "-i"] }
|
|
1053
|
+
{ "args": ["screenshot", "/tmp/lightpanda-text.png"] }
|
|
1054
|
+
{ "args": ["close"] }
|
|
1055
|
+
```
|
|
1056
|
+
|
|
1057
|
+
- Engine selection belongs on launch, not routine follow-ups. Changing an active wrapper-managed browser returns the existing fresh-session recovery; caller-owned sessions retain native lifecycle semantics.
|
|
1058
|
+
- Package `browser.defaultProfile` / `browser.executablePath` Chrome bootstrap defaults are omitted when CLI → environment → native config selects Lightpanda. Caller-selected profile, executable, config and environment values remain untouched; upstream validates combinations.
|
|
1059
|
+
- `open`, snapshots/refs, form interaction, getters, `eval --stdin`, native batch and browser code can work, but successful dispatch is not Chrome-parity visibility or layout evidence. By default Lightpanda skips external stylesheets, images, iframes and workers. Snapshot/hidden-element and overlay/scroll assumptions differ from Chrome.
|
|
1060
|
+
- Screenshots are PNGs rendering page markdown/text. Native engine metadata identifies them for direct, batch and selected code output; standalone geometry samples can also recognize the browser's Lightpanda brand on attachments or older native results. Identified images carry `rendering: "text"` with unknown geometry and no crop/scale mapping. They remain verified image attachments, not graphical captures or a basis for mouse coordinates. Missing provenance stays unknown.
|
|
1061
|
+
- Lightpanda currently permits one target per browser context; `tab new` can fail. Use separate native sessions for independent pages, or Chrome when the task needs multiple tabs. Do not infer broad authentication/restore parity from a single-origin storage result.
|
|
1062
|
+
- Native Lightpanda launch rejects Chrome profiles/extensions, headed mode, launch `--state`, WebGPU, file-access switches, Chromium `--args`, and private CA trust. Native WebMCP currently rejects this engine even though Lightpanda advertises its own WebMCP support. There is no wrapper workaround or automatic Chrome fallback; choose a fresh Chrome launch explicitly when those capabilities are needed.
|
|
1063
|
+
- Lightpanda 1.0 enforces CORS by default, so cross-origin page `fetch`/XHR can fail where earlier Lightpanda did not. Preserve that protection; navigation is a separate capability.
|
|
1064
|
+
- The 1.0 release provides macOS/Linux x86_64/ARM64 binaries (Linux glibc), not native Windows/Android. Remote CDP or WSL is a separate setup, not native-platform qualification. Telemetry is upstream-owned; `LIGHTPANDA_DISABLE_TELEMETRY=true` opts out.
|
|
1065
|
+
|
|
1042
1066
|
### Output, provider, policy, and AI flags
|
|
1043
1067
|
|
|
1044
1068
|
- `--json`: JSON output. The wrapper injects this automatically for normal tool execution. Environment: `AGENT_BROWSER_JSON`.
|
|
@@ -1049,7 +1073,7 @@ On Android/Termux, follow the README setup to install the packaged Linux-musl ar
|
|
|
1049
1073
|
- `--content-boundaries`: wrap page output in boundary markers. Environment: `AGENT_BROWSER_CONTENT_BOUNDARIES`.
|
|
1050
1074
|
- `--max-output <chars>`: truncate page output to N characters. Environment: `AGENT_BROWSER_MAX_OUTPUT`.
|
|
1051
1075
|
- `--allowed-domains <list>`: restrict browser and `read` traffic to exact or `*.` wildcard domain patterns. Environment: `AGENT_BROWSER_ALLOWED_DOMAINS`. Use a fresh local Chrome context; upstream 0.32.0 owns containment and incompatible-mode rejection and disables Chromium `RTCPeerConnection` while active. The wrapper passes the setting and result through unchanged.
|
|
1052
|
-
- `--action-policy <path>`: action policy JSON file. Environment: `AGENT_BROWSER_ACTION_POLICY`.
|
|
1076
|
+
- `--action-policy <path>`: action policy JSON file. Environment: `AGENT_BROWSER_ACTION_POLICY`. Explicit policy values also reach internal helpers for the current call. Keep the action-policy path consistent on follow-ups; changing or omitting it can restart the native daemon and discard pending confirmations. The established `--confirm-actions` setting is retained for same-session follow-ups.
|
|
1053
1077
|
- `--confirm-actions <list>`: action categories requiring confirmation. Environment: `AGENT_BROWSER_CONFIRM_ACTIONS`.
|
|
1054
1078
|
- `--confirm-interactive`: interactive confirmations; auto-denies when stdin is not a TTY. Environment: `AGENT_BROWSER_CONFIRM_INTERACTIVE`.
|
|
1055
1079
|
- `-p, --provider <name>`: provider such as `ios`, `browserbase`, `kernel`, `browseruse`, `browserless`, or `agentcore`. Environment: `AGENT_BROWSER_PROVIDER`.
|
|
@@ -1058,7 +1082,7 @@ On Android/Termux, follow the README setup to install the packaged Linux-musl ar
|
|
|
1058
1082
|
- `--model <name>`: AI model for `chat`. Environment: `AI_GATEWAY_MODEL`.
|
|
1059
1083
|
- `-v, --verbose`: show tool commands and raw output.
|
|
1060
1084
|
- `-q, --quiet`: show only AI text responses.
|
|
1061
|
-
- `--debug`: debug output. Environment: `AGENT_BROWSER_DEBUG`.
|
|
1085
|
+
- `--debug`: debug output. Environment: `AGENT_BROWSER_DEBUG`. Explicit `true`/`false` reaches all helpers in that call, including overriding true config defaults; `false` removes the debug variable so native presence checks cannot enable logging. Inherited env/config behavior is unchanged.
|
|
1062
1086
|
- `AGENT_BROWSER_PLUGINS`: JSON plugin registry override for the upstream `plugin` commands.
|
|
1063
1087
|
- `--version`, `-V`: show version.
|
|
1064
1088
|
|
|
@@ -1080,7 +1104,7 @@ Other useful environment variables include `AGENT_BROWSER_DEFAULT_TIMEOUT`, `AGE
|
|
|
1080
1104
|
## Wrapper-specific behavior worth knowing
|
|
1081
1105
|
|
|
1082
1106
|
- The extension may keep following one implicit managed session across later tool calls.
|
|
1083
|
-
- If launch-scoped flags like `--profile`, `--args`, `--user-agent`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--no-webmcp`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--enable`, `--provider` / `-p`, or provider device flags like `--device` would replace or be ignored by an already-active managed session, retry with `sessionMode: "fresh"`. When the call explicitly names the current managed session, the structured recovery payload removes that `--session` so the fresh rotation can succeed.
|
|
1107
|
+
- If launch-scoped flags like `--engine`, `--profile`, `--args`, `--user-agent`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--no-webmcp`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--enable`, `--provider` / `-p`, or provider device flags like `--device` would replace or be ignored by an already-active managed session, retry with `sessionMode: "fresh"`. When the call explicitly names the current managed session, the structured recovery payload removes that `--session` so the fresh rotation can succeed.
|
|
1084
1108
|
- If a `sessionMode: "fresh"` call fails (including upstream failure, timeout, missing binary, or **`qa`** reclassification after a nominally successful batch), read `details.managedSessionOutcome` before assuming where the next default call will go: `preserved` means the prior managed session remains current, while `abandoned` means no managed session became current. When the failure reason is not the fresh launch itself—for example `failureCategory: "qa-failure"`—`status`/`summary` may still describe the managed-session transition while `succeeded` on this object matches the final tool outcome.
|
|
1085
1109
|
<!-- agent-browser-playbook:start wrapper-tab-recovery -->
|
|
1086
1110
|
<!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
|
package/docs/RELEASE.md
CHANGED
|
@@ -21,7 +21,7 @@ Provide one concrete maintainer workflow for validating repo state, package cont
|
|
|
21
21
|
|
|
22
22
|
The owner's 2026-09-21 **“waive windows”** instruction makes Windows qualification nonblocking for this rollout, even with available runners, and supersedes older mandatory-Windows/no-waiver wording below and in platform guidance. Preserve failed diagnostics; do not claim a Windows full-suite pass or continue Windows repair experiments. Known policy-lock/restored Electron failures and bounded repeat results are tracked in [#191](https://github.com/fitchmultz/pi-agent-browser-native/issues/191) and the [current support matrix](SUPPORT_MATRIX.md#current-0616-rollout).
|
|
23
23
|
|
|
24
|
-
Linux/macOS, declared Node floors, official/fork host and consumer-artifact checks, and Native lifecycle Linux remain required.
|
|
24
|
+
Linux/macOS, declared Node floors, official/fork host and consumer-artifact checks, and Native lifecycle Linux remain required. This waiver changes neither runtime nor the composed `release`/`prepublishOnly` scripts: record individually satisfied gates and the explicit Windows waiver, never label the full composition passed.
|
|
25
25
|
|
|
26
26
|
Docs/workflow-only changes may reuse prior live/platform evidence when the complete compiled runtime, runtime dependencies and relevant tests are unchanged. Changed published files require a new retained tarball checked by `verify-package.mjs --smoke-pi`, including runtime-only SDK and actual CLI loading. Publish those exact checked bytes without repacking.
|
|
27
27
|
|
|
@@ -276,7 +276,7 @@ These show up often in cloud dev boxes and scripted smokes; they are maintainer
|
|
|
276
276
|
| **Pi CLI vs repo devDependencies** | Global `pi` older than the minimum Pi runtime floor for the release can change TUI behavior, `/reload`, package installs, and tool routing during lifecycle or checkout smokes. | Run `npm run doctor` and align `pi` with the current audited baseline before release gates (`pi update` or install the matching version). The published peer range stays wildcard per Pi package docs, and the doctor enforces the minimum Pi runtime floor before package validation. |
|
|
277
277
|
| **npm lockfile (`packageManager`)** | `package.json` pins **npm@12.2.0** for the Node 24.21.0 runtime baseline. Older npm versions can rewrite optional-platform metadata in `package-lock.json`; private registry proxies can also leak organization URLs into `resolved` entries. | Prefer `npx -y npm@12.2.0 install` when refreshing the lockfile; do not commit unrelated lockfile churn or WorkOS/private-registry URLs. |
|
|
278
278
|
| **`pi -p` / print mode** | Non-interactive `pi -p` may hang or emit no stdout for long real-browser smokes without a TTY. | Use **tmux**-driven interactive `pi` for release evidence and checkout smokes; reserve `-p` for short, non-browser checks. |
|
|
279
|
-
| **Real-browser cleanup** | `real-upstream`, Sauce Demo, and live-site runs can leave defunct Chrome/`agent-browser` children if a session aborts mid-flow. | Close via `agent_browser` / `agent-browser` `close`,
|
|
279
|
+
| **Real-browser cleanup** | `real-upstream`, Sauce Demo, and live-site runs can leave defunct Chrome/`agent-browser` children if a session aborts mid-flow. | Close only test-owned browsers via `agent_browser` / `agent-browser` `close`, stop this run's tmux sessions, and remove its disposable screenshots/HARs. Preserve unrelated resources and retained evidence. |
|
|
280
280
|
| **Automated prompt driving** | Grepping tmux pane text for words that also appear in the **user** prompt (`PASS`, `FAIL`, `checkout overview`, `Smoke result:`) can false-complete before the agent finishes. | Wait for pane idle (no `Working…`), `agent_browser close` / `Artifact lifecycle`, or JSONL tool results—not instruction phrases copied from the prompt. |
|
|
281
281
|
| **Lifecycle verify flags** | `npm run verify -- lifecycle --model` or `--timeout-ms` without the next argv token fails fast with a usage error—the `project.mjs` facade validates passthrough the same way as `scripts/verify-lifecycle.mjs`. | Always pair flags with values (`--model openai-codex/gpt-5.5:minimal`, `--timeout-ms 600000`) or omit `--model` / `--timeout-ms` to keep the harness defaults (`zai/glm-5.2`, **180000 ms** per-step waits). |
|
|
282
282
|
|
|
@@ -306,6 +306,7 @@ This suite requires the installed stable `agent-browser --version` to meet the m
|
|
|
306
306
|
- **Recording FPS paths and pinning:** the opt-in `test/agent-browser.batch-fidelity.test.ts` native case checks direct/raw/stdin preflight before dispatch, actual start/restart destinations and pinned targets with leading `--fps`, conservative start-ref protection, FPS-only restart continuity, and valid 12-fps WebM output after explicitly logged 12-second fixture captures. Run it with `PI_AGENT_BROWSER_REAL_UPSTREAM=1`; it requires native 0.37 or newer. Keep focused older-native controls for fresh-page starts and the start-then-ref batch latch when changing that policy.
|
|
307
307
|
- **Managed session core and safe diagnostic matrix:** fresh `open` on the contract fixture, then implicit reuse across `eval --stdin`, `snapshot -i`, interaction commands (`click`, `dblclick`, `fill`, `type`, `type --clear --delay`, `focus`, `keyboard` with `type` / `inserttext`, `press`, `hover`, `check`, `uncheck`, `select`, failed `select` no-match, `upload`, `drag`, `mouse`, `scroll`, off-viewport click, `scrollintoview`, `wait` on selectors in the main frame and a selected iframe), extraction (`get` variants, `is` variants, `find label … fill` via native `<label>`, `aria-label`, and `aria-labelledby`, inline `eval`), file outputs (`screenshot`, `pdf`), navigation (`back`, `forward`, `reload`, `tab list`, another `open` to the same fixture), `batch` stdin, `pushstate`, `vitals … --json`, network route/requests/HAR, diff snapshot/screenshot/url, trace/profiler, console/errors/highlight, stream enable/status/disable, and `cookies set --curl`.
|
|
308
308
|
- **Managed restore correctness and persistence:** while the restore-enabled managed daemon is active, assert raw argument and stdin batches containing nested `connect` fail before upstream spawn; a new harness replaying the owned launch result must also reject incompatible reuse of that live same-name daemon. Seed a cookie plus localStorage/sessionStorage, close the first managed browser while a conflicting parent namespace is set, verify the default-namespace daemon actually closed, create a new extension harness with the same cwd, reopen the fixture with `sessionMode: "fresh"`, and assert all three values restore before closing the second browser. On POSIX, separate isolated real-browser launches assert automatic restore stays disabled and no snapshot is written through either a symlinked `sessions` directory or a file symlink in `sessions/.tmp`; a relative `HOME`, untrusted writable HOME ancestry, and a non-Git cwd must fail closed. Verify a checkout rename preserves its generation identity but starts a fresh composite restore key (fail-closed, because the cwd-derived managed-session base name changes), a copied or path-replacement checkout gets a new key, and changing the Git-generation marker between planning and spawn prevents agent-browser from starting. Run two same-identity harnesses concurrently so a compatible launch publishes its daemon policy before a waiting incompatible launch re-inspects and fails without reaching its main spawn; also fail a fresh non-batch command after daemon creation and verify shutdown closes the retained identity.
|
|
309
|
+
- **Confirmation continuity:** the focused `contract suite matches confirmation launch policy surviving native nextActions` case follows exact confirm/deny actions without policy argv, checks a useful compound audit across resume followed immediately by title/body reads without a repairing URL getter, retains the native daemon PID on bare getters, and preserves an active recording and destination sentinel after denying restart. The nested recording-restart/navigation subcase requires the pinned native recording-policy patch; stock 0.38.1 skips only that subcase. With that patched binary first on `PATH`, run `PI_AGENT_BROWSER_REAL_UPSTREAM=1 PI_AGENT_BROWSER_NATIVE_COMPOUND_POLICY=1 node --import tsx --test --test-force-exit --test-name-pattern 'confirmation launch policy' test/agent-browser.real-upstream-contract.test.ts`. Both variables are test selectors, not runtime configuration. Focused read-confirmation, cold-resume, session-page-state and managed-daemon-policy owners additionally cover exact generic/native-read provenance, forged/foreign controls, completed click/tab and failed-inner action reconciliation, confirmed-close replay, definite unestablished policy requests and pending cold helper exposure. Qualify actual SDK resume plus immediate title/body follow-ups without a repairing getter; helpers must not replace the original native pending slot.
|
|
309
310
|
- **Cold first-read continuity:** the focused `contract suite matches cold URL reopen after quit` case starts an explicit `sessionMode: "fresh"` managed browser, runs actual quit cleanup, waits for the exact old daemon to exit, reloads the saved branch, and requests `snapshot -i` before any explicit navigation. Empty storage and origin storage must both reach their remembered non-root URL, report fresh refs in the main frame, and retain storage without pretending to retain unsaved forms or JavaScript memory. Separate deterministic cold-resume/boundary tests cover first `get url` and history commands, non-page daemon-starting calls and batch prefixes, hash routes, pending-state transcript replay, live missing tabs versus explicit URL destinations, close retirement, and caller-owned/attached/restore-disabled boundaries. Those fake-upstream checks do not replace native-browser or real Pi lifecycle qualification of the complete candidate.
|
|
310
311
|
- **Failure shape:** `react tree` on a page opened with `--enable react-devtools` but without a React app (expects a clear missing-renderer error with session-bound `details`).
|
|
311
312
|
- **Async download:** `open` on the `/download` fixture, anchor-triggered export, then `wait --download <path>` metadata and wrapper artifact reporting for the requested path.
|
|
@@ -330,11 +331,11 @@ node --test test/agent-browser.native-lifecycle.test.mjs
|
|
|
330
331
|
|
|
331
332
|
#### Routine GitHub PR checks
|
|
332
333
|
|
|
333
|
-
[`.github/workflows/pi-compatibility.yml`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/pi-compatibility.yml) runs on PRs to `main
|
|
334
|
+
[`.github/workflows/pi-compatibility.yml`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/pi-compatibility.yml) runs on PRs to `main`, manual dispatch, and release workflow calls. It resolves the latest stable official Pi version and latest maintained fork `main` commit once, or accepts those exact identities from the enclosing release run. Independent required Ubuntu jobs qualify each selected host's complete installed dependency graph on Node 24.21.0. Each runs `check:compat` once: host identity, typecheck, the full offline suite, and packed SDK/CLI loading. A parallel macOS Node 24.21.0 job typechecks and checks the packed Pi load against the same latest official graph without repeating the full suite. The host selector uses the immutable, reviewed `fitchmultz/.github` automation scripts; that automation pin is not a Pi qualification target. Locked development snapshots and the support floor are not validation lanes, and neither host is skipped based on a floor comparison.
|
|
334
335
|
|
|
335
|
-
[`.github/workflows/native-lifecycle.yml`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/native-lifecycle.yml) runs the separate **Native lifecycle Linux** gate through [`scripts/ci-native-lifecycle.sh`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/scripts/ci-native-lifecycle.sh). It independently checks out the PR head, builds the compiled extension on Node **24.21.0**, and runs portable native SDK lifecycle checks against isolated loopback browser fixtures. It replaces the fork-only checkpoint gate and requires
|
|
336
|
+
[`.github/workflows/native-lifecycle.yml`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/native-lifecycle.yml) runs the separate **Native lifecycle Linux** gate through [`scripts/ci-native-lifecycle.sh`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/scripts/ci-native-lifecycle.sh). It independently checks out the PR head, selects the latest official graph using its once-resolved version (or the exact version supplied by the enclosing release run), builds the compiled extension on Node **24.21.0**, and runs portable native SDK lifecycle checks against isolated loopback browser fixtures. It replaces the fork-only checkpoint gate and requires all 12 checks to pass with no failure, cancellation or skip. It does not repeat the official full suite. Sources use a private `/tmp` root, sanitized environment, non-root execution, and a loopback-only network namespace. No personal profile, credentials, or model calls are involved.
|
|
336
337
|
|
|
337
|
-
|
|
338
|
+
CI writes setup, provenance, command-reference, and native TAP results to the job log and uploads short-lived evidence: `official-host`, `fork-host`, and `macos-host` contain exact selected host receipts; `native-lifecycle` contains the host receipt, native TAP results and lifecycle transcripts. Each artifact is retained for three days. There are no persistent caches. The checks cover their exact source, host, and synthetic browser paths; they do not replace release/manual-publication gates, real-site authentication, or full personal-workspace restore qualification.
|
|
338
339
|
|
|
339
340
|
### Native Linux socket-root regression
|
|
340
341
|
|
package/docs/REQUIREMENTS.md
CHANGED
|
@@ -72,8 +72,10 @@ Define the product requirements and constraints for `pi-agent-browser-native`.
|
|
|
72
72
|
- Hold a cooperating-process execution lock keyed by actual socket context, canonical namespace, and session across direct helper/action work or the entire code cell. Use ordered multi-identity acquisition for replacement/cleanup and a namespace barrier for direct close-all. This does not coordinate humans, third-party clients, or older versions, guarantee atomic browser actions, or provide rollback.
|
|
73
73
|
- Persist one canonical begin/finish event format for direct/code/host work through native Pi custom entries and a shared reducer. Define full captures once, preserve exact targets/native identity, and persist artifact-row changes rather than unchanged views. Interrupted outcomes remain unknown until inspection; failed begin blocks dispatch and failed finish stops dependent work without mutation retry. Require actual journal publication before browser dispatch. Legacy browser/lease decoders belong only to validated offline stopped-copy conversion; preserve original bytes, all ancestry and owner UUIDs. A fork must never acquire its parent's cleanup rights. Bound extension replay with token projection and winning definitions on both hosts, while disclosing stock official Pi's eager native loader and individual-value memory ceilings; see [`SESSION_STORAGE.md`](SESSION_STORAGE.md).
|
|
74
74
|
- Use one canonical model-visible observation for JSON/prose/code with success/data/error, exact recovery, artifact receipts, and image observations. Native `outputSchema`/`structuredContent` under the Browser tool namespace expose that same bounded redacted projection, not raw internal details. Loader inventory and normalized search data keep their existing shapes. Details is audit/replay metadata, not the sole source of actionable failures. Return canonical native `isError` without a result-repair hook; preserve prose notices and explicit JSON. Inner calls avoid full model rendering/spills/image encoding. Bound outer output and name a complete redacted observation spill rather than truncating executable payloads.
|
|
75
|
-
- Attach only verified real images; code selects current-cell image handles explicitly. Report pixel dimensions and CSS viewport/DPR/crop/scroll/frame evidence conservatively. Unknown geometry remains unknown; neither full-page nor element captures are assumed viewport coordinates. Do not inject model-specific image-detail/provider payload changes.
|
|
75
|
+
- Attach only verified real images; code selects current-cell image handles explicitly. Report pixel dimensions and CSS viewport/DPR/crop/scroll/frame evidence conservatively. Unknown geometry remains unknown; neither full-page nor element captures are assumed viewport coordinates. Positively identified Lightpanda images are text-rendered, never graphical CSS/mouse-coordinate evidence. Do not inject model-specific image-detail/provider payload changes.
|
|
76
|
+
- Explicit native text (`--json false`) must retain redacted stdout and process failures without promoting JSON-looking, confirmation-looking or URL-shaped page content into trusted receipts or session targets; structured callers/helpers remain strict JSON. Optional output booleans follow native last-wins semantics. Native empty batch rows are accepted without rewriting stdin; malformed row shapes still fail before dispatch. A failed process's stderr takes precedence over absent/invalid JSON diagnostics, while zero-exit malformed machine output stays strict.
|
|
76
77
|
- Direct upstream capability remains available: explicit sessions, state/restore paths, profiles, config, file access, launch arguments, environment, local pages, output paths, and close arguments pass through under ordinary native semantics. Keep all session/state rows visible. Wrapper validation concerns input shape, lifecycle integrity, and truthful evidence, not authorization of resources.
|
|
78
|
+
- Keep Chrome default and separately installed Lightpanda opt-in. Treat `--engine` as launch-scoped through existing managed fresh-launch recovery. Omit only wrapper-supplied Chrome profile/executable defaults when Lightpanda is selected; preserve native caller settings and unsupported-command errors. No new driver, engine cache, browser bundle, or automatic Chrome fallback.
|
|
77
79
|
- Native download owns clicking and saving, including loopback, Blob exports, and redirects. Create artifact directories before dispatch and verify saved files; output-data writes must not alias browser artifacts. Pending recording is not saved video.
|
|
78
80
|
- Maintain a deliberate 0.7 migration map and package-level rollback that preserves saved authentication, explicit artifacts, and Pi transcripts. Restart every cooperating Pi participant on the updated package before relying on shared ordering. No universal performance, sandbox, authentication-persistence, or Windows qualification claims.
|
|
79
81
|
|
package/docs/SESSION_STORAGE.md
CHANGED
|
@@ -4,7 +4,7 @@ Pi's native JSONL journal owns conversation history and branch ancestry. Browser
|
|
|
4
4
|
|
|
5
5
|
## Events and snapshots
|
|
6
6
|
|
|
7
|
-
Every browser-affecting direct, code, or Electron host operation records a begin before dispatch and a finish after observing its effects. With a journal, each record requires confirmed publication. Direct browser use in a disabled/no-file session (`--no-session`) uses Pi's current in-memory ancestry without claiming persistence. Code still requires a published intent journal; a named but not-yet-published persistent conversation also refuses browser dispatch. Sessionless local inspections and browser-independent reads remain available before publication, including full redacted private spills and caller-selected output files. Their observation-only artifact receipts do not require a journal; browser effects, page changes and native confirmations retain their required records. `data.event` version 1 correlates them with an operation ID, the outer tool-call ID and inner command index. The same reducer applies committed live events and replays the selected branch.
|
|
7
|
+
Every browser-affecting direct, code, or Electron host operation records a begin before dispatch and a finish after observing its effects. With a journal, each record requires confirmed publication. Direct browser use in a disabled/no-file session (`--no-session`) uses Pi's current in-memory ancestry without claiming persistence. Code still requires a published intent journal; a named but not-yet-published persistent conversation also refuses browser dispatch. Sessionless local inspections and browser-independent reads remain available before publication, including full redacted private spills and caller-selected output files. Their observation-only artifact receipts do not require a journal; browser effects, page changes and native confirmations retain their required records. `data.event` version 1 correlates them with an operation ID, the outer tool-call ID and inner command index. The same reducer applies committed live events and replays the selected branch. Page events also retain the selected native confirmation string (including explicit `""`; native unset is not pinned) for each canonical namespace/session, including interrupted begins. A definite unestablished completion restores the previous setting only if no earlier browser helper could have established the request. Same-identity helpers and follow-ups use it before dispatch; successful close clears it with session state. Replacement cleanup uses the old identity's setting and keeps its continuity state on failed close. New explicit `--confirm-actions` / `--config` selections retain native precedence; bare follow-ups preserve the established setting over ambient defaults. Pending confirmation commands stay solely in native daemon memory. The existing ordered confirmation marker retains only exact ID/identity and compact command/native-action provenance, including helper controls; it never stores argv or replays commands. Exact guarded decisions avoid overwriting page helpers without claiming native ID validation or browser independence. Cold pending reopen keeps the target unknown and exposes the native decision before the getter can dispatch. Pending navigation rows cannot supply an observed target; completed compound confirmations, including native batch rows, reconcile their native target before follow-ups. Successful confirmed code close records realized retirement, not an unresolved intent; attachment replay consumes that compact close effect too. Confirmed full snapshot rows feed the ordinary ordered capture fold. Only a final successful complete confirmed capture marks the existing cleared confirmation as fresh for one next same-identity call. Semantic role click/check/fill and unique accessible select resolve through that same eligible capture. The call persists consumption, including direct/code/batch replay, so resume cannot resurrect the one-use sample. Ordinary membership, target, invalidation and generation checks still apply; the one use does not resample DOM-only renames after capture.
|
|
8
8
|
|
|
9
9
|
A finish explicitly reuses, replaces, invalidates, or leaves refs unknown. Each complete capture defines one `data.snapshot`: an ID, keyed ref membership and accessible metadata, exact private target, and verified native daemon generation when available. Ordinary completions name the ancestral definition instead of copying its map. Dedicated snapshot results and requested structured exports still retain their full data. Native delta baselines remain native; a Pi snapshot ID cannot restore them.
|
|
10
10
|
|