pi-agent-browser-native 0.6.16 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/CHANGELOG.md +18 -1
  2. package/README.md +59 -165
  3. package/dist/extensions/agent-browser/index.js +634 -501
  4. package/dist/extensions/agent-browser/lib/browser-transcript.js +33 -0
  5. package/dist/extensions/agent-browser/lib/command-taxonomy.js +8 -4
  6. package/dist/extensions/agent-browser/lib/execution-cwd.js +66 -0
  7. package/dist/extensions/agent-browser/lib/input-modes/job.js +2 -213
  8. package/dist/extensions/agent-browser/lib/input-modes/lookups.js +1 -1
  9. package/dist/extensions/agent-browser/lib/input-modes/params.js +146 -142
  10. package/dist/extensions/agent-browser/lib/input-modes/script.js +64 -92
  11. package/dist/extensions/agent-browser/lib/managed-session-policy-lock.js +170 -24
  12. package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-paths.js +27 -20
  13. package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostics.js +14 -2
  14. package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result.js +26 -13
  15. package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +21 -1
  16. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +4 -2
  17. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +1 -3
  18. package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +41 -24
  19. package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery.js +3 -1
  20. package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-observation.js +79 -0
  21. package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +3 -4
  22. package/dist/extensions/agent-browser/lib/orchestration/input-plan.js +19 -52
  23. package/dist/extensions/agent-browser/lib/orchestration/native-session-defaults.js +3 -3
  24. package/dist/extensions/agent-browser/lib/orchestration/operation-paths.js +76 -0
  25. package/dist/extensions/agent-browser/lib/orchestration/output-file.js +3 -3
  26. package/dist/extensions/agent-browser/lib/orchestration/script-mode.js +91 -278
  27. package/dist/extensions/agent-browser/lib/pi-tool-rendering.js +9 -9
  28. package/dist/extensions/agent-browser/lib/playbook.js +46 -25
  29. package/dist/extensions/agent-browser/lib/process-identity.js +22 -6
  30. package/dist/extensions/agent-browser/lib/process.js +16 -13
  31. package/dist/extensions/agent-browser/lib/results/action-recommendations.js +2 -2
  32. package/dist/extensions/agent-browser/lib/results/next-actions.js +1 -2
  33. package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +25 -4
  34. package/dist/extensions/agent-browser/lib/results/presentation/batch.js +6 -2
  35. package/dist/extensions/agent-browser/lib/results/presentation/common.js +2 -0
  36. package/dist/extensions/agent-browser/lib/results/presentation/content.js +39 -0
  37. package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +3 -3
  38. package/dist/extensions/agent-browser/lib/results/presentation/large-output.js +41 -1
  39. package/dist/extensions/agent-browser/lib/results/presentation.js +16 -11
  40. package/dist/extensions/agent-browser/lib/runtime.js +18 -6
  41. package/dist/extensions/agent-browser/lib/session-page-state.js +3 -4
  42. package/dist/extensions/agent-browser/lib/tool-surface.js +131 -0
  43. package/dist/extensions/agent-browser/script-worker.js +4 -1
  44. package/docs/ARCHITECTURE.md +47 -61
  45. package/docs/COMMAND_REFERENCE.md +78 -137
  46. package/docs/ELECTRON.md +51 -49
  47. package/docs/RELEASE.md +31 -15
  48. package/docs/REQUIREMENTS.md +18 -14
  49. package/docs/SUPPORT_MATRIX.md +33 -15
  50. package/docs/TOOL_CONTRACT.md +218 -401
  51. package/package.json +4 -4
  52. package/scripts/doctor.mjs +2 -2
package/docs/ELECTRON.md CHANGED
@@ -10,9 +10,9 @@ Related docs:
10
10
 
11
11
  ## Purpose
12
12
 
13
- This guide is the entry point for using `pi-agent-browser-native` against desktop **Electron** applications. The wrapper exposes a top-level `electron` shorthand that owns the awkward discover → launch → attach → probe → cleanup sequence so agents do not hand-build `--remote-debugging-port` argv, poll `DevToolsActivePort`, and `kill` profile directories. After attach, the rest of the native `agent_browser` surface (`snapshot`, `find`, `click`, `fill`, `get`, `eval --stdin`, `batch`, `qa.attached`, and similar) works the same way it does against a web page.
13
+ This guide is the entry point for using `pi-agent-browser-native` against desktop **Electron** applications. The wrapper exposes `agent_browser_electron` with flat action fields that owns the awkward discover → launch → attach → probe → cleanup sequence so agents do not hand-build `--remote-debugging-port` argv, poll `DevToolsActivePort`, and `kill` profile directories. After attach, the rest of the native `agent_browser` surface (`snapshot`, `find`, `click`, `fill`, `get`, `eval --stdin`, `batch`, persistent browser code, and advanced attached QA) works the same way it does against a web page.
14
14
 
15
- This document is structured for users, not implementers. Field-level rules live in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron); this guide focuses on **when** and **how** to use them, and on the safety and ownership boundary the wrapper enforces.
15
+ This document is structured for users, not implementers. Field-level rules live in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#agent_browser_electron); this guide focuses on **when** and **how** to use them, and on the safety and ownership boundary the wrapper enforces.
16
16
 
17
17
  ## Who this is for
18
18
 
@@ -20,10 +20,12 @@ This document is structured for users, not implementers. Field-level rules live
20
20
  - **Coding agents** that need a low-context lifecycle for desktop apps such as VS Code, Cursor, Obsidian, Slack, or any app built on Electron, without re-implementing the CDP attach dance every session.
21
21
  - **Maintainers and reviewers** validating the wrapper's Electron behavior before release; verification evidence lives under `RQ-0096` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md).
22
22
 
23
- It is **not** an upstream `agent-browser` reference and it does **not** replace the canonical [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron) for exact field semantics, validation rules, or failure categories.
23
+ It is **not** an upstream `agent-browser` reference and it does **not** replace the canonical [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#agent_browser_electron) for exact field semantics, validation rules, or failure categories.
24
24
 
25
25
  ## Mental model
26
26
 
27
+ Enable the tools with `agent_browser_tools { "enable": ["electron", "qa"] }`. Below, `electron.launch` and similar labels mean `agent_browser_electron { "action": "launch", ... }`; `qa.attached` means `agent_browser_qa { "attached": true, ... }`. These are separate tools, not nested inputs on `agent_browser`.
28
+
27
29
  ```
28
30
  electron.list → discover Electron apps (host-only; no upstream spawn)
29
31
  electron.launch → launch a wrapper-owned isolated app, attach via CDP, hand off (snapshot|tabs|connect)
@@ -42,14 +44,14 @@ Choosing between the two is a real decision, not a stylistic one. See [Wrapper-o
42
44
 
43
45
  ## Quick start
44
46
 
45
- Discover the app, launch with the default snapshot handoff, work with current refs, then clean up:
47
+ Use `agent_browser_electron` for the lifecycle rows below and `agent_browser` for the native snapshot: discover, launch with the default snapshot handoff, inspect, then clean up.
46
48
 
47
49
  ```json
48
- { "electron": { "action": "list", "query": "code" } }
49
- { "electron": { "action": "launch", "appName": "Visual Studio Code", "handoff": "snapshot" } }
50
+ { "action": "list", "query": "code" }
51
+ { "action": "launch", "appName": "Visual Studio Code", "handoff": "snapshot" }
50
52
  { "args": ["snapshot", "-i"] }
51
- { "electron": { "action": "probe", "timeoutMs": 5000 } }
52
- { "electron": { "action": "cleanup", "launchId": "electron-…" } }
53
+ { "action": "probe", "timeoutMs": 5000 }
54
+ { "action": "cleanup", "launchId": "electron-…" }
53
55
  ```
54
56
 
55
57
  The launch result carries both a `launchId` (used by `status`/`probe`/`cleanup`) and an attached `sessionName` (used by browser-style `snapshot`/`tab`/`click`/`find` calls). Read both from `details.electron.launch` and `details.electron.identifiers`. With default implicit session reuse, the quick-start `args: ["snapshot", "-i"]` line uses that attached session without an extra `--session` argument; pass `--session` explicitly when you target a named upstream session instead.
@@ -57,7 +59,7 @@ The launch result carries both a `launchId` (used by `status`/`probe`/`cleanup`)
57
59
  For a quick "is the app actually showing what we expect?" smoke check after attach:
58
60
 
59
61
  ```json
60
- { "qa": { "attached": true, "expectedText": "Explorer", "screenshotPath": ".dogfood/electron.png" } }
62
+ { "attached": true, "expectedText": "Explorer", "screenshotPath": ".dogfood/electron.png" }
61
63
  ```
62
64
 
63
65
  `qa.attached` runs against the **current managed session** without opening a URL, so it works for any attached app — wrapper-owned or manually launched.
@@ -92,7 +94,7 @@ Then attach and choose a ready target before using refs:
92
94
  { "args": ["tab", "list"] }
93
95
  { "args": ["tab", "t2"] }
94
96
  { "args": ["snapshot", "-i"] }
95
- { "qa": { "attached": true, "expectedText": "Channels" } }
97
+ { "attached": true, "expectedText": "Channels" }
96
98
  ```
97
99
 
98
100
  A successful `connect` means the CDP endpoint accepted the session; it does **not** prove the app has an active rendered page yet. Prefer `details.nextActions` when present: `verify-connected-session-url` performs the only page read allowed while the attached target is unverified, and `list-connected-session-tabs` inspects attached targets. A verified target clears the guard; otherwise navigate explicitly or select the intended tab. After the read-only list, select or confirm a stable `t<N>` target, verify it with `get url`, and run `snapshot -i` explicitly before trusting refs. If the first `snapshot -i` says `No active page`, follow `list-tabs-after-no-active-page`. If it returns no useful refs without that error, manually run `tab list`, select a stable `t<N>` id for the app surface, then retry a condition wait or `snapshot -i` on that selected target.
@@ -114,14 +116,14 @@ This project is not adding a first-class host-idle primitive yet. Revisit that o
114
116
 
115
117
  ## Action reference
116
118
 
117
- The exact field schemas, validation rules, and `details.*` payload shapes live in [`TOOL_CONTRACT.md#electron`](TOOL_CONTRACT.md#electron). This section is a usage-oriented overview.
119
+ The exact field schemas, validation rules, and `details.*` payload shapes live in [`TOOL_CONTRACT.md#agent_browser_electron`](TOOL_CONTRACT.md#agent_browser_electron). This section is a usage-oriented overview.
118
120
 
119
121
  ### `electron.list` — discover apps
120
122
 
121
123
  Host-only scan; does not spawn upstream `agent-browser`. macOS (`/Applications/*.app`, `~/Applications/*.app`) and Linux (`.desktop` launchers under standard XDG, Flatpak, and Snap locations) are supported in v1. On Windows (and any non-macOS/non-Linux host), `list` returns `details.electron.platform: "unsupported"` with an empty `apps` array—use `executablePath` (or a host `appPath` that resolves to a verifiable Electron binary) for `launch` instead; `inspectElectronExecutablePath` in `extensions/agent-browser/lib/electron/discovery.ts` still gates Windows executables before spawn.
122
124
 
123
125
  ```json
124
- { "electron": { "action": "list", "query": "code", "maxResults": 25 } }
126
+ { "action": "list", "query": "code", "maxResults": 25 }
125
127
  ```
126
128
 
127
129
  Returns app metadata under `details.electron.apps`: `name`, optional `bundleId`/`desktopId`, `appPath`, `executablePath`, `platform`, and optional non-blocking `sensitivity` annotations. Apps flagged as likely sensitive (categories such as `notes`, `chat`, `mail`, `developer-workspace`, or `passwords-auth`) are printed with `[likely sensitive: …]`. These are **advisory hints**, not enforcement; see [Safety and ownership](#safety-and-ownership) for the policy boundary.
@@ -132,14 +134,12 @@ Pass **exactly one** target: `appPath`, `appName`, `bundleId`, or `executablePat
132
134
 
133
135
  ```json
134
136
  {
135
- "electron": {
136
- "action": "launch",
137
- "appName": "Visual Studio Code",
138
- "handoff": "snapshot",
139
- "targetType": "page",
140
- "timeoutMs": 30000,
141
- "appArgs": ["--disable-telemetry"]
142
- }
137
+ "action": "launch",
138
+ "appName": "Visual Studio Code",
139
+ "handoff": "snapshot",
140
+ "targetType": "page",
141
+ "timeoutMs": 30000,
142
+ "appArgs": ["--disable-telemetry"]
143
143
  }
144
144
  ```
145
145
 
@@ -162,9 +162,9 @@ Wrapper-owned launches **always** use an isolated temp profile and an OS-chosen
162
162
  Read-only inspection of one or more tracked launches. Without `launchId` or `all`, it selects the single active wrapper launch when unambiguous.
163
163
 
164
164
  ```json
165
- { "electron": { "action": "status" } }
166
- { "electron": { "action": "status", "launchId": "electron-…" } }
167
- { "electron": { "action": "status", "all": true } }
165
+ { "action": "status" }
166
+ { "action": "status", "launchId": "electron-…" }
167
+ { "action": "status", "all": true }
168
168
  ```
169
169
 
170
170
  Reports `cleanupState`, current debug-port and PID liveness, bounded CDP targets, and freshly measured `userDataDirState` under `details.electron.statuses`. An explicit `launchId` can inspect a **historical cleaned launch record**; default and `all: true` selection exclude cleaned records. Cleanup history does not determine current liveness. The tracked profile path is `present`, `absent` (only native `lstat` ENOENT), or `unknown` (other filesystem errors); a dangling symlink is present. This measures that path only, not all app residue, and is not stored in the launch record. Its managed-session title/URL reads hold the normal daemon-policy lock and owned restore context. Mismatch fields surface when the current managed session or tab no longer matches a live wrapper launch target — typically the cue to follow `reattach-electron-launch` before trusting old refs.
@@ -174,8 +174,8 @@ Reports `cleanupState`, current debug-port and PID liveness, bounded CDP targets
174
174
  `probe` collapses what would otherwise be separate `get url` / `get title` / focused-element `eval` / `tab list` / `snapshot -i` calls into one bounded result. Use it instead of chaining those reads when you just need a quick "where are we?" check. The wrapper holds the managed-session daemon-policy lock for the probe and runs every underlying read with the session's owned restore decision, so probing cannot restart the daemon under a different restore key.
175
175
 
176
176
  ```json
177
- { "electron": { "action": "probe" } }
178
- { "electron": { "action": "probe", "launchId": "electron-…", "timeoutMs": 5000 } }
177
+ { "action": "probe" }
178
+ { "action": "probe", "launchId": "electron-…", "timeoutMs": 5000 }
179
179
  ```
180
180
 
181
181
  Output appears under `details.electron.probe`: `title`, `url`, `focusedElement`, `activeTab`, `tabs`, compact `snapshot` metadata (`refCount`, `refIds`, optional text preview and omission counts), and `errors`. If every underlying read fails, the tool fails with `failureCategory: "upstream-error"`; it does not report a successful empty partial probe. Probes reject an unverified target before helper reads, then verify the live URL again before title, eval, tab, or snapshot helpers so external target drift cannot expose the wrong page. A current-managed probe also persists top-level `details.namespace`, `sessionTabTarget`, and `refSnapshot` so Pi reload/branch replay keeps the same namespaced page identity; unverified transitions persist as `details.sessionTabTargetUnknown: true` until `get url` or explicit navigation establishes a trustworthy target. When `launchId` is given, the probe is tied to that tracked launch and will surface mismatch guidance if the wrapper sees a session or target drift; visible output also includes debug-port/pid liveness so a stale `about:blank` against a dead launch is unmistakable.
@@ -189,8 +189,8 @@ Closes the tracked managed session, stops only the wrapper-tracked process, veri
189
189
  Cleanup partial failures fail the tool result with `failureCategory: "cleanup-failed"` and the `retry-electron-cleanup` next action references the same `launchId` so retries are bounded.
190
190
 
191
191
  ```json
192
- { "electron": { "action": "cleanup", "launchId": "electron-…" } }
193
- { "electron": { "action": "cleanup", "all": true } }
192
+ { "action": "cleanup", "launchId": "electron-…" }
193
+ { "action": "cleanup", "all": true }
194
194
  ```
195
195
 
196
196
  `electron.cleanup` **never** targets:
@@ -206,7 +206,7 @@ On Pi `quit`, active wrapper-owned Electron launches are best-effort cleaned. On
206
206
 
207
207
  ### `timeoutMs` by action (quick reference)
208
208
 
209
- `electron.list` has no configurable timeout: neither top-level `timeoutMs` nor nested `electron.timeoutMs` is accepted for its host scan. For every other action, nested `timeoutMs` applies to **different surfaces**, not an end-to-end action deadline. Authoritative rules and env overrides live under **Validation and defaults** in [`TOOL_CONTRACT.md#electron`](TOOL_CONTRACT.md#electron).
209
+ `agent_browser_electron` with `action: "list"` has no `timeoutMs` field. For every other action, flat `timeoutMs` applies to **different surfaces**, not an end-to-end action deadline. Authoritative rules and env overrides live in the Electron contract in [`TOOL_CONTRACT.md#agent_browser_electron`](TOOL_CONTRACT.md#agent_browser_electron).
210
210
 
211
211
  | Action | What `timeoutMs` covers when set | Typical default when omitted |
212
212
  | --- | --- | --- |
@@ -217,28 +217,26 @@ On Pi `quit`, active wrapper-owned Electron launches are best-effort cleaned. On
217
217
 
218
218
  ## `qa.attached` — current-session smoke check
219
219
 
220
- `qa` has two forms: the URL form (`qa: { url, … }`) and the attached form (`qa: { attached: true, … }`). The attached form is the right tool for Electron smoke checks after either launch path because it does not open a URL and runs all checks against the current managed session.
220
+ `agent_browser_qa` has two flat forms: `{ url, … }` and `{ attached: true, … }`. The attached form is the right tool for Electron smoke checks after either launch path because it does not open a URL and runs all checks against the current managed session.
221
221
 
222
222
  ```json
223
223
  {
224
- "qa": {
225
- "attached": true,
226
- "expectedText": "Explorer",
227
- "expectedSelector": "@e1",
228
- "checkConsole": true,
229
- "checkErrors": true,
230
- "screenshotPath": ".dogfood/electron.png"
231
- }
224
+ "attached": true,
225
+ "expectedText": "Explorer",
226
+ "expectedSelector": "@e1",
227
+ "checkConsole": true,
228
+ "checkErrors": true,
229
+ "screenshotPath": ".dogfood/electron.png"
232
230
  }
233
231
  ```
234
232
 
235
- `qa.attached` rejects `url` and is incompatible with `sessionMode: "fresh"` — attach first with `electron.launch` or raw `connect`, then run `qa.attached`. Preserved-buffer diagnostics (`checkConsole`, `checkErrors`, `checkNetwork`) default to `false` for attached QA; opt into them when you want historical session buffers to fail the smoke. The full field rules and pass/fail classification live in [`TOOL_CONTRACT.md#qa`](TOOL_CONTRACT.md#qa).
233
+ `qa.attached` rejects `url` and is incompatible with `sessionMode: "fresh"` — attach first with `electron.launch` or raw `connect`, then run `qa.attached`. Preserved-buffer diagnostics (`checkConsole`, `checkErrors`, `checkNetwork`) default to `false` for attached QA; opt into them when you want historical session buffers to fail the smoke. The full field rules and pass/fail classification live in [`TOOL_CONTRACT.md#agent_browser_qa`](TOOL_CONTRACT.md#agent_browser_qa).
236
234
 
237
235
  In attached Electron sessions, broad selectors such as `body`, `html`, `main`, or `[role=application]` can read the entire app shell. When `get text <selector>` looks too broad, the wrapper may attach `details.electronGetTextScopeWarning` and a `snapshot-for-electron-text-scope` next action; prefer a fresh `snapshot -i`, a current `@ref`, or a narrower panel selector.
238
236
 
239
- ## `sourceLookup` against packaged Electron apps
237
+ ## Source lookup against packaged Electron apps
240
238
 
241
- `sourceLookup` is an experiment for hinting at the source file/component behind a visible element. It is **opt-in** and **evidence-based**: it reports confidence and evidence rather than claiming a guaranteed mapping. The same experimental helper works against packaged Electron apps, but with two important boundaries:
239
+ Enable `source` and use `agent_browser_source` for experimental hints about the source file/component behind a visible element. It is **opt-in** and **evidence-based**: it reports confidence and evidence rather than claiming a guaranteed mapping. The same experimental helper works against packaged Electron apps, but with two important boundaries:
242
240
 
243
241
  1. **Scope of the workspace scan.** `sourceLookup` walks the Pi session **cwd** (default `maxWorkspaceFiles: 2000`, hard cap 5000). It does **not** unpack `app.asar` or installed app resources. For packaged apps where the source lives inside `Contents/Resources/app.asar`, the workspace-search lane will commonly return no candidates.
244
242
  2. **React DevTools requirement.** `react inspect <id>` requires the session to have been launched with `--enable react-devtools` before first navigation. For Electron, the wrapper's `electron.launch` path does **not** inject `--enable react-devtools` into the Electron process; that flag belongs to upstream `agent-browser` Chromium launches. If the Electron app does not already expose a React DevTools backend, expect `react inspect` to fail; DOM-attribute and workspace-search candidates may still surface.
@@ -246,10 +244,10 @@ In attached Electron sessions, broad selectors such as `body`, `html`, `main`, o
246
244
  For wrapper-tracked packaged Electron sessions where `status` is `no-candidates`, the wrapper attaches `workspaceRoot` plus optional `electronContext` (`launchId?`, `appName?`, `appPath?`, `executablePath?`, `sessionName?`, `url?`) and limitations explaining the bundle/asar boundary, plus `snapshot-electron-session`, `probe-electron-launch`, and `list-electron-tabs` next actions so you can inspect the live app and decide whether to widen the workspace or pull source out-of-band before re-running the lookup.
247
245
 
248
246
  ```json
249
- { "sourceLookup": { "selector": "#save", "reactFiberId": "2", "componentName": "SaveButton" } }
247
+ { "selector": "#save", "reactFiberId": "2", "componentName": "SaveButton" }
250
248
  ```
251
249
 
252
- Treat `sourceLookup` output as a starting point for navigation, not a substitute for reading code. Full contract: [`TOOL_CONTRACT.md#sourcelookup`](TOOL_CONTRACT.md#sourcelookup).
250
+ Treat `sourceLookup` output as a starting point for navigation, not a substitute for reading code. Full contract: [`TOOL_CONTRACT.md#agent_browser_source`](TOOL_CONTRACT.md#agent_browser_source).
253
251
 
254
252
  ## Safety and ownership
255
253
 
@@ -277,12 +275,10 @@ Both lists match `appName`, `bundleId`, `desktopId`, `appPath`, or `executablePa
277
275
 
278
276
  ```json
279
277
  {
280
- "electron": {
281
- "action": "launch",
282
- "appName": "Slack",
283
- "allow": ["Slack"],
284
- "deny": ["1Password", "Bitwarden"]
285
- }
278
+ "action": "launch",
279
+ "appName": "Slack",
280
+ "allow": ["Slack"],
281
+ "deny": ["1Password", "Bitwarden"]
286
282
  }
287
283
  ```
288
284
 
@@ -353,6 +349,12 @@ Single-instance Electron behavior is a common cause of `timeout` and `upstream-e
353
349
  ### Mismatch between `status` and the active session
354
350
  - `electron.status` may report a live wrapper launch while the managed session has drifted to `about:blank`. Follow `reattach-electron-launch`, then refresh refs before reusing old `@e…` handles. For non-wrapper tab drift where `details.nextActions` names `select-intended-tab-after-drift`, use that stable `t<N>` action plus `snapshot-after-tab-recovery` before continuing.
355
351
 
352
+ ## Code and shared-session coordination
353
+
354
+ `agent_browser_code` can inspect/interact with the attached browser by selecting its returned `sessionName` (and namespace when present). The browser survives the cell; variables reset. Electron host discovery/launch/cleanup stays on `agent_browser_electron`. Updated cooperating Pi processes order helpers and actions with the shared socket/namespace/session lock; a whole code cell holds that boundary. Multiple named native sessions attached to the same external app are not a universal app lock. Humans and external clients can still change the app.
355
+
356
+ Recovery actions now name `agent_browser_electron` with flat parameters, or `agent_browser` for native operations. Enable `electron` through the loader if a returned lifecycle action is not active. See [0.7 migration](TOOL_CONTRACT.md#07-migration).
357
+
356
358
  ## Cleanup checklist
357
359
 
358
360
  Before ending the task:
@@ -371,7 +373,7 @@ If `cleanup` returns `failureCategory: "cleanup-failed"`, inspect `details.elect
371
373
 
372
374
  ## Where to go next
373
375
 
374
- - For exact field semantics, schemas, and `details.*` payloads: [`TOOL_CONTRACT.md#electron`](TOOL_CONTRACT.md#electron) and [`TOOL_CONTRACT.md#qa`](TOOL_CONTRACT.md#qa).
376
+ - For exact field semantics, schemas, and `details.*` payloads: [`TOOL_CONTRACT.md#agent_browser_electron`](TOOL_CONTRACT.md#agent_browser_electron) and [`TOOL_CONTRACT.md#agent_browser_qa`](TOOL_CONTRACT.md#agent_browser_qa).
375
377
  - For workflow examples woven into the broader command surface: [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#electron-desktop-apps).
376
378
  - For the closed `RQ-0068` recipe-layer decision that bounds why Electron support is a typed shorthand and not a generic recipe runtime: [`ARCHITECTURE.md`](ARCHITECTURE.md#no-reusable-recipe-layer-yet).
377
379
  - For the full release-readiness audit and the `RQ-0096` evidence row: [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md).
package/docs/RELEASE.md CHANGED
@@ -25,6 +25,22 @@ Linux/macOS, declared Node floors, official/fork host and consumer-artifact chec
25
25
 
26
26
  Docs/workflow-only changes may reuse prior live/platform evidence when the complete compiled runtime, runtime dependencies and relevant tests are unchanged. Changed published files require a new retained tarball checked by `verify-package.mjs --smoke-pi`, including runtime-only SDK and actual CLI loading. Publish those exact checked bytes without repacking.
27
27
 
28
+ ## 0.7 upgrade and rollback
29
+
30
+ The 0.7 tool contract is breaking: direct argv stays compact, `script` moves to `agent_browser_code.code` with a persistent browser, the job DSL is replaced by native batch/code, and five advanced tools use flat schemas after native activation. Publish with the [migration map](TOOL_CONTRACT.md#07-migration), not a parallel legacy public route. Preserve saved auth, explicit files, and Pi journals. Existing strict pre-0.7 isolated-script leases are only read for cleanup.
31
+
32
+ Before activating the update, finish or inspect in-flight mutations/recordings and restart every cooperating Pi process onto the same package. `/reload` alone is not reliable for picking up rebuilt compiled code. Mixed versions do not share the new execution-ordering guarantee.
33
+
34
+ Rollback is package-level: select the exact previously installed 0.6 release with `pi install npm:pi-agent-browser-native@<previous-0.6-version>`, then fully restart participating Pi processes with their existing session files. Do not delete auth/profile/state files, artifacts, or transcripts. The older package cannot interpret all new inner-transition records; inspect the live URL and obtain fresh refs before further actions. Rollback does not reverse submitted browser actions, stop external clients, or restore JavaScript variables.
35
+
36
+ ### Changed-path qualification
37
+
38
+ The minimum Pi runtime is 0.87.0, matching the native APIs used by the new tool contract. Wildcard peers remain host-owned dependency declarations, not support for older runtimes.
39
+
40
+ Qualify official Pi 0.87 and the fork independently with the existing host/package/lifecycle infrastructure. Required evidence for the redesign includes compact schema and additive activation/history, native prompt sections, shared direct/code auth continuity, fresh JavaScript globals, selected real images and conservative geometry, visible handled failures/exact recovery, real cross-process helper/action and whole-cell ordering, cancellation/reaping, and ordered transition replay after interruption/branch changes. Retain native batch and advanced action/QA/Electron/source outcome coverage. Small synthetic timing comparisons are not universal performance claims or sandbox certification.
41
+
42
+ Existing owner waivers remain authoritative. Windows qualification is waived; report diagnostic failures without claiming a pass or restarting waived repair work. GitHub Actions remains disabled under the current explicit owner instruction; use the authorized local checks and review evidence, and label CI as not run rather than passed. This does not revoke other approved verification or authorize unrequested platform work.
43
+
28
44
  ## Pre-release checks
29
45
 
30
46
  From the repository root:
@@ -85,13 +101,13 @@ crabbox list --provider parallels
85
101
 
86
102
  The Crabbox gate is only green when suite assertions and artifact manifests under `.artifacts/platform-smoke/` are green and no unexpected lease/clone remains.
87
103
 
88
- The deterministic dogfood mode clean-builds the compiled package, uses the extension harness and real `agent-browser` on `PATH` against a deterministic loopback HTTP fixture, then verifies top-level `script` conditional aggregation with isolated-session cleanup, `qa`, `semanticAction`, constrained `job`, screenshot artifact verification, and session close. Use `npm run verify -- dogfood --keep-artifacts` or `--artifact-dir <path>` only while debugging, then delete retained screenshots. This smoke complements, but does not replace, human-readable interactive transcript evidence.
104
+ The deterministic dogfood mode clean-builds the compiled package, uses the extension harness and real `agent-browser` on `PATH` against a deterministic loopback HTTP fixture, then verifies persistent browser code aggregation, flat advanced QA/actions, native batch, screenshot artifact verification, and explicit session close. Use `npm run verify -- dogfood --keep-artifacts` or `--artifact-dir <path>` only while debugging, then delete retained screenshots. This smoke complements, but does not replace, human-readable interactive transcript evidence.
89
105
 
90
- Every release also requires interactive `tmux`-driven Pi dogfood with the native `agent_browser` tool against real sites. For extension-focused release smokes, use `pi --approve --no-extensions --no-skills -e .` from the trusted checkout before publish so auto-loaded dogfood/QA skills cannot replace the bounded smoke workflow; omit `--approve` only when the smoke is explicitly testing Pi's Project Trust prompt. Run separate skill-enabled dogfood only when validating skill routing or report-generation behavior. Drive prompts with `tmux send-keys`, exercise at least one simple static site and one real documentation/product site, include the higher-level `qa` or `job`/`batch` surfaces when they changed, close every opened browser session, remove screenshots/temp artifacts, and record the outcome in the release notes or support-matrix evidence. Do not paste raw multi-line prompts into a tmux Pi pane: plain newlines submit separate queued user messages. For scripted smoke driving, collapse prompt files to one line before sending (`PROMPT=$(tr '\n' ' ' < /tmp/smoke-prompt.md); tmux send-keys -t "$SESSION":0.0 -l "$PROMPT"; tmux send-keys -t "$SESSION":0.0 Enter`). For manual multi-line editing, use Pi's external editor shortcut (`Ctrl+G`) or configure tmux extended keys so Pi can receive `Shift+Enter` for newlines; see the installed Pi `docs/tmux.md` guidance. Automated localhost, fake-upstream, and deterministic dogfood gates do not replace this human-readable live-site transcript evidence. When `script` changes, add a persisted-session code-mode pass that aggregates at least two real pages into one bounded emitted value, exercises one conditional branch, compares the exact row/value result with an ordinary-call baseline, confirms the transcript contains one top-level tool call rather than each inner call, verifies `details.scriptSession.cleanup: "closed"`, and checks no wrapper-owned browser/tmux/temp child remains. Also run a bounded timeout or quit/reload child-reaping check and a focused active-branch lease-recovery test; never use `--no-session` for script dogfood. When `agent_browser_web_search` or package config changed, add one key-free smoke proving the optional tool is absent without config, one fake/unit-backed smoke in the default suite, and one opt-in live Exa or Brave Search check with a real key while confirming the key does not appear in transcripts, stdout/stderr, config status, PR text, or artifacts. When `electron.*` surfaces, attached-session diagnostics, or `qa.attached` changed, add a local Electron pass: `electron.list` → `electron.launch` (expect isolated profile behavior) → `snapshot -i` or `electron.probe` / `qa.attached` → `electron.cleanup` with the returned `launchId`, verifying status/mismatch guidance if you simulate a dead renderer or stale refs. For dense-dashboard stress coverage, use the [public Grafana stress checklist](#public-grafana-stress-checklist) below; it is a maintainer workflow, not bundled product skill or recipe runtime.
106
+ Every release also requires interactive `tmux`-driven Pi dogfood with the native `agent_browser` tool against real sites. For extension-focused release smokes, use `pi --approve --no-extensions --no-skills -e .` from the trusted checkout before publish so auto-loaded dogfood/QA skills cannot replace the bounded smoke workflow; omit `--approve` only when the smoke is explicitly testing Pi's Project Trust prompt. Run separate skill-enabled dogfood only when validating skill routing or report-generation behavior. Drive prompts with `tmux send-keys`, exercise at least one simple static site and one real documentation/product site, include advanced QA, code, or native batch when those surfaces changed, close every opened browser session, remove screenshots/temp artifacts, and record the outcome in the release notes or support-matrix evidence. Do not paste raw multi-line prompts into a tmux Pi pane: plain newlines submit separate queued user messages. For scripted smoke driving, collapse prompt files to one line before sending (`PROMPT=$(tr '\n' ' ' < /tmp/smoke-prompt.md); tmux send-keys -t "$SESSION":0.0 -l "$PROMPT"; tmux send-keys -t "$SESSION":0.0 Enter`). For manual multi-line editing, use Pi's external editor shortcut (`Ctrl+G`) or configure tmux extended keys so Pi can receive `Shift+Enter` for newlines; see the installed Pi `docs/tmux.md` guidance. Automated localhost, fake-upstream, and deterministic dogfood gates do not replace this human-readable live-site transcript evidence. When code changes, add a persisted-session `agent_browser_code` pass that aggregates at least two real pages, exercises a conditional branch, compares selected output with an ordinary-call baseline, verifies selected screenshot bytes reach the model, and checks browser/auth continuity on a later direct/code call with fresh JS globals. Review ordered custom transition entries and interrupted/replayed state, not only the outer result. Exercise handled failures, timeout/cancellation, whole-cell ordering, and child reaping; close only test-owned browsers afterward and retain Pi transcripts. Never use `--no-session` for code dogfood. When `agent_browser_web_search` or package config changed, add one key-free smoke proving the optional tool is absent without config, one fake/unit-backed smoke in the default suite, and one opt-in live Exa or Brave Search check with a real key while confirming the key does not appear in transcripts, stdout/stderr, config status, PR text, or artifacts. When `electron.*` surfaces, attached-session diagnostics, or `qa.attached` changed, add a local Electron pass: `electron.list` → `electron.launch` (expect isolated profile behavior) → `snapshot -i` or `electron.probe` / `qa.attached` → `electron.cleanup` with the returned `launchId`, verifying status/mismatch guidance if you simulate a dead renderer or stale refs. For dense-dashboard stress coverage, use the [public Grafana stress checklist](#public-grafana-stress-checklist) below; it is a maintainer workflow, not bundled product skill or recipe runtime.
91
107
 
92
- When reviewing saved session JSONL after a failed smoke or a `qa` preset that reclassified an upstream-successful batch, expect `agent_browser` tool rows to carry `isError: true` whenever `details.resultCategory` is `failure`. For normal prose output, model-visible text should end with a `Pi tool isError: true` category line; for caller-requested `--json` output, the hook preserves parseable JSON and only patches `isError`. The extension applies that patch on the `tool_result` path so Pi’s transcript matches the wrapper contract ([`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details)). Preserve a normal Pi session directory for those checks; avoiding `--no-session` keeps this evidence intact ([`AGENTS.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/AGENTS.md) preferred validation workflow).
108
+ When reviewing saved session JSONL after a failed smoke or QA reclassification, expect browser tool rows to carry `isError: true` whenever `details.resultCategory` is `failure`. The model-visible observation must also report failure. Canonical JSON stays parseable; Pi's error patch must not append prose to it. The extension applies that patch on the `tool_result` path so Pi’s transcript matches the wrapper contract ([`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details)). Preserve a normal Pi session directory for those checks; avoiding `--no-session` keeps this evidence intact ([`AGENTS.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/AGENTS.md) preferred validation workflow).
93
109
 
94
- The configured-source lifecycle regression harness is required before release because it launches an interactive Pi 0.84.0+ process under `tmux` with `--approve` and validates `/reload`, full relaunch with the same exact `--session-id`, managed-session continuity, persisted artifacts, compiled-entrypoint pickup after process restart, and Pi failure-patch behavior. Branch-backed `session_tree` rehydration and cleanup ownership are validated by focused extension harness tests:
110
+ The configured-source lifecycle regression harness is required before release because it launches an interactive Pi 0.87.0+ process under `tmux` with `--approve` and validates `/reload`, full relaunch with the same exact `--session-id`, managed-session continuity, persisted artifacts, compiled-entrypoint pickup after process restart, and Pi failure-patch behavior. Branch-backed `session_tree` rehydration and cleanup ownership are validated by focused extension harness tests:
95
111
 
96
112
  ```bash
97
113
  npm run verify -- lifecycle
@@ -116,7 +132,7 @@ Minimum pass:
116
132
  1. Open the URL with the native `agent_browser` tool in a fresh session.
117
133
  2. Run `snapshot -i`; confirm the output is useful on a dense dashboard, including high-value controls and bounded spill behavior when needed.
118
134
  3. Exercise one dashboard scroll path. If page-level `scroll` does not move visible content, confirm `details.scrollNoop` / next actions or equivalent guidance points to snapshot/screenshot verification and nested-scroll recovery.
119
- 4. Exercise one explicit combobox-targeted action such as a role/name `semanticAction` on a dashboard variable. If it only focuses the field, confirm `details.comboboxFocus` / next actions point to `snapshot -i`, `press ArrowDown`, and `press Enter` when the closed-state evidence qualifies.
135
+ 4. Exercise one explicit combobox-targeted action such as a role/name `agent_browser_action` on a dashboard variable. If it only focuses the field, confirm `details.comboboxFocus` / next actions point to `snapshot -i`, `press ArrowDown`, and `press Enter` when the closed-state evidence qualifies.
120
136
  5. Capture at least one screenshot artifact and verify `details.artifactVerification` before using the file.
121
137
  6. If `ffmpeg` is on `PATH`, run `record start` / visible interaction / `record stop` and verify the video artifact. Native 0.37 checks ffmpeg before start; older natives may instead return pending output plus `details.recordingDependencyWarning`. Neither is usable recording evidence. Short/cold 0.37 captures have failed on Ubuntu while explicit 12-second fixture captures encoded successfully; record capture duration and every failed attempt, and never treat an empty file as a captured frame.
122
138
  7. Inspect `network requests`, `console`, and `errors` summaries. Treat Grafana Play-side noise such as analytics/Sentry requests, public-demo 403s, and console errors as site noise unless the wrapper leaks secrets, hides actionable failed rows, misclassifies artifacts, or suggests unsafe follow-ups.
@@ -131,10 +147,10 @@ Use this validation prompt after changing click enrichment, tab pinning, ref pre
131
147
  Run it in an isolated checkout session with skills disabled so the run validates the extension browser workflow instead of external dogfood/QA skill routing. It is fine to restrict active tools at launch so the checkout extension is the only browser surface, but keep those launch details out of the user prompt:
132
148
 
133
149
  ```bash
134
- pi --approve --no-extensions --no-skills -e . --model openai-codex/gpt-5.5:minimal --tools agent_browser --session-dir "$SESSION_DIR"
150
+ pi --approve --no-extensions --no-skills -e . --model openai-codex/gpt-6-astra:low --tools agent_browser,agent_browser_code,agent_browser_tools --session-dir "$SESSION_DIR"
135
151
  ```
136
152
 
137
- Repeat with `--model openai-codex/gpt-5.5:medium` when validating instruction-following robustness. Use unique temp paths for each run and delete them afterward. Run separate skill-enabled dogfood sessions only when the thing under test is skill integration, not this bounded release smoke.
153
+ Repeat with the task's chosen Astra reasoning level when validating instruction-following robustness. Use unique temp paths for each run and delete them afterward. Run separate skill-enabled dogfood sessions only when the thing under test is skill integration, not this bounded release smoke.
138
154
 
139
155
  Submit the prompt as one Pi message. In tmux automation, write it to a temp file with placeholders replaced, collapse newlines to spaces, and send that one line; for manual multiline entry, use Pi's `Ctrl+G` external editor or a tmux setup that preserves `Shift+Enter` newlines. Do not paste the raw block into a tmux pane line-by-line.
140
156
 
@@ -183,7 +199,7 @@ Evaluator expectations after the queued Sauce Demo fixes: the agent should indep
183
199
  `npm run verify -- package-pi` runs the same package-content checks and additionally confirms that:
184
200
 
185
201
  - the packed package can be loaded through Pi SDK resource loading with the same isolation principle as `pi --no-extensions -e <package-source>`
186
- - `agent_browser` is registered without requiring optional Brave config
202
+ - direct/code/loader tools and inactive advanced tools register without requiring optional search config; explicit host selection remains honored
187
203
  - any optional companion tools remain governed by their own configuration gates
188
204
  - the registered `agent_browser` source resolves inside the extracted packed package path, not the working checkout
189
205
  - the packaged `agent_browser` tool can be executed through Pi's loaded native tool definition with a deterministic fake upstream `agent-browser --version` binary
@@ -215,7 +231,7 @@ Before publishing, validate both local-checkout modes without mixing their assum
215
231
  5. Run a smoke prompt that exercises `agent_browser`.
216
232
  6. Restart Pi after extension edits; configured-source `/reload` is a separate validation mode.
217
233
 
218
- For expanded-surface validation, the smoke prompt should cover native tool invocation rather than shelling out to `agent-browser`: `--version`, `--help`, `skills list`, `skills get core --full`, `open` with `sessionMode: "fresh"`, `snapshot -i`, `click`, top-level `semanticAction` (locator shorthand compiled to upstream `find` and native dropdown selection compiled to upstream `select`, optionally with `semanticAction.session` when you need the same named upstream session as a prior explicit `--session` call), `eval --stdin`, `batch` via stdin, top-level `job`, `qa`, or experimental `sourceLookup` / `networkSourceLookup` (compiled batch smoke), `screenshot <path>`, explicit `--session … open` plus `--session … close`, `network requests`, `console` / `errors`, `diff snapshot`, `stream status` plus `stream disable`, `dashboard start` plus `dashboard stop`, and `chat <message>` (credential failure is acceptable evidence of wrapper pass-through when `AI_GATEWAY_API_KEY` is intentionally unset). Clean up any opened browser session with `close`, remove temporary files, and kill the tmux session before ending validation.
234
+ For expanded-surface validation, the smoke prompt should cover native tool invocation rather than shelling out to `agent-browser`: `--version`, `--help`, `skills list`, `skills get core --full`, `open` with `sessionMode: "fresh"`, `snapshot -i`, `click`, flat `agent_browser_action` after enabling `action` (including unique accessible select and explicit session routing), `eval --stdin`, `batch` via stdin, `agent_browser_code`, advanced QA, and experimental source/network tools after loader activation, `screenshot <path>`, explicit `--session … open` plus `--session … close`, `network requests`, `console` / `errors`, `diff snapshot`, `stream status` plus `stream disable`, `dashboard start` plus `dashboard stop`, and `chat <message>` (credential failure is acceptable evidence of wrapper pass-through when `AI_GATEWAY_API_KEY` is intentionally unset). Clean up any opened browser session with `close`, remove temporary files, and kill the tmux session before ending validation.
219
235
 
220
236
  This checklist assumes a real `agent-browser` on `PATH`. It complements, but does not overlap, `npm run verify -- lifecycle`: that harness swaps in a fake upstream binary and focuses on `/reload`, exact `--session-id` relaunch, managed-session continuity, spill-path persistence, and Pi `tool_result` failure-patch semantics (`scripts/verify-lifecycle.mjs`), not the full command matrix above.
221
237
 
@@ -241,7 +257,7 @@ Combine flags in one invocation when both apply (order after `lifecycle` is flex
241
257
  npm run verify -- lifecycle --model openai-codex/gpt-5.5:minimal --timeout-ms 600000
242
258
  ```
243
259
 
244
- On failure it retains transcripts/session artifacts; on success it performs best-effort cleanup. It does not replace occasional real-browser manual smoke testing.
260
+ It retains Pi transcripts and evidence, and removes disposable runtime inputs after success. It does not replace occasional real-browser manual smoke testing.
245
261
 
246
262
  **Lifecycle triage:** page checks require the first completed tool result to report a successful expected command and observed `data.url` (open) or `data.origin` (snapshot); recovery text and remembered targets are not page evidence. A cold post-quit `tab-drift` remains a continuity failure: upstream restores storage, and the wrapper must reopen the recorded URL after confirming shutdown before the first current-page read. Verify both empty storage and origin storage at non-root URLs and hash-routed URLs, including after `tab list` or explicit HTTP reads start the daemon and in batches with non-page prefixes, plus fresh refs and main-frame scope. A blank page, an origin root, or a URL mentioned only in an error is not the remembered page. Explicit URL QA recovery does not satisfy the same-page snapshot requirement; URL reopening does not recover unsaved forms, JavaScript memory, or history. A timeout on sentinel `v2` after exact-session relaunch means the new compiled entrypoint did not load after process restart. A reload-step timeout or missing post-reload snapshot often means Pi rejected reload while the TUI still showed `Working…` (`Wait for the current response to finish before reloading`), even when the session JSONL already has a final assistant message. Re-run with `--keep-artifacts --verbose`, inspect the retained pane capture, and confirm the configured model follows tool prompts reliably. Slower models may need a higher `--timeout-ms` than the **180000 ms** default.
247
263
 
@@ -263,7 +279,7 @@ Manual validation remains useful for release confidence and installed-package ch
263
279
  1. Configure exactly one active source for this extension in Pi settings: this checkout path before publishing, or the installed package after publishing.
264
280
  2. Launch plain `pi` so extension discovery is active.
265
281
  3. Validate managed-session continuity with `/reload` and a full restart plus exact `--session-id` relaunch or `/resume`.
266
- 4. Re-check local extension-side docs (`README.md`, `docs/COMMAND_REFERENCE.md`, `docs/TOOL_CONTRACT.md`, including the [`semanticAction`](TOOL_CONTRACT.md#semanticaction) rules when that shorthand or upstream `find` / `select` behavior changes) and regenerated prompt fragments from `extensions/agent-browser/lib/playbook.ts` via `npm run docs -- playbook check` or `npm run docs`. When the upstream `agent-browser` version or help surface changed, run `npm run verify -- command-reference`.
282
+ 4. Re-check local extension-side docs (`README.md`, `docs/COMMAND_REFERENCE.md`, `docs/TOOL_CONTRACT.md`, including the [`agent_browser_action`](TOOL_CONTRACT.md#agent_browser_action) rules when that shorthand or upstream `find` / `select` behavior changes) and regenerated prompt fragments from `extensions/agent-browser/lib/playbook.ts` via `npm run docs -- playbook check` or `npm run docs`. When the upstream `agent-browser` version or help surface changed, run `npm run verify -- command-reference`.
267
283
 
268
284
  ### Real upstream contract validation
269
285
 
@@ -292,7 +308,7 @@ The default unit suite also runs `agentBrowserExtension passes through core comm
292
308
 
293
309
  ### Native checkpoint qualification
294
310
 
295
- `test/agent-browser.checkpoint-native.test.mjs` is an opt-in, model-free test for a Pi SDK exposing `acquireCheckpoint` with native failed-journal repair/flush (qualified against core `c401cada69172dfd683e6d569b2cd9e5a45b4fcd`). It loads the compiled extension through native resource discovery, dispatches real tools through an awaited native command, and records real results with SessionManager. It checks idle/release/reload, live root and managed browsers (including a stable ambient socket override without a Pi override), off-branch state, detached and native-owned scripts, recording, failed journal appends/cleanup leases, routes, trace, caller-owned CDP attachment, and native checkpoint selection/entries plus root restore after replacing checkout/state directories with fresh inodes. No provider or fake readiness classifier is used.
311
+ `test/agent-browser.checkpoint-native.test.mjs` is an opt-in, model-free test for a Pi SDK exposing `acquireCheckpoint` with native failed-journal repair/flush (qualified against core `c401cada69172dfd683e6d569b2cd9e5a45b4fcd`). It loads the compiled extension through native resource discovery, dispatches real tools through an awaited native command, and records real results with SessionManager. It checks idle/release/reload, live root and managed browsers (including a stable ambient socket override without a Pi override), off-branch state, detached and native-owned code calls, recording, failed journal appends/cleanup leases, routes, trace, caller-owned CDP attachment, and native checkpoint selection/entries plus root restore after replacing checkout/state directories with fresh inodes. No provider or fake readiness classifier is used.
296
312
 
297
313
  Run as a non-root user in an isolated environment with stock `agent-browser`, Chrome for Testing and ffmpeg. Clear inherited credentials/configuration; do not use a personal browser. The test creates its own HOME/socket directory, copies only the supplied browser binary directory, serves a loopback synthetic page, explicitly closes its sessions, and removes its fixtures. `PI_CHECKPOINT_TEST_BROWSER_DIR` must name a binary installation directory such as `.../.agent-browser/browsers/chrome-<version>`, never a browser profile.
298
314
 
@@ -308,7 +324,7 @@ node --test test/agent-browser.checkpoint-native.test.mjs
308
324
 
309
325
  #### Required Linux PR gate
310
326
 
311
- [`.github/workflows/native-checkpoint.yml`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/native-checkpoint.yml) runs **Native checkpoint Linux** on every PR to `main`, pushes to `main`, and manual dispatch. Require this check in branch protection before merging checkpoint changes (repository settings are separate from the workflow). It does not replace release/manual-publication gates or full Linux personal-workspace sleep/resume qualification.
327
+ [`.github/workflows/native-checkpoint.yml`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/native-checkpoint.yml) runs **Native checkpoint Linux** on every PR to `main`, pushes to `main`, and manual dispatch. When Actions is authorized, this is the checkpoint PR check (repository settings are separate from the workflow). Under the current keep-Actions-disabled instruction, qualify locally and record the CI waiver rather than enabling workflows. It does not replace release/manual-publication gates or full Linux personal-workspace sleep/resume qualification.
312
328
 
313
329
  The CI-only recipe in [`scripts/ci-native-checkpoint.sh`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/scripts/ci-native-checkpoint.sh) builds the public fork target selected by `host-targets.json` in the workflow's pinned `fitchmultz/.github` automation revision, using the fork's lockfile and documented `npm ci --ignore-scripts`, `hydrate:model-data`, `build:offline` sequence. It uses explicit Node **26.9.0**, the extension's unchanged lockfile, stock **agent-browser 0.38.1**, a fresh stock Chrome-for-Testing download, and Ubuntu ffmpeg. No browser fork, personal profile, credentials, or model calls are involved.
314
330
 
@@ -385,8 +401,8 @@ Before publishing:
385
401
  - run `npm run doctor` and confirm any duplicate-source remediation matches the active package/checkout setup
386
402
  - run `npm run verify -- real-upstream` for upstream runtime, result-presentation, or managed-session changes
387
403
  - confirm both local-checkout modes still work for pre-release validation: isolated `pi --approve --no-extensions -e .` smoke testing for general trusted checkout loading (add `--no-skills` for extension-focused bounded smokes; omit `--approve` only to test the trust prompt) and configured-source lifecycle validation
388
- - complete interactive `tmux` live-site extension smoke with `pi --approve --no-extensions --no-skills -e .` and the native `agent_browser` tool (at least one simple static site and one real documentation/product site; include `qa` or `job`/`batch` when those surfaces changed; use the [public Grafana stress checklist](#public-grafana-stress-checklist) when dashboard/diagnostic/artifact behavior changed; close sessions and remove screenshots/temp artifacts; record evidence). Run separate skill-enabled dogfood only when validating skill routing/report-generation behavior—see [Pre-release checks](#pre-release-checks); automated gates are not a substitute
389
- - rerun `npm run verify -- release` and confirm the embedded Crabbox `platform-build` plus `browser-dogfood-smoke` matrix passed on `macos`, `ubuntu`, and `windows-native` with artifacts under `.artifacts/platform-smoke/`
404
+ - complete interactive `tmux` live-site extension smoke with `pi --approve --no-extensions --no-skills -e .` and the native `agent_browser` tool (at least one simple static site and one real documentation/product site; include advanced QA, code, or native batch when those surfaces changed; use the [public Grafana stress checklist](#public-grafana-stress-checklist) when dashboard/diagnostic/artifact behavior changed; close sessions and remove screenshots/temp artifacts; record evidence). Run separate skill-enabled dogfood only when validating skill routing/report-generation behavior—see [Pre-release checks](#pre-release-checks); automated gates are not a substitute
405
+ - satisfy the approved release checks and current owner waivers; record individual platform evidence and never label a Windows-inclusive composition passed when Windows was waived
390
406
  - run `npm run verify -- lifecycle` for configured-source `/reload`, exact `--session-id` relaunch, managed-session continuity, persisted-spill, and Pi failure-patch regression coverage (required before publish; see [Pre-release checks](#pre-release-checks))
391
407
  - confirm [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md) still maps every current baseline inventory section to docs, runtime handling, tests, and validation status
392
408
  - manually exercise real-browser `/reload` and full restart plus exact `--session-id` relaunch or `/resume` continuity when release risk warrants browser-level confidence beyond the fake upstream harness
@@ -62,16 +62,20 @@ Define the product requirements and constraints for `pi-agent-browser-native`.
62
62
  - Keep the handling simple and global-install-friendly.
63
63
  - Do not rely on package skill overrides as the primary answer.
64
64
 
65
- ### Native `agent_browser` inputs
66
-
67
- - Each tool invocation must supply **exactly one** of: top-level `script` (bounded one-shot JavaScript orchestration with `browser()` / `emit()` in a unique always-closed isolated session), `args` (full upstream argv after the binary name), top-level `semanticAction` (a small intent object compiled into existing upstream `find` argv for locator actions, direct selector/ref `click` / `check` / `fill` argv, or upstream `select <selector> <value...>` argv for native dropdown selection), `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron` (bounded desktop lifecycle: host `list`, wrapper-owned isolated `launch` with CDP attach, `status`, compact `probe`, and `cleanup`; mutually exclusive with caller `stdin`). Supplying multiple modes or none is rejected before launch (`extensions/agent-browser/index.ts`, `test/agent-browser.extension-validation.test.ts`). Contract and field rules: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#input-mode-chooser); operator workflow: [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-mental-model).
68
- - Direct upstream capability is the default: explicit sessions, state/restore paths, profiles, config, file access, launch arguments, environment variables, local pages, output paths, and close arguments pass through unchanged. Session/state lists keep all upstream rows and restore identifiers. Wrapper policy must not decide which valid upstream resources the agent or operator may use; validation is limited to input shape, process/lifecycle integrity, and truthful page/action evidence.
69
- - `script` is for one-shot loops, conditional page branches, or multi-page aggregation only. It must run in a separate permissioned child with no user-visible host objects/functions or host filesystem/network/process/import access; serialize a maximum of 25 inner calls through the complete ordinary native-tool executor; cap source, every complete inner response envelope, bounded inner summary/text, cumulative IPC, compact post-redaction output, and a 120-second default/300-second maximum deadline; inject a unique restore-disabled wrapper-owned session without touching the implicit conversation session; clear ambient upstream launch/proxy controls for every helper and cleanup subprocess; reject identity/lifecycle/attachment/local/persistent-launch/nested-mode controls; expose only script-policy-compatible inner next actions after removing the wrapper-owned isolated identity prefix; append an exact model-invisible persisted cleanup lease before first browser spawn; close in `finally`; abort and await cleanup on branch change/shutdown; and retry exact failed active-branch leases after restart. Pi `--no-session` must fail this mode before launch because durable cleanup recovery is unavailable. One approved top-level input may issue all 25 inner calls, so its custom Pi call renderer must show a bounded terminal-safe preview with visible line-break markers when collapsed and the full terminal-safe source when expanded, preserving JavaScript line terminators as visible newlines and visibly marking removed controls. This is not a reusable named recipe runtime and must not gain script names, a registry, imports, shared state, or workflow versioning without a separate design pass.
70
- - `semanticAction` is not a nested shape inside `batch` stdin; batch steps remain upstream argv string arrays, including `find` steps expressed as token lists.
71
- - Supported actions, locators, exclusivity rules, when `details.compiledSemanticAction` appears, and bounded `try-*-candidate` follow-ups on `selector-not-found` (specific action/locator pairs only; see contract) are specified in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#semanticaction), with workflow examples in [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md).
72
- - Constrained `job` remains a thin batch compiler, but its `click`/`fill` steps may use the same semantic locator fields as `semanticAction` so short workflows can avoid brittle selectors without adding a reusable recipe runtime, and `type` steps may expand to a bounded set of existing upstream focus/keyboard/wait/press rows for human-paced input while compacting model-visible batch text. `job` must default to fail-fast (`batch --bail`) so later mutating steps do not run after an earlier required step fails; `failFast: false` is the explicit opt-out.
73
- - `qa` must fail fast on failed readiness/text/selector assertions so missing expected text cannot burn the wrapper watchdog before reporting `qa-failure`. `qa.attached` must never erase existing session diagnostics; URL-opening `qa` may clear buffers to scope a fresh page load. Attached QA preserves buffers, reports that scope in `details.compiledQaPreset.checks.diagnosticsResetAtStart`, and defaults diagnostic checks off unless the caller opts into preserved-buffer failure checks.
74
- - Direct artifact workflows must create missing parent directories before spawning upstream and must verify saved files before downstream use. Native download must own the click and download, including loopback links, generated Blob exports, and redirects; the wrapper must not substitute an anchor-href fetch. `outputPath` may write a successful result payload to a caller-requested local file and must report `details.outputFile`; it must never overwrite a screenshot, download, recording, or other browser artifact from the same result, including through a filesystem alias. Missing non-pending artifacts, including diff screenshot outputs, must never use saved/verified wording; `record start` future files are pending/open until `record stop`, not missing.
65
+ ### Browser tool contract
66
+
67
+ - Keep the always-on interface compact: `agent_browser { args, stdin?, outputPath?, timeoutMs?, sessionMode? }`, strict-JSON `agent_browser_code { code, session?, namespace?, timeoutMs?, outputPath? }`, and `agent_browser_tools { enable? }`.
68
+ - Preserve the complete native command surface through one CLI executor. Native batch handles fixed sequences; code handles branches/loops/aggregation. Remove the redundant public job DSL and old isolated-script execution instead of carrying parallel legacy runtimes.
69
+ - Register advanced action, QA, Electron, UI-source, and network-source tools once, initially inactive. Enable additively through Pi's native tool selection/history without removing unrelated tools. Use flat schemas and return executable next actions naming the actual tool.
70
+ - Preserve unique visible accessible dropdown selection, semantic/ref recovery, truthful QA verdicts (including ambiguous error-buffer residue), attached diagnostic preservation/defaults, Electron process/profile ownership, and bounded source scans. Raw diagnostic-read success is not a QA verdict, and source hints remain candidates rather than authoritative blame.
71
+ - Code runs in a fresh permissioned Node child with no host APIs/imports, null-prototype task globals, JSON-only bounded IPC, serial native calls, and explicit JSON/image output. Keep current source/output/call/deadline limits. Browser/authentication persists under existing native ownership and configuration; JavaScript variables do not. Allow ordinary native commands including batch/auth/profile/state/connect/close, but prevent changes to the fixed parent-selected identity and namespace-wide close within a cell.
72
+ - Hold a cooperating-process execution lock keyed by actual socket context, canonical namespace, and session across direct helper/action work or the entire code cell. Use ordered multi-identity acquisition for replacement/cleanup and a namespace barrier for direct close-all. This does not coordinate humans, third-party clients, or older versions, guarantee atomic browser actions, or provide rollback.
73
+ - Persist ordered inner intent/completion transitions through Pi custom entries and existing browser-state reducers. Interrupted outcomes remain unknown until inspection. Require Pi persistence for code; retain exact old script lease readers only for upgrade cleanup, not public execution.
74
+ - Use one canonical model-visible observation for JSON/prose/code with success/data/error, exact recovery, artifact receipts, and image observations. Details is audit/replay metadata, not the sole source of actionable failures. Inner calls avoid full model rendering/spills/image encoding. Bound outer output and name a complete redacted observation spill rather than truncating executable payloads.
75
+ - Attach only verified real images; code selects current-cell image handles explicitly. Report pixel dimensions and CSS viewport/DPR/crop/scroll/frame evidence conservatively. Unknown geometry remains unknown; neither full-page nor element captures are assumed viewport coordinates. Do not inject model-specific image-detail/provider payload changes.
76
+ - Direct upstream capability remains available: explicit sessions, state/restore paths, profiles, config, file access, launch arguments, environment, local pages, output paths, and close arguments pass through under ordinary native semantics. Keep all session/state rows visible. Wrapper validation concerns input shape, lifecycle integrity, and truthful evidence, not authorization of resources.
77
+ - Native download owns clicking and saving, including loopback, Blob exports, and redirects. Create artifact directories before dispatch and verify saved files; output-data writes must not alias browser artifacts. Pending recording is not saved video.
78
+ - Maintain a deliberate 0.7 migration map and package-level rollback that preserves saved authentication, explicit artifacts, and Pi transcripts. Restart every cooperating Pi participant on the updated package before relying on shared ordering. No universal performance, sandbox, authentication-persistence, or Windows qualification claims.
75
79
 
76
80
  ### Documentation standard
77
81
 
@@ -109,13 +113,13 @@ The design should comfortably support workflows such as:
109
113
  - headless authenticated `chat.com` / ChatGPT / OpenAI and Cloudflare Dashboard browsing without forcing `--headed` or `--auto-connect`
110
114
  - upstream profile/debug workflows without adding a local profile-cloning layer in this package
111
115
  - provider-backed or iOS device launches where upstream owns credentials, env, and setup; the wrapper forwards argv and the parent environment without emulating those backends
112
- - desktop Electron targets using top-level `electron` for discover → isolated launch → attach → probe/cleanup, or raw `args: ["connect", …]` when the operator launches the real app with a debug port for signed-in state (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron) and [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#electron-desktop-apps))
113
- - one-shot public-page loops, conditional banner/dialog branches, and bounded multi-page extraction/aggregation through `script`, returning one compact emitted JSON value without exposing or persisting a reusable recipe (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#script) and [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#one-shot-code-mode))
116
+ - desktop Electron targets using `agent_browser_electron` for discover → isolated launch → attach → probe/cleanup, or raw `args: ["connect", …]` when the operator launches the real app with a debug port for signed-in state (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#agent_browser_electron) and [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#electron-desktop-apps))
117
+ - authenticated or public-page loops, branches, and bounded multi-page extraction through `agent_browser_code`, returning selected JSON/images while retaining the browser and resetting JavaScript variables
114
118
 
115
119
  ## Implications for the implementation
116
120
 
117
121
  - Package-manifest behavior matters more than repo-local development wiring.
118
- - The extension should use official `pi` hooks and package resources where possible.
122
+ - The extension uses native Pi tool registration/activation and named prompt sections rather than replacing the full system prompt. Qualify official Pi 0.87 and the fork independently; fork checkpoint hooks stay optional and hosted provider orchestration is not required.
119
123
  - The wrapper should stay thin, with upstream `agent-browser` remaining the source of truth for command semantics.
120
124
  - Successful and failed tool outcomes should surface bounded machine-readable fields on Pi-facing `details` (`resultCategory`, `successCategory`, `failureCategory`, optional structured `nextActions`, optional `pageChangeSummary` with explicit observed-vs-dispatched evidence and per-step summaries on `batch`, optional `artifactVerification` with the same shape on successful `batchSteps[]` rows, optional `outputFile`, optional `timeoutPartialProgress`) so agents can branch without parsing prose; browser-bearing `nextActions` must preserve a known session identity so recovery cannot inspect an unrelated implicit session; stateful commands (`auth`, `cookies`, `storage`, `dialog`, `frame`, `state`) plus other structured diagnostics (for example `network`, `diff`, `trace`, `stream`, `dashboard`, `chat`) and `batch` should redact secret-bearing payloads in model-facing `details.data`, including the compact per-step `batch` roll-up on the parent result (full per-step payloads live on `batchSteps[]`). Dialog/prompt-related timeouts should be bounded with recovery `nextActions`; non-dialog timeouts should prefer best-effort per-step progress and retry payloads when a plan is available; no-op scrolls should expose no-movement state instead of only an upstream success boolean; explicit page/container scroll helpers should expose before/after movement evidence. The contract lives in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), enums and classifier precedence live in `extensions/agent-browser/lib/results/categories.ts` and `contracts.ts`, and presentation-time summaries, redaction, network request follow-ups, and artifact verification rollups are assembled in `extensions/agent-browser/lib/results/presentation.ts` (`buildPageChangeSummary`, command taxonomy predicates from `command-taxonomy.ts`, `redactPresentationData`, `buildArtifactVerificationSummary`, `buildBatchPresentation`).
121
125
  - User-facing docs belong in `README.md` and the canonical published files under `docs/`.
@@ -132,4 +136,4 @@ The design should comfortably support workflows such as:
132
136
  - Treat argv-supplied `--allowed-domains` as launch-scoped so it starts in a fresh browser context. Keep upstream responsible for request/worker/popup/WebRTC containment and incompatible launch-mode rejection; pass its result through unchanged.
133
137
  - Upstream restore-state periodic autosave remains upstream-owned. Forward an explicit `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` unchanged when a daemon launches; otherwise default it to `0` for wrapper-owned headed launches, persist the effective launch-time interval, and retain that environment across helpers, follow-ups, still-owned off-current sessions, Electron cleanup closes, and transcript reload/resume to avoid upstream 0.33.2's visible temporary collector tabs and per-call daemon-configuration drift. Reject attempts to change that interval on an already-running wrapper-owned headed daemon until the caller closes it and launches fresh. If automatic cleanup of a replaced wrapper-owned session fails, persist that outcome and restore the older identity's ownership from the transcript so explicit follow-up and cleanup remain possible. Document the 30-second upstream default, native-close preservation, and the direct-window-close loss risk because headed browsers are exempt from idle shutdown; do not duplicate its timer or claim its restore files as wrapper artifacts.
134
138
  - Read-only upstream `skills list`, `skills get …`, and `skills path …` must stay free of implicit managed `--session` under default `sessionMode: "auto"` (still with `--json`), matching plain-text `--help` / `--version` inspection semantics so bundled skill text does not pin or rotate the active browser session; new `skills` subcommands pick up that behavior only after allowlisting in `extensions/agent-browser/lib/runtime.ts` with regression coverage.
135
- - Optional `semanticAction.session` on native `agent_browser` must compile to a leading `--session <name>` pair before upstream `find` or `select` argv so the shorthand can target a named upstream browser without hand-built `args`, while `buildExecutionPlan` still skips double-injecting the extension-managed implicit session whenever planned argv already starts with `--session`; stale-ref retries for compiled `find` actions and bounded `try-*` candidate `nextActions` must preserve that same prefix. Contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#semanticaction) / [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sessionmode); implementation in `extensions/agent-browser/index.ts` and `extensions/agent-browser/lib/runtime.ts`.
139
+ - Optional `session` on `agent_browser_action` must compile to a leading `--session <name>` pair before upstream `find` or `select` argv so the shorthand can target a named upstream browser without hand-built `args`, while `buildExecutionPlan` still skips double-injecting the extension-managed implicit session whenever planned argv already starts with `--session`; stale-ref retries for compiled `find` actions and bounded `try-*` candidate `nextActions` must preserve that same prefix. Contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#agent_browser_action) / [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sessionmode); implementation in `extensions/agent-browser/index.ts` and `extensions/agent-browser/lib/runtime.ts`.