pi-agent-browser-native 0.6.17 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +55 -167
- package/dist/extensions/agent-browser/index.js +317 -223
- package/dist/extensions/agent-browser/lib/browser-transcript.js +33 -0
- package/dist/extensions/agent-browser/lib/execution-cwd.js +4 -2
- package/dist/extensions/agent-browser/lib/input-modes/job.js +2 -213
- package/dist/extensions/agent-browser/lib/input-modes/params.js +146 -142
- package/dist/extensions/agent-browser/lib/input-modes/script.js +64 -92
- package/dist/extensions/agent-browser/lib/managed-session-policy-lock.js +170 -24
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostics.js +13 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result.js +26 -13
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +21 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +4 -2
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +1 -3
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +35 -20
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery.js +3 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-observation.js +79 -0
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +3 -4
- package/dist/extensions/agent-browser/lib/orchestration/input-plan.js +19 -52
- package/dist/extensions/agent-browser/lib/orchestration/native-session-defaults.js +3 -3
- package/dist/extensions/agent-browser/lib/orchestration/output-file.js +3 -3
- package/dist/extensions/agent-browser/lib/orchestration/script-mode.js +91 -278
- package/dist/extensions/agent-browser/lib/pi-tool-rendering.js +9 -9
- package/dist/extensions/agent-browser/lib/playbook.js +46 -25
- package/dist/extensions/agent-browser/lib/process-identity.js +22 -6
- package/dist/extensions/agent-browser/lib/process.js +16 -13
- package/dist/extensions/agent-browser/lib/results/action-recommendations.js +2 -2
- package/dist/extensions/agent-browser/lib/results/next-actions.js +1 -2
- package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +25 -4
- package/dist/extensions/agent-browser/lib/results/presentation/batch.js +6 -2
- package/dist/extensions/agent-browser/lib/results/presentation/common.js +2 -0
- package/dist/extensions/agent-browser/lib/results/presentation/content.js +39 -0
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +3 -3
- package/dist/extensions/agent-browser/lib/results/presentation/large-output.js +41 -1
- package/dist/extensions/agent-browser/lib/results/presentation.js +16 -11
- package/dist/extensions/agent-browser/lib/runtime.js +18 -6
- package/dist/extensions/agent-browser/lib/session-page-state.js +3 -4
- package/dist/extensions/agent-browser/lib/tool-surface.js +131 -0
- package/dist/extensions/agent-browser/script-worker.js +4 -1
- package/docs/ARCHITECTURE.md +46 -62
- package/docs/COMMAND_REFERENCE.md +77 -136
- package/docs/ELECTRON.md +51 -49
- package/docs/RELEASE.md +31 -15
- package/docs/REQUIREMENTS.md +18 -14
- package/docs/SUPPORT_MATRIX.md +33 -15
- package/docs/TOOL_CONTRACT.md +216 -405
- package/package.json +1 -1
- package/scripts/doctor.mjs +2 -2
package/docs/ELECTRON.md
CHANGED
|
@@ -10,9 +10,9 @@ Related docs:
|
|
|
10
10
|
|
|
11
11
|
## Purpose
|
|
12
12
|
|
|
13
|
-
This guide is the entry point for using `pi-agent-browser-native` against desktop **Electron** applications. The wrapper exposes
|
|
13
|
+
This guide is the entry point for using `pi-agent-browser-native` against desktop **Electron** applications. The wrapper exposes `agent_browser_electron` with flat action fields that owns the awkward discover → launch → attach → probe → cleanup sequence so agents do not hand-build `--remote-debugging-port` argv, poll `DevToolsActivePort`, and `kill` profile directories. After attach, the rest of the native `agent_browser` surface (`snapshot`, `find`, `click`, `fill`, `get`, `eval --stdin`, `batch`, persistent browser code, and advanced attached QA) works the same way it does against a web page.
|
|
14
14
|
|
|
15
|
-
This document is structured for users, not implementers. Field-level rules live in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#
|
|
15
|
+
This document is structured for users, not implementers. Field-level rules live in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#agent_browser_electron); this guide focuses on **when** and **how** to use them, and on the safety and ownership boundary the wrapper enforces.
|
|
16
16
|
|
|
17
17
|
## Who this is for
|
|
18
18
|
|
|
@@ -20,10 +20,12 @@ This document is structured for users, not implementers. Field-level rules live
|
|
|
20
20
|
- **Coding agents** that need a low-context lifecycle for desktop apps such as VS Code, Cursor, Obsidian, Slack, or any app built on Electron, without re-implementing the CDP attach dance every session.
|
|
21
21
|
- **Maintainers and reviewers** validating the wrapper's Electron behavior before release; verification evidence lives under `RQ-0096` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md).
|
|
22
22
|
|
|
23
|
-
It is **not** an upstream `agent-browser` reference and it does **not** replace the canonical [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#
|
|
23
|
+
It is **not** an upstream `agent-browser` reference and it does **not** replace the canonical [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#agent_browser_electron) for exact field semantics, validation rules, or failure categories.
|
|
24
24
|
|
|
25
25
|
## Mental model
|
|
26
26
|
|
|
27
|
+
Enable the tools with `agent_browser_tools { "enable": ["electron", "qa"] }`. Below, `electron.launch` and similar labels mean `agent_browser_electron { "action": "launch", ... }`; `qa.attached` means `agent_browser_qa { "attached": true, ... }`. These are separate tools, not nested inputs on `agent_browser`.
|
|
28
|
+
|
|
27
29
|
```
|
|
28
30
|
electron.list → discover Electron apps (host-only; no upstream spawn)
|
|
29
31
|
electron.launch → launch a wrapper-owned isolated app, attach via CDP, hand off (snapshot|tabs|connect)
|
|
@@ -42,14 +44,14 @@ Choosing between the two is a real decision, not a stylistic one. See [Wrapper-o
|
|
|
42
44
|
|
|
43
45
|
## Quick start
|
|
44
46
|
|
|
45
|
-
|
|
47
|
+
Use `agent_browser_electron` for the lifecycle rows below and `agent_browser` for the native snapshot: discover, launch with the default snapshot handoff, inspect, then clean up.
|
|
46
48
|
|
|
47
49
|
```json
|
|
48
|
-
{ "
|
|
49
|
-
{ "
|
|
50
|
+
{ "action": "list", "query": "code" }
|
|
51
|
+
{ "action": "launch", "appName": "Visual Studio Code", "handoff": "snapshot" }
|
|
50
52
|
{ "args": ["snapshot", "-i"] }
|
|
51
|
-
{ "
|
|
52
|
-
{ "
|
|
53
|
+
{ "action": "probe", "timeoutMs": 5000 }
|
|
54
|
+
{ "action": "cleanup", "launchId": "electron-…" }
|
|
53
55
|
```
|
|
54
56
|
|
|
55
57
|
The launch result carries both a `launchId` (used by `status`/`probe`/`cleanup`) and an attached `sessionName` (used by browser-style `snapshot`/`tab`/`click`/`find` calls). Read both from `details.electron.launch` and `details.electron.identifiers`. With default implicit session reuse, the quick-start `args: ["snapshot", "-i"]` line uses that attached session without an extra `--session` argument; pass `--session` explicitly when you target a named upstream session instead.
|
|
@@ -57,7 +59,7 @@ The launch result carries both a `launchId` (used by `status`/`probe`/`cleanup`)
|
|
|
57
59
|
For a quick "is the app actually showing what we expect?" smoke check after attach:
|
|
58
60
|
|
|
59
61
|
```json
|
|
60
|
-
{ "
|
|
62
|
+
{ "attached": true, "expectedText": "Explorer", "screenshotPath": ".dogfood/electron.png" }
|
|
61
63
|
```
|
|
62
64
|
|
|
63
65
|
`qa.attached` runs against the **current managed session** without opening a URL, so it works for any attached app — wrapper-owned or manually launched.
|
|
@@ -92,7 +94,7 @@ Then attach and choose a ready target before using refs:
|
|
|
92
94
|
{ "args": ["tab", "list"] }
|
|
93
95
|
{ "args": ["tab", "t2"] }
|
|
94
96
|
{ "args": ["snapshot", "-i"] }
|
|
95
|
-
{ "
|
|
97
|
+
{ "attached": true, "expectedText": "Channels" }
|
|
96
98
|
```
|
|
97
99
|
|
|
98
100
|
A successful `connect` means the CDP endpoint accepted the session; it does **not** prove the app has an active rendered page yet. Prefer `details.nextActions` when present: `verify-connected-session-url` performs the only page read allowed while the attached target is unverified, and `list-connected-session-tabs` inspects attached targets. A verified target clears the guard; otherwise navigate explicitly or select the intended tab. After the read-only list, select or confirm a stable `t<N>` target, verify it with `get url`, and run `snapshot -i` explicitly before trusting refs. If the first `snapshot -i` says `No active page`, follow `list-tabs-after-no-active-page`. If it returns no useful refs without that error, manually run `tab list`, select a stable `t<N>` id for the app surface, then retry a condition wait or `snapshot -i` on that selected target.
|
|
@@ -114,14 +116,14 @@ This project is not adding a first-class host-idle primitive yet. Revisit that o
|
|
|
114
116
|
|
|
115
117
|
## Action reference
|
|
116
118
|
|
|
117
|
-
The exact field schemas, validation rules, and `details.*` payload shapes live in [`TOOL_CONTRACT.md#
|
|
119
|
+
The exact field schemas, validation rules, and `details.*` payload shapes live in [`TOOL_CONTRACT.md#agent_browser_electron`](TOOL_CONTRACT.md#agent_browser_electron). This section is a usage-oriented overview.
|
|
118
120
|
|
|
119
121
|
### `electron.list` — discover apps
|
|
120
122
|
|
|
121
123
|
Host-only scan; does not spawn upstream `agent-browser`. macOS (`/Applications/*.app`, `~/Applications/*.app`) and Linux (`.desktop` launchers under standard XDG, Flatpak, and Snap locations) are supported in v1. On Windows (and any non-macOS/non-Linux host), `list` returns `details.electron.platform: "unsupported"` with an empty `apps` array—use `executablePath` (or a host `appPath` that resolves to a verifiable Electron binary) for `launch` instead; `inspectElectronExecutablePath` in `extensions/agent-browser/lib/electron/discovery.ts` still gates Windows executables before spawn.
|
|
122
124
|
|
|
123
125
|
```json
|
|
124
|
-
{ "
|
|
126
|
+
{ "action": "list", "query": "code", "maxResults": 25 }
|
|
125
127
|
```
|
|
126
128
|
|
|
127
129
|
Returns app metadata under `details.electron.apps`: `name`, optional `bundleId`/`desktopId`, `appPath`, `executablePath`, `platform`, and optional non-blocking `sensitivity` annotations. Apps flagged as likely sensitive (categories such as `notes`, `chat`, `mail`, `developer-workspace`, or `passwords-auth`) are printed with `[likely sensitive: …]`. These are **advisory hints**, not enforcement; see [Safety and ownership](#safety-and-ownership) for the policy boundary.
|
|
@@ -132,14 +134,12 @@ Pass **exactly one** target: `appPath`, `appName`, `bundleId`, or `executablePat
|
|
|
132
134
|
|
|
133
135
|
```json
|
|
134
136
|
{
|
|
135
|
-
"
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
"appArgs": ["--disable-telemetry"]
|
|
142
|
-
}
|
|
137
|
+
"action": "launch",
|
|
138
|
+
"appName": "Visual Studio Code",
|
|
139
|
+
"handoff": "snapshot",
|
|
140
|
+
"targetType": "page",
|
|
141
|
+
"timeoutMs": 30000,
|
|
142
|
+
"appArgs": ["--disable-telemetry"]
|
|
143
143
|
}
|
|
144
144
|
```
|
|
145
145
|
|
|
@@ -162,9 +162,9 @@ Wrapper-owned launches **always** use an isolated temp profile and an OS-chosen
|
|
|
162
162
|
Read-only inspection of one or more tracked launches. Without `launchId` or `all`, it selects the single active wrapper launch when unambiguous.
|
|
163
163
|
|
|
164
164
|
```json
|
|
165
|
-
{ "
|
|
166
|
-
{ "
|
|
167
|
-
{ "
|
|
165
|
+
{ "action": "status" }
|
|
166
|
+
{ "action": "status", "launchId": "electron-…" }
|
|
167
|
+
{ "action": "status", "all": true }
|
|
168
168
|
```
|
|
169
169
|
|
|
170
170
|
Reports `cleanupState`, current debug-port and PID liveness, bounded CDP targets, and freshly measured `userDataDirState` under `details.electron.statuses`. An explicit `launchId` can inspect a **historical cleaned launch record**; default and `all: true` selection exclude cleaned records. Cleanup history does not determine current liveness. The tracked profile path is `present`, `absent` (only native `lstat` ENOENT), or `unknown` (other filesystem errors); a dangling symlink is present. This measures that path only, not all app residue, and is not stored in the launch record. Its managed-session title/URL reads hold the normal daemon-policy lock and owned restore context. Mismatch fields surface when the current managed session or tab no longer matches a live wrapper launch target — typically the cue to follow `reattach-electron-launch` before trusting old refs.
|
|
@@ -174,8 +174,8 @@ Reports `cleanupState`, current debug-port and PID liveness, bounded CDP targets
|
|
|
174
174
|
`probe` collapses what would otherwise be separate `get url` / `get title` / focused-element `eval` / `tab list` / `snapshot -i` calls into one bounded result. Use it instead of chaining those reads when you just need a quick "where are we?" check. The wrapper holds the managed-session daemon-policy lock for the probe and runs every underlying read with the session's owned restore decision, so probing cannot restart the daemon under a different restore key.
|
|
175
175
|
|
|
176
176
|
```json
|
|
177
|
-
{ "
|
|
178
|
-
{ "
|
|
177
|
+
{ "action": "probe" }
|
|
178
|
+
{ "action": "probe", "launchId": "electron-…", "timeoutMs": 5000 }
|
|
179
179
|
```
|
|
180
180
|
|
|
181
181
|
Output appears under `details.electron.probe`: `title`, `url`, `focusedElement`, `activeTab`, `tabs`, compact `snapshot` metadata (`refCount`, `refIds`, optional text preview and omission counts), and `errors`. If every underlying read fails, the tool fails with `failureCategory: "upstream-error"`; it does not report a successful empty partial probe. Probes reject an unverified target before helper reads, then verify the live URL again before title, eval, tab, or snapshot helpers so external target drift cannot expose the wrong page. A current-managed probe also persists top-level `details.namespace`, `sessionTabTarget`, and `refSnapshot` so Pi reload/branch replay keeps the same namespaced page identity; unverified transitions persist as `details.sessionTabTargetUnknown: true` until `get url` or explicit navigation establishes a trustworthy target. When `launchId` is given, the probe is tied to that tracked launch and will surface mismatch guidance if the wrapper sees a session or target drift; visible output also includes debug-port/pid liveness so a stale `about:blank` against a dead launch is unmistakable.
|
|
@@ -189,8 +189,8 @@ Closes the tracked managed session, stops only the wrapper-tracked process, veri
|
|
|
189
189
|
Cleanup partial failures fail the tool result with `failureCategory: "cleanup-failed"` and the `retry-electron-cleanup` next action references the same `launchId` so retries are bounded.
|
|
190
190
|
|
|
191
191
|
```json
|
|
192
|
-
{ "
|
|
193
|
-
{ "
|
|
192
|
+
{ "action": "cleanup", "launchId": "electron-…" }
|
|
193
|
+
{ "action": "cleanup", "all": true }
|
|
194
194
|
```
|
|
195
195
|
|
|
196
196
|
`electron.cleanup` **never** targets:
|
|
@@ -206,7 +206,7 @@ On Pi `quit`, active wrapper-owned Electron launches are best-effort cleaned. On
|
|
|
206
206
|
|
|
207
207
|
### `timeoutMs` by action (quick reference)
|
|
208
208
|
|
|
209
|
-
`
|
|
209
|
+
`agent_browser_electron` with `action: "list"` has no `timeoutMs` field. For every other action, flat `timeoutMs` applies to **different surfaces**, not an end-to-end action deadline. Authoritative rules and env overrides live in the Electron contract in [`TOOL_CONTRACT.md#agent_browser_electron`](TOOL_CONTRACT.md#agent_browser_electron).
|
|
210
210
|
|
|
211
211
|
| Action | What `timeoutMs` covers when set | Typical default when omitted |
|
|
212
212
|
| --- | --- | --- |
|
|
@@ -217,28 +217,26 @@ On Pi `quit`, active wrapper-owned Electron launches are best-effort cleaned. On
|
|
|
217
217
|
|
|
218
218
|
## `qa.attached` — current-session smoke check
|
|
219
219
|
|
|
220
|
-
`
|
|
220
|
+
`agent_browser_qa` has two flat forms: `{ url, … }` and `{ attached: true, … }`. The attached form is the right tool for Electron smoke checks after either launch path because it does not open a URL and runs all checks against the current managed session.
|
|
221
221
|
|
|
222
222
|
```json
|
|
223
223
|
{
|
|
224
|
-
"
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
"screenshotPath": ".dogfood/electron.png"
|
|
231
|
-
}
|
|
224
|
+
"attached": true,
|
|
225
|
+
"expectedText": "Explorer",
|
|
226
|
+
"expectedSelector": "@e1",
|
|
227
|
+
"checkConsole": true,
|
|
228
|
+
"checkErrors": true,
|
|
229
|
+
"screenshotPath": ".dogfood/electron.png"
|
|
232
230
|
}
|
|
233
231
|
```
|
|
234
232
|
|
|
235
|
-
`qa.attached` rejects `url` and is incompatible with `sessionMode: "fresh"` — attach first with `electron.launch` or raw `connect`, then run `qa.attached`. Preserved-buffer diagnostics (`checkConsole`, `checkErrors`, `checkNetwork`) default to `false` for attached QA; opt into them when you want historical session buffers to fail the smoke. The full field rules and pass/fail classification live in [`TOOL_CONTRACT.md#
|
|
233
|
+
`qa.attached` rejects `url` and is incompatible with `sessionMode: "fresh"` — attach first with `electron.launch` or raw `connect`, then run `qa.attached`. Preserved-buffer diagnostics (`checkConsole`, `checkErrors`, `checkNetwork`) default to `false` for attached QA; opt into them when you want historical session buffers to fail the smoke. The full field rules and pass/fail classification live in [`TOOL_CONTRACT.md#agent_browser_qa`](TOOL_CONTRACT.md#agent_browser_qa).
|
|
236
234
|
|
|
237
235
|
In attached Electron sessions, broad selectors such as `body`, `html`, `main`, or `[role=application]` can read the entire app shell. When `get text <selector>` looks too broad, the wrapper may attach `details.electronGetTextScopeWarning` and a `snapshot-for-electron-text-scope` next action; prefer a fresh `snapshot -i`, a current `@ref`, or a narrower panel selector.
|
|
238
236
|
|
|
239
|
-
##
|
|
237
|
+
## Source lookup against packaged Electron apps
|
|
240
238
|
|
|
241
|
-
`
|
|
239
|
+
Enable `source` and use `agent_browser_source` for experimental hints about the source file/component behind a visible element. It is **opt-in** and **evidence-based**: it reports confidence and evidence rather than claiming a guaranteed mapping. The same experimental helper works against packaged Electron apps, but with two important boundaries:
|
|
242
240
|
|
|
243
241
|
1. **Scope of the workspace scan.** `sourceLookup` walks the Pi session **cwd** (default `maxWorkspaceFiles: 2000`, hard cap 5000). It does **not** unpack `app.asar` or installed app resources. For packaged apps where the source lives inside `Contents/Resources/app.asar`, the workspace-search lane will commonly return no candidates.
|
|
244
242
|
2. **React DevTools requirement.** `react inspect <id>` requires the session to have been launched with `--enable react-devtools` before first navigation. For Electron, the wrapper's `electron.launch` path does **not** inject `--enable react-devtools` into the Electron process; that flag belongs to upstream `agent-browser` Chromium launches. If the Electron app does not already expose a React DevTools backend, expect `react inspect` to fail; DOM-attribute and workspace-search candidates may still surface.
|
|
@@ -246,10 +244,10 @@ In attached Electron sessions, broad selectors such as `body`, `html`, `main`, o
|
|
|
246
244
|
For wrapper-tracked packaged Electron sessions where `status` is `no-candidates`, the wrapper attaches `workspaceRoot` plus optional `electronContext` (`launchId?`, `appName?`, `appPath?`, `executablePath?`, `sessionName?`, `url?`) and limitations explaining the bundle/asar boundary, plus `snapshot-electron-session`, `probe-electron-launch`, and `list-electron-tabs` next actions so you can inspect the live app and decide whether to widen the workspace or pull source out-of-band before re-running the lookup.
|
|
247
245
|
|
|
248
246
|
```json
|
|
249
|
-
{ "
|
|
247
|
+
{ "selector": "#save", "reactFiberId": "2", "componentName": "SaveButton" }
|
|
250
248
|
```
|
|
251
249
|
|
|
252
|
-
Treat `sourceLookup` output as a starting point for navigation, not a substitute for reading code. Full contract: [`TOOL_CONTRACT.md#
|
|
250
|
+
Treat `sourceLookup` output as a starting point for navigation, not a substitute for reading code. Full contract: [`TOOL_CONTRACT.md#agent_browser_source`](TOOL_CONTRACT.md#agent_browser_source).
|
|
253
251
|
|
|
254
252
|
## Safety and ownership
|
|
255
253
|
|
|
@@ -277,12 +275,10 @@ Both lists match `appName`, `bundleId`, `desktopId`, `appPath`, or `executablePa
|
|
|
277
275
|
|
|
278
276
|
```json
|
|
279
277
|
{
|
|
280
|
-
"
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
"deny": ["1Password", "Bitwarden"]
|
|
285
|
-
}
|
|
278
|
+
"action": "launch",
|
|
279
|
+
"appName": "Slack",
|
|
280
|
+
"allow": ["Slack"],
|
|
281
|
+
"deny": ["1Password", "Bitwarden"]
|
|
286
282
|
}
|
|
287
283
|
```
|
|
288
284
|
|
|
@@ -353,6 +349,12 @@ Single-instance Electron behavior is a common cause of `timeout` and `upstream-e
|
|
|
353
349
|
### Mismatch between `status` and the active session
|
|
354
350
|
- `electron.status` may report a live wrapper launch while the managed session has drifted to `about:blank`. Follow `reattach-electron-launch`, then refresh refs before reusing old `@e…` handles. For non-wrapper tab drift where `details.nextActions` names `select-intended-tab-after-drift`, use that stable `t<N>` action plus `snapshot-after-tab-recovery` before continuing.
|
|
355
351
|
|
|
352
|
+
## Code and shared-session coordination
|
|
353
|
+
|
|
354
|
+
`agent_browser_code` can inspect/interact with the attached browser by selecting its returned `sessionName` (and namespace when present). The browser survives the cell; variables reset. Electron host discovery/launch/cleanup stays on `agent_browser_electron`. Updated cooperating Pi processes order helpers and actions with the shared socket/namespace/session lock; a whole code cell holds that boundary. Multiple named native sessions attached to the same external app are not a universal app lock. Humans and external clients can still change the app.
|
|
355
|
+
|
|
356
|
+
Recovery actions now name `agent_browser_electron` with flat parameters, or `agent_browser` for native operations. Enable `electron` through the loader if a returned lifecycle action is not active. See [0.7 migration](TOOL_CONTRACT.md#07-migration).
|
|
357
|
+
|
|
356
358
|
## Cleanup checklist
|
|
357
359
|
|
|
358
360
|
Before ending the task:
|
|
@@ -371,7 +373,7 @@ If `cleanup` returns `failureCategory: "cleanup-failed"`, inspect `details.elect
|
|
|
371
373
|
|
|
372
374
|
## Where to go next
|
|
373
375
|
|
|
374
|
-
- For exact field semantics, schemas, and `details.*` payloads: [`TOOL_CONTRACT.md#
|
|
376
|
+
- For exact field semantics, schemas, and `details.*` payloads: [`TOOL_CONTRACT.md#agent_browser_electron`](TOOL_CONTRACT.md#agent_browser_electron) and [`TOOL_CONTRACT.md#agent_browser_qa`](TOOL_CONTRACT.md#agent_browser_qa).
|
|
375
377
|
- For workflow examples woven into the broader command surface: [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#electron-desktop-apps).
|
|
376
378
|
- For the closed `RQ-0068` recipe-layer decision that bounds why Electron support is a typed shorthand and not a generic recipe runtime: [`ARCHITECTURE.md`](ARCHITECTURE.md#no-reusable-recipe-layer-yet).
|
|
377
379
|
- For the full release-readiness audit and the `RQ-0096` evidence row: [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md).
|
package/docs/RELEASE.md
CHANGED
|
@@ -25,6 +25,22 @@ Linux/macOS, declared Node floors, official/fork host and consumer-artifact chec
|
|
|
25
25
|
|
|
26
26
|
Docs/workflow-only changes may reuse prior live/platform evidence when the complete compiled runtime, runtime dependencies and relevant tests are unchanged. Changed published files require a new retained tarball checked by `verify-package.mjs --smoke-pi`, including runtime-only SDK and actual CLI loading. Publish those exact checked bytes without repacking.
|
|
27
27
|
|
|
28
|
+
## 0.7 upgrade and rollback
|
|
29
|
+
|
|
30
|
+
The 0.7 tool contract is breaking: direct argv stays compact, `script` moves to `agent_browser_code.code` with a persistent browser, the job DSL is replaced by native batch/code, and five advanced tools use flat schemas after native activation. Publish with the [migration map](TOOL_CONTRACT.md#07-migration), not a parallel legacy public route. Preserve saved auth, explicit files, and Pi journals. Existing strict pre-0.7 isolated-script leases are only read for cleanup.
|
|
31
|
+
|
|
32
|
+
Before activating the update, finish or inspect in-flight mutations/recordings and restart every cooperating Pi process onto the same package. `/reload` alone is not reliable for picking up rebuilt compiled code. Mixed versions do not share the new execution-ordering guarantee.
|
|
33
|
+
|
|
34
|
+
Rollback is package-level: select the exact previously installed 0.6 release with `pi install npm:pi-agent-browser-native@<previous-0.6-version>`, then fully restart participating Pi processes with their existing session files. Do not delete auth/profile/state files, artifacts, or transcripts. The older package cannot interpret all new inner-transition records; inspect the live URL and obtain fresh refs before further actions. Rollback does not reverse submitted browser actions, stop external clients, or restore JavaScript variables.
|
|
35
|
+
|
|
36
|
+
### Changed-path qualification
|
|
37
|
+
|
|
38
|
+
The minimum Pi runtime is 0.87.0, matching the native APIs used by the new tool contract. Wildcard peers remain host-owned dependency declarations, not support for older runtimes.
|
|
39
|
+
|
|
40
|
+
Qualify official Pi 0.87 and the fork independently with the existing host/package/lifecycle infrastructure. Required evidence for the redesign includes compact schema and additive activation/history, native prompt sections, shared direct/code auth continuity, fresh JavaScript globals, selected real images and conservative geometry, visible handled failures/exact recovery, real cross-process helper/action and whole-cell ordering, cancellation/reaping, and ordered transition replay after interruption/branch changes. Retain native batch and advanced action/QA/Electron/source outcome coverage. Small synthetic timing comparisons are not universal performance claims or sandbox certification.
|
|
41
|
+
|
|
42
|
+
Existing owner waivers remain authoritative. Windows qualification is waived; report diagnostic failures without claiming a pass or restarting waived repair work. GitHub Actions remains disabled under the current explicit owner instruction; use the authorized local checks and review evidence, and label CI as not run rather than passed. This does not revoke other approved verification or authorize unrequested platform work.
|
|
43
|
+
|
|
28
44
|
## Pre-release checks
|
|
29
45
|
|
|
30
46
|
From the repository root:
|
|
@@ -85,13 +101,13 @@ crabbox list --provider parallels
|
|
|
85
101
|
|
|
86
102
|
The Crabbox gate is only green when suite assertions and artifact manifests under `.artifacts/platform-smoke/` are green and no unexpected lease/clone remains.
|
|
87
103
|
|
|
88
|
-
The deterministic dogfood mode clean-builds the compiled package, uses the extension harness and real `agent-browser` on `PATH` against a deterministic loopback HTTP fixture, then verifies
|
|
104
|
+
The deterministic dogfood mode clean-builds the compiled package, uses the extension harness and real `agent-browser` on `PATH` against a deterministic loopback HTTP fixture, then verifies persistent browser code aggregation, flat advanced QA/actions, native batch, screenshot artifact verification, and explicit session close. Use `npm run verify -- dogfood --keep-artifacts` or `--artifact-dir <path>` only while debugging, then delete retained screenshots. This smoke complements, but does not replace, human-readable interactive transcript evidence.
|
|
89
105
|
|
|
90
|
-
Every release also requires interactive `tmux`-driven Pi dogfood with the native `agent_browser` tool against real sites. For extension-focused release smokes, use `pi --approve --no-extensions --no-skills -e .` from the trusted checkout before publish so auto-loaded dogfood/QA skills cannot replace the bounded smoke workflow; omit `--approve` only when the smoke is explicitly testing Pi's Project Trust prompt. Run separate skill-enabled dogfood only when validating skill routing or report-generation behavior. Drive prompts with `tmux send-keys`, exercise at least one simple static site and one real documentation/product site, include
|
|
106
|
+
Every release also requires interactive `tmux`-driven Pi dogfood with the native `agent_browser` tool against real sites. For extension-focused release smokes, use `pi --approve --no-extensions --no-skills -e .` from the trusted checkout before publish so auto-loaded dogfood/QA skills cannot replace the bounded smoke workflow; omit `--approve` only when the smoke is explicitly testing Pi's Project Trust prompt. Run separate skill-enabled dogfood only when validating skill routing or report-generation behavior. Drive prompts with `tmux send-keys`, exercise at least one simple static site and one real documentation/product site, include advanced QA, code, or native batch when those surfaces changed, close every opened browser session, remove screenshots/temp artifacts, and record the outcome in the release notes or support-matrix evidence. Do not paste raw multi-line prompts into a tmux Pi pane: plain newlines submit separate queued user messages. For scripted smoke driving, collapse prompt files to one line before sending (`PROMPT=$(tr '\n' ' ' < /tmp/smoke-prompt.md); tmux send-keys -t "$SESSION":0.0 -l "$PROMPT"; tmux send-keys -t "$SESSION":0.0 Enter`). For manual multi-line editing, use Pi's external editor shortcut (`Ctrl+G`) or configure tmux extended keys so Pi can receive `Shift+Enter` for newlines; see the installed Pi `docs/tmux.md` guidance. Automated localhost, fake-upstream, and deterministic dogfood gates do not replace this human-readable live-site transcript evidence. When code changes, add a persisted-session `agent_browser_code` pass that aggregates at least two real pages, exercises a conditional branch, compares selected output with an ordinary-call baseline, verifies selected screenshot bytes reach the model, and checks browser/auth continuity on a later direct/code call with fresh JS globals. Review ordered custom transition entries and interrupted/replayed state, not only the outer result. Exercise handled failures, timeout/cancellation, whole-cell ordering, and child reaping; close only test-owned browsers afterward and retain Pi transcripts. Never use `--no-session` for code dogfood. When `agent_browser_web_search` or package config changed, add one key-free smoke proving the optional tool is absent without config, one fake/unit-backed smoke in the default suite, and one opt-in live Exa or Brave Search check with a real key while confirming the key does not appear in transcripts, stdout/stderr, config status, PR text, or artifacts. When `electron.*` surfaces, attached-session diagnostics, or `qa.attached` changed, add a local Electron pass: `electron.list` → `electron.launch` (expect isolated profile behavior) → `snapshot -i` or `electron.probe` / `qa.attached` → `electron.cleanup` with the returned `launchId`, verifying status/mismatch guidance if you simulate a dead renderer or stale refs. For dense-dashboard stress coverage, use the [public Grafana stress checklist](#public-grafana-stress-checklist) below; it is a maintainer workflow, not bundled product skill or recipe runtime.
|
|
91
107
|
|
|
92
|
-
When reviewing saved session JSONL after a failed smoke or
|
|
108
|
+
When reviewing saved session JSONL after a failed smoke or QA reclassification, expect browser tool rows to carry `isError: true` whenever `details.resultCategory` is `failure`. The model-visible observation must also report failure. Canonical JSON stays parseable; Pi's error patch must not append prose to it. The extension applies that patch on the `tool_result` path so Pi’s transcript matches the wrapper contract ([`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details)). Preserve a normal Pi session directory for those checks; avoiding `--no-session` keeps this evidence intact ([`AGENTS.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/AGENTS.md) preferred validation workflow).
|
|
93
109
|
|
|
94
|
-
The configured-source lifecycle regression harness is required before release because it launches an interactive Pi 0.
|
|
110
|
+
The configured-source lifecycle regression harness is required before release because it launches an interactive Pi 0.87.0+ process under `tmux` with `--approve` and validates `/reload`, full relaunch with the same exact `--session-id`, managed-session continuity, persisted artifacts, compiled-entrypoint pickup after process restart, and Pi failure-patch behavior. Branch-backed `session_tree` rehydration and cleanup ownership are validated by focused extension harness tests:
|
|
95
111
|
|
|
96
112
|
```bash
|
|
97
113
|
npm run verify -- lifecycle
|
|
@@ -116,7 +132,7 @@ Minimum pass:
|
|
|
116
132
|
1. Open the URL with the native `agent_browser` tool in a fresh session.
|
|
117
133
|
2. Run `snapshot -i`; confirm the output is useful on a dense dashboard, including high-value controls and bounded spill behavior when needed.
|
|
118
134
|
3. Exercise one dashboard scroll path. If page-level `scroll` does not move visible content, confirm `details.scrollNoop` / next actions or equivalent guidance points to snapshot/screenshot verification and nested-scroll recovery.
|
|
119
|
-
4. Exercise one explicit combobox-targeted action such as a role/name `
|
|
135
|
+
4. Exercise one explicit combobox-targeted action such as a role/name `agent_browser_action` on a dashboard variable. If it only focuses the field, confirm `details.comboboxFocus` / next actions point to `snapshot -i`, `press ArrowDown`, and `press Enter` when the closed-state evidence qualifies.
|
|
120
136
|
5. Capture at least one screenshot artifact and verify `details.artifactVerification` before using the file.
|
|
121
137
|
6. If `ffmpeg` is on `PATH`, run `record start` / visible interaction / `record stop` and verify the video artifact. Native 0.37 checks ffmpeg before start; older natives may instead return pending output plus `details.recordingDependencyWarning`. Neither is usable recording evidence. Short/cold 0.37 captures have failed on Ubuntu while explicit 12-second fixture captures encoded successfully; record capture duration and every failed attempt, and never treat an empty file as a captured frame.
|
|
122
138
|
7. Inspect `network requests`, `console`, and `errors` summaries. Treat Grafana Play-side noise such as analytics/Sentry requests, public-demo 403s, and console errors as site noise unless the wrapper leaks secrets, hides actionable failed rows, misclassifies artifacts, or suggests unsafe follow-ups.
|
|
@@ -131,10 +147,10 @@ Use this validation prompt after changing click enrichment, tab pinning, ref pre
|
|
|
131
147
|
Run it in an isolated checkout session with skills disabled so the run validates the extension browser workflow instead of external dogfood/QA skill routing. It is fine to restrict active tools at launch so the checkout extension is the only browser surface, but keep those launch details out of the user prompt:
|
|
132
148
|
|
|
133
149
|
```bash
|
|
134
|
-
pi --approve --no-extensions --no-skills -e . --model openai-codex/gpt-
|
|
150
|
+
pi --approve --no-extensions --no-skills -e . --model openai-codex/gpt-6-astra:low --tools agent_browser,agent_browser_code,agent_browser_tools --session-dir "$SESSION_DIR"
|
|
135
151
|
```
|
|
136
152
|
|
|
137
|
-
Repeat with
|
|
153
|
+
Repeat with the task's chosen Astra reasoning level when validating instruction-following robustness. Use unique temp paths for each run and delete them afterward. Run separate skill-enabled dogfood sessions only when the thing under test is skill integration, not this bounded release smoke.
|
|
138
154
|
|
|
139
155
|
Submit the prompt as one Pi message. In tmux automation, write it to a temp file with placeholders replaced, collapse newlines to spaces, and send that one line; for manual multiline entry, use Pi's `Ctrl+G` external editor or a tmux setup that preserves `Shift+Enter` newlines. Do not paste the raw block into a tmux pane line-by-line.
|
|
140
156
|
|
|
@@ -183,7 +199,7 @@ Evaluator expectations after the queued Sauce Demo fixes: the agent should indep
|
|
|
183
199
|
`npm run verify -- package-pi` runs the same package-content checks and additionally confirms that:
|
|
184
200
|
|
|
185
201
|
- the packed package can be loaded through Pi SDK resource loading with the same isolation principle as `pi --no-extensions -e <package-source>`
|
|
186
|
-
-
|
|
202
|
+
- direct/code/loader tools and inactive advanced tools register without requiring optional search config; explicit host selection remains honored
|
|
187
203
|
- any optional companion tools remain governed by their own configuration gates
|
|
188
204
|
- the registered `agent_browser` source resolves inside the extracted packed package path, not the working checkout
|
|
189
205
|
- the packaged `agent_browser` tool can be executed through Pi's loaded native tool definition with a deterministic fake upstream `agent-browser --version` binary
|
|
@@ -215,7 +231,7 @@ Before publishing, validate both local-checkout modes without mixing their assum
|
|
|
215
231
|
5. Run a smoke prompt that exercises `agent_browser`.
|
|
216
232
|
6. Restart Pi after extension edits; configured-source `/reload` is a separate validation mode.
|
|
217
233
|
|
|
218
|
-
For expanded-surface validation, the smoke prompt should cover native tool invocation rather than shelling out to `agent-browser`: `--version`, `--help`, `skills list`, `skills get core --full`, `open` with `sessionMode: "fresh"`, `snapshot -i`, `click`,
|
|
234
|
+
For expanded-surface validation, the smoke prompt should cover native tool invocation rather than shelling out to `agent-browser`: `--version`, `--help`, `skills list`, `skills get core --full`, `open` with `sessionMode: "fresh"`, `snapshot -i`, `click`, flat `agent_browser_action` after enabling `action` (including unique accessible select and explicit session routing), `eval --stdin`, `batch` via stdin, `agent_browser_code`, advanced QA, and experimental source/network tools after loader activation, `screenshot <path>`, explicit `--session … open` plus `--session … close`, `network requests`, `console` / `errors`, `diff snapshot`, `stream status` plus `stream disable`, `dashboard start` plus `dashboard stop`, and `chat <message>` (credential failure is acceptable evidence of wrapper pass-through when `AI_GATEWAY_API_KEY` is intentionally unset). Clean up any opened browser session with `close`, remove temporary files, and kill the tmux session before ending validation.
|
|
219
235
|
|
|
220
236
|
This checklist assumes a real `agent-browser` on `PATH`. It complements, but does not overlap, `npm run verify -- lifecycle`: that harness swaps in a fake upstream binary and focuses on `/reload`, exact `--session-id` relaunch, managed-session continuity, spill-path persistence, and Pi `tool_result` failure-patch semantics (`scripts/verify-lifecycle.mjs`), not the full command matrix above.
|
|
221
237
|
|
|
@@ -241,7 +257,7 @@ Combine flags in one invocation when both apply (order after `lifecycle` is flex
|
|
|
241
257
|
npm run verify -- lifecycle --model openai-codex/gpt-5.5:minimal --timeout-ms 600000
|
|
242
258
|
```
|
|
243
259
|
|
|
244
|
-
|
|
260
|
+
It retains Pi transcripts and evidence, and removes disposable runtime inputs after success. It does not replace occasional real-browser manual smoke testing.
|
|
245
261
|
|
|
246
262
|
**Lifecycle triage:** page checks require the first completed tool result to report a successful expected command and observed `data.url` (open) or `data.origin` (snapshot); recovery text and remembered targets are not page evidence. A cold post-quit `tab-drift` remains a continuity failure: upstream restores storage, and the wrapper must reopen the recorded URL after confirming shutdown before the first current-page read. Verify both empty storage and origin storage at non-root URLs and hash-routed URLs, including after `tab list` or explicit HTTP reads start the daemon and in batches with non-page prefixes, plus fresh refs and main-frame scope. A blank page, an origin root, or a URL mentioned only in an error is not the remembered page. Explicit URL QA recovery does not satisfy the same-page snapshot requirement; URL reopening does not recover unsaved forms, JavaScript memory, or history. A timeout on sentinel `v2` after exact-session relaunch means the new compiled entrypoint did not load after process restart. A reload-step timeout or missing post-reload snapshot often means Pi rejected reload while the TUI still showed `Working…` (`Wait for the current response to finish before reloading`), even when the session JSONL already has a final assistant message. Re-run with `--keep-artifacts --verbose`, inspect the retained pane capture, and confirm the configured model follows tool prompts reliably. Slower models may need a higher `--timeout-ms` than the **180000 ms** default.
|
|
247
263
|
|
|
@@ -263,7 +279,7 @@ Manual validation remains useful for release confidence and installed-package ch
|
|
|
263
279
|
1. Configure exactly one active source for this extension in Pi settings: this checkout path before publishing, or the installed package after publishing.
|
|
264
280
|
2. Launch plain `pi` so extension discovery is active.
|
|
265
281
|
3. Validate managed-session continuity with `/reload` and a full restart plus exact `--session-id` relaunch or `/resume`.
|
|
266
|
-
4. Re-check local extension-side docs (`README.md`, `docs/COMMAND_REFERENCE.md`, `docs/TOOL_CONTRACT.md`, including the [`
|
|
282
|
+
4. Re-check local extension-side docs (`README.md`, `docs/COMMAND_REFERENCE.md`, `docs/TOOL_CONTRACT.md`, including the [`agent_browser_action`](TOOL_CONTRACT.md#agent_browser_action) rules when that shorthand or upstream `find` / `select` behavior changes) and regenerated prompt fragments from `extensions/agent-browser/lib/playbook.ts` via `npm run docs -- playbook check` or `npm run docs`. When the upstream `agent-browser` version or help surface changed, run `npm run verify -- command-reference`.
|
|
267
283
|
|
|
268
284
|
### Real upstream contract validation
|
|
269
285
|
|
|
@@ -292,7 +308,7 @@ The default unit suite also runs `agentBrowserExtension passes through core comm
|
|
|
292
308
|
|
|
293
309
|
### Native checkpoint qualification
|
|
294
310
|
|
|
295
|
-
`test/agent-browser.checkpoint-native.test.mjs` is an opt-in, model-free test for a Pi SDK exposing `acquireCheckpoint` with native failed-journal repair/flush (qualified against core `c401cada69172dfd683e6d569b2cd9e5a45b4fcd`). It loads the compiled extension through native resource discovery, dispatches real tools through an awaited native command, and records real results with SessionManager. It checks idle/release/reload, live root and managed browsers (including a stable ambient socket override without a Pi override), off-branch state, detached and native-owned
|
|
311
|
+
`test/agent-browser.checkpoint-native.test.mjs` is an opt-in, model-free test for a Pi SDK exposing `acquireCheckpoint` with native failed-journal repair/flush (qualified against core `c401cada69172dfd683e6d569b2cd9e5a45b4fcd`). It loads the compiled extension through native resource discovery, dispatches real tools through an awaited native command, and records real results with SessionManager. It checks idle/release/reload, live root and managed browsers (including a stable ambient socket override without a Pi override), off-branch state, detached and native-owned code calls, recording, failed journal appends/cleanup leases, routes, trace, caller-owned CDP attachment, and native checkpoint selection/entries plus root restore after replacing checkout/state directories with fresh inodes. No provider or fake readiness classifier is used.
|
|
296
312
|
|
|
297
313
|
Run as a non-root user in an isolated environment with stock `agent-browser`, Chrome for Testing and ffmpeg. Clear inherited credentials/configuration; do not use a personal browser. The test creates its own HOME/socket directory, copies only the supplied browser binary directory, serves a loopback synthetic page, explicitly closes its sessions, and removes its fixtures. `PI_CHECKPOINT_TEST_BROWSER_DIR` must name a binary installation directory such as `.../.agent-browser/browsers/chrome-<version>`, never a browser profile.
|
|
298
314
|
|
|
@@ -308,7 +324,7 @@ node --test test/agent-browser.checkpoint-native.test.mjs
|
|
|
308
324
|
|
|
309
325
|
#### Required Linux PR gate
|
|
310
326
|
|
|
311
|
-
[`.github/workflows/native-checkpoint.yml`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/native-checkpoint.yml) runs **Native checkpoint Linux** on every PR to `main`, pushes to `main`, and manual dispatch.
|
|
327
|
+
[`.github/workflows/native-checkpoint.yml`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/native-checkpoint.yml) runs **Native checkpoint Linux** on every PR to `main`, pushes to `main`, and manual dispatch. When Actions is authorized, this is the checkpoint PR check (repository settings are separate from the workflow). Under the current keep-Actions-disabled instruction, qualify locally and record the CI waiver rather than enabling workflows. It does not replace release/manual-publication gates or full Linux personal-workspace sleep/resume qualification.
|
|
312
328
|
|
|
313
329
|
The CI-only recipe in [`scripts/ci-native-checkpoint.sh`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/scripts/ci-native-checkpoint.sh) builds the public fork target selected by `host-targets.json` in the workflow's pinned `fitchmultz/.github` automation revision, using the fork's lockfile and documented `npm ci --ignore-scripts`, `hydrate:model-data`, `build:offline` sequence. It uses explicit Node **26.9.0**, the extension's unchanged lockfile, stock **agent-browser 0.38.1**, a fresh stock Chrome-for-Testing download, and Ubuntu ffmpeg. No browser fork, personal profile, credentials, or model calls are involved.
|
|
314
330
|
|
|
@@ -385,8 +401,8 @@ Before publishing:
|
|
|
385
401
|
- run `npm run doctor` and confirm any duplicate-source remediation matches the active package/checkout setup
|
|
386
402
|
- run `npm run verify -- real-upstream` for upstream runtime, result-presentation, or managed-session changes
|
|
387
403
|
- confirm both local-checkout modes still work for pre-release validation: isolated `pi --approve --no-extensions -e .` smoke testing for general trusted checkout loading (add `--no-skills` for extension-focused bounded smokes; omit `--approve` only to test the trust prompt) and configured-source lifecycle validation
|
|
388
|
-
- complete interactive `tmux` live-site extension smoke with `pi --approve --no-extensions --no-skills -e .` and the native `agent_browser` tool (at least one simple static site and one real documentation/product site; include
|
|
389
|
-
-
|
|
404
|
+
- complete interactive `tmux` live-site extension smoke with `pi --approve --no-extensions --no-skills -e .` and the native `agent_browser` tool (at least one simple static site and one real documentation/product site; include advanced QA, code, or native batch when those surfaces changed; use the [public Grafana stress checklist](#public-grafana-stress-checklist) when dashboard/diagnostic/artifact behavior changed; close sessions and remove screenshots/temp artifacts; record evidence). Run separate skill-enabled dogfood only when validating skill routing/report-generation behavior—see [Pre-release checks](#pre-release-checks); automated gates are not a substitute
|
|
405
|
+
- satisfy the approved release checks and current owner waivers; record individual platform evidence and never label a Windows-inclusive composition passed when Windows was waived
|
|
390
406
|
- run `npm run verify -- lifecycle` for configured-source `/reload`, exact `--session-id` relaunch, managed-session continuity, persisted-spill, and Pi failure-patch regression coverage (required before publish; see [Pre-release checks](#pre-release-checks))
|
|
391
407
|
- confirm [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md) still maps every current baseline inventory section to docs, runtime handling, tests, and validation status
|
|
392
408
|
- manually exercise real-browser `/reload` and full restart plus exact `--session-id` relaunch or `/resume` continuity when release risk warrants browser-level confidence beyond the fake upstream harness
|
package/docs/REQUIREMENTS.md
CHANGED
|
@@ -62,16 +62,20 @@ Define the product requirements and constraints for `pi-agent-browser-native`.
|
|
|
62
62
|
- Keep the handling simple and global-install-friendly.
|
|
63
63
|
- Do not rely on package skill overrides as the primary answer.
|
|
64
64
|
|
|
65
|
-
###
|
|
66
|
-
|
|
67
|
-
-
|
|
68
|
-
-
|
|
69
|
-
-
|
|
70
|
-
-
|
|
71
|
-
-
|
|
72
|
-
-
|
|
73
|
-
-
|
|
74
|
-
-
|
|
65
|
+
### Browser tool contract
|
|
66
|
+
|
|
67
|
+
- Keep the always-on interface compact: `agent_browser { args, stdin?, outputPath?, timeoutMs?, sessionMode? }`, strict-JSON `agent_browser_code { code, session?, namespace?, timeoutMs?, outputPath? }`, and `agent_browser_tools { enable? }`.
|
|
68
|
+
- Preserve the complete native command surface through one CLI executor. Native batch handles fixed sequences; code handles branches/loops/aggregation. Remove the redundant public job DSL and old isolated-script execution instead of carrying parallel legacy runtimes.
|
|
69
|
+
- Register advanced action, QA, Electron, UI-source, and network-source tools once, initially inactive. Enable additively through Pi's native tool selection/history without removing unrelated tools. Use flat schemas and return executable next actions naming the actual tool.
|
|
70
|
+
- Preserve unique visible accessible dropdown selection, semantic/ref recovery, truthful QA verdicts (including ambiguous error-buffer residue), attached diagnostic preservation/defaults, Electron process/profile ownership, and bounded source scans. Raw diagnostic-read success is not a QA verdict, and source hints remain candidates rather than authoritative blame.
|
|
71
|
+
- Code runs in a fresh permissioned Node child with no host APIs/imports, null-prototype task globals, JSON-only bounded IPC, serial native calls, and explicit JSON/image output. Keep current source/output/call/deadline limits. Browser/authentication persists under existing native ownership and configuration; JavaScript variables do not. Allow ordinary native commands including batch/auth/profile/state/connect/close, but prevent changes to the fixed parent-selected identity and namespace-wide close within a cell.
|
|
72
|
+
- Hold a cooperating-process execution lock keyed by actual socket context, canonical namespace, and session across direct helper/action work or the entire code cell. Use ordered multi-identity acquisition for replacement/cleanup and a namespace barrier for direct close-all. This does not coordinate humans, third-party clients, or older versions, guarantee atomic browser actions, or provide rollback.
|
|
73
|
+
- Persist ordered inner intent/completion transitions through Pi custom entries and existing browser-state reducers. Interrupted outcomes remain unknown until inspection. Require Pi persistence for code; retain exact old script lease readers only for upgrade cleanup, not public execution.
|
|
74
|
+
- Use one canonical model-visible observation for JSON/prose/code with success/data/error, exact recovery, artifact receipts, and image observations. Details is audit/replay metadata, not the sole source of actionable failures. Inner calls avoid full model rendering/spills/image encoding. Bound outer output and name a complete redacted observation spill rather than truncating executable payloads.
|
|
75
|
+
- Attach only verified real images; code selects current-cell image handles explicitly. Report pixel dimensions and CSS viewport/DPR/crop/scroll/frame evidence conservatively. Unknown geometry remains unknown; neither full-page nor element captures are assumed viewport coordinates. Do not inject model-specific image-detail/provider payload changes.
|
|
76
|
+
- Direct upstream capability remains available: explicit sessions, state/restore paths, profiles, config, file access, launch arguments, environment, local pages, output paths, and close arguments pass through under ordinary native semantics. Keep all session/state rows visible. Wrapper validation concerns input shape, lifecycle integrity, and truthful evidence, not authorization of resources.
|
|
77
|
+
- Native download owns clicking and saving, including loopback, Blob exports, and redirects. Create artifact directories before dispatch and verify saved files; output-data writes must not alias browser artifacts. Pending recording is not saved video.
|
|
78
|
+
- Maintain a deliberate 0.7 migration map and package-level rollback that preserves saved authentication, explicit artifacts, and Pi transcripts. Restart every cooperating Pi participant on the updated package before relying on shared ordering. No universal performance, sandbox, authentication-persistence, or Windows qualification claims.
|
|
75
79
|
|
|
76
80
|
### Documentation standard
|
|
77
81
|
|
|
@@ -109,13 +113,13 @@ The design should comfortably support workflows such as:
|
|
|
109
113
|
- headless authenticated `chat.com` / ChatGPT / OpenAI and Cloudflare Dashboard browsing without forcing `--headed` or `--auto-connect`
|
|
110
114
|
- upstream profile/debug workflows without adding a local profile-cloning layer in this package
|
|
111
115
|
- provider-backed or iOS device launches where upstream owns credentials, env, and setup; the wrapper forwards argv and the parent environment without emulating those backends
|
|
112
|
-
- desktop Electron targets using
|
|
113
|
-
-
|
|
116
|
+
- desktop Electron targets using `agent_browser_electron` for discover → isolated launch → attach → probe/cleanup, or raw `args: ["connect", …]` when the operator launches the real app with a debug port for signed-in state (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#agent_browser_electron) and [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#electron-desktop-apps))
|
|
117
|
+
- authenticated or public-page loops, branches, and bounded multi-page extraction through `agent_browser_code`, returning selected JSON/images while retaining the browser and resetting JavaScript variables
|
|
114
118
|
|
|
115
119
|
## Implications for the implementation
|
|
116
120
|
|
|
117
121
|
- Package-manifest behavior matters more than repo-local development wiring.
|
|
118
|
-
- The extension
|
|
122
|
+
- The extension uses native Pi tool registration/activation and named prompt sections rather than replacing the full system prompt. Qualify official Pi 0.87 and the fork independently; fork checkpoint hooks stay optional and hosted provider orchestration is not required.
|
|
119
123
|
- The wrapper should stay thin, with upstream `agent-browser` remaining the source of truth for command semantics.
|
|
120
124
|
- Successful and failed tool outcomes should surface bounded machine-readable fields on Pi-facing `details` (`resultCategory`, `successCategory`, `failureCategory`, optional structured `nextActions`, optional `pageChangeSummary` with explicit observed-vs-dispatched evidence and per-step summaries on `batch`, optional `artifactVerification` with the same shape on successful `batchSteps[]` rows, optional `outputFile`, optional `timeoutPartialProgress`) so agents can branch without parsing prose; browser-bearing `nextActions` must preserve a known session identity so recovery cannot inspect an unrelated implicit session; stateful commands (`auth`, `cookies`, `storage`, `dialog`, `frame`, `state`) plus other structured diagnostics (for example `network`, `diff`, `trace`, `stream`, `dashboard`, `chat`) and `batch` should redact secret-bearing payloads in model-facing `details.data`, including the compact per-step `batch` roll-up on the parent result (full per-step payloads live on `batchSteps[]`). Dialog/prompt-related timeouts should be bounded with recovery `nextActions`; non-dialog timeouts should prefer best-effort per-step progress and retry payloads when a plan is available; no-op scrolls should expose no-movement state instead of only an upstream success boolean; explicit page/container scroll helpers should expose before/after movement evidence. The contract lives in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), enums and classifier precedence live in `extensions/agent-browser/lib/results/categories.ts` and `contracts.ts`, and presentation-time summaries, redaction, network request follow-ups, and artifact verification rollups are assembled in `extensions/agent-browser/lib/results/presentation.ts` (`buildPageChangeSummary`, command taxonomy predicates from `command-taxonomy.ts`, `redactPresentationData`, `buildArtifactVerificationSummary`, `buildBatchPresentation`).
|
|
121
125
|
- User-facing docs belong in `README.md` and the canonical published files under `docs/`.
|
|
@@ -132,4 +136,4 @@ The design should comfortably support workflows such as:
|
|
|
132
136
|
- Treat argv-supplied `--allowed-domains` as launch-scoped so it starts in a fresh browser context. Keep upstream responsible for request/worker/popup/WebRTC containment and incompatible launch-mode rejection; pass its result through unchanged.
|
|
133
137
|
- Upstream restore-state periodic autosave remains upstream-owned. Forward an explicit `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` unchanged when a daemon launches; otherwise default it to `0` for wrapper-owned headed launches, persist the effective launch-time interval, and retain that environment across helpers, follow-ups, still-owned off-current sessions, Electron cleanup closes, and transcript reload/resume to avoid upstream 0.33.2's visible temporary collector tabs and per-call daemon-configuration drift. Reject attempts to change that interval on an already-running wrapper-owned headed daemon until the caller closes it and launches fresh. If automatic cleanup of a replaced wrapper-owned session fails, persist that outcome and restore the older identity's ownership from the transcript so explicit follow-up and cleanup remain possible. Document the 30-second upstream default, native-close preservation, and the direct-window-close loss risk because headed browsers are exempt from idle shutdown; do not duplicate its timer or claim its restore files as wrapper artifacts.
|
|
134
138
|
- Read-only upstream `skills list`, `skills get …`, and `skills path …` must stay free of implicit managed `--session` under default `sessionMode: "auto"` (still with `--json`), matching plain-text `--help` / `--version` inspection semantics so bundled skill text does not pin or rotate the active browser session; new `skills` subcommands pick up that behavior only after allowlisting in `extensions/agent-browser/lib/runtime.ts` with regression coverage.
|
|
135
|
-
- Optional `
|
|
139
|
+
- Optional `session` on `agent_browser_action` must compile to a leading `--session <name>` pair before upstream `find` or `select` argv so the shorthand can target a named upstream browser without hand-built `args`, while `buildExecutionPlan` still skips double-injecting the extension-managed implicit session whenever planned argv already starts with `--session`; stale-ref retries for compiled `find` actions and bounded `try-*` candidate `nextActions` must preserve that same prefix. Contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#agent_browser_action) / [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sessionmode); implementation in `extensions/agent-browser/index.ts` and `extensions/agent-browser/lib/runtime.ts`.
|