pi-agent-browser-native 0.2.38 → 0.2.39
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -0
- package/README.md +18 -5
- package/docs/ARCHITECTURE.md +2 -2
- package/docs/COMMAND_REFERENCE.md +15 -12
- package/docs/RELEASE.md +19 -7
- package/docs/SUPPORT_MATRIX.md +9 -7
- package/docs/TOOL_CONTRACT.md +1 -1
- package/docs/platform-smoke.md +176 -0
- package/extensions/agent-browser/lib/playbook.ts +2 -2
- package/extensions/agent-browser/lib/process.ts +72 -13
- package/package.json +13 -1
- package/platform-smoke.config.mjs +18 -0
- package/scripts/agent-browser-capability-baseline.mjs +9 -6
- package/scripts/platform-smoke/artifacts.mjs +94 -0
- package/scripts/platform-smoke/browser-dogfood-windows.ps1 +110 -0
- package/scripts/platform-smoke/crabbox-runner.mjs +149 -0
- package/scripts/platform-smoke/doctor.mjs +307 -0
- package/scripts/platform-smoke/linux-image/Dockerfile +23 -0
- package/scripts/platform-smoke/platform-build-windows.ps1 +103 -0
- package/scripts/platform-smoke/targets.mjs +471 -0
- package/scripts/platform-smoke.mjs +161 -0
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,17 @@
|
|
|
2
2
|
|
|
3
3
|
## Unreleased
|
|
4
4
|
|
|
5
|
+
## 0.2.39 - 2026-06-02
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- Added a Crabbox-backed platform smoke gate for release validation across macOS, prepared Ubuntu Linux, and native Windows, including packed-package installation and deterministic browser dogfood suites.
|
|
10
|
+
|
|
11
|
+
### Changed
|
|
12
|
+
|
|
13
|
+
- Updated the upstream capability baseline, command reference, platform smoke images, and live-contract metadata for `agent-browser` `0.27.1`.
|
|
14
|
+
- Reduced per-target platform smoke cost by using a focused `verify -- platform-target` gate inside Crabbox targets instead of rerunning the full default verification suite on every OS.
|
|
15
|
+
|
|
5
16
|
## 0.2.38 - 2026-05-29
|
|
6
17
|
|
|
7
18
|
### Changed
|
package/README.md
CHANGED
|
@@ -340,7 +340,7 @@ For asynchronous exports, click first and then wait for the download:
|
|
|
340
340
|
{ "args": ["wait", "--download", "/tmp/report.csv"] }
|
|
341
341
|
```
|
|
342
342
|
|
|
343
|
-
When a user gives exact artifact paths for screenshots, recordings, downloads, PDFs, traces, or HAR files, use those paths or explicitly report why the artifact was unavailable; do not silently substitute a different path in the final report. With upstream `agent-browser 0.27.
|
|
343
|
+
When a user gives exact artifact paths for screenshots, recordings, downloads, PDFs, traces, or HAR files, use those paths or explicitly report why the artifact was unavailable; do not silently substitute a different path in the final report. With upstream `agent-browser 0.27.1`, treat `details.savedFilePath` as upstream-reported metadata and confirm `details.artifacts[].exists` before relying on the requested `wait --download <path>` file being present on disk.
|
|
344
344
|
|
|
345
345
|
For evidence-only screenshots or QA captures, branch on `details.artifactVerification` and `details.artifacts` before reporting PASS/FAIL; inline image attachments are optional when size limits allow—do not require vision review unless the user asked for visual inspection. If the latest prompt names exact required artifact paths, browser close can be blocked with `details.promptGuard` until those artifacts are saved and verified.
|
|
346
346
|
|
|
@@ -446,7 +446,7 @@ The full `npm run verify` gate runs:
|
|
|
446
446
|
- command-reference baseline checks
|
|
447
447
|
- live command-reference verification against the targeted installed upstream `agent-browser`
|
|
448
448
|
|
|
449
|
-
Step order and which subprocesses run live in [`scripts/project.mjs`](scripts/project.mjs); [`test/project-verify.test.ts`](test/project-verify.test.ts) locks default, `release`, `real-upstream`, `dogfood`, `package-pi`, and combined-docs orchestration so a gate cannot disappear accidentally. Run `npm run verify -- --help` for opt-in modes and supported passthrough flags.
|
|
449
|
+
Step order and which subprocesses run live in [`scripts/project.mjs`](scripts/project.mjs); [`test/project-verify.test.ts`](test/project-verify.test.ts) locks default, `release`, `real-upstream`, `dogfood`, `platform-target`, `platform-smoke`, `package-pi`, and combined-docs orchestration so a gate cannot disappear accidentally. Run `npm run verify -- --help` for opt-in modes and supported passthrough flags.
|
|
450
450
|
|
|
451
451
|
The deterministic agent-efficiency benchmark’s **standalone JSON/Markdown accounting run** is not part of default `npm run verify` (only `npm run verify -- benchmark` or `npm run benchmark:agent-browser` invokes the script). The full unit suite still exercises `test/agent-browser.efficiency-benchmark.test.ts`. Use the script before and after agent-facing abstractions to prove call-count, output-size, stale-ref, artifact, failure-category coverage, success-rate, and elapsed-time effects before changing the wrapper UX:
|
|
452
452
|
|
|
@@ -467,22 +467,34 @@ npm run verify -- real-upstream
|
|
|
467
467
|
|
|
468
468
|
That mode sets `PI_AGENT_BROWSER_REAL_UPSTREAM=1` and runs `test/agent-browser.real-upstream-contract.test.ts` against the real `agent-browser` on `PATH` (version must match the capability baseline). It covers inspection, skills, a broad core interaction and navigation matrix on localhost fixtures (including `batch` stdin and `pushstate`), plus `vitals`, network route/requests/HAR, diff snapshot/screenshot/url, trace/profiler, console/errors/highlight, stream enable/status/disable, `cookies set --curl`, a `react tree` missing-renderer path, and `wait --download` with the on-disk caveat documented in release notes. The harness uses a throwaway temp `HOME` and dedicated socket/screenshot directories so the run does not touch your normal browser profile paths. Browser-opening or credential-dependent families such as `inspect`, `dashboard`, `chat`, provider clouds, and OS clipboard flows stay in fake-upstream or manual validation unless a safe deterministic fixture is added. For prerequisites, isolation details, and troubleshooting, see [`docs/RELEASE.md`](docs/RELEASE.md#real-upstream-contract-validation).
|
|
469
469
|
|
|
470
|
-
A deterministic live-browser wrapper smoke is available without an LLM choosing tool calls:
|
|
470
|
+
A deterministic host-only live-browser wrapper smoke is available without an LLM choosing tool calls:
|
|
471
471
|
|
|
472
472
|
```bash
|
|
473
473
|
npm run verify -- dogfood
|
|
474
474
|
```
|
|
475
475
|
|
|
476
|
-
That mode drives the native wrapper through top-level `qa`, `semanticAction`,
|
|
476
|
+
That mode drives the native wrapper through top-level `qa`, `semanticAction`, constrained `job`, screenshot artifact verification, and session close against a deterministic local fixture. It complements, but does not replace, the interactive Pi/tmux release dogfood in [`docs/RELEASE.md`](docs/RELEASE.md#pre-release-checks).
|
|
477
|
+
|
|
478
|
+
Cross-platform release coverage uses Crabbox to run macOS, Ubuntu Linux, and native Windows target suites:
|
|
479
|
+
|
|
480
|
+
```bash
|
|
481
|
+
npm run check:platform-smoke
|
|
482
|
+
npm run smoke:platform:ubuntu-image
|
|
483
|
+
npm run smoke:platform:all
|
|
484
|
+
```
|
|
485
|
+
|
|
486
|
+
The required matrix is documented in [`docs/platform-smoke.md`](docs/platform-smoke.md). It runs `platform-build` (fast target-local verify, pack, clean packed Pi install, `pi list`) and `browser-dogfood-smoke` (real `agent-browser`/browser wrapper smoke) on every target.
|
|
477
487
|
|
|
478
488
|
For package release confidence, follow [`docs/RELEASE.md`](docs/RELEASE.md). The release gate is:
|
|
479
489
|
|
|
480
490
|
```bash
|
|
481
491
|
npm run doctor
|
|
492
|
+
npm run check:platform-smoke
|
|
493
|
+
npm run smoke:platform:ubuntu-image
|
|
482
494
|
npm run verify -- release
|
|
483
495
|
```
|
|
484
496
|
|
|
485
|
-
`npm run verify -- release` includes the default verification gate
|
|
497
|
+
`npm run verify -- release` includes the default verification gate, packaged Pi smoke coverage, and the release-blocking Crabbox platform matrix. The package also has a `prepublishOnly` hook that runs the same release gate and `npm pack --dry-run` during `npm publish`.
|
|
486
498
|
|
|
487
499
|
## How it works
|
|
488
500
|
|
|
@@ -584,6 +596,7 @@ These calls return plain text and stay stateless: the extension does not inject
|
|
|
584
596
|
| `docs/ARCHITECTURE.md` | Design decisions and implementation structure |
|
|
585
597
|
| `docs/REQUIREMENTS.md` | Product requirements and constraints |
|
|
586
598
|
| `docs/RELEASE.md` | Release, package, and lifecycle verification workflow |
|
|
599
|
+
| `docs/platform-smoke.md` | Crabbox macOS, Ubuntu, and native Windows release gate |
|
|
587
600
|
| `docs/SUPPORT_MATRIX.md` | Current upstream support audit and release-readiness matrix |
|
|
588
601
|
| `test/` | Wrapper, runtime, presentation, lifecycle, and package tests |
|
|
589
602
|
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -31,7 +31,7 @@ Why:
|
|
|
31
31
|
|
|
32
32
|
The extension should:
|
|
33
33
|
- resolve `agent-browser` from `PATH`
|
|
34
|
-
- invoke it directly,
|
|
34
|
+
- invoke it directly on POSIX; on Windows, route through PowerShell with single-quoted argv so npm launchers and the native `.exe` receive the same command tail that a user would type, and terminate the full PowerShell/agent-browser process tree with `taskkill /T /F` on timeout or abort before falling back to the direct child signal
|
|
35
35
|
- inject `--json`
|
|
36
36
|
- complete each upstream invocation when the direct `agent-browser` child exits even if Node delays `"close"`: piped stdio can stay referenced by longer-lived descendant processes, so `runAgentBrowserProcess` watches `exit` and `close` together, leaves stdio intact during a short post-`exit` grace so normal `close` can still win, destroys streams only when the post-`exit` fallback fires, and prefers `close` codes then wrapper timeout (`124`) over signal-shaped `exit` codes (`watchSpawnedChildCompletion` / `resolveSpawnedChildExitCode` in `extensions/agent-browser/lib/process.ts`) so the tool cannot hang after the CLI process has already terminated
|
|
37
37
|
- support optional stdin only for `eval --stdin`, `batch`, `auth save --password-stdin`, and wrapper-generated `batch` stdin from top-level `job`, `qa`, `sourceLookup`, or `networkSourceLookup`, rejecting other command/stdin combinations before launch; top-level `electron` never accepts caller `stdin` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron))
|
|
@@ -156,7 +156,7 @@ That failure should include a structured recovery hint pointing to `sessionMode:
|
|
|
156
156
|
Implementation detail lives in `extensions/agent-browser/lib/argv-descriptor.ts` and `extensions/agent-browser/lib/argv-grammar.ts` (command discovery, `VALUE_FLAGS`, `parseArgvDescriptor`) plus `extensions/agent-browser/lib/runtime.ts` (`getStartupScopedFlags`, `buildExecutionPlan`):
|
|
157
157
|
|
|
158
158
|
- **Command discovery:** Leading argv is scanned with a value-taking allowlist so known global flags and documented command flags consume their values before the upstream command word is identified. Missing-value prevalidation is intentionally limited to upstream global value flags; command-scoped flags and literal text are left to upstream parsing so values like `fill #field --password` are not rejected by wrapper heuristics before the CLI sees them. When upstream adds new global flags that take values ahead of the command, extend both the command-discovery and prevalidation allowlists; when it adds command-specific flags, extend only command discovery/redaction as needed. A smaller set of global boolean flags may be followed by an optional `true`/`false` literal; when present, that literal is consumed as the flag value before command discovery continues.
|
|
159
|
-
- **`--state` disambiguation:** Persisted browser `--state` before the command participates in launch-scoped validation and tab-correction hints. The same flag spelling after a `wait` command is excluded from startup-scoped detection so upstream help examples such as `wait @ref --state hidden` do not spuriously require `sessionMode: "fresh"` while an implicit session is active. As of upstream `agent-browser 0.27.
|
|
159
|
+
- **`--state` disambiguation:** Persisted browser `--state` before the command participates in launch-scoped validation and tab-correction hints. The same flag spelling after a `wait` command is excluded from startup-scoped detection so upstream help examples such as `wait @ref --state hidden` do not spuriously require `sessionMode: "fresh"` while an implicit session is active. As of upstream `agent-browser 0.27.1`, the parser does not implement those `wait --state` examples as distinct wait modes, so agent-facing docs recommend `wait --fn` predicates for disappearance checks instead.
|
|
160
160
|
- **`--auto-connect`:** Treated as launch-scoped only when enabled (`--auto-connect` bare or `true`). `--auto-connect false` is ignored for startup-scoped blocking so disabled attach hints do not force a fresh launch.
|
|
161
161
|
|
|
162
162
|
**Sessionless inspection and local commands:** Plain-text global help and version probes (`--help`, `-h`, `--version`, `-V`) must never allocate or bind the extension-managed session. The same session-ownership rule applies to read-only upstream `skills list`, `skills get …`, and `skills path …`, local auth profile management (`auth save/list/show/delete/remove`), plus local/setup surfaces such as `profiles`, `dashboard start/stop`, `device list`, `doctor`, `install`, `upgrade`, `session list`, and targeted/all local saved-state maintenance (`state list/show`, `state clear --all`, `state clear -a`, `state clear <session-name>`, `state clean --older-than <days>`, `state rename`). Non-plain-text sessionless commands still run with `--json` for machine-readable output, but the planner does not prepend the implicit managed `--session`, so an agent can inspect local capabilities or start/stop the standalone dashboard without consuming the implicit session slot before a real `open`. Browser-backed, context-dependent, or incomplete commands such as root `session`, untargeted `state clear`, bare `state clean`, `auth login`, `state save`, and `state load` keep normal managed-session injection. Command-shape allowlisting lives in `extensions/agent-browser/lib/command-policy.ts` (`needsManagedSession`), while `extensions/agent-browser/lib/runtime.ts` (`isPlainTextInspectionArgs`, `buildExecutionPlan`) applies that decision to execution planning.
|
|
@@ -18,7 +18,7 @@ This project intentionally blocks normal `agent-browser` bash usage in most agen
|
|
|
18
18
|
|
|
19
19
|
<!-- agent-browser-capability-baseline:start upstream-baseline -->
|
|
20
20
|
<!-- Generated from scripts/agent-browser-capability-baseline.mjs. Run `npm run docs -- command-reference write` to update. Do not edit manually. -->
|
|
21
|
-
This reference is baselined to the locally installed `agent-browser 0.27.
|
|
21
|
+
This reference is baselined to the locally installed `agent-browser 0.27.1` command/help surface, audited against vercel-labs/agent-browser@90050f2913159875e2c3719e424746396ccb3cbf. Upstream `agent-browser` remains the source of truth for command semantics; this file is the local fallback for Pi agent sessions where direct binary help is blocked or discouraged.
|
|
22
22
|
|
|
23
23
|
The lightweight drift check is `npm run verify -- command-reference`. Run it whenever the installed upstream `agent-browser` version changes or this reference is edited.
|
|
24
24
|
|
|
@@ -126,7 +126,7 @@ Use `vitals [url]` for Core Web Vitals plus React hydration timing when availabl
|
|
|
126
126
|
{ "args": ["pushstate", "/dashboard?tab=settings"] }
|
|
127
127
|
```
|
|
128
128
|
|
|
129
|
-
For first-navigation setup, start on `about:blank`, then stage routes, cookies, or init scripts before navigating. The relevant v0.27.
|
|
129
|
+
For first-navigation setup, start on `about:blank`, then stage routes, cookies, or init scripts before navigating. The relevant v0.27.1 surfaces are `network route <url> [--abort|--body <json>] [--resource-type <csv>]` and `cookies set --curl <file>`:
|
|
130
130
|
|
|
131
131
|
```json
|
|
132
132
|
{ "args": ["open"], "sessionMode": "fresh" }
|
|
@@ -330,7 +330,7 @@ For one-call flows, put the click and wait in `batch`; the wait step keeps the s
|
|
|
330
330
|
{ "args": ["batch"], "stdin": "[[\"click\",\"@export\"],[\"wait\",\"--download\",\"/tmp/report.csv\"]]" }
|
|
331
331
|
```
|
|
332
332
|
|
|
333
|
-
A successful wait-based download renders a readable summary such as `Download completed: /tmp/report.csv` and exposes top-level `details.savedFilePath` plus `details.savedFile` for non-batch calls. With the current upstream `agent-browser 0.27.
|
|
333
|
+
A successful wait-based download renders a readable summary such as `Download completed: /tmp/report.csv` and exposes top-level `details.savedFilePath` plus `details.savedFile` for non-batch calls. With the current upstream `agent-browser 0.27.1`, `wait --download <path>` may report the requested path before this environment can verify that the file was persisted there. Treat `details.savedFilePath` as upstream-reported metadata unless `details.artifacts[].exists` is true. Upstream tracking: [vercel-labs/agent-browser#1300](https://github.com/vercel-labs/agent-browser/issues/1300).
|
|
334
334
|
|
|
335
335
|
### Download, screenshot, and PDF files
|
|
336
336
|
|
|
@@ -598,7 +598,7 @@ When a snapshot is too large for inline output, the Pi wrapper renders a compact
|
|
|
598
598
|
| `wait --download [path]` | Wait for a download started by a previous action and optionally save it to `path`; successful wrapper results include upstream-reported `savedFilePath`/`savedFile`, while `details.artifacts[].exists` is the wrapper's on-disk verification signal. |
|
|
599
599
|
| `wait --download [path] --timeout <ms>` | Set download-start timeout in milliseconds. In the native Pi wrapper, use `25000` ms or less per call to stay under the upstream CLI IPC budget. |
|
|
600
600
|
|
|
601
|
-
Current v0.27.
|
|
601
|
+
Current v0.27.1 source does not parse `wait <selector> --state hidden` / `wait <selector> --state detached` as distinct wait modes even though upstream help mentions those examples. Use `wait --fn "!document.querySelector('#spinner')"` or another explicit JavaScript predicate for disappearance/detach checks until upstream parser support exists.
|
|
602
602
|
|
|
603
603
|
### Diff, debug, and streaming
|
|
604
604
|
|
|
@@ -607,7 +607,7 @@ Current v0.27.0 source does not parse `wait <selector> --state hidden` / `wait <
|
|
|
607
607
|
| `diff snapshot` | Compare current versus last snapshot. Use `diff snapshot --baseline <file> --selector <sel> --compact --depth <n>` when you need a saved baseline, scoped subtree, compact output, or depth bound. |
|
|
608
608
|
| `diff screenshot --baseline` | Compare current screenshot versus a baseline image. Use `diff screenshot --baseline <file> --output <file> --threshold <0-1> --selector <sel> --full` when you need a saved diff image, threshold tuning, element scope, or full-page capture. |
|
|
609
609
|
| `diff url <u1> <u2>` | Compare two pages. Use `diff url <u1> <u2> --screenshot --wait-until <strategy> --selector <sel> --compact --depth <n>` when you need screenshot comparison, navigation wait control, or scoped/compact snapshot comparison. |
|
|
610
|
-
| `trace start
|
|
610
|
+
| `trace start`, `trace stop [path]` | Record a Chrome DevTools trace. |
|
|
611
611
|
| `profiler start|stop [path]` | Record a Chrome DevTools profile. |
|
|
612
612
|
| `record start <path> [url]` | Start WebM video recording; output is written on `record stop`. Requires `ffmpeg` on `PATH` for the final encode. |
|
|
613
613
|
| `record stop` | Stop and save video. If this fails with `ffmpeg not found`, install `ffmpeg` / `ffmpeg-full` and rerun the recording. |
|
|
@@ -750,14 +750,14 @@ Other useful environment variables include `AGENT_BROWSER_DEFAULT_TIMEOUT`, `AGE
|
|
|
750
750
|
<!-- agent-browser-capability-baseline:start capability-token-baseline -->
|
|
751
751
|
<!-- Generated from scripts/agent-browser-capability-baseline.mjs. Run `npm run docs -- command-reference write` to update. Do not edit manually. -->
|
|
752
752
|
<details>
|
|
753
|
-
<summary>Generated verifier capability baseline for agent-browser 0.27.
|
|
753
|
+
<summary>Generated verifier capability baseline for agent-browser 0.27.1</summary>
|
|
754
754
|
|
|
755
755
|
This generated block is review data for maintainers. The human-authored reference sections above remain the readable command guide.
|
|
756
756
|
|
|
757
757
|
#### Source evidence
|
|
758
758
|
- repository: `vercel-labs/agent-browser`
|
|
759
|
-
- upstream HEAD: `
|
|
760
|
-
- upstream package version: `0.27.
|
|
759
|
+
- upstream HEAD: `90050f2913159875e2c3719e424746396ccb3cbf`
|
|
760
|
+
- upstream package version: `0.27.1`
|
|
761
761
|
- inspected: `agent-browser --version`
|
|
762
762
|
- inspected: `agent-browser --help`
|
|
763
763
|
- inspected: `selected agent-browser <command> --help output`
|
|
@@ -824,7 +824,7 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
824
824
|
- Built-in skills: 13 human-doc token(s), 13 upstream token(s)
|
|
825
825
|
- Core page, element, navigation, and extraction commands: 74 human-doc token(s), 74 upstream token(s)
|
|
826
826
|
- Sessions, state, tabs, frames, dialogs, and windows: 20 human-doc token(s), 16 upstream token(s)
|
|
827
|
-
- Network, storage, artifacts, diagnostics, and performance:
|
|
827
|
+
- Network, storage, artifacts, diagnostics, and performance: 43 human-doc token(s), 53 upstream token(s)
|
|
828
828
|
- Batch, auth, confirmations, setup, dashboard, devices, and AI commands: 24 human-doc token(s), 24 upstream token(s)
|
|
829
829
|
- Global flags, config, providers, policy, and environment: 117 human-doc token(s), 90 upstream token(s)
|
|
830
830
|
|
|
@@ -960,7 +960,8 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
960
960
|
- `diff screenshot --baseline <file> --output <file> --threshold <0-1> --selector <sel> --full`
|
|
961
961
|
- `diff url <u1> <u2>`
|
|
962
962
|
- `diff url <u1> <u2> --screenshot --wait-until <strategy> --selector <sel> --compact --depth <n>`
|
|
963
|
-
- `trace start
|
|
963
|
+
- `trace start`
|
|
964
|
+
- `trace stop [path]`
|
|
964
965
|
- `profiler start|stop [path]`
|
|
965
966
|
- `record start <path> [url]`
|
|
966
967
|
- `record restart <path> [url]`
|
|
@@ -1252,7 +1253,8 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1252
1253
|
- root help: `storage <local|session>`
|
|
1253
1254
|
- root help: `diff snapshot`
|
|
1254
1255
|
- root help: `diff screenshot --baseline`
|
|
1255
|
-
- root help: `trace start
|
|
1256
|
+
- root help: `trace start`
|
|
1257
|
+
- root help: `trace stop [path]`
|
|
1256
1258
|
- root help: `profiler start|stop [path]`
|
|
1257
1259
|
- root help: `record start <path> [url]`
|
|
1258
1260
|
- root help: `record stop`
|
|
@@ -1288,7 +1290,8 @@ This generated block is review data for maintainers. The human-authored referenc
|
|
|
1288
1290
|
- diff help: `--threshold <0-1>`
|
|
1289
1291
|
- diff help: `--wait-until <strategy>`
|
|
1290
1292
|
- diff help: `diff screenshot --baseline <f>`
|
|
1291
|
-
- trace help: `trace
|
|
1293
|
+
- trace help: `trace start`
|
|
1294
|
+
- trace help: `trace stop [path]`
|
|
1292
1295
|
- profiler help: `--categories <list>`
|
|
1293
1296
|
- record help: `record restart <path.webm> [url]`
|
|
1294
1297
|
- console help: `--clear`
|
package/docs/RELEASE.md
CHANGED
|
@@ -6,6 +6,7 @@ Related docs:
|
|
|
6
6
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md)
|
|
7
7
|
- [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md)
|
|
8
8
|
- [`ELECTRON.md`](ELECTRON.md)
|
|
9
|
+
- [`platform-smoke.md`](platform-smoke.md)
|
|
9
10
|
- [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md)
|
|
10
11
|
- Bounded `agent_browser` outcome metadata on `details` (`resultCategory`, `successCategory`, `failureCategory`, optional `nextActions`, optional `pageChangeSummary` with per-step summaries on `batch`): contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details); maintainer checklists under “Tool result categories” and “Page-change summaries” in [`../AGENTS.md`](../AGENTS.md)
|
|
11
12
|
- Post-success `get text` selector visibility (`RQ-0074`): optional `details.selectorTextVisibility` / `selectorTextVisibilityAll`, visible warnings, and `inspect-visible-text-candidates*` next actions after read-only visibility probes—[`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md), [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), and [`../AGENTS.md`](../AGENTS.md) maintainer checklist
|
|
@@ -23,6 +24,8 @@ From the repository root:
|
|
|
23
24
|
```bash
|
|
24
25
|
npm install
|
|
25
26
|
npm run doctor
|
|
27
|
+
npm run check:platform-smoke
|
|
28
|
+
npm run smoke:platform:ubuntu-image
|
|
26
29
|
npm run verify -- release
|
|
27
30
|
```
|
|
28
31
|
|
|
@@ -32,18 +35,27 @@ npm run verify -- release
|
|
|
32
35
|
|
|
33
36
|
1. `npm run verify` for generated playbook drift, TypeScript, unit/fake coverage, command-reference generated-block drift, and live command-reference verification against the targeted upstream on `PATH`
|
|
34
37
|
2. `npm run verify -- package-pi`, which first validates package contents via `npm pack --json --dry-run` and then smoke-loads the packed package in Pi isolation
|
|
38
|
+
3. `npm run smoke:platform:doctor` and the full Crabbox matrix from [`platform-smoke.md`](platform-smoke.md): macOS SSH, Ubuntu local-container, and native Windows Parallels targets running fast target-local `platform-build` plus `browser-dogfood-smoke`
|
|
35
39
|
|
|
36
|
-
`npm publish` runs npm’s `prepublishOnly` script from `package.json`, which executes the same `npm run verify -- release` gate and then `npm pack --dry-run`. That concatenated gate is everything in the default `npm run verify` step (generated playbook drift, TypeScript, the unit/fake suite, generated command-reference blocks, and live upstream command-reference sampling against the targeted `agent-browser` on `PATH`) plus the packaged Pi smoke in `package-pi
|
|
40
|
+
`npm publish` runs npm’s `prepublishOnly` script from `package.json`, which executes the same `npm run verify -- release` gate and then `npm pack --dry-run`. That concatenated gate is everything in the default `npm run verify` step (generated playbook drift, TypeScript, the unit/fake suite, generated command-reference blocks, and live upstream command-reference sampling against the targeted `agent-browser` on `PATH`) plus the packaged Pi smoke in `package-pi` and the release-blocking Crabbox platform matrix. Using `npm publish --ignore-scripts` skips that contract intentionally.
|
|
37
41
|
|
|
38
|
-
`prepublishOnly` intentionally does **not** run `npm run verify -- lifecycle`, `npm run verify -- real-upstream`, `npm run verify -- dogfood`, or `npm run verify -- benchmark
|
|
42
|
+
`prepublishOnly` intentionally does **not** run the standalone host-only `npm run verify -- lifecycle`, `npm run verify -- real-upstream`, `npm run verify -- dogfood`, or `npm run verify -- benchmark` modes; those remain separate `npm run verify` modes in [`scripts/project.mjs`](../scripts/project.mjs). The platform matrix includes its own fast target-local build/package gate and browser dogfood suite, and is automated through the `release` slice.
|
|
39
43
|
|
|
40
|
-
For a deterministic real-browser wrapper smoke without model choice in the loop, run:
|
|
44
|
+
For a deterministic host-only real-browser wrapper smoke without model choice in the loop, run:
|
|
41
45
|
|
|
42
46
|
```bash
|
|
43
47
|
npm run verify -- dogfood
|
|
44
48
|
```
|
|
45
49
|
|
|
46
|
-
|
|
50
|
+
For direct Crabbox diagnostics outside the full release compose, run:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
npm run smoke:platform:doctor
|
|
54
|
+
npm run smoke:platform:ubuntu-image
|
|
55
|
+
npm run smoke:platform:all
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
This mode uses the extension harness and the real `agent-browser` on `PATH` against a deterministic local file fixture, then verifies top-level `qa`, `semanticAction`, constrained `job`, screenshot artifact verification, and session close. Use `npm run verify -- dogfood --keep-artifacts` or `--artifact-dir <path>` only while debugging, then delete retained screenshots. This smoke complements, but does not replace, human-readable interactive transcript evidence.
|
|
47
59
|
|
|
48
60
|
Every release also requires interactive `tmux`-driven Pi dogfood with the native `agent_browser` tool against real sites. For extension-focused release smokes, use `pi --no-extensions --no-skills -e .` from the checkout before publish so auto-loaded dogfood/QA skills cannot replace the bounded smoke workflow; run separate skill-enabled dogfood only when validating skill routing or report-generation behavior. Drive prompts with `tmux send-keys`, exercise at least one simple static site and one real documentation/product site, include the higher-level `qa` or `job`/`batch` surfaces when they changed, close every opened browser session, remove screenshots/temp artifacts, and record the outcome in the release notes or support-matrix evidence. Automated localhost, fake-upstream, and deterministic dogfood gates do not replace this human-readable live-site transcript evidence. When `electron.*` surfaces, attached-session diagnostics, or `qa.attached` changed, add a local Electron pass: `electron.list` → `electron.launch` (expect isolated profile behavior) → `snapshot -i` or `electron.probe` / `qa.attached` → `electron.cleanup` with the returned `launchId`, verifying status/mismatch guidance if you simulate a dead renderer or stale refs. For dense-dashboard stress coverage, use the [public Grafana stress checklist](#public-grafana-stress-checklist) below; it is a maintainer workflow, not bundled product skill or recipe runtime.
|
|
49
61
|
|
|
@@ -260,7 +272,7 @@ The default unit suite also runs `agentBrowserExtension passes through core comm
|
|
|
260
272
|
- **Missing or extra `details` / `data` keys:** Update `test/fixtures/agent-browser-real-output-shapes.json` in the same change as the wrapper or presentation code that shifts those keys.
|
|
261
273
|
- **Timeouts:** A 120s bound covers the full matrix; repeated timeouts usually mean a hung browser, blocked loopback, or an environment preventing headful/headless launch—check upstream logs and local security tooling before loosening timeouts.
|
|
262
274
|
|
|
263
|
-
The current upstream `agent-browser 0.27.
|
|
275
|
+
The current upstream `agent-browser 0.27.1` `wait --download <path>` saveAs persistence limitation is tracked at [vercel-labs/agent-browser#1300](https://github.com/vercel-labs/agent-browser/issues/1300); until it is fixed, release validation must treat `details.savedFilePath` as upstream-reported metadata and use `details.artifacts[].exists` as the filesystem truth (the contract asserts the requested path is absent on disk while upstream still reports success). If the suite fails because JSON/detail keys drifted, update the wrapper behavior or refresh `test/fixtures/agent-browser-real-output-shapes.json` together with the presentation work that consumes those shapes.
|
|
264
276
|
|
|
265
277
|
Example smoke prompt:
|
|
266
278
|
|
|
@@ -280,7 +292,7 @@ Recommended configured-source lifecycle follow-up:
|
|
|
280
292
|
|
|
281
293
|
## Post-publish install validation
|
|
282
294
|
|
|
283
|
-
After publishing a release, validate the package-first path in isolation. `npm run verify -- release` includes the deterministic fake-binary packaged execution gate, but it does not replace a real-browser installed-package smoke:
|
|
295
|
+
After publishing a release, validate the package-first path in isolation. `npm run verify -- release` includes the deterministic fake-binary packaged execution gate and the pre-publish Crabbox platform matrix, but it does not replace a real-browser installed-package smoke against the published npm package:
|
|
284
296
|
|
|
285
297
|
```bash
|
|
286
298
|
npm exec --package pi-agent-browser-native -- pi-agent-browser-doctor
|
|
@@ -309,7 +321,7 @@ Before publishing:
|
|
|
309
321
|
- run `npm run verify -- real-upstream` for upstream runtime, result-presentation, or managed-session changes
|
|
310
322
|
- confirm both local-checkout modes still work for pre-release validation: isolated `pi --no-extensions -e .` smoke testing for general checkout loading (add `--no-skills` for extension-focused bounded smokes) and configured-source lifecycle validation
|
|
311
323
|
- complete interactive `tmux` live-site extension smoke with `pi --no-extensions --no-skills -e .` and the native `agent_browser` tool (at least one simple static site and one real documentation/product site; include `qa` or `job`/`batch` when those surfaces changed; use the [public Grafana stress checklist](#public-grafana-stress-checklist) when dashboard/diagnostic/artifact behavior changed; close sessions and remove screenshots/temp artifacts; record evidence). Run separate skill-enabled dogfood only when validating skill routing/report-generation behavior—see [Pre-release checks](#pre-release-checks); automated gates are not a substitute
|
|
312
|
-
- rerun `npm run verify -- release`
|
|
324
|
+
- rerun `npm run verify -- release` and confirm the embedded Crabbox `platform-build` plus `browser-dogfood-smoke` matrix passed on `macos`, `ubuntu`, and `windows-native` with artifacts under `.artifacts/platform-smoke/`
|
|
313
325
|
- run `npm run verify -- lifecycle` for configured-source `/reload`, exact `--session-id` relaunch, managed-session continuity, persisted-spill, and Pi failure-patch regression coverage (required before publish; see [Pre-release checks](#pre-release-checks))
|
|
314
326
|
- confirm [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md) still maps every current baseline inventory section to docs, runtime handling, tests, and validation status
|
|
315
327
|
- manually exercise real-browser `/reload` and full restart plus exact `--session-id` relaunch or `/resume` continuity when release risk warrants browser-level confidence beyond the fake upstream harness
|
package/docs/SUPPORT_MATRIX.md
CHANGED
|
@@ -7,6 +7,7 @@ Related docs:
|
|
|
7
7
|
- [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md)
|
|
8
8
|
- [`ELECTRON.md`](ELECTRON.md)
|
|
9
9
|
- [`RELEASE.md`](RELEASE.md)
|
|
10
|
+
- [`platform-smoke.md`](platform-smoke.md)
|
|
10
11
|
- [`REQUIREMENTS.md`](REQUIREMENTS.md)
|
|
11
12
|
|
|
12
13
|
## Purpose
|
|
@@ -25,10 +26,10 @@ When upstream ships a new `agent-browser` or the inventory changes:
|
|
|
25
26
|
|
|
26
27
|
## Audit result
|
|
27
28
|
|
|
28
|
-
- Target upstream: `agent-browser 0.27.
|
|
29
|
+
- Target upstream: `agent-browser 0.27.1` (must match `CAPABILITY_BASELINE.targetVersion` in [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs)).
|
|
29
30
|
- Source of truth: `CAPABILITY_BASELINE.inventorySections` in the same file (stable `id` keys: `skills`, `core-commands`, `state-tabs-frames-dialogs`, `network-storage-artifacts-diagnostics`, `batch-auth-setup-ai`, `options-and-env`).
|
|
30
31
|
- Status: supported for the current wrapper contract after the 2026-05-26 all-command audit.
|
|
31
|
-
- High-priority support gaps: 2026-05-26 audit found sessionless local commands and command-scoped value flags needed sharper wrapper handling; runtime/tests/docs now cover those paths. Remaining upstream-owned caveat: `agent-browser 0.27.
|
|
32
|
+
- High-priority support gaps: 2026-05-26 audit found sessionless local commands and command-scoped value flags needed sharper wrapper handling; runtime/tests/docs now cover those paths. Remaining upstream-owned caveat: `agent-browser 0.27.1` help mentions `wait <selector> --state hidden`, but source parsing does not implement that distinct wait mode, so wrapper docs steer agents to `wait --fn` predicates.
|
|
32
33
|
- Post-`v0.2.29` review state: commits `eb55320` through `86abbfb` add browser guidance/smoke coverage plus `RQ-0086` click-probe reduction, `RQ-0087` same-snapshot form fill batching, `RQ-0088` current-ref fallback on locator misses, `RQ-0089` direct-upstream click mutation investigation, and `RQ-0090` stop-boundary/artifact-path guidance. Verification gates below were rerun on 2026-05-18 after those tasks landed. Constrained `job` (`RQ-0064`), the lightweight `qa` preset (`RQ-0065`), the experimental `sourceLookup` helper (`RQ-0066`), and the experimental `networkSourceLookup` helper (`RQ-0067`) are implemented; see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#job), [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#qa), [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sourcelookup), and [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#networksourcelookup). Reusable browser recipes (`RQ-0068`) are intentionally not adopted as a runtime surface; see [`ARCHITECTURE.md`](ARCHITECTURE.md#no-reusable-recipe-layer-yet).
|
|
33
34
|
|
|
34
35
|
## Open UX/reliability follow-ups from 2026-05-29 agent feedback
|
|
@@ -57,14 +58,15 @@ Re-run the gates below before each release; this table records what the closure
|
|
|
57
58
|
|
|
58
59
|
| Gate | Evidence | Status |
|
|
59
60
|
| --- | --- | --- |
|
|
60
|
-
| Default local gate | `npm run verify` checks generated playbook drift, `tsc --noEmit`, unit/fake tests, generated command-reference blocks, and live command-reference sampling. | Pass on 2026-
|
|
61
|
-
| Real upstream contract | `npm run verify -- real-upstream` runs the localhost fixture matrix against the real installed `agent-browser` matching the baseline. | Pass on 2026-
|
|
61
|
+
| Default local gate | `npm run verify` checks generated playbook drift, `tsc --noEmit`, unit/fake tests, generated command-reference blocks, and live command-reference sampling. | Pass on 2026-06-02 as part of `npm run verify -- release` (`agent-browser 0.27.1` on `PATH`). |
|
|
62
|
+
| Real upstream contract | `npm run verify -- real-upstream` runs the localhost fixture matrix against the real installed `agent-browser` matching the baseline. | Pass on 2026-06-02 (`npm run verify -- real-upstream`, `agent-browser 0.27.1` on `PATH`; updated Web Vitals shape assertions for upstream 0.27.1 structured output). |
|
|
62
63
|
| Packaged Pi smoke | `npm run verify -- package-pi` validates package contents, loads exactly one packaged `agent_browser` tool, and executes fake-upstream `--version`. | Pass on 2026-05-29 (`npm run verify -- package-pi`). |
|
|
63
|
-
| Deterministic dogfood smoke | `npm run verify -- dogfood` (`scripts/verify-agent-browser-dogfood.ts`) drives the native wrapper against
|
|
64
|
+
| Deterministic dogfood smoke | `npm run verify -- dogfood` (`scripts/verify-agent-browser-dogfood.ts`) drives the native wrapper against a local file fixture through top-level `qa`, `semanticAction`, constrained `job`, screenshot artifact verification, and session close with the real `agent-browser` on `PATH`. | Pass on 2026-06-02 (`npm run verify -- dogfood`, `agent-browser 0.27.1`; artifacts cleaned by the harness). |
|
|
64
65
|
| Efficiency benchmark | `npm run verify -- benchmark` runs deterministic browser workflow accounting plus focused benchmark tests, including JSONL sampling fixtures and job/qa/sourceLookup/networkSourceLookup/Electron scenario coverage. | Pass on 2026-05-29 (`npm run verify -- benchmark`). |
|
|
65
|
-
|
|
|
66
|
+
| Crabbox platform smoke | `npm run check:platform-smoke` syntax-checks the harness and cheap invariants. `npm run smoke:platform:all` runs doctor first, then fast target-local `platform-build` (`npm run verify -- platform-target`, pack, clean Pi install) plus `browser-dogfood-smoke` on Crabbox `macos`, `ubuntu`, and `windows-native`; see [`platform-smoke.md`](platform-smoke.md). | Pass on 2026-06-02 (`npm run check:platform-smoke`, `npm run smoke:platform:ubuntu-image`, and `npm run smoke:platform:all`; artifacts cleaned after evidence capture). |
|
|
67
|
+
| `verify -- release` / `prepublishOnly` | `npm run verify -- release` chains the default gate with packaged Pi smoke and the release-blocking Crabbox platform matrix (`verifySteps` `release` in [`scripts/project.mjs`](../scripts/project.mjs)). `package.json` `prepublishOnly` runs that compose before `npm pack --dry-run` during `npm publish`. It intentionally omits standalone lifecycle, real-upstream, host-only dogfood, and benchmark modes—see [`RELEASE.md`](RELEASE.md#pre-release-checks). | Pass on 2026-06-02 (`npm run verify -- release`, including macOS/Ubuntu/native-Windows Crabbox matrix). |
|
|
66
68
|
| Configured-source lifecycle | `npm run verify -- lifecycle` (`scripts/verify-lifecycle.mjs`) drives `/reload`, closes and relaunches Pi with the same exact `--session-id`, checks the JSONL session header id, session continuity, slash-command sentinel tokens (`v1` then `v2` after rewriting the packaged extension to simulate pickup), persisted spill reachability, and real Pi `tool_result` failure-patch semantics for a QA reclassification with a fake upstream on `PATH`. Default Pi model is `zai/glm-5.1`; default per-step wait is **180000 ms** (`DEFAULT_TIMEOUT_MS`); override model with `--model <id>` and waits with `--timeout-ms <ms>`. Passthrough flags in [`scripts/project.mjs`](../scripts/project.mjs): `--keep-artifacts`, `--model`, `--verbose`, and `--timeout-ms` plus a value (for example `npm run verify -- lifecycle --model openai-codex/gpt-5.5:minimal --keep-artifacts --verbose --timeout-ms 600000`). | Pass on 2026-05-29 (`npm run verify -- lifecycle`). Treat any future unexplained red lifecycle gate as a release blocker. |
|
|
67
|
-
| Quick isolated Pi smoke | `pi --no-extensions --no-skills -e . --tools agent_browser` from repo root; native `agent_browser` only. |
|
|
69
|
+
| Quick isolated Pi smoke | `pi --no-extensions --no-skills -e . --tools agent_browser` from repo root; native `agent_browser` only. | Last interactive tmux checkout smoke pass on 2026-05-29 (`agent-browser 0.27.0` at the time). The 2026-06-02 Crabbox matrix now covers clean packed Pi install plus deterministic wrapper dogfood on all required platforms for `agent-browser 0.27.1`; run a new manual tmux smoke before publish when human-readable transcript evidence is required. Broader historical coverage also includes version/help/skills, open/snapshot/click, eval stdin, batch stdin, screenshot, explicit session, `sessionMode: "fresh"`, network requests, console/errors, diff snapshot, stream status/disable, dashboard start/stop, and chat credential-failure pass-through during RQ-0055. |
|
|
68
70
|
|
|
69
71
|
## Baseline checklist by inventory section
|
|
70
72
|
|
package/docs/TOOL_CONTRACT.md
CHANGED
|
@@ -88,7 +88,7 @@ The extension always plans normal browser commands with `--json` prepended in `e
|
|
|
88
88
|
- For Electron desktop apps, prefer top-level electron for wrapper-owned discovery, isolated launch, status, compact probe, and cleanup: list first, treat likely-sensitive annotations as hints rather than enforcement, launch with the default snapshot handoff unless handoff: "tabs" is the safer diagnostic starting point, use electron.probe or snapshot -i/qa.attached for current-session state, and always cleanup the returned launchId when done. electron.launch uses an isolated temporary profile; it does not reuse the app's normal signed-in profile or attach to an already-running authenticated app. For signed-in local app state, host-launch the normal app with --remote-debugging-port when appropriate, then use raw args connect <port|url>; after connect, inspect tab list, select the stable tab id such as tab t2, then run a condition wait or snapshot -i before using refs. close commands (`close`, `quit`, or `exit`) only close the browser/CDP session; leave manually launched app shutdown, profile cleanup, and explicit artifacts to the host owner.
|
|
89
89
|
- For provider or specialized app workflows, load version-matched upstream guidance with skills get agentcore|electron|slack|dogfood|vercel-sandbox through the native tool; add --full when you need references/templates, and use skills get --all only for broad skill audits. Provider launches such as -p ios, --provider browserbase/kernel/browseruse/browserless/agentcore, and iOS --device are upstream-owned setup paths; use sessionMode fresh when switching providers and expect external credentials or local Appium/Xcode setup to be required.
|
|
90
90
|
- For dialogs and frames, use dialog status/accept/dismiss and frame <selector|main> through native args; when --confirm-actions produces a pending confirmation, use details.nextActions or exact confirm <id> / deny <id> calls instead of inventing ids.
|
|
91
|
-
- If a session lands on the wrong page or tab, an interaction changes origin unexpectedly, or an open call returns blocked, blank, or otherwise unexpected results, use tab list / tab <tab-id-or-label> / snapshot -i to recover state before retrying different URLs or fallback strategies. For headed demos, put --headed on the first launch with sessionMode=fresh and verify with screenshot/tab/get-url evidence because tool success cannot prove the OS window is visible to the user. For desktop readiness, prefer real conditions first: wait --text, wait --url, wait --fn, wait --load <state>, wait --download, or qa.attached; for disappearance checks in agent-browser 0.27.
|
|
91
|
+
- If a session lands on the wrong page or tab, an interaction changes origin unexpectedly, or an open call returns blocked, blank, or otherwise unexpected results, use tab list / tab <tab-id-or-label> / snapshot -i to recover state before retrying different URLs or fallback strategies. For headed demos, put --headed on the first launch with sessionMode=fresh and verify with screenshot/tab/get-url evidence because tool success cannot prove the OS window is visible to the user. For desktop readiness, prefer real conditions first: wait --text, wait --url, wait --fn, wait --load <state>, wait --download, or qa.attached; for disappearance checks in agent-browser 0.27.1, use wait --fn predicates instead of stale upstream-help examples like wait <selector> --state hidden. Use electron.probe/status for wrapper-owned launch health or target mismatch. Fixed waits are a last resort, must stay below the wrapper IPC budget (wait 30000 is intentionally blocked), and a successful payload like "waited":"timeout" means elapsed time only—verify completion with an observed condition, fresh snapshot, or screenshot.
|
|
92
92
|
- For feed, timeline, or inbox reading tasks, focus on the main timeline/list region and read the first item there rather than unrelated composer or sidebar content.
|
|
93
93
|
- For read-only browsing tasks, prefer extracting the answer from the current snapshot, structured ref labels, or eval --stdin on the current page before navigating away. Only click into media viewers, detail routes, or new pages when the current view does not contain the needed information.
|
|
94
94
|
- For downloads, prefer download <selector> <path> when an element click should save a file. Do not rely on click alone when you need the downloaded file on disk.
|
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
# Platform smoke testing
|
|
2
|
+
|
|
3
|
+
`pi-agent-browser-native` uses a Crabbox-backed local platform smoke gate to prove the package on macOS, Ubuntu Linux, and native Windows before release.
|
|
4
|
+
|
|
5
|
+
This is a release-blocking gate. Missing Crabbox setup, Docker, macOS SSH, the native Windows template, upstream `agent-browser`, or browser runtime dependencies is a blocked release setup, not a skipped pass.
|
|
6
|
+
|
|
7
|
+
## Required release gate
|
|
8
|
+
|
|
9
|
+
Run the cheap harness checks first, then the full matrix:
|
|
10
|
+
|
|
11
|
+
```sh
|
|
12
|
+
npm run check:platform-smoke
|
|
13
|
+
npm run smoke:platform:ubuntu-image
|
|
14
|
+
npm run smoke:platform:all
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
`smoke:platform:all` runs `smoke:platform:doctor` before any target suite starts. The canonical `npm run verify -- release` gate also runs the same platform doctor and full `macos,ubuntu,windows-native` matrix after default verification and packaged Pi smoke, so `npm publish` cannot pass `prepublishOnly` without the platform gate.
|
|
18
|
+
|
|
19
|
+
Per-target commands are for diagnosis:
|
|
20
|
+
|
|
21
|
+
```sh
|
|
22
|
+
npm run smoke:platform:macos
|
|
23
|
+
npm run smoke:platform:ubuntu
|
|
24
|
+
npm run smoke:platform:windows-native
|
|
25
|
+
npm run verify -- platform-smoke run --target ubuntu --suite platform-build
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Targets
|
|
29
|
+
|
|
30
|
+
| Target | Crabbox provider | Shell contract | Release status |
|
|
31
|
+
| --- | --- | --- | --- |
|
|
32
|
+
| `macos` | `ssh` static localhost | POSIX shell on macOS | Required |
|
|
33
|
+
| `ubuntu` | `local-container` | POSIX shell in a Docker-compatible local container | Required |
|
|
34
|
+
| `windows-native` | `parallels` | native Windows PowerShell over OpenSSH | Required |
|
|
35
|
+
|
|
36
|
+
## Required environment
|
|
37
|
+
|
|
38
|
+
Install Crabbox on the macOS maintainer host and keep it on `PATH`:
|
|
39
|
+
|
|
40
|
+
```sh
|
|
41
|
+
brew install openclaw/tap/crabbox
|
|
42
|
+
crabbox --version
|
|
43
|
+
crabbox providers
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
Use `PLATFORM_SMOKE_CRABBOX=/path/to/crabbox` only when testing a non-default Crabbox binary.
|
|
47
|
+
|
|
48
|
+
Standard configuration knobs:
|
|
49
|
+
|
|
50
|
+
```sh
|
|
51
|
+
PLATFORM_SMOKE_MAC_HOST=localhost
|
|
52
|
+
PLATFORM_SMOKE_MAC_USER="$USER"
|
|
53
|
+
PLATFORM_SMOKE_MAC_WORK_ROOT="/Users/$USER/crabbox/pi-agent-browser-native"
|
|
54
|
+
|
|
55
|
+
# Default local image built by npm run smoke:platform:ubuntu-image.
|
|
56
|
+
PLATFORM_SMOKE_UBUNTU_IMAGE="pi-agent-browser-native-platform:node24-agent-browser0.27.1"
|
|
57
|
+
|
|
58
|
+
PLATFORM_SMOKE_WINDOWS_VM="pi-extension-windows-template"
|
|
59
|
+
PLATFORM_SMOKE_WINDOWS_SNAPSHOT="crabbox-ready"
|
|
60
|
+
PLATFORM_SMOKE_WINDOWS_USER="<windows-ssh-user>"
|
|
61
|
+
PLATFORM_SMOKE_WINDOWS_WORK_ROOT="C:\\crabbox\\pi-agent-browser-native"
|
|
62
|
+
|
|
63
|
+
# Optional: names of secret env vars to redact/forward if future live suites need them.
|
|
64
|
+
PLATFORM_SMOKE_AUTH_ENV=""
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
The Ubuntu target image is derived from `node:24-bookworm`, installs `agent-browser@0.27.1`, installs Debian Chromium through apt, creates a non-root `circleci` user, and sets `AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium`. Rebuild it after upstream rebaselining, or override `PLATFORM_SMOKE_UBUNTU_IMAGE` with an equivalent prepared local image. Do not install `agent-browser` ad hoc inside the Ubuntu smoke command.
|
|
68
|
+
|
|
69
|
+
The configured upstream `agent-browser` baseline is imported from [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs). Target-local browser suites verify that exact `agent-browser` version before running. Bake the exact upstream CLI and browser runtime into the Windows template/snapshot for speed and reproducibility; missing or stale Windows `agent-browser` / browser readiness is a blocked setup, not something the smoke command repairs. The Windows browser suite checks the preinstalled browser cache and prewarms one short local file URL before the extension harness runs.
|
|
70
|
+
|
|
71
|
+
## Target setup expectations
|
|
72
|
+
|
|
73
|
+
Crabbox does not install project runtime tools. The macOS host, Ubuntu image, and Windows template must already provide:
|
|
74
|
+
|
|
75
|
+
- Node/npm at or above the configured Node major baseline in [`platform-smoke.config.mjs`](../platform-smoke.config.mjs).
|
|
76
|
+
- Git and `tar`.
|
|
77
|
+
- Upstream `agent-browser` matching this wrapper’s capability baseline. The Ubuntu target gets it from [`scripts/platform-smoke/linux-image/Dockerfile`](../scripts/platform-smoke/linux-image/Dockerfile); the Windows template gets it from the shared `pi-extension-windows-template` / `crabbox-ready` snapshot.
|
|
78
|
+
- Browser/runtime dependencies needed by upstream `agent-browser`.
|
|
79
|
+
- Native PowerShell and OpenSSH Server on Windows.
|
|
80
|
+
|
|
81
|
+
For Windows, reuse `pi-extension-windows-template` with the shared canonical `crabbox-ready` power-off snapshot. Do not create one-off project VMs. If a reusable tool is missing, update the shared template, verify from a fresh SSH session, remove caches/secrets/checkouts, shut down cleanly, and promote a known-good power-off snapshot.
|
|
82
|
+
|
|
83
|
+
## What the suites prove
|
|
84
|
+
|
|
85
|
+
Each required target runs `platform-build` and `browser-dogfood-smoke` on one Crabbox lease, serially.
|
|
86
|
+
|
|
87
|
+
### `platform-build`
|
|
88
|
+
|
|
89
|
+
1. Verify the target Node major version.
|
|
90
|
+
2. Run `npm ci` in the synced checkout.
|
|
91
|
+
3. Run `npm run verify -- platform-target`, a fast target-local gate covering generated docs, TypeScript, package/platform harness tests, and runtime planning. The full unit/fake suite still runs once in the host default gate before the release matrix starts; target-local smoke must not duplicate that full suite on every OS. Browser subprocess behavior is then exercised by the target-local `browser-dogfood-smoke` suite against the real upstream binary.
|
|
92
|
+
4. Run `npm pack`.
|
|
93
|
+
5. Create a clean target-local Pi project.
|
|
94
|
+
6. Install the packed tarball with `npm install --no-save`.
|
|
95
|
+
7. Run `pi install -l ./node_modules/pi-agent-browser-native` from the clean project.
|
|
96
|
+
8. Run `pi list` and assert the package is registered from the packed install.
|
|
97
|
+
9. Assert the release proof did not use `pi -e .` or `pi --extension .`.
|
|
98
|
+
|
|
99
|
+
### `browser-dogfood-smoke`
|
|
100
|
+
|
|
101
|
+
1. Run `npm ci` in the synced checkout if needed.
|
|
102
|
+
2. Run the deterministic model-free browser smoke through `scripts/verify-agent-browser-dogfood.ts`.
|
|
103
|
+
3. Exercise native wrapper surfaces against the deterministic local file fixture from `scripts/verify-agent-browser-dogfood.ts`: top-level `qa`, `semanticAction`, constrained `job`, screenshot artifact verification, and session close.
|
|
104
|
+
4. Persist the dogfood JSON report and stdout/stderr evidence.
|
|
105
|
+
5. Fail on missing browser artifacts, failed tool calls, leaked secrets, or unclosed sessions.
|
|
106
|
+
|
|
107
|
+
The dogfood suite intentionally uses the checkout harness while `platform-build` proves packed Pi installation. Together they catch OS-specific packaging, install, path, process, browser, and wrapper bugs without using an LLM.
|
|
108
|
+
|
|
109
|
+
## Artifact contract
|
|
110
|
+
|
|
111
|
+
Every target suite writes host-side evidence under:
|
|
112
|
+
|
|
113
|
+
```text
|
|
114
|
+
.artifacts/platform-smoke/<run-id>/<target>/<suite>/
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
Required files include:
|
|
118
|
+
|
|
119
|
+
```text
|
|
120
|
+
summary.json
|
|
121
|
+
artifact-manifest.json
|
|
122
|
+
target.json
|
|
123
|
+
suite.json
|
|
124
|
+
command.txt
|
|
125
|
+
exit-code.txt
|
|
126
|
+
crabbox.stdout.txt
|
|
127
|
+
crabbox.stderr.txt
|
|
128
|
+
crabbox.timing.json
|
|
129
|
+
assertions.json
|
|
130
|
+
failures.md # only when assertions fail
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
`platform-build` also writes:
|
|
134
|
+
|
|
135
|
+
```text
|
|
136
|
+
node-version.txt
|
|
137
|
+
packed-tarball.txt
|
|
138
|
+
packed-node-install.stdout.txt
|
|
139
|
+
packed-node-install.stderr.txt
|
|
140
|
+
pi-install.stdout.txt
|
|
141
|
+
pi-install.stderr.txt
|
|
142
|
+
pi-list.stdout.txt
|
|
143
|
+
pi-list.stderr.txt
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
`browser-dogfood-smoke` also writes:
|
|
147
|
+
|
|
148
|
+
```text
|
|
149
|
+
node-version.txt
|
|
150
|
+
dogfood-artifacts.txt
|
|
151
|
+
dogfood.stdout.txt
|
|
152
|
+
dogfood.stderr.txt
|
|
153
|
+
dogfood-report.json
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Each target also writes a `lease-cleanup` artifact directory with `crabbox.stop.*` files. Cleanup failures are failing test results. Ubuntu and Windows runs also invoke Crabbox cleanup for stale direct-provider state after stopping the owned lease.
|
|
157
|
+
|
|
158
|
+
Passing suites must satisfy:
|
|
159
|
+
|
|
160
|
+
```text
|
|
161
|
+
summary.ok === assertions.ok
|
|
162
|
+
artifact-manifest.missing.length === 0
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
The harness redacts configured secret values and token-like text from persisted artifacts, then fails if a redaction scan still finds raw secrets.
|
|
166
|
+
|
|
167
|
+
## Source of truth
|
|
168
|
+
|
|
169
|
+
- Config: [`platform-smoke.config.mjs`](../platform-smoke.config.mjs)
|
|
170
|
+
- CLI: [`scripts/platform-smoke.mjs`](../scripts/platform-smoke.mjs)
|
|
171
|
+
- Crabbox wrapper: [`scripts/platform-smoke/crabbox-runner.mjs`](../scripts/platform-smoke/crabbox-runner.mjs)
|
|
172
|
+
- Target commands/assertions: [`scripts/platform-smoke/targets.mjs`](../scripts/platform-smoke/targets.mjs)
|
|
173
|
+
- Platform doctor: [`scripts/platform-smoke/doctor.mjs`](../scripts/platform-smoke/doctor.mjs)
|
|
174
|
+
- Artifact helpers: [`scripts/platform-smoke/artifacts.mjs`](../scripts/platform-smoke/artifacts.mjs)
|
|
175
|
+
- Windows build suite: [`scripts/platform-smoke/platform-build-windows.ps1`](../scripts/platform-smoke/platform-build-windows.ps1)
|
|
176
|
+
- Windows browser suite: [`scripts/platform-smoke/browser-dogfood-windows.ps1`](../scripts/platform-smoke/browser-dogfood-windows.ps1)
|
|
@@ -30,7 +30,7 @@ export const QUICK_START_GUIDELINES = [
|
|
|
30
30
|
] as const;
|
|
31
31
|
|
|
32
32
|
export const BRAVE_SEARCH_PROMPT_GUIDELINE =
|
|
33
|
-
"
|
|
33
|
+
"With BRAVE_API_KEY set, use Brave Search via bash/curl to find exact destination URLs, then open the chosen URL with agent_browser; do not browse search results just to locate a target.";
|
|
34
34
|
|
|
35
35
|
export const SHARED_BROWSER_PLAYBOOK_GUIDELINES = [
|
|
36
36
|
"Standard workflow: open the page, snapshot -i, interact using current @refs from that snapshot, and re-snapshot after navigation, scrolling, rerendering, or other major DOM changes because refs are page-scoped; the wrapper fails mutation-prone stale/recycled refs before upstream can silently target a different current-page element.",
|
|
@@ -51,7 +51,7 @@ export const SHARED_BROWSER_PLAYBOOK_GUIDELINES = [
|
|
|
51
51
|
"For Electron desktop apps, prefer top-level electron for wrapper-owned discovery, isolated launch, status, compact probe, and cleanup: list first, treat likely-sensitive annotations as hints rather than enforcement, launch with the default snapshot handoff unless handoff: \"tabs\" is the safer diagnostic starting point, use electron.probe or snapshot -i/qa.attached for current-session state, and always cleanup the returned launchId when done. electron.launch uses an isolated temporary profile; it does not reuse the app's normal signed-in profile or attach to an already-running authenticated app. For signed-in local app state, host-launch the normal app with --remote-debugging-port when appropriate, then use raw args connect <port|url>; after connect, inspect tab list, select the stable tab id such as tab t2, then run a condition wait or snapshot -i before using refs. close commands (`close`, `quit`, or `exit`) only close the browser/CDP session; leave manually launched app shutdown, profile cleanup, and explicit artifacts to the host owner.",
|
|
52
52
|
"For provider or specialized app workflows, load version-matched upstream guidance with skills get agentcore|electron|slack|dogfood|vercel-sandbox through the native tool; add --full when you need references/templates, and use skills get --all only for broad skill audits. Provider launches such as -p ios, --provider browserbase/kernel/browseruse/browserless/agentcore, and iOS --device are upstream-owned setup paths; use sessionMode fresh when switching providers and expect external credentials or local Appium/Xcode setup to be required.",
|
|
53
53
|
"For dialogs and frames, use dialog status/accept/dismiss and frame <selector|main> through native args; when --confirm-actions produces a pending confirmation, use details.nextActions or exact confirm <id> / deny <id> calls instead of inventing ids.",
|
|
54
|
-
"If a session lands on the wrong page or tab, an interaction changes origin unexpectedly, or an open call returns blocked, blank, or otherwise unexpected results, use tab list / tab <tab-id-or-label> / snapshot -i to recover state before retrying different URLs or fallback strategies. For headed demos, put --headed on the first launch with sessionMode=fresh and verify with screenshot/tab/get-url evidence because tool success cannot prove the OS window is visible to the user. For desktop readiness, prefer real conditions first: wait --text, wait --url, wait --fn, wait --load <state>, wait --download, or qa.attached; for disappearance checks in agent-browser 0.27.
|
|
54
|
+
"If a session lands on the wrong page or tab, an interaction changes origin unexpectedly, or an open call returns blocked, blank, or otherwise unexpected results, use tab list / tab <tab-id-or-label> / snapshot -i to recover state before retrying different URLs or fallback strategies. For headed demos, put --headed on the first launch with sessionMode=fresh and verify with screenshot/tab/get-url evidence because tool success cannot prove the OS window is visible to the user. For desktop readiness, prefer real conditions first: wait --text, wait --url, wait --fn, wait --load <state>, wait --download, or qa.attached; for disappearance checks in agent-browser 0.27.1, use wait --fn predicates instead of stale upstream-help examples like wait <selector> --state hidden. Use electron.probe/status for wrapper-owned launch health or target mismatch. Fixed waits are a last resort, must stay below the wrapper IPC budget (wait 30000 is intentionally blocked), and a successful payload like \"waited\":\"timeout\" means elapsed time only—verify completion with an observed condition, fresh snapshot, or screenshot.",
|
|
55
55
|
"For feed, timeline, or inbox reading tasks, focus on the main timeline/list region and read the first item there rather than unrelated composer or sidebar content.",
|
|
56
56
|
"For read-only browsing tasks, prefer extracting the answer from the current snapshot, structured ref labels, or eval --stdin on the current page before navigating away. Only click into media viewers, detail routes, or new pages when the current view does not contain the needed information.",
|
|
57
57
|
"For downloads, prefer download <selector> <path> when an element click should save a file. Do not rely on click alone when you need the downloaded file on disk.",
|