@parcha/pi-browser 0.1.0-beta.35 → 0.1.0-beta.36

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -2,6 +2,18 @@
2
2
 
3
3
  ## Unreleased
4
4
 
5
+ ## 0.1.0-beta.36 (2026-11-13)
6
+
7
+ - The browser tool has one stack: whether the agent's computer is a browser or the tool plane picks the vendor, every session is a browser computer of the computer registry's providers, made at the tool's call under the run's labels, and pi-browser's raw-CDP driver (`cdpDriver()`) drives it. Stagehand drives no run's browser now. Breaking: Browserbase and Kernel browsers now refuse run code and file uploads (browser_upload), and their snapshot is the raw-CDP list; every browser computer uses one driver. Breaking: pi-browser's `./providers/browserbase` and `./providers/kernel` exports and its `@browserbasehq/sdk` peer dependency are gone (a hosted browser is a browser computer provider of `@parcha/agentrun-computer`), and so are Browserbase's and Kernel's `create(spec, { settings })` vendor-field bag and its `BrowserbaseSessionSettings` and `KernelBrowserSettings` types (put a session's options in `spec.browser`).
8
+ - `@parcha/agentrun-computer`: a browser's own options ride the spec, `spec.browser` (`viewport`, `profile: { pointer, save }`, `stealth`, `headless`), and `capabilities().browser` declares what a provider's sessions offer (`timeout`, `profiles: { settleMs }`, `stealth`, `headless`, `downloads: { path }`). `check()` refuses an option the provider does not declare, in words. Browserbase and Kernel gain the optional `profiles { create, remove }` member, and Browserbase `downloads { list, read }`. Local Chrome declares `{ timeout: "wall-clock" }` and opens its window at the spec's viewport.
9
+ - `@parcha/pi-browser`: `ProviderCaps` is `profiles`, `sharedBrowser`, `settings`, `fixedSession` and `navigationAllow`; the session vault keeps a local Chrome's sessions alone. `cdpCall` is exported from the package root.
10
+ - agentrun-serve: on Browserbase and Kernel, the browser tool's `run` now takes actions only, as it does on Local Chrome (code is refused in words, and navigation is a `goto` action), its `snapshot` is the raw-CDP driver's, and `browser_upload` is refused (the driver sets no file input). A session carries the run's labels and its custody tag, as an agent's browser computer's does.
11
+ - `ScreenshotOptions.scale` (and `FakeScreenshotOptions.scale`) is a number, the fraction of the device-pixel capture to take, in place of `"css" | "device"`; a driver that passed `scale: "css"` passes `1 / devicePixelRatio`, and a custom `PageDriver.screenshot` honors a number below 1, else `captureWithinBudget` throws `ScreenshotBudgetError` for a capture over budget that it does not take smaller.
12
+
13
+ A screenshot over the 2,000 px long edge now shrinks to fit (CDP `clip.scale`) before its JPEG quality drops, so a long page on a device scale of 2 (stripe.com/pricing, a 2576x1248 viewport) comes back as a 1977x969 viewport in place of `ScreenshotBudgetError`; a full page whose fit would leave its short edge under 600 px is taken as the viewport, and the error is left for a browser that gives no capture that fits. On the runner, an image block in a tool's answer whose data is a PNG, JPEG, GIF or WebP of the type it names, with no held key in its bytes, is no longer cut to `[redacted]` by the run's key cut (any other `data` is cut as before) (which sent the model a `data:image/png;base64,[redacted]` URL and failed the run with a 400), and every image a model request carries is checked where it is attached: one with an unknown type, no data, data that is not base64, bytes of another type or a header that does not say its size goes as `[the image could not be attached: <why>]`.
14
+ - A run that asks its person a question with its browser open keeps that browser while it waits, for the person's window, `AGENTRUN_BROWSER_PERSON_WAIT_MIN` (15 minutes by default; the same setting now bounds a take-over with no input and an agent's hand-over ask). Before, the browser's own 3-minute idle release let it go while the run waited, so the page, the live view and Take over were gone just when the person might use them. At the window's end the browser is released with a timeline row saying so ("The browser was released after 15 min waiting for you"), and an answer after that opens a new browser. The Screen tab says the browser was released, and when, instead of "Waiting for the browser" (GET /v2/runs/{id}/computer's `none` carries `ended: {at, why}`). pi-browser's extension handle gains `keepForPerson(tag)`, which holds a session from its idle release until let go.
15
+ - The server and Pi browser READMEs state the Node minimum the packages enforce: 24.22.3 (the server image and the test suites run on it), not Node 22.
16
+
5
17
  ## 0.1.0-beta.35 (2026-11-11)
6
18
 
7
19
  - The `use_website` tool is removed from the durable adapter: no host passed `sites`. Gone with it: `createBrowserExtension`'s `sites` option, the handle's `docs.Children`, the root exports `useWebsite`, `useWebsiteContract` and `localSites`, and the types `SiteInfo`, `SiteRunPort` and `ChildRunner`. A host that wants an agent to hand a task to another agent runs that agent itself.
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  Your agent gets its own browser, and you can take over and hand it back.
4
4
 
5
- - **Its own browser.** The agent drives a Chrome of its own: a local Chrome, Browserbase or Kernel. It reads pages, clicks, types and takes screenshots. A local Chrome needs no key.
5
+ - **Its own browser.** The agent drives a Chrome of its own: a local Chrome, or a hosted browser its host provides (agentrun-serve's Browserbase and Kernel, through its computer providers). It reads pages, clicks, types and takes screenshots. A local Chrome needs no key.
6
6
  - **You can take over and hand it back.** A host mounts the live view in a few lines, and then a person sees the agent's page, uses the mouse and keys, and lets the agent go on. While the person is driving, the agent's browser calls are refused. The agent can also ask a person for a step only a person can do, and wait for the answer. See [Take over and hand back](#take-over-and-hand-back). For now, use the live view with test accounts on practice sites only: the model's own page code can read what a person types there.
7
7
  - **Credentials you register stay out of what the model sees.** Credentials you register, and keys of the common shapes, are replaced with `[redacted]` in every tool result, failure, filed page and log. See [Credentials](#credentials).
8
8
  - **Every page the agent reads is filed as evidence.** The file holds the page's URL, its title and its SHA-256.
@@ -76,23 +76,23 @@ It prints the page's accessibility tree, the same text `snapshot` gives a model:
76
76
 
77
77
  ## Demos
78
78
 
79
- Three runnable demos live in `examples/` in the repository. They are not in the npm package. Build once with `npm ci && npm run build`, then run one from the repository root. Each needs Chrome or Chromium (`CHROME_PATH` names it) except where it says otherwise.
79
+ Two runnable demos live in `examples/` in the repository. They are not in the npm package. Build once with `npm ci && npm run build`, then run one from the repository root. Each needs Chrome or Chromium (`CHROME_PATH` names it) except where it says otherwise.
80
80
 
81
81
  | Demo | Run | What it shows |
82
82
  |---|---|---|
83
- | [`examples/one-script-three-providers.mjs`](examples/one-script-three-providers.mjs) | `node packages/pi-browser/examples/one-script-three-providers.mjs https://example.com` | The same script on a local Chrome, Browserbase or Kernel. `PI_BROWSER_PROVIDER=browserbase` needs `BROWSERBASE_API_KEY` and the Browserbase SDK; `kernel` needs `KERNEL_API_KEY`. The tests run the local Chrome path only. |
84
83
  | [`examples/hold-an-irreversible-action.mjs`](examples/hold-an-irreversible-action.mjs) | `node packages/pi-browser/examples/hold-an-irreversible-action.mjs` | Under the `ask` policy a form post is held until a person approves it. The demo refuses, and the local server it posts to counts zero requests. |
85
84
  | [`examples/crash-and-resume.mjs`](examples/crash-and-resume.mjs) | `node packages/pi-browser/examples/crash-and-resume.mjs`, twice | The first run sets a page title and kills itself with SIGKILL between two tool calls. The second recovers the same Chrome from the run's own store and reads the title back. A scripted model stands in for a real one. It uses the pi-durable packages the repository already installs. Delete `$TMPDIR/pi-browser-crash-demo` to start over. |
86
85
 
87
86
  ## Providers
88
87
 
89
- The same tools run on three kinds of browser. You pick one when you set up the package.
88
+ The same tools run on any browser a `BrowserProvider` gives. The package carries the local one; a hosted browser is the
89
+ host's, through a provider of its own (agentrun-serve drives Browserbase and Kernel through its computer providers, so the
90
+ package names no vendor).
90
91
 
91
92
  | Provider | What it is | What it needs |
92
93
  |---|---|---|
93
94
  | Local Chrome (`/providers/cdp`) | A Chrome the package starts and stops by exact pid, with its own profile directory and a free debugging port. It can also attach to a Chrome you already run, at a CDP endpoint. | Chrome or Chromium 137 or later. No key. |
94
- | Browserbase (`/providers/browserbase`) | A Browserbase session, through its SDK. | `BROWSERBASE_API_KEY` and the `@browserbasehq/sdk` peer. |
95
- | Kernel (`/providers/kernel`) | A Kernel browser, through its REST API. | `KERNEL_API_KEY`. No SDK. |
95
+ | A host's own (`BrowserProvider`, `core/host.ts`) | Any browser that answers CDP: the host makes, finds and releases its sessions and hands over a CDP URL. | Whatever the host's vendor needs. |
96
96
 
97
97
  The local Chrome's options, in `cdpProvider({ chrome: { ... } })`:
98
98
 
@@ -103,7 +103,7 @@ The local Chrome's options, in `cdpProvider({ chrome: { ... } })`:
103
103
  | `args` | Extra Chrome flags, added after the package's own. |
104
104
  | `sandbox` | Chrome's own sandbox, on unless `false`. When absent, `PI_BROWSER_NO_SANDBOX=1` turns it off. Turn it off only where Chrome cannot start its sandbox (root, a container, restricted user namespaces), and only for pages you trust. A Chrome that exits before it opens its debugging port fails the create with its last lines of stderr, and with this hint when it aborted or named the sandbox. |
105
105
 
106
- A host passes the provider it wants; see [For hosts](#for-hosts). `examples/one-script-three-providers.mjs` runs one script on all three.
106
+ A host passes the provider it wants; see [For hosts](#for-hosts).
107
107
 
108
108
  ## Take over and hand back
109
109
 
@@ -187,12 +187,11 @@ This section says where a profile's state lives. A host can ask for a browser pr
187
187
  Where the profile lives:
188
188
 
189
189
  - **Local Chrome:** a directory `.profiles/<name>` under the profile root you give the provider. A session that saves runs on that directory. A session that does not save runs on a copy, and the profile stays as it was.
190
- - **Browserbase:** a Browserbase context. The provider keeps the map from profile name to context id in a file under the user's config directory, or in the store you pass.
191
- - **Kernel:** a Kernel profile of that name, created if it is not there.
190
+ - **A host's provider:** wherever its vendor keeps one (agentrun-serve opens a vendor's saved profile by the pointer it holds for the run's sign-in).
192
191
 
193
192
  It is never your own browser's profile.
194
193
 
195
- One session at a time may save to a profile in one process. A second saver is refused, and the message says why. Across processes, the last session to end wins. A profile name and a raw Browserbase context id cannot be asked for together.
194
+ One session at a time may save to a profile in one process. A second saver is refused, and the message says why. Across processes, the last session to end wins.
196
195
 
197
196
 
198
197
  ## The tools
@@ -259,7 +258,6 @@ Stagehand, undici, zod and the two HTML packages are dependencies. Three optiona
259
258
  | Peer | Needed for |
260
259
  |---|---|
261
260
  | `@earendil-works/pi-durable` and `@earendil-works/chord` | `/durable`, the adapter for a pi-durable Harness. |
262
- | `@browserbasehq/sdk` | the Browserbase provider. Kernel needs no SDK. |
263
261
 
264
262
  The pi peers are `@earendil-works/pi-durable` and `chord` `1.5.0` exactly. CI tests pi 1.5.0.
265
263
 
@@ -284,7 +282,9 @@ The pi peers are `@earendil-works/pi-durable` and `chord` `1.5.0` exactly. CI te
284
282
  sealed: no inference, no cache, and no trace endpoint, so no trace is exported. Stagehand attaches one client to a browser
285
283
  at a time: closing a driver disconnects it, so the next attach is allowed.
286
284
  - `/driver/cdp`: `cdpDriver()`, the driver of every AgentRun browser computer (Browserbase's, Kernel's, or the computer
287
- package's `local-chrome` provider's Chrome on this machine), over its CDP URL. It is raw CDP: no Stagehand, and nothing loaded into the browser. `snapshot` is Chrome's
285
+ package's `local-chrome` provider's Chrome on this machine), over its CDP URL. Browserbase and Kernel browsers now
286
+ refuse run code and file uploads (`browser_upload`), and their snapshot is the raw-CDP list; every browser computer
287
+ uses one driver, this one, which runs no code and sets no file input. It is raw CDP: no Stagehand, and nothing loaded into the browser. `snapshot` is Chrome's
288
288
  accessibility tree with the same "frame-node" ids (`[0-42]`: the frame's place, then Chrome's backend node id). `run`
289
289
  takes actions only. Each action is scrolled into view, hit-tested at its centre, and pressed and released there, or
290
290
  typed with inserted text and a real last key. Each answers one line: `[ok] click [0-42] button "Add" -> page changed`
@@ -292,9 +292,8 @@ The pi peers are `@earendil-works/pi-durable` and `chord` `1.5.0` exactly. CI te
292
292
  `{"op":"goto","url":"https://..."}`. A frame another site runs in its own process (a cross-origin iframe) is not read:
293
293
  the snapshot names each one it left out. Adapted from Browser Use Pi's `ax.ts`, `cdp.ts` and `page.ts` under its MIT
294
294
  license, in `src/vendor/browser-use-pi/` (see NOTICE and that folder's `UPSTREAM.json`).
295
- - `/providers/browserbase`, `/providers/kernel`, `/providers/cdp`: where browsers come from. Browserbase through its SDK,
296
- with the credential-proxy dial and the connect-URL rewrite. Kernel through its REST API on the package's own HTTP agent.
297
- Any CDP endpoint, or a local Chrome the provider starts and kills by exact pid.
295
+ - `/providers/cdp`: where the package's own browsers come from: any CDP endpoint, or a local Chrome the provider starts and
296
+ kills by exact pid. A hosted browser is the host's provider (agentrun-serve's, over its computer providers).
298
297
  - `/providers/chrome-args`: `chromeArgs()`, the arguments the cdp provider starts a local Chrome with, and `extensionOrigin()`,
299
298
  the one Origin its DevTools socket allows (Stagehand's extension). A host that starts its own Chrome for Stagehand takes them
300
299
  from here rather than copying Stagehand's `--remote-allow-origins=*`. `findChrome()` finds the Chrome to start: the one
@@ -334,8 +333,8 @@ await browser.reconcile(harness, () => true, context); // settles what a dead
334
333
  `createBrowserExtension` takes no key and sends nothing anywhere but the provider you give it. The options:
335
334
 
336
335
  - `driver`: `stagehandDriver()`, or your own `DriverFactory`.
337
- - `provider`: for example Browserbase from the run's env (`BROWSERBASE_API_KEY` or `BB_API_KEY`, the proxy URLs and the
338
- project id), with your own metadata stamped on every session.
336
+ - `provider`: the local Chrome (`cdpProvider`), or your own `BrowserProvider` over a hosted vendor, with your own metadata
337
+ stamped on every session.
339
338
  - `evidence`: a sink that files each read where your receipts already are.
340
339
  - `onSession`: the host channel. It receives each session's row and, only there, the live-view URLs. Nothing in the run log, a
341
340
  result or a log line carries them.
@@ -389,10 +388,6 @@ Two things to know before you give a model a browser:
389
388
 
390
389
  - **What a model may reach is your decision.** `publicFetch` refuses private addresses, but a driven browser goes
391
390
  wherever its network goes. A host that must not let a model reach internal addresses restricts that network or the URLs it passes.
392
- - **The Browserbase provider changes the process.** Its first call that needs the Browserbase client adds Browserbase's hosts to `NO_PROXY` and `no_proxy`, and
393
- the first attach replaces `globalThis.WebSocket` with a wrapper that dials a bound credential proxy instead of the raw
394
- connect URL. Both are process-wide. The wrapper passes every other URL through unchanged. Importing the package does
395
- neither.
396
391
 
397
392
  ## Development
398
393
 
@@ -406,7 +401,7 @@ The real-Chrome suites (`stagehand_real_chrome`, `cdp_local_chrome`, `effects-ch
406
401
  `readme_example`) skip with a typed reason where no Chrome can load Stagehand's extension, and fail instead under
407
402
  `AGENTRUN_REQUIRE_LOCAL_CHROME=1`, which CI sets. The crash matrix (`test/custody.test.mjs`) SIGKILLs a child process at each
408
403
  cut point of a session's life against the shared fake and reopens the run's SQLite file. The wire checks (`test/wire/`) plant
409
- sentinel credentials in a fake provider and scan every byte a run produced. The suites run on Node 22 and 24.
404
+ sentinel credentials in a fake provider and scan every byte a run produced. The suites run on Node 24.22.3, the minimum the package supports.
410
405
 
411
406
  Apache-2.0. Copyright 2026 Parcha Labs, Inc. `NOTICE` records the Stagehand (MIT) code this package copies: the Chrome launch
412
407
  flags and the vendored facade.
@@ -10,7 +10,7 @@ export declare const PERSON_HAD = "This call waited while the person had the bro
10
10
  export declare const PERSON_HAD_UNUSED = "This call waited while the person had the browser: they took it over through the live view and handed it back without using it, so the page is as you left it.";
11
11
  /** The mark (in a call's result details) of an act the person's take made stale: the person drove the session through the
12
12
  * live view while the call waited, so the snapshot its element ids came from no longer stands. Its timeline row reads
13
- * neutral, in plain words (pi-durable-agent's timeline), never as a failure. */
13
+ * neutral, in plain words (serve's timeline), never as a failure. */
14
14
  export declare const PAGE_CHANGED_WHILE_HELD = "page_changed_while_held";
15
15
  /** Told to the model by such an act: nothing was done, and its next step is a snapshot. */
16
16
  export declare const PAGE_CHANGED_FOR_MODEL = "The person changed the page while they had the browser, so this action was not done. Take a snapshot first, then act on the page as it is now.";
@@ -10,7 +10,7 @@ export const PERSON_HAD = "This call waited while the person had the browser: th
10
10
  export const PERSON_HAD_UNUSED = "This call waited while the person had the browser: they took it over through the live view and handed it back without using it, so the page is as you left it.";
11
11
  /** The mark (in a call's result details) of an act the person's take made stale: the person drove the session through the
12
12
  * live view while the call waited, so the snapshot its element ids came from no longer stands. Its timeline row reads
13
- * neutral, in plain words (pi-durable-agent's timeline), never as a failure. */
13
+ * neutral, in plain words (serve's timeline), never as a failure. */
14
14
  export const PAGE_CHANGED_WHILE_HELD = "page_changed_while_held";
15
15
  /** Told to the model by such an act: nothing was done, and its next step is a snapshot. */
16
16
  export const PAGE_CHANGED_FOR_MODEL = "The person changed the page while they had the browser, so this action was not done. Take a snapshot first, then act on the page as it is now.";
@@ -92,12 +92,8 @@ export type AttachTarget = {
92
92
  extensionCdpUrl?: string;
93
93
  };
94
94
  export type ProviderCaps = {
95
- timeoutModel: "wall-clock" | "inactivity";
96
- maxLifetimeS: number;
97
- survivesDisconnect: boolean;
98
- releaseIsAsync: boolean;
95
+ /** The provider can open a session on a saved sign-in (`SessionSpec.profile`); without it such a session is refused. */
99
96
  profiles: boolean;
100
- extension: "uploaded-per-launch" | "preinstalled" | "loaded-locally";
101
97
  /** Every session attaches to the same browser (an endpoint the host named), so two at once would drive the same pages. */
102
98
  sharedBrowser?: boolean;
103
99
  /** The session settings this provider applies; a setting left out is applied. */
@@ -140,13 +136,6 @@ export type RemoteFile = {
140
136
  /** What a session vault may keep of this provider's sessions. Absent: nothing (a browser the package cannot read, or one the host shares). */
141
137
  export type ProviderVault = {
142
138
  kind: "cdp";
143
- } | {
144
- kind: "context";
145
- provider: "browserbase" | "kernel";
146
- /** True when this provider writes the context ids it makes to a clear file: a vault refuses such a provider, so an id never lands outside it. */
147
- contextIdsInClear: boolean;
148
- /** The context or profile this session runs on, or null. A capability: it goes to the host's vault only. */
149
- contextOf(ref: LeaseRef): string | null;
150
139
  };
151
140
  export interface BrowserProvider {
152
141
  readonly name: string;
@@ -1,5 +1,8 @@
1
1
  export declare const SCREENSHOT_MAX_EDGE_PX = 2000;
2
2
  export declare const SCREENSHOT_MAX_BASE64_BYTES = 900000;
3
+ /** A full page shrunk to the long edge that would be narrower than this (its short edge) is unreadable: the viewport is
4
+ * taken in its place. */
5
+ export declare const SCREENSHOT_MIN_SHORT_EDGE_PX = 600;
3
6
  export type ImageSize = {
4
7
  type: "png" | "jpeg";
5
8
  width: number;
@@ -13,12 +16,13 @@ export type Clip = {
13
16
  width: number;
14
17
  height: number;
15
18
  };
16
- /** `scale: "css"` takes one image pixel per CSS pixel, so a high-density display does not double the size. `clip` captures that region alone. */
19
+ /** `clip` captures that region alone, `fullPage` the whole document. `scale` is the image's size as a share of the
20
+ * browser's own capture of the same region (its device pixels): 0.5 is half as wide and half as high; absent, 1. */
17
21
  export type ScreenshotOptions = {
18
22
  fullPage?: boolean;
19
23
  type?: "png" | "jpeg";
20
24
  quality?: number;
21
- scale?: "css" | "device";
25
+ scale?: number;
22
26
  clip?: Clip;
23
27
  };
24
28
  export type Capture = (options: ScreenshotOptions) => Promise<{
@@ -31,14 +35,38 @@ export type BudgetedScreenshot = {
31
35
  size: ImageSize;
32
36
  options: ScreenshotOptions;
33
37
  adjusted: {
34
- why: "long_edge" | "base64_bytes";
38
+ why: "long_edge" | "base64_bytes" | "unreadable";
35
39
  first: ImageSize | null;
36
40
  } | null;
37
41
  };
42
+ /** What CDP's Page.getLayoutMetrics says of a page, in CSS pixels: where the viewport is scrolled to and its size, and
43
+ * the whole document's size. */
44
+ export type Layout = {
45
+ cssVisualViewport: {
46
+ pageX: number;
47
+ pageY: number;
48
+ clientWidth: number;
49
+ clientHeight: number;
50
+ };
51
+ cssContentSize: {
52
+ width: number;
53
+ height: number;
54
+ };
55
+ };
56
+ /** CDP's Page.captureScreenshot parameters for `options` on a page laid out as `layout`. Chrome scales a clip alone, its
57
+ * image `clip.scale` times its device pixels: a clip (from the viewport) is moved to the document, a full page is the
58
+ * document's clip captured beyond the viewport, and a scaled viewport is the viewport's own clip. */
59
+ export declare function captureParams(options: ScreenshotOptions, layout: Layout): Record<string, unknown>;
38
60
  export declare class ScreenshotBudgetError extends Error {
39
61
  name: string;
40
62
  }
41
- /** Over budget, retakes as a viewport JPEG at CSS scale, quality 40, 25, 10 (of the same clip, when there is one); throws ScreenshotBudgetError. */
63
+ /**
64
+ * A capture as requested, held to the budget. Over the long edge, the same region is retaken scaled to fit it (a full
65
+ * page that would come out narrower than SCREENSHOT_MIN_SHORT_EDGE_PX is the viewport instead, itself scaled to fit);
66
+ * over the byte budget, as a JPEG at quality 40, 25 and 10 (at most the requested quality), then smaller still. A capture
67
+ * whose size cannot be read is retaken once as a viewport JPEG. Throws ScreenshotBudgetError only when no capture it can
68
+ * measure is made, or none fits within MAX_CAPTURES.
69
+ */
42
70
  export declare function captureWithinBudget(capture: Capture, requested: ScreenshotOptions, budget?: {
43
71
  maxEdgePx: number;
44
72
  maxBase64Bytes: number;
@@ -1,7 +1,11 @@
1
1
  // Tool images are stored verbatim and resent on every later request, and routes reject oversize ones: a capture is held
2
- // to a long edge and a base64 size.
2
+ // to a long edge and a base64 size, by shrinking its pixels and then its JPEG quality, never by failing while a capture
3
+ // can be made.
3
4
  export const SCREENSHOT_MAX_EDGE_PX = 2_000;
4
5
  export const SCREENSHOT_MAX_BASE64_BYTES = 900_000;
6
+ /** A full page shrunk to the long edge that would be narrower than this (its short edge) is unreadable: the viewport is
7
+ * taken in its place. */
8
+ export const SCREENSHOT_MIN_SHORT_EDGE_PX = 600;
5
9
  const PNG_SIGNATURE = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a];
6
10
  /** JPEG start-of-frame markers; C4 (DHT), C8 (JPG) and CC (DAC) are not frames. */
7
11
  const JPEG_SOF = new Set([0xc0, 0xc1, 0xc2, 0xc3, 0xc5, 0xc6, 0xc7, 0xc9, 0xca, 0xcb, 0xcd, 0xce, 0xcf]);
@@ -34,27 +38,88 @@ export function imageSize(bytes) {
34
38
  }
35
39
  return null;
36
40
  }
41
+ /** CDP's Page.captureScreenshot parameters for `options` on a page laid out as `layout`. Chrome scales a clip alone, its
42
+ * image `clip.scale` times its device pixels: a clip (from the viewport) is moved to the document, a full page is the
43
+ * document's clip captured beyond the viewport, and a scaled viewport is the viewport's own clip. */
44
+ export function captureParams(options, layout) {
45
+ const format = options.type ?? "png";
46
+ const scale = options.scale ?? 1;
47
+ const view = layout.cssVisualViewport;
48
+ const region = options.clip ? { ...options.clip, x: options.clip.x + view.pageX, y: options.clip.y + view.pageY }
49
+ : options.fullPage ? { x: 0, y: 0, width: Math.ceil(layout.cssContentSize.width), height: Math.ceil(layout.cssContentSize.height) }
50
+ : scale !== 1 ? { x: view.pageX, y: view.pageY, width: view.clientWidth, height: view.clientHeight }
51
+ : null;
52
+ return {
53
+ format,
54
+ // CDP takes a quality for jpeg only, and only as an integer.
55
+ ...(format === "jpeg" && options.quality !== undefined ? { quality: Math.round(options.quality) } : {}),
56
+ ...(options.fullPage && !options.clip ? { captureBeyondViewport: true } : {}),
57
+ ...(region ? { clip: { ...region, scale } } : {}),
58
+ };
59
+ }
37
60
  export class ScreenshotBudgetError extends Error {
38
61
  name = "ScreenshotBudgetError";
39
62
  }
40
- /** Over budget, retakes as a viewport JPEG at CSS scale, quality 40, 25, 10 (of the same clip, when there is one); throws ScreenshotBudgetError. */
63
+ /** The most captures one screenshot takes before it gives up. */
64
+ const MAX_CAPTURES = 12;
65
+ const QUALITIES = [40, 25, 10];
66
+ /**
67
+ * A capture as requested, held to the budget. Over the long edge, the same region is retaken scaled to fit it (a full
68
+ * page that would come out narrower than SCREENSHOT_MIN_SHORT_EDGE_PX is the viewport instead, itself scaled to fit);
69
+ * over the byte budget, as a JPEG at quality 40, 25 and 10 (at most the requested quality), then smaller still. A capture
70
+ * whose size cannot be read is retaken once as a viewport JPEG. Throws ScreenshotBudgetError only when no capture it can
71
+ * measure is made, or none fits within MAX_CAPTURES.
72
+ */
41
73
  export async function captureWithinBudget(capture, requested, budget = { maxEdgePx: SCREENSHOT_MAX_EDGE_PX, maxBase64Bytes: SCREENSHOT_MAX_BASE64_BYTES }) {
42
74
  const ceiling = requested.type === "png" ? 40 : (requested.quality ?? 40);
43
- const ladder = [
44
- { fullPage: requested.fullPage ?? false, type: requested.type ?? "jpeg", ...(requested.type === "png" ? {} : { quality: requested.quality ?? 40 }), ...(requested.scale ? { scale: requested.scale } : {}), ...(requested.clip ? { clip: requested.clip } : {}) },
45
- ...[40, 25, 10].map((quality) => ({ fullPage: false, type: "jpeg", quality: Math.min(ceiling, quality), scale: "css", ...(requested.clip ? { clip: requested.clip } : {}) })),
46
- ];
47
- const key = (o) => `${o.fullPage}:${o.type}:${o.quality}:${o.scale}:${o.clip ? `${o.clip.x},${o.clip.y},${o.clip.width},${o.clip.height}` : ""}`;
48
- const attempts = ladder.filter((o, i) => ladder.findIndex((other) => key(other) === key(o)) === i);
49
- let adjusted = null;
50
- for (const options of attempts) {
75
+ // The clip and the scale asked are kept on every capture; a retake only scales further down.
76
+ const clip = { ...(requested.clip ? { clip: requested.clip } : {}), ...(requested.scale === undefined ? {} : { scale: requested.scale }) };
77
+ let taken = 0;
78
+ const take = async (options) => {
79
+ taken += 1;
51
80
  const image = await capture(options);
52
81
  const size = imageSize(Buffer.from(image.data, "base64"));
53
- const why = size === null || Math.max(size.width, size.height) > budget.maxEdgePx ? "long_edge"
54
- : Buffer.byteLength(image.data, "utf8") > budget.maxBase64Bytes ? "base64_bytes" : null;
55
- if (why === null && size)
56
- return { data: image.data, mimeType: image.mimeType, size, options, adjusted };
57
- adjusted ??= { why: why ?? "long_edge", first: size };
82
+ const why = size === null ? "unreadable"
83
+ : Math.max(size.width, size.height) > budget.maxEdgePx ? "long_edge"
84
+ : Buffer.byteLength(image.data, "utf8") > budget.maxBase64Bytes ? "base64_bytes" : null;
85
+ return { image, size, why, options };
86
+ };
87
+ const fit = (size) => budget.maxEdgePx / Math.max(size.width, size.height);
88
+ const viewport = (options) => ({ fullPage: false, type: "jpeg", quality: Math.min(ceiling, options.type === "jpeg" ? options.quality ?? 40 : 40), ...clip });
89
+ let shot = await take({ fullPage: requested.fullPage ?? false, type: requested.type ?? "jpeg", ...(requested.type === "png" ? {} : { quality: ceiling }), ...clip });
90
+ const adjusted = shot.why === null ? null : { why: shot.why, first: shot.size };
91
+ if (shot.why === "unreadable")
92
+ shot = await take(viewport(shot.options));
93
+ // A full page that fits only narrower than a page can be read: the viewport, which the steps below fit.
94
+ if (shot.why === "long_edge" && shot.options.fullPage && shot.size && Math.min(shot.size.width, shot.size.height) * fit(shot.size) < SCREENSHOT_MIN_SHORT_EDGE_PX) {
95
+ shot = await take(viewport(shot.options));
96
+ }
97
+ while (shot.why !== null) {
98
+ if (shot.why === "unreadable" || !shot.size)
99
+ throw new ScreenshotBudgetError("The browser's capture is not a PNG or JPEG whose size can be read, and a viewport JPEG retake is not either.");
100
+ if (taken >= MAX_CAPTURES)
101
+ throw new ScreenshotBudgetError(`No capture fits ${budget.maxEdgePx} px on its long edge and ${budget.maxBase64Bytes} base64 bytes after ${taken} captures; the last was ${shot.size.width}x${shot.size.height}.`);
102
+ const { options, size } = shot;
103
+ const scale = options.scale ?? 1;
104
+ let next;
105
+ if (shot.why === "long_edge") {
106
+ next = { ...options, scale: scale * fit(size) };
107
+ }
108
+ else {
109
+ const quality = options.type === "png" ? undefined : options.quality ?? 40;
110
+ const lower = QUALITIES.map((q) => Math.min(ceiling, q)).find((q) => quality === undefined || q < quality);
111
+ // Its bytes go roughly with its area: a scale that takes them to nine tenths of the budget.
112
+ const smaller = scale * Math.min(0.9, 0.9 * Math.sqrt(budget.maxBase64Bytes / Buffer.byteLength(shot.image.data, "utf8")));
113
+ next = lower !== undefined ? { ...options, type: "jpeg", quality: lower } : { ...options, scale: smaller };
114
+ }
115
+ shot = await take(next);
116
+ // A driver that does not take the scale asked (Stagehand's, past one image pixel per CSS pixel): a full page is the
117
+ // viewport instead, and anything else is as small as this browser makes it.
118
+ if ((next.scale ?? 1) < scale && shot.size && Math.max(shot.size.width, shot.size.height) >= Math.max(size.width, size.height)) {
119
+ if (!options.fullPage)
120
+ throw new ScreenshotBudgetError(`The browser does not take this capture smaller than ${shot.size.width}x${shot.size.height}, which is over ${budget.maxEdgePx} px on its long edge or ${budget.maxBase64Bytes} base64 bytes.`);
121
+ shot = await take(viewport(options));
122
+ }
58
123
  }
59
- throw new ScreenshotBudgetError(`The screenshot stays over ${budget.maxEdgePx} px on its long edge or ${budget.maxBase64Bytes} base64 bytes after viewport JPEG retakes at CSS scale and quality 40, 25 and 10.`);
124
+ return { data: shot.image.data, mimeType: shot.image.mimeType, size: shot.size, options: shot.options, adjusted };
60
125
  }
@@ -421,7 +421,10 @@ export function pageTools({ guards = new Map(), reconnect: timings = RECONNECT,
421
421
  const url = pageUrl(call, await s.driver.url());
422
422
  const receipt = await file(s, call, { tool: "screenshot", args: region ? { ...requested, clip: region.clip, ...region.of } : requested, status: "ok", facts: { final_url: url }, body: Buffer.from(shot.data, "base64"), ext: shot.size.type === "png" ? "png" : "jpg" });
423
423
  const first = shot.adjusted?.first;
424
- const adjusted = shot.adjusted && `adjusted: the first capture${first ? ` (${first.width}x${first.height})` : ""} was over the ${shot.adjusted.why === "long_edge" ? "2,000 px long edge" : "size"} budget; this is ${region ? "the clip" : "the viewport"} at ${shot.size.width}x${shot.size.height}`;
424
+ const over = shot.adjusted?.why === "long_edge" ? "was over the 2,000 px long edge budget" : shot.adjusted?.why === "base64_bytes" ? "was over the size budget" : "could not be measured";
425
+ const taken = shot.options.clip ? "the clip" : shot.options.fullPage ? "the full page" : "the viewport";
426
+ const scaled = shot.options.scale !== undefined && shot.options.scale < 1 ? ", scaled down" : "";
427
+ const adjusted = shot.adjusted && `adjusted: the first capture${first ? ` (${first.width}x${first.height})` : ""} ${over}; this is ${taken} at ${shot.size.width}x${shot.size.height}${scaled}`;
425
428
  const lines = [news, `url: ${url}`, `evidence: ${receipt.path ?? "(not filed)"}`, adjusted].filter(Boolean).join("\n");
426
429
  const media = { kind: "screenshot", path: receipt.bodyPath, mime: shot.mimeType, bytes: Buffer.from(shot.data, "base64").byteLength, width: shot.size.width, height: shot.size.height, ...(region ? { clip: region.clip, ...region.of } : {}), ...(ask ? { ask } : {}) };
427
430
  return { content: [{ type: "text", text: lines }, { type: "image", data: shot.data, mimeType: shot.mimeType }], details: { evidence: receipt.path, sha256: receipt.sha256, media } };
@@ -19,17 +19,7 @@ export type CapturedState = {
19
19
  /** Only cookies the allowed origins would be sent (see `cookieKept`). Partitioned cookies are not kept. */
20
20
  cookies: VaultCookie[];
21
21
  localStorage: Record<string, Record<string, string>>;
22
- }
23
- /** `origins`: the origins the context was captured for. A context holds a whole browser profile, so it is restored only for a request that covers EVERY one of them (`coversEveryOrigin`); one that records none is refused. */
24
- | {
25
- kind: "provider-context";
26
- capturedAt: string;
27
- provider: "browserbase" | "kernel";
28
- contextId: string;
29
- origins?: string[];
30
22
  };
31
- /** Whether `requested` includes every origin in `saved` (as scheme://host[:port], any order): a provider context is all-or-nothing, so a request for some of its origins would bring back the others' sign-ins. */
32
- export declare function coversEveryOrigin(requested: readonly string[], saved: readonly string[] | undefined): boolean;
33
23
  export interface SessionVaultPort {
34
24
  /** At release and at the end of a handoff, and, for a context a new session made, right after the create (so a restart still finds it). The host supplied the origins; nothing outside them was captured.
35
25
  * A capture has a deadline (20 s) after which nothing more is read or written, but a write already handed to this port cannot be recalled: ignore one whose `capturedAt` is older than the one you hold. */
@@ -11,19 +11,6 @@ export function clampSessionCookieAgeMs(requested) {
11
11
  throw new RangeError("the session-cookie age cap is a positive number of milliseconds");
12
12
  return Math.min(requested, MAX_SESSION_COOKIE_AGE_MS);
13
13
  }
14
- /** Whether `requested` includes every origin in `saved` (as scheme://host[:port], any order): a provider context is all-or-nothing, so a request for some of its origins would bring back the others' sign-ins. */
15
- export function coversEveryOrigin(requested, saved) {
16
- if (!saved || saved.length === 0)
17
- return false;
18
- const origin = (value) => { try {
19
- return new URL(value).origin;
20
- }
21
- catch {
22
- return null;
23
- } };
24
- const have = new Set(requested.map(origin));
25
- return saved.every((value) => { const o = origin(value); return o !== null && have.has(o); });
26
- }
27
14
  const SESSION_KEY = /^[A-Za-z0-9][A-Za-z0-9_.:-]{0,127}$/;
28
15
  /** The host's allowed origins, each `scheme://host[:port]` of http or https, normalised and de-duplicated; anything else is refused. */
29
16
  export function vaultOrigins(origins) {
@@ -224,9 +224,9 @@ export function cdpDriver() {
224
224
  const selector = place?.selector ?? null;
225
225
  return page.evaluate(`(${LOCATE_JS}).call(document, ${JSON.stringify(selector)})`);
226
226
  },
227
- screenshot: async ({ type = "png", quality, fullPage, clip }) => {
228
- const data = await page.screenshot({ type, ...(quality === undefined ? {} : { quality }), ...(fullPage === undefined ? {} : { fullPage }), ...(clip ? { clip } : {}) });
229
- return { data, mimeType: type === "jpeg" ? "image/jpeg" : "image/png" };
227
+ screenshot: async (options) => {
228
+ const data = await page.screenshot(options);
229
+ return { data, mimeType: options.type === "jpeg" ? "image/jpeg" : "image/png" };
230
230
  },
231
231
  // One evaluate reads all four, so they are one document's.
232
232
  page: () => page.evaluate(() => ({
@@ -205,12 +205,16 @@ export function pageDriver(stagehand, deadlines) {
205
205
  selector = target.selector;
206
206
  return await facade.run(locateCode(selector));
207
207
  }),
208
+ // Stagehand's page.screenshot scales to its device's pixels or to one per CSS pixel, and only a clip: a scale below 1
209
+ // is a clip (the viewport's, when none is asked) at one image pixel per CSS pixel, which is no smaller than that on a
210
+ // display of one device pixel per CSS pixel. A full page it does not scale (captureWithinBudget takes the viewport).
208
211
  screenshot: async ({ type = "png", quality, fullPage, scale, clip }) => {
212
+ const css = scale !== undefined && scale < 1;
209
213
  // A clip is read from the viewport's corner, and the browser's own is from the document's: the scroll offset is added.
210
- const origin = clip ? await facade.run(`return await page.evaluate("({ x: scrollX, y: scrollY })");`) : null;
214
+ const view = clip || (css && !fullPage) ? await facade.run(`return await page.evaluate("({ x: scrollX, y: scrollY, width: innerWidth, height: innerHeight })");`) : null;
215
+ const region = view && (clip ? { ...clip, x: clip.x + view.x, y: clip.y + view.y } : view);
211
216
  const bytes = await (await active()).screenshot({
212
- type, ...(fullPage === undefined ? {} : { fullPage }), ...(scale === undefined ? {} : { scale }),
213
- ...(clip && origin ? { clip: { ...clip, x: clip.x + origin.x, y: clip.y + origin.y } } : {}),
217
+ type, ...(region ? { clip: region } : fullPage === undefined ? {} : { fullPage }), ...(css ? { scale: "css" } : {}),
214
218
  // CDP takes a quality for jpeg only, and only as an integer.
215
219
  ...(type === "jpeg" && quality !== undefined ? { quality: Math.round(quality) } : {}),
216
220
  });
package/dist/durable.d.ts CHANGED
@@ -107,6 +107,11 @@ export type BrowserExtensionHandle = {
107
107
  close(): Promise<void>;
108
108
  /** How many conversations the extension still holds in-memory state for: a repeat guard, an idle timer or a queued call. */
109
109
  retained(): number;
110
+ /** Keeps session `tag` from its idle release while the run waits for its person's answer (the host bounds that wait):
111
+ * an idle window that ends meanwhile releases nothing. The returned function lets it go, once; when the last keep on
112
+ * the tag goes, its idle window starts again from then. Releases for any other reason (the tool, a close, its end)
113
+ * are not held. */
114
+ keepForPerson(tag: string): () => void;
110
115
  };
111
116
  export { CONFIRM_POLLS, RELEASE_ATTEMPTS } from "./durable/store.js";
112
117
  export { HANDOFF_POLL_MS } from "./core/custody.js";
package/dist/durable.js CHANGED
@@ -238,7 +238,6 @@ async function create(c, scope, overrides, presetTag) {
238
238
  const unsupported = profileRefusal(spec, provider.caps) ?? navigationRefusal(spec, provider);
239
239
  if (unsupported)
240
240
  throw new BrowserFailureError(unsupported);
241
- // The vault may name a context or profile for the provider; the record stored below is made from the spec WITHOUT it, so no context id reaches the store.
242
241
  const plan = await planRestore(c.vault, config.vault, provider, spec);
243
242
  const record = newSessionRecord({ plane: scope.plane, tag, provider: provider.name, spec, at: now(), honours: provider.caps.settings });
244
243
  const row = rowOf(scope.cid, record);
@@ -283,9 +282,6 @@ async function create(c, scope, overrides, presetTag) {
283
282
  const live = { ...record, state: "live", resourceId: ref.id };
284
283
  try {
285
284
  const driver = await connect(c, scope, provider, ref, plan.afterAttach, plan.vaulted, plan.holdBack);
286
- // A context made for this session is given to the vault now, so a restart that loses the provider's memory of it still finds it.
287
- if (plan.fresh)
288
- await c.held.capture(ref.tag);
289
285
  await emit(c, scope.cid, config.label, live, "launched", provider);
290
286
  return { record: live, driver };
291
287
  }
@@ -660,6 +656,10 @@ async function releasePlane(c, conversation, reason, context, still = () => true
660
656
  const record = still() ? currentRecord(state, PLANE) : null;
661
657
  if (!record || !releasable(record, reason))
662
658
  return [];
659
+ // A run waiting for its person's answer keeps the session from an idle release; letting the keep go starts the idle
660
+ // window again (keepForPerson).
661
+ if (reason === "idle" && c.waiting.has(record.tag))
662
+ return [];
663
663
  // A person driving or watching the session through the live view keeps it from an idle release (#1098); it is
664
664
  // released one idle window after they leave.
665
665
  kept.gate = reason === "idle" ? personOn(c.options.gates ?? drivingGates, record.tag) : null;
@@ -834,7 +834,7 @@ export function createBrowserExtension(options) {
834
834
  stopped: (tag) => { c.stops.get(tag)?.stop(); c.stops.delete(tag); },
835
835
  });
836
836
  const c = {
837
- ...readers, name, Sessions, store, bill, held: new Attachments(), vault, fences: new Map(), leaving, serial: keyedSerializer(), guards: new Map(), owned, failed: false, stops: new Map(), ending: null, opened: null, incarnation: randomUUID(),
837
+ ...readers, name, Sessions, store, bill, held: new Attachments(), vault, fences: new Map(), leaving, waiting: new Map(), serial: keyedSerializer(), guards: new Map(), owned, failed: false, stops: new Map(), ending: null, opened: null, incarnation: randomUUID(),
838
838
  idle: idleTracker({ idleMs: DEFAULT_BROWSER_POLICY.idleReleaseS * 1000, onIdle: (cid, fired, idleMs) => void releaseIdle(c, cid, fired, idleMs).catch(() => undefined) }),
839
839
  };
840
840
  // Every browser tool is a model call on its session's driving gate: refused while a person drives the live view, and the viewer's input is held while it runs. The host's own
@@ -884,5 +884,24 @@ export function createBrowserExtension(options) {
884
884
  c.owned.clear();
885
885
  },
886
886
  retained: () => new Set([...c.guards.keys(), ...c.idle.keys(), ...c.serial.keys().map((key) => Number(key.slice(0, key.indexOf(":"))))]).size,
887
+ keepForPerson(tag) {
888
+ c.waiting.set(tag, (c.waiting.get(tag) ?? 0) + 1);
889
+ let kept = true;
890
+ return () => {
891
+ if (!kept)
892
+ return;
893
+ kept = false;
894
+ const left = (c.waiting.get(tag) ?? 1) - 1;
895
+ if (left > 0) {
896
+ c.waiting.set(tag, left);
897
+ return;
898
+ }
899
+ c.waiting.delete(tag);
900
+ // The idle window starts again from now, for a session still owned here.
901
+ const cid = c.owned.get(tag)?.cid;
902
+ if (cid !== undefined && c.idle.keys().includes(cid))
903
+ c.idle.activity(cid, 0);
904
+ };
905
+ },
887
906
  };
888
907
  }
package/dist/index.d.ts CHANGED
@@ -22,3 +22,4 @@ export * from "./core/tools.js";
22
22
  export * from "./core/usage.js";
23
23
  export * from "./core/web.js";
24
24
  export * from "./core/vault.js";
25
+ export { cdpCall, type CdpSession } from "./core/cdp-call.js";
package/dist/index.js CHANGED
@@ -23,3 +23,6 @@ export * from "./core/tools.js";
23
23
  export * from "./core/usage.js";
24
24
  export * from "./core/web.js";
25
25
  export * from "./core/vault.js";
26
+ // One CDP call over its own socket, kept open for a setting that lasts as long as it (Browser.setDownloadBehavior): what a
27
+ // host holding a vendor's browser for the tool needs beside the driver.
28
+ export { cdpCall } from "./core/cdp-call.js";
@@ -7,12 +7,12 @@ import path from "node:path";
7
7
  import { BrowserFailureError, PROFILE_NAME, refusedError } from "../core/custody.js";
8
8
  import { DownloadNotSettled } from "../core/downloads.js";
9
9
  import { browserFailure } from "../core/failures.js";
10
- import { cdpCall } from "./cdp-call.js";
10
+ import { cdpCall } from "../core/cdp-call.js";
11
11
  import { chromeTail, openChromeStderr, watchChromeStderr } from "./chrome-stderr.js";
12
12
  import { claimWriter, freeWriter, noteWriter } from "./profiles.js";
13
13
  import { chromeArgs } from "./chrome-args.js";
14
14
  import { COPY, copyExtension, copyId, isLoaded, loadConfined, soleStagehand } from "./worker-extension.js";
15
- const CAPS = { timeoutModel: "wall-clock", maxLifetimeS: 86_400, survivesDisconnect: true, releaseIsAsync: false, profiles: false, extension: "loaded-locally", settings: { proxies: false, verified: false, geolocation: false, region: false, context: false } };
15
+ const CAPS = { profiles: false, settings: { proxies: false, verified: false, geolocation: false, region: false, context: false } };
16
16
  const downloadsDir = (profileRoot, tag) => path.join(profileDir(profileRoot, tag) + ".downloads");
17
17
  const READY_MS = 20_000;
18
18
  /** The browser is gone and directories it used are not: a profile left behind holds the session's cookies, so a release that cannot remove