@divebell/agent-browser 0.33.1-divebell.1 → 0.33.2-divebell.1

package/README.md

@@ -1043,7 +1043,7 @@ Create an `agent-browser.json` file to set persistent defaults instead of repeat
 
 **Locations (lowest to highest priority):**
 
-1. `~/.agent-browser/config.json`: user-level defaults
+1. `~/.agent-browser/config.json`: user-level defaults (or `$AGENT_BROWSER_HOME/config.json`)
 2. `./agent-browser.json`: project-level overrides (in working directory)
 3. `AGENT_BROWSER_*` environment variables override config file values
 4. CLI flags override everything
@@ -1075,6 +1075,15 @@ agent-browser --config ./ci-config.json open example.com
 AGENT_BROWSER_CONFIG=./ci-config.json agent-browser open example.com
 ```
 
+For sandboxed agents whose home directory is read-only, set
+`AGENT_BROWSER_HOME` to a writable directory such as `/tmp/agent-browser`.
+This relocates agent-browser-owned configuration, session state, browser
+downloads, temporary artifacts, and daemon files without changing `HOME` for
+other tools. When no override is set and `~/.agent-browser` is not writable,
+agent-browser automatically uses a short per-user directory under `/tmp` on
+Unix. Sandboxes that prohibit local sockets or browser processes entirely
+still require a browser running outside the sandbox and a reachable CDP URL.
+
 All options from the table above can be set in the config file using camelCase keys (e.g., `--executable-path` becomes `"executablePath"`, `--proxy-bypass` becomes `"proxyBypass"`). Plugins are configured with the `"plugins"` array shown above. Unknown keys are ignored for forward compatibility.
 
 A [JSON Schema](agent-browser.schema.json) is available for IDE autocomplete and validation. Add a `$schema` key to your config file to enable it:
@@ -1448,6 +1457,24 @@ To bind to a specific port, set `AGENT_BROWSER_STREAM_PORT`:
 AGENT_BROWSER_STREAM_PORT=9223 agent-browser open example.com
 ```
 
+Frame encoding is daemon-wide:
+
+| Variable | Default | Description |
+|---|---|---|
+| `AGENT_BROWSER_STREAM_QUALITY` | `80` | JPEG quality, 0 to 100 |
+| `AGENT_BROWSER_STREAM_MAX_WIDTH` | the viewport | Caps frame width in pixels |
+| `AGENT_BROWSER_STREAM_MAX_HEIGHT` | the viewport | Caps frame height in pixels |
+
+Width and height cap the encoded frame and leave the page size alone, so a portrait or HiDPI viewport keeps its resolution unless you cap it. The live stream requests jpeg. An explicit `screencast_start` reconfigures the same screencast, so a client can see the format change mid-stream. On a busy page at 1280x720, quality 80 costs about 54 KB per frame, quality 20 about 25 KB, and quality 20 at 640x360 about 9 KB.
+
+```bash
+# Cheaper frames for a constrained link
+AGENT_BROWSER_STREAM_QUALITY=20 \
+AGENT_BROWSER_STREAM_MAX_WIDTH=640 \
+AGENT_BROWSER_STREAM_MAX_HEIGHT=360 \
+agent-browser open example.com
+```
+
 You can also manage streaming at runtime with `stream enable`, `stream disable`, and `stream status`:
 
 ```bash
@@ -1466,6 +1493,7 @@ Connect to `ws://localhost:9223` to receive frames and send input:
 ```json
 {
   "type": "frame",
+  "seq": 41,
   "data": "<base64-encoded-jpeg>",
   "metadata": {
     "deviceWidth": 1280,
@@ -1473,11 +1501,14 @@ Connect to `ws://localhost:9223` to receive frames and send input:
     "pageScaleFactor": 1,
     "offsetTop": 0,
     "scrollOffsetX": 0,
-    "scrollOffsetY": 0
+    "scrollOffsetY": 0,
+    "timestamp": 1785038682238
   }
 }
 ```
 
+`seq` is a monotonic frame id, echoed back in an `ack` message under ack pacing. `metadata.timestamp` is the capture time in epoch milliseconds, so a client can tell how old a frame is by the time it draws it.
+
 **Send mouse events:**
 
 ```json
@@ -1512,6 +1543,17 @@ Connect to `ws://localhost:9223` to receive frames and send input:
 }
 ```
 
+**Cap the frame rate (per client):**
+
+```json
+{
+  "type": "config",
+  "maxFps": 10
+}
+```
+
+Frames are delivered latest-first: the server picks the newest frame at send time, so frames produced while an earlier one is still being written are skipped rather than queued. `maxFps` (1 to 120, `0` = uncapped) limits delivery for that client only. A client that sends `{"type":"config","pacing":"ack"}` receives one frame at a time and acknowledges it with `{"type":"ack","seq":N}`, so nothing stale reaches the socket even if that client stalls; in the default push pacing, frames already handed to the transport are still delivered in order. Both settings can also be declared on the URL (`ws://127.0.0.1:<port>/?pacing=ack&maxFps=10`), which is the only way to cover the connection's opening frame. Input events are read on a dedicated task per connection, so clicks and keystrokes dispatch immediately even while frames are mid-write to a slow client. They are sent to the browser without waiting for its reply, so a click stays responsive behind a burst of mouse moves, and ordering is preserved.
+
 ## Architecture
 
 agent-browser uses a client-daemon architecture:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@divebell/agent-browser",
-  "version": "0.33.1-divebell.1",
+  "version": "0.33.2-divebell.1",
   "description": "Browser automation CLI for AI agents with Divebell memory diagnostics",
   "type": "module",
   "engines": {

package/skill-data/core/SKILL.md

@@ -513,6 +513,7 @@ That pulls in:
 - `references/session-management.md` — persistence, multi-session workflows
 - `references/profiling.md` — Chrome DevTools tracing and profiling
 - `references/video-recording.md` — video capture options
+- `references/streaming.md` covers live viewport streaming, remote input, per-client frame rate, and the encoding vars that set bandwidth cost
 - `references/proxy-support.md` — proxy configuration
 - `references/webgpu.md` — screenshots/video of WebGPU pages (three.js, Babylon.js), Linux/CI setup
 - `templates/*` — starter shell scripts for auth, capture, form automation

package/skill-data/core/references/commands.md

@@ -328,6 +328,17 @@ agent-browser state save auth.json    # Save cookies, storage, auth state
 agent-browser state load auth.json    # Restore saved state
 ```
 
+## Live Streaming
+
+```bash
+agent-browser stream status --json    # Enabled state, port, client count
+agent-browser stream enable           # Start the WebSocket stream server
+agent-browser stream enable --port 9223
+agent-browser stream disable          # Stop it
+```
+
+Clients connect to `ws://127.0.0.1:<port>` and receive `frame`, `status`, `tabs`, `url`, and `console` messages. They send `input_mouse`, `input_keyboard`, and `input_touch` to drive the page, `{"type":"config","maxFps":N}` (1 to 120, `0` = uncapped) to cap their own frame rate, and `{"type":"config","pacing":"ack"}` to receive one frame at a time, acknowledged with `{"type":"ack","seq":N}`. Both settings can be declared on the URL instead (`ws://127.0.0.1:<port>/?pacing=ack&maxFps=10`). See [streaming.md](streaming.md).
+
 ## MCP Server
 
 ```bash

package/skill-data/core/references/streaming.md

@@ -0,0 +1,104 @@
+# Live Streaming
+
+Stream a session's viewport over WebSocket and drive it with remote input. This is what a remote preview or embedded dashboard connects to: the browser runs wherever the daemon runs (a sandbox, a container, a CI box), and the client renders frames and sends clicks back.
+
+**Related**: [commands.md](commands.md) for full command reference, [SKILL.md](../SKILL.md) for quick start.
+
+## Contents
+
+- [Enabling the stream](#enabling-the-stream)
+- [Connecting](#connecting)
+- [Messages from the server](#messages-from-the-server)
+- [Messages from the client](#messages-from-the-client)
+- [Frame rate and staleness](#frame-rate-and-staleness)
+- [Limitations](#limitations)
+
+## Enabling the stream
+
+Streaming is always available; the server binds an OS-assigned localhost port unless told otherwise.
+
+```bash
+agent-browser stream status --json     # Report enabled state, port, client count
+agent-browser stream enable            # Create the server (--port to pin one)
+agent-browser stream disable           # Tear it down
+```
+
+`AGENT_BROWSER_STREAM_PORT` pins the port for the whole daemon instead of passing `--port`.
+
+Frame encoding is daemon-wide, read once at startup:
+
+| Variable | Default | Notes |
+|---|---|---|
+| `AGENT_BROWSER_STREAM_QUALITY` | `80` | 0 to 100, clamped |
+| `AGENT_BROWSER_STREAM_MAX_WIDTH` | the viewport | caps the frame, does not resize the page |
+| `AGENT_BROWSER_STREAM_MAX_HEIGHT` | the viewport | same |
+
+The live stream requests jpeg, since a `frame` message carries no format field. An explicit `screencast_start` reconfigures the same underlying screencast, so a client can still see the format change mid-stream; sniff the bytes rather than assuming. Measured on a busy page at 1280x720: quality 80 gives ~54 KB per frame, quality 20 gives ~25 KB, and quality 20 at 640x360 gives ~9 KB. An unusable value leaves the default.
+
+Read the port from `stream status --json` rather than assuming one; the OS-assigned default changes per daemon.
+
+## Connecting
+
+Connect a WebSocket client to `ws://127.0.0.1:<port>`. Frame delivery starts automatically once a client attaches, so there is no subscribe message. Browser clients must load from `localhost`, `127.0.0.1`, `::1` or `file://`. Any other origin gets a 403 on the upgrade and needs a proxy.
+
+## Messages from the server
+
+Every message is JSON text with a `type` field.
+
+- `frame`: a viewport image plus its metadata. Delivered latest-first (see below).
+
+```json
+{
+  "type": "frame",
+  "seq": 41,
+  "data": "<base64-encoded-jpeg>",
+  "metadata": {
+    "deviceWidth": 1280, "deviceHeight": 720, "pageScaleFactor": 1,
+    "offsetTop": 0, "scrollOffsetX": 0, "scrollOffsetY": 0,
+    "timestamp": 1785038682238
+  }
+}
+```
+
+`seq` is a monotonic frame id, echoed back under ack pacing and stable across browser relaunches. `metadata.timestamp` is the capture time in epoch milliseconds, so `Date.now() - timestamp` is the age of the frame being drawn. The other message types:
+
+- `status`: connection state, screencasting flag, viewport size, engine, recording flag. Sent once on connect and again on change.
+- `tabs`: the current tab list, sent on connect when tabs are known and on change.
+- `url`, `console`: navigation and console events.
+
+Status, tabs, url, and console travel on an ordered channel: they are delivered in order and are never replaced by a newer message the way frames are. They are not unconditionally durable. A client that falls far enough behind can lag out of that channel and lose messages it never saw, so treat console output as a live feed, not an audit log.
+
+## Messages from the client
+
+```json
+{"type": "input_mouse", "eventType": "mousePressed", "x": 40, "y": 40, "button": "left", "clickCount": 1}
+{"type": "input_keyboard", "eventType": "keyDown", "key": "a", "text": "a"}
+{"type": "input_touch", "eventType": "touchStart", "touchPoints": []}
+{"type": "config", "maxFps": 10}
+{"type": "config", "pacing": "ack"}
+{"type": "ack", "seq": 41}
+```
+
+Input dispatches to the browser on a task of its own, separate from frame delivery, so a click is not queued behind a frame write. Events are sent to the browser without waiting for its reply, so a click stays responsive behind a burst of mouse moves. Ordering is preserved: press never overtakes move. Mouse, keyboard, and touch input also reset the daemon idle timer, so an actively driven preview is not shut down by the idle timeout.
+
+`config` sets a per-client frame cap: 1 to 120, or `0` for uncapped (the default). It takes effect immediately, including when it loosens the cap. Each client's cap is its own; other connected clients are unaffected. A value above 120 is clamped to 120; a negative or non-numeric value is ignored, leaving the current cap in place. Neither rejects the connection.
+
+Both settings can also be declared on the URL, which is the only way to have them cover the connection's opening frame: `ws://127.0.0.1:<port>/?pacing=ack&maxFps=10`. A `config` message sent after connecting still wins.
+
+## Frame rate and staleness
+
+The server holds only the newest frame per client and reads it at send time. A frame produced while an earlier one is still being written is skipped, not queued, so the application never builds a backlog.
+
+Push pacing (the default) stops there, and the transport underneath is still ordered: frames already accepted by the socket are delivered in order, so a client that stalls drains whatever the kernel buffered before the writer blocked.
+
+Ack pacing closes that gap. Send `{"type":"config","pacing":"ack"}` and the server keeps at most one frame in flight, waiting for `{"type":"ack","seq":N}` before sending the next. Every frame carries a monotonic `seq`; echo the one you finished rendering. Frames produced while an ack is outstanding replace each other and never reach the socket, so a client that stalls for ten seconds and resumes gets the current page, not ten seconds of history.
+
+Under ack pacing one frame is in flight at a time, so the rate is one frame per transfer plus one acknowledgement round trip. Both the link's bandwidth and its latency bound it, and a link whose bandwidth-delay product exceeds a single frame goes underused. Ack pacing bounds one hop. With a proxy in the path, forward the renderer's acks; acks generated on receipt leave frames queued on the far side. Acks are cumulative, so acknowledging a newer id covers any older one. A client that opts in and then stops acking simply stops receiving frames; status, tabs, url, and console keep flowing.
+
+The two settings compose: `pacing` bounds how much is in flight, `maxFps` bounds the rate. A constrained preview usually wants both.
+
+## Limitations
+
+- Localhost only. Exposing the stream beyond the machine is the embedder's job (tunnel, proxy, or port forward), and the origin allowlist applies to browser clients.
+- Frames are images, not a video codec. Bandwidth scales with viewport size and page activity; cap the rate for constrained links.
+- In push pacing the server cannot tell a slow renderer from a fast one beyond transport backpressure. Use ack pacing when that distinction matters.