dsh-inner-browser-use 0.11.44 → 0.18.0

package/docs/ARCHITECTURE.md

@@ -1,172 +1,218 @@
 # Architecture
 
-Two halves, one wire.
+Two halves in two processes, and a Chrome that neither of them is.
 
 ```
-Host (plugin index.js)                Client (client.js, in the page)
-──────────────────────                ────────────────────────────────
-tools.register(browser_*)             the product's browser tab → <webview>
-        │                                       │
-        │  bridge.call(chat, op, args)          │  executeJavaScript(RUNTIME)
-        ▼                                       ▼
-   POST /push ────── SSE ──────►  dispatch op, reply via POST /uplink
-        ▲                                       │
-        └────────── /uplink ◄───────────────────┘
+server side  (index.js, in the DSH process)
+  sidecar_* tools ──> Sidecar ── CDP over a socket ──> headless Chrome
+                         │
+  HTTP + SSE routes <────┘
+        │
+        │ same origin as the product document, no other channel
+        │
+page side  (client.js, in the product document)
+  right column: JPEG picture, black arrow, input handlers, address row
 ```
 
-## Why the Host needs a wire at all
+## Why a wire
 
-The sidebar browser is a Client object. The `<webview>` is created in the product
-document, leased from the Desktop main process, and it only exists while the tab
-is mounted. A tool, however, executes on the Host. Nothing in the Host can reach
-a DOM element that lives in another process, so the two halves talk.
+The page half runs in the product document. It can paint a picture and read the mouse, and
+that is the limit: a page may not read another origin's DOM, and the product's own browser
+tab belongs to the product. The agent needs the page's text, its controls and real input
+events, which only the browser itself can give.
 
-## The transport
+So the server half owns a Chrome process and speaks CDP to it, and the two halves exchange
+two things over the plugin's own routes: frames going one way, input and navigation going the
+other. The picture is a consequence of that, not the mechanism.
 
-The Client `fetch`es relative URLs under `/dsh-inner-browser-use/`; the Host
-registers that prefix on its web server.
+## Chrome lives outside the product
 
-Same-origin is the whole trick, and it holds on both builds:
+The browser is a normal Chrome launched with `--user-data-dir` pointing at
+`~/.dsh/inner-browser-use/chrome/<profile>`, the profile name lowercased to `default`.
+Chrome's own storage is not used, and the product's browser storage is not used either: that
+one is created per process, so every sign-in in it is gone at the next restart.
 
-- **Web build:** the page already sits on the Host origin.
-- **Desktop build:** the product document is `dsh-app://app`. Its protocol
-  handler forwards every path it does not serve itself to the Host, with the
-  Host cookie attached. A relative `fetch` therefore lands on these routes with
-  no CORS preflight, no port number, and no token for the Client to learn.
+Other launch flags:
 
-| Route | Direction | Purpose |
-| --- | --- | --- |
-| `GET /downlink?session=` | Host → tab | Server-sent events, one long-lived response per open tab. |
-| `POST /uplink?session=` | tab → Host | Replies and lifecycle frames. |
-| `POST /push?session=` | Host → tab | Diagnostic push without a waiting call. Not used by tools. |
-| `GET /watch` | Host → page | Server-sent events, one long-lived response per page (not per tab): the channel the page uses to learn that some chat needs a tab. |
-| `GET /state` | either | Which chats have a tab, how many calls are in flight, last client heartbeat. |
-
-## Surviving a reload of the other half
-
-Each host half names itself (`epoch`) in every heartbeat reply. A page whose ask
-stream predates that name still holds an open socket to the old half, which can
-never open a tab. The next heartbeat (≤20 s) answers with a new name, and the page
-reopens the stream. Installing, or reloading the host, therefore needs no page
-reload.
-
-## Routing a chat to its tab
-
-The tool layer names a chat by its agent id; the tab the page opens may carry a
-different session id. Right after an ask, the page reports which tab it opened
-(`phase:"open"`, `opened:"current"`), and the host records that pair for the next
-30 s. Later calls resolve chat → tab through the pair, so two open tabs stay
-apart instead of both falling back to the single-seat rule.
-
-## Opening the tab on demand
-
-A tool cannot ask a user to click `+`, so the call carries the ask: when no seat
-exists, the Host pushes `wanted` down `/watch`, the page opens this chat's tab
-through `ctx.sidebarRight.openTabIn(sessionId, "browser")` — the product's own
-kind, the one with the address bar — and the call waits up to `openWaitMs` (8 s)
-for the seat to appear before it fails.
-
-`/watch` is per page rather than per tab on purpose: a page with no tab open has
-no `/downlink` stream, and it is exactly that page which must be able to obey.
-
-## Seats
-
-One open tab is one seat, keyed by the chat id. A tool resolves its seat by the
-agent id that called it and falls back to the only seat when exactly one tab is
-open — that fallback is what lets a chat drive a tab another chat opened, which
-is occasionally what you want while debugging.
-
-A seat that disappears mid-call rejects the waiter; it does not hang until the
-timeout. A seat that goes silent does hang until the timeout, which is
-deliberate: a page can be slow, and the model should see "did not finish" rather
-than a misleading success.
-
-## Frames
-
-```jsonc
-// Host → tab
-{ "type": "call", "id": "<random>", "tool": "click", "args": { "ref": 4 } }
-// tab → Host
-{ "type": "reply", "id": "<same>", "ok": true, "value": { "text": "…" } }
-{ "type": "reply", "id": "<same>", "ok": false, "error": "no control 4 on this page" }
-{ "type": "wanted", "session": "<chat id>" }
-{ "type": "client-alive", "phase": "registered", "tabType": true }
-```
-
-A navigation to the address the tab already shows is answered by `reloadIgnoringCache()`
-rather than `loadURL()`: a browser serves that second request from its own cache, so a
-page whose file changed on the machine would otherwise come back stale and the numbers
-handed to the model would point at text that no longer exists.
-
-Frames are JSON, not binary, and they carry text, not screenshots. A page
-observation is a few kilobytes of text; a screenshot would be megabytes over the
-same socket, and it is not what the model needs to decide where to click.
+| Flag | Why |
+| --- | --- |
+| `--headless=new` | No window on the desktop, still a full rendering engine. |
+| `--remote-debugging-port=0` | Chrome picks its own debugging address. | <!-- ui-copy-ignore -->
+| `--use-mock-keychain`, `--password-store=basic` | No keychain prompt on macOS. |
+| `--remote-allow-origins=*` | The CDP client connects over a plain WebSocket. |
+| `--disable-blink-features=AutomationControlled` | Sites that sniff for automation get the ordinary path. |
+| `--window-size=1440,900`, `--lang=zh-CN` | The viewport the pane mirrors, and a Chinese page locale. |
 
-## The guest runtime
+The binary comes from a fixed list: Google Chrome, Chromium or Canary under `/Applications`,
+then `/usr/bin/google-chrome` and `/usr/bin/chromium`. Nothing else is probed and there is no
+setting to add a path.
 
-`client.js` keeps the whole in-page side as one string, evaluated per call:
+## Attach and lifetime
 
-```js
-el.executeJavaScript(`(async function(){ ${RUNTIME}; return await run(${op}, ${args}) })()`)
-```
-
-Rewriting it on every call costs a few milliseconds and buys immunity from the
-one failure that would otherwise break everything: the page navigating and
-taking an earlier injection with it. Nothing persists in the guest except what
-the call itself creates.
+The browser outlives the plugin:
 
-`view` numbers every control it can see (`a`, `button`, `input`, `select`,
-`textarea`, `[role=…]`, `[onclick]`, `[tabindex]`) in document order, trimming
-invisible and zero-sized nodes, and returns the first 2400 characters of visible
-text. The numbers are only valid for the page the model last read, which is why
-every mutating tool returns a fresh `view` afterwards.
+1. `ensure()` reads `DevToolsActivePort` from the profile directory.
+2. It probes `GET http://127.0.0.1:<address>/json/version`. A stale file is deleted and Chrome
+   is launched again.
+3. Otherwise the running Chrome is adopted: the page target is read from `/json/list` (with
+   `/json/new?about:blank` as a fallback) and a client connects to it.
 
-`click` and `type` scroll the target into view, move the pointer there, ripple,
-and dispatch real `MouseEvent`s (full `pointerdown/mousedown/mouseup/click`
-sequence) so frameworks that listen on any of them react.
+One page per browser. `apply()` only constructs the `Sidecar`; no process exists until the
+first tool call, and `sidecar_state` reports a stopped browser without starting one.
 
-## The lease
+Shutdown asks Chrome to close (`Browser.close`), waits briefly, then sends `SIGTERM`, which is
+the point where Chrome flushes cookies. An idle timer closes the browser after 15 minutes; a
+tool call resets the clock, and a watching column defers the close when the timer fires, so a
+page someone is reading outlives the idle window.
 
-Only the rollback path (`NATIVE_MODE = false`) uses this. In the shipped mode the
-page drives the `<webview>` the product's browser tab already holds — it finds
-that node (right of the chat, `[data-sidebar-browser-frame="webview"]`, or a
-`webview` no other half parked) and never creates, moves, or disposes one.
+## Routes
 
-On the rollback path the tab does not create its own browser. It asks the Desktop
-main process:
+Everything hangs under `/dsh-inner-browser-use`, all same-origin, bodies capped at 1 MB.
 
-```js
-const reservation = await dshDesktop.browser.acquire("cwd:/path/to/workspace")
-webview.name = reservation.lease
-webview.partition = reservation.partition
-webview.src = `about:blank#${reservation.lease}`
-```
+| Route | Kind | Purpose |
+| --- | --- | --- |
+| `GET /sidecar/frames?session=` | SSE | The mirror: `hello`, then frame messages, plus pointer messages. |
+| `GET /sidecar/state` | JSON | What the browser is doing. |
+| `POST /sidecar/open` | JSON | Start or reuse the browser, optionally navigating. |
+| `POST /sidecar/input` | JSON | `hover`, `move`, `click`, `wheel`, `text`, `key`; answers with the page's cursor. |
+| `POST /sidecar/nav` | JSON | `back`, `forward`, `reload`, or a `goto` with an address. |
+| `POST /sidecar/stop` | JSON | Close the browser now. |
+| `GET /downlink?session=` | SSE | Per-session notices; the only message today is `sidecar-wanted`. |
+
+The page half never learns the debugging address. It knows the route prefix and nothing else;
+the CDP socket stays in the server half.
 
-All three attributes matter. The main process cancels `will-attach-webview` when
-the `src` hash does not match the lease it issued, so a webview configured
-loosely never attaches — it silently shows nothing. The partition is
-`dsh-sidebar-browser-<uuid>`, in memory, scoped to the workspace key: `cwd:<path>`
-for a chat in a workspace, `session:<id>` for an ungrouped one. Same key, same
-cookies as the built-in browser tab.
-
-In the shipped mode the partition is the product's own and outlives a restart, so
-no cookie bookkeeping is needed. On the rollback path that partition is in memory,
-so the page half files the cookies the page itself can read under `~/.dsh/inner-browser-use/cookies/<hash>.json`, mode 0600,
-keyed by the same workspace key: name, value, and the origin the page reported,
-nothing else, on this machine only. When that workspace's pane opens again, its
-remembered page loads, the kept cookies for that origin are written back, and the
-page loads once more — a cookie set from the page cannot affect a document that is
-already open. A workspace holds several sites, so a save replaces only the origin
-the page is on.
-
-The debugger is not an option for this: a `<webview>` element in the shipped app
-does not expose one, and the main process keeps the guest session to itself.
-Cookies marked `HttpOnly` are invisible to the page, so a login that relies only on
-those does not survive; anything the page reads from `document.cookie` does.
-
-## Failure modes, by design
-
-- **No tab:** the call asks for one, waits, and fails with one sentence if none opens.
-- **No Desktop:** the body renders "Desktop app only"; nothing else registers.
-- **Page refuses:** the error string reaches the model as a tool error.
-- **Plugin stops:** every seat closes, every waiter rejects.
+## Mirror
+
+The server half starts `Page.startScreencast` with JPEG quality 72, `maxWidth` and `maxHeight`
+at 1440×900, `optimizeForSpeed`, `everyNthFrame: 1`, and acknowledges every frame it receives
+so Chrome keeps sending. Frames are never queued: with two frames in flight, further frames
+are dropped and the next one that fits takes their place. A slow reader loses frames, never
+freshness.
+
+The viewport is pinned with `Emulation.setDeviceMetricsOverride` to the same 1440×900, so a
+frame is the whole page as laid out, not a crop.
+
+If `Browser.getVersion` reports `HeadlessChrome`, the user agent is overridden with an ordinary
+Chrome string, `acceptLanguage: 'zh-CN,zh;q=0.9,en;q=0.8'` and `platform: 'MacIntel'`, so a site
+that gates on headless gets the normal rendering.
+
+The stream is SSE with a comment beat every 15 seconds. A response that falls more than 3 MB
+behind is given up rather than buffered, Nagle is off on the socket, and a session that opens a
+second stream replaces its first seat: one column per session, no fan-out of stale frames.
+
+## Pointer, input, cursor
+
+Two pointers, two sources:
+
+- The agent's arrow comes from the tool calls. `movePointer()` sends `{type: 'pointer', w, h,
+  pointer}` on the same stream and the pane draws it at once, so the arrow does not wait for
+  the next frame. Hovering with your own mouse sends `mouseMoved` and never touches that state.
+- Your own cursor stays the system cursor. The pane asks the page which CSS cursor sits under
+  the hand (`cursorAt()`), the reply to `/sidecar/input` carries it, and the pane applies it
+  only when it is in the pane's allow-list.
+
+Clicks are `Input.dispatchMouseEvent` down and up at one point. Typing is `Input.insertText`:
+a composed Chinese word or a pasted string lands as one insertion rather than key by key.
+Named keys go through `Input.dispatchKeyEvent` with a table mapping Enter, Tab, Escape,
+Backspace, Delete, the arrows, Home, End, PageUp and PageDown to their key and code, which is
+what makes Backspace delete a character on the page instead of nothing.
+
+## Session scoping
+
+The right column is shared by every plugin and belongs to whichever session is on screen. A
+tool call pushes `sidecar-wanted` down that session's own `/downlink`, and the page half opens
+the mirror tab only when `sidebarRight.mounted` equals the session that asked. An unknown or
+empty `mounted` is a refusal, not permission.
+
+The frame stream is opened by the pane with its own `session` in the query. A chat whose column
+sits on another session gets no frames, and a tool call in the background drives the same
+browser without pulling the column away from the person reading it.
+
+## Tool surface
+
+`sidecarTools()` in `tools.js` builds the 28 definitions and `index.js` registers them on the
+plugin's tool scope. All of them return one text block. A page read numbers the controls and
+keeps those numbers across later reads, so `sidecar_click` by ref means the element the model
+last saw; navigation drops them and the model gets `REF_GONE` rather than a wrong click.
+
+Before a ref click or a ref focus, one page-side probe asks what is on top of that control.
+When something is, and that something carries its own safe way out (a banner with Accept /
+Close, a modal with 同意 / 关闭), the probe's answer names that button's point, the driver takes
+it, checks the control is clear, and then performs the action the model asked for. The result
+says both things. An overlay whose wording involves deleting, paying or signing out is only
+reported, never accepted on the person's behalf. The whole exchange is one `Runtime.evaluate`
+and a possible `mousePressed`/`mouseReleased` pair on an already-open CDP connection, so an
+uncovered click pays nothing.
+
+`sidecar_do` runs the same probe before a step that names a control — by `ref`, by
+text, or by field label — so a banner the first step clears is gone for the rest of
+the batch, and the step's own line says it in one clause (`after Accept on a banner,`).
+A batch therefore pays for one overlay, not one per step.
+
+An action answers with one line — address, title, focused field — and reads the page only when
+`read: true` is passed. That is what keeps a long batch cheap (see `BENCH.md`).
+
+Calls that touch the browser are marked not concurrency-safe; reading state, console and failed
+requests is. `sidecar_open`, `sidecar_view`, `sidecar_wait` and `sidecar_shot` carry longer
+timeouts than the rest.
+
+Sign-in state lives in the profile, and session cookies — the kind a sign-in usually sets,
+which Chrome would otherwise drop on exit — are saved next to it and restored on the next
+launch, so a restart of the app does not sign the person out.
+
+A system-prompt section (`browser-in-this-chat`) tells the model to reach for these tools
+without asking first when someone wants something done on a site.
+
+## Network
+
+`Network.enable` is already on for the mirror. A request is kept when it is
+sent and settled when its response arrives, so the list holds method, status,
+type and address of what the page actually got — 120 rows, oldest dropped.
+`sidecar_network` reads that list without touching the page, which is why it
+can answer while a dialog is open.
+
+## Downloads
+
+Chrome is pointed at `<profile>/downloads` when the page starts, so a click
+that saves a file writes it there instead of nowhere. The folder belongs to
+the profile: it is the same one on the next launch, and nothing outside the
+profile is touched. `sidecar_downloads` lists what is on disk, newest first;
+a `.crdownload` still in flight is not listed.
+
+## Frames and popups
+
+A page is not one document. Chrome keeps a table of frames and gives each one the
+JavaScript context its scripts run in; the plugin listens for those contexts as they
+are created and destroyed, so a read can be sent to the context that owns the
+controls it wants.
+
+- A read walks the frame tree main frame first and sends one snapshot per frame, so a
+  form inside an iframe is numbered with the page's own controls.
+- Refs from a frame are offset: `900000 + slot * 100000 + local`. The slot is the
+  frame's position in the tree, so the number says which context to resolve in, and a
+  frame's refs never collide with the page's.
+- A click on a framed control resolves in that frame, then adds the frame's rectangle
+  to the point it found, because input is always dispatched in viewport coordinates.
+- A frame Chrome runs in another process is a target of its own: the plugin keeps the
+  session Chrome hands it, so a read can be sent there and its controls still get a slot.
+  Its corner is asked from the page, through the frame's owner element, because the
+  frame's own process cannot see where it sits.
+
+A click can open a tab. A page target gets no handle on its own popups, so after an
+action the plugin asks the DevTools address list for page targets other than the one it
+drives. If one appears, the mirror navigates to its address and the popup is closed with
+the target HTTP endpoints. A person watching the column sees where the click went,
+and there is still one page to drive.
+
+## Files
+
+| File | Half |
+| --- | --- |
+| `index.js` | Server: routes, tools, CDP page reads, system prompt. |
+| `sidecar.js` | Server: Chrome launch, attach, screencast, input, shutdown. |
+| `client.js` | Page: the column, its stream, the arrow, the address row. |
+| `locale/*.json` | Page: pane labels. |
+
+Data on disk is confined to `~/.dsh/inner-browser-use/`: `chrome/<profile>` for the browser,
+`screenshots/` for `sidecar_shot`.

package/docs/BENCH.md

@@ -0,0 +1,234 @@
+<!-- ui-copy-ignore -->
+# Measured, not claimed
+
+`node scripts/bench.mjs` on a local page, headless, profile of its own. One JSON
+line per measurement; a phase is only done when its row improves.
+
+Method: a dense page (20 inputs, 120 links, two buttons), a text page (~60
+paragraphs), a light page. The dense page carries a viewport-fixed counter,
+because Chrome sends screencast frames only for visible change — measuring an
+idle page measures nothing.
+
+## Baseline — 0.12.1, before phase 1
+
+| What | Number |
+| --- | --- |
+| launch → first frame | 506 ms (cold profile) |
+| steady mirror | 60 fps, 72 KB/frame |
+| click → next frame | 13 ms |
+| click round trip | 1 ms |
+| key round trip | 13 ms |
+| screenshot | 82 ms, 133 KB PNG |
+| read: dense page | 4.7k chars ≈ **1383 tokens** |
+| read: text page | 8.0k chars ≈ **2363 tokens** |
+| read: light page | 69 chars ≈ 20 tokens |
+
+Targets: dense read ≤ 1200 tokens, text read ≤ 1500 tokens with the useful part
+kept, click → painted ≤ 120 ms (already 13 ms; the number to protect), and a
+batch of five steps in one tool call.
+
+## After phase 1 + 2 — read model v2 and batched steps
+
+Same bench, same machine. Read numbers are what a model actually receives with
+the default budget (4600 chars of page text).
+
+| What | Baseline | Now |
+| --- | --- | --- |
+| read: dense page | 1383 tokens | **436** tokens (90 controls, 1.5k chars) — `links` 514, `forms` 127, `full` 624 |
+| read: text page | 2363 tokens | **1130** tokens (screenful only, budget 4600) |
+| read: light page | 20 tokens | 17 tokens |
+| read round trip | ~30 ms | **1–4 ms** |
+| refs across reads | new numbers every read | **stable** (same refs, ticker-only text changes) |
+| click round trip | 1 ms | 1 ms |
+| key round trip | 13 ms | 0–2 ms |
+| launch → first frame | 506 ms | 129–539 ms (cold/warm profile) |
+| steady mirror | 60 fps, 72 KB/frame | 60 fps, 76 KB/frame |
+| screenshot | 82 ms / 133 KB | 79–86 ms / 115 KB |
+
+The win is structural: the old read took `document.body.innerText` (8k chars on
+the text page); the new one walks the viewport, dedupes parent text, numbers
+controls with refs that survive reads, and caps the text at the budget. Reading
+again returns the same numbers, so a batch of steps costs one read instead of
+one per step.
+
+## What one step costs the model
+
+Measured on the live tool surface (`/tmp/probe6.mjs`, 2026-10-27). Before this
+pass every action answered with a full page read; now it answers with one line
+and reads only when `read: true` is passed.
+
+| step | reply | ≈ tokens |
+| --- | --- | --- |
+| click | `ok click 1 · https://… · Title` (60 chars) | 18 |
+| type | `ok type · https://… · Title · focus email = a@b.c` (77) | 23 |
+| key | `ok Enter · https://… · Title` (65) | 20 |
+| check | `I agree is now on` (17) | 6 |
+| select | `chose Beta · https://… · Title · focus …` (65) | 20 |
+| wait | `saw hello` (9) | 3 |
+| console / failed | `error: message` or `no request failed` (17–47) | 6–14 |
+| dialog | `alert: sure?` (12) | 4 |
+| cookie | one line per cookie, up to 40 | — |
+| full read (dense page) | 1484 chars | 436 |
+
+A ten-step batch therefore spends ~150 tokens on confirmations instead of
+~4 400, and the page itself is read once when it changes.
+
+## Notes
+
+- Screencast frames are change-driven. A page with nothing moving sends nothing;
+  that is Chrome behaving, not the mirror failing. After an agent action the host
+  should force one frame so the person sees the result of a step that changed
+  nothing visible.
+- A tall page whose only animation is below the fold produces no frames: the
+  viewport is what is captured.
+
+## Phase 3 — frames and popups, 0.13.0
+
+Same bench, same page. Two changes landed: controls are capped at the read's own
+`max` after viewport-first sorting (the cap was applied to candidates, not to the
+answer), and a read now also asks for the frame tree.
+
+| What | Before | After |
+| --- | --- | --- |
+| read: dense, `full` | 3118 chars, 142 controls ≈ 917 tokens | 2128 chars, 90 controls ≈ **626 tokens** |
+| read: dense, `text` | 1510 chars, 142 controls ≈ 444 tokens | 1510 chars, 90 controls ≈ 444 tokens |
+| read: text page | 13 ms | 13 ms (one frame-tree call, ~1 ms) |
+| launch → first frame | 506 ms | 479 ms |
+| click round trip | 1 ms | 2 ms |
+| key round trip | 13 ms | 2 ms |
+| screenshot | 82 ms, 133 KB | 77 ms, 113 KB |
+
+`sidecar_extract` on a 30-row list: 1106 chars for all 30 rows (≈330 tokens),
+against 1291 chars for the same page read as text — and at a 500-char read
+budget the text read stops around row 12 while the list still returns 30.
+
+Live run (`npm run live`), 8 checks, all true on the open internet: open
+example.com in 566 ms, read it for 785 chars, click the language link on
+wikipedia.org and land on en.wikipedia.org in 2.3 s, go back to
+www.wikipedia.org, extract ten list rows from the Web scraping article in 3 ms
+(1813 chars), visit a host that cannot resolve (1.3 s to the browser's own
+error page, where a read, a click and a batch all refuse instead of quoting it),
+recover from there to a real page, and type into a search box and reach a
+results URL in 1.3 s. These are wall-clock end to end including the site's own
+load; the local numbers below are where the driver's own cost lives. Two hosts
+were unusable from this machine and are not plugin problems: Hacker News (12 s
+connect timeout, confirmed with curl) and Wikipedia's search redirect, which
+answers on the home page and times out on the query — the search check runs on
+Bing now, driven by the box's words rather than a control number.
+
+Breadth run (`npm run breadth`), 33 checks, all true: open shadow DOM read and
+click, form typing, select, checkbox, submit, `confirm()` seen and accepted,
+iframe read, iframe typing, iframe click, cross-site frame read and click,
+popup followed, download saved, network responses listed, a file uploaded
+into a file input, the page printed to a PDF, a changed-only read, names from `for=` and `aria-labelledby`, a click found by its words, a login filled by field names in one call,
+stale ref refused, a crash picked back up on the same page, a list read as 30 rows, a click under a full-screen cookie banner clears the banner and lands on the
+control, typing into a field under that banner works the same way, and an
+overlay that offers to delete the account is reported but never accepted.
+
+## Phase 4 — what a change costs
+
+| What | Number |
+| --- | --- |
+| change in the page → frame ready for the column | 7 ms median, 7 ms max (12 samples) |
+| `changed_only` after a field changes, dense page | 58 chars ≈ **17 tokens** (whole page: 444) |
+| read: text page, `changed_only` | 42 chars ≈ 12 tokens (whole page: 1517) |
+| renderer crashed → readable page again | **1.39 s** (detect, fresh tab, same address) |
+
+The mirror's delay is measured from the moment the page is told to change to
+the frame object the column would draw — capture, JPEG, and the CDP hop. It is
+the part of the latency that belongs to this side; what a person sees adds the
+browser's compositing and the SSE hop, both smaller.
+
+A cross-site frame (Chrome runs it in its own process) costs one more round
+trip than a same-process one: its controls arrive in the same read, and the
+frame's corner is one `DOM.getFrameOwner` plus one call on the owner element.
+
+## Phase 5 — what an overlay costs
+
+| What | Number |
+| --- | --- |
+| the "what is on top of this control" probe | **1 ms median, 2 ms max** (5 samples) — the same `Runtime.evaluate` hop, no round trip added |
+| `sidecar_click` on an uncovered control | 460 ms median (3 runs, page-load settle included) |
+| `sidecar_click` on the same control under a cookie banner | 630 ms median (3 runs) — the banner's own Accept click, a 150 ms beat, a re-probe, then the intended click |
+| the same under a modal that offers to delete the account | 456 ms — reported, nothing pressed |
+| one `sidecar_do` step on an uncovered control | 458 ms median (3 runs) |
+| the same step under the cookie banner | 633 ms median (3 runs) — cleared, then the step ran |
+
+A batch pays it once per step that has something over it, not once per call: a
+banner cleared by step 1 is gone for steps 2..n, so a five-step flow through a
+cookie wall costs one extra step's worth of time, not five.
+
+The probe costs nothing when nothing is in the way, which is the common case;
+the covered case pays about 170 ms to clear the banner and land the click in
+the same call, instead of making the model spend a whole extra turn (a read, a
+click, another read) telling it to close the banner first.
+
+## Phase 6 — what a stale number costs, 0.16.0
+
+A ref is a number for an element. A page that redraws a list, a table, a card
+grid — React, Vue, Svelte, or plain `innerHTML` — throws every element away and
+makes new ones, so the numbers from the last read stop pointing anywhere. The
+control is still on screen, saying the same words, usually in the same place.
+
+Before this phase that cost two round trips: the call came back `That control is
+gone`, the model read the page again to get a new number, then used it. The work
+is now done inside the call, by looking the control up by what it is.
+
+| What | Number |
+| --- | --- |
+| `sidecar_click` on a control the page just redrew | **462 ms median, 483 ms max** (8 samples) — one extra read of the page's visible text, 600 chars, to find the control again |
+| `sidecar_click` on a control that was not redrawn | 445 ms median, 446 ms max (8 runs) — the probe costs nothing when the number is still good |
+| stale refs recovered this way | **8/8** |
+| a control that really left the page | still refused, `Call sidecar_view and use the new number` — 2/2 in the breadth suite |
+
+So the recovery costs about **17 ms** when it is needed and nothing when it is
+not, against a model round trip of two calls plus the tokens of a fresh read.
+
+What counts as the same control: the same words and the same kind. A button does
+not become a field; a field never passes for the button beside it; a link keeps
+its destination, because the same words pointing somewhere else is a different
+thing to click. The words have to appear **exactly once** on the page — two
+"Delete" buttons in two rows are a choice only the person can make, so that case
+still asks for a read.
+
+The call says what it did: `(the page redrew, so "Learn more" has a new number — used where it is now)`, so the model knows its old number is not the one to keep
+using. On the live network the same thing happened on example.com: a link
+replaced by an identical clone, one click, 4.4 s including the iana.org page
+load, straight to `https://www.iana.org/help/example-domains`.
+
+## Phase 7 — what changing the shape of the browser costs, 0.17.0
+
+A site that refuses a windowless browser is the one case where the shape has to
+change mid-session. Changing it restarts the browser, so it costs a cold start:
+
+| what | median | notes |
+| --- | --- | --- |
+| windowless → windowed, same profile and page | 21.6 s | one process, cold start |
+| windowed → windowless, same profile and page | 22.1 s | one process, cold start |
+| asking for the shape it already has | 0 ms | nothing is restarted |
+
+Measured through the tool layer (`sidecar_open` → `sidecar_state` → `ps` spawn
+args), profile `probe-tool`, 1 run each; the same-shape row is the third call in
+the same run. A browser start on this machine is 20–22 s either way, so the
+switch is not what costs — the process is.
+
+Reading a check page costs nothing: the sentence is built from the title already
+in the read, and a normal page gets `''` for one comparison.
+
+## Phase 8 — saying it when nobody is drawing, 0.18.0
+
+The mirror is drawn by the tab in the chatting column, and that tab attaches
+while the page is loading. Waiting for it costs nothing when it arrives, and a
+bounded wait is the difference between a silent `viewers 0` and a sentence:
+
+| what | cost |
+| --- | --- |
+| `sidecar_open` with a tab watching | 0 ms |
+| `sidecar_open` with nothing watching | ≤ 1.2 s, and the line says so |
+| `sidecar_open` when the chat has no column to draw into | 0 ms (no wait) |
+| building the sentence on a normal state read | 0 ms, one comparison |
+
+Measured with `/tmp/probe-unseen.mjs` (tool layer, example.com, profile
+`probe-tool`): 1344 ms with nothing watching and no column asked for, 3236 ms in
+the first version of the same wait that did not skip the case where no column was
+ever requested — the whole difference was the wasted 1.2 s.