dsh-inner-browser-use 0.18.0 → 0.34.28

package/docs/BENCH.md

@@ -1,3 +1,238 @@
+## Phase 44 — 真站页游场地，0.34.28
+
+`node scripts/check-game.mjs <url>`，真 Chrome、自己的 profile、跑完自己关掉。两个真站：
+
+| 站点 | 空闲时页面刷新 | 全屏截图（`sidecar_shot` 同类）| 按住 `d` 200 ms 到达页面 |
+| --- | --- | --- | --- |
+| 奥拉星 `http://aoqi.100bt.com/` | **27 fps** | 中位数 **68 ms** | 是 |
+| 4399 `https://www.4399.com/` | **59.8 fps** | 中位数 **148 ms** | 是 |
+
+两个落地页都**没有 canvas**：游戏本体在链接后面，要么登录要么再点进去一层，所以这组数字说明的是「真站能不能实时跑」，不是「游戏画面已经能读」。
+
+诚实的一层：按键到达页面这件事在真站已经证实（页面自己记录的 `keydown`/`keyup`），但 4399 这种整屏重绘的页面，一张全屏图 1.8 MB、148 ms —— 做实时画面要压、要裁、要降缩放，这一层还没做，别拿它当「已经能玩」。
+
+## Phase 43 — 页游级输入：按住、组合键、按住鼠标，0.34.26
+
+本地画布页（每帧重绘，页面自己记录 `keydown`/`keyup`/`pointerdown`/`pointerup` 时间戳），真 Chrome，`node scripts/bench.mjs`：
+
+| 项 | 数字 |
+| --- | --- |
+| `sidecar_key` 带 `hold_ms: 300` 整次调用 | **305 ms**（含 300 ms 按住，收尾 5 ms） |
+| 页面测到的按住时长 | **301 ms** |
+| `ArrowLeft+ArrowUp` | 两键都收到 `keydown`，都收到 `keyup`，顺序为倒序松开 |
+| `sidecar_click` 带 `hold_ms: 200` 页面测到 | **211 ms** |
+
+对照 0.34.25 之前：同样的按住被 `settle` 拖到 727 ms，等于 0.7 s 才走一步——页游里这就是掉帧。
+
+## Phase 42 — the reload path, proved by two instances, 0.34.25
+
+`npm run adopt`（真 Chrome，临时 profile `adopt-check`，临时状态目录）：
+
+| 项 | 数字 |
+| --- | --- |
+| 第一个实例冷启动到页面可用 | 675 ms |
+| 第二个实例（模拟重载后的半边）接管 | 162 ms |
+| 第二个实例自己起的 Chrome | `null`（没有起第二个） |
+| 浏览器进程 | 同一个 pid，接管后仍活着 |
+| 页签与标题 | `title: "kept"`，内容原样 |
+
+这就是重载后「接回同一个浏览器、不重启、不丢页签和登录」的机器证据；之前只有手工重载才能走到这条路。
+
+## Phase 41 — the read tool that had never run, 0.34.23
+
+`node scripts/bench.mjs`（本机 headless 页面，Chrome 复用 `bench` profile）：
+
+| 项 | 数字 |
+| --- | --- |
+| 首次画面（复用已在跑的浏览器） | 133 ms |
+| 首帧字节 | 76 KB |
+| 帧率 | 60 fps |
+| 镜像延迟 | 中位 8 ms / 最大 8 ms |
+| 遮挡检测 | 中位 3 ms / 最大 15 ms |
+| 点一下（普通页 / 有遮罩，含读回） | 439 / 590 ms，遮罩那次 `cleared: true` |
+| 一步 `sidecar_do` | 440 / 590 ms |
+| 截图 | 73 ms / 115 KB |
+| 读 dense 页 text 档 | 6 ms / 1484 字 / ≈436 token |
+| 读 dense 页 forms 档 | 1 ms / 432 字 / ≈127 token |
+| 再读同一页 | 1–2 ms，给出「没有变化」一行 |
+| `changed_only` | 58 字 / ≈17 token（整页 1505 字） |
+
+这轮 bench 的作用不是数字，而是**它第一次跑到 `sidecar_view` 就报 `ReferenceError` 退出**：那个工具在真实会话里同样会失败，而单测只测了它的纯函数部分。修完之后 bench 一次性跑完，数字在上面。
+
+## Phase 40 — the poll that names its chat, 0.34.19
+
+The adoption rule from 0.34.16 never fired for the column itself: the pane
+polled `/sidecar/state` without a session, the host treated that as anonymous,
+and a reload could still leave the mirror empty over a live browser. The pane now
+names its chat, and asks again the moment a claim succeeds.
+
+No performance change was expected; the full bench was re-run anyway so the
+numbers under it are current (commit 0.34.19, local pages):
+
+| Label | Value |
+| --- | --- |
+| launch → first frame | 1029 ms |
+| steady stream | 60 fps, 76 KB per frame |
+| mirror delay (12 samples) | median 9 ms, max 13 ms |
+| dense page, text | 6 ms, 1484 chars, ≈436 tokens |
+| dense page, forms | 2 ms, 432 chars, ≈127 tokens |
+
+## Phase 39 — count what is on the screen, not what is in the DOM, 0.34.17
+
+`readFrames` counts child frames; the page's own count filters to frames that
+occupy the screen. Taking the larger made a hidden 1×1 frame claim a blind spot.
+
+- `blindSpots` now lets the page's visible count win; the reader's count is the
+  fallback for when the page could not count.
+- Live: two frames (400×300 and 1×1) → `blind.frames` 1, exactly one line.
+
+## Phase 38 — a reload must not blank the mirror, 0.34.16
+
+`/sidecar/state` answered from the new plugin instance, which owns nothing until
+something calls `open`: after a reload the owning chat's own poll said
+`alive:false`, hence "no page open" in the column while Chrome was fine.
+
+- The owning chat's state poll adopts the running browser and the tab remembered
+  for it before answering.
+- Live: `/dsh-unlag/reload` 200 → state `alive True`, url `https://www.doubao.com/chat/`,
+  Chrome pid 12833 with 11 m uptime, i.e. attached rather than relaunched.
+- Non-owning chats and empty `session` polls still answer without spawning anything.
+
+## Phase 37 — an observation that admits its blind spot, 0.34.15
+
+A text read and a screenshot claim to show the page. Both are lossy, and the loss
+was silent: a chart wired as `<canvas>`, a generated image with no alt text, a
+video, a cross-origin frame — nothing in the text said "there is something here
+you cannot read". A model then planned as if the page were fully described.
+
+- `readPage` counts the unreadable visible things on the page and in frames, and `renderPage` prints exactly one line when the count is non-zero; zero adds no line (measured token cost of the feature is 0 on pages that have nothing to hide).
+- `chrome-error://` is no longer rendered as an empty page: it says the page did not load and names `sidecar_nav`.
+- `navigate` retries that error page once (300 ms + 8 s cap). Cold launch on a fresh profile: first navigation recovered on the retry; an unreachable host still returns `bad` in well under the cap.
+- `sidecar_shot` without an image channel appends what the result is missing and which tool fills it; file path alone is no longer the whole answer.
+- Suite: 101 tests, 0 failures.
+
+## Phase 36 — a reload must not cost the browser, 0.34.14
+
+- 根因：交接看门狗 `while kill -0 ${process.pid}` 盯的是插件实例 pid，`dsh-unlag` 重载换 pid 后看门狗立刻 kill 共享 Chrome —— 镜像白屏、会话掉回 about:blank。
+- 改为盯应用 pid（`ps -o ppid=` 向上走到顶）+ `watcher.json` 去重；实测 node 14185 的应用 pid 是 13663。
+- `ensure()` 无活页面时回落到 `rememberedTab(id)`，首开即回原地址。
+- 测试 97 条全绿。
+
+## Phase 35 — a review round that can be checked off, 0.34.13## Phase 35 — a review round that can be checked off, 0.34.13
+
+- `markRound()` 追加 verdict 行，`lastRound()` 折回 `marks`；`brief({already})` 只保留 open 条目，实测提问里出现「Already handled from your previous review」，已处理点不再重复。
+- 工具面新增 `sidecar_review_verdict`（`mark: [{point,status,evidence}]`，状态枚举 fixed/verified/invalid/open）；越界编号丢弃、未知状态记为 open。
+- 测试 92 → 96 条全绿。
+
+## Phase 34 — one gate in front of every write, 0.34.12## Phase 34 — one gate in front of every write, 0.34.12
+
+- `acting()` 统一 400（未命名会话）与 403（非持有者）；实测匿名 `POST /sidecar/nav` → 400 `which chat asked for it`，非持有者 `POST /sidecar/stop` → 403，持有者 `forget` → 200。
+- 测试 90 → 92 条，全绿；文案与样式 lint 无 error。
+
+## Phase 33 — a lost picture comes back in one click, 0.34.11
+
+- 实测「切走 / 重载后画面没了」的下一步：`/sidecar/state?session=…` 与画面流 hello 都给出 `chat.remembered`（该会话最后地址）。
+- 右栏在没有画面且 `remembered` 有值时显示「恢复上次的页面」，点击 POST `/sidecar/open` 并重连画面流。
+- 同一会话第二次 open 命中已有标签页：107 ms（三次实测 2.08 s / 0.107 s / 0.108 s，首次含导航）。
+
+## Phase 32 — a call log belongs to its chat, 0.34.10
+
+`GET /sidecar/state` returned the last 40 mirror calls of every chat, each with
+an eight-character session id: one conversation could read where another had
+been browsing. The list is now filtered to the caller's own chat. The owner and
+the list of listening chats stay global, because those are what the column needs
+to claim the mirror.
+
+## Phase 31 — a reload keeps the mirror, 0.34.9
+
+`ctx.effect` cleanup ran `sidecar.shutdown()`, so each plugin reload closed
+Chrome. The page disappeared from the column and the next open was a cold
+start again. Cleanup now hands the browser to the next driver: cookies and a
+login snapshot hit disk, the browser keeps running, and a detached watcher
+closes it only after the app process is gone.
+
+| after a `dsh-unlag` reload | before | now |
+|---|---|---|
+| browser | closed, page lost | still running, page kept |
+| next interaction | cold launch (≈0.6 s, was 21 s) | attach, no launch |
+
+## Phase 30 — a brand-new chat's first tab, 0.34.8
+
+A fresh profile with no memory of a page means `openTab` asks for `about:blank`.
+`navigate` then polled for a non-blank address, which a blank page never becomes,
+and spent all 200 tries. Every first open of a new chat paid it.
+
+| cold start, fresh profile (`about:blank`) | before | now |
+|---|---|---|
+| round 1 | 21 322 ms | 817 ms |
+| round 2 | 20 882 ms | 607 ms |
+| round 3 | 20 900 ms | 516 ms |
+| page settings round trips | 4 | 1 |
+
+Measured with three launches of `Sidecar#ensure` against a throwaway profile;
+the first launch is now about 26× faster than before.
+
+## Phase 29 — a stale session cookie is not written back, 0.34.7
+
+`restoreSessionCookies` replayed every kept session cookie on launch, including
+ones the browser already had. A rotated `sessionid` from the note would replace
+the current one, and the site answers a retired session with a sign-in page.
+
+| case | before | now |
+|---|---|---|
+| note holds sessionid, browser holds rotated one | old value written | nothing written |
+| note holds a cookie the browser lost | written | written |
+| cookie key | — | domain + path + name |
+
+86 tests complete the suite in about 2 s.
+
+## Phase 28 — one chat, one tab, even in parallel, 0.34.6
+
+`tabFor` checked `pages`, then did async work, then wrote `pages`: two calls that
+started together both missed the map and both opened a tab. The orphan tab kept
+a live page nobody could see or stop.
+
+| case | before | now |
+|---|---|---|
+| two parallel calls, cold chat | 2 tabs, 1 orphan | 1 tab |
+| later call with another address | its own tab | same tab, navigated |
+| leftover pending note | — | cleared in `finally` |
+
+The fix is a per-chat promise in `opening`, joined by the second caller.
+
+## Phase 27 — one gate for the mirror, 0.34.5
+
+Three routes each carried their own copy of the ownership check. Copies drift:
+a route added later is exactly the one nobody remembers to guard.
+
+| case | result |
+|---|---|
+| rival chat → `/sidecar/input` | 403, owner named |
+| rival chat → `/sidecar/nav` | 403, owner named |
+| rival chat → `/sidecar/frames` | 403, no stream opened |
+| unit tests | 84 |
+
+The gate reads one function, so the question "may this chat act?" has one answer
+in one place.
+
+## Phase 26 — a screenshot the model can see, 0.34.3
+
+Observation was structured text only: `sidecar_shot` wrote a PNG and returned
+a path, which is a file name the model cannot look at. Codex and Claude Code
+both hand screenshots back as image content, and that is the difference
+between planning from the picture and guessing.
+
+| case | before | now |
+|---|---|---|
+| route with image input | path only | path + image block, same call |
+| route without image input | path | path, no image block, no error |
+| no attachment store | path | path |
+| read state across chats | one shared object | one per chat |
+| unit tests | 79 | 81 |
+
+Same-call cost: the PNG is already in memory for the file write, so the image
+block adds one attachment save, no extra round trip and no second screenshot.
+
 <!-- ui-copy-ignore -->
 # Measured, not claimed
 
@@ -232,3 +467,253 @@ Measured with `/tmp/probe-unseen.mjs` (tool layer, example.com, profile
 `probe-tool`): 1344 ms with nothing watching and no column asked for, 3236 ms in
 the first version of the same wait that did not skip the case where no column was
 ever requested — the whole difference was the wasted 1.2 s.
+
+## Phase 9 — finding the field behind the words, 0.19.0
+
+A read of a `/bare` form (two div labels, two bare inputs, no aria anywhere on
+the fields) and one `{field: "Work email"}` step:
+
+| what | result |
+| --- | --- |
+| the click lands on the field, not on the div | yes, value `me@x.y` in `input[name=wk]` |
+| the second, untouched field | still empty |
+| the tool's own words | `typed into "Work email"` |
+| worst case when nothing matches | one extra in-page query, then the old behaviour |
+
+The query walks the open shadow roots (capped at 400) and, per visible editable
+element, builds a list of candidate names from real labels, `aria-labelledby`,
+`aria-label`, `placeholder`, `title`, `name`, `label[for]`, the element right
+before it, and its wrapper's text — then scores exact (3) / contains (2) /
+reverse-contains (1).
+
+## Phase 10 — typing by name, 0.20.0
+
+| what the caller has | what it used to take | now |
+| --- | --- | --- |
+| a name for the field, no number | read the page, find the number, then type | one `sidecar_type` with `field`, one round trip |
+| the field's words live in a div | click landed on the div, text went wherever focus was | resolved to the field, `typed into "Work email"` |
+| a banner over the field | typed under the banner | banner cleared first, then typed |
+
+Breadth check `type-by-words` on the `/bare` form: value lands in
+`input[name=wk]`, the page does not move (no stray Enter).
+
+## Phase 11 — choosing a dropdown by name, 0.21.0
+
+`/bare` now carries a `select[name=seats]` whose only name on screen is a div
+saying `Seats`. One `sidecar_select` with `field: "Seats"`, `value: "Two"`:
+
+| what | result |
+| --- | --- |
+| the value after the call | `2` |
+| the tool's own words | `chose Two in Seats` (on-screen name, not `name="seats"`) |
+| a wrong option | answered with the first 12 options instead of a page read |
+| round trips to do it | one; no read beforehand, no read after |
+
+The naming rules (`labelSpot` and `selectSpot`) now live in one shared in-page
+block, so the two steps cannot drift apart.
+
+## Phase 12 — words split across nodes, 0.22.0
+
+`/words` now carries `<button><span>Sign</span><span>in</span></button>`, which
+the page reports as `Signin`.
+
+| what | result |
+| --- | --- |
+| `{text: "Sign in"}` before | not found; caller had to fall back to a number |
+| `{text: "Sign in"}` now | clicked, title changes, `ok click "Sign in"` |
+| extra work | none: the same single control list, one comparison changed |
+
+Exact equality still wins over the squeezed form, so `Sign in` never lands on
+`Sign up` when both exist.
+
+## Phase 13 — the field behind the box, 0.23.0
+
+`/bare` gains the date-picker shape: a visible rounded box with a transparent
+`<input>` laid over it, its words in a div two levels up.
+
+| what | before | now |
+| --- | --- | --- |
+| `{field: "Birthday"}` | not found, fell back to clicking the div | found the hidden input, clicked its visible box |
+| value after one `sidecar_do` step | whatever already had focus | `2001-02-03` in `#bd`, `typed into "Birthday"` |
+| extra round trips | read the page, guess a number | none |
+
+The visible-box rule: a candidate is usable when it, or its nearest visible
+ancestor, has a real box — so a 0-opacity input still counts if it covers
+something the person can see.
+
+## Phase 15 — the answer is the picture, 0.25.0
+
+`/pic` is one 140x90 PNG and nothing else.
+
+| what | result |
+| --- | --- |
+| `sidecar_grab` | 6 ms, `picture-check.png`, 140x90 |
+| before | three round trips: list the images, ask for the full address, fetch it |
+
+A picture page has no text at all, so a read of it costs one line and no tokens;
+the picture comes back as a file path the person can open.
+
+## Phase 16 — say what the picture looks like, 0.26.0
+
+Two 140x90 PNGs: one a gradient, one a single colour.
+
+| page | what `sidecar_grab` answers | time |
+| --- | --- | --- |
+| `/pic` | `... 140x90 a picture with detail` | 21 ms (first, page cold) |
+| `/flat` | `... 140x90 mostly one colour` | 4 ms |
+
+The sampling is 32x32 inside the page, so it costs no round trip and its result
+travels in the same line as the path.
+
+## Phase 17 — the answer is a picture, in the read itself, 0.27.0
+
+| page | read line | extra round trips |
+| --- | --- | --- |
+| `/pic` (gradient, no words) | `picture 140x90 a picture with detail "Chart" — sidecar_grab saves it as a file` | 0 |
+| `/flat` (single colour) | `picture 140x90 mostly one colour "Chart" — …` | 0 |
+| `/words` (text page) | no picture line | 0 |
+
+The sampling is 32x32 inside the page, in the same call that reads the page, so a
+blank chart is visible before anything is saved.
+
+## Phase 18 — the picture that never arrived, 0.28.0
+
+| page | read line | time |
+| --- | --- | --- |
+| `/broken` (320x240 `<img>` pointing at a 404, no words) | `picture 320x240 "Chart" did not load — the page may need a reload, or the picture is gone` | 123 ms (cold page) |
+
+A failed image collapses to an error icon, so the declared `width`/`height` is
+what the page meant to show; the read reports that size, not the icon's.
+
+## Phase 19 — the page said again, 0.29.0
+
+| read | characters | time |
+| --- | --- | --- |
+| first read of `/dense` (40 paragraphs) | 2057 | — |
+| second read, nothing changed | 105 | 1 ms |
+
+The comparison is free: both reads already happened. Only the report shrinks.
+A changed scroll offset is still reported, so "nothing changed" is never a lie.
+
+## Phase 20 — one call carries the question and the answer, 0.30.0
+
+| step | before | `sidecar_ask` |
+| --- | --- | --- |
+| type the message | `sidecar_type` + `submit` | same, inside the call |
+| read the page | `sidecar_view` | — |
+| read it again until the answer stops growing | 2–6 more `sidecar_view` calls | — |
+| **turns for a chat reply** | 3–7 | **1** |
+| measured `sidecar_ask` on `/chat` (answer over 2 arrivals, 700 ms tail) | — | 2.5 s, one turn |
+
+## Phase 21 — a review round that closes the loop, 0.31.0
+
+| step | before | `sidecar_review` |
+| --- | --- | --- |
+| write the request (repo, commit, focus) | by hand, every round | built from three fields |
+| send it, wait for the answer | `sidecar_ask` | same call |
+| turn the answer into a list | read it, retype it | parsed, numbering kept |
+| know what the last round was about | — | round log, with the commit and the time |
+| **turns for a review round** | 3–7 + bookkeeping | **1** |
+| measured round on `/chat` (echoed question + 2-item list) | — | 2.6 s, 2 points |
+
+## Phase 21b — an answer that is not an answer, 0.31.1
+
+A chat box on a page that is not signed in still accepts text, still redraws,
+and still looks busy. Three cheap checks keep that from being reported as a
+review.
+
+| what the page does | before | now | cost |
+| --- | --- | --- | --- |
+| keeps the words in the box | waits 4 s, then reads the page | says so at 0.5 s | 1 evaluate |
+| redraws but never shows the message | the new words were the answer | says the page did not take it | none (text compare) |
+| shows the message, no reply | "answered in prose" | "has the message but has not answered yet" | none (text compare) |
+| real page, signed out (Doubao) | a review-shaped paragraph of menu chips | the sign-in is named, 2.6 s | — |
+
+## Phase 22 — the change travels with the request, 0.32.0
+
+The reviewer inside the column cannot open a repository. Asking it to review
+"the newest commit" of a repository it cannot see is asking for a guess.
+
+| request | what the reviewer gets | cost |
+| --- | --- | --- |
+| repo + commit | an address and a hash it cannot open | 1 message |
+| repo + commit + diff | the change itself, capped at 5000 characters | +diff characters in that one message |
+
+Measured: `review-carries-diff` green in 2.6 s; the recorded round carries
+`+const cap = 60` verbatim.
+
+## Phase 22b — the mirror outlives the visit, 0.33.0
+
+A column tab only exists while the chat is on screen. The ask used to be a
+single frame pushed at request time: if that chat had no stream, the frame was
+lost and the mirror never appeared on return.
+
+| situation | before | now |
+| --- | --- | --- |
+| the chat is on screen | tab opens | tab opens |
+| the chat is elsewhere, then returns | nothing | tab is waiting |
+| the page was reloaded | nothing | tab is waiting |
+| the ask is 3 minutes old | nothing | nothing (spent) |
+| a different chat comes back | nothing | nothing (not its mirror) |
+
+Data flow never needed the column: measured with zero viewers — boot 626 ms,
+click and read back 114 ms, read 4 ms, shot 48 ms / 12.5 KB.
+
+## Phase 22c — a duplicate cannot take the pane down, 0.33.2
+
+A scan of the six shipped modules finds no repeated top-level name, no module
+imported twice, no two tools with one name, one tab id and one style sheet.
+The two places a reload can still meet itself are guarded instead of trusted:
+
+| case | before | now |
+| --- | --- | --- |
+| same tool name registered again | thrown error, surface down | warning, first one kept |
+| same tab mounted again | possible duplicate registration | no-op |
+| style sheet re-applied | second sheet stacked | replaced by id |
+| two mount entry points | possible double effect | one entry per half |
+
+Guards are covered by `test/no-duplicate.test.mjs` plus a double-mount test:
+65 unit tests, 66 breadth checks, 20 packaged files.
+
+## Phase 25 — one chat cannot take another chat's page, 0.34.2
+
+The audit on 0.34.0 named five things a driver written for one chat got wrong.
+Each was reproduced against 0.34.1 first: two were already fixed (the
+`sessionScope`/`KEYS` import, and every `ensure()` route already carrying a
+session), three were real.
+
+| What | Before | Now |
+| --- | --- | --- |
+| popup lookup | any non-main tab, then navigated or closed | only a target that is neither this page nor another chat's (`isForeign`) |
+| focused password in a result line | echoed the `.value` (`DUMMY_SECRET_123`) | name only, value always empty |
+| `sidecar_do` on a step that did not land | kept going, typed into the wrong page | stops at that step, says to re-plan |
+| overlay that would not clear | clicked the control under it anyway | refuses the click/typing, names the overlay |
+| `changed_only` twice | repeated the same change | diff becomes the new baseline |
+| `/sidecar/state` | top-level `url` could be another chat's tab | `?session=` adds `chat.url/title/viewers` (live: doubao page, viewers 1) |
+| suite | 77 pass | 79 pass (isolation, leak) |
+
+## Phase 24 — a number that is checked before it is used, 0.34.1
+
+| What | Before | Now |
+| --- | --- | --- |
+| ref on a control whose words changed | pressed the new control | refused, says both words (0 ms extra on an unchanged ref) |
+| sidecar call after a hot reload | `sessionScope is not defined` | works; the Host half is checked by a test |
+
+## Phase 23 — one tab per chat, typing that looks like hands, 0.34.0
+
+The browser used to be one page for the whole app, so any chat could navigate
+the page another chat was reading. Tabs are now per chat; one windowless Chrome
+and one profile still back all of them, so a login is shared and an address is
+not.
+
+| what | measured |
+| --- | --- |
+| first chat, browser cold | 725 ms to a driven page |
+| second chat, own tab | 1 round trip, no second Chrome |
+| chat A navigates | chat B's URL and the first tab unchanged |
+| frames per chat | `/sidecar/frames?session=` streams only that tab |
+| paste, 46 chars | 5 ms |
+| human typing, 46 chars | 1965–2238 ms (43–49 ms per char) |
+| human typing, >160 chars | 8-char bursts, 16–58 ms gaps, ≈6 ms per char |
+| headless UA | desktop Chrome UA, no `HeadlessChrome` |
+| suite | 70 pass, breadth 66, ui-copy clean |

package/docs/ROADMAP.md

@@ -174,3 +174,149 @@ Next in this direction: the drawing half could re-attach by itself. A tab that
 reloads re-subscribes on its own, but a tab that never loaded the half cannot be
 told to — that is a load-order problem, so the fix belongs in how the panel half
 is delivered, not in the wait.
+
+## Phase 10 — words are not where the field is, 0.19.0
+
+The tool used to treat "the control that says Email" as "the field called
+Email". Real forms separate the two, so the writing step now resolves the words
+to the element that takes text, and says which one it used.
+
+Next in this direction: the same resolution for `sidecar_type` when the caller
+names a field by its words instead of a ref, and for date/time pickers where the
+visible box is a div and the real input is hidden beside it.
+
+## Phase 11 — say where, not which number, 0.20.0
+
+Typing by name is in. The same resolution is what other steps should be able to
+use when the caller knows the destination but not the number — the next place
+worth doing this is `sidecar_select` (choose an option by the field it belongs
+to), and after that the date/time case where the visible box is a div and the
+real input sits hidden beside it.
+
+## Phase 12 — the same words, the same place, 0.21.0
+
+`sidecar_select` joined `sidecar_type` and `sidecar_do` on the shared resolver.
+Left in this direction: the date/time case where the box the user sees is a div
+and the real input is hidden beside it, and `sidecar_click` on a control whose
+words are split across sibling nodes (`<span>Sign</span><span>in</span>`).
+
+## Phase 13 — the page's spelling is not the caller's, 0.22.0
+
+Whitespace is normalised in both halves now (tool side and in-page side). Next
+in this line: the date/time box where the visible box is a div and the real
+input is hidden beside it, which needs the resolver to accept a hidden target
+and click the thing in front of it.
+
+## Phase 14 — behind the box, 0.23.0
+
+Both halves now resolve a name to a place: visible field, hidden field behind a
+visible box, words split over nodes, words outside the wrapper. Next: a page
+whose visible box is a `<canvas>` or an `<iframe>` — nothing in the tree to
+name, so the fallback has to be a picture plus a click, not a name lookup.
+
+## Phase 16 — a file, not a path, 0.25.0
+
+`sidecar_grab` returns a path. Next: say one line about the picture (size, and
+whether it is mostly one colour) so a model that cannot open files still learns
+"the chart is blank" without a full read.
+
+## Phase 17 — a picture nobody can see, 0.26.0
+
+The description is coarse on purpose. Next: when a page's answer is a picture,
+let a read say so in one line ("the answer is a picture, and it is blank") so a
+model that cannot open files does not need the tool at all to notice a failure.
+
+## Phase 18 — a picture that never arrived, 0.27.0
+
+A picture is only an answer if it loaded. Next: report a large picture element
+whose source failed (`naturalWidth === 0`) as a broken picture rather than
+staying silent, since that is the commonest way a generated image "disappears".
+
+## Phase 19 — the same picture again, 0.28.0
+
+Reads are stateless: the same page costs the same tokens twice. Next: remember
+the last read of a page by address and say only what changed, with a hard reset
+when the address or the document changes.
+
+## Phase 20 — two browsers, one conversation, 0.29.0
+
+The column is a real browser, so the model inside it can be a reviewer, not just
+a tool: hand it the repository address, ask for a review, read the answer back
+through the same mirror, and take the next round from it. Nothing opens on the
+desktop while that happens: a site that refuses a headless build gets a window
+at -3000,-3000, and the column shows the only picture.
+
+## Phase 21 — a review round that closes the loop, 0.30.0
+
+`sidecar_ask` sends and returns an answer. Next: a round trip that carries the
+repository address there and the review back, and records in the round log which
+commit the review was about, so a later round can tell whether the advice was
+already taken.
+
+## Phase 22 — the reviewer sees the code, not a description of it, 0.31.0
+
+The round log now records what was asked and what came back. Next: paste the
+diff the review is about into the same message (bounded, so the column does not
+fill with code), and let a later round say which points were already acted on,
+so the loop stops repeating advice.
+
+## Phase 23 — the drawing half, 0.31.0
+
+A page that never shipped an image still has a canvas or a video frame; take
+that surface as the picture when the page has no file to grab.
+
+## Phases 24–39, condensed
+
+The phases above were written one at a time as the work happened; the run from
+0.32 to 0.34.18 is dense enough that the per-phase prose lives in
+`docs/BENCH.md` and the changelog. What each one changed, in one line:
+
+| Version | Phase | What it moved |
+| --- | --- | --- |
+| 0.32.0 | 22 | the diff a review is about travels with the request |
+| 0.33.0 | 22b | the mirror outlives the visit: claim, takeover, remembered address |
+| 0.33.2 | 22c | a duplicate registration cannot take the pane down |
+| 0.34.0 | 23 | one tab per chat, a UA that does not say "headless", typing that looks like hands |
+| 0.34.1 | 24 | a ref is checked against what the control says before it is used |
+| 0.34.2 | 25 | one chat cannot take another chat's page; a password never comes back in a result |
+| 0.34.3 | 26 | a picture reaches the model as a picture; read state is per chat |
+| 0.34.4 | 27 | an empty mirror says what to do; remembering an address is written, not crashed |
+| 0.34.5 | 28 | one ownership gate for every acting route |
+| 0.34.6 | 29 | one tab per chat even when two calls race |
+| 0.34.7 | 30 | a live cookie is never overwritten by a stale copy |
+| 0.34.8 | 31 | a cold first tab in 0.6 s instead of 21 s |
+| 0.34.9 | 32 | a plugin reload no longer closes the browser |
+| 0.34.10 | 33 | a call log never shows another chat's calls |
+| 0.34.11 | 34 | the column offers "restore the last page" in one click |
+| 0.34.12 | 35 | every write needs the chat that asked for it |
+| 0.34.13 | 36 | a review round is verifiable: each point marked fixed / verified / invalid / open |
+| 0.34.14 | 36b | a reload must not cost the browser: the watchdog follows the application pid |
+| 0.34.15 | 37 | an observation admits its blind spot: pictures, canvases, videos, frames; an error page says so |
+| 0.34.16 | 38 | a reload does not blank the mirror: the owning chat's poll adopts the running browser |
+| 0.34.17 | 39 | the blind-spot frame count follows the screen, not the DOM |
+| 0.34.18 | — | the adoption rule is a named function with a test |
+
+## 页游这一档（2026-10-28 起）
+
+输入这一层已经到位：按住键、组合键、按住鼠标，一项 300 ms 的按住整次调用 305 ms，事件由页面自己证实。实时画面沿用同一条 CDP 帧路（60 fps、镜像延迟中位数 8 ms）。
+
+还差的一层是「看」：4399 / 7k7k / 奥拉星这类页游把画面画在 canvas 上，文本树里没有可读的东西，判断走位、Boss 血条、按钮位置都要靠像素。本会话的模型没有视觉能力，所以这里的正确做法不是假装能玩，而是把可判读的信号做出来：canvas 局部像素采样、颜色/区域变化率、画面是否在动（`blind` 行已经在报 canvas 数量）。有视觉能力的模型接着用同一套帧路就能玩。
+
+## Where the next round should go
+
+Ranked by what a real task still loses:
+
+1. **End-to-end proof of the places unit tests cannot reach.** The restore-in-one-click
+   path (0.34.11) and the mirror coming back after a reload (0.34.16) were checked by
+   HTTP and by pid, not by a person clicking. A browser-level test that opens the
+   column, kills the driver, and clicks restore is the missing evidence.
+2. **One-click "open here" when this chat has no seat.** `state.chats` can lack this
+   chat, and the column then only tells the model to call `sidecar_open`. The button
+   belongs in the column.
+3. **Competitor behaviours still unmatched.** Playwright MCP's accessibility refs are
+   ours, and the blind line covers the vision-less read; Stagehand v4's
+   act/observe/extract split with cached selectors is only partly ours, and its
+   self-healing of a selector between rounds has no equivalent here.
+4. **Token numbers for a real session, not a synthetic page.** Reads are measured on
+   fixed pages; an actual 20-step task has never had its cumulative token cost
+   recorded.

package/docs/VERIFY.md

@@ -149,3 +149,17 @@ hand on the mirror: while the person is using it a read still works
 (`hold-refuses-action`), and `sidecar_resume` hands the page back so the same
 click goes through (`resume-lets-go`).
 
+## Who the mirror serves
+
+The bar that matters most cannot be reached from `node --test`, because it is
+about a column that belongs to one conversation. With DSH running:
+
+```sh
+node scripts/live-safety.mjs http://127.0.0.1:49552
+```
+
+It reads the state, then tries the two ways in — a frame stream and a
+navigation — as a session that is not the owner. Both must answer 403, and the
+owner's stream must answer 200. Three `true` lines is the bar; `open · owner
+none` as the first line only means nothing is on screen, and the two refusals
+still have to hold.