@forsion/tangu-computer-use 0.5.8

package/UPSTREAM.md

@@ -0,0 +1,277 @@
+# Upstream & vendor strategy
+
+Forked from **[injaneity/pi-computer-use](https://github.com/injaneity/pi-computer-use)**
+@ `4b8dbd7e` (v0.5.1, 2026-08-31), MIT. (Previously `c838d3a` = v0.5.0; resynced 2026-09-07.) Upstream license: `LICENSE.upstream`.
+
+> Repo shape: this repo **is a Forsion bundle** (`manifest.json` + `main.js` + `skills/` at the root,
+> engine plugin under `tangu-plugins/computer-use/`). See "Bundle layout" at the bottom.
+
+## Vendor strategy — vendor is verbatim except a mechanical brand rename
+
+`src/vendor/` is the upstream `src/` copied verbatim **except one mechanical, greppable rename**
+(see "Brand rename" below — the helper's identity/path constants must match the branded native
+app or nothing connects) **and one deliberate behavioural constant**: `HELPER_PROTOCOL_VERSION` in
+`platform/macos/helper.ts`, which we hold above upstream's (see "Helper protocol version" below).
+Re-syncing must not blindly restore upstream's number — that silently disables every native feature
+we added. The entire *code* coupling to pi (`@earendil-works/pi-coding-agent`)
+is 5 files:
+
+| upstream file | pi import | how it's satisfied |
+|---|---|---|
+| `bridge.ts` | `AgentToolResult`, `AgentToolUpdateCallback`, `ExtensionContext` (type) | `src/pi-compat.ts` |
+| `permissions.ts`, `platform/types.ts`, `platform/macos/permissions.ts` | `ExtensionContext` (type) | `src/pi-compat.ts` |
+| `config.ts` | `getAgentDir` (runtime) | `src/pi-compat.ts` |
+
+`tsconfig.json` `paths` + `scripts/build.mjs` esbuild `alias` both map
+`@earendil-works/pi-coding-agent` → `src/pi-compat.ts`, so **the vendor import lines are not
+edited**. Config is injected by mutating the `activeConfig` object returned by
+`getComputerUseConfig()` (see `src/tools.ts` → `syncSettings`), so `config.ts` is unmodified too.
+
+### Tangu adaptation layer (NOT vendored — hand-written)
+- `src/pi-compat.ts` — the pi shim (only the shapes vendor actually uses).
+- `src/tools.ts` — upstream's 11 `defineTool` → Tangu `ToolDef[]`; typebox schema hand-translated
+  to plain JSON Schema; `AgentToolResult.content` → return string + `ctx.collectImage` (screenshots).
+- `src/settings.ts` — 3 keys (browser_use / headless / cursor_overlay).
+- `src/setup.ts` — `tangu computer-use setup|doctor|stop` (readline `ExtensionContext` reuses
+  vendor `ensureComputerUseSetup`).
+- `src/index.ts` — the `TanguPlugin` entry.
+
+### Brand rename (the one vendor patch)
+The native helper is branded (`com.forsion.tangu-computer-use`, `/Applications/tangu-computer-use.app`,
+socket `~/Library/Caches/tangu-computer-use/bridge.sock`, Windows exe under `tangu-computer-use/`).
+The vendor **runtime** must point at those exact names — `HELPER_BUNDLE_ID` (used by `open -b`),
+`HELPER_APP_PATH`, the socket dir, and `WINDOWS_HELPER_PATH`. So the vendor gets one mechanical
+rename applied across `src/vendor/**.ts`:
+
+```
+s/\bpi-computer-use\b/tangu-computer-use/g   # lowercase token: bundle id, app/socket/exe paths, user-facing strings, config filename
+s/\binjaneity\b/forsion/g                    # com.injaneity.… → com.forsion.…
+```
+
+Plus these **exact user-facing phrases** (never a blanket `\bpi\b` — that would hit the
+`@earendil-works/pi-coding-agent` import specifier the alias depends on):
+
+```
+Restart Pi to use the installed helper.        → Restart Tangu to use the installed helper.
+Restart Pi so the canonical helper is used.    → Restart Tangu so the canonical helper is used.
+Start pi in interactive mode.                  → Run 'tangu computer-use setup' in a terminal.   (single quotes — the string sits inside a template literal)
+Windows helper closed because the Pi session ended. → … the Tangu session ended.
+Enable browser_use in ~/.pi/agent/extensions/… or .pi/computer-use.json to allow browser windows.
+                                               → Enable the computer-use plugin's browser_use setting to allow browser windows.
+```
+
+Also `Linux helper closed because the Pi session ended.` → `… the Tangu session ended.` (v0.5.0's
+Linux backend mirrors the Windows string).
+
+Case matters: the `PI_COMPUTER_USE_*` / `PI_CU_*` **env-var names are a different token** (uppercase +
+underscores) and are deliberately left unchanged, as are code comments mentioning Pi. After the rename,
+`grep -rn "pi-computer-use\|injaneity" src/vendor` must be empty (the rename now also covers `.mjs` /
+`.d.mts` — v0.5.0 added `platform/macos/helper-path.mjs`). This mismatch was a
+real bug once (native branded, vendor still `pi-*`): the runtime launched an unregistered bundle id and
+connected to the wrong socket, so `doctor` reported ready while a real observe/act failed.
+
+### PACKAGE_ROOT — no longer patched, but still guarded
+`platform/{macos,windows,linux}/helper.ts` locate `scripts/setup-helper.mjs` via
+`path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..", "..")`. Correct in the source
+tree (`src/vendor/platform/<os>/` → package root). Once bundled it depends on **where the bundle lands**:
+
+| bundle output | 3 levels up | verdict |
+|---|---|---|
+| `<root>/dist/index.js` (v0.1, engine-plugin layout) | two levels *above* the repo | ✗ — needed a build-time rewrite |
+| `<root>/tangu-plugins/computer-use/dist/index.js` (bundle layout) | `<root>` | ✔ — upstream's own math is right |
+
+So the bundle move **deleted a whole vendor patch**. `scripts/build.mjs` keeps two guards instead:
+① the expression must still be present in every platform helper (upstream changing it fails the build),
+② after the build it resolves 3 levels up from the real output dir and asserts `scripts/setup-helper.mjs`
+is there. Move the output and the build tells you.
+
+`scripts/setup-helper.mjs` gets one path patch of its own: upstream imports
+`../src/platform/macos/helper-path.mjs`; ours is `../src/vendor/platform/macos/helper-path.mjs`
+(and that file is listed in package.json `files`, or the published package can't install its helper).
+Since 0.5.3 the macOS installer is intentionally diverged: **never restore upstream runtime
+certificate discovery/import or signing**. `build-native.mjs` creates sealed App ZIPs and checksum
+manifests, `macos-bundle.mjs` verifies and atomically installs them, and onboarding uses the same
+complete-app check. ZIPs keep Electron's outer `--deep` signature from rewriting the helper.
+`check:installer` traps every subprocess and rejects keychain/private-key operations. Keep this
+guard in both plugin and Genesis release pipelines.
+
+Historical cause (removed in 0.5.3): `codesign --sign <local cert>` needs the private key, so it raises a
+**keychain dialog** — run it anywhere nobody can click and it hangs, and killing it leaves a
+`MacOS/*.cstemp` behind. The next `codesign --deep` then seals that temp file into `CodeResources`
+and deletes it on the way out, so the signature fails verification **forever**
+(`a sealed resource is missing or invalid`). Because TCC matches the *designated requirement*, the
+app silently loses Accessibility/Screen Recording and **toggling the checkbox in System Settings can
+never fix it** — the stored grant no longer matches the binary. Re-signing cleanly with the same cert
+restores the original requirement and the existing grants light up again; no uninstall, no
+`tccutil reset`. (2026-07-28, cost a long debugging session.)
+
+### Native
+`native/macos` (Swift), `native/windows` (Rust) and `native/linux` (Rust) are vendored with the same
+brand strings rewritten. **Never change the bundle id or signing identity across releases** — macOS ties
+Accessibility/Screen-Recording (TCC) grants to them.
+
+> ⚠️ v0.5.0 changed where the macOS helper installs: an existing **writable** `/Applications` install
+> stays put, everything else goes to `~/Applications` (no admin needed). Users whose `/Applications`
+> is not writable will be asked to re-grant TCC once, because the path moved.
+
+**Helper protocol version — bumped by us.** Upstream v0.5.1 is still at 6; we run **12**, in two places that must
+always match: `native/macos/bridge.swift` (`private let protocolVersion`) and
+`src/vendor/platform/macos/helper.ts` (`HELPER_PROTOCOL_VERSION`). Bump it whenever the helper gains a
+command **or changes native behaviour** — `src/onboarding.ts` now notices a stale binary *on disk*, but a
+**already-running** daemon is only re-validated through the protocol number, so without a bump the old
+process keeps serving after the update lands. 0.4.0 is the cautionary case: it changed only overlay window
+levels and one CGWindowList call, no new commands — a same-protocol daemon would have gone on drawing
+invisible overlays and the user would have reported the bug a fourth time. Without a bump, an old installed helper keeps serving observe/act while
+every new native feature silently does nothing (that is exactly how the first live-view build shipped
+"working"). With a bump, `ensureProtocol()` restarts the daemon and then throws a clear
+"reinstall or rebuild the helper app" error. On resync, if upstream has moved past 6, go one above theirs.
+
+**Tangu-only native additions (not upstream, replay them after every sync):**
+- `native/macos/agent_highlight.swift` + `agent_highlight_tests.swift` — the glowing edge around the
+  window being controlled. Listed in `helperSourcePaths` (setup-helper.mjs) and `macosSourcePaths`
+  (build-native.mjs); forget either and the helper compiles without it.
+- `native/macos/live_stream.swift` — `LiveWindowStream`, a **persistent `SCStream`** for the controlled
+  window plus a window-sized one-shot (`captureOnce`). Same two source lists as the highlight.
+  Two things it exists to fix, both measured, both easy to undo by accident:
+  ① `SCScreenshotManager.captureImage` renegotiates a capture session per frame — fine for one
+  screenshot, a 1fps slideshow for a live view. The stream keeps the latest frame; a fetch is ~12ms.
+  ② upstream's `captureWindow` sets **no** `config.width/height`, so ScreenCaptureKit renders at
+  *display* size with the window pinned top-left and the rest blank — a 230×408 Calculator comes back
+  as 1920×1080 mostly white. That whitespace is baked into the JPEG; no amount of CSS fixes it.
+  Anything serving the live view must set width/height. `captureWindow` itself is left alone because
+  `look` depends on its coordinate space.
+- `bridge.swift` `act()`: **coordinate clicks try an AX hit-test first** (`backgroundPressAtPoint`).
+  Upstream sends coordinate clicks straight to HID/foreground (`needsForeground` in `actions.ts`)
+  because `CGEventPostToPid` is dropped by non-key windows — so "look at the screenshot and click"
+  always stole the foreground. Hit-testing **against the target app element** (never systemWide: the
+  occluding window would answer) and pressing via AX keeps it in the background. Guarded to
+  left-button single clicks, non-web, non-text, and the hit must be inside the target window's subtree.
+  `npm run check:blindclick` proves all of it on a real machine.
+- `bridge.swift`: `noteControlledWindow()` called at the top of `act()`, a matching write inside
+  `storeLookRecord()`, and the `liveView` command (case + method). The highlight reuses the existing
+  `cursorOverlay` request flag — no new config key, so `platform/macos/backend.ts` is untouched.
+- `agent_cursor.swift` (vendor file, small patch): the overlay window spans **every** display and is
+  re-framed on each `animate()`, and the global CG point is converted to overlay-local coordinates via
+  `OverlayGeometry.canvasPoint`. Upstream sizes it to `NSScreen.main?.frame` only, so a click on a
+  secondary display draws the cursor outside the window — invisible, which reads exactly like "the
+  assistive cursor never appears". The geometry lives in `agent_highlight.swift` (our file) so
+  `npm run check:highlight` can pin it; single-display setups are an exact identity, unchanged.
+  ⚠️ "Primary" means the screen whose frame origin is `(0,0)` — not `NSScreen.main` (that follows the
+  key window) and not necessarily `screens.first`.
+- `bridge.swift` `postMouseClick()`: save/restore the **physical cursor** around HID clicks. A
+  `.cghidEventTap` post drags the user's real pointer to the target and leaves it there; having the
+  foreground taken is bad enough without also losing your mouse. Guarded — it only warps back when the
+  pointer is still where we put it (so it never fights a user who grabbed the mouse mid-click), and
+  `CGAssociateMouseAndMouseCursorPosition(1)` after the warp or the pointer appears stuck. The
+  background (pid) path needs none of this: it never moves the system pointer, it draws `AgentCursor`.
+
+- **Overlay z-order and window bounds — two bugs that made both overlays invisible (0.4.0).** Replay
+  these after any sync; both are one-liners and both fail *silently*.
+  ① `cgWindowBounds()` must use `CGWindowListCopyWindowInfo([.optionIncludingWindow], id)`.
+  **Never** `CGWindowListCreateDescriptionFromArray` — it returns an **empty array** for windows owned
+  by another process (measured on the same window id: 0 vs 1 entries). `AgentHighlight.show()` bails on
+  a nil rect, so the glowing edge had never rendered once since it was written.
+  ② Both overlay windows must set `window.level` (`.floating` for the glow, `.screenSaver` for the
+  cursor). They live in an `.accessory` app that never activates, so at the default level 0 they sink
+  below the controlled app the moment it is activated. Upstream's
+  `window.order(.above, relativeTo: <foreign window number>)` is a **no-op across processes** (measured:
+  the window neither rises nor disappears) — cross-process stacking is only expressible as a level.
+  `npm run check:overlay` samples the on-screen window stack at 100ms and asserts both overlays are
+  present **and above the controlled window**. Geometry unit tests cannot see either bug: "the rect is
+  correct", "the window exists" and "the user can see it" are three different claims.
+
+- **Never spawn `Contents/MacOS/bridge` directly** — launch the daemon with
+  `open -n -g <APP> --args serve --socket <path>` (what `launchDaemon()` does). TCC attributes grants to
+  the *responsible process*, so a direct spawn inherits the caller's identity and `diagnostics` reports
+  "no Accessibility permission". It looks exactly like lost grants and is not.
+
+- **Windows helper links the MSVC runtime statically (0.5.8)** — `scripts/build-native.mjs` (Windows branch)
+  appends `-C target-feature=+crt-static` to `RUSTFLAGS` before running cargo. Upstream links it dynamically,
+  so `windows-bridge.exe` imports `VCRUNTIME140.dll`, which Windows does not ship (it comes with the VC++
+  Redistributable): on a clean machine the helper never starts. Not a `.cargo/config.toml` in the crate —
+  cargo reads config from the cwd, and `build-native.mjs` runs cargo from the caller's cwd with
+  `--manifest-path`; an inherited `RUSTFLAGS` would override config rustflags anyway. `scripts/verify-package.mjs`
+  fails the release if the helper's bytes name any VC++ runtime DLL, so losing this patch in a sync cannot ship.
+  Side effect: a set `RUSTFLAGS` makes cargo ignore config-level rustflags, so a cross-build (`--target`) that
+  needs extra flags passes them in `RUSTFLAGS`. Not applied to `setup-helper.mjs`'s own `cargo build`, which runs
+  when the prebuilt exe is missing (`--runtime`, which the runtime always passes, `--allow-build`, or
+  `PI_COMPUTER_USE_ALLOW_BUILD=1`): that binary only runs on the machine that built it, which has MSVC and its runtime.
+
+### Onboarding — the helper installs itself (0.4.0)
+Since 0.5.2, a successful in-place macOS upgrade also awaits `macosHelper.restart()`. Replacing a
+binary does not replace a running process image, and matching protocol/path checks alone cannot
+detect that case. Keep installer and restart failures in the shared failure handler. Ship changed
+native bits under a new bundle version: Genesis deliberately skips same-version installed bundles.
+`check:helper-refresh` verifies this boundary; `check:helper-signal` probes the selected packaged
+executable on an isolated socket rather than trusting source code or protocol numbers.
+
+`ensureInstalled()` already shells out to `scripts/setup-helper.mjs` (re-entering Electron/Bun via
+`ELECTRON_RUN_AS_NODE`), so a **first** install needs no terminal. Two things break that, and both were
+ours:
+- Any "helper not installed → return instructions" early-return in `src/tools.ts` **defeats it**. Don't
+  add one back.
+- `ensureInstalled()` returns as soon as the executable *exists* and never notices a **stale** binary, so
+  plugin updates used to dead-end at a protocol-mismatch error telling the user to open a terminal.
+  `src/onboarding.ts` `helperNeedsUpdate()` compares the bundle's `prebuilt/macos/<arch>/bridge` against
+  the installed executable by hash (memoised per process) and reinstalls when they differ. This is why
+  the bundle must keep shipping the prebuilt binary — `npm run check:helper-path` asserts it does.
+
+Permissions deliberately still require the user: granting Accessibility/Screen Recording is a system
+security setting. `guideMissingPermissions()` pre-registers the app in the Privacy panes and opens the
+right pane (once per kind per process, or a retrying agent spams System Settings).
+
+### Known residual: same-app window pairing
+`backgroundPressAtPoint` scopes its hit-test to the target app and then requires the hit to sit inside
+the AX subtree of `windowElement(pid:windowId:)`. That last step relies on upstream's greedy
+title/geometry pairing between CG window ids and AX windows, which has no tie-break for two windows of
+one app with the same title **and** near-identical overlapping frames. In that (rare) case the press
+can land in the front window instead of the one the screenshot came from. The `ref` path has exactly
+the same exposure — fixing it means real AX↔CG window identity, which needs private API
+(`_AXUIElementGetWindow`). Deliberately not done; noted so nobody assumes the subtree check is airtight.
+
+### Bundle layout
+```
+manifest.json  main.js  check.mjs  skills/computer-use/     ← Forsion 桌面侧(视图 + 配套技能)
+tangu-plugins/computer-use/{tangu-plugin.json,dist/}        ← 引擎侧插件(12 工具:上游 11 + 自研 ensure_app)
+src/  native/  scripts/  prebuilt/                          ← 源码与原生 helper(位置不变)
+```
+`bundles.ts` finds the engine plugin at `<bundle>/tangu-plugins/<pid>/tangu-plugin.json` and skills at
+`<bundle>/skills/<slug>/SKILL.md`.
+
+**Bundled into Forsion Desktop (0.5.0+).** The desktop depends on this repo as a vendored tarball
+(`Forsion-Genesis/desktop/vendor/tangu-computer-use.tgz`, refreshed by `npm run vendor:cu` there = `npm pack` of this repo
+after `npm run build` + `build:native --arch all`). electron-builder copies it to `resources/bundled-plugins/tangu-computer-use`
+and `desktop/electron/builtinPlugins.ts` seeds it into `<home>/plugins/tangu-computer-use/` at startup (replace only when the
+shipped version is newer; never downgrade a user-installed copy). The plugin therefore shows up as 「内置」 with no uninstall
+button; `install.sh dev|prod` stays as the developer loop for iterating on the bundle alone. There is **no root `tangu-plugin.json` any more** — a stale
+`tangu install --link` symlink from v0.1 now points at a directory without one.
+
+## Re-syncing upstream
+1. `git clone https://github.com/injaneity/pi-computer-use` and `git diff <pinned>..HEAD` to see the scope.
+2. Copy its `src/` over `src/vendor/`, then re-run the brand rename over `src/vendor/**` (`.ts` + `.mjs`
+   + `.mts`; the two `perl -pe` subs plus the exact phrases in "Brand rename"). `grep -rn
+   "pi-computer-use\|injaneity" src/vendor` must come back empty. Import lines need no change — the alias
+   handles pi.
+3. Copy `scripts/{setup-helper,build-native,make-signing-cert}` over, rename, then re-apply our two
+   setup-helper patches (vendor path for `helper-path.mjs`, short signing CN) and the `build-native.mjs`
+   hooks listed under "Native" (macOS source list, Windows `+crt-static`).
+4. Copy `native/{macos,windows,linux}` over, rename — then **replay the Tangu native additions** listed
+   under "Native" (the highlight file and the three `bridge.swift` hooks). `grep -n
+   "noteControlledWindow\|liveView\|AgentHighlight" native/macos/bridge.swift` must find them.
+5. If `extensions/computer-use.ts` changed (new tools or schema), mirror it in `src/tools.ts` (`SPECS`),
+   and mirror new config keys in `src/settings.ts` + `syncSettings`.
+6. `npm run build && npm run check`, then **`node scripts/build-native.mjs --arch all`** — not just a compile
+   probe: the bundle ships `prebuilt/macos/{arm64,x64}/bridge`, and the desktop vendors this repo as a tarball
+   (`npm run vendor:cu` in `Forsion-Genesis/desktop`), so a stale prebuilt is what users get. ⚠️ Upstream's own
+   git-tracked `prebuilt/` is *not* rebuilt per release (v0.5.0↔v0.5.1 blobs were byte-identical while
+   `bridge.swift` changed) — never copy their binaries, always rebuild from `native/`. Bump the commit at the
+   top of this file and the protocol number whenever `native/` changed behaviour.
+7. Real-machine greens (need an installed, authorized helper — not runnable in CI, hence not in
+   `npm run check`): `npm run check:live` = `check:blindclick` (coordinate click goes AX, keeps the
+   foreground, **and the Calculator readout actually changes**) + `check:liveview` (frames come from
+   the stream, sub-100ms, actually differ, and every frame matches the window's aspect ratio).
+   For the view's layout, serve the repo and open `harness/harness.html`, then press 「量一量」 —
+   it walks every window shape and asserts canvas-aspect / fill / snug / no-overflow. The DOM shim in
+   `check.mjs` does no layout, so CSS regressions are only visible there.
+
+- Mini Panel foreground activity: `native/macos/foreground_activity.swift` emits a bounded, data-only lease beside the daemon socket. Hooked into actual HID/focus delivery and recursive input scopes, not the requested delivery policy. Included in both native-build and source-install inputs. No change to upstream tool schemas or input behavior.

package/check.mjs

@@ -0,0 +1,369 @@
+/**
+ * 桌面侧插件自检:用宿主同款 `new Function('ctx', src)` 求值 main.js,断言贡献点齐全,
+ * 并在 DOM 垫片上跑一轮真实挂载 —— 重点验三件**错了不会报错、只会静默烧 CPU 或骗人**的事:
+ *   ① 卸载必须停掉轮询(视图关了还在每秒截屏 = 白烧,而且用户看不见)
+ *   ② 看不见时(document.hidden / 滚出可视区)不发请求
+ *   ③ 没在操控 / 拿不到画面时,绝不留着上一帧假装实时
+ * 跑法:node check.mjs
+ */
+import { readFileSync } from 'node:fs'
+import { strict as A } from 'node:assert'
+
+// ── 极简 DOM 垫片(只实现 main.js 真正用到的那点) ──
+/**
+ * 画布(.cu-live-viewport)的尺寸。实现自己用 layoutViewport 把宽高写进 style,所以垫片
+ * 照读即可 —— 不必复刻任何布局规则,量到的就是实现算出来的那个数。
+ * (⚠️曾经想让 CSS `aspect-ratio` 干这件事,真浏览器里不成立,见 main.js 里那段注释。)
+ */
+const viewportBoxOfNode = (n) => ({
+  w: parseFloat(n.style.width) || 0,
+  h: parseFloat(n.style.height) || 0,
+})
+const mkNode = (tag) => {
+  const style = { setProperty(k, v) { style[k] = String(v) }, removeProperty(k) { delete style[k] } }
+  const n = {
+    tag, children: [], attrs: {}, parentNode: null, style, dataset: {},
+    className: '', id: '', src: '', alt: '', title: '',
+  }
+  let own = ''
+  Object.defineProperty(n, 'textContent', {
+    get: () => own + n.children.map((c) => c.textContent || '').join(''),
+    set: (x) => { n.children.forEach((c) => { c.parentNode = null }); n.children.length = 0; own = String(x) },
+  })
+  n.appendChild = (c) => { n.children.push(c); c.parentNode = n; return c }
+  n.append = (...cs) => cs.forEach((c) => n.appendChild(c))
+  n.replaceChildren = (...cs) => { n.children.forEach((c) => { c.parentNode = null }); n.children.length = 0; own = ''; cs.forEach((c) => n.appendChild(c)) }
+  n.setAttribute = (k, v) => { n.attrs[k] = v }
+  n.querySelector = () => null
+  n.listeners = {}
+  n.addEventListener = (t, fn) => { (n.listeners[t] ||= []).push(fn) }
+  n.removeEventListener = (t, fn) => { n.listeners[t] = (n.listeners[t] || []).filter((f) => f !== fn) }
+  n.fire = (t, ev = {}) => (n.listeners[t] || []).slice().forEach((f) => f({ preventDefault() {}, ...ev }))
+  // 真几何:垫片里 clientWidth/Height 恒 0 的话,缩放那一整套代码在自检里等于裸奔(codex 指出)。
+  // 画布是个例外 —— 它的尺寸不是谁设进来的,而是由自己的宽高比 + 舞台算出来的(同 CSS)。
+  let ownW = 0
+  let ownH = 0
+  const isViewport = () => n.className.includes('cu-live-viewport')
+  Object.defineProperty(n, 'clientWidth', {
+    get: () => (isViewport() ? viewportBoxOfNode(n).w : ownW),
+    set: (v) => { ownW = v },
+  })
+  Object.defineProperty(n, 'clientHeight', {
+    get: () => (isViewport() ? viewportBoxOfNode(n).h : ownH),
+    set: (v) => { ownH = v },
+  })
+  n.getBoundingClientRect = () => ({ left: 0, top: 0, width: n.clientWidth, height: n.clientHeight })
+  n.captured = []
+  n.setPointerCapture = (id) => { n.captured.push(id) }
+  n.releasePointerCapture = (id) => { n.captured = n.captured.filter((x) => x !== id) }
+  return n
+}
+const head = mkNode('head')
+globalThis.document = {
+  hidden: false,
+  head,
+  createElement: mkNode,
+  createTextNode: (t) => { const n = mkNode('#text'); n.textContent = t; return n },
+  getElementById: (id) => (head.children.find((c) => c.id === id) || null),
+}
+let observed = null
+globalThis.IntersectionObserver = class {
+  constructor(cb) { this.cb = cb; observed = this }
+  observe(el) { this.el = el }
+  disconnect() { this.disconnected = true }
+  fire(isIntersecting) { this.cb([{ isIntersecting }]) }
+}
+// 必须有:没有它,`resizeObserver.disconnect()` 漏掉的回归自检根本发现不了(codex 指出)。
+let resized = null
+globalThis.ResizeObserver = class {
+  constructor(cb) { this.cb = cb; resized = this }
+  observe(el) { this.el = el }
+  disconnect() { this.disconnected = true }
+  fire() { this.cb([{}]) }
+}
+
+// ── 求值 main.js(宿主同款) ──
+const src = readFileSync(new URL('./main.js', import.meta.url), 'utf8')
+const views = []
+const ctx = { registerView: (v) => views.push(v) }
+new Function('ctx', src)(ctx)
+
+A.equal(views.length, 1, '应只注册一个视图')
+A.equal(views[0].id, 'live', '视图 id 必须是 live —— 技能里写死了 plugin:tangu-computer-use:live')
+A.equal(typeof views[0].mount, 'function', 'mount 必须是函数')
+
+const T = globalThis.__CU_LIVE_TEST__
+A.ok(T, 'main.js 必须暴露 __CU_LIVE_TEST__ 供自检')
+// 技能 SKILL.md 里写死的视图 id 必须与插件 id/视图 id 拼得出来,改一处不改另一处就是死链
+const skill = readFileSync(new URL('./skills/computer-use/SKILL.md', import.meta.url), 'utf8')
+A.ok(skill.includes(`plugin:${T.PLUGIN_ID}:${T.VIEW_ID}`), 'SKILL.md 里的 desk_present 视图 id 与插件不一致')
+const manifest = JSON.parse(readFileSync(new URL('./manifest.json', import.meta.url), 'utf8'))
+A.equal(manifest.id, T.PLUGIN_ID, 'manifest.id 必须与 main.js 里的 PLUGIN_ID 一致(视图 id 由它拼)')
+A.equal(manifest.main, 'main.js')
+
+// ── 纯函数 ──
+A.equal(T.ago(0), '刚刚')
+A.equal(T.ago(3000), '3 秒前')
+A.equal(T.ago(120000), '2 分钟前')
+A.equal(T.describe({ app: 'TextEdit', title: '未命名' }), 'TextEdit — 未命名')
+A.equal(T.describe({ app: 'TextEdit' }), 'TextEdit')
+A.equal(T.describe({ windowId: 7 }), '窗口 #7')
+A.equal(T.explain(''), '', '无错误码 → 空串')
+A.ok(T.explain('helper_not_running').includes('还没有窗口'))
+A.ok(T.explain('screen_recording_denied').includes('屏幕录制'))
+A.ok(T.explain('some_new_code').includes('some_new_code'), '未知错误码必须原样带出,不许吞掉线索')
+A.ok(T.explain('client_timeout').includes('超时'), '客户端超时与 helper 截图超时是两码事,都要有说法')
+
+// ── 缩放几何(纯函数)──
+// 默认比例 = 整幅塞进舞台
+A.equal(T.fitScaleOf(1000, 500, 400, 400), 0.4, '宽受限时按宽算')
+A.equal(T.fitScaleOf(500, 1000, 400, 400), 0.4, '高受限时按高算')
+A.equal(T.fitScaleOf(0, 0, 400, 400), 1, '尺寸未知时不乱缩')
+A.equal(T.fitScaleOf(1000, 500, 0, 0), 1, '舞台还没布局时不乱缩')
+
+// 画布按窗口比例变形:宽窗口 → 画布变矮,高窗口 → 画布变窄。窗口比舞台小也**放大填满**
+// (这是取景框不是原尺寸预览),所以两个方向上都必然有一边贴满舞台 —— 那正是"没有留白"。
+{
+  const wide = T.viewportBoxOf(1600, 500, 400, 300)
+  A.ok(Math.abs(wide.w - 400) < 1e-6 && Math.abs(wide.h - 125) < 1e-6, `宽窗口画布应变矮,得到 ${JSON.stringify(wide)}`)
+  const tall = T.viewportBoxOf(500, 1600, 400, 300)
+  A.ok(Math.abs(tall.h - 300) < 1e-6 && Math.abs(tall.w - 93.75) < 1e-6, `高窗口画布应变窄,得到 ${JSON.stringify(tall)}`)
+  const box = T.viewportBoxOf(800, 600, 400, 300)
+  A.ok(Math.abs(box.w / box.h - 800 / 600) < 1e-9, '画布宽高比必须等于窗口宽高比')
+  A.ok(Math.abs(box.w - 400) < 1e-6 || Math.abs(box.h - 300) < 1e-6, '画布必须有一边贴满舞台,否则就是白留了空间')
+}
+
+// ⚠️实时画面的节奏线:helper 已是常驻取景流,这里再回到 1fps 就又变回用户实报的"掉帧幻灯片"。
+A.ok(T.TICK_MS <= 200, `实时画面轮询必须 ≤200ms(现在 ${T.TICK_MS}ms)—— 慢过这个就不是 screen sharing 了`)
+
+// 平移夹紧:图比舞台大 → 不许把边拖进舞台里面(拖出空白就再也找不回图了)
+A.equal(T.clampOffset(500, 2000, 400), 0, '右边界:最多贴住左沿')
+A.equal(T.clampOffset(-5000, 2000, 400), -1600, '左边界:最多贴住右沿')
+A.equal(T.clampOffset(-100, 2000, 400), -100, '范围内原样保留')
+// 图比舞台小 → 一律居中
+A.equal(T.clampOffset(999, 200, 400), 100, '图比舞台小时强制居中')
+
+// 以光标为锚缩放:锚点下的那个像素必须不动
+{
+  const anchor = 300
+  const off0 = -100
+  const s0 = 1
+  const s1 = 2
+  const off1 = T.anchoredOffset(off0, anchor, s0, s1)
+  // 锚点在图片自身坐标系里的位置,缩放前后应一致
+  A.ok(Math.abs((anchor - off0) / s0 - (anchor - off1) / s1) < 1e-9, '缩放后锚点像素发生了漂移')
+}
+
+// 倍率夹紧:下限是"整幅可见",上限 MAX_SCALE
+A.equal(T.clampScale(0.01, 0.4), 0.4, '不许缩到比默认比例还小(只会徒增空白)')
+A.equal(T.clampScale(999, 0.4), T.MAX_SCALE, '上限封顶')
+A.equal(T.clampScale(2, 0.4), 2, '范围内原样保留')
+
+// ── 挂载行为 ──
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms))
+const calls = []
+let reply = { active: false }
+// 假 API 必须**照实现主进程的语义**:image:false 就是不给图。
+// (第一版偷懒无视这个参数,于是"没要图时保留画面"那条断言压根没被执行到 —— 没牙的断言比没有更坏。)
+globalThis.window = {
+  tangu: {
+    computerUseLiveView: async (o) => {
+      calls.push(o)
+      if (o && o.image === false) { const { jpegBase64, width, height, ...rest } = reply; return rest }
+      return reply
+    },
+  },
+}
+
+const host = mkNode('div')
+const dispose = views[0].mount(host)
+const rootOf = () => host.children[0]
+const stageOf = () => rootOf().children[1]
+const imgsIn = (n) => { const o = []; ;(function w(x) { if (x.tag === 'img') o.push(x); (x.children || []).forEach(w) })(n); return o }
+const msgsIn = (n) => { const o = []; ;(function w(x) { if (x.className === 'cu-live-msg') o.push(x); (x.children || []).forEach(w) })(n); return o }
+// 给舞台真尺寸,否则缩放几何在自检里全走"量不到→按 1"的退化分支
+stageOf().clientWidth = 400
+stageOf().clientHeight = 300
+
+// ⚠️可见性**未知**必须按可见处理 —— 这是「永久静默」那个 bug 的回归线:
+// 隐藏页面里浏览器根本不做交集计算,IntersectionObserver 连首次回调都不来,
+// 初值若当成「不可见」就永远翻不了身(实测 ioHits=0,一帧都不请求)。
+await sleep(30)
+A.ok(calls.length >= 1, 'IO 还没表态时必须按可见处理并正常轮询(否则永久静默)')
+A.ok(rootOf().textContent.includes('暂无正在被操控的窗口'), '没在操控时应给出说明')
+
+// 有画面 → 显示 img,且头部换成窗口名。**必须带真尺寸**:没有尺寸就走不到缩放几何。
+reply = { active: true, windowId: 3, app: 'Safari', title: '标签页', ageMs: 500, jpegBase64: 'AAAA', width: 800, height: 600 }
+await sleep(T.IDLE_TICK_MS + 60)
+A.equal(imgsIn(rootOf()).length, 1, '有画面时舞台上应只有一个 img')
+A.equal(imgsIn(rootOf())[0].src, 'data:image/jpeg;base64,AAAA')
+A.equal(msgsIn(stageOf()).length, 0, '有画面时舞台上不许还留着说明文案(图与文案互斥)')
+A.ok(rootOf().textContent.includes('Safari'), '头部应显示 App 名')
+// 舞台 400x300、画面 800x600(同为 4:3)→ 画布铺满舞台,默认比例 = 400/800 = 50%
+const pctNow = () => { let t = null; ;(function w(n) { if (n.className === 'cu-live-pct') t = n.textContent; (n.children || []).forEach(w) })(rootOf()); return t }
+A.equal(pctNow(), '50%', '默认比例应按真实几何算出(800x600 塞进 400x300 = 50%)')
+// ⚠️"没有适配窗口 size"的回归线:画布的宽高比必须每帧跟着被操控窗口写进去。
+const viewportOf = () => { let v = null; ;(function w(n) { if (n.className.includes('cu-live-viewport')) v = n; (n.children || []).forEach(w) })(rootOf()); return v }
+A.equal(`${viewportOf().style.width}x${viewportOf().style.height}`, '400px x300px'.replace(' ', ''), '画布必须按这一帧的窗口比例变形(800x600 → 铺满 400x300)')
+
+// ⚠️静止窗口:helper 回 unchanged(不含图)时必须**保留**现有画面,而不是当成"拿不到画面"。
+// 同时必须真的把上次的 frameSeq 带回去,否则 helper 永远认不出"没变",每秒重编 8 次 JPEG。
+{
+  reply = { active: true, windowId: 3, app: 'Safari', ageMs: 300, jpegBase64: 'SEQ1', width: 800, height: 600, frameSeq: '42' }
+  await sleep(T.TICK_MS + 120)
+  A.equal(imgsIn(rootOf()).length, 1, '先要有一帧画面')
+  await sleep(T.TICK_MS + 120)
+  A.equal(calls[calls.length - 1].sinceFrame, '42', '必须把上次的 frameSeq 带回去,否则省不掉重编')
+  reply = { active: true, windowId: 3, app: 'Safari', ageMs: 400, unchanged: true }
+  await sleep(T.TICK_MS + 120)
+  A.equal(imgsIn(rootOf()).length, 1, 'unchanged 时必须保留现有画面,不许换成"拿不到画面"')
+}
+
+// 转为「拿不到画面」→ 必须把旧图撤下,不能留着上一帧假装实时
+reply = { active: true, windowId: 3, app: 'Safari', error: 'screen_recording_denied' }
+await sleep(T.TICK_MS + 80)
+A.equal(imgsIn(rootOf()).length, 0, '拿不到画面时必须撤下旧图(不许留着上一帧冒充实时)')
+A.ok(rootOf().textContent.includes('屏幕录制'), '应说明是权限问题')
+
+// 滚出可视区 → **降频**(不是停),且主动不要图(省掉真正贵的那一步:一次真实截屏)
+reply = { active: true, windowId: 3, app: 'Safari', ageMs: 10, jpegBase64: 'BBBB', width: 800, height: 600 }
+await sleep(T.TICK_MS + 80)               // 先在可见态拿到一帧真画面
+observed.fire(false)
+const before = calls.length
+await sleep(T.HIDDEN_TICK_MS + 200)
+A.ok(calls.length > before, '看不见也必须继续问状态 —— 只降频,绝不完全停(永久静默的教训)')
+A.equal(calls[calls.length - 1].image, false, '看不见时不该要图')
+// 而且不许因为"这一帧没图"把已经在显示的画面撤成错误文案
+const imgs3 = []
+;(function walk(n) { if (n.tag === 'img') imgs3.push(n); (n.children || []).forEach(walk) })(rootOf())
+A.equal(imgs3.length, 1, '我们主动没要图时,应保留正在显示的那一帧,而不是换成"拿不到画面"')
+observed.fire(true)
+await sleep(120)
+A.ok(calls[calls.length - 1].image !== false, '重新可见后应立刻恢复要图')
+
+// ── 缩放/取景的行为(几何在垫片里是真的了,这些才有意义)──
+const scaleNow = () => { let t = null; ;(function w(n) { if (n.tag === 'img') t = n.style.transform; (n.children || []).forEach(w) })(rootOf()); return t }
+reply = { active: true, windowId: 3, app: 'Safari', ageMs: 10, jpegBase64: 'CCCC', width: 800, height: 600 }
+observed.fire(true)
+await sleep(T.TICK_MS + 120)
+const fitTf = scaleNow()
+// 放大一档(点 + 按钮)
+let btnIn = null
+;(function w(n) { if (n.tag === 'button' && n.textContent === '+') btnIn = n; (n.children || []).forEach(w) })(rootOf())
+btnIn.fire('click')
+const zoomedTf = scaleNow()
+A.notEqual(zoomedTf, fitTf, '点 + 应该真的改变缩放')
+
+// ⚠️用户放大着,新帧到来不许把取景弹回默认
+await sleep(T.TICK_MS + 120)
+A.equal(scaleNow(), zoomedTf, '常规新帧不得重置用户的取景')
+
+// ⚠️用户一放大,我们就主动要更高清的原图(maxDimension 800→1600)→ 同一个窗口回来的像素翻倍。
+// scale 是「相对原始像素」的倍率,像素翻倍而 scale 不动,画面就凭空放大一倍、锚点跟着跳。
+// 判据只能是「用户看到的宽度」不变,不能是 scale 不变。
+const visualWidthOf = (nat) => { const m = /scale\(([\d.]+)\)/.exec(scaleNow()); return nat * Number(m[1]) }
+{
+  const before = visualWidthOf(800)
+  reply = { ...reply, jpegBase64: 'CCCC2', width: 1600, height: 1200 }
+  await sleep(T.TICK_MS + 120)
+  const after = visualWidthOf(1600)
+  A.ok(Math.abs(after - before) < 0.01, `换高清源不得改变看到的大小:${before} → ${after}`)
+}
+
+// ⚠️用户放大着,目标窗口改了尺寸 → 守住倍率,但平移必须夹回新边界(不能让画面滑出舞台)
+reply = { ...reply, jpegBase64: 'DDDD', width: 300, height: 200 }
+await sleep(T.TICK_MS + 120)
+{
+  const vp = viewportOf()
+  const ar = parseFloat(vp.style.width) / parseFloat(vp.style.height)
+  A.ok(Math.abs(ar - 300 / 200) < 1e-6, `窗口改了尺寸,画布比例必须跟着改,实得 ${vp.style.width}x${vp.style.height}`)
+}
+{
+  const m = /translate\((-?[\d.]+)px, (-?[\d.]+)px\) scale\(([\d.]+)\)/.exec(scaleNow())
+  A.ok(m, '变形应可解析')
+  const [, ox, oy, sc] = m.map(Number)
+  // 边界是**画布**的,不是舞台的 —— 画布已经按 300x200 变形过了
+  const { w: sw, h: sh } = T.viewportBoxOf(300, 200, 400, 300)
+  const w = 300 * sc, h = 200 * sc
+  const okX = w <= sw ? Math.abs(ox - (sw - w) / 2) < 0.5 : ox <= 0.001 && ox >= sw - w - 0.001
+  const okY = h <= sh ? Math.abs(oy - (sh - h) / 2) < 0.5 : oy <= 0.001 && oy >= sh - h - 0.001
+  A.ok(okX && okY, `窗口改尺寸后平移必须夹回边界,现在是 ${scaleNow()}`)
+}
+
+// ⚠️换了窗口(哪怕尺寸恰好相同)必须复位,不能套用上一个窗口的取景
+reply = { active: true, windowId: 99, app: 'Notes', ageMs: 10, jpegBase64: 'EEEE', width: 300, height: 200 }
+await sleep(T.TICK_MS + 120)
+A.equal(scaleNow(), defaultTfFor(300, 200), '换窗口必须回到默认比例(哪怕尺寸与上一个窗口相同)')
+/** 用与实现同一条规则算期望值,别手推 —— 画布先按比例变形,再在画布里贴合。 */
+function defaultTfFor(w, h) {
+  const { w: vw, h: vh } = T.viewportBoxOf(w, h, 400, 300)
+  const s = T.fitScaleOf(w, h, vw, vh)
+  const off = (box, scaled) => (scaled <= box ? (box - scaled) / 2 : Math.min(0, Math.max(box - scaled, (box - scaled) / 2)))
+  return `translate(${off(vw, w * s)}px, ${off(vh, h * s)}px) scale(${s})`
+}
+
+// 卸载 → 停轮询 + 断开两个 observer + 清空宿主
+dispose()
+A.ok(resized.disconnected, '卸载后必须断开 ResizeObserver')
+const afterDispose = calls.length
+await sleep(T.TICK_MS + T.IDLE_TICK_MS + 120)
+A.equal(calls.length, afterDispose, '卸载后必须停止轮询')
+A.ok(observed.disconnected, '卸载后必须断开 IntersectionObserver')
+A.equal(host.children.length, 0, '卸载后必须清空宿主元素')
+
+// ── 跨实例单飞:每帧都是一次真实截屏(helper 最长 8s),两个实例同时在场不能各截各的 ──
+{
+  let release
+  const hang = new Promise((r) => { release = r })
+  let hits = 0
+  globalThis.window = { tangu: { computerUseLiveView: async () => { hits++; await hang; return { active: false } } } }
+  const a = mkNode('div')
+  const b = mkNode('div')
+  const da = views[0].mount(a)
+  const oa = observed
+  const db = views[0].mount(b)
+  const ob = observed
+  oa.fire(true)
+  ob.fire(true)
+  await sleep(30)
+  A.equal(hits, 1, '两个实例同时取景只应发一次请求(共享单飞)')
+  release({ active: false })
+  await sleep(10)
+  da(); db()
+}
+
+// ⚠️落地时只许清**自己**那一份单飞槽:另一个桶的请求可能已经把槽换成它的了,
+// 无条件清会让那个还在飞的请求失去保护 —— 下一个实例立刻又发一次真实取景(codex 指出)。
+{
+  const gates = []
+  const seen = []
+  globalThis.window = { tangu: { computerUseLiveView: (o) => { seen.push(o.image); return new Promise((r) => gates.push(() => r({ active: false }))) } } }
+  const a = T.fetchFrame(globalThis.window.tangu.computerUseLiveView, { image: false })
+  const b = T.fetchFrame(globalThis.window.tangu.computerUseLiveView, { maxDimension: 800 })
+  A.equal(seen.length, 2, '两个桶各应发一次')
+  gates[0]()                       // 先落地的是**另一个桶**的 a
+  await a
+  const c = T.fetchFrame(globalThis.window.tangu.computerUseLiveView, { maxDimension: 800 })
+  A.equal(seen.length, 2, 'a 落地不得清掉 b 的单飞槽,否则 c 会再发一次真实取景')
+  A.equal(c, b, 'c 应复用还在飞的 b')
+  gates[1]()
+  await Promise.all([b, c])
+}
+
+// ⚠️单飞不许跨"要不要图"复用:慢速实例先发的 {image:false} 被要图的实例捡走,
+// 那边看到没有 jpegBase64 就会误判成"拿不到画面" —— 我们没要 ≠ 拿不到。
+{
+  const seen = []
+  let release
+  const hang = new Promise((r) => { release = r })
+  globalThis.window = { tangu: { computerUseLiveView: async (o) => { seen.push(o.image); await hang; return { active: false } } } }
+  const p1 = T.fetchFrame(globalThis.window.tangu.computerUseLiveView, { image: false })
+  const p2 = T.fetchFrame(globalThis.window.tangu.computerUseLiveView, { maxDimension: 1280 })
+  A.notEqual(p1, p2, '要图与不要图的请求不得共用同一个 Promise')
+  A.deepEqual(seen, [false, undefined], '两种请求都应真的发出去')
+  release()
+  await Promise.all([p1, p2])
+}
+
+console.log('✅ computer-use 插件自检通过')
+process.exit(0)