@forsion/tangu-computer-use 0.5.8

package/CHANGELOG.md

@@ -0,0 +1,223 @@
+# 更新日志
+
+## 0.5.8 — 2026-09-26
+
+- 修复 Windows 助手在没装过 VC++ 运行库的干净 Windows 上起不来：助手改为静态链接 MSVC 运行库，不再依赖 `VCRUNTIME140.dll`。发布前的包内容检查会拦下仍依赖 VC++ 运行库的 Windows 助手。
+
+The Windows helper now links the MSVC runtime statically, so it starts on clean Windows machines without the Visual C++ Redistributable (it no longer needs `VCRUNTIME140.dll`). The release check rejects a Windows helper that still depends on the Visual C++ runtime.
+
+## 0.5.7 — 2026-09-25
+
+- 修复 Windows / Linux 上 `tangu computer-use setup` 与 `doctor` 打印结果后进程不退出：这两个一次性命令结束前会关掉助手进程。macOS 不受影响。
+
+On Windows and Linux, `tangu computer-use setup` and `doctor` now exit after printing their result. Both one-off commands close the helper process before they return. macOS was not affected.
+
+## 0.5.6 — 2026-09-25
+
+- macOS 助手改用固定证书签名。从这一版起，助手更新会保留「辅助功能」与「屏幕录制」授权，不用再重新授予。从旧版升级上来时需要最后授权一次。安装过程仍然不访问钥匙串，也不会要你输入密码。
+- 第一个发布到 npm 的版本，随包带齐 macOS（arm64 / x64）、Windows 与 Linux 助手。已安装 Forsion 的机器会在后台下载新版，下次启动时换上。
+
+The macOS helper is now signed with a fixed certificate, so later helper updates keep the Accessibility and Screen Recording grants. Upgrading from an earlier version asks for them one last time. Installation still never touches the keychain or asks for a password. This is also the first version published to npm, and it ships the macOS (arm64 and x64), Windows and Linux helpers. Installed copies of Forsion download new versions in the background and switch to them on the next launch.
+
+## 0.5.5 — 2026-09-21
+
+- 首次引导改由 Forsion 桌面端实测「辅助功能」与「屏幕录制」两项授权：都已授权就不再弹引导卡，缺哪项就在卡里直接安装 helper 并请求授权；引导说明补齐英文。
+
+The setup card is now driven by the Accessibility and Screen Recording grants that Forsion Desktop checks itself. It only appears while a grant is missing, and it can install the helper and request access in place. The setup guide now has English text.
+
+## 0.5.4 — 2026-09-10
+
+- 修正 Windows / Linux 桌面操作指引：从窗口发现开始，不再要求调用仅 macOS 支持的应用启动工具。
+- 明确已可用的工具可以直接调用，避免误把重复加载提示当成整个电脑控制功能故障。
+- 新增三平台离线契约检查；这是工具与指引校验，不代替 Windows 真机操作验收。
+
+Windows and Linux guidance now starts with window discovery instead of the macOS-only app launcher. Available tools can be called directly without loading them again. Offline checks cover tool visibility and instructions on all three platforms; native Windows GUI acceptance testing remains separate.
+
+## 0.5.3 — 2026-09-09
+
+- 修复首次安装 Computer Use 时弹出 `codesign` 钥匙串密钥访问框：macOS 助手在构建阶段组装、签名，以完整 App ZIP 随包交付；安装时只校验和复制，不再查找用户 Developer ID、导入本地证书或访问私钥。
+- 修复拒绝旧签名弹窗后留下的半安装 App；校验完整签名与随包内容，采用临时目录验证、互斥安装和失败回滚，并识别同协议号的旧助手。
+- 新增 `check:installer` 与 `scripts/verify-macos-bundles.mjs`，验证首次安装、升级、半安装修复、损坏包、并发和回滚；桌面打包之后再次校验 ZIP，防止递归签名改写内置助手。
+- 当前构建使用 ad-hoc 签名，不等于 Developer ID 或 Apple 公证。从旧本地证书迁移及助手升级后，macOS 可能要求重新授予辅助功能与屏幕录制；安装无需钥匙串密码，已有钥匙串条目不会自动删除。
+
+macOS helpers now ship as complete, sealed app archives. Installation never discovers signing identities, imports certificates or accesses private keys. Interrupted older installations are repaired, concurrent installs are serialized, and failed replacements restore the previous app. The current ad-hoc build is not Developer ID-signed or notarized; macOS may request Accessibility and Screen Recording access again after migration or updates.
+
+## 0.5.2 — 2026-09-08
+
+- 修复旧 0.5.1 插件与新版前台信号 helper 同版本导致桌面播种跳过更新：发布独立版本，随包携带已构建的 macOS arm64 / x64 helper。
+- 自动升级 helper 后显式重启常驻进程，即使协议号和安装路径相同，也不会继续运行旧二进制；安装或重启失败均交给现有引导反馈。
+- 新增真实二进制启动信号检查 `check:helper-signal` 与安装/重启顺序回归 `check:helper-refresh`。
+
+Computer Use now ships as a distinct version so Desktop replaces older 0.5.1 bundles. Successful helper upgrades restart the daemon even when its protocol and path are unchanged. Tests cover actual packaged signal output and installer/restart failures.
+
+## 0.5.1 — 2026-09-08
+
+- 为 Genesis Mini Panel 提供 macOS 前台输入活动信号：实际 HID 输入/前台激活开始发布短期租约，递归输入作用域结束后停止；AX 和 PID 后台路径不触发。
+- 信号在 helper socket 同目录的 `foreground.json`，带 PID 与过期时间；进行中每秒续期，避免每帧写盘。构建/安装脚本已纳入 reporter，新行为需要重新构建 helper。
+- 新增隔离 Swift 回归仪器 `npm run check:mini-foreground`，覆盖后台静默、嵌套输入、结束及短点击宽限。
+
+为 Forsion Desktop 权限引导提供独立的原生接口：
+
+- `permissionStatus` 只读取辅助功能信任状态、屏幕录制预检和授权主体，不抓图、不请求权限、不缓存。
+  同时返回 `settingsFrontmost` 和可选 `settingsWindow: {x,y,width,height}`，坐标为 CG 全局坐标
+  （左上原点、y 向下）；窗口按系统设置的 PID、layer 0、可见有效矩形筛选，取最大窗口，不依赖 AX 或窗口标题。
+  调用端必须只连接已有 socket：`macosHelper.command()` 会自动启动 helper，不适合页面首屏状态读取。
+- `registerPermissions` 可选 `kind: 'accessibility' | 'screenRecording'`，只请求该项；单项屏幕录制
+  返回系统 request 的 `screenRecording` 结果，不做 capturable probe。省略 `kind` 保持原先的双项请求行为。
+- `checkPermissions` 可选 `fresh: true`，绕过并重新建立成功缓存，验证失败会清除旧成功；只用于用户主动
+  授权后的验证。此参数不清除 macOS 自身的 TCC 缓存，预检通过仍不等于实际可采集。
+
+原生 arm64 / x64 产物随包更新；兼容原有命令，协议号保持 12。新增隔离系统 API 的权限检查，
+验证状态读取无授权副作用、单项请求隔离、撤权后 fresh 不残留成功缓存及未授权时的窗口矩形筛选。
+
+## 0.5.0 — 2026-09-07
+
+**随 Forsion Desktop 内置。** 不再需要从市场安装或跑 `install.sh`:桌面启动时把随包的捆绑包播种进
+`<home>/plugins/tangu-computer-use/`(只在随包版本比已装的新时替换;不降级、不碰更新的手装副本),
+设置里显示「内置」且不提供卸载 —— 不想用就关掉开关。native helper 仍在第一次用到工具时自动装。
+开发者装法 `sh install.sh dev` 保留,用于只迭代本包。
+
+**同步上游 v0.5.1**(自 v0.5.0,`4b8dbd7e`,2026-08-31)。工具契约零变化;拿到三处 macOS 修复:
+`find_roots` 的性能与 ScreenCaptureKit 探测死锁(窗口封顶 128、广度发现时 AX 超时 1.0s→0.25s、
+探测改回调)、**小窗口截图不再落进 1920×1080 的默认画布**(此前坐标映射错乱的根因)、同进程的
+AXDialog 不再抢走显式选中的目标窗口(新文件 `root-selection.ts`);`setup-helper` 下载加 120s 超时;
+Windows helper 路径可用 `PI_COMPUTER_USE_WINDOWS_HELPER_PATH` 覆盖。
+⚠️ 上游 git 里跟踪的 prebuilt 二进制在两版之间字节相同而 `bridge.swift` 改了 —— 同步只拷源码、
+native 一律本地重编;本包随附 arm64 + x64 两份新编 helper(`build-native.mjs --arch all`)。
+
+**协议号 11 → 12**:上游改了 helper 的原生行为(截图画布尺寸、根列表时序),按纪律 bump。
+已在跑的老 daemon 会被识别出来重启,并按 0.4.0 的引导自愈就地重装。
+
+顺手修掉文档漂移:包里一直是 12 个工具(上游 11 + 自研 `ensure_app`),六处写着 11。
+
+## 0.4.0 — 2026-07-29
+
+**示意光标和边缘光效终于真的画在屏幕上了。** 0.3.0 那轮修的是几何(算得对不对),这轮才发现算得再对
+也没用 —— 两个覆盖层压根没被用户看见,而且是两个互不相干的原因:
+
+- **边缘光效从未出现过一次**。定位窗口矩形用的 `CGWindowListCreateDescriptionFromArray` 对**别的进程**
+  的窗口恒返回空数组(同一个 windowId:它 count=0,`CGWindowListCopyWindowInfo(.optionIncludingWindow)`
+  count=1 且 bounds 正确)。拿不到矩形就 `return`,窗口连建都没建过。这个功能自加进来就一直是死的。
+- **示意光标随时会被压在下面**。两个覆盖层都没设 `window.level`,待在普通层(0);而宿主是 `.accessory`
+  永不激活,所以只要被操控的 App 被激活一次,覆盖层就沉到它下面。上游想用
+  `window.order(.above, relativeTo: 目标窗口号)` 解决,但那个 API 只在**本进程自己的窗口之间**有意义,
+  传别的进程的窗口号是**空操作**(实测:既不上移也不消失)。现在光效 `.floating`、光标 `.screenSaver`。
+
+留了仪器:`npm run check:overlay` 按 100ms 采样 `CGWindowListCopyWindowInfo` 的窗口栈,断言两个覆盖层
+**都上了屏、且 z 序在被操控窗口之上**。"窗口存在"和"用户看得见"是两件事,只有前者的测试骗了我们两轮。
+
+**首次安装和更新不再需要开终端。** 也是两半:
+
+- vendor 的 `ensureInstalled()` 本来就会自动跑 `scripts/setup-helper.mjs`(还正确处理了
+  `ELECTRON_RUN_AS_NODE`)—— 是我们自己在 tools.ts 里加的「没装就返回指导文本」抢在它前面 return,
+  把上游的自动安装堵死了。
+- 而 `ensureInstalled()` 只判断可执行文件**存不存在**,不判断新旧;插件升级后老二进制留在原地 →
+  协议不匹配 → 报错让用户去终端。现在比对 bundle 随包二进制与已装二进制的哈希,过期就地重装。
+
+权限仍然只能用户自己拨(系统安全设置),但现在会自动把 App 预登记进隐私面板并**打开对应面板**,
+用户只需拨一下开关;每个面板每进程只开一次,免得 agent 重试时刷屏。
+
+四处用户可见文案同步改掉了「去终端敲 `tangu computer-use setup`」:`manifest.json` 的引导卡
+(用户真正看到的那张)、`skills/computer-use/SKILL.md`(不然 agent 还会照旧念给用户听)、
+`main.js` 的旧 helper 提示、`install.sh` 结尾。CLI 本身保留,当修复入口用。
+
+**Codex 评审后修掉的(16 条,全收)**,其中三条足以让上面两件事白做:
+
+- **协议号还停在 10** —— 装新二进制不等于换掉正在跑的 daemon。协议号不变的话,0.3 的 daemon 能通过
+  `ensureProtocol()` 继续服务,你重启 Forsion 也还是看不见叠层。现为 **11**。
+  这轮只改了窗口层级和一个 CGWindowList 调用、没加新命令,正是最容易忘 bump 的形状。
+- **「过期」判定恒为真** —— `installHelperApp()` 是复制后**在原地重签**,Mach-O 字节必然变,
+  拿随包源二进制去比已装的可执行文件永远不等 → 每个新进程都跑一遍完整安装,还可能每次弹钥匙串。
+  改比 `Contents/Resources/source.sha256`(签名前的源哈希,`installHelperApp` 专门写它就是为这个)。
+- **发布只编了一个架构** —— CI 跑在 arm64 runner 上,`npm run build:native` 不带参数只编 `process.arch`,
+  发出去的包里没有 x64 那份。Intel Mac 的「随包自动安装」会退化成本地 Swift 编译,没装 Xcode 命令行
+  工具就直接失败。改 `--arch all` 并加发版前的硬门槛。
+
+其余:权限真值改问 `checkPermissions`(`diagnostics` 那项只是 `CGPreflight` 缓存值,bridge.swift 自己的
+注释就写着这点);一次只开一个隐私面板(系统设置是单窗口,连开两个后一个会顶掉前一个,而两个都被记成
+"开过了");安装的共享 Promise 只跟子进程生死绑定,超时/取消只中断**等待**;`cgWindowBounds` 增加
+`kCGWindowIsOnscreen` 检查(最小化的窗口也会返回 bounds,光效会留在它原来的位置);光标层级从
+`.screenSaver`(1000)降到 `.popUpMenu + 1`(102)—— 够盖住 agent 会碰到的一切,但不再盖住屏保和系统警告;
+就绪失败时只有「自动安装真的失败了」才提终端命令(否则 Linux 缺 AT-SPI 这类错误会被这句话盖住)。
+
+仪器也修了三条,其中两条是**假绿灯通道**:`check:overlay` 现在要求先有干净起跑线(光标空闲 8 秒才隐藏,
+上一个仪器留下的窗口足以让断言通过)、断言只看动作**之后**的帧;不再写死 `/Applications`
+(标准用户装在 `~/Applications`);`killCalculator` 超时改抛错而不是静默放行。
+并在脚本头写清它**不证明**什么:只看窗口的创建/层级/z 序,画布画了个空照样能过。
+
+## 0.3.0 — 2026-07-27
+
+**后台点击不再抢前台**。之前 agent 看着截图"盲点"(只给坐标、没有元素引用)一定会把目标 App 拉到前台、
+顺手把你的鼠标拽走 —— 这是上游的设计:物理事件走后台通道会被非活动窗口直接丢掉,所以坐标点击一律直连前台。
+现在坐标点击**先在目标 App 自己的层级里做一次 AX 命中测试**,命中可按压控件就直接按下去:不抢焦点、不动
+真鼠标、照常画示意光标和边缘光效。命不中(网页内容、文本框、画布)才退回原来的前台路径,行为与上游一致。
+右键/中键/双击语义 AX 表达不了,仍走前台。
+
+**实时画面真的是实时的了**。helper 改为给被操控的窗口开一条**常驻 ScreenCaptureKit 取景流**,取一帧只是
+从缓存拿最新那张(实测中位 12ms,之前每帧都要重新协商一次采集会话),视图轮询从 1 秒提到 120ms ≈ 8fps。
+没人看了自动停流。分辨率相应调低(默认最长边 800),放大取景时才要高清原图。
+
+**画面不再有留白**。两处各占一半:
+
+- helper 那头:上游截窗口时不设输出尺寸,ScreenCaptureKit 会**按整块屏幕出图**、把窗口摆在左上角、
+  其余全是白的 —— 230×408 的计算器出来是一张 1920×1080 的大白图。留白是烤进 JPEG 的,前端救不回来。
+  现在取景流和单帧兜底都按窗口尺寸出图。
+- 视图那头:画布(取景框)会**按窗口的宽高比变形**,画面严丝合缝铺满它;卡片里剩下的空间是卡片底色,
+  不再是画中的黑边。宽窗口 → 矮画布,竖窗口 → 窄画布。
+
+**Codex 评审后的第二轮**(11 条,9 条修掉):
+
+- 示意光标的覆盖层原来只开**主屏**那么大 —— 副屏上的点击把光标画到了窗口之外,看不见。现在横跨所有
+  显示器,单屏行为完全不变(几何有单测钉住)。**这多半才是"辅助鼠标没有出现"剩下的那一半。**
+- 后台 AX 点击原来会被**反着报**成「抢了前台」:坐标点击在 TS 层永远被判 needsForeground,
+  于是 policy 恒为 foreground,而 helper 已经在后台完成了它。
+- 用户放大取景时我们会换更高清的原图,像素翻倍而倍率不动 → 画面凭空放大一倍。现在按像素比补偿,
+  保住看到的大小。
+- 拖动窗口边缘时尺寸每帧都在变,一变就重开流 = 整个拖动过程都在重开(而重开期间只能回落单帧截图)。
+  加了 0.6 秒的稳定去抖。
+- 静止的窗口原来每秒被重编 8 次 JPEG。现在带帧号来,没变就只回一句 `unchanged`。
+- 三个竞态:停流前会复核是不是真的还闲着;启动后、采纳前就失败的流不再被当活流装进来(否则永远
+  不吐帧且再也不会重开,加了看门狗兜底);共享单飞落地时只清自己那一份槽。
+
+**协议号 7 → 10**。helper 必须重装(`tangu computer-use setup`;本地 ad-hoc 签名的机器加
+`PI_COMPUTER_USE_ALLOW_ADHOC_UPDATE=1`),否则新功能一件都不会出现。
+
+**留下的仪器**(下次从跑脚本开始,不从重新推演开始):`npm run check:live` 真机验后台点击与实时画面
+(会开一下计算器);`harness/harness.html` 的「量一量」逐一走过每种窗口形状验布局。
+
+## 0.2.0 — 2026-07-26
+
+**改成 Forsion 捆绑包**。一个目录同时带上引擎侧的 11 个工具、配套技能和桌面视图,装一次就位,不用再单独
+`tangu install`。装法改为 `sh install.sh dev|prod`。旧的独立引擎插件(0.1.x,装在 `<home>/tangu/plugins/`)
+与新捆绑包是同一个插件 id,两份同时在会重复装载 —— `install.sh` 会检测并提示你自行删掉旧的。
+
+**看得见 agent 在动哪个窗口**:
+
+- 被操控的窗口会亮起一圈**边缘光效**(原生绘制,点击穿透、不抢焦点、跟着窗口移动;窗口关掉就熄灭)。
+  开关沿用「操作可视化」那一项,和示意光标同一个 —— 它俩本来就是同一件事。
+- 新增视图**「被操控的窗口」**:实时显示那个窗口的画面。配套技能会让 agent 在开始操作前把它摆到
+  Agent Desk 上,于是你能直接看着它做事。看不见的时候(面板收起/切走标签页)自动停止取画面,不白烧。
+  目前仅 macOS —— Windows 的 helper 是个 stdio 子进程,桌面端够不着它。
+- 拿不到画面时如实说明原因(最常见是没给「屏幕录制」权限),不会留着上一帧假装实时。
+- 实时画面的取景请求做了收敛:「最近被操控的窗口还算数多久」封了 2 分钟硬上界 —— 没有这道闸,一个插件
+  就能把早已结束的一次操作变成对那个窗口的长期取景权。多个视图同时在场只发一次请求(每帧都是真实截屏)。
+
+**同步上游 pi-computer-use v0.5.0**(自 v0.4.3,50 个提交):
+
+- **工具输出封顶**:超长结果会被截断并给一个 `@o` 续读句柄(`read_text`),不再有一次 observe 或
+  evaluate_browser 就把上下文撑爆的可能。
+- **helper 不再需要管理员**:已有的可写 `/Applications` 安装留在原地,其余一律装进 `~/Applications`。
+  ⚠️ 如果你的 `/Applications` 不可写,helper 会搬家,macOS 会要求重新授权一次。
+- **启动 helper 改用直接路径**而不是 bundle id —— 系统里若残留同 id 的旧副本,不会再被它抢走。
+  运行中的 daemon 也会校验是不是当前这份二进制,不匹配就重启。
+- 工具契约收紧(**破坏性**):删掉 `doubleClick`(用 `clickCount`)与 `wait` 动作(用 `wait_for` /
+  `expect`);`click` 拆成「按 ref」与「按坐标」两个互斥形态;`observe_ui` 只认 `@r` root,不再猜
+  app/窗口标题;`search_ui` 的 `action` 改名 `capability` 且不再分页;`wait_for` 与 `act_ui.expect`
+  改用同一套条件字段;`act_ui.headless` 参数取消 —— 严格后台改由插件设置控制。
+- `launch_browser` 不再收 `browser`/`port` 参数,改由新设置项**受管浏览器**(Chrome / Helium)决定。
+- 新增 **Linux 支持**(AT-SPI2 + X11);macOS 的窗口发现范围收窄、幽灵光标生命周期修复、浏览器条件与
+  鼠标键校验、wait 结果 JSON 安全等一批上游修复。
+
+## 0.1.0 — 2026-07-16
+
+- 首发。fork `injaneity/pi-computer-use`(MIT)成 Tangu 引擎插件,macOS + Windows。

package/LICENSE

@@ -0,0 +1,26 @@
+MIT License
+
+Copyright (c) 2026 Forsion
+
+Portions of this software are derived from pi-computer-use
+(https://github.com/injaneity/pi-computer-use), Copyright (c) injaneity,
+licensed under the MIT License. The upstream license text is retained in
+LICENSE.upstream.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/LICENSE.upstream

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Zane Chee
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,169 @@
+# Tangu Computer Use
+
+Let a Forsion agent **observe and control desktop apps** on macOS, Windows and Linux — see the screen,
+click, type, scroll, and wait for UI changes — through the accessibility tree + OCR + screenshots.
+A **Forsion 捆绑包**: one directory that carries the engine plugin (12 tools), a companion skill, and a
+desktop view that shows the window being controlled. Forked from
+[pi-computer-use](https://github.com/injaneity/pi-computer-use) (MIT), synced at v0.5.0.
+
+> macOS 14+ (Swift helper), Windows (Rust UIA helper), Linux (Rust AT-SPI2/X11 helper).
+> On macOS the helper needs Accessibility + Screen Recording; the other platforms need no system grant.
+
+## Install
+
+**It ships inside Forsion Desktop (0.5.0+).** Nothing to install: the desktop seeds this bundle into
+`<home>/plugins/tangu-computer-use/` on startup (replacing it only when the shipped version is newer, never
+downgrading a copy you installed yourself), and it appears under **设置 → 插件** as 「内置」 with no uninstall
+button — turn the **电脑操作** switch off if you do not want it. The native helper still installs itself the
+first time a tool runs, and reinstalls itself when an update ships a newer helper (the binary rides along in
+the bundle, so this works offline). No terminal required.
+
+macOS will ask you to grant **Accessibility** and **Screen Recording** to *Tangu Computer Use*
+in System Settings → Privacy & Security — the agent opens the right pane for you, but only you can
+flip the switch. The agent cannot see or touch anything until you do.
+Since v0.5.0 the helper installs to `~/Applications` unless a writable `/Applications` copy already
+exists — **no administrator password needed**.
+
+The CLI is still there for repairs and for scripting:
+
+```bash
+tangu computer-use doctor     # check helper / Accessibility / Screen Recording / macOS version
+tangu computer-use setup      # force a reinstall of the helper
+tangu computer-use stop       # stop the helper
+```
+
+### Developer loop (iterating on this bundle alone)
+
+```bash
+npm install && npm run build     # 引擎入口必须先构建出来
+sh install.sh dev                # → ~/.forsion-dev/plugins/tangu-computer-use   (prod = ~/.forsion)
+```
+
+> 0.2.0 起不再走 `tangu install`:仓根已没有 `tangu-plugin.json`,引擎侧内容在 `tangu-plugins/computer-use/`。
+> 若你装过 0.1.x,`<home>/tangu/plugins/` 或 `~/.tangu/plugins/` 里那份要删掉——同一个插件 id 会重复装载,
+> `install.sh` 会把它们列出来。
+
+### Release / 发布
+
+Bump `version` in both `package.json` and `manifest.json`, add a CHANGELOG entry, then push a `v<version>` tag.
+`.github/workflows/release.yml` builds every native helper, runs the checks, verifies the package contents
+(`node scripts/verify-package.mjs`), attaches the helpers and the `.tgz` to the GitHub Release, and publishes the
+`.tgz` to npm through trusted publishing (no npm token is stored anywhere). Running the workflow by hand is a dry
+run: it builds and verifies, but creates no release and publishes nothing. The macOS helper is signed with the
+release certificate from the secrets `CU_SIGNING_P12` (base64 of the `.p12`) and `CU_SIGNING_P12_PASSWORD`; the
+workflow fails without them rather than shipping an ad-hoc helper.
+
+Forsion Desktop picks a new version up in two ways. Running desktops check npm in the background, download the
+new version, and switch to it on the next launch. New desktop builds bundle it: Dependabot opens a PR that bumps
+the pinned version in `Forsion-Genesis/desktop`. A copy is only replaced when the version number is higher, so
+forgetting the bump means the update silently never lands. Declare `minAppVersion` in `manifest.json` whenever a
+version needs a newer desktop; older desktops then skip it instead of loading something they cannot run.
+
+**First publish (once).** npm only lets you register a trusted publisher for a package that already exists:
+
+```bash
+gh release download v<version> -R Changan-Su/Tangu-Computer-Use -p 'forsion-tangu-computer-use-*.tgz'
+npm publish forsion-tangu-computer-use-<version>.tgz --access public
+```
+
+Then on npmjs.com open the package → Settings → Trusted publishing → GitHub Actions, with owner `Changan-Su`,
+repository `Tangu-Computer-Use`, workflow `release.yml` (or `npm trust github @forsion/tangu-computer-use
+--repo Changan-Su/Tangu-Computer-Use --file release.yml --allow-publish` with a current npm 11). Every later tag publishes by itself.
+
+发版 = 两处 version 一起升 + CHANGELOG + 推 `v<version>` tag,CI 构建全部 helper、校验包内容、挂 Release、经 trusted
+publishing 发 npm(仓库里没有 npm 令牌)。桌面端两路拿到新版:在跑的桌面后台从 npm 下载、下次启动换上;新桌面版由
+Dependabot 提 PR 升级内置版本。首个版本需按上面两条命令人工发一次,再到 npm 包设置里登记 trusted publisher。
+
+## What's in the bundle
+
+| path | what the host does with it |
+|---|---|
+| `manifest.json` + `main.js` | 桌面插件:注册视图 `plugin:tangu-computer-use:live`(被操控窗口的实时画面) |
+| `skills/computer-use/SKILL.md` | 配套技能:工具循环、后台优先、上 Agent Desk、不可逆动作的确认规则 |
+| `tangu-plugins/computer-use/` | 引擎插件:12 个工具 + `tangu computer-use` 子命令 |
+| `native/`, `scripts/`, `prebuilt/` | 三平台原生 helper 与它的构建/安装脚本 |
+
+## Tools
+
+`find_roots` · `observe_ui` · `search_ui` · `expand_ui` · `inspect_ui` · `act_ui` · `ensure_app` ·
+`read_text` · `wait_for` · `launch_browser` · `navigate_browser` · `evaluate_browser`
+
+All are host-only (require `hostExec`) and gated on the plugin being enabled + a supported platform.
+Action tools (`act_ui`, `launch/navigate/evaluate_browser`) require approval (same tier as `run_bash`);
+observation tools do not. `observe_ui` screenshots are fed back to the model as images. Oversized tool
+output is truncated with an `@o` continuation ref you read back through `read_text`.
+
+## Seeing what the agent is doing
+
+- **Edge glow** — the window currently being acted on gets a glowing border drawn by the native helper
+  (click-through, never takes focus, follows the window as it moves). Off switch: the *操作可视化*
+  setting, same one that controls the agent cursor.
+- **Live view** — `plugin:tangu-computer-use:live` shows that window's picture, polled ~8fps from the
+  helper's background capture. The skill tells the agent to put it on the **Agent Desk** at the start of
+  a session (`desk_present`), so the user watches instead of guessing. macOS only for now — the Windows
+  helper is a stdio child process with no server for the desktop to talk to.
+
+## When to use it vs. browser tools
+
+Computer Use drives **any on-screen desktop app**. For **pure web** tasks prefer `browser_task` /
+`browser_*` (lighter), or plain `curl` when the page is directly fetchable. Use the Computer Use browser
+tools only when a desktop workflow must also touch a web page in the same root forest.
+
+## Settings
+
+- **browser_use** (default on) — allow the CDP browser tools.
+- **managed_browser** (default Chrome) — which browser `launch_browser` starts.
+- **headless / 严格后台** (default off) — actions stay in the background; no foreground focus grab or
+  cursor move. Things the background genuinely cannot do then fail instead of stealing focus.
+- **cursor_overlay / 操作可视化** (default on) — the agent cursor *and* the controlled-window edge glow.
+
+## Development
+
+```bash
+npm install
+npm run build          # esbuild → tangu-plugins/computer-use/dist/index.js  +  tsc --noEmit
+npm run check          # foreground note · helper path · highlight geometry · desktop plugin
+npm run check:live     # real machine, needs an authorized helper:
+                       #   check:blindclick  background click lands without taking the foreground
+                       #   check:liveview    the live view really streams, at the window's aspect
+                       #   check:overlay     both overlays are ON SCREEN and ABOVE the target window
+                       # (briefly opens Calculator; not runnable in CI, so not part of `check`)
+npm run build:native   # build the macOS Swift helper (arm64 + x86_64)
+npm run build:windows  # Windows Rust helper (must run on Windows)
+npm run build:linux    # Linux Rust helper (must run on Linux)
+sh install.sh dev      # deploy the bundle to ~/.forsion-dev
+```
+
+See `UPSTREAM.md` for the vendor strategy and how to re-sync upstream.
+
+## License
+
+MIT (see `LICENSE`). Derived from pi-computer-use — upstream license in `LICENSE.upstream`.
+
+### Mini Panel foreground signal
+
+The macOS helper publishes a short-lived `foreground.json` beside its socket for Genesis Mini Panel. It becomes active only when real HID input is posted or the input path activates the target app; successful AX and PID background operations do not activate it. Nested physical-input scopes keep the signal active until the outer action finishes. A 1-second heartbeat renews a lease of at most 2500ms; short completed clicks remain observable for 350ms. Desktop checks expiry and helper liveness, never starts the helper or requests screenshots for this feature.
+
+Build the helper with `npm run build:native` and pair it with Genesis's Mini Panel adapter update. Builds produce a sealed App ZIP alongside each binary; installation never compiles or signs on the user's machine. `npm run check:mini-foreground` tests signal lifetime without controlling any application.
+
+macOS helper 会向 socket 同目录发布短时前台输入信号，供 Genesis Mini Panel 跟随光标。后台 AX/PID 调用不触发；必须配套更新 Genesis 并按现有签名流程重建 helper。验证命令为 `npm run check:mini-foreground`，不会操控用户应用。
+
+### macOS installation and signing / 安装与签名
+
+Run `node scripts/build-native.mjs --arch all` before packaging, then `node scripts/verify-macos-bundles.mjs` and `npm run check:installer`. Both architectures must include `tangu-computer-use.app.zip` and its JSON checksum manifest. ZIP transport preserves the helper signature through Electron's recursive signing. Runtime setup only verifies and copies these archives; missing or damaged artifacts fail without trying a user's signing identity or creating keys. `setup-helper.mjs --check` is read-only (exit 0: current, 10: installation/repair needed, 1: package error).
+
+Release builds are signed with the project's fixed self-signed certificate (`releaseCertSha1` in `scripts/macos-bundle.mjs`). macOS files the Accessibility and Screen Recording grants under the bundle id plus that certificate's hash, so helper updates keep the grants as long as neither changes. Never replace the certificate: every user would have to grant access again. `scripts/verify-package.mjs` refuses to release a helper signed by anything else. Local builds without `--sign-identity` stay ad-hoc and are for development only. This is not Developer ID signing or notarization.
+
+打包前构建完整双架构 App，运行上述两项校验。首次安装、升级及旧签名被拒绝后的修复都不再访问用户钥匙串；缺失或损坏的随包件会明确失败。发布版用固定的自签名证书签名(指纹见 `scripts/macos-bundle.mjs` 的 `releaseCertSha1`),macOS 按「bundle id + 证书指纹」记授权,两者不变则 helper 更新不用重新授权;**证书永远不能换**。本地不带 `--sign-identity` 的构建仍是 ad-hoc,只供开发。这不是 Developer ID 签名,也没有公证。
+
+### Mini Panel helper delivery / 辅助程序交付
+
+工具与平台指引离线回归：`npm run build && npm run check:platform`。在隔离进程中验证 macOS / Windows / Linux 工具可见性、macOS 专用启动工具的边界，以及常驻工具直接调用的说明，不启动 helper 或操作用户界面。Windows 真机仍需另验原生助手启动、发现窗口、观察和动作；不能把该检查等同于整机验收。
+
+Offline platform regression: `npm run build && npm run check:platform` checks tool visibility and guidance in isolated processes without starting a helper or controlling a UI. Native Windows acceptance still requires verifying helper startup, window discovery, observation and actions on Windows hardware.
+
+Genesis 的自动 Mini 依赖 helper socket 同目录的 `foreground.json`。升级 helper 行为时必须提升捆绑包版本并重建所有随包 native 产物；桌面按 manifest 版本播种，不覆盖同版本副本。0.5.2 起自动更新成功后重启常驻 helper，避免协议号与路径相同但内存中仍为旧版本。
+
+运行 `npm run check:helper-refresh` 检查升级顺序，`npm run check:helper-signal` 启动随包真实二进制并检查初始闲置信号；后者可追加已安装的可执行文件路径。完整前台触发与 Mini 过渡在 Genesis desktop 的 `npm run check:mininative` 中验证，需要已安装并授权的 macOS helper。该测试只操作隔离测试窗口。
+
+Automatic Mini requires the helper’s foreground signal. Bump the bundle version whenever shipping changed helper bits, rebuild all native artifacts, and restart the daemon after replacement. `check:helper-refresh` covers upgrade ordering; `check:helper-signal` probes real packaged bits. Genesis `check:mininative` covers actual physical input, external focus, current-session Mini and cursor motion against an isolated window.