deepseek-computer-use 0.1.0 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,16 @@
1
+ # 更新记录
2
+
3
+ ## 0.1.2 — 2026-10-18
4
+
5
+ - 适配真实 DeepSeek Harness:标准 JSON Schema 参数、模型可见的截图附件、审批审计,
6
+ 以及会话独立的应用授权、helper 和观测缓存;修正截图缩放后的坐标换算。
7
+ - 超时或取消后停止原生请求,确认旧 helper 退出后再返回失败。长文本输入可中途取消,
8
+ 按键和鼠标按钮在退出路径释放;回合结束时主动关闭 helper 与覆盖层。
9
+ - 输入或抓屏失败后清除观测,要求重新执行 `get_window_state`;截图 id 跨 helper 重启唯一。
10
+ 更新时若旧 helper 正被占用,可将新版编译到独立路径。
11
+ - 原生协议升级到 **2**。默认 helper 会自动重编译;使用自定义 `helperPath` 时需同步更新。
12
+ - 增加 18 项参数、策略和进程生命周期回归,并在 DSH `0.2.1-rc.2`、
13
+ `0.2.1-alpha.5` 中验证真实截图、审批、输入、取消及回合清理。
14
+
15
+ 本版未验证音频录制、多应用长任务或与 Codex 的模型任务成功率对比。
16
+ Codex SDK 和二进制未纳入分发,改进为独立实现。
package/README.md CHANGED
@@ -1,14 +1,22 @@
1
1
  # deepseek-computer-use
2
2
 
3
+ ![deepseek-computer-use 大肥鱼桌面控制封面](https://raw.githubusercontent.com/STARDUSTLC666/deepseek-computer-use/main/assets/cover-whale-girl.png)
4
+
5
+ [![npm](https://img.shields.io/npm/v/deepseek-computer-use.svg)](https://www.npmjs.com/package/deepseek-computer-use)
6
+
3
7
  > 社区插件,**非 DeepSeek 官方出品**。官方包在 `@deepseek-ai/*` scope 下。
4
8
 
5
- 给 DeepSeek Harness 的 **Windows 桌面控制(Computer Use)** 插件。
9
+ 让 **DeepSeek Harness 操作 Windows 桌面软件**:查看窗口截图、读取界面控件、
10
+ 点击按钮、填写表单。适用于没有可用 API、需要直接操作图形界面的任务。
11
+
12
+ 提供 16 个工具,覆盖窗口发现、截图观测、鼠标键盘、控件操作和可选的系统音频录制。
13
+ 支持会话独立授权、敏感操作审批与审计;取消或回合结束时停止原生助手,
14
+ 操作失败后要求重新观察,减少结果不明时的重复操作。
15
+
16
+ 底层采用 **UI Automation + SendInput + Windows.Graphics.Capture**,
17
+ 工具语义参考 Codex window2,并针对 DSH 的会话、审批和截图附件机制适配。
6
18
 
7
- 工具面对齐 Codex 的 window2 API(`list_apps` / `list_windows` / `get_window` /
8
- `launch_app` / `get_window_state` / `click` / `press_key` / `type_text` / `scroll` /
9
- `set_value` / `drag` / `perform_secondary_action` / `activate_window` /
10
- `start_audio_recording` / `stop_audio_recording` / `computer_use_health`),底层是
11
- Windows 原生三件套:**UI Automation 观测 + SendInput 注入 + Windows.Graphics.Capture 抓屏**。
19
+ 欢迎使用,遇到问题或有改进建议,请提交 [issues](https://github.com/STARDUSTLC666/deepseek-computer-use/issues) 和 [PR](https://github.com/STARDUSTLC666/deepseek-computer-use/pulls)。
12
20
 
13
21
  零 npm 原生依赖、零 .NET 运行时依赖:helper 和覆盖层用系统自带的 `csc.exe` 现编,
14
22
  抓屏(WGC)和录音(WASAPI)是 net48 程序,Windows 10/11 自带运行时。
@@ -19,13 +27,11 @@ Windows 原生三件套:**UI Automation 观测 + SendInput 注入 + Windows.Gr
19
27
  UI Automation、WGC)。
20
28
 
21
29
  ```bash
22
- # 发布到 npm 之后
23
- npm install --save deepseek-computer-use
24
-
25
- # 现在还只能用 GitHub 源
26
- npm install --save github:STARDUSTLC666/deepseek-computer-use
30
+ npm install deepseek-computer-use
27
31
  ```
28
32
 
33
+ [npmjs.com/package/deepseek-computer-use](https://www.npmjs.com/package/deepseek-computer-use)
34
+
29
35
  然后在本机 profile 的 `cordis.patch.yml` 里启用(详见本文件末尾的配置示例):
30
36
 
31
37
  ```yaml
@@ -46,10 +52,42 @@ npm run build # helper + overlay(csc),再尝试 dotnet 编 captur
46
52
  ## 自检
47
53
 
48
54
  ```bash
49
- npm test # 13 项断言:注册、观测、点击、审批、硬拒绝、Esc 阻断
50
- npm run test:all # 追加 helper 协议/音频 与插件级一致性测试
55
+ npm test # DSH 参数/策略回归 + 原生 WinForms 集成验证
56
+ npm run test:contracts # 只运行参数与策略测试,不操作桌面
57
+ npm run test:harness -- --dsh-root /path/to/node_modules/@deepseek-ai/dsh
58
+ # test:harness 启动隔离的真实 DSH CLI/profile,验证附件、审批和原生操作。
59
+ # npm run test:all 另含真实系统音频录制测试,按需运行。
60
+ ```
61
+
62
+ ## DeepSeek Harness 接入
63
+
64
+ 这是通过 DSH 的 `apply(ctx)` / `inject` / bundle patch 接入的独立插件。
65
+ 工具注册使用标准 JSON Schema,截图通过 `attachments` 保存并输出模型可见的 image block,
66
+ 应用授权与危险动作通过 `approval.request` 审批并写入会话审计。
67
+ 每个会话独立保存授权、helper、截图与元素快照;同一桌面的工具调用串行执行。
68
+ DSH 附件存储缩小截图时,坐标会自动换算回原始抓屏像素。
69
+
70
+ 在需要使用的 DSH profile 中安装,例如 Git Bash 下:
71
+
72
+ ```bash
73
+ dsh plugin --profile web add deepseek-computer-use
74
+ dsh web --no-open
51
75
  ```
52
76
 
77
+ 开发时可将包名换成 `file:E:/deepseek/dsh-computer-use` 来加载本地源码。
78
+ 以下宿主适配和失败恢复改进包含在 0.1.2 及后续版本中,详见 [更新记录](CHANGELOG.md)。
79
+ `test:harness` 不调用模型、不录音,只操作自己创建的测试窗口,结束后移除隔离 profile。
80
+
81
+ 中止与失败恢复:
82
+
83
+ - 超时或取消会先通知当前原生请求停止;输入循环检查取消标记,并在 `finally` 释放按键/鼠标按钮。
84
+ helper 未在 500ms 内退出时会被终止;调用失败返回前确认旧进程已经退出。
85
+ - DSH 的 `session/event` 出现 `turn/end` 时主动停止该会话的 helper 和覆盖层,不依赖下一次工具调用。
86
+ - 输入或抓屏失败会清除已缓存的快照。后续输入返回 `fresh_observation_required`,
87
+ 必须成功调用 `get_window_state` 后再决定动作;不会自动重复结果不明的输入。
88
+ - 新 helper 使用不同的截图 id,防止重启前的截图编号误命中新缓存。
89
+ 原生协议已升级为 2,自定义 `helperPath` 也需要匹配版本。
90
+
53
91
  ## 它解决什么问题
54
92
 
55
93
  DSH 本体没有鼠标键盘工具,只能跑命令、读写文件。需要"操作图形界面"时(点按钮、
@@ -73,7 +111,7 @@ DSH 宿主
73
111
  驱动层是接口(`IPlatformDriver`),今天是 `windows` 一个实现;macOS(AX + CGEvent)
74
112
  和 Linux(AT-SPI)可以照着补,插件层不用改。
75
113
 
76
- helper 常驻的原因:元素编号必须和"那一次观测"绑在同一个进程里,否则
114
+ 每个会话的 helper 常驻的原因:元素编号必须和"那一次观测"绑在同一个进程里,否则
77
115
  `click(element_index)` 无从解析。因此插件串行发请求、按 id 配对响应。
78
116
 
79
117
  ## 工具
@@ -111,7 +149,17 @@ Codex 用两层覆盖层告诉用户"AI 正在动你的电脑",这里做的是
111
149
  **Cursor Overlay — 虚拟光标 + 系统光标抑制**
112
150
  - 覆盖层存在期间常驻一个带描边和投影的白色指针,启动时落在真实指针位置,
113
151
  之后 0.35 缓动滑向每个动作点;背后一层极淡的强调色光晕
114
- - 默认抑制真实系统光标,退出/Esc/超时用 `SPI_SETCURSORS` 恢复;
152
+ - 默认抑制真实系统光标(`SetSystemCursor` 换成透明图标),五层恢复保障:
153
+ 1. 正常退出 / Esc / 超时 → 退出钩子里 `SPI_SETCURSORS`
154
+ 2. **独立守卫进程**(`--cursor-guard <ownerPid>`)盯着覆盖层,覆盖层被强杀后
155
+ 它负责还原——和 Codex 的 `system-cursor-manager` 同一思路
156
+ 3. helper 的 **500ms 看门狗**:标记里的 PID 失效就还原,不需要任何请求
157
+ 4. helper **启动即自检**:上次会话残留的标记会被清掉
158
+ 5. 插件 dispose 时发 `restore_cursor` RPC
159
+ - 抑制期间 `%LOCALAPPDATA%\dsh-computer-use\cursor-hidden.flag` 记录自己的 PID,
160
+ 守卫只清理属于自己 owner 的标记,不会误伤新一代覆盖层
161
+ - Windows 在休眠恢复 / 换分辨率后会重置光标,覆盖层每约 1.8 秒**重新抑制**一次
162
+ - 完全不想抑制:`DSH_CU_KEEP_CURSOR=1`
115
163
  helper 强杀覆盖层时也会兜底恢复一次
116
164
 
117
165
  **DPI 关键点**:覆盖层进程必须 Per-Monitor-V2 感知,否则在 150% 缩放屏上
@@ -129,6 +177,8 @@ Codex 用两层覆盖层告诉用户"AI 正在动你的电脑",这里做的是
129
177
  `launch_app` 总是问
130
178
  - `always`:每次动作都问
131
179
  审批走 DSH 自己的 approval seam;**没有审批服务时直接拒绝**,绝不静默降级。
180
+ DSH 宿主的 `approval` 策略 `never` 表示拒绝需要审批的请求,与本插件配置项的
181
+ `approval: never`(跳过动作级询问)含义不同。应用授权仍由 `appConsent` 单独控制。
132
182
  3. **前台证明**:打字、按键、坐标点击、滚动必须先证明目标窗口真的拿到了前台,
133
183
  否则抛 `focus_failed` 并告诉你当前是谁占着前台。SendInput 会打到有焦点的窗口,
134
184
  这一步省不得。
@@ -161,17 +211,22 @@ cd "E:\path\to\dsh-computer-use"
161
211
  approval: sensitive # never | sensitive | always
162
212
  allowApps: ["notepad", "chrome"] # 空 = 不限制
163
213
  denyApps: []
164
- defaultIncludeScreenshot: false # 默认不抓图,省 token
214
+ defaultIncludeScreenshot: true # 默认抓图;false 可显式关闭
165
215
  maxElements: 260
166
216
  snapshotTtlMs: 120000
167
217
  dryRun: false
168
218
  toolPrefix: '' # 填 'cu_' 可避免与其他插件的 click 重名
169
- timeoutMs: 20000
219
+ timeoutMs: 10000
170
220
  ```
171
221
 
172
- ## 契约对齐(对着 Codex 逐条核过)
222
+ ## 与 Codex Windows window2 的关系
223
+
224
+ 工具以 `@oai/sky` 的 Windows window2 动作和观测语义为参照,底层也采用 UIA、SendInput、WGC。
225
+ 它是 DSH 的独立实现,不能据此认为具有相同的模型能力、成功率或全部运行时行为。
173
226
 
174
- 工具名与参数按 `@oai/sky` 的 window2 契约实现,不是"看着像":
227
+ DSH 接口有这些明确差异:`list_windows` / `list_apps` 分别返回 `{ windows, count }` /
228
+ `{ apps, count }`;`window.app` 返回应用元数据对象,输入也接受字符串 app id 或只提供窗口 id;
229
+ `get_window` 接收 `{ window: { id } }`;截图返回 DSH 附件引用和 image block,Codex 使用图片 URL。
175
230
 
176
231
  | 契约项 | Codex | 本插件 |
177
232
  |---|---|---|
@@ -184,22 +239,22 @@ cd "E:\path\to\dsh-computer-use"
184
239
  | 元素失效 | 编号只对产生它的观测有效 | 一致(TTL + helper 进程内缓存) |
185
240
  | 最小化窗口 | 不该给出无意义画面 | 本插件直接报 `window_minimized`(实测旧版会吐出 334×48 的假图) |
186
241
  | 键名 | X keysym + `Numpad_*` 别名 | 一致(`KP_0`/`Numpad_Add`/…) |
187
- | `list_apps` | `lastUsedDate`、`useCount` | `lastUsedDate` 有;`useCount` 无数据源,不编造 |
242
+ | `list_apps` | `lastUsedDate`、`useCount` | `lastUsedDate` 用进程启动时间近似,并非用户最近使用时间;`useCount` 留空 |
188
243
  | `launch_app` | app id / `.exe` 路径(含 UWP) | exe 路径、AUMID(`shell:AppsFolder\\…`)、shell URI |
189
244
  | Win/Meta 键 | 硬拒绝 | 硬拒绝(插件层 + helper 层双保险) |
190
245
  | 终端/Run/密码管理器/安全软件/认证对话框/宿主自身 | 硬拒绝 | 硬拒绝(不可被审批覆盖) |
191
246
  | 按应用授权 | "Computer Use was not approved to use <app>" | 每会话每应用一次审批(`appConsent: session`),批准后下发 `approve_app`,**由 helper 强制**:未批准的应用直接 `app_not_approved` |
192
- | 回合边界 | 新 turn 时向 helper 发 `end_turn`,清元素快照 | 一致:token 变化即 `end_turn`(清快照/截图区/Esc 阻断,保留应用授权) |
247
+ | 回合边界 | 回合与 helper 生命周期关联 | DSH 回合结束事件主动停止旧 helper;新回合创建新 helper,并重新下发已获应用授权 |
193
248
  | 请求预算 | 普通 10s、`launch_app` 15s | 一致(`timeoutMs` / `launchTimeoutMs`) |
194
- | 回合结束停止输入 | `turn_ended` 钩子 | 读会话日志判断 open turn,`turn_ended` 拒绝注入 |
249
+ | 回合结束停止输入 | `turn_ended` 钩子 | 主动停止正在执行的请求,排队中的旧回合请求也被拒绝 |
195
250
 
196
- ## 明确没做的(不属于 Windows window2 面)
251
+ ## 能力边界
197
252
 
198
253
  - **`paste` / `select_text`**:属于 macOS 的 window 客户端(`target: "mac"`),Windows 上不存在。
199
- - **播客/音频录制**(`start_audio_recording`):契约里是可选方法,未实现。
254
+ - **系统音频录制**(`start_audio_recording`):已实现 WASAPI 环回录音,默认关闭,需要构建 audio helper 并单独授权。
200
255
  - **full-desktop 面**(`clipboard_read/write`、`key_down/up`、`move`、`drag_handle`、`get_screenshot`):那是 `target: "linux"` 的 API。
201
- - **区域裁剪截图**:Codex 是"bounded screenshot",我们截整窗(长边封顶 2560)。token 成本略高,语义一致。
202
- - **Windows.Graphics.Capture**:Codex 用 WGC;我们用 `PrintWindow(PW_RENDERFULLCONTENT)`,遮挡可用、最小化拒绝,不需要 D3D 互操作。
256
+ - **区域裁剪截图**:当前抓整窗(长边封顶 2560),不能认为与 Codex 的 bounded screenshot 完全等价。
257
+ - **Windows.Graphics.Capture**:构建 capture helper 后优先使用 WGC;不可用时回退到 PrintWindow / BitBlt,GPU 窗口的兼容性取决于实际使用的抓屏后端。
203
258
 
204
259
  ## 抓屏链路(对齐 Codex 的 WGC)
205
260
 
@@ -238,7 +293,7 @@ node test/integration.mjs
238
293
 
239
294
  固定装置是一个独立的 WinForms 可执行文件(`test/bin/dsh-cu-fixture.exe`,一个文本框 +
240
295
  一个名为 "Delete all" 的按钮;特意不用 shell 托管——shell 窗口本身被硬拒绝)。覆盖:
241
- 14 个工具注册、截图数组与 origin/zIndex、`accessibility:null` 语义、`set_value`、
296
+ 16 个工具注册、截图数组与 origin/zIndex、`accessibility:null` 语义、`set_value`、
242
297
  `type_text`、危险按钮触发审批、`dryRun` 不落地、`denyApps` 拒绝、Win 键硬拒绝、
243
298
  `scroll` 契约。
244
299
 
@@ -252,7 +307,7 @@ MIT。第三方商用前请自行做安全评审:这个插件能让模型操
252
307
  | 文件 | 用途 | 大小 |
253
308
  |---|---|---|
254
309
  | `dsh-computer-use-helper.exe` | UIA / SendInput / 抓屏调度 / 音频调度 | ~50KB |
255
- | `dsh-cu-overlay.exe` | 鲸鱼娘 + 虚拟光标覆盖层 | ~30KB |
310
+ | `dsh-cu-overlay.exe` | 呼吸光边 + 虚拟光标覆盖层 | ~30KB |
256
311
  | `capture/` | WGC 抓屏(.NET Framework 4.8 + SharpDX) | 0.68MB |
257
312
  | `audio/` | WASAPI 环回录音(NAudio) | 0.60MB |
258
313
  | **合计** | | **~1.4MB** |
@@ -263,7 +318,7 @@ MIT。第三方商用前请自行做安全评审:这个插件能让模型操
263
318
 
264
319
  | 能力 | 实现 |
265
320
  |---|---|
266
- | 协议握手 | helper 提供 `hello` → `{ protocolVersion: 1, capabilities: [...] }`;bridge 首次连接校验,不匹配直接报 `protocol_mismatch` |
321
+ | 协议握手 | helper 提供 `hello` → `{ protocolVersion: 2, capabilities: [...] }`;bridge 首次连接校验,不匹配直接报 `protocol_mismatch` |
267
322
  | 每请求预算 | 客户端把预算随请求下发(`__budgetMs`,对应 Codex 的 `x-oai-cua-request-budget-ms`),helper 用它收缩抓屏等待 |
268
323
  | 坐标越界 | `point_outside_screenshot`,文案同 Codex("is outside screenshot bounds … Take a fresh screenshot before retrying") |
269
324
  | 截图失效 | `stale_screenshot` 带同一句 "Take a fresh screenshot before retrying" |
@@ -289,6 +344,9 @@ marker 文件按 **会话 + turn** 命名,避免旧会话的 Esc 标记误伤
289
344
  audio: false # 开启 WASAPI 环回录音(start/stop_audio_recording)
290
345
  redactApps: [] # 这些应用的截图永不出机器,例如 ["1password", "银行"]
291
346
  defaultIncludeScreenshot: true
347
+ timeoutMs: 10000 # 单次 RPC 超时(launch_app 用 launchTimeoutMs)
348
+ helperPath: null # 覆盖 helper 路径(仅信任场景)
349
+ riskyPattern: '' # 自定义「危险动作」正则,命中走审批
292
350
  toolPrefix: '' # 例如 cu_ 可避免和其他插件的 click 重名
293
351
  ```
294
352
 
@@ -297,6 +355,8 @@ marker 文件按 **会话 + turn** 命名,避免旧会话的 Esc 标记误伤
297
355
  - 只支持 Windows;驱动层是接口(`IPlatformDriver`),macOS / Linux 可照着补。
298
356
  - Codex window2 没有的接口(`paste` / `clipboard_*` / `key_down` / `move`)这里也没有。
299
357
  - `App.useCount` 没有数据来源,按可选字段留空,不伪造。
358
+ - `redactApps` 在**插件层**执行:直接调用 helper 的 RPC 不受它约束,别把 helper 单独暴露给非受信调用方。
359
+ - 抢焦点类工具(如 ProcessLasso)会让 `type_text` 返回 `focus_failed`,这是设计行为——宁可不输入,也不打到别的窗口。
300
360
 
301
361
  ## License
302
362
 
Binary file
package/lib/bridge.js CHANGED
@@ -1,22 +1,14 @@
1
- /**
2
- * Lifecycle and framing for the native helper process.
3
- *
4
- * One long-lived helper owns the UI Automation element cache, so element
5
- * indexes returned by get_window_state stay resolvable by click/set_value.
6
- * Requests are serialized: UI Automation calls are not reentrant and a
7
- * desynchronized element cache is worse than waiting a few milliseconds.
8
- *
9
- * @module deepseek-computer-use/bridge
10
- */
1
+ /** Serialized native RPC, with cancellation that waits for the owned process to stop. */
11
2
  import { spawn } from 'node:child_process';
3
+ import { randomUUID } from 'node:crypto';
4
+ import fs from 'node:fs';
5
+ import os from 'node:os';
6
+ import path from 'node:path';
7
+ import { setTimeout as delay } from 'node:timers/promises';
12
8
 
13
- /** Protocol version this plugin speaks; the helper reports its own in `hello`. */
14
- const MIN_PROTOCOL = 1;
15
-
16
- /** Codex's wording, reused verbatim so the model reacts the same way. */
9
+ const MIN_PROTOCOL = 2;
17
10
  const ESC_MESSAGE = 'Computer Use was stopped by the user with the physical Escape key. Stop your work, do not call further Computer Use tools in this turn, and send a final message noting that the user stopped Computer Use.';
18
11
 
19
- /** An error raised by the helper, carrying its machine-readable code. */
20
12
  export class HelperError extends Error {
21
13
  constructor(code, message, details) {
22
14
  super(message);
@@ -28,197 +20,208 @@ export class HelperError extends Error {
28
20
 
29
21
  export class HelperBridge {
30
22
  #child = null;
23
+ #closed = new WeakMap();
24
+ #termination = new WeakMap();
25
+ #reasons = new WeakMap();
31
26
  #buffer = '';
32
27
  #pending = new Map();
33
28
  #seq = 0;
34
29
  #queue = Promise.resolve();
30
+ #stopping = Promise.resolve();
35
31
  #exe;
32
+ #args;
33
+ #cancelDirectory;
36
34
  #timeoutMs;
37
35
  #protocol = null;
38
- #interruptedTurn = null;
36
+ #stoppedTurns = new Map();
39
37
  #lastTurn = null;
40
38
  #disposed = false;
41
39
  #log;
42
40
 
43
- constructor({ exe, timeoutMs = 20000, log } = {}) {
41
+ constructor({ exe, args = [], cancelDirectory, timeoutMs = 20000, log } = {}) {
44
42
  if (typeof exe !== 'string' || exe.length === 0) throw new Error('HelperBridge requires an executable path');
45
43
  this.#exe = exe;
44
+ this.#args = args;
45
+ this.#cancelDirectory = cancelDirectory ?? path.join(os.tmpdir(), 'dsh-computer-use-requests');
46
46
  this.#timeoutMs = timeoutMs;
47
47
  this.#log = log;
48
48
  }
49
49
 
50
- get running() {
51
- return this.#child !== null && this.#child.exitCode === null && !this.#child.killed;
52
- }
50
+ get running() { return this.#child !== null && this.#child.exitCode === null && this.#child.signalCode === null; }
51
+ get pid() { return this.running ? this.#child.pid : null; }
53
52
 
54
53
  #ensureChild() {
55
54
  if (this.#disposed) throw new HelperError('disposed', 'the Computer Use bridge was disposed');
56
55
  if (this.running) return this.#child;
57
- const child = spawn(this.#exe, [], { stdio: ['pipe', 'pipe', 'pipe'], windowsHide: true });
56
+ const child = spawn(this.#exe, this.#args, { stdio: ['pipe', 'pipe', 'pipe'], windowsHide: true });
57
+ this.#child = child;
58
+ this.#buffer = '';
59
+ this.#protocol = null;
60
+ this.#closed.set(child, new Promise(resolve => child.once('close', resolve)));
58
61
  child.stdout.setEncoding('utf8');
59
62
  child.stderr.setEncoding('utf8');
60
- child.stdout.on('data', chunk => this.#onData(chunk));
61
- child.stderr.on('data', chunk => {
62
- const text = String(chunk).trim();
63
- if (text.length > 0) this.#log?.('[dsh-computer-use] helper stderr: ' + text);
63
+ child.stdout.on('data', chunk => { if (this.#child === child) this.#onData(child, chunk); });
64
+ child.stderr.on('data', chunk => this.#log?.('[dsh-computer-use] helper: ' + String(chunk).trim().slice(0, 500)));
65
+ child.stdin.on('error', error => this.#stop(child, new HelperError('write_failed', error.message)));
66
+ child.on('error', error => {
67
+ this.#reasons.set(child, new HelperError('helper_start_failed', error.message));
64
68
  });
65
- child.on('exit', (code, signal) => {
66
- this.#child = null;
67
- this.#protocol = null;
68
- // Codex's helper exits 130 when the user presses Esc; the transport turns that
69
- // into the interrupt error so the model stops instead of retrying.
70
- const interrupted = code === 130;
71
- if (interrupted && this.#lastTurn !== null) this.#interruptedTurn = this.#lastTurn;
72
- const reason = interrupted
73
- ? ESC_MESSAGE
74
- : 'the Computer Use helper exited (code ' + code + ', signal ' + signal + ')';
75
- for (const [, entry] of this.#pending) {
76
- clearTimeout(entry.timer);
77
- entry.reject(new HelperError(interrupted ? 'cancelled_by_user' : 'helper_exited', reason));
69
+ child.on('close', (code, signal) => {
70
+ const escaped = code === 130;
71
+ if (escaped && this.#lastTurn !== null) this.#stoppedTurns.set(this.#lastTurn, new HelperError('turn_stopped', ESC_MESSAGE));
72
+ const error = this.#reasons.get(child) ?? new HelperError(escaped ? 'cancelled_by_user' : 'helper_exited',
73
+ escaped ? ESC_MESSAGE : 'the Computer Use helper exited (code ' + code + ', signal ' + signal + ')');
74
+ if (this.#child === child) {
75
+ this.#child = null;
76
+ this.#protocol = null;
77
+ this.#buffer = '';
78
78
  }
79
- this.#pending.clear();
79
+ this.#rejectChild(child, error);
80
80
  });
81
- child.on('error', error => {
82
- this.#log?.('[dsh-computer-use] helper spawn error: ' + error.message);
83
- });
84
- this.#child = child;
85
81
  return child;
86
82
  }
87
83
 
88
- #onData(chunk) {
89
- this.#buffer += chunk;
90
- let index = this.#buffer.indexOf('\n');
91
- while (index >= 0) {
92
- const line = this.#buffer.slice(0, index).trim();
93
- this.#buffer = this.#buffer.slice(index + 1);
94
- if (line.length > 0) this.#settle(line);
95
- index = this.#buffer.indexOf('\n');
84
+ #rejectChild(child, error) {
85
+ for (const [id, entry] of this.#pending) {
86
+ if (entry.child !== child) continue;
87
+ this.#pending.delete(id);
88
+ entry.cleanup();
89
+ entry.reject(error);
90
+ }
91
+ }
92
+
93
+ #stop(child, error) {
94
+ const stopped = this.#terminate(child, error);
95
+ stopped.catch(failure => this.#rejectChild(child, failure));
96
+ return stopped;
97
+ }
98
+
99
+ #terminate(child, error) {
100
+ if (!child) return this.#stopping;
101
+ if (this.#termination.has(child)) return this.#termination.get(child);
102
+ this.#reasons.set(child, error);
103
+ for (const entry of this.#pending.values()) {
104
+ if (entry.child !== child) continue;
105
+ entry.stopping = true;
106
+ clearTimeout(entry.timer);
107
+ try { fs.writeFileSync(entry.cancelFile, error.code, 'utf8'); } catch { /* forced stop below remains available */ }
96
108
  }
109
+ const stop = (async () => {
110
+ const closed = this.#closed.get(child);
111
+ // Native input loops poll the marker and release held input in finally.
112
+ // The close frame is processed once that request unwinds.
113
+ try { child.stdin.write(JSON.stringify({ id: ++this.#seq, method: 'close', params: {} }) + '\n'); } catch { /* process already exited */ }
114
+ let ended = await Promise.race([closed.then(() => true), delay(500).then(() => false)]);
115
+ if (!ended) {
116
+ child.kill('SIGKILL');
117
+ ended = await Promise.race([closed.then(() => true), delay(2000).then(() => false)]);
118
+ }
119
+ if (!ended) {
120
+ this.#disposed = true;
121
+ throw new HelperError('helper_stop_failed', 'the Computer Use helper could not be stopped; no further Computer Use requests are allowed');
122
+ }
123
+ })();
124
+ this.#termination.set(child, stop);
125
+ this.#stopping = stop;
126
+ return stop;
97
127
  }
98
128
 
99
- #settle(line) {
100
- let payload;
101
- try {
102
- payload = JSON.parse(line);
103
- } catch {
104
- this.#log?.('[dsh-computer-use] helper produced non-JSON output: ' + line.slice(0, 200));
105
- return;
129
+ #onData(child, chunk) {
130
+ this.#buffer += chunk;
131
+ let newline;
132
+ while ((newline = this.#buffer.indexOf('\n')) >= 0) {
133
+ const line = this.#buffer.slice(0, newline).trim();
134
+ this.#buffer = this.#buffer.slice(newline + 1);
135
+ if (!line) continue;
136
+ let payload;
137
+ try { payload = JSON.parse(line); }
138
+ catch {
139
+ this.#stop(child, new HelperError('protocol_error', 'the Computer Use helper returned invalid JSON; take a fresh observation before retrying'));
140
+ return;
141
+ }
142
+ const entry = this.#pending.get(payload?.id);
143
+ if (!entry || entry.child !== child || entry.stopping) continue;
144
+ if (typeof payload.ok !== 'boolean') {
145
+ this.#stop(child, new HelperError('protocol_error', 'the Computer Use helper returned an invalid response'));
146
+ return;
147
+ }
148
+ const code = payload.error?.code;
149
+ if (!payload.ok && ['timeout', 'cancelled', 'cancelled_by_user', 'turn_stopped'].includes(code)) {
150
+ if (['cancelled_by_user', 'turn_stopped'].includes(code) && entry.turnToken !== undefined) {
151
+ this.#stoppedTurns.set(entry.turnToken, new HelperError('turn_stopped', ESC_MESSAGE));
152
+ }
153
+ this.#stop(child, new HelperError(code, payload.error.message));
154
+ continue;
155
+ }
156
+ this.#pending.delete(payload.id);
157
+ entry.cleanup();
158
+ if (payload.ok) entry.resolve(payload.result);
159
+ else entry.reject(new HelperError(code ?? 'helper_error', payload.error?.message ?? 'the helper request failed', payload.error));
106
160
  }
107
- const entry = this.#pending.get(payload.id);
108
- if (entry === undefined) return;
109
- this.#pending.delete(payload.id);
110
- clearTimeout(entry.timer);
111
- if (payload.ok === true) entry.resolve(payload.result);
112
- else entry.reject(new HelperError(payload.error?.code ?? 'helper_error', payload.error?.message ?? 'the helper reported an unknown failure', payload.error));
113
161
  }
114
162
 
115
- /**
116
- * Send one request. Calls are serialized through the queue; a caller that
117
- * never gets an answer fails on the timeout rather than hanging the turn.
118
- */
119
163
  request(method, params = {}, { signal, timeoutMs, turnToken } = {}) {
120
- const wire = turnToken === undefined ? params : { ...params, turnToken };
121
- if (turnToken !== undefined) this.#lastTurn = turnToken;
122
164
  const run = async () => {
123
- // Codex stops issuing input for the rest of a turn once the user hits Esc.
124
- // The helper persists the same fact, but failing here keeps the error
125
- // immediate even while a fresh helper process is spinning up.
126
- if (turnToken !== undefined && this.#interruptedTurn === turnToken) {
127
- throw new HelperError('turn_stopped', ESC_MESSAGE);
128
- }
129
- await this.#ensureProtocol(signal);
130
- try {
131
- return await this.#call(method, wire, { signal, timeoutMs });
132
- } catch (error) {
133
- if ((error?.code === 'cancelled_by_user' || error?.code === 'turn_stopped') && turnToken !== undefined) {
134
- this.#interruptedTurn = turnToken;
165
+ await this.#stopping;
166
+ if (this.#disposed) throw new HelperError('disposed', 'the Computer Use bridge was disposed');
167
+ if (signal?.aborted) throw new HelperError('cancelled', 'the request was cancelled before it started');
168
+ if (turnToken !== undefined && this.#stoppedTurns.has(turnToken)) throw this.#stoppedTurns.get(turnToken);
169
+ if (turnToken !== undefined) this.#lastTurn = turnToken;
170
+ if (this.#protocol === null) {
171
+ const hello = await this.#call('hello', {}, { signal, timeoutMs: 5000, turnToken });
172
+ if (!Number.isInteger(hello?.protocolVersion) || hello.protocolVersion < MIN_PROTOCOL) {
173
+ const error = new HelperError('protocol_mismatch', 'the Computer Use helper must support cancellation protocol ' + MIN_PROTOCOL + '; rebuild the plugin helper');
174
+ await this.#terminate(this.#child, error);
175
+ throw error;
135
176
  }
136
- throw error;
177
+ this.#protocol = hello.protocolVersion;
137
178
  }
179
+ if (turnToken !== undefined && this.#stoppedTurns.has(turnToken)) throw this.#stoppedTurns.get(turnToken);
180
+ return this.#call(method, params, { signal, timeoutMs, turnToken });
138
181
  };
139
182
  const next = this.#queue.then(run, run);
140
- this.#queue = next.then(() => undefined, () => undefined);
183
+ this.#queue = next.catch(() => {});
141
184
  return next;
142
185
  }
143
186
 
144
- /**
145
- * The helper reports `protocolVersion` from `hello`. Codex exposes the same fact as
146
- * `window_protocol_version`; checking it up front turns a silent mismatch into one
147
- * clear error instead of a stream of malformed-stream failures.
148
- */
149
- async #ensureProtocol(signal) {
150
- if (this.#protocol !== null) return;
151
- const hello = await this.#call('hello', {}, { signal, timeoutMs: 5000 });
152
- const version = Number(hello?.protocolVersion);
153
- if (!Number.isFinite(version) || version < MIN_PROTOCOL) {
154
- throw new HelperError('protocol_mismatch',
155
- 'the Computer Use helper speaks protocol ' + (hello?.protocolVersion ?? 'unknown') +
156
- ', this plugin needs ' + MIN_PROTOCOL + ' or newer. Reinstall deepseek-computer-use so the helper and the plugin match.');
157
- }
158
- this.#protocol = version;
159
- }
160
-
161
- #call(method, params, { signal, timeoutMs }) {
187
+ #call(method, params, { signal, timeoutMs, turnToken }) {
162
188
  if (signal?.aborted) return Promise.reject(new HelperError('cancelled', 'the request was cancelled before it started'));
163
189
  const child = this.#ensureChild();
190
+ fs.mkdirSync(this.#cancelDirectory, { recursive: true });
191
+ const cancelFile = path.join(this.#cancelDirectory, randomUUID() + '.flag');
164
192
  const id = ++this.#seq;
165
- const budgetMs = Number.isFinite(timeoutMs) ? timeoutMs : this.#timeoutMs;
166
- // Codex ships the budget with every call as x-oai-cua-request-budget-ms so the
167
- // helper can bound its own waits; the same deadline travels here.
168
- const payload = JSON.stringify({ id, method, params: { ...params, __budgetMs: budgetMs } });
169
-
170
- const raw = new Promise((resolve, reject) => {
171
- const budget = Number.isFinite(timeoutMs) ? timeoutMs : this.#timeoutMs;
172
- const timer = setTimeout(() => {
173
- this.#pending.delete(id);
174
- reject(new HelperError('timeout', 'the Computer Use helper did not answer ' + method + ' within ' + budget + 'ms'));
175
- }, budget);
176
- timer.unref?.();
177
-
193
+ const budget = timeoutMs ?? this.#timeoutMs;
194
+ return new Promise((resolve, reject) => {
195
+ const timer = setTimeout(() => this.#stop(child, new HelperError('timeout',
196
+ 'the Computer Use helper exceeded ' + budget + 'ms for ' + method + '; take a fresh observation before retrying')), budget);
178
197
  const onAbort = () => {
179
- this.#pending.delete(id);
198
+ if (turnToken !== undefined) this.#stoppedTurns.set(turnToken, new HelperError('turn_stopped', 'Computer Use was cancelled in this turn; wait for a new turn'));
199
+ this.#stop(child, new HelperError('cancelled', 'the Computer Use request was cancelled and its helper stopped'));
200
+ };
201
+ const cleanup = () => {
180
202
  clearTimeout(timer);
181
- reject(new HelperError('cancelled', 'the request was cancelled'));
203
+ signal?.removeEventListener('abort', onAbort);
204
+ try { fs.unlinkSync(cancelFile); } catch { /* no marker was needed */ }
182
205
  };
206
+ this.#pending.set(id, { child, cancelFile, timer, cleanup, resolve, reject, turnToken, stopping: false });
183
207
  signal?.addEventListener('abort', onAbort, { once: true });
184
-
185
- this.#pending.set(id, {
186
- timer,
187
- resolve: value => {
188
- signal?.removeEventListener('abort', onAbort);
189
- resolve(value);
190
- },
191
- reject: error => {
192
- signal?.removeEventListener('abort', onAbort);
193
- reject(error);
194
- },
195
- });
196
-
208
+ if (signal?.aborted) { onAbort(); return; }
197
209
  try {
198
- child.stdin.write(payload + '\n');
199
- } catch (error) {
200
- this.#pending.delete(id);
201
- clearTimeout(timer);
202
- reject(new HelperError('write_failed', 'cannot talk to the Computer Use helper: ' + error.message));
203
- }
210
+ child.stdin.write(JSON.stringify({ id, method, params: { ...params, ...(turnToken === undefined ? {} : { turnToken }), __budgetMs: budget, __cancelFile: cancelFile } }) + '\n', error => {
211
+ if (error) this.#stop(child, new HelperError('write_failed', error.message));
212
+ });
213
+ } catch (error) { this.#stop(child, new HelperError('write_failed', error.message)); }
204
214
  });
205
- // The caller always handles this promise; the extra branch only stops Node from
206
- // treating a momentarily unobserved rejection as a fatal unhandled one.
207
- raw.catch(() => { /* observed by the caller; this branch only silences Node */ });
208
- return raw;
215
+ }
216
+
217
+ stopTurn(turnToken) {
218
+ const error = new HelperError('turn_ended', 'the requesting turn has ended; wait for a new turn and take a fresh observation');
219
+ this.#stoppedTurns.set(turnToken, error);
220
+ return this.#terminate(this.#child, error);
209
221
  }
210
222
 
211
223
  dispose() {
212
224
  this.#disposed = true;
213
- const child = this.#child;
214
- this.#child = null;
215
- if (child !== null) {
216
- // Codex closes the helper with a close RPC before tearing the process down.
217
- try {
218
- child.stdin.write(JSON.stringify({ id: ++this.#seq, method: 'close', params: {} }) + '\n');
219
- } catch { /* already gone */ }
220
- try { child.stdin.end(); } catch { /* already gone */ }
221
- setTimeout(() => { try { child.kill(); } catch { /* already gone */ } }, 150).unref?.();
222
- }
225
+ return this.#terminate(this.#child, new HelperError('disposed', 'the Computer Use bridge was disposed'));
223
226
  }
224
227
  }