deepseek-computer-use 0.1.1 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +22 -0
- package/README.md +82 -24
- package/assets/cover-whale-girl.png +0 -0
- package/lib/bridge.js +153 -150
- package/lib/build-helper.mjs +18 -4
- package/lib/build.mjs +2 -2
- package/lib/index.js +58 -35
- package/lib/parameters.js +50 -0
- package/lib/safety.js +2 -9
- package/lib/tools.js +131 -32
- package/lib/update-client.js +408 -0
- package/native/Driver.cs +1906 -1857
- package/native/Json.cs +274 -274
- package/native/Overlay.cs +732 -725
- package/native/Program.cs +160 -157
- package/native/audio/Program.cs +95 -95
- package/native/wgcfw/Program.cs +249 -249
- package/package.json +26 -8
- package/skills/computer-use/SKILL.md +5 -2
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# 更新记录
|
|
2
|
+
|
|
3
|
+
## 0.2.0 (2026-10-26)
|
|
4
|
+
|
|
5
|
+
- Add this plugin’s own update controls and independent preferences. Automatic installation stays off until explicitly enabled.
|
|
6
|
+
- Confirm manual installs, preserve local sources and disabled state, pause for tasks or unsaved changes, and use native DSH recovery without forced restarts.
|
|
7
|
+
- Require the public DSH 0.2.1-rc.3 update APIs; Suite components remain managed through Suite.
|
|
8
|
+
|
|
9
|
+
## 0.1.2 — 2026-10-18
|
|
10
|
+
|
|
11
|
+
- 适配真实 DeepSeek Harness:标准 JSON Schema 参数、模型可见的截图附件、审批审计,
|
|
12
|
+
以及会话独立的应用授权、helper 和观测缓存;修正截图缩放后的坐标换算。
|
|
13
|
+
- 超时或取消后停止原生请求,确认旧 helper 退出后再返回失败。长文本输入可中途取消,
|
|
14
|
+
按键和鼠标按钮在退出路径释放;回合结束时主动关闭 helper 与覆盖层。
|
|
15
|
+
- 输入或抓屏失败后清除观测,要求重新执行 `get_window_state`;截图 id 跨 helper 重启唯一。
|
|
16
|
+
更新时若旧 helper 正被占用,可将新版编译到独立路径。
|
|
17
|
+
- 原生协议升级到 **2**。默认 helper 会自动重编译;使用自定义 `helperPath` 时需同步更新。
|
|
18
|
+
- 增加 18 项参数、策略和进程生命周期回归,并在 DSH `0.2.1-rc.2`、
|
|
19
|
+
`0.2.1-alpha.5` 中验证真实截图、审批、输入、取消及回合清理。
|
|
20
|
+
|
|
21
|
+
本版未验证音频录制、多应用长任务或与 Codex 的模型任务成功率对比。
|
|
22
|
+
Codex SDK 和二进制未纳入分发,改进为独立实现。
|
package/README.md
CHANGED
|
@@ -1,16 +1,25 @@
|
|
|
1
1
|
# deepseek-computer-use
|
|
2
2
|
|
|
3
|
+

|
|
4
|
+
|
|
3
5
|
[](https://www.npmjs.com/package/deepseek-computer-use)
|
|
6
|
+
[](#安装)
|
|
7
|
+
[](package.json)
|
|
8
|
+
[](LICENSE)
|
|
4
9
|
|
|
5
10
|
> 社区插件,**非 DeepSeek 官方出品**。官方包在 `@deepseek-ai/*` scope 下。
|
|
6
11
|
|
|
7
|
-
|
|
12
|
+
让 **DeepSeek Harness 操作 Windows 桌面软件**:查看窗口截图、读取界面控件、
|
|
13
|
+
点击按钮、填写表单。适用于没有可用 API、需要直接操作图形界面的任务。
|
|
14
|
+
|
|
15
|
+
提供 16 个工具,覆盖窗口发现、截图观测、鼠标键盘、控件操作和可选的系统音频录制。
|
|
16
|
+
支持会话独立授权、敏感操作审批与审计;取消或回合结束时停止原生助手,
|
|
17
|
+
操作失败后要求重新观察,减少结果不明时的重复操作。
|
|
18
|
+
|
|
19
|
+
底层采用 **UI Automation + SendInput + Windows.Graphics.Capture**,
|
|
20
|
+
工具语义参考 Codex window2,并针对 DSH 的会话、审批和截图附件机制适配。
|
|
8
21
|
|
|
9
|
-
|
|
10
|
-
`launch_app` / `get_window_state` / `click` / `press_key` / `type_text` / `scroll` /
|
|
11
|
-
`set_value` / `drag` / `perform_secondary_action` / `activate_window` /
|
|
12
|
-
`start_audio_recording` / `stop_audio_recording` / `computer_use_health`),底层是
|
|
13
|
-
Windows 原生三件套:**UI Automation 观测 + SendInput 注入 + Windows.Graphics.Capture 抓屏**。
|
|
22
|
+
欢迎使用,遇到问题或有改进建议,请提交 [issues](https://github.com/STARDUSTLC666/deepseek-computer-use/issues) 和 [PR](https://github.com/STARDUSTLC666/deepseek-computer-use/pulls)。
|
|
14
23
|
|
|
15
24
|
零 npm 原生依赖、零 .NET 运行时依赖:helper 和覆盖层用系统自带的 `csc.exe` 现编,
|
|
16
25
|
抓屏(WGC)和录音(WASAPI)是 net48 程序,Windows 10/11 自带运行时。
|
|
@@ -46,15 +55,51 @@ npm run build # helper + overlay(csc),再尝试 dotnet 编 captur
|
|
|
46
55
|
## 自检
|
|
47
56
|
|
|
48
57
|
```bash
|
|
49
|
-
npm test #
|
|
50
|
-
npm run test:
|
|
58
|
+
npm test # DSH 参数/策略回归 + 原生 WinForms 集成验证
|
|
59
|
+
npm run test:contracts # 只运行参数与策略测试,不操作桌面
|
|
60
|
+
npm run test:harness -- --dsh-root /path/to/node_modules/@deepseek-ai/dsh
|
|
61
|
+
# test:harness 启动隔离的真实 DSH CLI/profile,验证附件、审批和原生操作。
|
|
62
|
+
# npm run test:all 另含真实系统音频录制测试,按需运行。
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## DeepSeek Harness 接入
|
|
66
|
+
|
|
67
|
+
这是通过 DSH 的 `apply(ctx)` / `inject` / bundle patch 接入的独立插件。
|
|
68
|
+
工具注册使用标准 JSON Schema,截图通过 `attachments` 保存并输出模型可见的 image block,
|
|
69
|
+
应用授权与危险动作通过 `approval.request` 审批并写入会话审计。
|
|
70
|
+
每个会话独立保存授权、helper、截图与元素快照;同一桌面的工具调用串行执行。
|
|
71
|
+
DSH 附件存储缩小截图时,坐标会自动换算回原始抓屏像素。
|
|
72
|
+
|
|
73
|
+
在需要使用的 DSH profile 中安装,例如 Git Bash 下:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
dsh plugin --profile web add deepseek-computer-use
|
|
77
|
+
dsh web --no-open
|
|
51
78
|
```
|
|
52
79
|
|
|
80
|
+
开发时可将包名换成 `file:E:/deepseek/dsh-computer-use` 来加载本地源码。
|
|
81
|
+
以下宿主适配和失败恢复改进包含在 0.1.2 及后续版本中,详见 [更新记录](CHANGELOG.md)。
|
|
82
|
+
`test:harness` 不调用模型、不录音,只操作自己创建的测试窗口,结束后移除隔离 profile。
|
|
83
|
+
|
|
84
|
+
中止与失败恢复:
|
|
85
|
+
|
|
86
|
+
- 超时或取消会先通知当前原生请求停止;输入循环检查取消标记,并在 `finally` 释放按键/鼠标按钮。
|
|
87
|
+
helper 未在 500ms 内退出时会被终止;调用失败返回前确认旧进程已经退出。
|
|
88
|
+
- DSH 的 `session/event` 出现 `turn/end` 时主动停止该会话的 helper 和覆盖层,不依赖下一次工具调用。
|
|
89
|
+
- 输入或抓屏失败会清除已缓存的快照。后续输入返回 `fresh_observation_required`,
|
|
90
|
+
必须成功调用 `get_window_state` 后再决定动作;不会自动重复结果不明的输入。
|
|
91
|
+
- 新 helper 使用不同的截图 id,防止重启前的截图编号误命中新缓存。
|
|
92
|
+
原生协议已升级为 2,自定义 `helperPath` 也需要匹配版本。
|
|
93
|
+
|
|
53
94
|
## 它解决什么问题
|
|
54
95
|
|
|
55
|
-
DSH
|
|
56
|
-
|
|
57
|
-
|
|
96
|
+
DSH 官方已有需要显式启用的实验性 Computer Use 方案,包括 Cua Driver MCP 与原生
|
|
97
|
+
提供方;其上游 Cua Driver 也支持 Windows。详见 [官方计算机操作文档](https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/subsystems/computer-use.zh.md)
|
|
98
|
+
和 [Cua Driver 平台支持](https://github.com/trycua/cua/blob/main/docs/content/docs/reference/cua-driver/contracts.mdx)。
|
|
99
|
+
|
|
100
|
+
本插件是另一套独立的 Windows 社区实现,提供 UI Automation、窗口截图、鼠标键盘、
|
|
101
|
+
会话授权、审批审计和中断恢复,用于点击按钮、填写表单或操作没有 API 的桌面软件。
|
|
102
|
+
它与官方方案有功能重合;本项目的验收结果不代表已与官方驱动做过效果或性能对比。
|
|
58
103
|
|
|
59
104
|
## 架构
|
|
60
105
|
|
|
@@ -73,7 +118,7 @@ DSH 宿主
|
|
|
73
118
|
驱动层是接口(`IPlatformDriver`),今天是 `windows` 一个实现;macOS(AX + CGEvent)
|
|
74
119
|
和 Linux(AT-SPI)可以照着补,插件层不用改。
|
|
75
120
|
|
|
76
|
-
helper 常驻的原因:元素编号必须和"那一次观测"绑在同一个进程里,否则
|
|
121
|
+
每个会话的 helper 常驻的原因:元素编号必须和"那一次观测"绑在同一个进程里,否则
|
|
77
122
|
`click(element_index)` 无从解析。因此插件串行发请求、按 id 配对响应。
|
|
78
123
|
|
|
79
124
|
## 工具
|
|
@@ -139,6 +184,8 @@ Codex 用两层覆盖层告诉用户"AI 正在动你的电脑",这里做的是
|
|
|
139
184
|
`launch_app` 总是问
|
|
140
185
|
- `always`:每次动作都问
|
|
141
186
|
审批走 DSH 自己的 approval seam;**没有审批服务时直接拒绝**,绝不静默降级。
|
|
187
|
+
DSH 宿主的 `approval` 策略 `never` 表示拒绝需要审批的请求,与本插件配置项的
|
|
188
|
+
`approval: never`(跳过动作级询问)含义不同。应用授权仍由 `appConsent` 单独控制。
|
|
142
189
|
3. **前台证明**:打字、按键、坐标点击、滚动必须先证明目标窗口真的拿到了前台,
|
|
143
190
|
否则抛 `focus_failed` 并告诉你当前是谁占着前台。SendInput 会打到有焦点的窗口,
|
|
144
191
|
这一步省不得。
|
|
@@ -171,17 +218,22 @@ cd "E:\path\to\dsh-computer-use"
|
|
|
171
218
|
approval: sensitive # never | sensitive | always
|
|
172
219
|
allowApps: ["notepad", "chrome"] # 空 = 不限制
|
|
173
220
|
denyApps: []
|
|
174
|
-
defaultIncludeScreenshot:
|
|
221
|
+
defaultIncludeScreenshot: true # 默认抓图;false 可显式关闭
|
|
175
222
|
maxElements: 260
|
|
176
223
|
snapshotTtlMs: 120000
|
|
177
224
|
dryRun: false
|
|
178
225
|
toolPrefix: '' # 填 'cu_' 可避免与其他插件的 click 重名
|
|
179
|
-
timeoutMs:
|
|
226
|
+
timeoutMs: 10000
|
|
180
227
|
```
|
|
181
228
|
|
|
182
|
-
##
|
|
229
|
+
## 与 Codex Windows window2 的关系
|
|
183
230
|
|
|
184
|
-
|
|
231
|
+
工具以 `@oai/sky` 的 Windows window2 动作和观测语义为参照,底层也采用 UIA、SendInput、WGC。
|
|
232
|
+
它是 DSH 的独立实现,不能据此认为具有相同的模型能力、成功率或全部运行时行为。
|
|
233
|
+
|
|
234
|
+
DSH 接口有这些明确差异:`list_windows` / `list_apps` 分别返回 `{ windows, count }` /
|
|
235
|
+
`{ apps, count }`;`window.app` 返回应用元数据对象,输入也接受字符串 app id 或只提供窗口 id;
|
|
236
|
+
`get_window` 接收 `{ window: { id } }`;截图返回 DSH 附件引用和 image block,Codex 使用图片 URL。
|
|
185
237
|
|
|
186
238
|
| 契约项 | Codex | 本插件 |
|
|
187
239
|
|---|---|---|
|
|
@@ -194,22 +246,22 @@ cd "E:\path\to\dsh-computer-use"
|
|
|
194
246
|
| 元素失效 | 编号只对产生它的观测有效 | 一致(TTL + helper 进程内缓存) |
|
|
195
247
|
| 最小化窗口 | 不该给出无意义画面 | 本插件直接报 `window_minimized`(实测旧版会吐出 334×48 的假图) |
|
|
196
248
|
| 键名 | X keysym + `Numpad_*` 别名 | 一致(`KP_0`/`Numpad_Add`/…) |
|
|
197
|
-
| `list_apps` | `lastUsedDate`、`useCount` | `lastUsedDate`
|
|
249
|
+
| `list_apps` | `lastUsedDate`、`useCount` | `lastUsedDate` 用进程启动时间近似,并非用户最近使用时间;`useCount` 留空 |
|
|
198
250
|
| `launch_app` | app id / `.exe` 路径(含 UWP) | exe 路径、AUMID(`shell:AppsFolder\\…`)、shell URI |
|
|
199
251
|
| Win/Meta 键 | 硬拒绝 | 硬拒绝(插件层 + helper 层双保险) |
|
|
200
252
|
| 终端/Run/密码管理器/安全软件/认证对话框/宿主自身 | 硬拒绝 | 硬拒绝(不可被审批覆盖) |
|
|
201
253
|
| 按应用授权 | "Computer Use was not approved to use <app>" | 每会话每应用一次审批(`appConsent: session`),批准后下发 `approve_app`,**由 helper 强制**:未批准的应用直接 `app_not_approved` |
|
|
202
|
-
| 回合边界 |
|
|
254
|
+
| 回合边界 | 回合与 helper 生命周期关联 | DSH 回合结束事件主动停止旧 helper;新回合创建新 helper,并重新下发已获应用授权 |
|
|
203
255
|
| 请求预算 | 普通 10s、`launch_app` 15s | 一致(`timeoutMs` / `launchTimeoutMs`) |
|
|
204
|
-
| 回合结束停止输入 | `turn_ended` 钩子 |
|
|
256
|
+
| 回合结束停止输入 | `turn_ended` 钩子 | 主动停止正在执行的请求,排队中的旧回合请求也被拒绝 |
|
|
205
257
|
|
|
206
|
-
##
|
|
258
|
+
## 能力边界
|
|
207
259
|
|
|
208
260
|
- **`paste` / `select_text`**:属于 macOS 的 window 客户端(`target: "mac"`),Windows 上不存在。
|
|
209
|
-
-
|
|
261
|
+
- **系统音频录制**(`start_audio_recording`):已实现 WASAPI 环回录音,默认关闭,需要构建 audio helper 并单独授权。
|
|
210
262
|
- **full-desktop 面**(`clipboard_read/write`、`key_down/up`、`move`、`drag_handle`、`get_screenshot`):那是 `target: "linux"` 的 API。
|
|
211
|
-
-
|
|
212
|
-
- **Windows.Graphics.Capture
|
|
263
|
+
- **区域裁剪截图**:当前抓整窗(长边封顶 2560),不能认为与 Codex 的 bounded screenshot 完全等价。
|
|
264
|
+
- **Windows.Graphics.Capture**:构建 capture helper 后优先使用 WGC;不可用时回退到 PrintWindow / BitBlt,GPU 窗口的兼容性取决于实际使用的抓屏后端。
|
|
213
265
|
|
|
214
266
|
## 抓屏链路(对齐 Codex 的 WGC)
|
|
215
267
|
|
|
@@ -273,7 +325,7 @@ MIT。第三方商用前请自行做安全评审:这个插件能让模型操
|
|
|
273
325
|
|
|
274
326
|
| 能力 | 实现 |
|
|
275
327
|
|---|---|
|
|
276
|
-
| 协议握手 | helper 提供 `hello` → `{ protocolVersion:
|
|
328
|
+
| 协议握手 | helper 提供 `hello` → `{ protocolVersion: 2, capabilities: [...] }`;bridge 首次连接校验,不匹配直接报 `protocol_mismatch` |
|
|
277
329
|
| 每请求预算 | 客户端把预算随请求下发(`__budgetMs`,对应 Codex 的 `x-oai-cua-request-budget-ms`),helper 用它收缩抓屏等待 |
|
|
278
330
|
| 坐标越界 | `point_outside_screenshot`,文案同 Codex("is outside screenshot bounds … Take a fresh screenshot before retrying") |
|
|
279
331
|
| 截图失效 | `stale_screenshot` 带同一句 "Take a fresh screenshot before retrying" |
|
|
@@ -313,6 +365,12 @@ marker 文件按 **会话 + turn** 命名,避免旧会话的 Esc 标记误伤
|
|
|
313
365
|
- `redactApps` 在**插件层**执行:直接调用 helper 的 RPC 不受它约束,别把 helper 单独暴露给非受信调用方。
|
|
314
366
|
- 抢焦点类工具(如 ProcessLasso)会让 `type_text` 返回 `focus_failed`,这是设计行为——宁可不输入,也不打到别的窗口。
|
|
315
367
|
|
|
368
|
+
## 插件更新
|
|
369
|
+
|
|
370
|
+
需要 DSH 0.2.1-rc.3 或更新版本。每个插件在自己的页面或设置项提供检查、更新记录与确认安装入口,开关独立保存。自动检查默认开启;**自动安装默认关闭**,只有明确授权后,兼容的稳定补丁才会在空闲且没有未保存修改时安装。可单独关闭、推迟或忽略提醒;不会强制重启 DSH、升级 DSH 本体或替换本地源码安装。Suite 安装的组件会明确提示通过 Suite 整包更新。
|
|
371
|
+
|
|
372
|
+
旧版用户先在官方插件管理中手动更新一次,之后才会获得此入口。发布到 npm 不会直接替换用户已经安装的版本。
|
|
373
|
+
|
|
316
374
|
## License
|
|
317
375
|
|
|
318
376
|
MIT
|
|
Binary file
|
package/lib/bridge.js
CHANGED
|
@@ -1,22 +1,14 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Lifecycle and framing for the native helper process.
|
|
3
|
-
*
|
|
4
|
-
* One long-lived helper owns the UI Automation element cache, so element
|
|
5
|
-
* indexes returned by get_window_state stay resolvable by click/set_value.
|
|
6
|
-
* Requests are serialized: UI Automation calls are not reentrant and a
|
|
7
|
-
* desynchronized element cache is worse than waiting a few milliseconds.
|
|
8
|
-
*
|
|
9
|
-
* @module deepseek-computer-use/bridge
|
|
10
|
-
*/
|
|
1
|
+
/** Serialized native RPC, with cancellation that waits for the owned process to stop. */
|
|
11
2
|
import { spawn } from 'node:child_process';
|
|
3
|
+
import { randomUUID } from 'node:crypto';
|
|
4
|
+
import fs from 'node:fs';
|
|
5
|
+
import os from 'node:os';
|
|
6
|
+
import path from 'node:path';
|
|
7
|
+
import { setTimeout as delay } from 'node:timers/promises';
|
|
12
8
|
|
|
13
|
-
|
|
14
|
-
const MIN_PROTOCOL = 1;
|
|
15
|
-
|
|
16
|
-
/** Codex's wording, reused verbatim so the model reacts the same way. */
|
|
9
|
+
const MIN_PROTOCOL = 2;
|
|
17
10
|
const ESC_MESSAGE = 'Computer Use was stopped by the user with the physical Escape key. Stop your work, do not call further Computer Use tools in this turn, and send a final message noting that the user stopped Computer Use.';
|
|
18
11
|
|
|
19
|
-
/** An error raised by the helper, carrying its machine-readable code. */
|
|
20
12
|
export class HelperError extends Error {
|
|
21
13
|
constructor(code, message, details) {
|
|
22
14
|
super(message);
|
|
@@ -28,197 +20,208 @@ export class HelperError extends Error {
|
|
|
28
20
|
|
|
29
21
|
export class HelperBridge {
|
|
30
22
|
#child = null;
|
|
23
|
+
#closed = new WeakMap();
|
|
24
|
+
#termination = new WeakMap();
|
|
25
|
+
#reasons = new WeakMap();
|
|
31
26
|
#buffer = '';
|
|
32
27
|
#pending = new Map();
|
|
33
28
|
#seq = 0;
|
|
34
29
|
#queue = Promise.resolve();
|
|
30
|
+
#stopping = Promise.resolve();
|
|
35
31
|
#exe;
|
|
32
|
+
#args;
|
|
33
|
+
#cancelDirectory;
|
|
36
34
|
#timeoutMs;
|
|
37
35
|
#protocol = null;
|
|
38
|
-
#
|
|
36
|
+
#stoppedTurns = new Map();
|
|
39
37
|
#lastTurn = null;
|
|
40
38
|
#disposed = false;
|
|
41
39
|
#log;
|
|
42
40
|
|
|
43
|
-
constructor({ exe, timeoutMs = 20000, log } = {}) {
|
|
41
|
+
constructor({ exe, args = [], cancelDirectory, timeoutMs = 20000, log } = {}) {
|
|
44
42
|
if (typeof exe !== 'string' || exe.length === 0) throw new Error('HelperBridge requires an executable path');
|
|
45
43
|
this.#exe = exe;
|
|
44
|
+
this.#args = args;
|
|
45
|
+
this.#cancelDirectory = cancelDirectory ?? path.join(os.tmpdir(), 'dsh-computer-use-requests');
|
|
46
46
|
this.#timeoutMs = timeoutMs;
|
|
47
47
|
this.#log = log;
|
|
48
48
|
}
|
|
49
49
|
|
|
50
|
-
get running() {
|
|
51
|
-
|
|
52
|
-
}
|
|
50
|
+
get running() { return this.#child !== null && this.#child.exitCode === null && this.#child.signalCode === null; }
|
|
51
|
+
get pid() { return this.running ? this.#child.pid : null; }
|
|
53
52
|
|
|
54
53
|
#ensureChild() {
|
|
55
54
|
if (this.#disposed) throw new HelperError('disposed', 'the Computer Use bridge was disposed');
|
|
56
55
|
if (this.running) return this.#child;
|
|
57
|
-
const child = spawn(this.#exe,
|
|
56
|
+
const child = spawn(this.#exe, this.#args, { stdio: ['pipe', 'pipe', 'pipe'], windowsHide: true });
|
|
57
|
+
this.#child = child;
|
|
58
|
+
this.#buffer = '';
|
|
59
|
+
this.#protocol = null;
|
|
60
|
+
this.#closed.set(child, new Promise(resolve => child.once('close', resolve)));
|
|
58
61
|
child.stdout.setEncoding('utf8');
|
|
59
62
|
child.stderr.setEncoding('utf8');
|
|
60
|
-
child.stdout.on('data', chunk => this.#onData(chunk));
|
|
61
|
-
child.stderr.on('data', chunk =>
|
|
62
|
-
|
|
63
|
-
|
|
63
|
+
child.stdout.on('data', chunk => { if (this.#child === child) this.#onData(child, chunk); });
|
|
64
|
+
child.stderr.on('data', chunk => this.#log?.('[dsh-computer-use] helper: ' + String(chunk).trim().slice(0, 500)));
|
|
65
|
+
child.stdin.on('error', error => this.#stop(child, new HelperError('write_failed', error.message)));
|
|
66
|
+
child.on('error', error => {
|
|
67
|
+
this.#reasons.set(child, new HelperError('helper_start_failed', error.message));
|
|
64
68
|
});
|
|
65
|
-
child.on('
|
|
66
|
-
|
|
67
|
-
this.#
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
: 'the Computer Use helper exited (code ' + code + ', signal ' + signal + ')';
|
|
75
|
-
for (const [, entry] of this.#pending) {
|
|
76
|
-
clearTimeout(entry.timer);
|
|
77
|
-
entry.reject(new HelperError(interrupted ? 'cancelled_by_user' : 'helper_exited', reason));
|
|
69
|
+
child.on('close', (code, signal) => {
|
|
70
|
+
const escaped = code === 130;
|
|
71
|
+
if (escaped && this.#lastTurn !== null) this.#stoppedTurns.set(this.#lastTurn, new HelperError('turn_stopped', ESC_MESSAGE));
|
|
72
|
+
const error = this.#reasons.get(child) ?? new HelperError(escaped ? 'cancelled_by_user' : 'helper_exited',
|
|
73
|
+
escaped ? ESC_MESSAGE : 'the Computer Use helper exited (code ' + code + ', signal ' + signal + ')');
|
|
74
|
+
if (this.#child === child) {
|
|
75
|
+
this.#child = null;
|
|
76
|
+
this.#protocol = null;
|
|
77
|
+
this.#buffer = '';
|
|
78
78
|
}
|
|
79
|
-
this.#
|
|
79
|
+
this.#rejectChild(child, error);
|
|
80
80
|
});
|
|
81
|
-
child.on('error', error => {
|
|
82
|
-
this.#log?.('[dsh-computer-use] helper spawn error: ' + error.message);
|
|
83
|
-
});
|
|
84
|
-
this.#child = child;
|
|
85
81
|
return child;
|
|
86
82
|
}
|
|
87
83
|
|
|
88
|
-
#
|
|
89
|
-
this.#
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
84
|
+
#rejectChild(child, error) {
|
|
85
|
+
for (const [id, entry] of this.#pending) {
|
|
86
|
+
if (entry.child !== child) continue;
|
|
87
|
+
this.#pending.delete(id);
|
|
88
|
+
entry.cleanup();
|
|
89
|
+
entry.reject(error);
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
#stop(child, error) {
|
|
94
|
+
const stopped = this.#terminate(child, error);
|
|
95
|
+
stopped.catch(failure => this.#rejectChild(child, failure));
|
|
96
|
+
return stopped;
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
#terminate(child, error) {
|
|
100
|
+
if (!child) return this.#stopping;
|
|
101
|
+
if (this.#termination.has(child)) return this.#termination.get(child);
|
|
102
|
+
this.#reasons.set(child, error);
|
|
103
|
+
for (const entry of this.#pending.values()) {
|
|
104
|
+
if (entry.child !== child) continue;
|
|
105
|
+
entry.stopping = true;
|
|
106
|
+
clearTimeout(entry.timer);
|
|
107
|
+
try { fs.writeFileSync(entry.cancelFile, error.code, 'utf8'); } catch { /* forced stop below remains available */ }
|
|
96
108
|
}
|
|
109
|
+
const stop = (async () => {
|
|
110
|
+
const closed = this.#closed.get(child);
|
|
111
|
+
// Native input loops poll the marker and release held input in finally.
|
|
112
|
+
// The close frame is processed once that request unwinds.
|
|
113
|
+
try { child.stdin.write(JSON.stringify({ id: ++this.#seq, method: 'close', params: {} }) + '\n'); } catch { /* process already exited */ }
|
|
114
|
+
let ended = await Promise.race([closed.then(() => true), delay(500).then(() => false)]);
|
|
115
|
+
if (!ended) {
|
|
116
|
+
child.kill('SIGKILL');
|
|
117
|
+
ended = await Promise.race([closed.then(() => true), delay(2000).then(() => false)]);
|
|
118
|
+
}
|
|
119
|
+
if (!ended) {
|
|
120
|
+
this.#disposed = true;
|
|
121
|
+
throw new HelperError('helper_stop_failed', 'the Computer Use helper could not be stopped; no further Computer Use requests are allowed');
|
|
122
|
+
}
|
|
123
|
+
})();
|
|
124
|
+
this.#termination.set(child, stop);
|
|
125
|
+
this.#stopping = stop;
|
|
126
|
+
return stop;
|
|
97
127
|
}
|
|
98
128
|
|
|
99
|
-
#
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
this.#
|
|
105
|
-
|
|
129
|
+
#onData(child, chunk) {
|
|
130
|
+
this.#buffer += chunk;
|
|
131
|
+
let newline;
|
|
132
|
+
while ((newline = this.#buffer.indexOf('\n')) >= 0) {
|
|
133
|
+
const line = this.#buffer.slice(0, newline).trim();
|
|
134
|
+
this.#buffer = this.#buffer.slice(newline + 1);
|
|
135
|
+
if (!line) continue;
|
|
136
|
+
let payload;
|
|
137
|
+
try { payload = JSON.parse(line); }
|
|
138
|
+
catch {
|
|
139
|
+
this.#stop(child, new HelperError('protocol_error', 'the Computer Use helper returned invalid JSON; take a fresh observation before retrying'));
|
|
140
|
+
return;
|
|
141
|
+
}
|
|
142
|
+
const entry = this.#pending.get(payload?.id);
|
|
143
|
+
if (!entry || entry.child !== child || entry.stopping) continue;
|
|
144
|
+
if (typeof payload.ok !== 'boolean') {
|
|
145
|
+
this.#stop(child, new HelperError('protocol_error', 'the Computer Use helper returned an invalid response'));
|
|
146
|
+
return;
|
|
147
|
+
}
|
|
148
|
+
const code = payload.error?.code;
|
|
149
|
+
if (!payload.ok && ['timeout', 'cancelled', 'cancelled_by_user', 'turn_stopped'].includes(code)) {
|
|
150
|
+
if (['cancelled_by_user', 'turn_stopped'].includes(code) && entry.turnToken !== undefined) {
|
|
151
|
+
this.#stoppedTurns.set(entry.turnToken, new HelperError('turn_stopped', ESC_MESSAGE));
|
|
152
|
+
}
|
|
153
|
+
this.#stop(child, new HelperError(code, payload.error.message));
|
|
154
|
+
continue;
|
|
155
|
+
}
|
|
156
|
+
this.#pending.delete(payload.id);
|
|
157
|
+
entry.cleanup();
|
|
158
|
+
if (payload.ok) entry.resolve(payload.result);
|
|
159
|
+
else entry.reject(new HelperError(code ?? 'helper_error', payload.error?.message ?? 'the helper request failed', payload.error));
|
|
106
160
|
}
|
|
107
|
-
const entry = this.#pending.get(payload.id);
|
|
108
|
-
if (entry === undefined) return;
|
|
109
|
-
this.#pending.delete(payload.id);
|
|
110
|
-
clearTimeout(entry.timer);
|
|
111
|
-
if (payload.ok === true) entry.resolve(payload.result);
|
|
112
|
-
else entry.reject(new HelperError(payload.error?.code ?? 'helper_error', payload.error?.message ?? 'the helper reported an unknown failure', payload.error));
|
|
113
161
|
}
|
|
114
162
|
|
|
115
|
-
/**
|
|
116
|
-
* Send one request. Calls are serialized through the queue; a caller that
|
|
117
|
-
* never gets an answer fails on the timeout rather than hanging the turn.
|
|
118
|
-
*/
|
|
119
163
|
request(method, params = {}, { signal, timeoutMs, turnToken } = {}) {
|
|
120
|
-
const wire = turnToken === undefined ? params : { ...params, turnToken };
|
|
121
|
-
if (turnToken !== undefined) this.#lastTurn = turnToken;
|
|
122
164
|
const run = async () => {
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
if (turnToken !== undefined && this.#
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
this.#interruptedTurn = turnToken;
|
|
165
|
+
await this.#stopping;
|
|
166
|
+
if (this.#disposed) throw new HelperError('disposed', 'the Computer Use bridge was disposed');
|
|
167
|
+
if (signal?.aborted) throw new HelperError('cancelled', 'the request was cancelled before it started');
|
|
168
|
+
if (turnToken !== undefined && this.#stoppedTurns.has(turnToken)) throw this.#stoppedTurns.get(turnToken);
|
|
169
|
+
if (turnToken !== undefined) this.#lastTurn = turnToken;
|
|
170
|
+
if (this.#protocol === null) {
|
|
171
|
+
const hello = await this.#call('hello', {}, { signal, timeoutMs: 5000, turnToken });
|
|
172
|
+
if (!Number.isInteger(hello?.protocolVersion) || hello.protocolVersion < MIN_PROTOCOL) {
|
|
173
|
+
const error = new HelperError('protocol_mismatch', 'the Computer Use helper must support cancellation protocol ' + MIN_PROTOCOL + '; rebuild the plugin helper');
|
|
174
|
+
await this.#terminate(this.#child, error);
|
|
175
|
+
throw error;
|
|
135
176
|
}
|
|
136
|
-
|
|
177
|
+
this.#protocol = hello.protocolVersion;
|
|
137
178
|
}
|
|
179
|
+
if (turnToken !== undefined && this.#stoppedTurns.has(turnToken)) throw this.#stoppedTurns.get(turnToken);
|
|
180
|
+
return this.#call(method, params, { signal, timeoutMs, turnToken });
|
|
138
181
|
};
|
|
139
182
|
const next = this.#queue.then(run, run);
|
|
140
|
-
this.#queue = next.
|
|
183
|
+
this.#queue = next.catch(() => {});
|
|
141
184
|
return next;
|
|
142
185
|
}
|
|
143
186
|
|
|
144
|
-
|
|
145
|
-
* The helper reports `protocolVersion` from `hello`. Codex exposes the same fact as
|
|
146
|
-
* `window_protocol_version`; checking it up front turns a silent mismatch into one
|
|
147
|
-
* clear error instead of a stream of malformed-stream failures.
|
|
148
|
-
*/
|
|
149
|
-
async #ensureProtocol(signal) {
|
|
150
|
-
if (this.#protocol !== null) return;
|
|
151
|
-
const hello = await this.#call('hello', {}, { signal, timeoutMs: 5000 });
|
|
152
|
-
const version = Number(hello?.protocolVersion);
|
|
153
|
-
if (!Number.isFinite(version) || version < MIN_PROTOCOL) {
|
|
154
|
-
throw new HelperError('protocol_mismatch',
|
|
155
|
-
'the Computer Use helper speaks protocol ' + (hello?.protocolVersion ?? 'unknown') +
|
|
156
|
-
', this plugin needs ' + MIN_PROTOCOL + ' or newer. Reinstall deepseek-computer-use so the helper and the plugin match.');
|
|
157
|
-
}
|
|
158
|
-
this.#protocol = version;
|
|
159
|
-
}
|
|
160
|
-
|
|
161
|
-
#call(method, params, { signal, timeoutMs }) {
|
|
187
|
+
#call(method, params, { signal, timeoutMs, turnToken }) {
|
|
162
188
|
if (signal?.aborted) return Promise.reject(new HelperError('cancelled', 'the request was cancelled before it started'));
|
|
163
189
|
const child = this.#ensureChild();
|
|
190
|
+
fs.mkdirSync(this.#cancelDirectory, { recursive: true });
|
|
191
|
+
const cancelFile = path.join(this.#cancelDirectory, randomUUID() + '.flag');
|
|
164
192
|
const id = ++this.#seq;
|
|
165
|
-
const
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
const raw = new Promise((resolve, reject) => {
|
|
171
|
-
const budget = Number.isFinite(timeoutMs) ? timeoutMs : this.#timeoutMs;
|
|
172
|
-
const timer = setTimeout(() => {
|
|
173
|
-
this.#pending.delete(id);
|
|
174
|
-
reject(new HelperError('timeout', 'the Computer Use helper did not answer ' + method + ' within ' + budget + 'ms'));
|
|
175
|
-
}, budget);
|
|
176
|
-
timer.unref?.();
|
|
177
|
-
|
|
193
|
+
const budget = timeoutMs ?? this.#timeoutMs;
|
|
194
|
+
return new Promise((resolve, reject) => {
|
|
195
|
+
const timer = setTimeout(() => this.#stop(child, new HelperError('timeout',
|
|
196
|
+
'the Computer Use helper exceeded ' + budget + 'ms for ' + method + '; take a fresh observation before retrying')), budget);
|
|
178
197
|
const onAbort = () => {
|
|
179
|
-
this.#
|
|
198
|
+
if (turnToken !== undefined) this.#stoppedTurns.set(turnToken, new HelperError('turn_stopped', 'Computer Use was cancelled in this turn; wait for a new turn'));
|
|
199
|
+
this.#stop(child, new HelperError('cancelled', 'the Computer Use request was cancelled and its helper stopped'));
|
|
200
|
+
};
|
|
201
|
+
const cleanup = () => {
|
|
180
202
|
clearTimeout(timer);
|
|
181
|
-
|
|
203
|
+
signal?.removeEventListener('abort', onAbort);
|
|
204
|
+
try { fs.unlinkSync(cancelFile); } catch { /* no marker was needed */ }
|
|
182
205
|
};
|
|
206
|
+
this.#pending.set(id, { child, cancelFile, timer, cleanup, resolve, reject, turnToken, stopping: false });
|
|
183
207
|
signal?.addEventListener('abort', onAbort, { once: true });
|
|
184
|
-
|
|
185
|
-
this.#pending.set(id, {
|
|
186
|
-
timer,
|
|
187
|
-
resolve: value => {
|
|
188
|
-
signal?.removeEventListener('abort', onAbort);
|
|
189
|
-
resolve(value);
|
|
190
|
-
},
|
|
191
|
-
reject: error => {
|
|
192
|
-
signal?.removeEventListener('abort', onAbort);
|
|
193
|
-
reject(error);
|
|
194
|
-
},
|
|
195
|
-
});
|
|
196
|
-
|
|
208
|
+
if (signal?.aborted) { onAbort(); return; }
|
|
197
209
|
try {
|
|
198
|
-
child.stdin.write(
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
reject(new HelperError('write_failed', 'cannot talk to the Computer Use helper: ' + error.message));
|
|
203
|
-
}
|
|
210
|
+
child.stdin.write(JSON.stringify({ id, method, params: { ...params, ...(turnToken === undefined ? {} : { turnToken }), __budgetMs: budget, __cancelFile: cancelFile } }) + '\n', error => {
|
|
211
|
+
if (error) this.#stop(child, new HelperError('write_failed', error.message));
|
|
212
|
+
});
|
|
213
|
+
} catch (error) { this.#stop(child, new HelperError('write_failed', error.message)); }
|
|
204
214
|
});
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
215
|
+
}
|
|
216
|
+
|
|
217
|
+
stopTurn(turnToken) {
|
|
218
|
+
const error = new HelperError('turn_ended', 'the requesting turn has ended; wait for a new turn and take a fresh observation');
|
|
219
|
+
this.#stoppedTurns.set(turnToken, error);
|
|
220
|
+
return this.#terminate(this.#child, error);
|
|
209
221
|
}
|
|
210
222
|
|
|
211
223
|
dispose() {
|
|
212
224
|
this.#disposed = true;
|
|
213
|
-
|
|
214
|
-
this.#child = null;
|
|
215
|
-
if (child !== null) {
|
|
216
|
-
// Codex closes the helper with a close RPC before tearing the process down.
|
|
217
|
-
try {
|
|
218
|
-
child.stdin.write(JSON.stringify({ id: ++this.#seq, method: 'close', params: {} }) + '\n');
|
|
219
|
-
} catch { /* already gone */ }
|
|
220
|
-
try { child.stdin.end(); } catch { /* already gone */ }
|
|
221
|
-
setTimeout(() => { try { child.kill(); } catch { /* already gone */ } }, 150).unref?.();
|
|
222
|
-
}
|
|
225
|
+
return this.#terminate(this.#child, new HelperError('disposed', 'the Computer Use bridge was disposed'));
|
|
223
226
|
}
|
|
224
227
|
}
|
package/lib/build-helper.mjs
CHANGED
|
@@ -9,6 +9,7 @@
|
|
|
9
9
|
* @module deepseek-computer-use/build-helper
|
|
10
10
|
*/
|
|
11
11
|
import { execFileSync } from 'node:child_process';
|
|
12
|
+
import { createHash } from 'node:crypto';
|
|
12
13
|
import fs from 'node:fs';
|
|
13
14
|
import path from 'node:path';
|
|
14
15
|
import { fileURLToPath } from 'node:url';
|
|
@@ -76,11 +77,24 @@ export function ensureHelper({ force = false, exe = HELPER_EXE, log } = {}) {
|
|
|
76
77
|
}
|
|
77
78
|
|
|
78
79
|
fs.mkdirSync(path.dirname(exe), { recursive: true });
|
|
79
|
-
const args = ['/nologo', '/target:exe', '/platform:anycpu', '/out:' + exe, ...SOURCES, ...refs];
|
|
80
80
|
log?.('[dsh-computer-use] compiling helper with ' + compiler);
|
|
81
|
-
const
|
|
82
|
-
|
|
83
|
-
|
|
81
|
+
const compile = output => execFileSync(compiler, ['/nologo', '/target:exe', '/platform:anycpu', '/out:' + output, ...SOURCES, ...refs], { encoding: 'utf8', windowsHide: true });
|
|
82
|
+
try {
|
|
83
|
+
compile(exe);
|
|
84
|
+
if (!fs.existsSync(exe)) throw new Error('helper compilation produced no executable');
|
|
85
|
+
return { exe, built: true };
|
|
86
|
+
} catch (error) {
|
|
87
|
+
// Windows locks a running exe. Compile alongside it during an update rather
|
|
88
|
+
// than stopping a helper owned by another DSH session.
|
|
89
|
+
if (!fs.existsSync(exe) || !String(error.stdout ?? '').includes('CS0016')) throw error;
|
|
90
|
+
const hash = createHash('sha256');
|
|
91
|
+
for (const source of SOURCES) hash.update(fs.readFileSync(source));
|
|
92
|
+
const cached = path.join(path.dirname(exe), path.basename(exe, '.exe') + '-' + hash.digest('hex').slice(0, 16) + '.exe');
|
|
93
|
+
if (isHelperFresh(cached)) return { exe: cached, built: false };
|
|
94
|
+
log?.('[dsh-computer-use] helper is in use; compiling alongside it');
|
|
95
|
+
compile(cached);
|
|
96
|
+
return { exe: cached, built: true };
|
|
97
|
+
}
|
|
84
98
|
}
|
|
85
99
|
|
|
86
100
|
/** The in-use indicator (breathing frame + virtual cursor) is a separate win-exe. */
|
package/lib/build.mjs
CHANGED
|
@@ -6,13 +6,13 @@
|
|
|
6
6
|
* capture Windows.Graphics.Capture (needs .NET SDK)
|
|
7
7
|
* audio WASAPI loopback recorder (needs .NET SDK)
|
|
8
8
|
*/
|
|
9
|
-
import { ensureHelper, ensureOverlay,
|
|
9
|
+
import { ensureHelper, ensureOverlay, OVERLAY_EXE } from './build-helper.mjs';
|
|
10
10
|
import { buildNative } from './build-native.mjs';
|
|
11
11
|
|
|
12
12
|
const log = message => console.log(message);
|
|
13
13
|
|
|
14
14
|
const helper = ensureHelper({ force: process.argv.includes('--force'), log });
|
|
15
|
-
log('[dsh-computer-use] helper ' + (helper.built ? 'built' : 'up to date') + ' -> ' +
|
|
15
|
+
log('[dsh-computer-use] helper ' + (helper.built ? 'built' : 'up to date') + ' -> ' + helper.exe);
|
|
16
16
|
|
|
17
17
|
const overlay = ensureOverlay({ force: process.argv.includes('--force'), log });
|
|
18
18
|
log('[dsh-computer-use] overlay ' + (overlay.built ? 'built' : 'up to date') + ' -> ' + OVERLAY_EXE);
|