dsh-codex-style-computer-use 1.2.1 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,5 +1,9 @@
1
1
  # dsh-codex-style-computer-use · Codex-style desktop control
2
2
 
3
+ **Give DeepSeek eyes and hands for Windows.**
4
+
5
+ Ask your agent to work with desktop apps: see the screen, find controls, click and type. Take over with your own mouse or keyboard whenever needed; the agent waits before continuing. Approval prompts remain enabled by default.
6
+
3
7
  Windows desktop computer use for **DeepSeek Harness**: 18 desktop tools plus an Agent activation entry that observe and drive
4
8
  native Windows applications through prebuilt C# workers included in the package. No PowerShell 7,
5
9
  SDK or install-time builds are needed.
@@ -8,8 +12,16 @@ English | [中文](README.zh.md)
8
12
 
9
13
  Windows only (`os: win32`). Requires Node.js ≥ 22 and the .NET Framework 4.x that Windows ships with.
10
14
 
15
+ **1.3.0 desktop lifecycle update:** the deep-blue cue now matches actual actuation, with a 16-pixel pointer jitter allowance and a visible 1.5-second purple notice before input. Human takeover returns a required continuation checkpoint. Session/agent ownership and a native cross-process lease allow only one controller. Call `computer_ctrl {action:"release"}` after desktop work and verification; the conversation continues and later desktop work can open a fresh cycle in the same turn. Normal completion releases automatically without an exit question card. Explicit pauses, exits and pending questions remain authoritative.
16
+
17
+ The indicators use continuous 2-pixel rails with square screen corners and an 8-pixel glow. Normal working states have no floating badges; handoff, pause and preparation use compact top hints. Content interiors remain transparent. The visual revision preserves state logic and animation timing.
18
+
19
+ **1.2.3 automatic fix:** normal installation/update now installs the approval-attribution correction for computer-use calls on affected DSH hosts. No manual host patch is required for those calls. Host approvals and audit remain authoritative; follow the host's restart indication after updating an already loaded plugin. See [1.2.3](docs/RELEASE-1.2.3.md).
20
+
11
21
  **1.2.1 attention fix:** fixes a missing crypto import that prevented 1.2.0's native `probe` and `raise` requests from reaching the helper. Updating from 1.2.0 is required for this runtime fix; reinstalling 1.2.0 does not fix it. New releases are staged on npm's `next` tag before promotion to `latest`; the market's normal update follows `latest`. See [1.2.1 validation and release status](docs/RELEASE-1.2.1.md).
12
22
 
23
+ **1.2.2 GitHub pre-release:** includes a separately deployed DSH host approval-attribution patch and 14 regression cases. Plugin input and approval behavior is retained from 1.2.1. See [release contents and deployment scope](docs/RELEASE-1.2.2.md); GitHub publication does not update npm or the market automatically.
24
+
13
25
  ## What it does
14
26
 
15
27
  - **Observe** — window list, screenshots (region / JPEG / PNG / downscaled), a Set-of-Marks map of
@@ -31,7 +43,7 @@ under `$DSH_HOME/data/computer-use/`.
31
43
 
32
44
  Once the catalog submission is merged and synced, search **computer-use** and select **dsh-codex-style-computer-use**, by **ltmroberthk915** (npm maintainer: **ltmroberthk**). Click **Install**. Later, use **Update** or **Update all** for versions admitted by the host's release policy. The package includes both native helpers; no PowerShell 7 or build permission is needed.
33
45
 
34
- Before catalog sync, use **Settings → Plugins → Add plugin** and enter `dsh-codex-style-computer-use`. The default non-strict pnpm 11.7.0 configuration installs this published package immediately; an explicitly strict host policy can require waiting as described below. A newly added bundle can load live on the official Desktop host; check that its tools and skill appear. Restart from the tray when replacing an already loaded version, when the client remains stale, or when DSH reports **restart-required**.
46
+ Before catalog sync, use **Settings → Plugins → Add plugin**. The corrected version is **`dsh-codex-style-computer-use@1.2.1`**; the unversioned name still resolves to 1.2.0 until promotion. Do not paste the GitHub repository URL: that selects the source-download path instead of the small prebuilt npm package. The default non-strict pnpm 11.7.0 configuration can install 1.2.1 immediately; an explicitly strict host policy can require waiting as described below. A newly added bundle can load live on the official Desktop host; check that its tools and skill appear. Restart from the tray when replacing an already loaded version, when the client remains stale, or when DSH reports **restart-required**.
35
47
 
36
48
  ### Releases less than 24 hours old
37
49
 
@@ -113,6 +125,8 @@ clicked at the wrong place.
113
125
 
114
126
  ## Approval modes
115
127
 
128
+ For a host that reports a disabled-prompt automatic refusal as “the user rejected tool”, see the [DSH approval attribution patch and regression test](docs/APPROVAL-ATTRIBUTION.md).
129
+
116
130
  `automationMode` is set in the plugin's settings and defaults to `standard`:
117
131
 
118
132
  | Mode | Observation | Actuation |
@@ -127,7 +141,7 @@ fields are detected through UI Automation's `IsPassword` and refused by default.
127
141
 
128
142
  ## Safety model
129
143
 
130
- - **Shared input** — physical typing, scrolling, mouse movement and mouse buttons immediately yield control. Continued input or a held key/button for 2 seconds enters automatic waiting. After all keys/buttons are released and 3 seconds pass without physical input, the agent takes a fresh observation and continues the authorized task. A brief touch also waits for the 3-second quiet interval. A gentle pale-yellow edge gradient shows human ownership.
144
+ - **Shared input** — ordinary user activity is allowed while the agent is thinking, observing or preparing. During actual actuation, typing, scrolling and buttons yield immediately; pointer movement has a 16-pixel displacement tolerance. Continued input or a held key/button for 2 seconds enters automatic waiting. After release and 3 quiet seconds, the agent must inspect the current target and continue the original task, without blindly replaying partial writes. A gentle pale-yellow edge gradient shows human ownership.
131
145
  - **Manual pause** — only physical `Ctrl+Esc` raises a persistent input brake; `Ctrl+Alt+R` resumes. Ordinary Esc, typing, scrolling, pointer distance/frequency and the screen corner do not create this pause. Explicit question/diagnostic holds remain separate.
132
146
  - **Exit** — `Ctrl+Alt+Q` ends the session: the overlay goes away, its listeners are torn down, and
133
147
  further actuation is refused until a later turn opens a new one.
package/README.zh.md CHANGED
@@ -1,5 +1,9 @@
1
1
  # dsh-codex-style-computer-use · Codex 风格桌面操控
2
2
 
3
+ **让 DeepSeek 看得见,也点得动。**
4
+
5
+ 把 Windows 应用里的重复操作交给 AI:看屏幕、找控件、点鼠标、敲键盘。AI 实际操控前提示 1.5 秒,非操控期间你可以正常使用电脑;操控中接管后,AI 等待交回并接续原任务。默认保留操作审批。
6
+
3
7
  给 **DeepSeek Harness** 用的 Windows 桌面操控插件:18 个桌面工具加一个 Agent 激活入口,通过一个 C# worker
4
8
  观察并驱动原生 Windows 应用。发布包已包含编译好的原生组件,无需安装 PowerShell 7、SDK 或执行编译命令。
5
9
 
@@ -7,8 +11,18 @@
7
11
 
8
12
  仅支持 Windows(`os: win32`)。需要 Node.js ≥ 22 与 Windows 自带的 .NET Framework 4.x。
9
13
 
14
+ **1.3.0 桌面控制周期升级:**实际操控与深蓝色指示同步;鼠标轻微抖动有 16 像素容差;紫色操作前提示持续 1.5 秒;接管后返回必须继续任务的断点;同一时间只有一个 agent/worker 拥有控制周期。桌面工作和验证结束后调用 `computer_ctrl {action:"release"}` 即可释放,聊天继续,同一轮后来仍可重新开始。正常收尾自动释放,不再弹出结束确认选项单。人工暂停、明确退出和待回答的问题保持有效。
15
+
16
+ 灯效统一为贴合屏幕直角的 2 像素光线和最多 8 像素柔光,四角没有圆弧缺口。普通工作状态不显示悬浮文字牌;让出、暂停和预告使用紧凑提示,正文区域保持透明。此次美工调整不改变状态逻辑或闪烁节奏。
17
+
18
+ **1.2.3 自动修复:**正常安装或更新插件,即会为受影响宿主中的 computer-use 调用安装审批归因修正,无需手工修改宿主。审批决定与审计仍由宿主处理;更新已加载插件后按宿主提示重启。详见 [1.2.3](docs/RELEASE-1.2.3.md)。
19
+
10
20
  **1.2.1 提醒修复:**补齐 crypto 导入,修复 1.2.0 原生 `probe` / `raise` 请求还未发送就抛错的问题。这次是运行时代码修复,需要从 1.2.0 更新;重复安装 1.2.0 无效。新版本先发布到 npm 的 `next`,再提升为 `latest`,市场普通更新跟随 `latest`。详见 [1.2.1 验证和发布状态](docs/RELEASE-1.2.1.md)。
11
21
 
22
+ 宿主在禁用审批弹窗时仍提示“用户拒绝工具”的问题,详见 [DSH 审批归因补丁与回归测试](docs/APPROVAL-ATTRIBUTION.md)。
23
+
24
+ **1.2.2 GitHub 预发布:**收录独立部署的宿主审批归因补丁和 14 项回归测试。插件输入与审批行为延续 1.2.1,详见 [发布内容及部署范围](docs/RELEASE-1.2.2.md)。GitHub 发布不会自动更新 npm 或市场版本。
25
+
12
26
  ## 它能做什么
13
27
 
14
28
  - **观察** —— 窗口列表、截图(区域 / JPEG / PNG / 降采样)、可点元素的 Set-of-Marks 编号图,
@@ -27,7 +41,7 @@
27
41
 
28
42
  市场收录同步后,搜索 **computer-use**,选择 **dsh-codex-style-computer-use**,作者 **ltmroberthk915**(npm 维护者 **ltmroberthk**),点击**安装**。以后通过市场的**更新**或**全部更新**安装符合宿主发布策略的版本。两份原生组件都已随包提供,无需 PowerShell 7、SDK 或构建白名单。
29
43
 
30
- 目录尚未同步时,可在 **设置 → 插件 → 添加插件** 输入 `dsh-codex-style-computer-use`。默认非严格 pnpm 11.7.0 配置可以直接安装这个已发布的包;宿主显式启用严格策略时,才可能需要按下节等待冷却。首次新增 bundle 在官方桌面端可热加载;确认工具与 skill 已出现即可使用。替换已经加载的版本、客户端界面未刷新或界面明确提示 **restart-required / 需要重启** 时,再从托盘完整退出 DSH 并重新打开。
44
+ 目录尚未同步时,在 **设置 → 插件 → 添加插件** 输入修复版 **`dsh-codex-style-computer-use@1.2.1`**。正式标签提升前,省略版本号仍会装到 1.2.0。这里填 npm 包名即可;填写 GitHub 仓库网址会转为下载源码,容易受 GitHub 下载线路影响。默认非严格 pnpm 11.7.0 配置可以直接安装 1.2.1;宿主显式启用严格策略时,才可能需要等待冷却。首次新增 bundle 在官方桌面端可热加载;确认工具与 skill 已出现即可使用。替换已经加载的版本、客户端界面未刷新或界面明确提示 **restart-required / 需要重启** 时,再从托盘完整退出 DSH 并重新打开。
31
45
 
32
46
  ### 发布未满 24 小时
33
47
 
@@ -121,7 +135,9 @@ ID**:派发输入前会核对窗口身份与采样像素,因此过期的 mar
121
135
 
122
136
  ## 安全模型
123
137
 
124
- - **人机共用输入** —— 实体键盘、滚轮、鼠标移动或按钮操作立即让出控制。持续输入或按住键/按钮 2 秒进入自动等待;全部松开并静默 3 秒后,重新观察目标并继续已授权任务。短暂触碰也等待 3 秒静默。人操作时显示温和的淡黄色边缘渐变。
138
+ - **人机共用输入** —— 思考、观察和预告期间允许用户正常操作。实际操控时,键盘、滚轮和按钮立即触发让出;鼠标位移有 16 像素容差。持续输入或按住键/按钮 2 秒进入自动等待;全部松开并静默 3 秒后,agent 必须重新观察并接续原任务,不盲目重放部分写入。人操作时显示温和的淡黄色边缘渐变。
139
+ - **工具用完即释放** —— `computer_ctrl {action:"release"}` 结束电脑控制但保持聊天继续;正常收尾自动释放,不弹确认选项单。暂停和待回答问题不会被自动清除。
140
+ - **一个操作者** —— 会话/agent 所有权与跨 worker 独占锁覆盖整个控制周期,另一 agent 必须等当前操作者释放。
125
141
  - **人工暂停** —— 只有实体 `Ctrl+Esc` 触发持续输入刹车,`Ctrl+Alt+R` 恢复。普通 Esc、打字、滚动、鼠标距离/频率和屏幕左上角不再触发这种暂停;显式提问、诊断暂停保持独立。
126
142
  - **退出** —— `Ctrl+Alt+Q` 结束会话:覆盖层消失、监听全部拆除,后续注入被拒,直到之后某一轮重新开启。
127
143
  - **刹车是关于机器的事实,不属于某一个进程** —— 闩锁状态落盘,每个 worker 都会遵守,包括之后从命令行启动的。
@@ -0,0 +1,39 @@
1
+ // Compatibility with hosts that conflate policy refusals and answerer refusals.
2
+ // Delegate first so the host still owns every approval and durable audit pair.
3
+ export function installApprovalAttribution (ctx, log) {
4
+ const tools = ctx.get('tools')
5
+ const original = tools?.serviceAsk
6
+ if (typeof original !== 'function') {
7
+ log.warn('approval attribution: host compatibility seam unavailable')
8
+ return
9
+ }
10
+ const ownDescriptor = Object.getOwnPropertyDescriptor(tools, 'serviceAsk')
11
+ const wrapped = async function (exec, ask) {
12
+ let policy
13
+ if (exec?.name?.startsWith('computer_') && exec.agent) {
14
+ const approval = this.ctx?.get?.('approval')
15
+ if (typeof approval?.effectivePolicy === 'function') policy = approval.effectivePolicy(exec.agent.session)
16
+ }
17
+ const result = await original.call(this, exec, ask)
18
+ // Newer/fixed hosts already provide their own reason. Only repair the exact
19
+ // known legacy attribution; never reinterpret cancellation or a grant.
20
+ if (!exec?.name?.startsWith('computer_') || result?.decision?.kind !== 'deny' ||
21
+ result.decision.reason !== `the user rejected tool "${exec.name}"`) return result
22
+ const reason = (policy === 'never'
23
+ ? `tool "${exec.name}" requires approval, but approval prompts are disabled in this session (approval policy: never); the request was automatically rejected without asking the user`
24
+ : `approval for tool "${exec.name}" was rejected by the approval channel`) +
25
+ (ask?.reason ? `; approval required because: ${ask.reason}` : '')
26
+ return { ...result, decision: { ...result.decision, reason } }
27
+ }
28
+ ctx.effect(() => {
29
+ tools.serviceAsk = wrapped
30
+ return () => {
31
+ // Cordis returns a traced callable on get(); compare the stored method,
32
+ // not that newly created callable, to identify our own replacement.
33
+ if (Object.getOwnPropertyDescriptor(tools, 'serviceAsk')?.value !== wrapped) return
34
+ if (ownDescriptor) Object.defineProperty(tools, 'serviceAsk', ownDescriptor)
35
+ else delete tools.serviceAsk
36
+ }
37
+ })
38
+ log.info('approval attribution: installed for computer-use; host approvals and audit retained')
39
+ }
package/lib/batch.js CHANGED
@@ -1,4 +1,5 @@
1
1
  // Serial execution and observation selection; no desktop or lifecycle side effects here.
2
+ import { handoffResult } from './continuation.js'
2
3
  const reads = new Set(['computer_state', 'computer_uia', 'computer_marks', 'computer_shot', 'computer_wait'])
3
4
  const isRead = (step) => reads.has(step?.tool) || (step?.tool === 'computer_clip' && step.args?.text === undefined)
4
5
  const isReadback = (step) => ['computer_state', 'computer_uia', 'computer_marks', 'computer_shot'].includes(step.tool) ||
@@ -10,37 +11,52 @@ export function actionRefusal (step, result) {
10
11
  return 'target window activation was not confirmed; subsequent actions were not dispatched'
11
12
  }
12
13
  if (step.tool === 'computer_ctrl' && step.args?.action === 'recover' && result?.recovered !== true) return result.reason || 'temporary pause recovery refused'
14
+ if (step.tool === 'computer_ctrl' && step.args?.action === 'release' && !result?.released && !result?.alreadyReleased) return result.reason || 'cycle release refused'
13
15
  return null
14
16
  }
15
17
 
16
- export async function runBatch ({ steps, handlers, signal, stopped, screenshot, shot = 'auto', exec }) {
18
+ export async function runBatch ({ steps, handlers, signal, stopped, screenshot, shot = 'auto', exec, assertOwnership }) {
17
19
  if (!Array.isArray(steps) || !['auto', 'always', 'never'].includes(shot)) throw new Error('invalid batch actions or shot mode')
18
20
  const actions = []
19
21
  let failed = false
22
+ let continuation = null
20
23
  for (const step of steps) {
21
24
  const name = step?.tool
22
25
  if (failed || signal?.aborted) {
23
- actions.push({ tool: name || '', ok: false, outcome: 'not-dispatched', error: signal?.aborted ? 'cancelled' : 'earlier action failed' })
26
+ actions.push({ tool: name || '', ok: false, outcome: 'not-dispatched',
27
+ ...(continuation && !signal?.aborted
28
+ ? { status: 'deferred', message: 'Resume this unfinished step after fresh target evidence; the original task must continue.' }
29
+ : { error: signal?.aborted ? 'cancelled' : 'earlier action failed' }) })
24
30
  failed = true
25
31
  continue
26
32
  }
27
33
  const fn = Object.hasOwn(handlers, name) && handlers[name]
28
34
  if (!fn) { actions.push({ tool: name || '', ok: false, outcome: 'not-dispatched', error: 'unknown tool' }); failed = true; continue }
29
35
  try {
36
+ assertOwnership?.()
30
37
  const stoppedBefore = stopped()
31
38
  const result = await fn(step.args || {}, exec)
32
39
  const refusal = actionRefusal(step, result)
33
40
  actions.push({ tool: name, ok: !refusal, result, ...(refusal ? { error: refusal } : {}) })
34
41
  failed = !!refusal || (!stoppedBefore && stopped()) ||
35
- (name === 'computer_ctrl' && ['stop', 'exit'].includes(step.args?.action))
42
+ (name === 'computer_ctrl' && ['stop', 'exit', 'release'].includes(step.args?.action))
36
43
  } catch (error) {
37
- actions.push({ tool: name, ok: false, error: String(error.message || error),
38
- code: error.code || 'action-error', outcome: error.outcome || 'unknown' })
44
+ const checkpoint = !signal?.aborted && !stopped() ? handoffResult(error) : null
45
+ if (checkpoint) {
46
+ continuation = { ...checkpoint.continuation, resumeFromActionIndex: actions.length }
47
+ actions.push({ tool: name, ...checkpoint, continuation })
48
+ } else {
49
+ actions.push({ tool: name, ok: false, error: String(error.message || error),
50
+ code: error.code || 'action-error', outcome: error.outcome || 'unknown' })
51
+ }
39
52
  failed = true
40
53
  }
41
54
  }
42
55
  const needsShot = shot === 'always' || (shot === 'auto' && steps.some(s => !isRead(s)) && !isReadback(steps.at(-1) || {}))
43
56
  const result = { actions, attachedShot: null }
57
+ if (continuation) Object.assign(result, {
58
+ status: 'continuation-required', taskStatus: 'continuing', transient: true, continuation,
59
+ })
44
60
  if (!failed && !signal?.aborted && !stopped() && needsShot) {
45
61
  try { result.attachedShot = await screenshot() }
46
62
  catch (error) { result.observationError = String(error.message || error) }
@@ -0,0 +1,14 @@
1
+ // Temporary ownership handoff is a task checkpoint, not a failed/cancelled task.
2
+ export function handoffResult (error) {
3
+ if (error?.code !== 'HUMAN_REOBSERVE' || !error.continuation?.required) return null
4
+ return {
5
+ ok: false,
6
+ status: 'continuation-required',
7
+ taskStatus: 'continuing',
8
+ transient: true,
9
+ code: error.code,
10
+ outcome: error.outcome || 'unknown',
11
+ message: error.message,
12
+ continuation: { ...error.continuation },
13
+ }
14
+ }
@@ -2,6 +2,6 @@
2
2
  "format": 1,
3
3
  "platform": "win32",
4
4
  "architecture": "x64",
5
- "sourceSha256": "7e24c3fa73fed54eb8209efc253753d44664037954d936573e7c80150caf5856",
6
- "binarySha256": "ee890a28d1aec984f98b48caf30acab536477ea4d9261892761ea6e8bd485c46"
5
+ "sourceSha256": "885c7935d5520bc6f865da92e3ab63b29c6515a853e2a3e39ceb2221606acf08",
6
+ "binarySha256": "c7ae5f0f77912674a8f6f310b95809eef10941cf986f8aa7691aed7a7f08bff5"
7
7
  }
package/lib/core/index.js CHANGED
@@ -10,7 +10,7 @@
10
10
  // {"event":"ready"} announcement.
11
11
 
12
12
  import { spawn, execFileSync } from 'node:child_process'
13
- import { createHash } from 'node:crypto'
13
+ import { createHash, randomUUID } from 'node:crypto'
14
14
  import { EventEmitter } from 'node:events'
15
15
  import { AsyncLocalStorage } from 'node:async_hooks'
16
16
  import { existsSync, readFileSync, statSync } from 'node:fs'
@@ -24,6 +24,7 @@ const DEFAULT_TIMEOUT_MS = 15_000
24
24
  const CAPTURE_TIMEOUT_MS = 30_000
25
25
  const INPUT_OPS = new Set(['click', 'shiftClick', 'move', 'drag', 'scroll', 'selectRange',
26
26
  'type', 'key', 'uiaAct', 'windowOp', 'activate', 'clipWrite', 'calibrate', 'selftest'])
27
+ const isInputOp = (op, args = {}) => INPUT_OPS.has(op) || (op === 'focus' && args.refocus === true)
27
28
  // THE CORE'S ASK DEADLINE, AND IT IS A BARE LITERAL ON PURPOSE (merge-design.md §3, §3.2).
28
29
  //
29
30
  // It bounds ONE thing: a wedged worker inside `beginAsk`'s raise sequence (a hung UIA provider), or
@@ -48,6 +49,28 @@ export function workerCallError (message, code, op, requestId, outcome = 'unknow
48
49
  { code, op, requestId, outcome })
49
50
  }
50
51
 
52
+ /** A handoff interrupts an input operation, never the authorized task. */
53
+ export function humanContinuation (error = {}) {
54
+ return {
55
+ required: true,
56
+ taskStatus: 'continuing',
57
+ handback: 'received',
58
+ nextAction: 'observe-current-target-then-continue-original-task',
59
+ replayInterruptedInput: false,
60
+ interruptedOperation: error.op || null,
61
+ requestId: error.requestId ?? null,
62
+ outcome: error.outcome === 'not-dispatched' ? 'not-dispatched' : 'unknown',
63
+ instruction: 'Temporary input interruption only. Control has been handed back. You MUST immediately obtain fresh target evidence, verify any partial effect, and continue the original authorized task from its unfinished step until complete. Do not abandon the task, issue a final wrap-up, or ask for permission to continue because of this handoff. Never blindly replay an interrupted write. Explicit user cancellation, manual pause, or exit still takes precedence.',
64
+ }
65
+ }
66
+
67
+ function humanReobserveError (error = {}) {
68
+ const continuation = humanContinuation(error)
69
+ return Object.assign(workerCallError(continuation.instruction, 'HUMAN_REOBSERVE',
70
+ continuation.interruptedOperation, continuation.requestId, continuation.outcome),
71
+ { transient: true, taskStatus: 'continuing', continuation })
72
+ }
73
+
51
74
  /** Interactive roles that receive Set-of-Marks labels. */
52
75
  const SOM_ROLES = new Set([
53
76
  'Button', 'MenuItem', 'ListItem', 'TreeItem', 'DataItem', 'TabItem',
@@ -220,8 +243,11 @@ export class ComputerUse extends EventEmitter {
220
243
  p.cleanup?.()
221
244
  if (msg.data?.cooperation) this.syncCooperation(msg.data.cooperation)
222
245
  if (msg.ok) p.resolve(msg.data)
223
- else p.reject(workerCallError(msg.error || 'worker error', msg.code || 'worker-refusal-or-error', p.op, msg.id,
224
- msg.outcome === 'not-dispatched' ? 'not-dispatched' : 'unknown'))
246
+ else {
247
+ const error = workerCallError(msg.error || 'worker error', msg.code || 'worker-refusal-or-error', p.op, msg.id,
248
+ msg.outcome === 'not-dispatched' ? 'not-dispatched' : 'unknown')
249
+ p.reject(error.code === 'HUMAN_REOBSERVE' ? humanReobserveError(error) : error)
250
+ }
225
251
  }
226
252
 
227
253
  /**
@@ -231,12 +257,18 @@ export class ComputerUse extends EventEmitter {
231
257
  * @param {boolean} [killOnTimeout] kill a wedged worker when that cap expires (default true)
232
258
  * @returns {Promise<any>} worker `data` payload
233
259
  */
234
- async call (op, args = {}, timeoutMs = DEFAULT_TIMEOUT_MS, killOnTimeout = true) {
235
- const signal = INPUT_OPS.has(op) ? this.inputContext.getStore()?.signal : undefined
260
+ async call (op, args = {}, timeoutMs = DEFAULT_TIMEOUT_MS, killOnTimeout = true, beforeDispatch) {
261
+ const input = isInputOp(op, args)
262
+ const context = input ? this.inputContext.getStore() : undefined
263
+ const signal = context?.signal
236
264
  if (signal?.aborted) throw workerCallError('owning tool cancelled', 'INPUT_CANCELLED', op, null, 'not-dispatched')
265
+ beforeDispatch?.()
266
+ context?.assertOwnership?.()
237
267
  try { await this.ensureWorker() }
238
268
  catch (error) { throw workerCallError(error.message || String(error), 'worker-unavailable', op, null, 'not-dispatched') }
239
269
  if (signal?.aborted) throw workerCallError('owning tool cancelled', 'INPUT_CANCELLED', op, null, 'not-dispatched')
270
+ beforeDispatch?.()
271
+ context?.assertOwnership?.()
240
272
  const id = ++this.seq
241
273
  // `__cycle` is this layer's SIGNATURE on a call made on the agent's behalf, and the worker
242
274
  // opens (or keeps lit) a computer-use CYCLE only for signed calls. A one-shot
@@ -245,6 +277,9 @@ export class ComputerUse extends EventEmitter {
245
277
  // LAST so an op's own args can never forge it.
246
278
  const signed = { ...args, __cycle: true }
247
279
  signed.__kickoff = this.kickoffRequired
280
+ // One warning per external call/batch, with a fresh identity for each context.
281
+ // Never let a caller-supplied argument reuse another tool's announcement.
282
+ if (input) signed.__inputSeries = this.inputContext.getStore()?.inputSeries || randomUUID()
248
283
  return new Promise((resolve, reject) => {
249
284
  let written = false
250
285
  const cleanup = () => signal?.removeEventListener('abort', abort)
@@ -275,7 +310,7 @@ export class ComputerUse extends EventEmitter {
275
310
  : ' — worker left alive so the brake can still be released'),
276
311
  'worker-timeout', op, id))
277
312
  }, timeoutMs)
278
- this.pending.set(id, { resolve, reject, timer, op, cleanup, abort })
313
+ this.pending.set(id, { resolve, reject, timer, op, input, cleanup, abort })
279
314
  signal?.addEventListener('abort', abort, { once: true })
280
315
  if (signal?.aborted) { abort(); return }
281
316
  written = true
@@ -293,12 +328,17 @@ export class ComputerUse extends EventEmitter {
293
328
  // ---------------- screen observation ----------------
294
329
  syncCooperation (state) {
295
330
  if (!state || Number(state.revision) < Number(this.cooperation.revision)) return
331
+ if (Number(state.revision) === Number(this.cooperation.revision) && Number.isFinite(state.inputEpoch) &&
332
+ state.inputEpoch < (this.cooperation.inputEpoch || 0)) return
333
+ const inputChanged = Number.isFinite(state.inputEpoch) && state.inputEpoch !== (this.cooperation.inputEpoch || 0)
296
334
  this.cooperation = state
297
- if (state.active) { this.maps.clear(); this.lastMarks = []; this.lastMarkFrame = null }
335
+ if (state.active || inputChanged) { this.maps.clear(); this.lastMarks = []; this.lastMarkFrame = null }
298
336
  this.emit('handoff', state)
299
337
  }
300
338
 
301
- withInputContext (context, fn) { return this.inputContext.run({ signal: context?.signal }, fn) }
339
+ withInputContext (context, fn) {
340
+ return this.inputContext.run({ signal: context?.signal, assertOwnership: context?.assertOwnership, inputSeries: randomUUID() }, fn)
341
+ }
302
342
 
303
343
  waitForHuman () {
304
344
  const signal = this.inputContext.getStore()?.signal
@@ -324,13 +364,13 @@ export class ComputerUse extends EventEmitter {
324
364
  async waitInputReady () {
325
365
  if (!this.cooperation.active) return
326
366
  await this.waitForHuman()
327
- throw workerCallError('Human input ended and the three-second quiet interval completed. Observe the current target, then continue this task automatically. Previous coordinates/focus may have changed.',
328
- 'HUMAN_REOBSERVE', null, null, 'not-dispatched')
367
+ this.inputContext.getStore()?.assertOwnership?.()
368
+ throw humanReobserveError({ outcome: 'not-dispatched' })
329
369
  }
330
370
 
331
371
  cancelInputs () {
332
372
  this.emit('input-cancel')
333
- for (const pending of [...this.pending.values()]) if (INPUT_OPS.has(pending.op)) pending.abort?.()
373
+ for (const pending of [...this.pending.values()]) if (pending.input || INPUT_OPS.has(pending.op)) pending.abort?.()
334
374
  }
335
375
 
336
376
  /** Raw capture; returns { image: Buffer, mime, region, cursor, settled, settleMs, ts, scaledTo }. */
@@ -463,7 +503,7 @@ export class ComputerUse extends EventEmitter {
463
503
  * emergency brake. Called when the agent's turn ends, so a lingering blue border can never
464
504
  * claim the machine is still under control after control actually stopped.
465
505
  */
466
- async calm () { return this.call('calm', {}) }
506
+ async calm ({ beforeDispatch } = {}) { return this.call('calm', {}, DEFAULT_TIMEOUT_MS, true, beforeDispatch) }
467
507
 
468
508
  /// **Ask the human one question, machine paused — the ONE question path.**
469
509
  ///
@@ -584,7 +624,9 @@ export class ComputerUse extends EventEmitter {
584
624
  * the intended target, whether it had to be re-asserted, and an integrity-level warning when
585
625
  * Windows (UIPI) would silently discard our input. Text-only — no screenshot needed.
586
626
  */
587
- async focus (opts = {}) { return this.call('focus', opts) }
627
+ async focus (opts = {}) {
628
+ return opts.refocus === true ? this.actuate('focus', opts) : this.call('focus', opts)
629
+ }
588
630
 
589
631
  /** Sampled fingerprint plus window identity/geometry; a change invalidates cached marks. */
590
632
  async frameSig (opts = {}) { return this.call('frameSig', opts) }
@@ -763,10 +805,23 @@ export class ComputerUse extends EventEmitter {
763
805
  if (err.code === 'HUMAN_YIELD') {
764
806
  // The hook's event can arrive just after its operation refusal. One cheap local
765
807
  // status read closes that race; waiting spends no model requests and never replays input.
766
- await this.call('ping', {})
767
- await this.waitForHuman()
768
- err.code = 'HUMAN_REOBSERVE'
769
- err.message += ' Human input is now idle for at least three seconds. Inspect the current target and continue automatically; do not repeat any partial write without a readback.'
808
+ try {
809
+ await this.call('ping', {})
810
+ await this.waitForHuman()
811
+ this.inputContext.getStore()?.assertOwnership?.()
812
+ err = humanReobserveError({ ...err, op })
813
+ } catch (waitError) {
814
+ // Cancelling the local wait cannot erase input already accepted before handoff.
815
+ err = workerCallError(waitError.message || String(waitError), waitError.code || 'handoff-wait-failed',
816
+ op, err.requestId ?? null, err.outcome === 'not-dispatched' ? 'not-dispatched' : 'unknown')
817
+ }
818
+ }
819
+ if (err.code === 'HUMAN_REOBSERVE') {
820
+ try { this.inputContext.getStore()?.assertOwnership?.() }
821
+ catch (ownerError) {
822
+ err = workerCallError(ownerError.message, ownerError.code || 'COMPUTER_OWNER_CHANGED', op,
823
+ err.requestId ?? null, err.outcome)
824
+ }
770
825
  }
771
826
  rec.error = String(err.message || err)
772
827
  throw err