dsh-gui-control 0.0.0-stage → 0.1.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 dsh-gui-control contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,210 @@
-# Temporary Holding Version
+<div align="center">
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+<img src="https://raw.githubusercontent.com/seetown/dsh-gui-control/main/assets/icon-180.png" width="88" alt="dsh-gui-control">
+
+# dsh-gui-control
+
+**给 DeepSeek Harness 的 Windows 桌面操控：不是「帮我点一下」,而是「以后这个活都交给你」。**
+
+<p>
+  <a href="https://www.npmjs.com/package/dsh-gui-control"><img alt="npm" src="https://img.shields.io/npm/v/dsh-gui-control.svg?color=cb3837"></a>
+  <a href="https://www.npmjs.com/package/dsh-gui-control"><img alt="downloads" src="https://img.shields.io/npm/dm/dsh-gui-control.svg"></a>
+  <img alt="platform" src="https://img.shields.io/badge/platform-Windows-0078d4.svg">
+  <img alt="license" src="https://img.shields.io/badge/license-MIT-blue.svg">
+  <a href="https://github.com/topics/dsh-plugin"><img alt="dsh-plugin" src="https://img.shields.io/badge/topic-dsh--plugin-6f42c1.svg"></a>
+</p>
+
+[中文说明](#中文说明) · [English](#english)
+
+</div>
+
+---
+
+## 中文说明
+
+### 它和别的桌面 Agent 有什么不同
+
+大多数桌面自动化解决的是**一次性操作**：看一眼屏幕、点一下、再看一眼。这个插件多做一层——**把一套动作固化成任务文件**，跑的时候模型完全不参与。
+
+| | 常见做法 | dsh-gui-control |
+|---|---|---|
+| 感知 | 截图 → 模型猜坐标 | **无障碍树优先**，按控件名定位；树不可用时才截图 |
+| 一次任务的往返 | 每步一次模型调用 | 一次调用跑完整个任务文件 |
+| 失败之后 | 不知道点没点上 | **投递判定**：没发出去 / 可能发出去了 / 已验证 |
+| 重复执行 | 每次重跑一遍 | 任务文件可复用，带条件等待与重试 |
+
+### 演示
+
+**① 打开 B 站 → 搜番剧 → 播到指定集数**
+
+![B 站选集](https://raw.githubusercontent.com/seetown/dsh-gui-control/main/demo/4-bilibili.gif)
+
+**② 语义点击：鼠标位移 0 像素**（直接调用控件的 Invoke，鼠标不动、窗口在后台也能点）
+
+![语义点击](https://raw.githubusercontent.com/seetown/dsh-gui-control/main/demo/3-calc.gif)
+
+**③ 填一列数据**（17 步 / 0.9 秒 / 一次调用）
+
+![填表](https://raw.githubusercontent.com/seetown/dsh-gui-control/main/demo/2-sheet.gif)
+
+### 安装
+
+```bash
+dsh plugin add dsh-gui-control
+```
+
+或从源码：
+
+```bash
+dsh plugin add seetown/dsh-gui-control
+```
+
+要求：Windows 10 1809+、PowerShell 5.1、DeepSeek Harness 0.1.5-rc.2 ~ 0.2.x。装完不需要重启。
+
+### 三个工具
+
+| 工具 | 作用 |
+|---|---|
+| `gui_snapshot` | 读窗口的无障碍树（文本，精确、便宜、与位置无关），或带屏幕坐标刻度尺的截图 |
+| `gui_act` | 执行一个动作，带身份校验和投递判定 |
+| `gui_task` | 跑声明式任务文件：一次调用完成多步，带条件、重试、风险闸门 |
+
+```
+gui_snapshot { window: "Notepad", mode: "tree" }
+→ Butto 关闭标签页 / Docum 文本编辑器 / MenuI 文件 ...
+
+gui_act { action: "click_text", window: "Notepad", text: "最大化" }
+→ click ok via InvokePattern | verified: text appeared
+```
+
+**优先用 `click_text` 而不是坐标**：它按控件名定位，鼠标不动，窗口布局变了也不会点错。
+
+### 任务文件
+
+```json
+{
+  "name": "fill-sheet",
+  "steps": [
+    { "do": "activate", "win": "工作簿1" },
+    { "do": "type", "text": "项目" },
+    { "do": "key", "key": "tab" },
+    { "do": "type", "text": "金额" },
+    { "do": "key", "key": "enter",
+      "until": { "op": "pathexists", "text": "C:\\out\\report.xlsx" }, "untilTimeout": 10 }
+  ]
+}
+```
+
+每步字段：`do` + 该动作参数，外加 `until`（轮询到条件成立才算过）、`untilTimeout`、`retry`、`risk`。
+
+- `risk: "high"` 的步骤默认**拒绝执行**，必须显式 `confirm: true`
+- `dryRun: true` 只返回计划，不碰桌面
+
+### 投递语义
+
+每个动作返回三选一：
+
+| 判定 | 含义 | 能否重试 |
+|---|---|---|
+| `not_delivered` | 什么都没发出去（派发前就被拒） | ✅ 安全 |
+| `delivered_unverified` | 发出去了，效果未确认 | ❌ 不能盲目重放 |
+| `delivered_verified` | 发出去了，且后置条件验证通过 | ❌ |
+
+这一条决定了「发消息 / 删文件 / 提交表单」这类不可逆操作是否安全。
+
+### 安全
+
+- 只监听 `127.0.0.1:38471`，不对外暴露
+- 密码字段一律输出 `<redacted>`
+- 高风险步骤需要显式确认
+- 结果不明时**绝不自动重试**
+
+> 这个插件能操控整台机器的鼠标键盘。装之前请知情，只在可信环境使用。
+
+### 实测数据
+
+| 操作 | 耗时 |
+|---|---|
+| 单步动作（常驻进程） | 10.8 ms |
+| 语义点击（UIA 树） | 26–34 ms |
+| 紧凑树读取 | 86 ms，865 字符（比完整 dump 省 88%） |
+| 7 步任务（含启动计算器） | 680 ms，一次调用 |
+| 桌面建 3 个文件夹 | 3.45 s，3/3 成功 |
+| WPS 填 2×3 表格 | 0.9 s，17 步 |
+
+首次调用会预热常驻进程（约 1 秒），之后每个动作是一次 loopback 往返。
+
+### 已知限制
+
+- **仅 Windows**（Win32 + UI Automation）
+- 需要 PowerShell 5.1
+- 画布类应用（WPS 表格、游戏、自绘列表）没有无障碍树，只能截图 + OCR，准确率下降
+- 网页任务目前基于截图；CDP 通道已实现但尚未接入工具
+
+### Roadmap
+
+- [ ] 截图自动编号（Set-of-Marks），把坐标估计降级成编号分类
+- [ ] 接入 CDP：网页任务按选择器操作，而不是按坐标
+- [ ] 任务库与意图路由：一句话命中现成任务
+- [ ] macOS / Linux 适配
+
+---
+
+## English
+
+**Windows desktop control for DeepSeek Harness.** Not "click that button for me" — "this job is yours from now on."
+
+Most desktop agents do one-off operations: look, click, look again. This one adds a layer underneath: **it turns a sequence into a task file** that runs with no model in the loop.
+
+### What makes it different
+
+| | Typical approach | dsh-gui-control |
+|---|---|---|
+| Perception | Screenshot, then the model guesses coordinates | **Accessibility tree first**, addressed by control name; screenshots only when no tree exists |
+| Round trips per task | One model call per step | One call runs the whole task file |
+| After a failure | Unknown whether it landed | **Delivery verdict**: not delivered / may have landed / verified |
+| Repeating a flow | Replay it by hand | Task files are reusable, with conditions and retries |
+
+### Install
+
+```bash
+dsh plugin add dsh-gui-control
+```
+
+Windows 10 1809+, PowerShell 5.1, DeepSeek Harness 0.1.5-rc.2 – 0.2.x. No restart needed.
+
+### Three tools
+
+| Tool | Purpose |
+|---|---|
+| `gui_snapshot` | Read a window's accessibility tree as text, or a screenshot with a screen-coordinate ruler |
+| `gui_act` | One action, with an identity check and a delivery verdict |
+| `gui_task` | Run a declarative task file: many steps in one call, with conditions, retries and a risk gate |
+
+### Delivery semantics
+
+| Verdict | Meaning | Retry |
+|---|---|---|
+| `not_delivered` | Nothing was sent | ✅ safe |
+| `delivered_unverified` | Input may have landed, effect unconfirmed | ❌ never blind-retry |
+| `delivered_verified` | Landed, and the post-condition held | ❌ |
+
+That distinction is what makes "send a message / delete a file / submit a form" safe to automate.
+
+### Security
+
+Loopback-only (`127.0.0.1:38471`), password fields are redacted, high-risk steps require explicit confirmation, and an uncertain outcome is never retried automatically.
+
+> This plugin drives the real mouse and keyboard of the whole machine. Install it knowingly, in a trusted environment.
+
+### Limitations
+
+Windows only; PowerShell 5.1 required; canvas-like apps (spreadsheets, games, self-drawn lists) expose no accessibility tree and fall back to screenshot + OCR with lower accuracy; web tasks currently use screenshots (the CDP path exists but is not wired into the tools yet).
+
+---
+
+<div align="center">
+
+MIT © [seetown](https://github.com/seetown)
+
+</div>

package/cordis.patch.yml

@@ -0,0 +1,4 @@
+- insert:
+    - id: gui-control
+      name: dsh-gui-control
+      config: {}

package/icon.svg

@@ -0,0 +1,8 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" width="64" height="64" role="img" aria-label="Desktop control">
+  <rect x="4" y="8" width="56" height="40" rx="5" fill="none" stroke="#3b82f6" stroke-width="4"/>
+  <line x1="4" y1="18" x2="60" y2="18" stroke="#3b82f6" stroke-width="3"/>
+  <circle cx="11" cy="13" r="2.2" fill="#3b82f6"/>
+  <circle cx="18" cy="13" r="2.2" fill="#3b82f6"/>
+  <path d="M30 28 L30 50 L36 44 L40 53 L45 51 L41 42 L49 42 Z" fill="#f59e0b" stroke="#0f172a" stroke-width="2" stroke-linejoin="round"/>
+  <line x1="22" y1="56" x2="42" y2="56" stroke="#3b82f6" stroke-width="4" stroke-linecap="round"/>
+</svg>

package/lib/index.js

@@ -0,0 +1,346 @@
+/**
+ * dsh-gui-control — Windows desktop control for DeepSeek Harness.
+ *
+ * The heavy lifting lives in a long-lived PowerShell daemon shipped under lib/scripts.
+ * It compiles the Win32/UIA helpers once and then serves operations over loopback TCP,
+ * so a click costs ~10 ms instead of the ~700 ms of starting PowerShell and recompiling.
+ *
+ * This module only exposes three model-facing tools and keeps the surface small on
+ * purpose: read, act, or run a declarative task.
+ */
+import { execFileSync } from 'node:child_process';
+import { existsSync } from 'node:fs';
+import net from 'node:net';
+import { tmpdir } from 'node:os';
+import { dirname, join } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { defineTool } from '@deepseek-ai/dsh-tools';
+
+const HERE = dirname(fileURLToPath(import.meta.url));
+const SCRIPTS = join(HERE, 'scripts');
+const PORT = 38471;
+const HOST = '127.0.0.1';
+
+const isWindows = process.platform === 'win32';
+
+/** Resolve PowerShell explicitly: the host process PATH is not guaranteed to include it. */
+const POWERSHELL = (() => {
+	const systemRoot = process.env.SystemRoot ?? process.env.windir ?? 'C:\\Windows';
+	const full = join(systemRoot, 'System32', 'WindowsPowerShell', 'v1.0', 'powershell.exe');
+	return existsSync(full) ? full : 'powershell.exe';
+})();
+
+let launchPromise = null;
+
+function text(value) {
+	return [{ type: 'text', text: typeof value === 'string' ? value : JSON.stringify(value) }];
+}
+
+function assertWindows() {
+	if (!isWindows) {
+		throw new Error('dsh-gui-control only works on Windows; the native helpers are Win32 and UI Automation based.');
+	}
+}
+
+function pingDaemon(timeoutMs = 400) {
+	return new Promise((resolve) => {
+		const socket = net.createConnection({ host: HOST, port: PORT });
+		const done = (ok) => {
+			socket.removeAllListeners();
+			socket.destroy();
+			resolve(ok);
+		};
+		socket.setTimeout(timeoutMs);
+		socket.once('connect', () => done(true));
+		socket.once('timeout', () => done(false));
+		socket.once('error', () => done(false));
+	});
+}
+
+/** Start the daemon once per process; concurrent callers share one launch. */
+function ensureDaemon() {
+	if (launchPromise !== null) return launchPromise;
+	launchPromise = (async () => {
+		if (await pingDaemon()) return;
+		// Launch through Start-Process: a bare detached spawn of PowerShell can exit
+		// immediately on Windows depending on console inheritance from the host.
+		const daemon = join(SCRIPTS, 'gui-daemon.ps1');
+		const quote = (value) => `'${String(value).replace(/'/g, "''")}'`;
+		const args = ['-NoProfile', '-STA', '-ExecutionPolicy', 'Bypass', '-WindowStyle', 'Hidden', '-File', daemon, '-Port', String(PORT)];
+		const command = `Start-Process -FilePath ${quote(POWERSHELL)} -ArgumentList @(${args.map(quote).join(',')}) -WindowStyle Hidden`;
+		try {
+			execFileSync(POWERSHELL, ['-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command', command], { stdio: 'ignore', timeout: 15000, windowsHide: true });
+		} catch (error) {
+			launchPromise = null;
+			throw new Error(`Could not launch the GUI daemon: ${error.message}`);
+		}
+		for (let i = 0; i < 120; i += 1) {
+			await new Promise((r) => setTimeout(r, 100));
+			if (await pingDaemon()) return;
+		}
+		launchPromise = null;
+		throw new Error('The GUI daemon did not start. Check that PowerShell 5.1 is available and see lib/scripts/gui-daemon.log.');
+	})();
+	return launchPromise;
+}
+
+/** One request, one response line. Resolves with the daemon envelope. */
+function callDaemon(op, timeoutMs = 20000) {
+	return new Promise((resolve, reject) => {
+		const socket = net.createConnection({ host: HOST, port: PORT });
+		let buffer = '';
+		let settled = false;
+		const finish = (fn, value) => {
+			if (settled) return;
+			settled = true;
+			socket.destroy();
+			fn(value);
+		};
+		socket.setEncoding('utf8');
+		socket.setTimeout(timeoutMs);
+		socket.once('connect', () => socket.write(`${JSON.stringify(op)}\n`));
+		socket.on('data', (chunk) => {
+			buffer += chunk;
+			const nl = buffer.indexOf('\n');
+			if (nl === -1) return;
+			const line = buffer.slice(0, nl);
+			try {
+				finish(resolve, JSON.parse(line));
+			} catch (error) {
+				finish(reject, new Error(`Bad daemon response: ${line.slice(0, 200)}`));
+			}
+		});
+		socket.once('timeout', () => finish(reject, new Error(`GUI operation timed out after ${timeoutMs} ms (${op.op}).`)));
+		socket.once('error', (error) => finish(reject, error));
+	});
+}
+
+/** Call, and if the daemon died since the last call, relaunch it once and retry. */
+async function callResilient(op, timeoutMs) {
+	await ensureDaemon();
+	try {
+		return await callDaemon(op, timeoutMs);
+	} catch (error) {
+		const code = error?.code;
+		if (code !== 'ECONNREFUSED' && code !== 'EPIPE' && code !== 'ECONNRESET') throw error;
+		launchPromise = null;
+		await ensureDaemon();
+		return callDaemon(op, timeoutMs);
+	}
+}
+
+async function run(op, timeoutMs) {
+	const envelope = await callResilient(op, timeoutMs);
+	if (envelope.ok !== true) {
+		const parts = [String(envelope.error ?? 'operation failed')];
+		if (envelope.delivery !== undefined) parts.push(`delivery=${envelope.delivery}`);
+		if (envelope.retry !== undefined) parts.push(`retry=${envelope.retry}`);
+		throw new Error(parts.join('  '));
+	}
+	return envelope.result;
+}
+
+/** Compact tree read: one line per control, deduped, no rectangles. */
+function formatTree(lines) {
+	if (!Array.isArray(lines) || lines.length === 0) return '(no controls)';
+	return lines.join('\n');
+}
+
+function actOp(args) {
+	const action = String(args.action ?? '').toLowerCase();
+	const base = {};
+	if (typeof args.window === 'string' && args.window.length > 0) base.win = args.window;
+	if (Number.isFinite(args.x) && Number.isFinite(args.y)) {
+		base.x = Math.round(args.x);
+		base.y = Math.round(args.y);
+	}
+	switch (action) {
+		case 'click':
+		case 'double_click':
+		case 'right_click': {
+			const out = { op: 'click', ...base };
+			if (action === 'double_click') out.count = 2;
+			if (action === 'right_click') out.button = 'right';
+			return out;
+		}
+		case 'type':
+			return { op: 'type', ...base, text: String(args.text ?? '') };
+		case 'key':
+			return { op: 'key', ...base, key: String(args.key ?? '') };
+		case 'hotkey':
+			return { op: 'hotkey', ...base, key: String(args.key ?? '') };
+		case 'scroll':
+			return { op: 'scroll', ...base, amount: Number(args.amount ?? -3) };
+		case 'focus':
+			return { op: 'activate', win: String(args.window ?? '') };
+		case 'click_text': {
+			// Semantic click: locate the control by name instead of by coordinate.
+			return { op: 'uia-click', win: String(args.window ?? ''), text: String(args.text ?? ''), type: args.controlType };
+		}
+		default:
+			throw new Error(`Unknown action "${action}". Use click, double_click, right_click, type, key, hotkey, scroll, focus, or click_text.`);
+	}
+}
+
+const SNAPSHOT = {
+	name: 'gui_snapshot',
+	description: [
+		'Read a Windows window as text: its accessibility tree (control names, roles, bounds-free) or a cropped screenshot with a coordinate ruler.',
+		'Use the tree first — it is exact, cheap, and position independent. Fall back to a screenshot only when the window publishes no tree (a canvas, a game, or a self-drawn list).',
+		'Always snapshot before acting, and again afterwards to confirm what changed.',
+		'A screenshot is returned as a file path; read that file with the image reader. The ruler labels are screen coordinates, so you can pass them straight back to gui_act.',
+	].join(' '),
+	parameters: {
+		window: { type: 'string', description: 'Window title or process-name substring, case-insensitive (for example "Notepad" or "计算器").' },
+		mode: { type: 'string', enum: ['tree', 'shot'], description: 'tree (default): control names as text. shot: PNG with a screen-coordinate ruler.' },
+		lines: { type: 'number', description: 'Maximum control lines to return (default 40, tree mode only).' },
+		chars: { type: 'number', description: 'Truncate each control name to this many characters (default 60, tree mode only).' },
+		region: { type: 'string', description: "Screenshot crop in screen pixels as 'x,y,width,height' (shot mode). Defaults to the whole window." },
+	},
+	output: {
+		schema: { type: 'string' },
+		render: (_args, value) => text(value),
+	},
+	isConcurrencySafe: () => true,
+	async execute(args) {
+		assertWindows();
+		const window = typeof args.window === 'string' ? args.window : '';
+		if (window.length === 0) throw new Error('gui_snapshot needs a window (title or process substring).');
+		const mode = args.mode === 'shot' ? 'shot' : 'tree';
+		if (mode === 'tree') {
+			const lines = await run({ op: 'uia-mini', win: window, lines: args.lines ?? 40, chars: args.chars ?? 60 });
+			return formatTree(lines);
+		}
+		const op = { op: 'shot', win: window, rulers: true, path: join(tmpdir(), `gui-shot-${Date.now()}.png`) };
+		if (typeof args.region === 'string' && args.region.length > 0) {
+			const nums = args.region.split(',').map((v) => Number(v.trim()));
+			if (nums.length === 4 && nums.every((n) => Number.isFinite(n))) {
+				[op.x, op.y, op.w, op.h] = nums;
+			}
+		}
+		const shot = await run(op);
+		return `shot ${shot.w}x${shot.h} at ${shot.x},${shot.y} scale ${shot.scale}\n${shot.path}\nRead the file to see it. Ruler labels are screen coordinates.`;
+	},
+	presentCall(args) {
+		return { card: 'generic', title: `Snapshot ${args.window ?? 'window'}`, kind: 'read' };
+	},
+};
+
+const ACT = {
+	name: 'gui_act',
+	description: [
+		'Perform ONE desktop action and optionally verify its effect.',
+		'Prefer click_text over click with coordinates: it addresses the control by name, so the mouse does not move and a window layout change cannot make it hit the wrong thing.',
+		'Every action reports a delivery verdict. "not_delivered" means nothing was sent and a retry is safe; "delivered_unverified" means the input may have landed, so repeating it risks a duplicate — verify first, never blind-retry.',
+		'For more than two steps use gui_task instead: one call, conditions, and a single failure report.',
+	].join(' '),
+	parameters: {
+		action: {
+			type: 'string',
+			enum: ['click', 'double_click', 'right_click', 'type', 'key', 'hotkey', 'scroll', 'focus', 'click_text'],
+			description: 'click_text = click the control whose name matches text (recommended). key/hotkey take a key name such as enter or ctrl+s. focus brings a window to the front.',
+		},
+		window: { type: 'string', description: 'Target window (title or process substring). Required for click_text, focus, click, type, and key.' },
+		text: { type: 'string', description: 'Text to type, or the control name for click_text.' },
+		x: { type: 'number', description: 'Screen x for click/double_click/right_click (use only when no control name exists).' },
+		y: { type: 'number', description: 'Screen y for click/double_click/right_click.' },
+		key: { type: 'string', description: "Key or chord: 'enter', 'tab', 'esc', 'f5', 'ctrl+s', 'alt+tab'." },
+		amount: { type: 'number', description: 'Scroll notches; negative scrolls down (default -3).' },
+		controlType: { type: 'string', description: 'Optional control type filter for click_text, for example Button or TabItem.' },
+		verifyText: { type: 'string', description: 'After acting, poll until this text appears in the window (proves the action landed).' },
+		verifyTimeout: { type: 'number', description: 'Seconds to poll for verifyText (default 4).' },
+	},
+	output: {
+		schema: { type: 'string' },
+		render: (_args, value) => text(value),
+	},
+	isConcurrencySafe: () => false,
+	async execute(args) {
+		assertWindows();
+		const op = actOp(args);
+		const envelope = await callResilient(op);
+		if (envelope.ok !== true) {
+			const parts = [String(envelope.error ?? 'action failed')];
+			if (envelope.delivery !== undefined) parts.push(`delivery=${envelope.delivery}`);
+			if (envelope.retry !== undefined) parts.push(`retry=${envelope.retry}`);
+			throw new Error(parts.join('  '));
+		}
+		let verified = 'unverified';
+		if (typeof args.verifyText === 'string' && args.verifyText.length > 0) {
+			verified = await verifyText(args.window, args.verifyText, (args.verifyTimeout ?? 4) * 1000);
+		}
+		const how = envelope.result?.how !== undefined ? ` via ${envelope.result.how}` : '';
+		return `${op.op} ok${how}${verified === 'ok' ? ' | verified: text appeared' : ''}${verified === 'timeout' ? ' | delivery=delivered_unverified: text never appeared, do NOT blind-retry' : ''}`;
+	},
+	presentCall(args) {
+		return { card: 'generic', title: `${args.action ?? 'act'} ${args.window ?? ''}`.trim(), kind: 'execute' };
+	},
+};
+
+async function verifyText(window, needle, timeoutMs) {
+	const deadline = Date.now() + timeoutMs;
+	while (Date.now() < deadline) {
+		try {
+			const envelope = await callResilient({ op: 'uia-get', win: String(window ?? ''), text: needle }, 5000);
+			if (envelope.ok === true) return 'ok';
+		} catch {
+			// keep polling
+		}
+		await new Promise((r) => setTimeout(r, 200));
+	}
+	return 'timeout';
+}
+
+const TASK = {
+	name: 'gui_task',
+	description: [
+		'Run a declarative task file: an ordered list of desktop steps executed inside the daemon, with no model in the loop.',
+		'Each step names an operation plus its parameters and may add "until" (a condition polled until it holds), "untilTimeout", "retry", and "risk".',
+		'A task returns per-step timing and a delivery verdict, so one call replaces a long back-and-forth of single actions.',
+		'Pass dryRun to see the plan without touching the desktop. Steps marked risk "high" are refused unless confirm is true.',
+		'Example step: { "do": "uia-click", "win": "Calculator", "text": "7", "until": { "op": "uia-get", "win": "Calculator", "text": "Display is 7" } }',
+	].join(' '),
+	parameters: {
+		file: { type: 'string', description: 'Absolute path to the task JSON file.' },
+		dryRun: { type: 'boolean', description: 'Return the step plan without executing anything.' },
+		confirm: { type: 'boolean', description: 'Allow steps marked risk high (otherwise they are refused before dispatch).' },
+	},
+	output: {
+		schema: { type: 'string' },
+		render: (_args, value) => text(value),
+	},
+	isConcurrencySafe: () => false,
+	async execute(args) {
+		assertWindows();
+		const file = typeof args.file === 'string' ? args.file : '';
+		if (file.length === 0) throw new Error('gui_task needs file (absolute path to a task JSON).');
+		const op = { op: 'task', file, dryRun: args.dryRun === true, confirm: args.confirm === true };
+		const envelope = await callResilient(op, 600000);
+		const result = envelope.result ?? {};
+		if (result.dryRun === true) {
+			const plan = (result.steps ?? []).map((step) => `${String(step.n).padStart(2)}. ${String(step.do).padEnd(10)} risk=${step.risk ?? 'low'}`);
+			return [`task ${result.name ?? ''} plan (dry run, nothing executed)`, ...plan].join('\n');
+		}
+		const lines = [];
+		for (const step of result.steps ?? []) {
+			const flag = step.ok ? 'ok  ' : 'FAIL';
+			const extra = step.note ? ` ${step.note}` : '';
+			lines.push(`${String(step.n).padStart(2)}. ${flag} ${String(step.do).padEnd(10)} ${String(step.ms).padStart(6)} ms  ${step.delivery ?? ''}${extra}`.trimEnd());
+		}
+		const head = envelope.ok === true
+			? `task ${result.name ?? ''} ok in ${result.total ?? '?'} ms`
+			: `task failed: ${envelope.error}${envelope.delivery ? `  delivery=${envelope.delivery}` : ''}${envelope.retry ? `  retry=${envelope.retry}` : ''}`;
+		return [head, ...lines].join('\n');
+	},
+	presentCall(args) {
+		return { card: 'generic', title: `Run task ${String(args.file ?? '').split(/[\\/]/).pop() ?? ''}`, kind: 'execute' };
+	},
+};
+
+export const inject = ['tools'];
+
+export function apply(ctx) {
+	ctx.effect(() => ctx.tools.register(defineTool(SNAPSHOT)));
+	ctx.effect(() => ctx.tools.register(defineTool(ACT)));
+	ctx.effect(() => ctx.tools.register(defineTool(TASK)));
+}