dsh-gui-control 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 dsh-gui-control contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,188 @@
-# Temporary Holding Version
+# dsh-gui-control
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+**给 DeepSeek Harness 的 Windows 桌面操控：不是"帮我点一下这个按钮"，而是"以后这个活都交给你"。**
+
+大多数桌面 Agent 解决的是「一次性操作」——看一眼、点一下、再看一眼。这个插件多做一层：**把一套动作固化成任务文件**，跑的时候模型完全不参与，带条件校验、失败分类、以及"这一次到底有没有真的发出去"的判定。
+
+<!-- TODO: 15 秒 GIF：左边敲一句话，右边 WPS 表格被填好 -->
+
+## 演示
+
+**打开 B 站 → 搜索番剧 → 播到指定集数**（模型自己找搜索框、点卡片、跳剧集）
+
+![B 站选集](https://raw.githubusercontent.com/seetown/dsh-gui-control/main/demo/4-bilibili.gif)
+
+**语义点击：鼠标位移 0 像素**（直接调用控件的 Invoke，鼠标不动、窗口在后台也能点）
+
+![语义点击](https://raw.githubusercontent.com/seetown/dsh-gui-control/main/demo/3-calc.gif)
+
+**填一列数据**（17 步 / 0.9 秒 / 一次调用，执行期间模型完全不参与）
+
+![填表](https://raw.githubusercontent.com/seetown/dsh-gui-control/main/demo/2-sheet.gif)
+
+> 把 `seetown` 换成你的 GitHub 用户名，图才会显示。
+
+## 三个工具
+
+| 工具 | 作用 |
+|---|---|
+| `gui_snapshot` | 读窗口的无障碍树（文本，精确、便宜、与位置无关），或带屏幕坐标刻度尺的截图 |
+| `gui_act` | 做一个动作，带**身份校验**和**投递判定** |
+| `gui_task` | 跑一个声明式任务文件：一步调用完成多步，带条件等待、重试、风险闸门 |
+
+## 为什么不是又一个 pyautogui 封装
+
+### 1. 投递语义：知道"有没有真的发出去"
+
+每个动作返回三选一的判定：
+
+| 判定 | 含义 | 能不能重试 |
+|---|---|---|
+| `not_delivered` | 什么都没发出去（派发前就被拒） | ✅ 安全重试 |
+| `delivered_unverified` | 发出去了，但效果没确认 | ❌ **绝对不能盲目重放** |
+| `delivered_verified` | 发出去了，且后置条件验证通过 | ❌ |
+
+这条区分决定了「发送消息 / 删除文件 / 提交表单」这类不可逆操作是否安全。实测：控件不存在时判 `not_delivered/retry=safe`；点了但校验超时判 `delivered_unverified/retry=unsafe`。
+
+### 2. 元素身份，而不是坐标或序号
+
+`uia-pick` 支持 `expect`（名字）和 `expectId`（AutomationId）。页面重排后序号漂移，它会**在派发前拒绝**并回报实际值：
+
+```
+identity mismatch at index 0: expected id "NoSuchAutomationId", found "CloseButton"
+```
+
+### 3. 三通道并存，不赌单一来源
+
+- **UIA**：能读树的窗口，语义点击，鼠标不动、不抢焦点
+- **OCR**：文字密集但无树的界面（本地 Windows.Media.Ocr，不外传图片）
+- **坐标 + 刻度尺截图**：画布、游戏、自绘列表
+
+WPS 表格这类「有窗口但没有单元格节点」的应用，只有后两条路能走。
+
+## 安装
+
+```bash
+# 从本地目录（开发时）
+dsh plugin add /absolute/path/to/dsh-gui-control
+
+# 从 npm
+dsh plugin add dsh-gui-control
+```
+
+装完**无需重启**：三个工具会在当前会话直接出现。
+
+要求：Windows 10 1809+，PowerShell 5.1（系统自带），DeepSeek Harness 0.1.5-rc.2 ~ 0.2.x。
+
+## 用法
+
+```
+gui_snapshot { window: "Notepad", mode: "tree" }
+→ MenuI 文件
+  Butto 关闭标签页
+  Docum 文本编辑器
+  ...
+
+gui_act { action: "click_text", window: "Notepad", text: "最大化" }
+→ click ok via InvokePattern | verified: text appeared
+
+gui_task { file: "C:\\tasks\\fill-sheet.json", dryRun: true }
+→ task fill-sheet ok in 0 ms
+   1. ok   click         412 ms  not_delivered
+   2. ok   type          380 ms  delivered_unverified
+```
+
+### 任务文件
+
+```json
+{
+  "name": "fill-sheet",
+  "steps": [
+    { "do": "activate", "win": "工作簿1" },
+    { "do": "click", "x": 129, "y": 449 },
+    { "do": "type", "text": "项目" },
+    { "do": "key", "key": "tab" },
+    { "do": "type", "text": "金额" },
+    { "do": "key", "key": "enter",
+      "until": { "op": "pathexists", "text": "C:\\out\\report.xlsx" }, "untilTimeout": 10 }
+  ]
+}
+```
+
+步骤字段：`do` + 该动作的参数，外加 `until`（轮询到条件成立才算过）、`untilTimeout`、`retry`、`risk`。
+
+`risk: "high"` 的步骤默认**拒绝执行**，必须显式 `confirm: true`；`dryRun: true` 只看计划不碰桌面。
+
+## 性能实测
+
+| 操作 | 耗时 |
+|---|---|
+| 动作（单步，守护进程） | 10.8 ms |
+| UIA 点击（语义） | 26–34 ms |
+| 紧凑树读取（`uia-mini`） | 86 ms，865 字符（比完整 dump 省 88%） |
+| 7 步任务（含启动计算器） | 680 ms，一次调用 |
+| 桌面建 3 个文件夹 | 3.45 s，3 次全部成功 |
+
+守护进程常驻，首次调用预热约 1 秒；之后每个动作是一次 loopback TCP 往返。
+
+## 安全
+
+- 只监听 `127.0.0.1:38471`，不对外暴露
+- 密码字段读取一律输出 `<redacted>`
+- 高风险步骤需要显式确认
+- 失败区分「没发出去」和「可能发出去了」，后者绝不自动重放
+
+**注意**：这个插件能操控整台机器的鼠标键盘。装之前请知情，只在可信环境使用。
+
+## 已知限制
+
+- **仅 Windows**（Win32 + UI Automation）
+- 需要 PowerShell 5.1；没有它在 PATH 上的发行版会失败
+- 部分应用不暴露无障碍树（游戏、画布、未开无障碍的 Electron），此时只能走截图 + OCR，速度和准确率都会下降
+- 坐标是物理像素，前提是进程声明了 per-monitor DPI 感知（已内置）
+
+## Roadmap
+
+- [ ] 截图自动编号（Set-of-Marks），把坐标估计降级成编号分类
+- [ ] 任务库与意图路由：一句话命中现成任务
+- [ ] macOS / Linux 适配（目前是 Win32 专用）
+
+## License
+
+MIT
+
+---
+
+# English
+
+**Windows desktop control for DeepSeek Harness.** Not "click that button for me" — "this job is yours from now on."
+
+Most desktop agents do one-off operations: look, click, look again. This plugin adds a layer underneath: **turn a sequence into a task file** that runs with no model in the loop, with conditions, failure classification, and a verdict on whether the input actually reached the application.
+
+## Three tools
+
+| Tool | Purpose |
+|---|---|
+| `gui_snapshot` | Read a window's accessibility tree as text, or a screenshot with a screen-coordinate ruler |
+| `gui_act` | One action, with an identity check and a delivery verdict |
+| `gui_task` | Run a declarative task file: many steps in one call, with conditions, retries and a risk gate |
+
+## Delivery semantics
+
+| Verdict | Meaning | Retry |
+|---|---|---|
+| `not_delivered` | Nothing was sent | ✅ safe |
+| `delivered_unverified` | Input may have landed, effect unconfirmed | ❌ never blind-retry |
+| `delivered_verified` | Landed and the post-condition held | ❌ |
+
+That distinction is what makes "send a message / delete a file / submit a form" safe to automate.
+
+## Install
+
+```bash
+dsh plugin add dsh-gui-control
+```
+
+Windows 10 1809+, PowerShell 5.1, DeepSeek Harness 0.1.5-rc.2 – 0.2.x. Tools appear immediately, no restart.
+
+MIT.

package/cordis.patch.yml

@@ -0,0 +1,4 @@
+- insert:
+    - id: gui-control
+      name: dsh-gui-control
+      config: {}

package/icon.svg

@@ -0,0 +1,8 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" width="64" height="64" role="img" aria-label="Desktop control">
+  <rect x="4" y="8" width="56" height="40" rx="5" fill="none" stroke="#3b82f6" stroke-width="4"/>
+  <line x1="4" y1="18" x2="60" y2="18" stroke="#3b82f6" stroke-width="3"/>
+  <circle cx="11" cy="13" r="2.2" fill="#3b82f6"/>
+  <circle cx="18" cy="13" r="2.2" fill="#3b82f6"/>
+  <path d="M30 28 L30 50 L36 44 L40 53 L45 51 L41 42 L49 42 Z" fill="#f59e0b" stroke="#0f172a" stroke-width="2" stroke-linejoin="round"/>
+  <line x1="22" y1="56" x2="42" y2="56" stroke="#3b82f6" stroke-width="4" stroke-linecap="round"/>
+</svg>

package/lib/index.js

@@ -0,0 +1,346 @@
+/**
+ * dsh-gui-control — Windows desktop control for DeepSeek Harness.
+ *
+ * The heavy lifting lives in a long-lived PowerShell daemon shipped under lib/scripts.
+ * It compiles the Win32/UIA helpers once and then serves operations over loopback TCP,
+ * so a click costs ~10 ms instead of the ~700 ms of starting PowerShell and recompiling.
+ *
+ * This module only exposes three model-facing tools and keeps the surface small on
+ * purpose: read, act, or run a declarative task.
+ */
+import { execFileSync } from 'node:child_process';
+import { existsSync } from 'node:fs';
+import net from 'node:net';
+import { tmpdir } from 'node:os';
+import { dirname, join } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { defineTool } from '@deepseek-ai/dsh-tools';
+
+const HERE = dirname(fileURLToPath(import.meta.url));
+const SCRIPTS = join(HERE, 'scripts');
+const PORT = 38471;
+const HOST = '127.0.0.1';
+
+const isWindows = process.platform === 'win32';
+
+/** Resolve PowerShell explicitly: the host process PATH is not guaranteed to include it. */
+const POWERSHELL = (() => {
+	const systemRoot = process.env.SystemRoot ?? process.env.windir ?? 'C:\\Windows';
+	const full = join(systemRoot, 'System32', 'WindowsPowerShell', 'v1.0', 'powershell.exe');
+	return existsSync(full) ? full : 'powershell.exe';
+})();
+
+let launchPromise = null;
+
+function text(value) {
+	return [{ type: 'text', text: typeof value === 'string' ? value : JSON.stringify(value) }];
+}
+
+function assertWindows() {
+	if (!isWindows) {
+		throw new Error('dsh-gui-control only works on Windows; the native helpers are Win32 and UI Automation based.');
+	}
+}
+
+function pingDaemon(timeoutMs = 400) {
+	return new Promise((resolve) => {
+		const socket = net.createConnection({ host: HOST, port: PORT });
+		const done = (ok) => {
+			socket.removeAllListeners();
+			socket.destroy();
+			resolve(ok);
+		};
+		socket.setTimeout(timeoutMs);
+		socket.once('connect', () => done(true));
+		socket.once('timeout', () => done(false));
+		socket.once('error', () => done(false));
+	});
+}
+
+/** Start the daemon once per process; concurrent callers share one launch. */
+function ensureDaemon() {
+	if (launchPromise !== null) return launchPromise;
+	launchPromise = (async () => {
+		if (await pingDaemon()) return;
+		// Launch through Start-Process: a bare detached spawn of PowerShell can exit
+		// immediately on Windows depending on console inheritance from the host.
+		const daemon = join(SCRIPTS, 'gui-daemon.ps1');
+		const quote = (value) => `'${String(value).replace(/'/g, "''")}'`;
+		const args = ['-NoProfile', '-STA', '-ExecutionPolicy', 'Bypass', '-WindowStyle', 'Hidden', '-File', daemon, '-Port', String(PORT)];
+		const command = `Start-Process -FilePath ${quote(POWERSHELL)} -ArgumentList @(${args.map(quote).join(',')}) -WindowStyle Hidden`;
+		try {
+			execFileSync(POWERSHELL, ['-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command', command], { stdio: 'ignore', timeout: 15000, windowsHide: true });
+		} catch (error) {
+			launchPromise = null;
+			throw new Error(`Could not launch the GUI daemon: ${error.message}`);
+		}
+		for (let i = 0; i < 120; i += 1) {
+			await new Promise((r) => setTimeout(r, 100));
+			if (await pingDaemon()) return;
+		}
+		launchPromise = null;
+		throw new Error('The GUI daemon did not start. Check that PowerShell 5.1 is available and see lib/scripts/gui-daemon.log.');
+	})();
+	return launchPromise;
+}
+
+/** One request, one response line. Resolves with the daemon envelope. */
+function callDaemon(op, timeoutMs = 20000) {
+	return new Promise((resolve, reject) => {
+		const socket = net.createConnection({ host: HOST, port: PORT });
+		let buffer = '';
+		let settled = false;
+		const finish = (fn, value) => {
+			if (settled) return;
+			settled = true;
+			socket.destroy();
+			fn(value);
+		};
+		socket.setEncoding('utf8');
+		socket.setTimeout(timeoutMs);
+		socket.once('connect', () => socket.write(`${JSON.stringify(op)}\n`));
+		socket.on('data', (chunk) => {
+			buffer += chunk;
+			const nl = buffer.indexOf('\n');
+			if (nl === -1) return;
+			const line = buffer.slice(0, nl);
+			try {
+				finish(resolve, JSON.parse(line));
+			} catch (error) {
+				finish(reject, new Error(`Bad daemon response: ${line.slice(0, 200)}`));
+			}
+		});
+		socket.once('timeout', () => finish(reject, new Error(`GUI operation timed out after ${timeoutMs} ms (${op.op}).`)));
+		socket.once('error', (error) => finish(reject, error));
+	});
+}
+
+/** Call, and if the daemon died since the last call, relaunch it once and retry. */
+async function callResilient(op, timeoutMs) {
+	await ensureDaemon();
+	try {
+		return await callDaemon(op, timeoutMs);
+	} catch (error) {
+		const code = error?.code;
+		if (code !== 'ECONNREFUSED' && code !== 'EPIPE' && code !== 'ECONNRESET') throw error;
+		launchPromise = null;
+		await ensureDaemon();
+		return callDaemon(op, timeoutMs);
+	}
+}
+
+async function run(op, timeoutMs) {
+	const envelope = await callResilient(op, timeoutMs);
+	if (envelope.ok !== true) {
+		const parts = [String(envelope.error ?? 'operation failed')];
+		if (envelope.delivery !== undefined) parts.push(`delivery=${envelope.delivery}`);
+		if (envelope.retry !== undefined) parts.push(`retry=${envelope.retry}`);
+		throw new Error(parts.join('  '));
+	}
+	return envelope.result;
+}
+
+/** Compact tree read: one line per control, deduped, no rectangles. */
+function formatTree(lines) {
+	if (!Array.isArray(lines) || lines.length === 0) return '(no controls)';
+	return lines.join('\n');
+}
+
+function actOp(args) {
+	const action = String(args.action ?? '').toLowerCase();
+	const base = {};
+	if (typeof args.window === 'string' && args.window.length > 0) base.win = args.window;
+	if (Number.isFinite(args.x) && Number.isFinite(args.y)) {
+		base.x = Math.round(args.x);
+		base.y = Math.round(args.y);
+	}
+	switch (action) {
+		case 'click':
+		case 'double_click':
+		case 'right_click': {
+			const out = { op: 'click', ...base };
+			if (action === 'double_click') out.count = 2;
+			if (action === 'right_click') out.button = 'right';
+			return out;
+		}
+		case 'type':
+			return { op: 'type', ...base, text: String(args.text ?? '') };
+		case 'key':
+			return { op: 'key', ...base, key: String(args.key ?? '') };
+		case 'hotkey':
+			return { op: 'hotkey', ...base, key: String(args.key ?? '') };
+		case 'scroll':
+			return { op: 'scroll', ...base, amount: Number(args.amount ?? -3) };
+		case 'focus':
+			return { op: 'activate', win: String(args.window ?? '') };
+		case 'click_text': {
+			// Semantic click: locate the control by name instead of by coordinate.
+			return { op: 'uia-click', win: String(args.window ?? ''), text: String(args.text ?? ''), type: args.controlType };
+		}
+		default:
+			throw new Error(`Unknown action "${action}". Use click, double_click, right_click, type, key, hotkey, scroll, focus, or click_text.`);
+	}
+}
+
+const SNAPSHOT = {
+	name: 'gui_snapshot',
+	description: [
+		'Read a Windows window as text: its accessibility tree (control names, roles, bounds-free) or a cropped screenshot with a coordinate ruler.',
+		'Use the tree first — it is exact, cheap, and position independent. Fall back to a screenshot only when the window publishes no tree (a canvas, a game, or a self-drawn list).',
+		'Always snapshot before acting, and again afterwards to confirm what changed.',
+		'A screenshot is returned as a file path; read that file with the image reader. The ruler labels are screen coordinates, so you can pass them straight back to gui_act.',
+	].join(' '),
+	parameters: {
+		window: { type: 'string', description: 'Window title or process-name substring, case-insensitive (for example "Notepad" or "计算器").' },
+		mode: { type: 'string', enum: ['tree', 'shot'], description: 'tree (default): control names as text. shot: PNG with a screen-coordinate ruler.' },
+		lines: { type: 'number', description: 'Maximum control lines to return (default 40, tree mode only).' },
+		chars: { type: 'number', description: 'Truncate each control name to this many characters (default 60, tree mode only).' },
+		region: { type: 'string', description: "Screenshot crop in screen pixels as 'x,y,width,height' (shot mode). Defaults to the whole window." },
+	},
+	output: {
+		schema: { type: 'string' },
+		render: (_args, value) => text(value),
+	},
+	isConcurrencySafe: () => true,
+	async execute(args) {
+		assertWindows();
+		const window = typeof args.window === 'string' ? args.window : '';
+		if (window.length === 0) throw new Error('gui_snapshot needs a window (title or process substring).');
+		const mode = args.mode === 'shot' ? 'shot' : 'tree';
+		if (mode === 'tree') {
+			const lines = await run({ op: 'uia-mini', win: window, lines: args.lines ?? 40, chars: args.chars ?? 60 });
+			return formatTree(lines);
+		}
+		const op = { op: 'shot', win: window, rulers: true, path: join(tmpdir(), `gui-shot-${Date.now()}.png`) };
+		if (typeof args.region === 'string' && args.region.length > 0) {
+			const nums = args.region.split(',').map((v) => Number(v.trim()));
+			if (nums.length === 4 && nums.every((n) => Number.isFinite(n))) {
+				[op.x, op.y, op.w, op.h] = nums;
+			}
+		}
+		const shot = await run(op);
+		return `shot ${shot.w}x${shot.h} at ${shot.x},${shot.y} scale ${shot.scale}\n${shot.path}\nRead the file to see it. Ruler labels are screen coordinates.`;
+	},
+	presentCall(args) {
+		return { card: 'generic', title: `Snapshot ${args.window ?? 'window'}`, kind: 'read' };
+	},
+};
+
+const ACT = {
+	name: 'gui_act',
+	description: [
+		'Perform ONE desktop action and optionally verify its effect.',
+		'Prefer click_text over click with coordinates: it addresses the control by name, so the mouse does not move and a window layout change cannot make it hit the wrong thing.',
+		'Every action reports a delivery verdict. "not_delivered" means nothing was sent and a retry is safe; "delivered_unverified" means the input may have landed, so repeating it risks a duplicate — verify first, never blind-retry.',
+		'For more than two steps use gui_task instead: one call, conditions, and a single failure report.',
+	].join(' '),
+	parameters: {
+		action: {
+			type: 'string',
+			enum: ['click', 'double_click', 'right_click', 'type', 'key', 'hotkey', 'scroll', 'focus', 'click_text'],
+			description: 'click_text = click the control whose name matches text (recommended). key/hotkey take a key name such as enter or ctrl+s. focus brings a window to the front.',
+		},
+		window: { type: 'string', description: 'Target window (title or process substring). Required for click_text, focus, click, type, and key.' },
+		text: { type: 'string', description: 'Text to type, or the control name for click_text.' },
+		x: { type: 'number', description: 'Screen x for click/double_click/right_click (use only when no control name exists).' },
+		y: { type: 'number', description: 'Screen y for click/double_click/right_click.' },
+		key: { type: 'string', description: "Key or chord: 'enter', 'tab', 'esc', 'f5', 'ctrl+s', 'alt+tab'." },
+		amount: { type: 'number', description: 'Scroll notches; negative scrolls down (default -3).' },
+		controlType: { type: 'string', description: 'Optional control type filter for click_text, for example Button or TabItem.' },
+		verifyText: { type: 'string', description: 'After acting, poll until this text appears in the window (proves the action landed).' },
+		verifyTimeout: { type: 'number', description: 'Seconds to poll for verifyText (default 4).' },
+	},
+	output: {
+		schema: { type: 'string' },
+		render: (_args, value) => text(value),
+	},
+	isConcurrencySafe: () => false,
+	async execute(args) {
+		assertWindows();
+		const op = actOp(args);
+		const envelope = await callResilient(op);
+		if (envelope.ok !== true) {
+			const parts = [String(envelope.error ?? 'action failed')];
+			if (envelope.delivery !== undefined) parts.push(`delivery=${envelope.delivery}`);
+			if (envelope.retry !== undefined) parts.push(`retry=${envelope.retry}`);
+			throw new Error(parts.join('  '));
+		}
+		let verified = 'unverified';
+		if (typeof args.verifyText === 'string' && args.verifyText.length > 0) {
+			verified = await verifyText(args.window, args.verifyText, (args.verifyTimeout ?? 4) * 1000);
+		}
+		const how = envelope.result?.how !== undefined ? ` via ${envelope.result.how}` : '';
+		return `${op.op} ok${how}${verified === 'ok' ? ' | verified: text appeared' : ''}${verified === 'timeout' ? ' | delivery=delivered_unverified: text never appeared, do NOT blind-retry' : ''}`;
+	},
+	presentCall(args) {
+		return { card: 'generic', title: `${args.action ?? 'act'} ${args.window ?? ''}`.trim(), kind: 'execute' };
+	},
+};
+
+async function verifyText(window, needle, timeoutMs) {
+	const deadline = Date.now() + timeoutMs;
+	while (Date.now() < deadline) {
+		try {
+			const envelope = await callResilient({ op: 'uia-get', win: String(window ?? ''), text: needle }, 5000);
+			if (envelope.ok === true) return 'ok';
+		} catch {
+			// keep polling
+		}
+		await new Promise((r) => setTimeout(r, 200));
+	}
+	return 'timeout';
+}
+
+const TASK = {
+	name: 'gui_task',
+	description: [
+		'Run a declarative task file: an ordered list of desktop steps executed inside the daemon, with no model in the loop.',
+		'Each step names an operation plus its parameters and may add "until" (a condition polled until it holds), "untilTimeout", "retry", and "risk".',
+		'A task returns per-step timing and a delivery verdict, so one call replaces a long back-and-forth of single actions.',
+		'Pass dryRun to see the plan without touching the desktop. Steps marked risk "high" are refused unless confirm is true.',
+		'Example step: { "do": "uia-click", "win": "Calculator", "text": "7", "until": { "op": "uia-get", "win": "Calculator", "text": "Display is 7" } }',
+	].join(' '),
+	parameters: {
+		file: { type: 'string', description: 'Absolute path to the task JSON file.' },
+		dryRun: { type: 'boolean', description: 'Return the step plan without executing anything.' },
+		confirm: { type: 'boolean', description: 'Allow steps marked risk high (otherwise they are refused before dispatch).' },
+	},
+	output: {
+		schema: { type: 'string' },
+		render: (_args, value) => text(value),
+	},
+	isConcurrencySafe: () => false,
+	async execute(args) {
+		assertWindows();
+		const file = typeof args.file === 'string' ? args.file : '';
+		if (file.length === 0) throw new Error('gui_task needs file (absolute path to a task JSON).');
+		const op = { op: 'task', file, dryRun: args.dryRun === true, confirm: args.confirm === true };
+		const envelope = await callResilient(op, 600000);
+		const result = envelope.result ?? {};
+		if (result.dryRun === true) {
+			const plan = (result.steps ?? []).map((step) => `${String(step.n).padStart(2)}. ${String(step.do).padEnd(10)} risk=${step.risk ?? 'low'}`);
+			return [`task ${result.name ?? ''} plan (dry run, nothing executed)`, ...plan].join('\n');
+		}
+		const lines = [];
+		for (const step of result.steps ?? []) {
+			const flag = step.ok ? 'ok  ' : 'FAIL';
+			const extra = step.note ? ` ${step.note}` : '';
+			lines.push(`${String(step.n).padStart(2)}. ${flag} ${String(step.do).padEnd(10)} ${String(step.ms).padStart(6)} ms  ${step.delivery ?? ''}${extra}`.trimEnd());
+		}
+		const head = envelope.ok === true
+			? `task ${result.name ?? ''} ok in ${result.total ?? '?'} ms`
+			: `task failed: ${envelope.error}${envelope.delivery ? `  delivery=${envelope.delivery}` : ''}${envelope.retry ? `  retry=${envelope.retry}` : ''}`;
+		return [head, ...lines].join('\n');
+	},
+	presentCall(args) {
+		return { card: 'generic', title: `Run task ${String(args.file ?? '').split(/[\\/]/).pop() ?? ''}`, kind: 'execute' };
+	},
+};
+
+export const inject = ['tools'];
+
+export function apply(ctx) {
+	ctx.effect(() => ctx.tools.register(defineTool(SNAPSHOT)));
+	ctx.effect(() => ctx.tools.register(defineTool(ACT)));
+	ctx.effect(() => ctx.tools.register(defineTool(TASK)));
+}