@ai-agent-forge/plugin-desktop-macos-input 0.0.0-stage → 0.88.3

package/MANUAL.md

@@ -0,0 +1,16 @@
+# desktop-macos-input 能力手册(AI 用)
+
+macOS 桌面真实输入:CGEvent HID 级鼠标/键盘注入(与物理硬件事件同一队列)+ AX 元素动作。与 desktop-macos-observe 配对(observe 拿元素坐标 → execute → reobserve)。
+
+## 真实度(重要)
+
+- **鼠标**:CGEventPost(kCGHIDEventTap)——用户态最接近真实硬件的档位,行为信号层面与物理鼠标同队列。
+- **键盘**:System Events keystroke/keyCode——同为 HID 注入。
+- 这是"反自动化检测/原生对话框/无 DOM 页面"场景的真实操作通道;CDP 系工具(Playwright)生成的事件与之不同源。
+
+## 使用要点
+
+1. **TCC**:辅助功能授权对象是 `/usr/bin/osascript`;-25201/-25211 让用户加白(见 observe 手册)。
+2. **坐标输入前先聚焦目标窗口**,执行期间用户碰键鼠会打断。
+3. 锁屏时注入无效(窗口不在 AX 树,事件落空)——先确认解锁。
+4. 打字中文等非 ASCII 文本走 AX set value 或剪贴板粘贴路径比逐键注入可靠;逐键注入适合 ASCII 与快捷键。

package/README.md

@@ -1,3 +1,56 @@
-# Temporary Holding Version
+# @agent-forge/plugin-desktop-macos-input
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Bounded macOS desktop input capability plugin for [Agent Forge](https://github.com/agent-forge/agent-forge), structured as the macOS counterpart of
+`plugins/desktop-windows-input`. It registers `desktop_execute` — focus, click, type, key, scroll — backed by the macOS Accessibility tree through a
+JXA driver (`osascript -l JavaScript`). A readonly host mode registers no input tools at all, and the shipped facade deliberately exposes no replay
+method, so `desktop_replay` never registers out of the box.
+
+Action surface: click accepts observed control identities (AXPress with a CGEvent coordinate-click fallback) and — click-only — explicit
+`coordinates` targets (integer, non-negative screen coordinates); key uses a minimal SendKeys-aligned grammar (`+` shift, `^` control, `%` option,
+`{ENTER}`-style special keys) translated to System Events keystroke/keyCode; scroll posts CGEvent scroll-wheel events. High-risk actions (uac,
+password, admin, payment, external-submit) stay behind the shared `requiresDesktopUserConfirmation` fail-closed gate and always return `needs_user`.
+Every action re-observes so results carry fresh state.
+
+On non-macOS platforms importing the package does not throw; every tool reports an explicit `unsupported` result. A missing Accessibility
+(assistive access) grant is reported fail-closed as the probe reason `accessibility-permission-required`.
+
+## Install
+
+```sh
+# from a local checkout
+agent-forge plugin install ./plugins/desktop-macos-input
+
+# from npm
+agent-forge plugin install @agent-forge/plugin-desktop-macos-input
+```
+
+The package ships two manifests: `agent-forge.json` (ecosystem manifest for the deterministic plugin resolver, `platform.os: ["macos"]`) and
+`plugin.json` (host-load manifest consumed by `loadCapabilityPluginFromManifest`, entry `./dist/entry.js`).
+
+## Library surface
+
+`import { DesktopMacosInputAdapter, MacosAccessibilityDriver } from "@agent-forge/plugin-desktop-macos-input";`
+
+- `DesktopMacosInputAdapter` — production input facade (no replay method).
+- `MacosAccessibilityDriver(options?)` — JXA driver; supports `command`/`timeoutMs`/`spawnSync`/`platform` injection for brokers and tests.
+- `createDesktopInputCapabilityFactoryV1(runtime)` — registers `desktop_execute` (plus `desktop_replay` only when the injected runtime exposes a
+  replay method) on a `CapabilityAPI`.
+- Default export of `dist/entry.js` — the precomposed factory used by external plugin loading.
+
+The session/observation/projection contract and the confirmation gate are deliberate inlined copies shared with
+`@agent-forge/plugin-desktop-macos-observe` so both packages stay self-contained.
+
+## Real-host status
+
+The driver and adapter are contract-tested with an injected fake `spawnSync` (canned JSON, no real desktop). Real darwin behavior — the JXA
+enumeration and action scripts, Accessibility permission classification, keystroke/keyCode delivery and CGEvent clicks — is verified only by the
+explicitly gated test (`AGENT_FORGE_MACOS_AX_INPUT_REAL_TEST=1` on macOS, scroll-only to stay reversible) and has not been validated on a real
+macOS host as of this change.
+
+## Development
+
+```sh
+npm install --ignore-scripts        # repo root, creates the workspace symlink
+npm run build                       # plugins/desktop-macos-input (tsgo)
+npm test                            # vitest; the entry fidelity test requires `npm run build` first
+```

package/agent-forge.json

@@ -0,0 +1,31 @@
+{
+	"packageFormatVersion": 1,
+	"id": "agent-forge.plugin.desktop-macos-input",
+	"pluginVersion": "0.88.3",
+	"apiVersion": "1",
+	"minHostVersion": "0.84.4",
+	"capabilityRefs": [
+		{
+			"id": "desktop-session",
+			"version": 1
+		},
+		{
+			"id": "desktop-uia",
+			"version": 1
+		},
+		{
+			"id": "desktop-input",
+			"version": 1
+		}
+	],
+	"platform": {
+		"os": [
+			"macos"
+		],
+		"runtime": [
+			{
+				"name": "node"
+			}
+		]
+	}
+}

package/dist/capability.d.ts

@@ -0,0 +1,21 @@
+/**
+ * Bounded desktop input capability factory, mirroring the Windows package's
+ * `capability.ts` (migrated from agent-forge
+ * `src/profile/runtime-capabilities-desktop-input.ts`,
+ * `desktopInputFactory`, D-075 S3 extraction). Differences from the Windows
+ * copy:
+ * - The runtime type is the macOS full-control adapter from `types.ts`.
+ * - The target union gains the `coordinates` variant (`x`/`y` integer,
+ *   non-negative screen coordinates); the coordinate target is accepted for
+ *   the `click` action only — any other action kind with a coordinate target
+ *   is rejected as `desktop_input_invalid` (same ruling the Windows package
+ *   implements for its coordinate click).
+ * Tool names, parameters, budgets, result shapes, the readonly-mode refusal
+ * to register input tools, and the optional replay registration when the
+ * runtime exposes a replay method are unchanged; the shipped macOS facade
+ * exposes no replay, so only `desktop_execute` registers out of the box.
+ */
+import type { CapabilityPluginSyncFactory } from "@agent-forge/plugin-sdk";
+import type { DesktopFullControlAdapterV1 } from "./types.ts";
+export declare function createDesktopInputCapabilityFactoryV1(runtime: DesktopFullControlAdapterV1): CapabilityPluginSyncFactory;
+//# sourceMappingURL=capability.d.ts.map

package/dist/capability.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"capability.d.ts","sourceRoot":"","sources":["../src/capability.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;GAgBG;AACH,OAAO,KAAK,EAAE,2BAA2B,EAAE,MAAM,yBAAyB,CAAC;AAc3E,OAAO,KAAK,EACX,2BAA2B,EAI3B,MAAM,YAAY,CAAC;AA6TpB,wBAAgB,qCAAqC,CACpD,OAAO,EAAE,2BAA2B,GAClC,2BAA2B,CA0N7B","sourcesContent":["/**\n * Bounded desktop input capability factory, mirroring the Windows package's\n * `capability.ts` (migrated from agent-forge\n * `src/profile/runtime-capabilities-desktop-input.ts`,\n * `desktopInputFactory`, D-075 S3 extraction). Differences from the Windows\n * copy:\n * - The runtime type is the macOS full-control adapter from `types.ts`.\n * - The target union gains the `coordinates` variant (`x`/`y` integer,\n *   non-negative screen coordinates); the coordinate target is accepted for\n *   the `click` action only — any other action kind with a coordinate target\n *   is rejected as `desktop_input_invalid` (same ruling the Windows package\n *   implements for its coordinate click).\n * Tool names, parameters, budgets, result shapes, the readonly-mode refusal\n * to register input tools, and the optional replay registration when the\n * runtime exposes a replay method are unchanged; the shipped macOS facade\n * exposes no replay, so only `desktop_execute` registers out of the box.\n */\nimport type { CapabilityPluginSyncFactory } from \"@agent-forge/plugin-sdk\";\nimport { Type } from \"typebox\";\nimport {\n\tDESKTOP_MAX_POSTCONDITIONS_V1,\n\tDESKTOP_MAX_REPLAY_ACTIONS_V1,\n\tDESKTOP_MAX_STRING_CHARS_V1,\n\tDESKTOP_MAX_TYPE_TEXT_CHARS_V1,\n\tdesktopBoundedString,\n\tdesktopFailure,\n\tdesktopRequiredString,\n\tdesktopResultWithinBudget,\n\tprojectDesktopObservation,\n} from \"./projection.ts\";\nimport { signalFrom, toolInvocation } from \"./shared.ts\";\nimport type {\n\tDesktopFullControlAdapterV1,\n\tDesktopOperationRequestV1,\n\tDesktopPostconditionV1,\n\tDesktopReplayRequestV1,\n} from \"./types.ts\";\n\nfunction desktopCoordinates(input: Record<string, unknown>): { readonly x: number; readonly y: number } {\n\tconst x = input.x;\n\tconst y = input.y;\n\tif (typeof x !== \"number\" || typeof y !== \"number\" || !Number.isInteger(x) || !Number.isInteger(y) || x < 0 || y < 0)\n\t\tthrow new Error(\"desktop-target-invalid\");\n\treturn { x, y };\n}\n\nfunction desktopTarget(input: unknown): DesktopOperationRequestV1[\"action\"][\"target\"] {\n\tif (input === null || typeof input !== \"object\" || Array.isArray(input)) throw new Error(\"desktop-target-invalid\");\n\tconst target = input as Record<string, unknown>;\n\tif (target.kind === \"window\") {\n\t\tdesktopExactKeys(target, [\"kind\", \"windowId\"]);\n\t\treturn { kind: \"window\", windowId: desktopRequiredString(target.windowId, \"windowId\") };\n\t}\n\tif (target.kind === \"control\") {\n\t\tdesktopExactKeys(target, [\"kind\", \"controlId\", \"windowId\"]);\n\t\treturn {\n\t\t\tkind: \"control\",\n\t\t\tcontrolId: desktopRequiredString(target.controlId, \"controlId\"),\n\t\t\t...(target.windowId === undefined ? {} : { windowId: desktopRequiredString(target.windowId, \"windowId\") }),\n\t\t};\n\t}\n\tif (target.kind === \"coordinates\") {\n\t\tdesktopExactKeys(target, [\"kind\", \"x\", \"y\"]);\n\t\treturn { kind: \"coordinates\", ...desktopCoordinates(target) };\n\t}\n\tthrow new Error(\"desktop-target-invalid\");\n}\n\nfunction desktopExactKeys(value: Record<string, unknown>, allowed: readonly string[]): void {\n\tif (Object.keys(value).some((key) => !allowed.includes(key))) throw new Error(\"desktop-input-invalid\");\n}\n\nfunction desktopPostconditions(input: unknown): readonly DesktopPostconditionV1[] | undefined {\n\tif (input === undefined) return undefined;\n\tif (!Array.isArray(input) || input.length > DESKTOP_MAX_POSTCONDITIONS_V1)\n\t\tthrow new Error(\"desktop-postconditions-invalid\");\n\treturn input.map((entry) => {\n\t\tif (entry === null || typeof entry !== \"object\" || Array.isArray(entry))\n\t\t\tthrow new Error(\"desktop-postcondition-invalid\");\n\t\tconst value = entry as Record<string, unknown>;\n\t\tif (value.kind === \"window-exists\") {\n\t\t\tdesktopExactKeys(value, [\"kind\", \"windowId\"]);\n\t\t\treturn { kind: \"window-exists\", windowId: desktopRequiredString(value.windowId, \"windowId\") } as const;\n\t\t}\n\t\tif (value.kind === \"control-exists\") {\n\t\t\tdesktopExactKeys(value, [\"kind\", \"controlId\"]);\n\t\t\treturn { kind: \"control-exists\", controlId: desktopRequiredString(value.controlId, \"controlId\") } as const;\n\t\t}\n\t\tif (value.kind === \"focused\") {\n\t\t\tdesktopExactKeys(value, [\"kind\", \"controlId\"]);\n\t\t\treturn { kind: \"focused\", controlId: desktopRequiredString(value.controlId, \"controlId\") } as const;\n\t\t}\n\t\tif (value.kind === \"property-equals\") {\n\t\t\tdesktopExactKeys(value, [\"kind\", \"target\", \"property\", \"expected\"]);\n\t\t\tif (value.property !== \"enabled\" && value.property !== \"value\" && value.property !== \"name\")\n\t\t\t\tthrow new Error(\"desktop-postcondition-invalid\");\n\t\t\tif (\n\t\t\t\t(value.property === \"enabled\" && typeof value.expected !== \"boolean\") ||\n\t\t\t\t(value.property !== \"enabled\" && typeof value.expected !== \"string\")\n\t\t\t)\n\t\t\t\tthrow new Error(\"desktop-postcondition-invalid\");\n\t\t\treturn {\n\t\t\t\tkind: \"property-equals\",\n\t\t\t\ttarget: desktopTarget(value.target),\n\t\t\t\tproperty: value.property,\n\t\t\t\texpected:\n\t\t\t\t\ttypeof value.expected === \"string\"\n\t\t\t\t\t\t? desktopBoundedString(value.expected, \"postcondition expected\")\n\t\t\t\t\t\t: value.expected,\n\t\t\t} as DesktopPostconditionV1;\n\t\t}\n\t\tthrow new Error(\"desktop-postcondition-invalid\");\n\t});\n}\n\nfunction desktopExecuteRequest(input: Record<string, unknown>, operationId: string): DesktopOperationRequestV1 {\n\tdesktopExactKeys(input, [\"sessionId\", \"action\", \"postconditions\"]);\n\tconst actionInput = input.action;\n\tif (actionInput === null || typeof actionInput !== \"object\" || Array.isArray(actionInput))\n\t\tthrow new Error(\"desktop-action-invalid\");\n\tconst action = actionInput as Record<string, unknown>;\n\tdesktopExactKeys(action, [\"kind\", \"target\", \"text\", \"key\"]);\n\tif (\n\t\taction.kind !== \"focus\" &&\n\t\taction.kind !== \"click\" &&\n\t\taction.kind !== \"type\" &&\n\t\taction.kind !== \"key\" &&\n\t\taction.kind !== \"scroll\"\n\t)\n\t\tthrow new Error(\"desktop-action-invalid\");\n\tconst target = desktopTarget(action.target);\n\tif (target === undefined) throw new Error(\"desktop-target-invalid\");\n\t// Coordinate targets are a click-only affordance: pointing elsewhere is a\n\t// caller contract violation, not a fallback path.\n\tif (target.kind === \"coordinates\" && action.kind !== \"click\") throw new Error(\"desktop-coordinates-click-only\");\n\tconst text =\n\t\taction.kind === \"type\" || action.kind === \"scroll\" ? desktopBoundedString(action.text, \"text\") : undefined;\n\tif (text !== undefined && text.length > DESKTOP_MAX_TYPE_TEXT_CHARS_V1) throw new Error(\"desktop-type-text-budget\");\n\tconst key = action.kind === \"key\" ? desktopBoundedString(action.key, \"key\") : undefined;\n\tif (key !== undefined && key.length > DESKTOP_MAX_TYPE_TEXT_CHARS_V1) throw new Error(\"desktop-type-text-budget\");\n\tconst postconditions = desktopPostconditions(input.postconditions);\n\treturn {\n\t\tversion: 1,\n\t\toperationId,\n\t\tsessionId: desktopRequiredString(input.sessionId, \"sessionId\"),\n\t\taction: {\n\t\t\tversion: 1,\n\t\t\tkind: action.kind,\n\t\t\ttarget,\n\t\t\t...(text === undefined ? {} : { text }),\n\t\t\t...(key === undefined ? {} : { key }),\n\t\t},\n\t\t...(postconditions === undefined ? {} : { postconditions }),\n\t\t// Input plugin operations always request a fresh observation; callers may\n\t\t// not turn this off because target identity otherwise goes stale.\n\t\treobserveAfterAction: true,\n\t};\n}\n\nfunction desktopVerification(\n\tvalue: unknown,\n): { readonly status: \"passed\" | \"failed\" | \"unknown\"; readonly checks: readonly string[] } | undefined {\n\tif (value === undefined) return undefined;\n\tif (value === null || typeof value !== \"object\" || Array.isArray(value)) throw new Error(\"desktop-result-invalid\");\n\tconst verification = value as Record<string, unknown>;\n\tif (\n\t\t(verification.status !== \"passed\" && verification.status !== \"failed\" && verification.status !== \"unknown\") ||\n\t\t!Array.isArray(verification.checks) ||\n\t\tverification.checks.length > DESKTOP_MAX_POSTCONDITIONS_V1\n\t)\n\t\tthrow new Error(\"desktop-result-invalid\");\n\treturn {\n\t\tstatus: verification.status,\n\t\tchecks: verification.checks.map((check) => desktopBoundedString(check, \"verification check\")),\n\t};\n}\n\nfunction desktopExecuteResult(value: unknown, request: DesktopOperationRequestV1) {\n\tif (value === null || typeof value !== \"object\" || Array.isArray(value)) throw new Error(\"desktop-result-invalid\");\n\tconst result = value as Record<string, unknown>;\n\tif (\n\t\tresult.version !== 1 ||\n\t\tresult.operationId !== request.operationId ||\n\t\t(result.status !== \"succeeded\" &&\n\t\t\tresult.status !== \"failed\" &&\n\t\t\tresult.status !== \"cancelled\" &&\n\t\t\tresult.status !== \"needs_user\" &&\n\t\t\tresult.status !== \"unsupported\")\n\t)\n\t\tthrow new Error(\"desktop-result-invalid\");\n\tconst observation = result.observation === undefined ? undefined : projectDesktopObservation(result.observation);\n\tif (\n\t\tresult.status === \"succeeded\" &&\n\t\t(observation === undefined || observation.observation.sessionId !== request.sessionId)\n\t) {\n\t\treturn desktopFailure(\n\t\t\trequest.operationId,\n\t\t\t\"verification_unavailable\",\n\t\t\t\"Desktop action did not produce a fresh observation\",\n\t\t);\n\t}\n\tconst base = {\n\t\tversion: 1 as const,\n\t\toperationId: request.operationId,\n\t\tstatus: result.status,\n\t\tsummary: result.status === \"succeeded\" ? \"Desktop action completed\" : \"Desktop action could not be completed\",\n\t\t...(observation === undefined ? {} : observation),\n\t\t...(desktopVerification(result.verification) === undefined\n\t\t\t? {}\n\t\t\t: { verification: desktopVerification(result.verification) }),\n\t};\n\tif (result.status === \"succeeded\") return desktopResultWithinBudget(base);\n\tconst error = result.error;\n\tif (error === null || typeof error !== \"object\" || Array.isArray(error)) throw new Error(\"desktop-result-invalid\");\n\tconst errorValue = error as Record<string, unknown>;\n\tconst causeCode =\n\t\tresult.status === \"needs_user\" && typeof errorValue.causeCode === \"string\"\n\t\t\t? desktopBoundedString(errorValue.causeCode, \"error causeCode\")\n\t\t\t: undefined;\n\treturn desktopResultWithinBudget({\n\t\t...base,\n\t\terror: {\n\t\t\tversion: 1 as const,\n\t\t\tcode: result.status === \"needs_user\" ? \"needs_user\" : \"desktop_action_failed\",\n\t\t\tmessage: result.status === \"needs_user\" ? \"Desktop action requires user action\" : \"Desktop action failed\",\n\t\t\t...(causeCode === undefined ? {} : { causeCode }),\n\t\t},\n\t});\n}\n\nfunction desktopReplayActionInput(raw: unknown): DesktopOperationRequestV1[\"action\"] {\n\tif (raw === null || typeof raw !== \"object\" || Array.isArray(raw)) throw new Error(\"desktop-action-invalid\");\n\tconst action = raw as Record<string, unknown>;\n\tdesktopExactKeys(action, [\"kind\", \"target\", \"text\", \"key\"]);\n\tif (\n\t\taction.kind !== \"focus\" &&\n\t\taction.kind !== \"click\" &&\n\t\taction.kind !== \"type\" &&\n\t\taction.kind !== \"key\" &&\n\t\taction.kind !== \"scroll\"\n\t)\n\t\tthrow new Error(\"desktop-action-invalid\");\n\tconst target = desktopTarget(action.target);\n\tif (target === undefined) throw new Error(\"desktop-target-invalid\");\n\tif (target.kind === \"coordinates\" && action.kind !== \"click\") throw new Error(\"desktop-coordinates-click-only\");\n\tconst text =\n\t\taction.kind === \"type\" || action.kind === \"scroll\" ? desktopBoundedString(action.text, \"text\") : undefined;\n\tif (text !== undefined && text.length > DESKTOP_MAX_TYPE_TEXT_CHARS_V1) throw new Error(\"desktop-type-text-budget\");\n\tconst key = action.kind === \"key\" ? desktopBoundedString(action.key, \"key\") : undefined;\n\tif (key !== undefined && key.length > DESKTOP_MAX_TYPE_TEXT_CHARS_V1) throw new Error(\"desktop-type-text-budget\");\n\treturn {\n\t\tversion: 1,\n\t\tkind: action.kind,\n\t\ttarget,\n\t\t...(text === undefined ? {} : { text }),\n\t\t...(key === undefined ? {} : { key }),\n\t};\n}\n\nfunction desktopReplayRequest(input: Record<string, unknown>, operationId: string): DesktopReplayRequestV1 {\n\tdesktopExactKeys(input, [\"sessionId\", \"actions\", \"stopOnFailure\"]);\n\tif (\n\t\t!Array.isArray(input.actions) ||\n\t\tinput.actions.length === 0 ||\n\t\tinput.actions.length > DESKTOP_MAX_REPLAY_ACTIONS_V1\n\t)\n\t\tthrow new Error(\"desktop-replay-actions-invalid\");\n\tconst sessionId = desktopRequiredString(input.sessionId, \"sessionId\");\n\tconst operations = input.actions.map((raw, index) => {\n\t\tconst action = desktopReplayActionInput(raw);\n\t\treturn {\n\t\t\tversion: 1 as const,\n\t\t\toperationId: `${operationId}-${index}`,\n\t\t\tsessionId,\n\t\t\taction,\n\t\t\t// Replay always re-observes so per-action results carry fresh state.\n\t\t\treobserveAfterAction: true,\n\t\t} satisfies DesktopOperationRequestV1;\n\t});\n\treturn {\n\t\tversion: 1,\n\t\tsessionId,\n\t\toperations,\n\t\tstopOnFailure: input.stopOnFailure === undefined ? true : input.stopOnFailure === true,\n\t};\n}\n\nfunction desktopReplayResult(value: unknown, operationId: string): unknown {\n\tif (value === null || typeof value !== \"object\" || Array.isArray(value)) throw new Error(\"desktop-result-invalid\");\n\tconst result = value as Record<string, unknown>;\n\tif (\n\t\tresult.version !== 1 ||\n\t\t(result.status !== \"succeeded\" &&\n\t\t\tresult.status !== \"failed\" &&\n\t\t\tresult.status !== \"cancelled\" &&\n\t\t\tresult.status !== \"needs_user\") ||\n\t\t!Array.isArray(result.results)\n\t)\n\t\tthrow new Error(\"desktop-result-invalid\");\n\tconst results = result.results.map((entry) => {\n\t\tif (entry === null || typeof entry !== \"object\" || Array.isArray(entry))\n\t\t\tthrow new Error(\"desktop-result-invalid\");\n\t\tconst record = entry as Record<string, unknown>;\n\t\tif (\n\t\t\trecord.version !== 1 ||\n\t\t\ttypeof record.operationId !== \"string\" ||\n\t\t\t(record.status !== \"succeeded\" &&\n\t\t\t\trecord.status !== \"failed\" &&\n\t\t\t\trecord.status !== \"cancelled\" &&\n\t\t\t\trecord.status !== \"needs_user\" &&\n\t\t\t\trecord.status !== \"unsupported\")\n\t\t)\n\t\t\tthrow new Error(\"desktop-result-invalid\");\n\t\treturn {\n\t\t\toperationId: desktopBoundedString(record.operationId, \"replay operationId\"),\n\t\t\tstatus: record.status,\n\t\t\tsummary: desktopBoundedString(record.summary, \"replay summary\"),\n\t\t};\n\t});\n\tconst last = result.results[result.results.length - 1] as Record<string, unknown> | undefined;\n\tconst observation =\n\t\tlast?.observation === undefined ? undefined : projectDesktopObservation(last.observation).observation;\n\tconst needsUser = result.results.find((entry) => (entry as Record<string, unknown>).status === \"needs_user\") as\n\t\t| Record<string, unknown>\n\t\t| undefined;\n\tconst causeCode =\n\t\tneedsUser !== undefined &&\n\t\tneedsUser.error !== null &&\n\t\ttypeof needsUser.error === \"object\" &&\n\t\t\"causeCode\" in needsUser.error &&\n\t\ttypeof (needsUser.error as Record<string, unknown>).causeCode === \"string\"\n\t\t\t? desktopBoundedString((needsUser.error as Record<string, unknown>).causeCode, \"replay causeCode\")\n\t\t\t: undefined;\n\treturn desktopResultWithinBudget({\n\t\tversion: 1,\n\t\toperationId,\n\t\tstatus: result.status,\n\t\tsummary: result.status === \"succeeded\" ? \"Desktop replay completed\" : \"Desktop replay stopped before completion\",\n\t\tresults,\n\t\t...(observation === undefined ? {} : { observation }),\n\t\t...(result.status === \"needs_user\" && causeCode !== undefined\n\t\t\t? {\n\t\t\t\t\terror: {\n\t\t\t\t\t\tversion: 1 as const,\n\t\t\t\t\t\tcode: \"needs_user\",\n\t\t\t\t\t\tmessage: \"Desktop replay requires user action\",\n\t\t\t\t\t\tcauseCode,\n\t\t\t\t\t},\n\t\t\t\t}\n\t\t\t: {}),\n\t});\n}\n\nexport function createDesktopInputCapabilityFactoryV1(\n\truntime: DesktopFullControlAdapterV1,\n): CapabilityPluginSyncFactory {\n\treturn (api) => {\n\t\tif (api.host.mode === \"readonly\") return;\n\t\tapi.registerTool({\n\t\t\tname: \"desktop_execute\",\n\t\t\tlabel: \"Execute desktop action\",\n\t\t\tdescription:\n\t\t\t\t\"Execute one bounded Accessibility focus, click, type, key, or scroll action against an observed target (click also accepts screen coordinates) and return a fresh bounded observation\",\n\t\t\tparameters: Type.Object(\n\t\t\t\t{\n\t\t\t\t\tsessionId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\taction: Type.Object(\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tkind: Type.Union([\n\t\t\t\t\t\t\t\tType.Literal(\"focus\"),\n\t\t\t\t\t\t\t\tType.Literal(\"click\"),\n\t\t\t\t\t\t\t\tType.Literal(\"type\"),\n\t\t\t\t\t\t\t\tType.Literal(\"key\"),\n\t\t\t\t\t\t\t\tType.Literal(\"scroll\"),\n\t\t\t\t\t\t\t]),\n\t\t\t\t\t\t\ttarget: Type.Union([\n\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"window\"),\n\t\t\t\t\t\t\t\t\t\twindowId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"control\"),\n\t\t\t\t\t\t\t\t\t\tcontrolId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t\twindowId: Type.Optional(\n\t\t\t\t\t\t\t\t\t\t\tType.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"coordinates\"),\n\t\t\t\t\t\t\t\t\t\tx: Type.Integer({ minimum: 0 }),\n\t\t\t\t\t\t\t\t\t\ty: Type.Integer({ minimum: 0 }),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t]),\n\t\t\t\t\t\t\ttext: Type.Optional(Type.String({ maxLength: DESKTOP_MAX_TYPE_TEXT_CHARS_V1 })),\n\t\t\t\t\t\t\tkey: Type.Optional(Type.String({ minLength: 1, maxLength: DESKTOP_MAX_TYPE_TEXT_CHARS_V1 })),\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t),\n\t\t\t\t\tpostconditions: Type.Optional(\n\t\t\t\t\t\tType.Array(\n\t\t\t\t\t\t\tType.Union([\n\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"window-exists\"),\n\t\t\t\t\t\t\t\t\t\twindowId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"control-exists\"),\n\t\t\t\t\t\t\t\t\t\tcontrolId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"focused\"),\n\t\t\t\t\t\t\t\t\t\tcontrolId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"property-equals\"),\n\t\t\t\t\t\t\t\t\t\ttarget: Type.Union([\n\t\t\t\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"window\"),\n\t\t\t\t\t\t\t\t\t\t\t\t\twindowId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"control\"),\n\t\t\t\t\t\t\t\t\t\t\t\t\tcontrolId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t\t\t\t\twindowId: Type.Optional(\n\t\t\t\t\t\t\t\t\t\t\t\t\t\tType.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\t\t\t]),\n\t\t\t\t\t\t\t\t\t\tproperty: Type.Union([\n\t\t\t\t\t\t\t\t\t\t\tType.Literal(\"enabled\"),\n\t\t\t\t\t\t\t\t\t\t\tType.Literal(\"value\"),\n\t\t\t\t\t\t\t\t\t\t\tType.Literal(\"name\"),\n\t\t\t\t\t\t\t\t\t\t]),\n\t\t\t\t\t\t\t\t\t\texpected: Type.Union([\n\t\t\t\t\t\t\t\t\t\t\tType.Boolean(),\n\t\t\t\t\t\t\t\t\t\t\tType.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t\t]),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t]),\n\t\t\t\t\t\t\t{ maxItems: DESKTOP_MAX_POSTCONDITIONS_V1 },\n\t\t\t\t\t\t),\n\t\t\t\t\t),\n\t\t\t\t},\n\t\t\t\t{ additionalProperties: false },\n\t\t\t),\n\t\t\texecute: async (first: unknown, second?: unknown, third?: unknown) => {\n\t\t\t\tconst { input, operationId } = toolInvocation(first, second);\n\t\t\t\tconst signal = signalFrom(third);\n\t\t\t\tif (signal?.aborted)\n\t\t\t\t\treturn desktopFailure(operationId, \"cancelled\", \"Desktop operation was cancelled\", \"cancelled\");\n\t\t\t\tlet request: DesktopOperationRequestV1;\n\t\t\t\ttry {\n\t\t\t\t\trequest = desktopExecuteRequest(input, operationId);\n\t\t\t\t} catch {\n\t\t\t\t\treturn desktopFailure(operationId, \"desktop_input_invalid\", \"Desktop action input is invalid\");\n\t\t\t\t}\n\t\t\t\ttry {\n\t\t\t\t\tconst result = runtime.execute(request);\n\t\t\t\t\treturn signal?.aborted\n\t\t\t\t\t\t? desktopFailure(operationId, \"cancelled\", \"Desktop operation was cancelled\", \"cancelled\")\n\t\t\t\t\t\t: desktopExecuteResult(result, request);\n\t\t\t\t} catch {\n\t\t\t\t\treturn desktopFailure(operationId, \"desktop_adapter_failed\", \"The selected desktop adapter failed\");\n\t\t\t\t}\n\t\t\t},\n\t\t});\n\t\tconst replay = runtime.replay;\n\t\tif (replay === undefined) return;\n\t\tapi.registerTool({\n\t\t\tname: \"desktop_replay\",\n\t\t\tlabel: \"Replay desktop actions\",\n\t\t\tdescription:\n\t\t\t\t\"Replay a bounded sequence of desktop actions against one session; stops at the first failure, cancellation, or user-confirmation pause by default\",\n\t\t\tparameters: Type.Object(\n\t\t\t\t{\n\t\t\t\t\tsessionId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\tactions: Type.Array(\n\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tkind: Type.Union([\n\t\t\t\t\t\t\t\t\tType.Literal(\"focus\"),\n\t\t\t\t\t\t\t\t\tType.Literal(\"click\"),\n\t\t\t\t\t\t\t\t\tType.Literal(\"type\"),\n\t\t\t\t\t\t\t\t\tType.Literal(\"key\"),\n\t\t\t\t\t\t\t\t\tType.Literal(\"scroll\"),\n\t\t\t\t\t\t\t\t]),\n\t\t\t\t\t\t\t\ttarget: Type.Union([\n\t\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"window\"),\n\t\t\t\t\t\t\t\t\t\t\twindowId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"control\"),\n\t\t\t\t\t\t\t\t\t\t\tcontrolId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t\t\twindowId: Type.Optional(\n\t\t\t\t\t\t\t\t\t\t\t\tType.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),\n\t\t\t\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\t\tType.Object(\n\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\tkind: Type.Literal(\"coordinates\"),\n\t\t\t\t\t\t\t\t\t\t\tx: Type.Integer({ minimum: 0 }),\n\t\t\t\t\t\t\t\t\t\t\ty: Type.Integer({ minimum: 0 }),\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t\t\t\t),\n\t\t\t\t\t\t\t\t]),\n\t\t\t\t\t\t\t\ttext: Type.Optional(Type.String({ maxLength: DESKTOP_MAX_TYPE_TEXT_CHARS_V1 })),\n\t\t\t\t\t\t\t\tkey: Type.Optional(Type.String({ minLength: 1, maxLength: DESKTOP_MAX_TYPE_TEXT_CHARS_V1 })),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{ additionalProperties: false },\n\t\t\t\t\t\t),\n\t\t\t\t\t\t{ minItems: 1, maxItems: DESKTOP_MAX_REPLAY_ACTIONS_V1 },\n\t\t\t\t\t),\n\t\t\t\t\tstopOnFailure: Type.Optional(Type.Boolean()),\n\t\t\t\t},\n\t\t\t\t{ additionalProperties: false },\n\t\t\t),\n\t\t\texecute: async (first: unknown, second?: unknown, third?: unknown) => {\n\t\t\t\tconst { input, operationId } = toolInvocation(first, second);\n\t\t\t\tconst signal = signalFrom(third);\n\t\t\t\tif (signal?.aborted)\n\t\t\t\t\treturn desktopFailure(operationId, \"cancelled\", \"Desktop operation was cancelled\", \"cancelled\");\n\t\t\t\tlet request: DesktopReplayRequestV1;\n\t\t\t\ttry {\n\t\t\t\t\trequest = desktopReplayRequest(input, operationId);\n\t\t\t\t} catch {\n\t\t\t\t\treturn desktopFailure(operationId, \"desktop_input_invalid\", \"Desktop replay input is invalid\");\n\t\t\t\t}\n\t\t\t\ttry {\n\t\t\t\t\tconst result = replay(request);\n\t\t\t\t\treturn signal?.aborted\n\t\t\t\t\t\t? desktopFailure(operationId, \"cancelled\", \"Desktop operation was cancelled\", \"cancelled\")\n\t\t\t\t\t\t: desktopReplayResult(result, operationId);\n\t\t\t\t} catch {\n\t\t\t\t\treturn desktopFailure(operationId, \"desktop_adapter_failed\", \"The selected desktop adapter failed\");\n\t\t\t\t}\n\t\t\t},\n\t\t});\n\t};\n}\n"]}

package/dist/capability.js

@@ -0,0 +1,456 @@
+import { Type } from "typebox";
+import { DESKTOP_MAX_POSTCONDITIONS_V1, DESKTOP_MAX_REPLAY_ACTIONS_V1, DESKTOP_MAX_STRING_CHARS_V1, DESKTOP_MAX_TYPE_TEXT_CHARS_V1, desktopBoundedString, desktopFailure, desktopRequiredString, desktopResultWithinBudget, projectDesktopObservation, } from "./projection.js";
+import { signalFrom, toolInvocation } from "./shared.js";
+function desktopCoordinates(input) {
+    const x = input.x;
+    const y = input.y;
+    if (typeof x !== "number" || typeof y !== "number" || !Number.isInteger(x) || !Number.isInteger(y) || x < 0 || y < 0)
+        throw new Error("desktop-target-invalid");
+    return { x, y };
+}
+function desktopTarget(input) {
+    if (input === null || typeof input !== "object" || Array.isArray(input))
+        throw new Error("desktop-target-invalid");
+    const target = input;
+    if (target.kind === "window") {
+        desktopExactKeys(target, ["kind", "windowId"]);
+        return { kind: "window", windowId: desktopRequiredString(target.windowId, "windowId") };
+    }
+    if (target.kind === "control") {
+        desktopExactKeys(target, ["kind", "controlId", "windowId"]);
+        return {
+            kind: "control",
+            controlId: desktopRequiredString(target.controlId, "controlId"),
+            ...(target.windowId === undefined ? {} : { windowId: desktopRequiredString(target.windowId, "windowId") }),
+        };
+    }
+    if (target.kind === "coordinates") {
+        desktopExactKeys(target, ["kind", "x", "y"]);
+        return { kind: "coordinates", ...desktopCoordinates(target) };
+    }
+    throw new Error("desktop-target-invalid");
+}
+function desktopExactKeys(value, allowed) {
+    if (Object.keys(value).some((key) => !allowed.includes(key)))
+        throw new Error("desktop-input-invalid");
+}
+function desktopPostconditions(input) {
+    if (input === undefined)
+        return undefined;
+    if (!Array.isArray(input) || input.length > DESKTOP_MAX_POSTCONDITIONS_V1)
+        throw new Error("desktop-postconditions-invalid");
+    return input.map((entry) => {
+        if (entry === null || typeof entry !== "object" || Array.isArray(entry))
+            throw new Error("desktop-postcondition-invalid");
+        const value = entry;
+        if (value.kind === "window-exists") {
+            desktopExactKeys(value, ["kind", "windowId"]);
+            return { kind: "window-exists", windowId: desktopRequiredString(value.windowId, "windowId") };
+        }
+        if (value.kind === "control-exists") {
+            desktopExactKeys(value, ["kind", "controlId"]);
+            return { kind: "control-exists", controlId: desktopRequiredString(value.controlId, "controlId") };
+        }
+        if (value.kind === "focused") {
+            desktopExactKeys(value, ["kind", "controlId"]);
+            return { kind: "focused", controlId: desktopRequiredString(value.controlId, "controlId") };
+        }
+        if (value.kind === "property-equals") {
+            desktopExactKeys(value, ["kind", "target", "property", "expected"]);
+            if (value.property !== "enabled" && value.property !== "value" && value.property !== "name")
+                throw new Error("desktop-postcondition-invalid");
+            if ((value.property === "enabled" && typeof value.expected !== "boolean") ||
+                (value.property !== "enabled" && typeof value.expected !== "string"))
+                throw new Error("desktop-postcondition-invalid");
+            return {
+                kind: "property-equals",
+                target: desktopTarget(value.target),
+                property: value.property,
+                expected: typeof value.expected === "string"
+                    ? desktopBoundedString(value.expected, "postcondition expected")
+                    : value.expected,
+            };
+        }
+        throw new Error("desktop-postcondition-invalid");
+    });
+}
+function desktopExecuteRequest(input, operationId) {
+    desktopExactKeys(input, ["sessionId", "action", "postconditions"]);
+    const actionInput = input.action;
+    if (actionInput === null || typeof actionInput !== "object" || Array.isArray(actionInput))
+        throw new Error("desktop-action-invalid");
+    const action = actionInput;
+    desktopExactKeys(action, ["kind", "target", "text", "key"]);
+    if (action.kind !== "focus" &&
+        action.kind !== "click" &&
+        action.kind !== "type" &&
+        action.kind !== "key" &&
+        action.kind !== "scroll")
+        throw new Error("desktop-action-invalid");
+    const target = desktopTarget(action.target);
+    if (target === undefined)
+        throw new Error("desktop-target-invalid");
+    // Coordinate targets are a click-only affordance: pointing elsewhere is a
+    // caller contract violation, not a fallback path.
+    if (target.kind === "coordinates" && action.kind !== "click")
+        throw new Error("desktop-coordinates-click-only");
+    const text = action.kind === "type" || action.kind === "scroll" ? desktopBoundedString(action.text, "text") : undefined;
+    if (text !== undefined && text.length > DESKTOP_MAX_TYPE_TEXT_CHARS_V1)
+        throw new Error("desktop-type-text-budget");
+    const key = action.kind === "key" ? desktopBoundedString(action.key, "key") : undefined;
+    if (key !== undefined && key.length > DESKTOP_MAX_TYPE_TEXT_CHARS_V1)
+        throw new Error("desktop-type-text-budget");
+    const postconditions = desktopPostconditions(input.postconditions);
+    return {
+        version: 1,
+        operationId,
+        sessionId: desktopRequiredString(input.sessionId, "sessionId"),
+        action: {
+            version: 1,
+            kind: action.kind,
+            target,
+            ...(text === undefined ? {} : { text }),
+            ...(key === undefined ? {} : { key }),
+        },
+        ...(postconditions === undefined ? {} : { postconditions }),
+        // Input plugin operations always request a fresh observation; callers may
+        // not turn this off because target identity otherwise goes stale.
+        reobserveAfterAction: true,
+    };
+}
+function desktopVerification(value) {
+    if (value === undefined)
+        return undefined;
+    if (value === null || typeof value !== "object" || Array.isArray(value))
+        throw new Error("desktop-result-invalid");
+    const verification = value;
+    if ((verification.status !== "passed" && verification.status !== "failed" && verification.status !== "unknown") ||
+        !Array.isArray(verification.checks) ||
+        verification.checks.length > DESKTOP_MAX_POSTCONDITIONS_V1)
+        throw new Error("desktop-result-invalid");
+    return {
+        status: verification.status,
+        checks: verification.checks.map((check) => desktopBoundedString(check, "verification check")),
+    };
+}
+function desktopExecuteResult(value, request) {
+    if (value === null || typeof value !== "object" || Array.isArray(value))
+        throw new Error("desktop-result-invalid");
+    const result = value;
+    if (result.version !== 1 ||
+        result.operationId !== request.operationId ||
+        (result.status !== "succeeded" &&
+            result.status !== "failed" &&
+            result.status !== "cancelled" &&
+            result.status !== "needs_user" &&
+            result.status !== "unsupported"))
+        throw new Error("desktop-result-invalid");
+    const observation = result.observation === undefined ? undefined : projectDesktopObservation(result.observation);
+    if (result.status === "succeeded" &&
+        (observation === undefined || observation.observation.sessionId !== request.sessionId)) {
+        return desktopFailure(request.operationId, "verification_unavailable", "Desktop action did not produce a fresh observation");
+    }
+    const base = {
+        version: 1,
+        operationId: request.operationId,
+        status: result.status,
+        summary: result.status === "succeeded" ? "Desktop action completed" : "Desktop action could not be completed",
+        ...(observation === undefined ? {} : observation),
+        ...(desktopVerification(result.verification) === undefined
+            ? {}
+            : { verification: desktopVerification(result.verification) }),
+    };
+    if (result.status === "succeeded")
+        return desktopResultWithinBudget(base);
+    const error = result.error;
+    if (error === null || typeof error !== "object" || Array.isArray(error))
+        throw new Error("desktop-result-invalid");
+    const errorValue = error;
+    const causeCode = result.status === "needs_user" && typeof errorValue.causeCode === "string"
+        ? desktopBoundedString(errorValue.causeCode, "error causeCode")
+        : undefined;
+    return desktopResultWithinBudget({
+        ...base,
+        error: {
+            version: 1,
+            code: result.status === "needs_user" ? "needs_user" : "desktop_action_failed",
+            message: result.status === "needs_user" ? "Desktop action requires user action" : "Desktop action failed",
+            ...(causeCode === undefined ? {} : { causeCode }),
+        },
+    });
+}
+function desktopReplayActionInput(raw) {
+    if (raw === null || typeof raw !== "object" || Array.isArray(raw))
+        throw new Error("desktop-action-invalid");
+    const action = raw;
+    desktopExactKeys(action, ["kind", "target", "text", "key"]);
+    if (action.kind !== "focus" &&
+        action.kind !== "click" &&
+        action.kind !== "type" &&
+        action.kind !== "key" &&
+        action.kind !== "scroll")
+        throw new Error("desktop-action-invalid");
+    const target = desktopTarget(action.target);
+    if (target === undefined)
+        throw new Error("desktop-target-invalid");
+    if (target.kind === "coordinates" && action.kind !== "click")
+        throw new Error("desktop-coordinates-click-only");
+    const text = action.kind === "type" || action.kind === "scroll" ? desktopBoundedString(action.text, "text") : undefined;
+    if (text !== undefined && text.length > DESKTOP_MAX_TYPE_TEXT_CHARS_V1)
+        throw new Error("desktop-type-text-budget");
+    const key = action.kind === "key" ? desktopBoundedString(action.key, "key") : undefined;
+    if (key !== undefined && key.length > DESKTOP_MAX_TYPE_TEXT_CHARS_V1)
+        throw new Error("desktop-type-text-budget");
+    return {
+        version: 1,
+        kind: action.kind,
+        target,
+        ...(text === undefined ? {} : { text }),
+        ...(key === undefined ? {} : { key }),
+    };
+}
+function desktopReplayRequest(input, operationId) {
+    desktopExactKeys(input, ["sessionId", "actions", "stopOnFailure"]);
+    if (!Array.isArray(input.actions) ||
+        input.actions.length === 0 ||
+        input.actions.length > DESKTOP_MAX_REPLAY_ACTIONS_V1)
+        throw new Error("desktop-replay-actions-invalid");
+    const sessionId = desktopRequiredString(input.sessionId, "sessionId");
+    const operations = input.actions.map((raw, index) => {
+        const action = desktopReplayActionInput(raw);
+        return {
+            version: 1,
+            operationId: `${operationId}-${index}`,
+            sessionId,
+            action,
+            // Replay always re-observes so per-action results carry fresh state.
+            reobserveAfterAction: true,
+        };
+    });
+    return {
+        version: 1,
+        sessionId,
+        operations,
+        stopOnFailure: input.stopOnFailure === undefined ? true : input.stopOnFailure === true,
+    };
+}
+function desktopReplayResult(value, operationId) {
+    if (value === null || typeof value !== "object" || Array.isArray(value))
+        throw new Error("desktop-result-invalid");
+    const result = value;
+    if (result.version !== 1 ||
+        (result.status !== "succeeded" &&
+            result.status !== "failed" &&
+            result.status !== "cancelled" &&
+            result.status !== "needs_user") ||
+        !Array.isArray(result.results))
+        throw new Error("desktop-result-invalid");
+    const results = result.results.map((entry) => {
+        if (entry === null || typeof entry !== "object" || Array.isArray(entry))
+            throw new Error("desktop-result-invalid");
+        const record = entry;
+        if (record.version !== 1 ||
+            typeof record.operationId !== "string" ||
+            (record.status !== "succeeded" &&
+                record.status !== "failed" &&
+                record.status !== "cancelled" &&
+                record.status !== "needs_user" &&
+                record.status !== "unsupported"))
+            throw new Error("desktop-result-invalid");
+        return {
+            operationId: desktopBoundedString(record.operationId, "replay operationId"),
+            status: record.status,
+            summary: desktopBoundedString(record.summary, "replay summary"),
+        };
+    });
+    const last = result.results[result.results.length - 1];
+    const observation = last?.observation === undefined ? undefined : projectDesktopObservation(last.observation).observation;
+    const needsUser = result.results.find((entry) => entry.status === "needs_user");
+    const causeCode = needsUser !== undefined &&
+        needsUser.error !== null &&
+        typeof needsUser.error === "object" &&
+        "causeCode" in needsUser.error &&
+        typeof needsUser.error.causeCode === "string"
+        ? desktopBoundedString(needsUser.error.causeCode, "replay causeCode")
+        : undefined;
+    return desktopResultWithinBudget({
+        version: 1,
+        operationId,
+        status: result.status,
+        summary: result.status === "succeeded" ? "Desktop replay completed" : "Desktop replay stopped before completion",
+        results,
+        ...(observation === undefined ? {} : { observation }),
+        ...(result.status === "needs_user" && causeCode !== undefined
+            ? {
+                error: {
+                    version: 1,
+                    code: "needs_user",
+                    message: "Desktop replay requires user action",
+                    causeCode,
+                },
+            }
+            : {}),
+    });
+}
+export function createDesktopInputCapabilityFactoryV1(runtime) {
+    return (api) => {
+        if (api.host.mode === "readonly")
+            return;
+        api.registerTool({
+            name: "desktop_execute",
+            label: "Execute desktop action",
+            description: "Execute one bounded Accessibility focus, click, type, key, or scroll action against an observed target (click also accepts screen coordinates) and return a fresh bounded observation",
+            parameters: Type.Object({
+                sessionId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                action: Type.Object({
+                    kind: Type.Union([
+                        Type.Literal("focus"),
+                        Type.Literal("click"),
+                        Type.Literal("type"),
+                        Type.Literal("key"),
+                        Type.Literal("scroll"),
+                    ]),
+                    target: Type.Union([
+                        Type.Object({
+                            kind: Type.Literal("window"),
+                            windowId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                        }, { additionalProperties: false }),
+                        Type.Object({
+                            kind: Type.Literal("control"),
+                            controlId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                            windowId: Type.Optional(Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 })),
+                        }, { additionalProperties: false }),
+                        Type.Object({
+                            kind: Type.Literal("coordinates"),
+                            x: Type.Integer({ minimum: 0 }),
+                            y: Type.Integer({ minimum: 0 }),
+                        }, { additionalProperties: false }),
+                    ]),
+                    text: Type.Optional(Type.String({ maxLength: DESKTOP_MAX_TYPE_TEXT_CHARS_V1 })),
+                    key: Type.Optional(Type.String({ minLength: 1, maxLength: DESKTOP_MAX_TYPE_TEXT_CHARS_V1 })),
+                }, { additionalProperties: false }),
+                postconditions: Type.Optional(Type.Array(Type.Union([
+                    Type.Object({
+                        kind: Type.Literal("window-exists"),
+                        windowId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                    }, { additionalProperties: false }),
+                    Type.Object({
+                        kind: Type.Literal("control-exists"),
+                        controlId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                    }, { additionalProperties: false }),
+                    Type.Object({
+                        kind: Type.Literal("focused"),
+                        controlId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                    }, { additionalProperties: false }),
+                    Type.Object({
+                        kind: Type.Literal("property-equals"),
+                        target: Type.Union([
+                            Type.Object({
+                                kind: Type.Literal("window"),
+                                windowId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                            }, { additionalProperties: false }),
+                            Type.Object({
+                                kind: Type.Literal("control"),
+                                controlId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                                windowId: Type.Optional(Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 })),
+                            }, { additionalProperties: false }),
+                        ]),
+                        property: Type.Union([
+                            Type.Literal("enabled"),
+                            Type.Literal("value"),
+                            Type.Literal("name"),
+                        ]),
+                        expected: Type.Union([
+                            Type.Boolean(),
+                            Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                        ]),
+                    }, { additionalProperties: false }),
+                ]), { maxItems: DESKTOP_MAX_POSTCONDITIONS_V1 })),
+            }, { additionalProperties: false }),
+            execute: async (first, second, third) => {
+                const { input, operationId } = toolInvocation(first, second);
+                const signal = signalFrom(third);
+                if (signal?.aborted)
+                    return desktopFailure(operationId, "cancelled", "Desktop operation was cancelled", "cancelled");
+                let request;
+                try {
+                    request = desktopExecuteRequest(input, operationId);
+                }
+                catch {
+                    return desktopFailure(operationId, "desktop_input_invalid", "Desktop action input is invalid");
+                }
+                try {
+                    const result = runtime.execute(request);
+                    return signal?.aborted
+                        ? desktopFailure(operationId, "cancelled", "Desktop operation was cancelled", "cancelled")
+                        : desktopExecuteResult(result, request);
+                }
+                catch {
+                    return desktopFailure(operationId, "desktop_adapter_failed", "The selected desktop adapter failed");
+                }
+            },
+        });
+        const replay = runtime.replay;
+        if (replay === undefined)
+            return;
+        api.registerTool({
+            name: "desktop_replay",
+            label: "Replay desktop actions",
+            description: "Replay a bounded sequence of desktop actions against one session; stops at the first failure, cancellation, or user-confirmation pause by default",
+            parameters: Type.Object({
+                sessionId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                actions: Type.Array(Type.Object({
+                    kind: Type.Union([
+                        Type.Literal("focus"),
+                        Type.Literal("click"),
+                        Type.Literal("type"),
+                        Type.Literal("key"),
+                        Type.Literal("scroll"),
+                    ]),
+                    target: Type.Union([
+                        Type.Object({
+                            kind: Type.Literal("window"),
+                            windowId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                        }, { additionalProperties: false }),
+                        Type.Object({
+                            kind: Type.Literal("control"),
+                            controlId: Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 }),
+                            windowId: Type.Optional(Type.String({ minLength: 1, maxLength: DESKTOP_MAX_STRING_CHARS_V1 })),
+                        }, { additionalProperties: false }),
+                        Type.Object({
+                            kind: Type.Literal("coordinates"),
+                            x: Type.Integer({ minimum: 0 }),
+                            y: Type.Integer({ minimum: 0 }),
+                        }, { additionalProperties: false }),
+                    ]),
+                    text: Type.Optional(Type.String({ maxLength: DESKTOP_MAX_TYPE_TEXT_CHARS_V1 })),
+                    key: Type.Optional(Type.String({ minLength: 1, maxLength: DESKTOP_MAX_TYPE_TEXT_CHARS_V1 })),
+                }, { additionalProperties: false }), { minItems: 1, maxItems: DESKTOP_MAX_REPLAY_ACTIONS_V1 }),
+                stopOnFailure: Type.Optional(Type.Boolean()),
+            }, { additionalProperties: false }),
+            execute: async (first, second, third) => {
+                const { input, operationId } = toolInvocation(first, second);
+                const signal = signalFrom(third);
+                if (signal?.aborted)
+                    return desktopFailure(operationId, "cancelled", "Desktop operation was cancelled", "cancelled");
+                let request;
+                try {
+                    request = desktopReplayRequest(input, operationId);
+                }
+                catch {
+                    return desktopFailure(operationId, "desktop_input_invalid", "Desktop replay input is invalid");
+                }
+                try {
+                    const result = replay(request);
+                    return signal?.aborted
+                        ? desktopFailure(operationId, "cancelled", "Desktop operation was cancelled", "cancelled")
+                        : desktopReplayResult(result, operationId);
+                }
+                catch {
+                    return desktopFailure(operationId, "desktop_adapter_failed", "The selected desktop adapter failed");
+                }
+            },
+        });
+    };
+}
+//# sourceMappingURL=capability.js.map