@bastani/atomic 0.9.28-alpha.3 → 0.9.28-alpha.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +39 -0
- package/dist/builtin/intercom/CHANGELOG.md +6 -0
- package/dist/builtin/intercom/README.md +3 -2
- package/dist/builtin/intercom/broker/broker.ts +90 -14
- package/dist/builtin/intercom/broker/paths.ts +4 -0
- package/dist/builtin/intercom/broker/socket-liveness.ts +32 -0
- package/dist/builtin/intercom/broker/socket-replacement-lock.ts +104 -0
- package/dist/builtin/intercom/broker/spawn.ts +108 -63
- package/dist/builtin/intercom/index.bundle.mjs +114 -61
- package/dist/builtin/intercom/package.json +2 -2
- package/dist/builtin/mcp/package.json +1 -1
- package/dist/builtin/subagents/CHANGELOG.md +6 -0
- package/dist/builtin/subagents/package.json +3 -3
- package/dist/builtin/subagents/skills/agent-browser/SKILL.md +1 -1
- package/dist/builtin/subagents/skills/cua-driver/BROWSER.md +60 -39
- package/dist/builtin/subagents/skills/cua-driver/EMBEDDING.md +119 -36
- package/dist/builtin/subagents/skills/cua-driver/LINUX.md +225 -183
- package/dist/builtin/subagents/skills/cua-driver/MACOS.md +120 -89
- package/dist/builtin/subagents/skills/cua-driver/README.md +16 -7
- package/dist/builtin/subagents/skills/cua-driver/RECORDING.md +85 -115
- package/dist/builtin/subagents/skills/cua-driver/RUNTIME.md +113 -0
- package/dist/builtin/subagents/skills/cua-driver/SKILL.md +55 -1071
- package/dist/builtin/subagents/skills/cua-driver/VISUAL.md +125 -0
- package/dist/builtin/subagents/skills/cua-driver/WINDOWS.md +145 -185
- package/dist/builtin/subagents/skills/cua-driver/WORKFLOW.md +173 -0
- package/dist/builtin/web-access/package.json +2 -2
- package/dist/builtin/workflows/CHANGELOG.md +16 -0
- package/dist/builtin/workflows/README.md +1 -1
- package/dist/builtin/workflows/builtin/{chunk-aqv8ny38.js → chunk-9ywkdsa9.js} +1 -1
- package/dist/builtin/workflows/builtin/{chunk-p6jagvyf.js → chunk-bnz4knhy.js} +1 -1
- package/dist/builtin/workflows/builtin/{chunk-br4ac9a3.js → chunk-dx5sjaxe.js} +1 -1
- package/dist/builtin/workflows/builtin/{chunk-p3xmajmj.js → chunk-v4scsq3y.js} +41 -13
- package/dist/builtin/workflows/builtin/goal.js +2 -2
- package/dist/builtin/workflows/builtin/index.js +3 -3
- package/dist/builtin/workflows/builtin/ralph.js +2 -2
- package/dist/builtin/workflows/package.json +2 -2
- package/dist/builtin/workflows/skills/impeccable/SKILL.md +8 -2
- package/dist/builtin/workflows/skills/impeccable/reference/adapt.md +6 -0
- package/dist/builtin/workflows/skills/impeccable/reference/audit.md +2 -1
- package/dist/builtin/workflows/skills/impeccable/reference/component-review.md +63 -0
- package/dist/builtin/workflows/skills/impeccable/reference/degraded/asset-producer.md +5 -0
- package/dist/builtin/workflows/skills/impeccable/reference/degraded/finish-reviewer.md +1 -1
- package/dist/builtin/workflows/skills/impeccable/reference/generate.md +101 -0
- package/dist/builtin/workflows/skills/impeccable/reference/harden.md +9 -0
- package/dist/builtin/workflows/skills/impeccable/reference/hooks.md +3 -1
- package/dist/builtin/workflows/skills/impeccable/reference/mode-operate.md +21 -0
- package/dist/builtin/workflows/skills/impeccable/reference/mode-persuade.md +19 -0
- package/dist/builtin/workflows/skills/impeccable/reference/mode-read.md +21 -0
- package/dist/builtin/workflows/skills/impeccable/reference/new-work.md +19 -12
- package/dist/builtin/workflows/skills/impeccable/reference/region-map.md +26 -0
- package/dist/builtin/workflows/skills/impeccable/reference/routing.md +1 -1
- package/dist/builtin/workflows/skills/impeccable/reference/visualize.md +5 -5
- package/dist/builtin/workflows/skills/impeccable/scripts/VERSION +1 -1
- package/dist/builtin/workflows/skills/impeccable/scripts/command-metadata.json +4 -0
- package/dist/builtin/workflows/skills/impeccable/scripts/impeccable.cmd +2 -2
- package/dist/builtin/workflows/skills/impeccable/scripts/live-browser-session.js +5 -1
- package/dist/builtin/workflows/skills/impeccable/scripts/live-browser.js +524 -4
- package/dist/builtin/workflows/skills/prompt-engineer/SKILL.md +1 -0
- package/dist/builtin/workflows/skills/prompt-engineer/references/claude_haiku_5_5.md +81 -0
- package/dist/builtin/workflows/src/extension/index.bundle.mjs +38 -14
- package/dist/builtin/workflows/src/index.js +38 -14
- package/dist/cli/startup-ui.d.ts +3 -2
- package/dist/cli/startup-ui.d.ts.map +1 -1
- package/dist/cli/startup-ui.js +19 -4
- package/dist/cli/startup-ui.js.map +1 -1
- package/dist/codemode-worker.js +2 -2
- package/dist/core/agent-session-compaction.d.ts +3 -3
- package/dist/core/agent-session-compaction.d.ts.map +1 -1
- package/dist/core/agent-session-compaction.js +5 -1
- package/dist/core/agent-session-compaction.js.map +1 -1
- package/dist/core/agent-session-events.d.ts.map +1 -1
- package/dist/core/agent-session-events.js +1 -0
- package/dist/core/agent-session-events.js.map +1 -1
- package/dist/core/agent-session-extension-bindings.d.ts.map +1 -1
- package/dist/core/agent-session-extension-bindings.js +4 -0
- package/dist/core/agent-session-extension-bindings.js.map +1 -1
- package/dist/core/agent-session-methods.d.ts +6 -1
- package/dist/core/agent-session-methods.d.ts.map +1 -1
- package/dist/core/agent-session-methods.js.map +1 -1
- package/dist/core/agent-session-prompt.d.ts.map +1 -1
- package/dist/core/agent-session-prompt.js +3 -2
- package/dist/core/agent-session-prompt.js.map +1 -1
- package/dist/core/agent-session-runtime.d.ts +5 -1
- package/dist/core/agent-session-runtime.d.ts.map +1 -1
- package/dist/core/agent-session-runtime.js +13 -9
- package/dist/core/agent-session-runtime.js.map +1 -1
- package/dist/core/agent-session-types.d.ts +1 -0
- package/dist/core/agent-session-types.d.ts.map +1 -1
- package/dist/core/agent-session-types.js.map +1 -1
- package/dist/core/agent-session.d.ts +1 -0
- package/dist/core/agent-session.d.ts.map +1 -1
- package/dist/core/agent-session.js.map +1 -1
- package/dist/core/export-html/tool-renderer.d.ts.map +1 -1
- package/dist/core/export-html/tool-renderer.js +2 -0
- package/dist/core/export-html/tool-renderer.js.map +1 -1
- package/dist/core/extensions/agent-events.d.ts +2 -0
- package/dist/core/extensions/agent-events.d.ts.map +1 -1
- package/dist/core/extensions/agent-events.js.map +1 -1
- package/dist/core/extensions/context-types.d.ts +2 -0
- package/dist/core/extensions/context-types.d.ts.map +1 -1
- package/dist/core/extensions/context-types.js.map +1 -1
- package/dist/core/extensions/mcp-server-registry.d.ts +4 -1
- package/dist/core/extensions/mcp-server-registry.d.ts.map +1 -1
- package/dist/core/extensions/mcp-server-registry.js +7 -4
- package/dist/core/extensions/mcp-server-registry.js.map +1 -1
- package/dist/core/extensions/runner.d.ts +3 -0
- package/dist/core/extensions/runner.d.ts.map +1 -1
- package/dist/core/extensions/runner.js +19 -0
- package/dist/core/extensions/runner.js.map +1 -1
- package/dist/core/extensions/session-events.d.ts +2 -0
- package/dist/core/extensions/session-events.d.ts.map +1 -1
- package/dist/core/extensions/session-events.js.map +1 -1
- package/dist/core/extensions/tool-types.d.ts +2 -0
- package/dist/core/extensions/tool-types.d.ts.map +1 -1
- package/dist/core/extensions/tool-types.js.map +1 -1
- package/dist/core/fast-model-variants.d.ts.map +1 -1
- package/dist/core/fast-model-variants.js +3 -2
- package/dist/core/fast-model-variants.js.map +1 -1
- package/dist/core/model-resolver-defaults.d.ts.map +1 -1
- package/dist/core/model-resolver-defaults.js +2 -0
- package/dist/core/model-resolver-defaults.js.map +1 -1
- package/dist/core/nested-tool-calls.d.ts +1 -0
- package/dist/core/nested-tool-calls.d.ts.map +1 -1
- package/dist/core/nested-tool-calls.js +1 -0
- package/dist/core/nested-tool-calls.js.map +1 -1
- package/dist/core/provider-alias.d.ts.map +1 -1
- package/dist/core/provider-alias.js +8 -0
- package/dist/core/provider-alias.js.map +1 -1
- package/dist/core/provider-composer-internal.d.ts.map +1 -1
- package/dist/core/provider-composer-internal.js +6 -3
- package/dist/core/provider-composer-internal.js.map +1 -1
- package/dist/core/resource-loader-reload.d.ts.map +1 -1
- package/dist/core/resource-loader-reload.js +1 -0
- package/dist/core/resource-loader-reload.js.map +1 -1
- package/dist/core/resource-loader-types.d.ts +1 -0
- package/dist/core/resource-loader-types.d.ts.map +1 -1
- package/dist/core/resource-loader-types.js.map +1 -1
- package/dist/core/settings-types.js.map +1 -1
- package/dist/core/tools/bash.d.ts.map +1 -1
- package/dist/core/tools/bash.js +7 -3
- package/dist/core/tools/bash.js.map +1 -1
- package/dist/core/tools/edit.js +1 -1
- package/dist/core/tools/edit.js.map +1 -1
- package/dist/extensions/codemode/execute.d.ts.map +1 -1
- package/dist/extensions/codemode/execute.js +55 -7
- package/dist/extensions/codemode/execute.js.map +1 -1
- package/dist/extensions/codemode/index.d.ts.map +1 -1
- package/dist/extensions/codemode/index.js +7 -0
- package/dist/extensions/codemode/index.js.map +1 -1
- package/dist/extensions/codemode/tool.d.ts +2 -0
- package/dist/extensions/codemode/tool.d.ts.map +1 -1
- package/dist/extensions/codemode/tool.js +2 -0
- package/dist/extensions/codemode/tool.js.map +1 -1
- package/dist/extensions/herdr/index.d.ts.map +1 -1
- package/dist/extensions/herdr/index.js +20 -2
- package/dist/extensions/herdr/index.js.map +1 -1
- package/dist/extensions/herdr/pane-owner.d.ts +7 -0
- package/dist/extensions/herdr/pane-owner.d.ts.map +1 -1
- package/dist/extensions/herdr/pane-owner.js +53 -11
- package/dist/extensions/herdr/pane-owner.js.map +1 -1
- package/dist/index.d.ts +1 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js.map +1 -1
- package/dist/main-deferred-startup.d.ts +8 -0
- package/dist/main-deferred-startup.d.ts.map +1 -1
- package/dist/main-deferred-startup.js +6 -0
- package/dist/main-deferred-startup.js.map +1 -1
- package/dist/main.d.ts.map +1 -1
- package/dist/main.js +11 -3
- package/dist/main.js.map +1 -1
- package/dist/modes/interactive/components/bash-execution.d.ts +5 -1
- package/dist/modes/interactive/components/bash-execution.d.ts.map +1 -1
- package/dist/modes/interactive/components/bash-execution.js +17 -14
- package/dist/modes/interactive/components/bash-execution.js.map +1 -1
- package/dist/modes/interactive/components/branch-summary-message.d.ts +2 -1
- package/dist/modes/interactive/components/branch-summary-message.d.ts.map +1 -1
- package/dist/modes/interactive/components/branch-summary-message.js +5 -2
- package/dist/modes/interactive/components/branch-summary-message.js.map +1 -1
- package/dist/modes/interactive/components/chat-message-renderer.d.ts +1 -0
- package/dist/modes/interactive/components/chat-message-renderer.d.ts.map +1 -1
- package/dist/modes/interactive/components/chat-message-renderer.js +19 -8
- package/dist/modes/interactive/components/chat-message-renderer.js.map +1 -1
- package/dist/modes/interactive/components/compaction-boundary-message.d.ts +3 -2
- package/dist/modes/interactive/components/compaction-boundary-message.d.ts.map +1 -1
- package/dist/modes/interactive/components/compaction-boundary-message.js +7 -4
- package/dist/modes/interactive/components/compaction-boundary-message.js.map +1 -1
- package/dist/modes/interactive/components/custom-entry.d.ts +3 -1
- package/dist/modes/interactive/components/custom-entry.d.ts.map +1 -1
- package/dist/modes/interactive/components/custom-entry.js +8 -2
- package/dist/modes/interactive/components/custom-entry.js.map +1 -1
- package/dist/modes/interactive/components/custom-message.d.ts.map +1 -1
- package/dist/modes/interactive/components/custom-message.js +1 -0
- package/dist/modes/interactive/components/custom-message.js.map +1 -1
- package/dist/modes/interactive/components/settings-selector-items.js +1 -1
- package/dist/modes/interactive/components/settings-selector-items.js.map +1 -1
- package/dist/modes/interactive/components/skill-invocation-message.d.ts +2 -1
- package/dist/modes/interactive/components/skill-invocation-message.d.ts.map +1 -1
- package/dist/modes/interactive/components/skill-invocation-message.js +5 -2
- package/dist/modes/interactive/components/skill-invocation-message.js.map +1 -1
- package/dist/modes/interactive/components/tool-execution.d.ts +4 -0
- package/dist/modes/interactive/components/tool-execution.d.ts.map +1 -1
- package/dist/modes/interactive/components/tool-execution.js +10 -0
- package/dist/modes/interactive/components/tool-execution.js.map +1 -1
- package/dist/modes/interactive/interactive-agent-events.js +3 -1
- package/dist/modes/interactive/interactive-agent-events.js.map +1 -1
- package/dist/modes/interactive/interactive-auth-login.js +6 -0
- package/dist/modes/interactive/interactive-auth-login.js.map +1 -1
- package/dist/modes/interactive/interactive-auth-routing.js +2 -2
- package/dist/modes/interactive/interactive-auth-routing.js.map +1 -1
- package/dist/modes/interactive/interactive-bash-compact.js +2 -2
- package/dist/modes/interactive/interactive-bash-compact.js.map +1 -1
- package/dist/modes/interactive/interactive-extension-dialogs.js +19 -2
- package/dist/modes/interactive/interactive-extension-dialogs.js.map +1 -1
- package/dist/modes/interactive/interactive-mode-base.d.ts +4 -0
- package/dist/modes/interactive/interactive-mode-base.d.ts.map +1 -1
- package/dist/modes/interactive/interactive-mode-base.js +7 -0
- package/dist/modes/interactive/interactive-mode-base.js.map +1 -1
- package/dist/modes/interactive/interactive-mode-helpers.js +1 -1
- package/dist/modes/interactive/interactive-mode-helpers.js.map +1 -1
- package/dist/modes/interactive/interactive-mode-surface.d.ts +2 -1
- package/dist/modes/interactive/interactive-mode-surface.d.ts.map +1 -1
- package/dist/modes/interactive/interactive-mode-surface.js.map +1 -1
- package/dist/modes/interactive/interactive-mode-types.d.ts +1 -0
- package/dist/modes/interactive/interactive-mode-types.d.ts.map +1 -1
- package/dist/modes/interactive/interactive-mode-types.js.map +1 -1
- package/dist/modes/interactive/interactive-process-lifecycle.js +1 -1
- package/dist/modes/interactive/interactive-process-lifecycle.js.map +1 -1
- package/dist/modes/interactive/interactive-render-chat.js +7 -5
- package/dist/modes/interactive/interactive-render-chat.js.map +1 -1
- package/dist/modes/interactive/interactive-selectors.js +8 -1
- package/dist/modes/interactive/interactive-selectors.js.map +1 -1
- package/dist/modes/interactive/interactive-session-runtime.js +1 -0
- package/dist/modes/interactive/interactive-session-runtime.js.map +1 -1
- package/dist/modes/interactive/interactive-startup-trust.d.ts +6 -0
- package/dist/modes/interactive/interactive-startup-trust.d.ts.map +1 -0
- package/dist/modes/interactive/interactive-startup-trust.js +15 -0
- package/dist/modes/interactive/interactive-startup-trust.js.map +1 -0
- package/dist/modes/interactive/interactive-startup.d.ts.map +1 -1
- package/dist/modes/interactive/interactive-startup.js +4 -0
- package/dist/modes/interactive/interactive-startup.js.map +1 -1
- package/dist/modes/interactive/program-status-reporter.d.ts +24 -0
- package/dist/modes/interactive/program-status-reporter.d.ts.map +1 -0
- package/dist/modes/interactive/program-status-reporter.js +96 -0
- package/dist/modes/interactive/program-status-reporter.js.map +1 -0
- package/dist/modes/interactive-engine/engine-child-liveness.d.ts +1 -0
- package/dist/modes/interactive-engine/engine-child-liveness.d.ts.map +1 -1
- package/dist/modes/interactive-engine/engine-child-liveness.js +15 -1
- package/dist/modes/interactive-engine/engine-child-liveness.js.map +1 -1
- package/dist/modes/interactive-engine/engine-custom-ui.d.ts.map +1 -1
- package/dist/modes/interactive-engine/engine-custom-ui.js +1 -0
- package/dist/modes/interactive-engine/engine-custom-ui.js.map +1 -1
- package/dist/modes/interactive-engine/engine-monitor.d.ts +4 -0
- package/dist/modes/interactive-engine/engine-monitor.d.ts.map +1 -1
- package/dist/modes/interactive-engine/engine-monitor.js +17 -0
- package/dist/modes/interactive-engine/engine-monitor.js.map +1 -1
- package/dist/modes/interactive-engine/engine-quit-intent.d.ts +16 -0
- package/dist/modes/interactive-engine/engine-quit-intent.d.ts.map +1 -0
- package/dist/modes/interactive-engine/engine-quit-intent.js +27 -0
- package/dist/modes/interactive-engine/engine-quit-intent.js.map +1 -0
- package/dist/modes/interactive-engine/engine-render-service.d.ts.map +1 -1
- package/dist/modes/interactive-engine/engine-render-service.js +3 -1
- package/dist/modes/interactive-engine/engine-render-service.js.map +1 -1
- package/dist/modes/interactive-engine/extension-ui-bridge.d.ts +1 -0
- package/dist/modes/interactive-engine/extension-ui-bridge.d.ts.map +1 -1
- package/dist/modes/interactive-engine/extension-ui-bridge.js +5 -0
- package/dist/modes/interactive-engine/extension-ui-bridge.js.map +1 -1
- package/dist/modes/interactive-engine/isolated-runtime.d.ts +4 -1
- package/dist/modes/interactive-engine/isolated-runtime.d.ts.map +1 -1
- package/dist/modes/interactive-engine/isolated-runtime.js +19 -2
- package/dist/modes/interactive-engine/isolated-runtime.js.map +1 -1
- package/dist/modes/interactive-engine/protocol.d.ts +19 -1
- package/dist/modes/interactive-engine/protocol.d.ts.map +1 -1
- package/dist/modes/interactive-engine/protocol.js +7 -0
- package/dist/modes/interactive-engine/protocol.js.map +1 -1
- package/dist/modes/interactive-engine/remote-renderer.d.ts +3 -0
- package/dist/modes/interactive-engine/remote-renderer.d.ts.map +1 -1
- package/dist/modes/interactive-engine/remote-renderer.js +7 -0
- package/dist/modes/interactive-engine/remote-renderer.js.map +1 -1
- package/dist/modes/rpc/rpc-client.d.ts +11 -0
- package/dist/modes/rpc/rpc-client.d.ts.map +1 -1
- package/dist/modes/rpc/rpc-client.js +42 -0
- package/dist/modes/rpc/rpc-client.js.map +1 -1
- package/dist/modes/rpc/rpc-input-scheduler.js +2 -2
- package/dist/modes/rpc/rpc-input-scheduler.js.map +1 -1
- package/dist/modes/rpc/rpc-mode.d.ts.map +1 -1
- package/dist/modes/rpc/rpc-mode.js +23 -6
- package/dist/modes/rpc/rpc-mode.js.map +1 -1
- package/dist/modes/rpc/rpc-provider-auth.js +1 -1
- package/dist/modes/rpc/rpc-provider-auth.js.map +1 -1
- package/docs/codemode.md +10 -6
- package/docs/compaction.md +1 -1
- package/docs/computer-use.md +9 -9
- package/docs/custom-provider/streaming.md +4 -0
- package/docs/environment-variables.md +2 -0
- package/docs/extensions/api-reference.md +1 -1
- package/docs/extensions/authoring.md +2 -0
- package/docs/extensions/events.md +8 -1
- package/docs/getting-started/first-session.md +1 -1
- package/docs/herdr.md +11 -2
- package/docs/json.md +6 -1
- package/docs/models/evals.md +34 -3
- package/docs/models.md +1 -1
- package/docs/providers.md +10 -0
- package/docs/sdk.md +8 -2
- package/docs/settings.md +2 -2
- package/docs/skills.md +1 -1
- package/docs/terminal-setup.md +14 -0
- package/docs/workflows/authoring.md +3 -3
- package/examples/extensions/sandbox/package-lock.json +3 -3
- package/npm-shrinkwrap.json +67 -67
- package/package.json +8 -8
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# cua-driver
|
|
1
|
+
# cua-driver: Windows
|
|
2
2
|
|
|
3
3
|
Orchestrates Windows app automation via the `cua-driver` binary (`cua-driver.exe`). Whenever a user
|
|
4
4
|
asks to drive a native Windows app, follow the loop in this doc
|
|
5
|
-
rather than calling tools ad-hoc
|
|
5
|
+
rather than calling tools ad-hoc: the snapshot-before-action
|
|
6
6
|
invariant is not optional and silently breaks if you skip it.
|
|
7
7
|
|
|
8
8
|
`SKILL.md` in this directory describes the cross-platform core;
|
|
@@ -11,29 +11,31 @@ the snapshot invariant, MCP-vs-CLI choice, agent cursor overlay, and
|
|
|
11
11
|
recording flow are identical. The launch, click, and accessibility-
|
|
12
12
|
tree mechanics in this file replace the macOS ones.
|
|
13
13
|
|
|
14
|
-
##
|
|
14
|
+
## Background window delivery
|
|
15
15
|
|
|
16
|
-
**
|
|
16
|
+
**Background window actions must preserve the user's frontmost app.**
|
|
17
|
+
Foreground and desktop actions follow the authorization boundary in
|
|
18
|
+
[RUNTIME.md](RUNTIME.md#foreground-boundary). Users pay for the right
|
|
17
19
|
to keep typing in their editor while an agent drives another app in
|
|
18
20
|
the background. Violate this rule and every other nice property the
|
|
19
21
|
driver gives you (no cursor warp, no taskbar flash, no window
|
|
20
|
-
restore-and-raise) stops mattering
|
|
22
|
+
restore-and-raise) stops mattering: you just shipped a `SendInput`
|
|
21
23
|
wrapper with extra steps.
|
|
22
24
|
|
|
23
25
|
### How the contract is enforced per call: the `delivery_mode` field
|
|
24
26
|
|
|
25
27
|
Every Windows input tool (`click`, `double_click`, `right_click`,
|
|
26
28
|
`drag`, `scroll`, `press_key`, `hotkey`, `type_text`) accepts an
|
|
27
|
-
optional `delivery_mode` field
|
|
28
|
-
surface (same name, same two values). The default is `"background"
|
|
29
|
+
optional `delivery_mode` field: this mirrors the macOS `delivery_mode`
|
|
30
|
+
surface (same name, same two values). The default is `"background"`:
|
|
29
31
|
strict no-foreground:
|
|
30
32
|
|
|
31
33
|
| `delivery_mode` | Behavior on Windows |
|
|
32
34
|
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
33
|
-
| `"background"` (DEFAULT) | Never fronts and **never raises/restacks** the target
|
|
34
|
-
| `"foreground"` | SendInput with brief `SetForegroundWindow(target)` → restore. The explicit, agent-chosen rung where fronting IS allowed
|
|
35
|
+
| `"background"` (DEFAULT) | Never fronts and **never raises/restacks** the target: macOS-aligned (mirrors CGEvent-to-pid). **Pixel clicks**: a UIA hit-test at the point first (accessibility-channel Invoke, works on UWP / WinUI3 / Win11 packaged apps, no flash); if that misses, coordinate-injected pen/touch, **but only when the target is the _visible_ window at that point**; PostMessage for plain Win32. It returns a structured `background_unavailable` error (rather than raising or fronting) when the target is **occluded** at the point, or the event kind is known-dropped (Chromium DOM mouse + key-combos, GTK buttons, VCL/LibreOffice accelerators, terminal / WPF text with no `element_token`). **No foreground swap and no z-order raise, ever.** |
|
|
36
|
+
| `"foreground"` | SendInput with brief `SetForegroundWindow(target)` → restore. The explicit, agent-chosen rung where fronting IS allowed: required to reach occluded targets, Chromium DOM content, GTK buttons, VCL accelerators, WPF drag, terminals, and canvas / custom-drawn surfaces with no UIA peer. Implemented for **every** input tool: `type_text` (SendInput Unicode via `send_text_synthesized`) and `scroll` (SendInput wheel via `send_wheel_synthesized`) included. The activation and restoration are scoped to that action. |
|
|
35
37
|
|
|
36
|
-
> **macOS is the source of truth
|
|
38
|
+
> **macOS is the source of truth: `background` never alters the screen.**
|
|
37
39
|
> Earlier Windows builds "cheated" in background with three tricks that this
|
|
38
40
|
> pass **removed**: (1) a z-order raise (`ZorderGuard`) to win the pointer
|
|
39
41
|
> hit-test on occluded windows, (2) a full focus-activate for WPF drags, and
|
|
@@ -41,14 +43,14 @@ strict no-foreground:
|
|
|
41
43
|
> otherwise drop. macOS does none of these (pure CGEvent-to-pid + focus
|
|
42
44
|
> suppression), so Windows now does none either: when strict no-front /
|
|
43
45
|
> no-raise delivery can't land, the tool returns `background_unavailable` and
|
|
44
|
-
> **the agent
|
|
46
|
+
> **the agent (not the driver) decides** whether to escalate to
|
|
45
47
|
> `delivery_mode:"foreground"` (the rung where fronting is explicitly opted
|
|
46
48
|
> into). A `background` call will never raise, restack, flash, or steal focus.
|
|
47
49
|
|
|
48
50
|
> **Removed: the legacy `"auto"` mode.** Earlier builds had a third
|
|
49
51
|
> Windows-only `dispatch:"auto"` mode (silent SendInput fallback on
|
|
50
52
|
> known-problematic targets). It was removed in the macOS-alignment pass
|
|
51
|
-
> because it could front the target _without the caller opting in_
|
|
53
|
+
> because it could front the target _without the caller opting in_:
|
|
52
54
|
> breaking the no-foreground contract macOS guarantees. Any unrecognised
|
|
53
55
|
> value (including a stray `"auto"`) now resolves to `"background"`. If you
|
|
54
56
|
> have notes/snippets that pass `dispatch:"auto"`, switch to an explicit
|
|
@@ -60,7 +62,7 @@ The hit-test fallback above means **`delivery_mode:"background"` is the correct
|
|
|
60
62
|
default even when the target is a XAML host whose input stack drops raw
|
|
61
63
|
PostMessage** (UWP Calculator, Win11 Notepad, WinUI3 apps, etc.). cua-driver
|
|
62
64
|
turns the pixel coord into a UIA Invoke at that point and delivers
|
|
63
|
-
through the accessibility channel
|
|
65
|
+
through the accessibility channel: no flash, no focus steal. Only
|
|
64
66
|
escalate to `delivery_mode:"foreground"` when you actually see a
|
|
65
67
|
`background_unavailable` structured error. Retry only that action with
|
|
66
68
|
`delivery_mode:"foreground"`; cua-driver briefly activates the target, delivers
|
|
@@ -71,7 +73,7 @@ Calculator on Win11 (Number-pad buttons + operators) consistently
|
|
|
71
73
|
resolve through `try_invoke_in_window_at_point` and produce
|
|
72
74
|
`"✅ Performed UIA Invoke at (sx,sy) for pid X."` with zero visible
|
|
73
75
|
flash. `delivery_mode:"foreground"` on the same coords also works but
|
|
74
|
-
flashes the Calculator window foreground for ~40 ms
|
|
76
|
+
flashes the Calculator window foreground for ~40 ms: it's the
|
|
75
77
|
costlier path; only use it for surfaces with no UIA peer.
|
|
76
78
|
|
|
77
79
|
`background_unavailable` error shape:
|
|
@@ -97,7 +99,7 @@ macOS / X11, where a background pixel click can still land in the background.)
|
|
|
97
99
|
|
|
98
100
|
The normal flow when an agent gets that error:
|
|
99
101
|
|
|
100
|
-
1.
|
|
102
|
+
1. Confirm the user authorized visible control, then reissue only the refused action with `delivery_mode:"foreground"`.
|
|
101
103
|
2. cua-driver activates the target, delivers through SendInput, and restores
|
|
102
104
|
the previous foreground.
|
|
103
105
|
3. Continue with `delivery_mode:"background"` for later actions unless they
|
|
@@ -113,7 +115,7 @@ the remote surface from accepting input.
|
|
|
113
115
|
The `bring_to_front` tool uses an `AttachThreadInput` trick to _attempt_
|
|
114
116
|
the foreground swap even when the daemon isn't at UIAccess integrity (the
|
|
115
117
|
same trick that powers `send_key_synthesized`). Returns
|
|
116
|
-
`{previous_fg_hwnd, now_fg_hwnd, landed_on_target}
|
|
118
|
+
`{previous_fg_hwnd, now_fg_hwnd, landed_on_target}`: **check
|
|
117
119
|
`landed_on_target`**. Without UIAccess, Windows' foreground-lock can still
|
|
118
120
|
reject the swap (and a subsequent `delivery_mode:"foreground"` call will
|
|
119
121
|
bail with the "Foreground swap … was rejected by Windows" diagnostic
|
|
@@ -122,7 +124,7 @@ target genuinely cannot be driven by SendInput/keystrokes in this session:
|
|
|
122
124
|
use an interactively launched High-IL daemon. The reserved `cua-driver-uia`
|
|
123
125
|
worker is a daemon-internal, default-off service boundary and public clients
|
|
124
126
|
must never connect to its pipe directly. Alternatively, for tasks
|
|
125
|
-
that produce a file
|
|
127
|
+
that produce a file: generate the document and `launch_app` it instead of
|
|
126
128
|
driving the GUI (e.g. building a spreadsheet and opening it in LibreOffice
|
|
127
129
|
Calc rather than typing into the grid, which is dropped on the VCL
|
|
128
130
|
background path).
|
|
@@ -134,63 +136,63 @@ is therefore forbidden unless the user **explicitly** asked for
|
|
|
134
136
|
frontmost state:
|
|
135
137
|
|
|
136
138
|
- **`Start-Process <exe>` / `Start-Process <url>` / `Start-Process
|
|
137
|
-
-FilePath
|
|
139
|
+
-FilePath ...`**: defaults to launching with `SW_SHOWNORMAL` which
|
|
138
140
|
_activates_ the new window. Windows treats new processes as
|
|
139
141
|
user-initiated foreground apps. The CmdLine flag `-WindowStyle Hidden`
|
|
140
142
|
helps but does not block activation for apps that call
|
|
141
143
|
`SetForegroundWindow` themselves on startup (Edge, most browsers,
|
|
142
144
|
Office, most installers). **Never use `Start-Process` to launch a
|
|
143
|
-
visible app.** Use `launch_app
|
|
145
|
+
visible app.** Use `launch_app`: it goes through
|
|
144
146
|
`SW_SHOWNOACTIVATE` and wraps an internal `VK_NONAME keybd_event`
|
|
145
147
|
trick that the OS treats as "user activity from another window," so
|
|
146
148
|
the target's own `SetForegroundWindow` call is denied per Windows
|
|
147
149
|
focus-stealing prevention rules.
|
|
148
|
-
- **`& "C:\path\to\app.exe"` from PowerShell
|
|
150
|
+
- **`& "C:\path\to\app.exe"` from PowerShell**: same as
|
|
149
151
|
`Start-Process` on activation. PowerShell's `&` invocation operator
|
|
150
152
|
spawns the process with foreground intent.
|
|
151
|
-
- **`cmd /c start <thing>` and `start /b <thing
|
|
153
|
+
- **`cmd /c start <thing>` and `start /b <thing>`**: `start` on
|
|
152
154
|
Windows is the equivalent of macOS's `open`: it goes through the
|
|
153
155
|
shell's protocol-handler / file-association lookup and activates
|
|
154
156
|
the resulting process. `start /min` helps for taskbar minimization
|
|
155
157
|
but still activates the new window before minimizing it (flash
|
|
156
158
|
visible to the user). Forbidden for the same reason.
|
|
157
159
|
- **`explorer.exe shell:AppsFolder\<AUMID>` / `explorer.exe ms-edge:
|
|
158
|
-
<url
|
|
160
|
+
<url>`**: these are the Windows-shell equivalents of `open -a` /
|
|
159
161
|
`open <url>` on macOS. They go through `IApplicationActivationManager`
|
|
160
162
|
with the wrong activation kind and foreground the target. Use
|
|
161
|
-
`launch_app({aumid})` or `launch_app({urls})` instead
|
|
163
|
+
`launch_app({aumid})` or `launch_app({urls})` instead: those route
|
|
162
164
|
through `IApplicationActivationManager::ActivateApplication` with
|
|
163
165
|
the correct flag combination that respects `SW_SHOWNOACTIVATE`.
|
|
164
166
|
- **`SetForegroundWindow(hwnd)` / `BringWindowToTop(hwnd)` /
|
|
165
167
|
`SwitchToThisWindow(hwnd, fAltTab)` / `ShowWindow(hwnd, SW_SHOW)` /
|
|
166
|
-
`SetActiveWindow(hwnd)
|
|
168
|
+
`SetActiveWindow(hwnd)`**: all foreground the named window by
|
|
167
169
|
definition. Never call these from Bash via PowerShell add-types or
|
|
168
170
|
C# inline. If you find yourself doing this to "make a click land,"
|
|
169
171
|
the click is already wrong: re-read "Click semantics" below.
|
|
170
|
-
- **`AttachThreadInput` + `SetForegroundWindow` trickery
|
|
172
|
+
- **`AttachThreadInput` + `SetForegroundWindow` trickery**: the
|
|
171
173
|
classic Windows focus-bypass hack. Even when it works it's a
|
|
172
174
|
visible focus pop and a UIPI/UIAccess violation. Forbidden.
|
|
173
|
-
- **`SendInput(MOUSEEVENTF_*)` over the user's real cursor
|
|
175
|
+
- **`SendInput(MOUSEEVENTF_*)` over the user's real cursor**: moves
|
|
174
176
|
the cursor and synthesizes input. The cursor warps to coordinates,
|
|
175
177
|
any handler in the topmost window at those coordinates fires, and
|
|
176
178
|
for most apps the receiving window activates because input arrived
|
|
177
179
|
from the OS-trusted pipeline. Use `click({pid, x, y})` or
|
|
178
|
-
`click({pid,
|
|
180
|
+
`click({pid, element_token})` instead: both route per-pid and
|
|
179
181
|
never touch the OS cursor.
|
|
180
|
-
- **`SendInput(KEYBDINPUT)` with no target HWND
|
|
182
|
+
- **`SendInput(KEYBDINPUT)` with no target HWND**: same idea, it goes
|
|
181
183
|
to the focused window, not your target. Use `hotkey({pid, keys:
|
|
182
184
|
[...]})` which uses `PostMessage(WM_KEYDOWN/UP)` to the named pid's
|
|
183
185
|
focused window.
|
|
184
|
-
- **Keyboard shortcuts that semantically mean "focus here"
|
|
186
|
+
- **Keyboard shortcuts that semantically mean "focus here":
|
|
185
187
|
Chromium / Edge / Firefox `Ctrl+L` (focus address bar),
|
|
186
188
|
Explorer's `Ctrl+L` / `Alt+D` (focus path bar), `F6` (focus
|
|
187
|
-
cycle).** These aren't pure key events
|
|
189
|
+
cycle).** These aren't pure key events: the receiving app
|
|
188
190
|
interprets "user wants to type here" as activation intent and
|
|
189
191
|
raises its window to be key. Even when delivered to a backgrounded
|
|
190
192
|
pid via `hotkey`, the downstream app pulls focus. **For omnibox
|
|
191
193
|
navigation specifically**, the correct path is `launch_app({path:
|
|
192
194
|
"...msedge.exe", urls: ["https://…"]})` (or `{aumid:
|
|
193
|
-
"Microsoft.MicrosoftEdge.Stable_…!App", urls: [...]}`)
|
|
195
|
+
"Microsoft.MicrosoftEdge.Stable_…!App", urls: [...]}`). No
|
|
194
196
|
omnibox dance, no `Ctrl+L`, no focus-steal. The browser opens the
|
|
195
197
|
URL in a new window without activating it.
|
|
196
198
|
- **Tab-switching shortcuts in browsers (`Ctrl+1..9`, `Ctrl+Tab`,
|
|
@@ -198,37 +200,37 @@ frontmost state:
|
|
|
198
200
|
even when delivered to a backgrounded pid. The app's key handler
|
|
199
201
|
processes the shortcut, the window re-renders the new tab's
|
|
200
202
|
content, the user sees their tabs flipping. Same as macOS:
|
|
201
|
-
there is no UIA-only workaround
|
|
203
|
+
there is no UIA-only workaround: page content (HTML, DOM,
|
|
202
204
|
WebView2's accessible tree) populates only for the focused tab;
|
|
203
205
|
inspecting a background tab requires activating it, which is the
|
|
204
206
|
visible flip.
|
|
205
207
|
|
|
206
208
|
**Prefer the windows-over-tabs pattern**: for each URL you need to
|
|
207
|
-
drive backgrounded, use `launch_app({urls: [url]})
|
|
209
|
+
drive backgrounded, use `launch_app({urls: [url]})`. Chromium-
|
|
208
210
|
family browsers open each URL in a new **window**. Each window has
|
|
209
211
|
its own `window_id`, its own UIA tree, and can be inspected /
|
|
210
|
-
interacted with via `
|
|
212
|
+
interacted with via `element_token` without activating or switching
|
|
211
213
|
anything. Tabs are a UX grouping for humans; cua-driver-rs
|
|
212
214
|
workflows should default to windows.
|
|
213
215
|
|
|
214
|
-
- **Win+key shortcuts owned by the shell
|
|
216
|
+
- **Win+key shortcuts owned by the shell**: `Win+E` (Explorer),
|
|
215
217
|
`Win+R` (Run), `Win+S` / `Win+Q` (Search), `Win+number` (taskbar
|
|
216
218
|
pinned-app activation), `Win+Tab` (Task View), `Alt+Tab` (window
|
|
217
219
|
switcher), `Win+D` (show desktop), `Win+L` (lock), `Win+M` /
|
|
218
220
|
`Win+Up/Down` (minimize / maximize). All hard-coded to the shell
|
|
219
|
-
and visibly disruptive
|
|
221
|
+
and visibly disruptive: they bypass any per-pid routing. Forbidden
|
|
220
222
|
in `hotkey` calls regardless of target pid.
|
|
221
|
-
- **`taskkill /F /IM <exe>` to close an app the user is using
|
|
223
|
+
- **`taskkill /F /IM <exe>` to close an app the user is using**:
|
|
222
224
|
not a focus-steal but a data-loss vector. Use
|
|
223
225
|
`hotkey({pid, keys:["alt","f4"]})` and let the app close cleanly,
|
|
224
226
|
or ask the user first.
|
|
225
227
|
|
|
226
228
|
Reading state is fine. Listing windows, reading registry, querying
|
|
227
229
|
process info via `Get-Process`, calling `tasklist`, walking UIA trees
|
|
228
|
-
via `cua-driver get_window_state
|
|
230
|
+
via `cua-driver get_window_state`: none of these change focus.
|
|
229
231
|
**Mutating state via shell shims is the line.**
|
|
230
232
|
|
|
231
|
-
**Corollary
|
|
233
|
+
**Corollary: the Win+Search rule.** Don't use Win+S/Win+Q "open Start
|
|
232
234
|
search, type query, press Enter" patterns to launch apps. They (a)
|
|
233
235
|
foreground the Search UI, (b) leave the Search UI populated with the
|
|
234
236
|
agent's typed text, (c) trigger the new app via the Start Menu's own
|
|
@@ -236,7 +238,7 @@ activation path which `SW_SHOWNORMAL`s it. Use `launch_app({name})` /
|
|
|
236
238
|
`{aumid}` / `{path}` instead.
|
|
237
239
|
|
|
238
240
|
**"Open \<app\>" in user speech means launch, not activate.**
|
|
239
|
-
`cua-driver launch_app` is the one correct path for process startup
|
|
241
|
+
`cua-driver launch_app` is the one correct path for process startup:
|
|
240
242
|
it's idempotent (no-op on a running app), returns the pid, and
|
|
241
243
|
internally uses `SW_SHOWNOACTIVATE` plus the AppX-broker activation
|
|
242
244
|
flow for packaged apps so the target's window comes up without
|
|
@@ -256,26 +258,26 @@ spawned app has actually activated. This mirrors the macOS
|
|
|
256
258
|
default browser, no app-identifying field) the restore is **skipped**,
|
|
257
259
|
because the user explicitly asked for the page to come up. The restore is
|
|
258
260
|
best-effort: `SetForegroundWindow` from non-UIAccess processes is subject
|
|
259
|
-
to Windows' foreground-lock and may silently no-op
|
|
261
|
+
to Windows' foreground-lock and may silently no-op: failures are logged
|
|
260
262
|
at `tracing::trace!` and not surfaced as errors.
|
|
261
263
|
|
|
262
264
|
**`hotkey` modifier dispatch on Chromium/Electron (Windows).** When
|
|
263
265
|
the target is detected as a Chromium-family window (`Chrome_WidgetWin_*`
|
|
264
|
-
class
|
|
266
|
+
class, covering every Chromium browser AND every Electron app: Chrome,
|
|
265
267
|
Edge, Brave, Arc, Vivaldi, Slack, VS Code, Discord, Teams, Notion),
|
|
266
268
|
`hotkey` with modifiers routes through `PostMessage(WM_KEYDOWN/UP)`
|
|
267
|
-
instead of `SendInput
|
|
269
|
+
instead of `SendInput`: no foreground swap. Chromium's
|
|
268
270
|
`Browser::HandleKeyboardEvent` reads modifier state from the WM_KEYDOWN
|
|
269
271
|
LPARAM bits, NOT from `GetKeyState`, so PostMessage delivery works for
|
|
270
272
|
real accelerators (Ctrl+T, Ctrl+W, Ctrl+L, Ctrl+Shift+B, etc.). The
|
|
271
273
|
SendInput-swap path (`send_key_synthesized`) remains the dispatch for
|
|
272
|
-
**non-Chromium Win32 + modifiers
|
|
274
|
+
**non-Chromium Win32 + modifiers**: classic apps (LibreOffice, FAR,
|
|
273
275
|
classic Notepad) use `TranslateAccelerator` which requires the system
|
|
274
276
|
modifier state updated, and PostMessage can't do that.
|
|
275
277
|
|
|
276
278
|
**`modifier` on a _background_ click is a Windows residual.** A
|
|
277
279
|
backgrounded click delivers through UIA `Invoke` or `PostMessage`, and
|
|
278
|
-
neither carries live keyboard state
|
|
280
|
+
neither carries live keyboard state, so a `modifier` (Ctrl/Shift/etc.)
|
|
279
281
|
passed alongside a `delivery_mode:"background"` click **is not honored**
|
|
280
282
|
on Windows. The `modifier` parameter is part of the shared schema and is
|
|
281
283
|
accepted everywhere; it only takes effect on the SendInput rung, i.e. a
|
|
@@ -293,11 +295,11 @@ Windows-relevant notes:
|
|
|
293
295
|
- **`session` is now accepted on every action/cursor tool.** Earlier
|
|
294
296
|
Windows builds rejected `session` via `additionalProperties:false` (it
|
|
295
297
|
was effectively a macOS-only key); the shared contract makes it
|
|
296
|
-
uniformly schema-accepted
|
|
298
|
+
uniformly schema-accepted: cursor-wired where a cursor glides,
|
|
297
299
|
accepted elsewhere.
|
|
298
300
|
- **`debug_window_info` is a Windows-only tool** (window-handle / class /
|
|
299
301
|
rect / z-order diagnostics for triaging the click chain). It is
|
|
300
|
-
deliberately not part of the cross-platform surface
|
|
302
|
+
deliberately not part of the cross-platform surface: there is no macOS
|
|
301
303
|
or Linux counterpart.
|
|
302
304
|
- **`launch_app` identifiers are platform-specific.** Windows takes
|
|
303
305
|
`aumid` / `launch_path` / `path` / `start_minimized` (plus `bundle_id`
|
|
@@ -307,57 +309,33 @@ Windows-relevant notes:
|
|
|
307
309
|
**Chromium pixel-click foreground polling restore.** `click({pid, x, y})`
|
|
308
310
|
on a Chromium target falls through to `send_click_synthesized`
|
|
309
311
|
(SendInput + brief foreground swap) because Chromium's input thread filters by
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
##
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
PowerShell 5.1's argv quoting quirks for strings containing both
|
|
326
|
-
quotes and spaces). MCP tools (prefix `mcp__cua-driver__*`) only when
|
|
327
|
-
the user explicitly asks for them. CLI wins because it picks up
|
|
328
|
-
rebuilds instantly, failures are easier to diagnose, and there's no
|
|
329
|
-
per-tool schema-load overhead.
|
|
330
|
-
|
|
331
|
-
Every reference to `click(...)`, `get_window_state(...)` etc. in this
|
|
332
|
-
doc means `cua-driver <name>` with JSON piped via stdin — translate
|
|
333
|
-
to MCP form only when MCP is requested.
|
|
334
|
-
|
|
335
|
-
### CLI argument plumbing on Windows
|
|
336
|
-
|
|
337
|
-
Three equivalent shapes for passing JSON to `cua-driver <tool>`:
|
|
338
|
-
|
|
339
|
-
1. **Stdin pipe (recommended)** — avoids PS quoting bugs entirely:
|
|
340
|
-
```powershell
|
|
341
|
-
'{"pid":1234,"text":"hello world"}' | & cua-driver call type_text
|
|
342
|
-
```
|
|
343
|
-
2. **Positional with escaped quotes** — works for JSON without spaces
|
|
344
|
-
in string values:
|
|
345
|
-
```powershell
|
|
346
|
-
& cua-driver call list_windows '{\"app_name\":\"Calculator\"}'
|
|
347
|
-
```
|
|
348
|
-
(Windows PowerShell 5.1 mangles `{"x":"with space"}` when both `"`
|
|
349
|
-
and ` ` appear unquoted in argv. Use stdin for those.)
|
|
350
|
-
3. **`--%` stop-parser directive** (PowerShell 5.1 specific):
|
|
351
|
-
```powershell
|
|
352
|
-
& cua-driver call type_text --% {"pid":1234,"text":"hello world"}
|
|
353
|
-
```
|
|
312
|
+
queue-origin and PostMessage-delivered clicks don't fire DOM events. The
|
|
313
|
+
synchronous restore inside `send_click_synthesized` covers the
|
|
314
|
+
immediate swap; an additional polling guard (same shape as `launch_app`'s
|
|
315
|
+
`FocusRestoreGuard`) catches the **asynchronous** Chromium re-activation
|
|
316
|
+
that can happen as the renderer's input handler processes the click
|
|
317
|
+
(focus().activate() / WebContents::Activate(), 100-500 ms later). The
|
|
318
|
+
guard is gated on `GetWindowThreadProcessId(fg_now) == pid` so user
|
|
319
|
+
Alt-Tabs are respected. The polling guard is asynchronous and best-effort,
|
|
320
|
+
so the tool response is not proof that the previous foreground has already
|
|
321
|
+
been restored.
|
|
322
|
+
|
|
323
|
+
## Transport
|
|
324
|
+
|
|
325
|
+
Follow [RUNTIME.md](RUNTIME.md) for CLI and MCP lifecycle ownership.
|
|
326
|
+
On PowerShell, prefer piping JSON to the CLI to avoid argument-quoting errors:
|
|
354
327
|
|
|
355
|
-
|
|
328
|
+
```powershell
|
|
329
|
+
'{"pid":6004,"session":"run-1"}' | cua-driver list_windows
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
Reuse the same explicit label for a multi-call CLI run and end it afterward.
|
|
333
|
+
For recording or browser preparation, use one persistent MCP connection.
|
|
356
334
|
|
|
357
335
|
### Intent → tool mapping
|
|
358
336
|
|
|
359
337
|
If you find yourself reaching for the right column, something has
|
|
360
|
-
gone wrong
|
|
338
|
+
gone wrong: re-read "The no-foreground contract" above.
|
|
361
339
|
|
|
362
340
|
| Intent | Use | Don't use |
|
|
363
341
|
| ---------------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
|
|
@@ -365,11 +343,11 @@ gone wrong — re-read "The no-foreground contract" above.
|
|
|
365
343
|
| Open / launch a UWP / packaged app | `launch_app({aumid: "Microsoft.Foo_8wekyb3d8bbwe!App"})` | `explorer.exe shell:AppsFolder\\<AUMID>`, Start Menu typing |
|
|
366
344
|
| Open a URL in the default browser | `launch_app({urls: ["https://example.com"]})` | `Start-Process "https://…"`, `explorer.exe ms-edge:…`, `cmd /c start "" "https://…"` |
|
|
367
345
|
| Find a pid | `list_apps` or `launch_app`'s return | `Get-Process`, `tasklist`, Win+S typing |
|
|
368
|
-
| Enumerate an app's windows | `list_windows({pid})
|
|
369
|
-
| Move or resize one exact window | `set_window_frame({pid, window_id, x, y, width, height})` | PowerShell Add-Type wrappers, Win+Arrow, or title-bar dragging
|
|
346
|
+
| Enumerate an app's windows | `list_windows({pid})`, or read the `windows` array `launch_app` already returns | `Get-Process \| Where-Object { $_.MainWindowHandle }` |
|
|
347
|
+
| Move or resize one exact window | `set_window_frame({pid, window_id, x, y, width, height})` | PowerShell Add-Type wrappers, Win+Arrow, or title-bar dragging |
|
|
370
348
|
| Click / type / scroll / keys | `click`, `type_text`, `scroll`, `press_key`, `hotkey` | `SendInput`, `cliclick`-style C# add-types, AutoHotkey scripts |
|
|
371
349
|
| Drag / drag-and-drop | `drag({pid, from_x, from_y, to_x, to_y})` | `SendInput` with `MOUSEEVENTF_MOVE`, mouse_event |
|
|
372
|
-
| Screenshot | `
|
|
350
|
+
| Screenshot | `get_window_state` (window) or authorized `get_desktop_state` (desktop) | `[System.Windows.Forms.Screen]::CopyFromScreen`, `nircmd savescreenshot` |
|
|
373
351
|
| Quit an app | ask the user first, then `hotkey({pid, keys:["alt","f4"]})` | `taskkill /F`, `Stop-Process -Force`, `Get-Process \| Stop-Process` |
|
|
374
352
|
| Hand a file/URL to an app | `launch_app({urls:[<path>]})` (default app) or `{path: "...exe", args:[<file>]}` (specific app) | `& "app.exe" "file"`, `Invoke-Item`, shell associations |
|
|
375
353
|
|
|
@@ -379,37 +357,36 @@ The **only** legitimate use of `SetForegroundWindow` or
|
|
|
379
357
|
`Start-Process` with a foreground app is when the user **explicitly**
|
|
380
358
|
asked for frontmost state ("bring Edge to the front", "make
|
|
381
359
|
Calculator visible", "I want to see it"). Reaching for it because a
|
|
382
|
-
tool call returned something confusing is wrong
|
|
360
|
+
tool call returned something confusing is wrong: diagnose first.
|
|
383
361
|
|
|
384
|
-
When a cua-driver
|
|
362
|
+
When a cua-driver surprises you, diagnose cua-driver first:
|
|
385
363
|
|
|
386
364
|
- **`Posted click to pid X` instead of `Performed UIA Invoke ...`?**
|
|
387
365
|
The (x,y) UIA hit-test didn't find an `InvokePattern`-bearing
|
|
388
366
|
element at that pixel inside the target HWND, so it fell through
|
|
389
367
|
to `PostMessage(WM_LBUTTONDOWN)`. For UWP / WebView2 surfaces,
|
|
390
|
-
PostMessage silently no-ops
|
|
391
|
-
`get_window_state(pid, window_id)` and use `
|
|
368
|
+
PostMessage silently no-ops: re-snapshot via
|
|
369
|
+
`get_window_state(pid, window_id)` and use `element_token` so the
|
|
392
370
|
daemon can invoke the cached UIA element by identity instead of by
|
|
393
371
|
point.
|
|
394
|
-
- **`
|
|
395
|
-
skipped `get_window_state` this turn or
|
|
396
|
-
|
|
397
|
-
keyed on `(pid, window_id)` — indices don't carry across windows of
|
|
372
|
+
- **`stale_element_token` / `No cached UIA state`?** You either
|
|
373
|
+
skipped `get_window_state` this turn or used a token from an older
|
|
374
|
+
snapshot. Snapshots are keyed on `(pid, window_id)`: tokens don't carry across windows of
|
|
398
375
|
the same app. Re-snapshot with the same window_id you're about to
|
|
399
376
|
click in.
|
|
400
377
|
- **`Invalid window handle (0x80070578)`?** The HWND you passed is
|
|
401
378
|
stale (window closed, recreated). Re-resolve via `list_windows`.
|
|
402
379
|
- **Empty `tree_markdown` / sparse UIA tree?** Some apps populate
|
|
403
380
|
their UIA tree lazily on first call; retry `get_window_state`
|
|
404
|
-
once. If still empty, the app has no UIA provider
|
|
381
|
+
once. If still empty, the app has no UIA provider: fall back to an
|
|
405
382
|
element px action (x,y clicks off the screenshot) on visible content
|
|
406
383
|
(acceptable for exploration; pair with screenshots).
|
|
407
384
|
- **Empty tree, or a snapshot with no image?** `get_window_state`
|
|
408
|
-
returns **both** the UIA tree and a screenshot by default
|
|
385
|
+
returns **both** the UIA tree and a screenshot by default: there is no
|
|
409
386
|
capture mode to pick. If the tree came back empty, the response is
|
|
410
|
-
`degraded` (no UIA provider
|
|
387
|
+
`degraded` (no UIA provider: retry once for lazy trees, see the note
|
|
411
388
|
above); act by **px** off the screenshot in the same response. The
|
|
412
|
-
`capture_mode` param is **deprecated and ignored
|
|
389
|
+
`capture_mode` param is **deprecated and ignored**: it's still accepted
|
|
413
390
|
so old callers don't error, but both the tree and the image come back
|
|
414
391
|
regardless of what you pass.
|
|
415
392
|
- **`list_windows` returns Win32 windows but misses UWP / WebView2
|
|
@@ -422,13 +399,13 @@ When a cua-driver call surprises you, diagnose cua-driver first:
|
|
|
422
399
|
`get_window_state(pid, window_id)` instead; it works while backgrounded.
|
|
423
400
|
Desktop actions use
|
|
424
401
|
`target:{"kind":"desktop","display_id":"primary"}`. Don't reach for
|
|
425
|
-
`get_desktop_state` as a casual screenshot
|
|
402
|
+
`get_desktop_state` as a casual screenshot: it is the desktop capture
|
|
426
403
|
surface, not window inspection.
|
|
427
404
|
- **`Calc display stuck at 0 after my clicks`?** Almost always
|
|
428
405
|
means UWP and you're on the PostMessage path. UWP processes
|
|
429
406
|
pointer input via `Windows.UI.Input`, NOT through HWND message
|
|
430
|
-
queues
|
|
431
|
-
`
|
|
407
|
+
queues: PostMessage(WM_LBUTTONDOWN) gets ignored. Use
|
|
408
|
+
`element_token` instead of (x,y) for UWP targets.
|
|
432
409
|
|
|
433
410
|
Only after those are ruled out should you fall through to the
|
|
434
411
|
activate fallback. Always name the focus steal in your response
|
|
@@ -440,16 +417,16 @@ Before every `Bash` call whose command line touches any Windows app
|
|
|
440
417
|
(launching, opening, clicking, typing, scripting, screenshotting),
|
|
441
418
|
run the self-check:
|
|
442
419
|
|
|
443
|
-
1. **Does this command foreground the target?** If yes
|
|
420
|
+
1. **Does this command foreground the target?** If yes, stop and
|
|
444
421
|
translate to the cua-driver equivalent from the mapping table.
|
|
445
422
|
2. **Does this command move the user's real cursor?** (`SendInput`,
|
|
446
423
|
`SetCursorPos` from inline C#, AutoHotkey scripts, `nircmd
|
|
447
|
-
sendmouse`.) If yes
|
|
424
|
+
sendmouse`.) If yes, stop; use `click({pid, x, y})` which routes
|
|
448
425
|
per-HWND via PostMessage / per-element via UIA Invoke and never
|
|
449
426
|
warps the cursor.
|
|
450
427
|
3. **Does this command bypass cua-driver entirely?** (PowerShell
|
|
451
428
|
GUI scripts, AutoHotkey, `SendKeys`, AppleScript-equivalent
|
|
452
|
-
automation tools.) If yes
|
|
429
|
+
automation tools.) If yes, stop; find the cua-driver tool that
|
|
453
430
|
does the intent.
|
|
454
431
|
|
|
455
432
|
If all three are "no," the command is safe. If you can't answer,
|
|
@@ -457,9 +434,9 @@ default to stop and ask rather than proceed. A single `Start-Process`
|
|
|
457
434
|
run by accident steals the user's foreground and kills the trust
|
|
458
435
|
your prior tool calls earned.
|
|
459
436
|
|
|
460
|
-
## Prerequisites
|
|
437
|
+
## Prerequisites: check before starting
|
|
461
438
|
|
|
462
|
-
1. **`cua-driver` is on `$PATH
|
|
439
|
+
1. **`cua-driver` is on `$PATH`**: `Get-Command cua-driver` or
|
|
463
440
|
`where.exe cua-driver`. Install location:
|
|
464
441
|
`%LOCALAPPDATA%\Programs\trycua\cua-driver-rs\bin\cua-driver.exe`,
|
|
465
442
|
added to the user PATH by the install script.
|
|
@@ -480,15 +457,15 @@ your prior tool calls earned.
|
|
|
480
457
|
`SessionId == 0` is refused before runtime actions. The autostart Scheduled Task uses
|
|
481
458
|
`LogonType=Interactive` so the daemon lands in the user's logon
|
|
482
459
|
session. If you started the daemon via SSH-into-Windows, that
|
|
483
|
-
session is usually Session 0
|
|
460
|
+
session is usually Session 0: kick the autostart task instead:
|
|
484
461
|
```powershell
|
|
485
462
|
schtasks /Run /TN cua-driver-serve
|
|
486
463
|
```
|
|
487
|
-
3. **Run `cua-driver doctor
|
|
464
|
+
3. **Run `cua-driver doctor`**: reports session ID, COM apartment
|
|
488
465
|
status, UIA desktop-enumeration reachability, install paths,
|
|
489
466
|
version. If anything reads `false` / `error`, fix that before
|
|
490
467
|
tool-calling.
|
|
491
|
-
4. **Permissions
|
|
468
|
+
4. **Permissions**: Windows has no TCC equivalent. cua-driver-rs
|
|
492
469
|
needs:
|
|
493
470
|
- No admin elevation for normal use (UIA, PostMessage, UWP
|
|
494
471
|
activation all work from a standard user token).
|
|
@@ -502,66 +479,49 @@ your prior tool calls earned.
|
|
|
502
479
|
## Using cua-driver from the shell
|
|
503
480
|
|
|
504
481
|
Tool names are `snake_case`, management subcommands are
|
|
505
|
-
`kebab-case
|
|
482
|
+
`kebab-case`: no ambiguity. Tools invoked as `cua-driver
|
|
506
483
|
<tool-name>` with JSON via stdin or positional arg. Management
|
|
507
484
|
subcommands:
|
|
508
485
|
|
|
509
|
-
- **`cua-driver serve
|
|
486
|
+
- **`cua-driver serve`**: start the persistent daemon used by one-shot CLI
|
|
510
487
|
calls or by MCP clients that explicitly select it with `--socket`. Bare
|
|
511
488
|
`cua-driver mcp` owns its runtime directly on Windows.
|
|
512
|
-
Normally not run manually
|
|
489
|
+
Normally not run manually: the autostart Scheduled Task fires it
|
|
513
490
|
at every interactive logon. If you stopped it (`Stop-Process`),
|
|
514
491
|
re-run with `schtasks /Run /TN cua-driver-serve`, not by spawning
|
|
515
492
|
`cua-driver serve` from SSH (Session 0 problem).
|
|
516
|
-
- **`cua-driver stop`** / **`status
|
|
517
|
-
- **`cua-driver doctor
|
|
518
|
-
- **`cua-driver list-tools`** / **`describe <tool
|
|
493
|
+
- **`cua-driver stop`** / **`status`**: daemon lifecycle.
|
|
494
|
+
- **`cua-driver doctor`**: full diagnostics.
|
|
495
|
+
- **`cua-driver list-tools`** / **`describe <tool>`**: tool surface
|
|
519
496
|
discovery.
|
|
520
|
-
- **`cua-driver autostart {enable|disable|status|kick}
|
|
497
|
+
- **`cua-driver autostart {enable|disable|status|kick}`**: manage the
|
|
521
498
|
Scheduled Task that auto-starts the daemon at logon. `enable`
|
|
522
|
-
registers it (idempotent
|
|
499
|
+
registers it (idempotent: replaces existing). `kick` runs it
|
|
523
500
|
immediately without waiting for a fresh logon.
|
|
524
|
-
- **`cua-driver recording start|stop|status
|
|
501
|
+
- **`cua-driver recording start|stop|status`**: see `RECORDING.md`.
|
|
525
502
|
Windows video uses ffmpeg with `gdigrab`; trajectory evidence continues
|
|
526
503
|
without video when ffmpeg is unavailable.
|
|
527
504
|
|
|
528
505
|
Over SSH, never use bare `cua-driver mcp`: the direct runtime rejects Session 0. Start the daemon in the interactive user session and run `cua-driver mcp
|
|
529
506
|
--socket \\.\pipe\cua-driver` from SSH.
|
|
530
507
|
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
# Daemon is already running via Scheduled Task.
|
|
535
|
-
# Launch UWP Calculator without focus-stealing.
|
|
536
|
-
'{"aumid":"Microsoft.WindowsCalculator_8wekyb3d8bbwe!App"}' | & cua-driver call launch_app
|
|
537
|
-
# → {pid: 6004, windows: [{window_id: 459672, ...}]}
|
|
538
|
-
|
|
539
|
-
# Snapshot the UIA tree.
|
|
540
|
-
'{"pid":6004,"window_id":459672}' | & cua-driver call get_window_state
|
|
541
|
-
# Returns: tree_markdown with [N] indices plus structured element_token values,
|
|
542
|
-
# snapshot_id, screenshot, and dimensions.
|
|
543
|
-
|
|
544
|
-
# Click the "Equals" row with its opaque token from that response.
|
|
545
|
-
'{"pid":6004,"element_token":"s0000002a:22"}' | & cua-driver call click
|
|
546
|
-
# → "✅ Performed UIA Invoke on [22] ..."
|
|
547
|
-
|
|
548
|
-
# Re-snapshot to verify the action landed.
|
|
549
|
-
'{"pid":6004,"window_id":459672}' | & cua-driver call get_window_state
|
|
550
|
-
```
|
|
508
|
+
For an ordered action loop, use the named CLI or persistent MCP examples in
|
|
509
|
+
[WORKFLOW.md](WORKFLOW.md), with an observed AUMID, PID, HWND, and fresh token.
|
|
510
|
+
Do not assume the window array is populated immediately on a cold launch.
|
|
551
511
|
|
|
552
|
-
## The core invariant
|
|
512
|
+
## The core invariant: snapshot before AND after every action
|
|
553
513
|
|
|
554
514
|
**Every action MUST be bracketed by `get_window_state(pid, window_id)`**:
|
|
555
515
|
|
|
556
|
-
- **Before
|
|
557
|
-
you're about to use.
|
|
558
|
-
|
|
559
|
-
|
|
560
|
-
|
|
516
|
+
- **Before**: the pre-action snapshot resolves the `element_token`
|
|
517
|
+
you're about to use. Targets from previous turns are stale; the
|
|
518
|
+
server replaces the snapshot on every read, keyed on
|
|
519
|
+
`(pid, window_id)`, and lists the replaced ids in
|
|
520
|
+
`invalidated_snapshot_ids`. Tokens from turn N don't resolve in turn
|
|
561
521
|
N+1, and targets from window A don't resolve against window B of
|
|
562
|
-
the same app. Skip this and element
|
|
563
|
-
`stale_element_token
|
|
564
|
-
- **After
|
|
522
|
+
the same app. Skip this and element actions fail with
|
|
523
|
+
`stale_element_token`, which names the current snapshots.
|
|
524
|
+
- **After**: the post-action snapshot verifies the action actually
|
|
565
525
|
landed. Without it you can't tell a silent no-op from a real
|
|
566
526
|
effect. The UIA tree change (new value, new window, disappeared
|
|
567
527
|
menu, disabled button, etc.) is your evidence that the action
|
|
@@ -594,7 +554,7 @@ Properties:
|
|
|
594
554
|
minimized-but-not-iconic, and cross-virtual-desktop windows.
|
|
595
555
|
- **Cross-process boundaries work.** UIA marshals across the
|
|
596
556
|
`ApplicationFrameHost.exe` → inner UWP process boundary
|
|
597
|
-
(CalculatorApp.exe, Notepad-Win11.exe, etc.)
|
|
557
|
+
(CalculatorApp.exe, Notepad-Win11.exe, etc.): this is how it can
|
|
598
558
|
click Win11 Calc buttons even though they live in a different
|
|
599
559
|
process from the AppFrame HWND.
|
|
600
560
|
- **Falls back to `PostMessage(WM_LBUTTONDOWN/UP)` to the deepest
|
|
@@ -627,13 +587,13 @@ Window-client coordinates (origin at the top-left of the screenshot
|
|
|
627
587
|
the agent saw). The driver:
|
|
628
588
|
|
|
629
589
|
1. Converts to screen coords via `ClientToScreen(hwnd, ...)`.
|
|
630
|
-
2. **UIA hit-test in target HWND's subtree
|
|
590
|
+
2. **UIA hit-test in target HWND's subtree**: `ElementFromHandle`
|
|
631
591
|
resolves the root, `FindAll(TreeScope_Subtree)` enumerates
|
|
632
592
|
descendants, picks the smallest-area `InvokePattern`-bearing
|
|
633
593
|
element whose `CurrentBoundingRectangle` contains the screen
|
|
634
594
|
point, calls `Invoke()`. This is the only path that lands on UWP
|
|
635
595
|
/ WebView2 / DirectComposition surfaces.
|
|
636
|
-
3. **PostMessage fallback
|
|
596
|
+
3. **PostMessage fallback**: if step 2 returned false (no Invokable
|
|
637
597
|
element under the pixel inside `hwnd`, or `hwnd` has no useful
|
|
638
598
|
UIA tree at all), fires `PostMessage(WM_LBUTTONDOWN/UP)` against
|
|
639
599
|
the deepest child HWND at the screen point. Covers plain Win32
|
|
@@ -657,15 +617,15 @@ DOM nodes inside a WebView2 viewport).
|
|
|
657
617
|
### What `(x, y)` mode does NOT solve
|
|
658
618
|
|
|
659
619
|
Apps with **no useful UIA tree** AND that **ignore `WM_LBUTTONDOWN`**
|
|
660
|
-
on the HWND queue
|
|
620
|
+
on the HWND queue: primarily DirectX / OpenGL / Vulkan-rendered
|
|
661
621
|
surfaces (games, custom renderers). The click chain falls all the
|
|
662
622
|
way through and the click no-ops. For those, the only options are:
|
|
663
623
|
|
|
664
|
-
- Bring the window to top first (focus steal
|
|
624
|
+
- Bring the window to top first (focus steal: ask the user before
|
|
665
625
|
doing this, and document why), then synthesize input
|
|
666
626
|
- Use the app's keyboard interface via `hotkey` if available
|
|
667
627
|
|
|
668
|
-
`SendInput` is **not** a silent-fallback option here
|
|
628
|
+
`SendInput` is **not** a silent-fallback option here: it would
|
|
669
629
|
steal focus from whatever the user is doing.
|
|
670
630
|
|
|
671
631
|
### Right-click and multi-click
|
|
@@ -674,17 +634,17 @@ steal focus from whatever the user is doing.
|
|
|
674
634
|
go directly through the PostMessage path. Reason: UIA has no clean
|
|
675
635
|
by-coord equivalent of `ShowContextMenu`, and `Invoke()` is single-
|
|
676
636
|
fire by definition. The action result reports `route:"synthetic_events"`
|
|
677
|
-
regardless of the target's UWP-ness
|
|
637
|
+
regardless of the target's UWP-ness: this is expected and
|
|
678
638
|
**will not work for UWP context menus**. To open a UWP context menu,
|
|
679
639
|
prefer `hotkey({pid, keys: ["shift", "f10"]})` against the focused
|
|
680
640
|
UWP element.
|
|
681
641
|
|
|
682
|
-
## UWP / packaged apps
|
|
642
|
+
## UWP / packaged apps: the AUMID layer
|
|
683
643
|
|
|
684
644
|
Modern Win11 apps (Calculator, Notepad, Settings, Photos, Edge UI
|
|
685
645
|
chrome, Microsoft Store) are **packaged apps** with an `App User
|
|
686
646
|
Model ID` (AUMID) rather than a plain `.exe` path. The AUMID looks
|
|
687
|
-
like `Microsoft.WindowsCalculator_8wekyb3d8bbwe!App
|
|
647
|
+
like `Microsoft.WindowsCalculator_8wekyb3d8bbwe!App`: package family
|
|
688
648
|
name + `!` + AppId.
|
|
689
649
|
|
|
690
650
|
Windows architectural quirks that matter:
|
|
@@ -696,7 +656,7 @@ Windows architectural quirks that matter:
|
|
|
696
656
|
spawns the stub, which exits, which redirects through the AppX
|
|
697
657
|
broker, which spawns the real process with a different pid. You
|
|
698
658
|
end up holding a pid that's already gone. `launch_app` handles
|
|
699
|
-
this transparently
|
|
659
|
+
this transparently: it detects AUMID-looking strings and routes
|
|
700
660
|
through `IApplicationActivationManager::ActivateApplication`,
|
|
701
661
|
which returns the **real** packaged-process pid.
|
|
702
662
|
2. **UWP windows are hosted by `ApplicationFrameHost.exe`.** The
|
|
@@ -708,7 +668,7 @@ Windows architectural quirks that matter:
|
|
|
708
668
|
`GetWindowRect`, `PostMessage`, `BitBlt` all target. UIA
|
|
709
669
|
transparently crosses the boundary into the inner process when
|
|
710
670
|
you walk the tree.
|
|
711
|
-
3. **AUMID resolution by name
|
|
671
|
+
3. **AUMID resolution by name**: `launch_app({name: "calc"})` will
|
|
712
672
|
first try a `shell:AppsFolder` lookup, matching against
|
|
713
673
|
installed-package display names. On a hit it goes through the
|
|
714
674
|
packaged-app path (real pid). Otherwise it falls back to
|
|
@@ -763,20 +723,20 @@ typed browser tools yet.
|
|
|
763
723
|
|
|
764
724
|
## Common failure modes (Windows-specific)
|
|
765
725
|
|
|
766
|
-
- **`Session 0` daemon
|
|
726
|
+
- **`Session 0` daemon**: `cua-driver doctor` reports
|
|
767
727
|
`SessionId: 0`. UIA enumeration returns empty, screenshot
|
|
768
728
|
returns blank. Fix: stop the daemon, kick the autostart task with
|
|
769
729
|
`schtasks /Run /TN cua-driver-serve`.
|
|
770
|
-
- **Stale HWND** (`Invalid window handle 0x80070578`)
|
|
730
|
+
- **Stale HWND** (`Invalid window handle 0x80070578`): the window
|
|
771
731
|
was closed, re-created (e.g. UWP shutdown-on-idle), or moved to
|
|
772
732
|
a different desktop session. Re-resolve via `list_windows`.
|
|
773
|
-
- **Calc display stuck at "0" after pixel clicks
|
|
733
|
+
- **Calc display stuck at "0" after pixel clicks**: the (x,y) UIA
|
|
774
734
|
hit-test missed and PostMessage fell through (PostMessage is a
|
|
775
|
-
silent no-op on UWP). Switch to `
|
|
735
|
+
silent no-op on UWP). Switch to `element_token` mode. Symptom:
|
|
776
736
|
the action result reports `route:"synthetic_events"` instead of
|
|
777
737
|
`route:"accessibility"`.
|
|
778
738
|
- **LibreOffice (VCL) `type_text` / `hotkey` reported success but
|
|
779
|
-
nothing happened
|
|
739
|
+
nothing happened**: VCL/SAL apps route accelerators through
|
|
780
740
|
`TranslateAccelerator` (reads `GetKeyState`, which PostMessage doesn't
|
|
781
741
|
update) and the Calc/Writer document grid only takes real keystrokes
|
|
782
742
|
when a cell is in edit mode, so background `WM_CHAR` / key-combos are
|
|
@@ -785,7 +745,7 @@ typed browser tools yet.
|
|
|
785
745
|
- **`hotkey` / `press_key`** (keystroke + key-combo): `delivery_mode:"background"`
|
|
786
746
|
surfaces a `background_unavailable` error for VCL.
|
|
787
747
|
- **`type_text`** does a **UIA read-back** and returns the shared
|
|
788
|
-
`ActionResult`. With an `
|
|
748
|
+
`ActionResult`. With an `element_token`, the ValuePattern path returns
|
|
789
749
|
`effect:"confirmed"` with `evidence:[{"kind":"value_readback"}]` only
|
|
790
750
|
when the complete expected value is synchronously visible and differs from
|
|
791
751
|
the prior value. If SetValue succeeded but the provider still exposes the
|
|
@@ -799,31 +759,31 @@ typed browser tools yet.
|
|
|
799
759
|
contains the requested text, the result stays `effect:"unverifiable"` because
|
|
800
760
|
WM_CHAR does not expose the insertion point. Take a fresh snapshot before
|
|
801
761
|
retrying. It may recommend foreground when a background insert appears
|
|
802
|
-
dropped. Passing an `
|
|
762
|
+
dropped. Passing an `element_token` makes the
|
|
803
763
|
read-back target that exact element by handle (ValuePattern → TextPattern),
|
|
804
764
|
independent of foreground focus. Without one, PostMessage verification
|
|
805
765
|
falls back to system-wide `GetFocusedElement`, which normally resolves only
|
|
806
766
|
for the foreground app; an unreadable result is therefore not proof of
|
|
807
767
|
failure.
|
|
808
768
|
Escalate to `delivery_mode:"foreground"` for both (SendInput Unicode /
|
|
809
|
-
accelerator). **But** foreground needs the swap to actually land
|
|
769
|
+
accelerator). **But** foreground needs the swap to actually land: if the
|
|
810
770
|
daemon lacks UIAccess and `bring_to_front` returns `landed_on_target:false`
|
|
811
771
|
(or it reverts before the next call), you can't drive it by input at all:
|
|
812
772
|
produce the artifact and `launch_app` it (build the `.xlsx` / `.docx` and
|
|
813
773
|
open it) rather than typing into the GUI.
|
|
814
774
|
- **Edge / Chrome shows tab switching even though I used pid-scoped
|
|
815
|
-
hotkey
|
|
775
|
+
hotkey**: `Ctrl+Tab` / `Ctrl+1..9` aren't pid-scopable; the
|
|
816
776
|
receiver activates. Use the windows-per-URL pattern.
|
|
817
777
|
- **`Get-StartApps` returns no AUMID for an app I see in Start
|
|
818
|
-
Menu
|
|
778
|
+
Menu**: the app might be a Win32 desktop app, not a packaged
|
|
819
779
|
app. Use `{path: "..."}` instead of `{aumid}`. (Win11 Calculator
|
|
820
780
|
IS packaged; Win10 classic Notepad was not.)
|
|
821
781
|
- **`launch_app` returns pid N but `list_windows({pid: N})` returns
|
|
822
|
-
empty
|
|
782
|
+
empty**: UWP cold-launch race: the AppFrame HWND hasn't
|
|
823
783
|
materialized yet. Re-call `list_windows({pid: N})` after 500ms;
|
|
824
784
|
for chronic cases, key off the app name in `list_windows({})`
|
|
825
785
|
output.
|
|
826
|
-
- **Need a screenshot file
|
|
786
|
+
- **Need a screenshot file**: use `get_window_state` with
|
|
827
787
|
`screenshot_out_file`; the result is PNG. The removed standalone screenshot
|
|
828
788
|
compatibility inputs (`format` and `quality`) are rejected by the live tool
|
|
829
789
|
schema. Downsample or convert the saved PNG outside cua-driver if needed.
|
|
@@ -846,15 +806,15 @@ trace back to one of these checks reading "false."
|
|
|
846
806
|
`cua-driver autostart status` reports whether the daemon is
|
|
847
807
|
registered to auto-start at logon AND whether it's currently running:
|
|
848
808
|
|
|
849
|
-
- `not-registered
|
|
809
|
+
- `not-registered`: Task Scheduler explicitly reported that the named task
|
|
850
810
|
does not exist. Re-register via `cua-driver autostart enable`.
|
|
851
|
-
- `registered (not running)
|
|
811
|
+
- `registered (not running)`: autostart task exists but no daemon
|
|
852
812
|
process. Kick it with `cua-driver autostart kick`.
|
|
853
|
-
- `registered (running)
|
|
854
|
-
- `permission-denied
|
|
813
|
+
- `registered (running)`: happy path.
|
|
814
|
+
- `permission-denied`: the current process cannot inspect Task Scheduler;
|
|
855
815
|
registration is unknown. Re-run the status check from a context that can
|
|
856
816
|
read the task rather than re-registering it blindly.
|
|
857
|
-
- `unknown
|
|
817
|
+
- `unknown`: the query failed for another reason. The command exits non-zero
|
|
858
818
|
and includes the original `schtasks` diagnostic; do not treat this as an
|
|
859
819
|
absent registration.
|
|
860
820
|
|