@bastani/atomic 0.9.28-alpha.2 → 0.9.28-alpha.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +50 -0
- package/dist/builtin/intercom/package.json +2 -2
- package/dist/builtin/mcp/native-client-licenses/NOTICE +2 -0
- package/dist/builtin/mcp/package.json +1 -1
- package/dist/builtin/subagents/CHANGELOG.md +6 -0
- package/dist/builtin/subagents/package.json +3 -3
- package/dist/builtin/subagents/skills/agent-browser/SKILL.md +1 -1
- package/dist/builtin/subagents/skills/cua-driver/BROWSER.md +60 -39
- package/dist/builtin/subagents/skills/cua-driver/EMBEDDING.md +119 -36
- package/dist/builtin/subagents/skills/cua-driver/LINUX.md +225 -183
- package/dist/builtin/subagents/skills/cua-driver/MACOS.md +120 -89
- package/dist/builtin/subagents/skills/cua-driver/README.md +16 -7
- package/dist/builtin/subagents/skills/cua-driver/RECORDING.md +85 -115
- package/dist/builtin/subagents/skills/cua-driver/RUNTIME.md +113 -0
- package/dist/builtin/subagents/skills/cua-driver/SKILL.md +55 -1071
- package/dist/builtin/subagents/skills/cua-driver/VISUAL.md +125 -0
- package/dist/builtin/subagents/skills/cua-driver/WINDOWS.md +145 -185
- package/dist/builtin/subagents/skills/cua-driver/WORKFLOW.md +173 -0
- package/dist/builtin/web-access/package.json +2 -2
- package/dist/builtin/workflows/CHANGELOG.md +21 -0
- package/dist/builtin/workflows/README.md +1 -1
- package/dist/builtin/workflows/package.json +2 -2
- package/dist/builtin/workflows/skills/impeccable/SKILL.md +8 -2
- package/dist/builtin/workflows/skills/impeccable/reference/adapt.md +6 -0
- package/dist/builtin/workflows/skills/impeccable/reference/audit.md +2 -1
- package/dist/builtin/workflows/skills/impeccable/reference/component-review.md +63 -0
- package/dist/builtin/workflows/skills/impeccable/reference/degraded/asset-producer.md +5 -0
- package/dist/builtin/workflows/skills/impeccable/reference/degraded/finish-reviewer.md +1 -1
- package/dist/builtin/workflows/skills/impeccable/reference/generate.md +101 -0
- package/dist/builtin/workflows/skills/impeccable/reference/harden.md +9 -0
- package/dist/builtin/workflows/skills/impeccable/reference/hooks.md +3 -1
- package/dist/builtin/workflows/skills/impeccable/reference/mode-operate.md +21 -0
- package/dist/builtin/workflows/skills/impeccable/reference/mode-persuade.md +19 -0
- package/dist/builtin/workflows/skills/impeccable/reference/mode-read.md +21 -0
- package/dist/builtin/workflows/skills/impeccable/reference/new-work.md +19 -12
- package/dist/builtin/workflows/skills/impeccable/reference/region-map.md +26 -0
- package/dist/builtin/workflows/skills/impeccable/reference/routing.md +1 -1
- package/dist/builtin/workflows/skills/impeccable/reference/visualize.md +5 -5
- package/dist/builtin/workflows/skills/impeccable/scripts/VERSION +1 -1
- package/dist/builtin/workflows/skills/impeccable/scripts/command-metadata.json +4 -0
- package/dist/builtin/workflows/skills/impeccable/scripts/impeccable.cmd +2 -2
- package/dist/builtin/workflows/skills/impeccable/scripts/live-browser-session.js +5 -1
- package/dist/builtin/workflows/skills/impeccable/scripts/live-browser.js +524 -4
- package/dist/builtin/workflows/skills/prompt-engineer/SKILL.md +1 -0
- package/dist/builtin/workflows/skills/prompt-engineer/references/claude_haiku_5_5.md +81 -0
- package/dist/builtin/workflows/src/extension/index.bundle.mjs +14733 -14704
- package/dist/cli/args.d.ts.map +1 -1
- package/dist/cli/args.js +8 -1
- package/dist/cli/args.js.map +1 -1
- package/dist/codemode-worker.js +2 -2
- package/dist/config.d.ts +2 -1
- package/dist/config.d.ts.map +1 -1
- package/dist/config.js +4 -2
- package/dist/config.js.map +1 -1
- package/dist/core/agent-session-compaction.d.ts +3 -3
- package/dist/core/agent-session-compaction.d.ts.map +1 -1
- package/dist/core/agent-session-compaction.js +5 -1
- package/dist/core/agent-session-compaction.js.map +1 -1
- package/dist/core/agent-session-events.d.ts.map +1 -1
- package/dist/core/agent-session-events.js +1 -0
- package/dist/core/agent-session-events.js.map +1 -1
- package/dist/core/agent-session-extension-bindings.d.ts.map +1 -1
- package/dist/core/agent-session-extension-bindings.js +7 -2
- package/dist/core/agent-session-extension-bindings.js.map +1 -1
- package/dist/core/agent-session-methods.d.ts +7 -1
- package/dist/core/agent-session-methods.d.ts.map +1 -1
- package/dist/core/agent-session-methods.js.map +1 -1
- package/dist/core/agent-session-prompt.d.ts.map +1 -1
- package/dist/core/agent-session-prompt.js +3 -2
- package/dist/core/agent-session-prompt.js.map +1 -1
- package/dist/core/agent-session-tool-registry.js.map +1 -1
- package/dist/core/agent-session-types.d.ts +2 -0
- package/dist/core/agent-session-types.d.ts.map +1 -1
- package/dist/core/agent-session-types.js.map +1 -1
- package/dist/core/agent-session.d.ts +2 -0
- package/dist/core/agent-session.d.ts.map +1 -1
- package/dist/core/agent-session.js +6 -1
- package/dist/core/agent-session.js.map +1 -1
- package/dist/core/bash-executor.d.ts.map +1 -1
- package/dist/core/bash-executor.js +22 -4
- package/dist/core/bash-executor.js.map +1 -1
- package/dist/core/child-session-options.d.ts.map +1 -1
- package/dist/core/child-session-options.js +17 -2
- package/dist/core/child-session-options.js.map +1 -1
- package/dist/core/export-html/tool-renderer.d.ts.map +1 -1
- package/dist/core/export-html/tool-renderer.js +2 -0
- package/dist/core/export-html/tool-renderer.js.map +1 -1
- package/dist/core/extensions/agent-events.d.ts +2 -0
- package/dist/core/extensions/agent-events.d.ts.map +1 -1
- package/dist/core/extensions/agent-events.js.map +1 -1
- package/dist/core/extensions/context-types.d.ts +2 -0
- package/dist/core/extensions/context-types.d.ts.map +1 -1
- package/dist/core/extensions/context-types.js.map +1 -1
- package/dist/core/extensions/loader-virtual-modules.d.ts +3 -4
- package/dist/core/extensions/loader-virtual-modules.d.ts.map +1 -1
- package/dist/core/extensions/loader-virtual-modules.js +14 -9
- package/dist/core/extensions/loader-virtual-modules.js.map +1 -1
- package/dist/core/extensions/mcp-server-registry.d.ts +4 -1
- package/dist/core/extensions/mcp-server-registry.d.ts.map +1 -1
- package/dist/core/extensions/mcp-server-registry.js +7 -4
- package/dist/core/extensions/mcp-server-registry.js.map +1 -1
- package/dist/core/extensions/runner.d.ts +3 -0
- package/dist/core/extensions/runner.d.ts.map +1 -1
- package/dist/core/extensions/runner.js +19 -0
- package/dist/core/extensions/runner.js.map +1 -1
- package/dist/core/extensions/tool-events.d.ts +1 -1
- package/dist/core/extensions/tool-events.js.map +1 -1
- package/dist/core/extensions/tool-types.d.ts +2 -0
- package/dist/core/extensions/tool-types.d.ts.map +1 -1
- package/dist/core/extensions/tool-types.js.map +1 -1
- package/dist/core/extensions/ts-source-specifiers.d.ts +18 -0
- package/dist/core/extensions/ts-source-specifiers.d.ts.map +1 -0
- package/dist/core/extensions/ts-source-specifiers.js +142 -0
- package/dist/core/extensions/ts-source-specifiers.js.map +1 -0
- package/dist/core/fast-model-variants.d.ts.map +1 -1
- package/dist/core/fast-model-variants.js +3 -2
- package/dist/core/fast-model-variants.js.map +1 -1
- package/dist/core/model-resolver-defaults.d.ts.map +1 -1
- package/dist/core/model-resolver-defaults.js +2 -0
- package/dist/core/model-resolver-defaults.js.map +1 -1
- package/dist/core/nested-tool-calls.d.ts +1 -0
- package/dist/core/nested-tool-calls.d.ts.map +1 -1
- package/dist/core/nested-tool-calls.js +1 -0
- package/dist/core/nested-tool-calls.js.map +1 -1
- package/dist/core/sdk-exports.d.ts +2 -2
- package/dist/core/sdk-exports.js.map +1 -1
- package/dist/core/sdk-types.d.ts +3 -0
- package/dist/core/sdk-types.d.ts.map +1 -1
- package/dist/core/sdk-types.js.map +1 -1
- package/dist/core/sdk.d.ts.map +1 -1
- package/dist/core/sdk.js +17 -8
- package/dist/core/sdk.js.map +1 -1
- package/dist/core/settings-manager-basic-accessors.js.map +1 -1
- package/dist/core/settings-manager-core.js.map +1 -1
- package/dist/core/settings-merge.d.ts +3 -0
- package/dist/core/settings-merge.d.ts.map +1 -1
- package/dist/core/settings-merge.js +18 -4
- package/dist/core/settings-merge.js.map +1 -1
- package/dist/core/settings-storage.js.map +1 -1
- package/dist/core/settings-types.js.map +1 -1
- package/dist/core/tools/bash.d.ts.map +1 -1
- package/dist/core/tools/bash.js +7 -3
- package/dist/core/tools/bash.js.map +1 -1
- package/dist/core/tools/edit.js +1 -1
- package/dist/core/tools/edit.js.map +1 -1
- package/dist/extensions/codemode/execute.d.ts.map +1 -1
- package/dist/extensions/codemode/execute.js +69 -8
- package/dist/extensions/codemode/execute.js.map +1 -1
- package/dist/extensions/codemode/index.d.ts.map +1 -1
- package/dist/extensions/codemode/index.js +7 -0
- package/dist/extensions/codemode/index.js.map +1 -1
- package/dist/extensions/codemode/tool.d.ts +2 -0
- package/dist/extensions/codemode/tool.d.ts.map +1 -1
- package/dist/extensions/codemode/tool.js +4 -2
- package/dist/extensions/codemode/tool.js.map +1 -1
- package/dist/extensions/llama/provider.d.ts.map +1 -1
- package/dist/extensions/llama/provider.js +19 -8
- package/dist/extensions/llama/provider.js.map +1 -1
- package/dist/extensions/mcp/cli.js +18 -24
- package/dist/extensions/mcp/cli.js.map +1 -1
- package/dist/extensions/mcp/client/auth-provider.d.ts +4 -0
- package/dist/extensions/mcp/client/auth-provider.d.ts.map +1 -1
- package/dist/extensions/mcp/client/auth-provider.js.map +1 -1
- package/dist/extensions/mcp/client/oauth/discovery.d.ts +3 -0
- package/dist/extensions/mcp/client/oauth/discovery.d.ts.map +1 -1
- package/dist/extensions/mcp/client/oauth/discovery.js +8 -5
- package/dist/extensions/mcp/client/oauth/discovery.js.map +1 -1
- package/dist/extensions/mcp/client/oauth/flow.d.ts +4 -0
- package/dist/extensions/mcp/client/oauth/flow.d.ts.map +1 -1
- package/dist/extensions/mcp/client/oauth/flow.js +12 -2
- package/dist/extensions/mcp/client/oauth/flow.js.map +1 -1
- package/dist/extensions/mcp/client/transports/streamable-http.d.ts +3 -0
- package/dist/extensions/mcp/client/transports/streamable-http.d.ts.map +1 -1
- package/dist/extensions/mcp/client/transports/streamable-http.js +17 -14
- package/dist/extensions/mcp/client/transports/streamable-http.js.map +1 -1
- package/dist/extensions/mcp/index.d.ts.map +1 -1
- package/dist/extensions/mcp/index.js +150 -72
- package/dist/extensions/mcp/index.js.map +1 -1
- package/dist/extensions/mcp/oauth.d.ts +5 -3
- package/dist/extensions/mcp/oauth.d.ts.map +1 -1
- package/dist/extensions/mcp/oauth.js +36 -10
- package/dist/extensions/mcp/oauth.js.map +1 -1
- package/dist/extensions/mcp/ui.d.ts +3 -3
- package/dist/extensions/mcp/ui.d.ts.map +1 -1
- package/dist/extensions/mcp/ui.js +10 -2
- package/dist/extensions/mcp/ui.js.map +1 -1
- package/dist/index.d.ts +2 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js.map +1 -1
- package/dist/main-deferred-startup.d.ts +1 -0
- package/dist/main-deferred-startup.d.ts.map +1 -1
- package/dist/main-deferred-startup.js +9 -21
- package/dist/main-deferred-startup.js.map +1 -1
- package/dist/main.d.ts.map +1 -1
- package/dist/main.js +8 -6
- package/dist/main.js.map +1 -1
- package/dist/modes/interactive/components/bash-execution.d.ts +5 -1
- package/dist/modes/interactive/components/bash-execution.d.ts.map +1 -1
- package/dist/modes/interactive/components/bash-execution.js +17 -14
- package/dist/modes/interactive/components/bash-execution.js.map +1 -1
- package/dist/modes/interactive/components/branch-summary-message.d.ts +2 -1
- package/dist/modes/interactive/components/branch-summary-message.d.ts.map +1 -1
- package/dist/modes/interactive/components/branch-summary-message.js +5 -2
- package/dist/modes/interactive/components/branch-summary-message.js.map +1 -1
- package/dist/modes/interactive/components/chat-message-renderer.d.ts +1 -0
- package/dist/modes/interactive/components/chat-message-renderer.d.ts.map +1 -1
- package/dist/modes/interactive/components/chat-message-renderer.js +19 -8
- package/dist/modes/interactive/components/chat-message-renderer.js.map +1 -1
- package/dist/modes/interactive/components/compaction-boundary-message.d.ts +3 -2
- package/dist/modes/interactive/components/compaction-boundary-message.d.ts.map +1 -1
- package/dist/modes/interactive/components/compaction-boundary-message.js +7 -4
- package/dist/modes/interactive/components/compaction-boundary-message.js.map +1 -1
- package/dist/modes/interactive/components/custom-entry.d.ts +3 -1
- package/dist/modes/interactive/components/custom-entry.d.ts.map +1 -1
- package/dist/modes/interactive/components/custom-entry.js +8 -2
- package/dist/modes/interactive/components/custom-entry.js.map +1 -1
- package/dist/modes/interactive/components/custom-message.d.ts.map +1 -1
- package/dist/modes/interactive/components/custom-message.js +1 -0
- package/dist/modes/interactive/components/custom-message.js.map +1 -1
- package/dist/modes/interactive/components/settings-selector-items.js +1 -1
- package/dist/modes/interactive/components/settings-selector-items.js.map +1 -1
- package/dist/modes/interactive/components/skill-invocation-message.d.ts +2 -1
- package/dist/modes/interactive/components/skill-invocation-message.d.ts.map +1 -1
- package/dist/modes/interactive/components/skill-invocation-message.js +5 -2
- package/dist/modes/interactive/components/skill-invocation-message.js.map +1 -1
- package/dist/modes/interactive/components/tool-execution.d.ts +4 -0
- package/dist/modes/interactive/components/tool-execution.d.ts.map +1 -1
- package/dist/modes/interactive/components/tool-execution.js +10 -0
- package/dist/modes/interactive/components/tool-execution.js.map +1 -1
- package/dist/modes/interactive/interactive-agent-events.js +3 -1
- package/dist/modes/interactive/interactive-agent-events.js.map +1 -1
- package/dist/modes/interactive/interactive-auth-login.js +6 -0
- package/dist/modes/interactive/interactive-auth-login.js.map +1 -1
- package/dist/modes/interactive/interactive-bash-compact.js +2 -2
- package/dist/modes/interactive/interactive-bash-compact.js.map +1 -1
- package/dist/modes/interactive/interactive-extension-dialogs.js +11 -2
- package/dist/modes/interactive/interactive-extension-dialogs.js.map +1 -1
- package/dist/modes/interactive/interactive-input-handling.d.ts.map +1 -1
- package/dist/modes/interactive/interactive-input-handling.js +1 -47
- package/dist/modes/interactive/interactive-input-handling.js.map +1 -1
- package/dist/modes/interactive/interactive-mode-base.d.ts +3 -1
- package/dist/modes/interactive/interactive-mode-base.d.ts.map +1 -1
- package/dist/modes/interactive/interactive-mode-base.js +6 -1
- package/dist/modes/interactive/interactive-mode-base.js.map +1 -1
- package/dist/modes/interactive/interactive-mode-surface.d.ts +2 -2
- package/dist/modes/interactive/interactive-mode-surface.d.ts.map +1 -1
- package/dist/modes/interactive/interactive-mode-surface.js.map +1 -1
- package/dist/modes/interactive/interactive-prompt-restore.d.ts.map +1 -1
- package/dist/modes/interactive/interactive-prompt-restore.js +0 -5
- package/dist/modes/interactive/interactive-prompt-restore.js.map +1 -1
- package/dist/modes/interactive/interactive-render-chat.js +7 -10
- package/dist/modes/interactive/interactive-render-chat.js.map +1 -1
- package/dist/modes/interactive/interactive-selectors.js +8 -1
- package/dist/modes/interactive/interactive-selectors.js.map +1 -1
- package/dist/modes/interactive/interactive-session-runtime.js +1 -0
- package/dist/modes/interactive/interactive-session-runtime.js.map +1 -1
- package/dist/modes/interactive/interactive-startup.js +1 -0
- package/dist/modes/interactive/interactive-startup.js.map +1 -1
- package/dist/modes/interactive/program-status-reporter.d.ts +24 -0
- package/dist/modes/interactive/program-status-reporter.d.ts.map +1 -0
- package/dist/modes/interactive/program-status-reporter.js +96 -0
- package/dist/modes/interactive/program-status-reporter.js.map +1 -0
- package/dist/modes/interactive-engine/engine-custom-ui.d.ts.map +1 -1
- package/dist/modes/interactive-engine/engine-custom-ui.js +1 -0
- package/dist/modes/interactive-engine/engine-custom-ui.js.map +1 -1
- package/dist/modes/interactive-engine/engine-render-service.d.ts.map +1 -1
- package/dist/modes/interactive-engine/engine-render-service.js +3 -1
- package/dist/modes/interactive-engine/engine-render-service.js.map +1 -1
- package/dist/modes/interactive-engine/protocol.d.ts +1 -0
- package/dist/modes/interactive-engine/protocol.d.ts.map +1 -1
- package/dist/modes/interactive-engine/protocol.js +1 -0
- package/dist/modes/interactive-engine/protocol.js.map +1 -1
- package/dist/modes/interactive-engine/remote-renderer.d.ts +3 -0
- package/dist/modes/interactive-engine/remote-renderer.d.ts.map +1 -1
- package/dist/modes/interactive-engine/remote-renderer.js +7 -0
- package/dist/modes/interactive-engine/remote-renderer.js.map +1 -1
- package/dist/modes/rpc/rpc-client-process.d.ts +7 -0
- package/dist/modes/rpc/rpc-client-process.d.ts.map +1 -1
- package/dist/modes/rpc/rpc-client-process.js +20 -3
- package/dist/modes/rpc/rpc-client-process.js.map +1 -1
- package/dist/modes/rpc/rpc-command-timeouts.d.ts.map +1 -1
- package/dist/modes/rpc/rpc-command-timeouts.js +1 -0
- package/dist/modes/rpc/rpc-command-timeouts.js.map +1 -1
- package/dist/utils/ansi.d.ts +8 -0
- package/dist/utils/ansi.d.ts.map +1 -1
- package/dist/utils/ansi.js +29 -10
- package/dist/utils/ansi.js.map +1 -1
- package/dist/utils/clipboard.d.ts.map +1 -1
- package/dist/utils/clipboard.js +13 -3
- package/dist/utils/clipboard.js.map +1 -1
- package/dist/utils/image-resize-core.d.ts +8 -0
- package/dist/utils/image-resize-core.d.ts.map +1 -1
- package/dist/utils/image-resize-core.js +1 -0
- package/dist/utils/image-resize-core.js.map +1 -1
- package/dist/utils/image-resize-worker.js +3 -2
- package/dist/utils/image-resize-worker.js.map +1 -1
- package/dist/utils/image-resize.d.ts +2 -2
- package/dist/utils/image-resize.d.ts.map +1 -1
- package/dist/utils/image-resize.js +7 -10
- package/dist/utils/image-resize.js.map +1 -1
- package/docs/codemode.md +30 -6
- package/docs/compaction.md +1 -1
- package/docs/computer-use.md +9 -9
- package/docs/custom-provider/streaming.md +4 -0
- package/docs/environment-variables.md +2 -0
- package/docs/extensions/api-reference.md +1 -1
- package/docs/extensions/authoring.md +2 -0
- package/docs/extensions/events.md +5 -1
- package/docs/getting-started/first-session.md +1 -1
- package/docs/json.md +6 -1
- package/docs/llama-cpp.md +9 -2
- package/docs/mcp-servers.md +3 -3
- package/docs/models/evals.md +34 -3
- package/docs/models.md +10 -0
- package/docs/providers.md +35 -4
- package/docs/reference/cli.md +2 -2
- package/docs/sdk/reference.md +3 -3
- package/docs/sdk.md +7 -1
- package/docs/settings.md +3 -3
- package/docs/skills.md +1 -1
- package/docs/terminal-setup.md +14 -0
- package/docs/tools.md +1 -1
- package/docs/workflows/authoring.md +3 -3
- package/docs/workflows/operations.md +2 -2
- package/examples/extensions/sandbox/package-lock.json +3 -3
- package/npm-shrinkwrap.json +67 -67
- package/package.json +8 -8
|
@@ -1,12 +1,10 @@
|
|
|
1
|
-
# cua-driver
|
|
1
|
+
# cua-driver: Linux
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
`launch_app`, `list_apps`, `list_windows`, `get_window_state`, and
|
|
9
|
-
session recording.
|
|
3
|
+
Start with `cua-driver doctor` in the graphical user's session. X11,
|
|
4
|
+
XWayland, and native Wayland expose different capture and input facilities;
|
|
5
|
+
successful discovery does not prove either input delivery or window capture.
|
|
6
|
+
Use [WORKFLOW.md](WORKFLOW.md) for exact targets and verification and
|
|
7
|
+
[RUNTIME.md](RUNTIME.md) for transport and permission ownership.
|
|
10
8
|
|
|
11
9
|
On X11, `set_window_frame({pid, window_id, x, y, width, height})` sends an
|
|
12
10
|
EWMH window-manager request and confirms it against `list_windows` geometry.
|
|
@@ -14,137 +12,166 @@ Wayland has no portable protocol for setting another application's top-level
|
|
|
14
12
|
geometry, so this tool refuses there unless a future compositor-owned adapter
|
|
15
13
|
can provide exact targeting and readback.
|
|
16
14
|
|
|
17
|
-
AT-SPI is talked to natively over D-Bus (the `atspi`/zbus crate)
|
|
15
|
+
AT-SPI is talked to natively over D-Bus (the `atspi`/zbus crate): no
|
|
18
16
|
`pyatspi` or GObject-introspection typelibs are required at runtime.
|
|
19
17
|
|
|
20
|
-
##
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
`
|
|
70
|
-
`
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
`
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
`
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
`
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
`
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
`
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
`
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
`
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
18
|
+
## Delivery
|
|
19
|
+
|
|
20
|
+
Background window input must not activate or raise the target. AT-SPI actions
|
|
21
|
+
can reach semantic controls without raw pointer delivery; X11 pixel dispatch
|
|
22
|
+
can use AT-SPI hit-testing or a virtual-pointer route. Do not assume all input
|
|
23
|
+
uses XSendEvent or that every toolkit accepts it. Inspect the result's public
|
|
24
|
+
route and independently verify the application.
|
|
25
|
+
|
|
26
|
+
`delivery_mode:"foreground"` is a user-visible takeover. Never select it automatically.
|
|
27
|
+
Use explicitly authorized `delivery_mode:"foreground"` only after evidence
|
|
28
|
+
that the chosen background route is unavailable or ineffective. Target-specific
|
|
29
|
+
activation and restoration depend on the compositor adapter. If activation
|
|
30
|
+
cannot be proven, accept the refusal instead of sending keys to whatever is focused.
|
|
31
|
+
|
|
32
|
+
The overlay is separate from the physical pointer; cursor-bearing and
|
|
33
|
+
keyboard actions re-show it automatically. Distinct overlays do not isolate
|
|
34
|
+
desktop input. Keep one controller for a global-input workflow.
|
|
35
|
+
|
|
36
|
+
## Capture recovery
|
|
37
|
+
|
|
38
|
+
`get_window_state` requests both AT-SPI and a window screenshot by default.
|
|
39
|
+
A valid image is not guaranteed. If `screenshot_error.code` is
|
|
40
|
+
`surface_identity_unproven`, the driver cannot attest that output pixels
|
|
41
|
+
belong to the requested surface. Preserve that distinction from an empty or
|
|
42
|
+
untrusted accessibility tree. Never crop a desktop image and label it an
|
|
43
|
+
attested window capture.
|
|
44
|
+
|
|
45
|
+
1. Read any returned tree and `degraded_reason`; semantic interaction may still
|
|
46
|
+
be possible without pixels.
|
|
47
|
+
2. Do not retry a window-pixel action without a valid window image. Changing
|
|
48
|
+
`capture_mode`, inventing bounds, or repeatedly increasing timeouts cannot
|
|
49
|
+
establish surface identity.
|
|
50
|
+
3. If the task permits full-display capture and visible desktop control,
|
|
51
|
+
explain the broader scope and obtain authorization if not already given.
|
|
52
|
+
Use `get_desktop_state`, then
|
|
53
|
+
`target:{"kind":"desktop","display_id":"primary"}` on input, and verify
|
|
54
|
+
through another desktop snapshot. See [the desktop loop](WORKFLOW.md#desktop-loop).
|
|
55
|
+
4. If desktop scope is not admitted, report the limitation and stop that route.
|
|
56
|
+
|
|
57
|
+
A capture request may be waiting for an OS/portal permission dialog. Inspect
|
|
58
|
+
window discovery for a pending prompt, or ask the user to check the desktop.
|
|
59
|
+
Let the user approve or deny it. Do not repeatedly start captures, automate
|
|
60
|
+
approval, or change security settings as a workaround. After approval, take a
|
|
61
|
+
fresh capture and verify its dimensions/content before continuing. A timeout
|
|
62
|
+
alone does not prove which permission or backend failed.
|
|
63
|
+
|
|
64
|
+
Supporting smoke evidence: Driver 0.23.2 on KDE/KWin Wayland opened an Electron
|
|
65
|
+
music app, searched, selected an album track, and showed playback through this
|
|
66
|
+
desktop route after user screenshot approval. The window route returned
|
|
67
|
+
`surface_identity_unproven`; desktop actions returned `global_input` and
|
|
68
|
+
`unverifiable`, so screenshots and user confirmation established the result.
|
|
69
|
+
This is not certification of background targeting, other compositors, or video.
|
|
70
|
+
|
|
71
|
+
## X11 observation and input details
|
|
72
|
+
|
|
73
|
+
**What is open over the window.** `get_window_state` lists the pid's transient
|
|
74
|
+
dialogs (`dialogs: [{window_id, title, transient_for, modal, bounds}]`) and
|
|
75
|
+
override-redirect popups (`popups: [...]`) that are mapped, with a `follow_up`
|
|
76
|
+
sentence naming the call that targets each one
|
|
77
|
+
(`get_window_state(pid, window_id=<that id>)`). When one overlaps the window
|
|
78
|
+
the screenshot is taken from the screen (`screenshot_composited:true`) so the
|
|
79
|
+
menu or dialog is visible; the window's own drawable never shows them. The
|
|
80
|
+
payload states its frame (`coordinate_frame:"window"`, `frame_note`): `x`/`y`
|
|
81
|
+
of the pointer tools are pixels of THIS screenshot. Closed menus are listed
|
|
82
|
+
with a `description` ("closed menu with N items…") and are not walked;
|
|
83
|
+
click the menu (a real press) and read the popup by its window_id.
|
|
84
|
+
`press_key` / `hotkey` that map a new top-level report it as `window_opened`
|
|
85
|
+
with `window_change` evidence. Action results carry the same text as
|
|
86
|
+
`summary` inside `structuredContent`, for clients that only show that.
|
|
87
|
+
|
|
88
|
+
**Screenshot scale.** The window screenshot is delivered at or below 1.15
|
|
89
|
+
megapixels (long edge ≤ `max_image_dimension`, 1568 by default): larger
|
|
90
|
+
images are downsized before a model reads them, and its pixel coordinates
|
|
91
|
+
would then be uniformly short. Pointer `x`/`y` and each element's
|
|
92
|
+
`screenshot_frame` are pixels of the delivered image (the element's `frame`
|
|
93
|
+
stays in screen coordinates, as on every platform); `frame_scale` < 1 and
|
|
94
|
+
`screenshot_original_width` report the downsizing, and the driver scales your
|
|
95
|
+
pixels back to the window.
|
|
96
|
+
An explicit per-call `max_image_dimension` (0 = native) replaces this cap.
|
|
97
|
+
|
|
98
|
+
**Budget the pixels before you aim.** The cap is a pixel budget, so on a
|
|
99
|
+
high-resolution window the delivered image is a small fraction of native and
|
|
100
|
+
fine controls stop being readable. Measured on a 3840x2342 window: `frame_scale`
|
|
101
|
+
0.357, so a 24px toolbar icon arrives at ~8.6px and a 1px border at ~0.36px:
|
|
102
|
+
sub-pixel for any reader, sighted or not. Check `frame_scale` in the
|
|
103
|
+
`get_window_state` payload and, when a target is smaller than roughly 30px, frame
|
|
104
|
+
it with `zoom` before choosing a coordinate. Zooming a 196x728 window-px region
|
|
105
|
+
returned a 235x873 image (about 1.2x native), which restores a 24px icon to
|
|
106
|
+
~29px. `zoom` takes screenshot pixels and the follow-up action needs
|
|
107
|
+
`from_zoom: true`. It also requires a screenshot-owning snapshot from the same
|
|
108
|
+
session first, and refuses with `screenshot_context_missing` otherwise, so
|
|
109
|
+
`get_window_state` with a screenshot must precede the first `zoom`.
|
|
110
|
+
|
|
111
|
+
**When the active model takes no images.** Some clients drop image content
|
|
112
|
+
entirely, so `get_window_state` and `zoom` return an image the reader never sees.
|
|
113
|
+
The loop still closes for most questions, and none of this needs a vision model:
|
|
114
|
+
`verify_state` proves state and returns a real `satisfied` / `unsatisfied` /
|
|
115
|
+
`unknown` verdict; an out-of-band read such as
|
|
116
|
+
`xprop -root _NET_ACTIVE_WINDOW` proves which window holds focus; and scanning the
|
|
117
|
+
written capture for colour transitions (PIL or equivalent) proves coarse layout.
|
|
118
|
+
What none of them prove is that a specific small control was activated. For that
|
|
119
|
+
use the `cua-perception` extension (see `VISUAL.md`) or an app that exposes a
|
|
120
|
+
usable accessibility tree.
|
|
121
|
+
|
|
122
|
+
**Windows without `_NET_WM_PID`.** Tk, many Java/AWT builds, Wine, and legacy
|
|
123
|
+
Xlib/Xt clients do not publish `_NET_WM_PID`. On X11, `list_windows` then asks
|
|
124
|
+
the X server's X-Resource extension (XRes 1.2 `LocalClientPID`) which local
|
|
125
|
+
process created the window, so these windows still carry a `pid` and accept
|
|
126
|
+
`get_window_state` / `click` by `pid` + `window_id`. The driver never guesses:
|
|
127
|
+
`pid` stays `null` when XRes is missing or older than 1.2, when the client
|
|
128
|
+
is remote (TCP, or SSH-forwarded with a `WM_CLIENT_MACHINE` naming another
|
|
129
|
+
host), or when the X server cannot report the driver's own PID correctly
|
|
130
|
+
(remote `DISPLAY`, container PID namespace). A `pid: null` window can only be
|
|
131
|
+
reached through desktop scope. An SSH-forwarded client that publishes
|
|
132
|
+
neither `_NET_WM_PID` nor `WM_CLIENT_MACHINE` is attributed to the local
|
|
133
|
+
`ssh` process, because that is the socket peer the server sees.
|
|
134
|
+
|
|
135
|
+
**Keys while the app's own popup is open.** A Qt combo list / completer or a
|
|
136
|
+
GTK/VCL menu holds a keyboard grab that makes the X server drop keys from the
|
|
137
|
+
virtual keyboard. `press_key` / `hotkey` / `type_text` / `set_value` then go
|
|
138
|
+
through the core keyboard (`path: "xtest_core_grab"`, the result names the
|
|
139
|
+
popup and states that the core focus and active window were verified
|
|
140
|
+
unchanged), or, when another application holds the core focus, are refused
|
|
141
|
+
with `code: "popup_keyboard_grab"` and a hint: dismiss the popup (click
|
|
142
|
+
outside it, or click one of its rows by element_token after
|
|
143
|
+
`get_window_state(pid, window_id=<popup>)`) and retry. Typing an absolute path
|
|
144
|
+
into a Qt file dialog opens its completer after the first `/`; prefer
|
|
145
|
+
`set_value` on the "File name" field, which writes the path in one go.
|
|
146
|
+
|
|
147
|
+
**Popup walks, labels, descriptions.** `get_window_state(pid, window_id=<popup>)`
|
|
148
|
+
returns the popup's own rows (list / tree / menu items), never the main
|
|
149
|
+
window's menubar. Elements carry `description` when the toolkit publishes one
|
|
150
|
+
(Qt keeps a button's tooltip there: `push button "" (description "Pause")`;
|
|
151
|
+
`label` falls back to it). A control's `label` is never its value: a spin
|
|
152
|
+
button or slider nobody names is `unlabelled: true` with its place in
|
|
153
|
+
`description` (`unlabelled spin button; 3rd of 4 spin buttons in this panel`,
|
|
154
|
+
in visual order). `parent_index` is the nearest indexed real ancestor.
|
|
155
|
+
|
|
156
|
+
**Grid presses.** After a click that lands on a table cell (LibreOffice Calc),
|
|
157
|
+
the result appends `focus: cell D2` (`focused_cell`, evidence) read from the
|
|
158
|
+
focus log, so a one-row miss is visible before you type.
|
|
159
|
+
|
|
160
|
+
**Multi-click.** `click` takes `count: 2` (double) or `count: 3` (triple, a
|
|
161
|
+
line/paragraph selection in editors) as one press train with real double-click
|
|
162
|
+
cadence in both delivery modes; there is no `triple_click` tool.
|
|
163
|
+
|
|
164
|
+
**Typing throughput.** Key-event typing (XTest in foreground, the virtual
|
|
165
|
+
keyboard in background) runs at roughly 45-60 characters per second, so a
|
|
166
|
+
1,300-character script takes about 30 s: scale a client-side `type_text`
|
|
167
|
+
deadline with the text length instead of using a flat one (0.04 s per
|
|
168
|
+
character leaves margin over those rates). A terminal can still be echoing when the call returns; read the
|
|
169
|
+
result again before retyping text that looks truncated.
|
|
170
|
+
|
|
171
|
+
**Cross-application drops.** A pointer action whose point lies over another
|
|
172
|
+
application's window (a drag dropped onto VLC) reports that window's title
|
|
173
|
+
change and the windows its pid opened as `foreign_window` / `window_change`
|
|
174
|
+
evidence; the focus guard alone only watches the target pid.
|
|
148
175
|
|
|
149
176
|
## Native application menus
|
|
150
177
|
|
|
@@ -158,13 +185,13 @@ native `do_action` acknowledgement alone is not task completion.
|
|
|
158
185
|
|
|
159
186
|
## AT-SPI needs the session bus (headless / containers / `runuser`)
|
|
160
187
|
|
|
161
|
-
AT-SPI
|
|
162
|
-
clicks, and focus-free `type_text`
|
|
188
|
+
AT-SPI (the accessibility tree behind `get_window_state`, `element_token`
|
|
189
|
+
clicks, and focus-free `type_text`) lives **entirely on the desktop
|
|
163
190
|
session's D-Bus**. cua-driver reaches it via `DBUS_SESSION_BUS_ADDRESS`. When
|
|
164
|
-
the daemon is started
|
|
165
|
-
exported and everything works. When it is started **outside** the session
|
|
166
|
-
a container entrypoint, a headless box, `runuser`/`su` into the desktop user,
|
|
167
|
-
a systemd
|
|
191
|
+
the daemon is started _inside_ a normal desktop login that variable is already
|
|
192
|
+
exported and everything works. When it is started **outside** the session
|
|
193
|
+
(a container entrypoint, a headless box, `runuser`/`su` into the desktop user,
|
|
194
|
+
a systemd _system_ unit, or a VNC session running its own ad-hoc bus), the
|
|
168
195
|
variable is unset, the AT-SPI registry walk comes back empty, and
|
|
169
196
|
`get_window_state` reports **every** window as having no elements.
|
|
170
197
|
|
|
@@ -175,10 +202,10 @@ process's `/proc/<pid>/environ` (`xfce4-session`, `gnome-session`, …). So the
|
|
|
175
202
|
common headless cases now "just work". The two things that still must be true:
|
|
176
203
|
|
|
177
204
|
1. **An accessibility bus must be running** in that session, and
|
|
178
|
-
**`toolkit-accessibility` must be on
|
|
205
|
+
**`toolkit-accessibility` must be on**: cua-driver advertises a screen
|
|
179
206
|
reader at startup to flip it, but a session with no a11y bus at all
|
|
180
207
|
(`/usr/libexec/at-spi-bus-launcher`) can't expose a tree. `cua-driver
|
|
181
|
-
|
|
208
|
+
doctor` now probes `org.a11y.Bus` for real (not just "is there a bus?")
|
|
182
209
|
and tells you which of the two is missing.
|
|
183
210
|
2. The daemon must run **as the desktop user** (so it can read that user's
|
|
184
211
|
session-process environ and the `/run/user/<uid>/bus` socket). Running the
|
|
@@ -194,20 +221,26 @@ bridge isn't up / the daemon isn't on the session bus".
|
|
|
194
221
|
|
|
195
222
|
Each input rung and its stable public route:
|
|
196
223
|
|
|
197
|
-
| Modality
|
|
198
|
-
|
|
199
|
-
| Element click (`
|
|
200
|
-
| **element px action (x,y)**
|
|
201
|
-
| Pixel (px) click, escalated
|
|
202
|
-
| `type_text` into editable
|
|
203
|
-
| `type_text`, non-editable focus | `background`/`foreground` | `synthetic_events` or `global_input`
|
|
224
|
+
| Modality | `delivery_mode` | `route` | Postcondition proof |
|
|
225
|
+
| ------------------------------- | ------------------------- | -------------------------------------------------------------------- | -------------------------------------------------------- |
|
|
226
|
+
| Element click (`element_token`) | `background` | `accessibility` | Use `verify_state`; invocation alone is not confirmation |
|
|
227
|
+
| **element px action (x,y)** | `background` | `accessibility` when AT-SPI-at-point lands, otherwise `global_input` | Use `verify_state` or multimodal reading |
|
|
228
|
+
| Pixel (px) click, escalated | `foreground` | `global_input` | Use `verify_state` or multimodal reading |
|
|
229
|
+
| `type_text` into editable | `background` | `accessibility` | `confirmed` only with `value_readback` evidence |
|
|
230
|
+
| `type_text`, non-editable focus | `background`/`foreground` | `synthetic_events` or `global_input` | Use `verify_state` or multimodal reading |
|
|
204
231
|
|
|
205
|
-
**A background element px action does land on X11
|
|
232
|
+
**A background element px action does land on X11**: for an AX-exposing app it
|
|
206
233
|
takes the focus-free AT-SPI `do_action`-at-point path (`x11_atspi`), exactly
|
|
207
234
|
like the macOS/Windows background pixel click. It falls to the MPX
|
|
208
235
|
virtual-pointer path (`x11_pixel`) only for non-AX surfaces, **and that path
|
|
209
|
-
needs a real Xorg + `/dev/uinput
|
|
210
|
-
uinput, escalate to `delivery_mode:"foreground"`.
|
|
236
|
+
needs a real Xorg + `/dev/uinput`**: under Xvnc / minimal containers without
|
|
237
|
+
uinput, escalate to `delivery_mode:"foreground"`. The AT-SPI path only fires
|
|
238
|
+
a control under the point: when the hit test finds nothing deeper than the
|
|
239
|
+
application's own frame or window (Chromium page content before its AT-SPI
|
|
240
|
+
tree is populated), no action is fired. Chromium/Electron targets without a
|
|
241
|
+
real focus-free pointer return `background_unavailable` before the capture is
|
|
242
|
+
consumed, so retry the same capture-bound click with
|
|
243
|
+
`delivery_mode:"foreground"` when foreground input is authorized. (`type_text` in the
|
|
211
244
|
`background` rung is focus-dependent for non-editable widgets; that's the one
|
|
212
245
|
genuine background limitation, and `foreground` is the documented escalation.)
|
|
213
246
|
|
|
@@ -277,12 +310,14 @@ identity refuses. Package eligibility does not certify every LibreOffice
|
|
|
277
310
|
application or operation.
|
|
278
311
|
|
|
279
312
|
The plugin separately binds the exact live native surface and checks geometry,
|
|
280
|
-
desktop availability, primary-client and other-lane conflicts
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
313
|
+
desktop availability, and primary-client and other-lane conflicts. Each
|
|
314
|
+
background lane publishes a private canonical `evdev`/`pc105`/`us` keymap, so
|
|
315
|
+
user options such as Caps Lock remapped to Ctrl or Super do not alter agent key
|
|
316
|
+
semantics. A physical keymap transition cancels existing authority; use a fresh
|
|
317
|
+
action afterward. Multiple agent layout groups, Unicode, IME input, arbitrary
|
|
318
|
+
held-key streams, and modified pointer gestures remain outside this scope.
|
|
319
|
+
Chromium, Electron, and XWayland raw background input are outside this scope.
|
|
320
|
+
AT-SPI routes retain their separate behavior.
|
|
286
321
|
|
|
287
322
|
Two compositor seats, `Cua-Agent` and `Cua-Agent-2`, persist across configuration
|
|
288
323
|
disable/re-enable. Each connection claims one lane, and each admitted action
|
|
@@ -302,7 +337,15 @@ native harness covers defined GTK3, Electron, and Tauri foreground cases. It ref
|
|
|
302
337
|
held physical input, grabs, constraints, drag-and-drop, ambiguous primary seat
|
|
303
338
|
bindings, and non-neutral keyboard modifiers. Background refusal never selects
|
|
304
339
|
this route automatically. Driver expands bounded ASCII text under the exact
|
|
305
|
-
US keymap; Unicode and IME remain outside its raw-input scope.
|
|
340
|
+
US keymap; Unicode and IME remain outside its raw-input scope. Foreground
|
|
341
|
+
pointer-only actions are layout-independent. Foreground keyboard actions leave
|
|
342
|
+
the user's Num Lock and Caps Lock untouched. They work with Num Lock on and with
|
|
343
|
+
keymap options that leave every typing and modifier key unchanged, such as
|
|
344
|
+
`compose:caps`. They refuse before any input under Caps Lock
|
|
345
|
+
(`foreground_keyboard_caps_lock`), for a keypad key that Num Lock changes
|
|
346
|
+
(`foreground_keyboard_numlock_keypad`), and for a different layout or remapped
|
|
347
|
+
key (`foreground_unsupported_layout`). Ask the user to turn Caps Lock off, or
|
|
348
|
+
use the equivalent non-keypad key, instead of retrying the same call.
|
|
306
349
|
|
|
307
350
|
The retained bounded app evidence at source
|
|
308
351
|
`f180e8828b8f31cc153e3c44eaa89a9c13c5bc68` includes instrumented Calc/Inkscape
|
|
@@ -320,45 +363,44 @@ Fleet image.
|
|
|
320
363
|
|
|
321
364
|
If a tool call surprises you on Linux:
|
|
322
365
|
|
|
323
|
-
1. `cua-driver doctor
|
|
366
|
+
1. `cua-driver doctor`: reports the display server (X11 / Wayland),
|
|
324
367
|
**whether `org.a11y.Bus` actually answers on the session bus** (not just
|
|
325
368
|
"is there a bus"), the discovered `DBUS_SESSION_BUS_ADDRESS`, and
|
|
326
369
|
`ffmpeg` availability (for recording).
|
|
327
|
-
2. Check `XDG_SESSION_TYPE
|
|
370
|
+
2. Check `XDG_SESSION_TYPE`: X11 still has toolkit-specific delivery limits; `wayland`
|
|
328
371
|
needs `CUA_DRIVER_RS_ENABLE_WAYLAND=1` for the native backend,
|
|
329
372
|
else XWayland.
|
|
330
|
-
3. **Empty AT-SPI tree** (`get_window_state` returns `degraded:true`)
|
|
373
|
+
3. **Empty AT-SPI tree** (`get_window_state` returns `degraded:true`): in
|
|
331
374
|
order of likelihood: (a) the daemon isn't on the desktop session bus
|
|
332
|
-
(headless / container / `runuser` / root-against-user-session
|
|
333
|
-
|
|
375
|
+
(headless / container / `runuser` / root-against-user-session; see
|
|
376
|
+
_AT-SPI needs the session bus_ above; doctor will say
|
|
334
377
|
`DBUS_SESSION_BUS_ADDRESS unset`); (b) the a11y bridge is off
|
|
335
378
|
(`gsettings set org.gnome.desktop.interface toolkit-accessibility true`);
|
|
336
|
-
(c) GTK4 / Qt6 / Chromium populate lazily
|
|
379
|
+
(c) GTK4 / Qt6 / Chromium populate lazily: re-snapshot after an
|
|
337
380
|
interaction or an AX-enable settle.
|
|
338
381
|
|
|
339
382
|
## Forbidden vectors
|
|
340
383
|
|
|
341
|
-
Same idea as macOS / Windows
|
|
384
|
+
Same idea as macOS / Windows: don't shell out to anything that
|
|
342
385
|
foregrounds a target:
|
|
343
386
|
|
|
344
|
-
- `wmctrl -a <window>` / `wmctrl -R <window
|
|
345
|
-
- `xdotool windowactivate <wid
|
|
346
|
-
- `xdotool key --window <wid> alt+Tab
|
|
387
|
+
- `wmctrl -a <window>` / `wmctrl -R <window>`: activates / raises.
|
|
388
|
+
- `xdotool windowactivate <wid>`: activates.
|
|
389
|
+
- `xdotool key --window <wid> alt+Tab`: focus churn.
|
|
347
390
|
|
|
348
391
|
Prefer cua-driver tools with an explicit `window_id`. When in doubt,
|
|
349
392
|
ask the user.
|
|
350
393
|
|
|
351
394
|
## What to expect
|
|
352
395
|
|
|
353
|
-
| Environment
|
|
354
|
-
|
|
355
|
-
| X11/Openbox
|
|
356
|
-
| Sway/wlroots
|
|
357
|
-
| Hyprland/Omarchy
|
|
358
|
-
| GNOME/Mutter
|
|
359
|
-
| KDE/KWin
|
|
360
|
-
| Nested `cua-compositor` | Versioned direct per-surface input, native GTK 31/31, capture/scope 5/5, and partial Electron coverage
|
|
361
|
-
|
|
362
|
-
See
|
|
363
|
-
pixel-click contract, failure modes) and `RECORDING.md` for session
|
|
396
|
+
| Environment | Proven baseline | Main limits |
|
|
397
|
+
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
398
|
+
| X11/Openbox | AT-SPI trees and actions, foreground pointer and keyboard input, window and desktop capture, and video | Raw background delivery remains toolkit-specific; unsupported shapes refuse |
|
|
399
|
+
| Sway/wlroots | AT-SPI, native discovery, full-display and cropped-window screencopy, foreground input, semantic background actions, and video | Raw background pointer and keyboard input remains focus-bound |
|
|
400
|
+
| Hyprland/Omarchy | Experimental source candidate with separate discovery-foundation and bounded two-seat app evidence | Default plugin is discovery-only; raw background v3 qualification is limited to the exact native Calc/Inkscape packages and a private agent US keymap; user Caps-to-Ctrl/Super remaps do not alter agent semantics; complete native harness and release acceptance are separate gates |
|
|
401
|
+
| GNOME/Mutter | AT-SPI, WinRects geometry and activation, capture, and portal/libei foreground input | Requires the helper and portal grant; portal video parity remains open |
|
|
402
|
+
| KDE/KWin | AT-SPI and generic discovery where exposed | Target-specific activation and behavioral coverage remain experimental |
|
|
403
|
+
| Nested `cua-compositor` | Versioned direct per-surface input, native GTK 31/31, capture/scope 5/5, and partial Electron coverage | The complete shared matrix remains experimental; do not infer standard-Wayland support |
|
|
404
|
+
|
|
405
|
+
See [WORKFLOW.md](WORKFLOW.md) for exact targeting and verification and `RECORDING.md` for session
|
|
364
406
|
recording.
|