@bastani/atomic 0.9.56-alpha.1 → 0.9.57-alpha.1

package/dist/builtin/subagents/skills/cua-driver/WINDOWS.md

@@ -1,832 +0,0 @@
-# cua-driver: Windows
-
-Orchestrates Windows app automation via the `cua-driver` binary (`cua-driver.exe`). Whenever a user
-asks to drive a native Windows app, follow the loop in this doc
-rather than calling tools ad-hoc: the snapshot-before-action
-invariant is not optional and silently breaks if you skip it.
-
-`SKILL.md` in this directory describes the cross-platform core;
-this file is the Windows-specific extension. Read both:
-the snapshot invariant, MCP-vs-CLI choice, agent cursor overlay, and
-recording flow are identical. The launch, click, and accessibility-
-tree mechanics in this file replace the macOS ones.
-
-## Background window delivery
-
-**Background window actions must preserve the user's frontmost app.**
-Foreground and desktop actions follow the authorization boundary in
-[RUNTIME.md](RUNTIME.md#foreground-boundary). Users pay for the right
-to keep typing in their editor while an agent drives another app in
-the background. Violate this rule and every other nice property the
-driver gives you (no cursor warp, no taskbar flash, no window
-restore-and-raise) stops mattering: you just shipped a `SendInput`
-wrapper with extra steps.
-
-### How the contract is enforced per call: the `delivery_mode` field
-
-Every Windows input tool (`click`, `double_click`, `right_click`,
-`drag`, `scroll`, `press_key`, `hotkey`, `type_text`) accepts an
-optional `delivery_mode` field: this mirrors the macOS `delivery_mode`
-surface (same name, same two values). The default is `"background"`:
-strict no-foreground:
-
-| `delivery_mode`          | Behavior on Windows                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
-| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `"background"` (DEFAULT) | Never fronts and **never raises/restacks** the target: macOS-aligned (mirrors CGEvent-to-pid). **Pixel clicks**: a UIA hit-test at the point first (accessibility-channel Invoke, works on UWP / WinUI3 / Win11 packaged apps, no flash); if that misses, coordinate-injected pen/touch, **but only when the target is the _visible_ window at that point**; PostMessage for plain Win32. It returns a structured `background_unavailable` error (rather than raising or fronting) when the target is **occluded** at the point, or the event kind is known-dropped (Chromium DOM mouse + key-combos, GTK buttons, VCL/LibreOffice accelerators, terminal / WPF text with no `element_token`). **No foreground swap and no z-order raise, ever.** |
-| `"foreground"`           | SendInput with brief `SetForegroundWindow(target)` → restore. The explicit, agent-chosen rung where fronting IS allowed: required to reach occluded targets, Chromium DOM content, GTK buttons, VCL accelerators, WPF drag, terminals, and canvas / custom-drawn surfaces with no UIA peer. Implemented for **every** input tool: `type_text` (SendInput Unicode via `send_text_synthesized`) and `scroll` (SendInput wheel via `send_wheel_synthesized`) included. The activation and restoration are scoped to that action.                                                                                                                                                                                                                       |
-
-> **macOS is the source of truth: `background` never alters the screen.**
-> Earlier Windows builds "cheated" in background with three tricks that this
-> pass **removed**: (1) a z-order raise (`ZorderGuard`) to win the pointer
-> hit-test on occluded windows, (2) a full focus-activate for WPF drags, and
-> (3) a _cloaked_ (hidden) focus-grab for keystrokes/text the target would
-> otherwise drop. macOS does none of these (pure CGEvent-to-pid + focus
-> suppression), so Windows now does none either: when strict no-front /
-> no-raise delivery can't land, the tool returns `background_unavailable` and
-> **the agent (not the driver) decides** whether to escalate to
-> `delivery_mode:"foreground"` (the rung where fronting is explicitly opted
-> into). A `background` call will never raise, restack, flash, or steal focus.
-
-> **Removed: the legacy `"auto"` mode.** Earlier builds had a third
-> Windows-only `dispatch:"auto"` mode (silent SendInput fallback on
-> known-problematic targets). It was removed in the macOS-alignment pass
-> because it could front the target _without the caller opting in_:
-> breaking the no-foreground contract macOS guarantees. Any unrecognised
-> value (including a stray `"auto"`) now resolves to `"background"`. If you
-> have notes/snippets that pass `dispatch:"auto"`, switch to an explicit
-> `delivery_mode:"foreground"` for the cases that need fronting.
-
-### Always try `delivery_mode:"background"` first
-
-The hit-test fallback above means **`delivery_mode:"background"` is the correct
-default even when the target is a XAML host whose input stack drops raw
-PostMessage** (UWP Calculator, Win11 Notepad, WinUI3 apps, etc.). cua-driver
-turns the pixel coord into a UIA Invoke at that point and delivers
-through the accessibility channel: no flash, no focus steal. Only
-escalate to `delivery_mode:"foreground"` when you actually see a
-`background_unavailable` structured error. Retry only that action with
-`delivery_mode:"foreground"`; cua-driver briefly activates the target, delivers
-the input, and restores the previous foreground.
-
-Empirical: pixel-clicks via `delivery_mode:"background"` against the UWP
-Calculator on Win11 (Number-pad buttons + operators) consistently
-resolve through `try_invoke_in_window_at_point` and produce
-`"✅ Performed UIA Invoke at (sx,sy) for pid X."` with zero visible
-flash. `delivery_mode:"foreground"` on the same coords also works but
-flashes the Calculator window foreground for ~40 ms: it's the
-costlier path; only use it for surfaces with no UIA peer.
-
-`background_unavailable` error shape:
-
-```json
-{
-  "isError": true,
-  "structuredContent": {
-    "code": "background_unavailable",
-    "target_class": "Chrome_WidgetWin_1",
-    "event_kind": "mouse_click",
-    "escalation": { "recommended": "foreground", "reason": "occluded / known-dropped event kind" },
-    "suggestion": "Retry this action with delivery_mode:\"foreground\"."
-  }
-}
-```
-
-Errors retain their diagnostic `escalation.recommended` hint. Successful
-action results use the narrower `escalation.target` contract instead (see
-`SKILL.md` → behavior matrix). On Windows this error recommendation is
-`"foreground"` because the dropped event needs the fronting rung. (Contrast
-macOS / X11, where a background pixel click can still land in the background.)
-
-The normal flow when an agent gets that error:
-
-1. Confirm the user authorized visible control, then reissue only the refused action with `delivery_mode:"foreground"`.
-2. cua-driver activates the target, delivers through SendInput, and restores
-   the previous foreground.
-3. Continue with `delivery_mode:"background"` for later actions unless they
-   are also refused.
-
-### Persistent focus-proxy exception
-
-`bring_to_front` is not part of the normal input ladder. Use it only when a
-focus-proxy surface must remain foreground across multiple calls, such as an
-RDP or Windows App session, or when repeated action-scoped activation prevents
-the remote surface from accepting input.
-
-The `bring_to_front` tool uses an `AttachThreadInput` trick to _attempt_
-the foreground swap even when the daemon isn't at UIAccess integrity (the
-same trick that powers `send_key_synthesized`). Returns
-`{previous_fg_hwnd, now_fg_hwnd, landed_on_target}`: **check
-`landed_on_target`**. Without UIAccess, Windows' foreground-lock can still
-reject the swap (and a subsequent `delivery_mode:"foreground"` call will
-bail with the "Foreground swap … was rejected by Windows" diagnostic
-rather than landing input on the wrong window). When that happens the
-target genuinely cannot be driven by SendInput/keystrokes in this session:
-use an interactively launched High-IL daemon. The reserved `cua-driver-uia`
-worker is a daemon-internal, default-off service boundary and public clients
-must never connect to its pipe directly. Alternatively, for tasks
-that produce a file: generate the document and `launch_app` it instead of
-driving the GUI (e.g. building a spreadsheet and opening it in LibreOffice
-Calc rather than typing into the grid, which is dropped on the VCL
-background path).
-
-Before running any shell command, ask: **"does this raise, activate,
-foreground, or steal focus from any app?"** If yes, don't run it.
-Every one of the commands below activates the target on Windows and
-is therefore forbidden unless the user **explicitly** asked for
-frontmost state:
-
-- **`Start-Process <exe>` / `Start-Process <url>` / `Start-Process
--FilePath ...`**: defaults to launching with `SW_SHOWNORMAL` which
-  _activates_ the new window. Windows treats new processes as
-  user-initiated foreground apps. The CmdLine flag `-WindowStyle Hidden`
-  helps but does not block activation for apps that call
-  `SetForegroundWindow` themselves on startup (Edge, most browsers,
-  Office, most installers). **Never use `Start-Process` to launch a
-  visible app.** Use `launch_app`: it goes through
-  `SW_SHOWNOACTIVATE` and wraps an internal `VK_NONAME keybd_event`
-  trick that the OS treats as "user activity from another window," so
-  the target's own `SetForegroundWindow` call is denied per Windows
-  focus-stealing prevention rules.
-- **`& "C:\path\to\app.exe"` from PowerShell**: same as
-  `Start-Process` on activation. PowerShell's `&` invocation operator
-  spawns the process with foreground intent.
-- **`cmd /c start <thing>` and `start /b <thing>`**: `start` on
-  Windows is the equivalent of macOS's `open`: it goes through the
-  shell's protocol-handler / file-association lookup and activates
-  the resulting process. `start /min` helps for taskbar minimization
-  but still activates the new window before minimizing it (flash
-  visible to the user). Forbidden for the same reason.
-- **`explorer.exe shell:AppsFolder\<AUMID>` / `explorer.exe ms-edge:
-<url>`**: these are the Windows-shell equivalents of `open -a` /
-  `open <url>` on macOS. They go through `IApplicationActivationManager`
-  with the wrong activation kind and foreground the target. Use
-  `launch_app({aumid})` or `launch_app({urls})` instead: those route
-  through `IApplicationActivationManager::ActivateApplication` with
-  the correct flag combination that respects `SW_SHOWNOACTIVATE`.
-- **`SetForegroundWindow(hwnd)` / `BringWindowToTop(hwnd)` /
-  `SwitchToThisWindow(hwnd, fAltTab)` / `ShowWindow(hwnd, SW_SHOW)` /
-  `SetActiveWindow(hwnd)`**: all foreground the named window by
-  definition. Never call these from Bash via PowerShell add-types or
-  C# inline. If you find yourself doing this to "make a click land,"
-  the click is already wrong: re-read "Click semantics" below.
-- **`AttachThreadInput` + `SetForegroundWindow` trickery**: the
-  classic Windows focus-bypass hack. Even when it works it's a
-  visible focus pop and a UIPI/UIAccess violation. Forbidden.
-- **`SendInput(MOUSEEVENTF_*)` over the user's real cursor**: moves
-  the cursor and synthesizes input. The cursor warps to coordinates,
-  any handler in the topmost window at those coordinates fires, and
-  for most apps the receiving window activates because input arrived
-  from the OS-trusted pipeline. Use `click({pid, x, y})` or
-  `click({pid, element_token})` instead: both route per-pid and
-  never touch the OS cursor.
-- **`SendInput(KEYBDINPUT)` with no target HWND**: same idea, it goes
-  to the focused window, not your target. Use `hotkey({pid, keys:
-[...]})` which uses `PostMessage(WM_KEYDOWN/UP)` to the named pid's
-  focused window.
-- **Keyboard shortcuts that semantically mean "focus here":
-  Chromium / Edge / Firefox `Ctrl+L` (focus address bar),
-  Explorer's `Ctrl+L` / `Alt+D` (focus path bar), `F6` (focus
-  cycle).** These aren't pure key events: the receiving app
-  interprets "user wants to type here" as activation intent and
-  raises its window to be key. Even when delivered to a backgrounded
-  pid via `hotkey`, the downstream app pulls focus. **For omnibox
-  navigation specifically**, the correct path is `launch_app({path:
-"...msedge.exe", additional_arguments: ["https://example.com/"]})`.
-  This hands the URL to the selected browser process; no omnibox dance
-  or `Ctrl+L` is needed. Do not combine `path` or `aumid` with `urls`
-  for selected-browser navigation: `urls` opens separately in the default
-  browser. For the default browser, use `urls` only: `launch_app({urls: [url]})`.
-- **Tab-switching shortcuts in browsers (`Ctrl+1..9`, `Ctrl+Tab`,
-  `Ctrl+Shift+Tab`, `Ctrl+PageUp/Down`)** are visibly disruptive
-  even when delivered to a backgrounded pid. The app's key handler
-  processes the shortcut, the window re-renders the new tab's
-  content, the user sees their tabs flipping. Same as macOS:
-  there is no UIA-only workaround: page content (HTML, DOM,
-  WebView2's accessible tree) populates only for the focused tab;
-  inspecting a background tab requires activating it, which is the
-  visible flip.
-
-  **Prefer the windows-over-tabs pattern**: for each URL you need to
-  drive backgrounded, use `launch_app({urls: [url]})`. Chromium-
-  family browsers open each URL in a new **window**. Each window has
-  its own `window_id`, its own UIA tree, and can be inspected /
-  interacted with via `element_token` without activating or switching
-  anything. Tabs are a UX grouping for humans; cua-driver-rs
-  workflows should default to windows.
-
-- **Win+key shortcuts owned by the shell**: `Win+E` (Explorer),
-  `Win+R` (Run), `Win+S` / `Win+Q` (Search), `Win+number` (taskbar
-  pinned-app activation), `Win+Tab` (Task View), `Alt+Tab` (window
-  switcher), `Win+D` (show desktop), `Win+L` (lock), `Win+M` /
-  `Win+Up/Down` (minimize / maximize). All hard-coded to the shell
-  and visibly disruptive: they bypass any per-pid routing. Forbidden
-  in `hotkey` calls regardless of target pid.
-- **`taskkill /F /IM <exe>` to close an app the user is using**:
-  not a focus-steal but a data-loss vector. Use
-  `hotkey({pid, keys:["alt","f4"]})` and let the app close cleanly,
-  or ask the user first.
-
-Reading state is fine. Listing windows, reading registry, querying
-process info via `Get-Process`, calling `tasklist`, walking UIA trees
-via `cua-driver get_window_state`: none of these change focus.
-**Mutating state via shell shims is the line.**
-
-**Corollary: the Win+Search rule.** Don't use Win+S/Win+Q "open Start
-search, type query, press Enter" patterns to launch apps. They (a)
-foreground the Search UI, (b) leave the Search UI populated with the
-agent's typed text, (c) trigger the new app via the Start Menu's own
-activation path which `SW_SHOWNORMAL`s it. Use `launch_app({name})` /
-`{aumid}` / `{path}` instead.
-
-**"Open \<app\>" in user speech means launch, not activate.**
-`cua-driver launch_app` is the one correct path for process startup:
-it's idempotent (no-op on a running app), returns the pid, and
-internally uses `SW_SHOWNOACTIVATE` plus the AppX-broker activation
-flow for packaged apps so the target's window comes up without
-becoming foreground. The macOS `FocusRestoreGuard` is replaced on
-Windows by the activation-deny dance: the launcher pumps a
-`VK_NONAME` keystroke before activating, which Windows interprets as
-"another app just had user activity" and rejects the target's own
-`SetForegroundWindow` call per [Windows focus-stealing prevention
-rules](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setforegroundwindow).
-
-**`launch_app` foreground-restore behaviour (Windows).** In addition to
-`SW_SHOWNOACTIVATE`, `launch_app` captures `GetForegroundWindow()` before
-the spawn and schedules a post-spawn polling restore (every 100 ms for up
-to 3 s) that flips the foreground back to the prior window once the
-spawned app has actually activated. This mirrors the macOS
-`FocusRestoreGuard`. For `urls`-only invocations (open these links in the
-default browser, no app-identifying field) the restore is **skipped**,
-because the user explicitly asked for the page to come up. The restore is
-best-effort: `SetForegroundWindow` from non-UIAccess processes is subject
-to Windows' foreground-lock and may silently no-op: failures are logged
-at `tracing::trace!` and not surfaced as errors.
-
-**`hotkey` modifier dispatch on Chromium/Electron (Windows).** When
-the target is detected as a Chromium-family window (`Chrome_WidgetWin_*`
-class, covering every Chromium browser AND every Electron app: Chrome,
-Edge, Brave, Arc, Vivaldi, Slack, VS Code, Discord, Teams, Notion),
-`hotkey` with modifiers routes through `PostMessage(WM_KEYDOWN/UP)`
-instead of `SendInput`: no foreground swap. Chromium's
-`Browser::HandleKeyboardEvent` reads modifier state from the WM_KEYDOWN
-LPARAM bits, NOT from `GetKeyState`, so PostMessage delivery works for
-real accelerators (Ctrl+T, Ctrl+W, Ctrl+L, Ctrl+Shift+B, etc.). The
-SendInput-swap path (`send_key_synthesized`) remains the dispatch for
-**non-Chromium Win32 + modifiers**: classic apps (LibreOffice, FAR,
-classic Notepad) use `TranslateAccelerator` which requires the system
-modifier state updated, and PostMessage can't do that.
-
-**`modifier` on a _background_ click is a Windows residual.** A
-backgrounded click delivers through UIA `Invoke` or `PostMessage`, and
-neither carries live keyboard state, so a `modifier` (Ctrl/Shift/etc.)
-passed alongside a `delivery_mode:"background"` click **is not honored**
-on Windows. The `modifier` parameter is part of the shared schema and is
-accepted everywhere; it only takes effect on the SendInput rung, i.e. a
-`delivery_mode:"foreground"` click, where SendInput sets real modifier state.
-If you need a modifier-click on Windows, escalate that one action to
-`foreground`. Reserve `bring_to_front` for the persistent focus-proxy exception
-described above.
-
-### Cross-platform schema residuals (Windows)
-
-The capture/dispatch/addressing params are a shared cross-platform
-contract (see `SKILL.md` → _Cross-platform parameter contract_). Three
-Windows-relevant notes:
-
-- **`session` is now accepted on every action/cursor tool.** Earlier
-  Windows builds rejected `session` via `additionalProperties:false` (it
-  was effectively a macOS-only key); the shared contract makes it
-  uniformly schema-accepted: cursor-wired where a cursor glides,
-  accepted elsewhere.
-- **`debug_window_info` is a Windows-only tool** (window-handle / class /
-  rect / z-order diagnostics for triaging the click chain). It is
-  deliberately not part of the cross-platform surface: there is no macOS
-  or Linux counterpart.
-- **`launch_app` identifiers are platform-specific.** Windows takes
-  `aumid` / `launch_path` / `path` / `start_minimized` (plus `bundle_id`
-  overloaded for AUMIDs); macOS takes `bundle_id` / `urls`. `name` is the
-  portable fallback. See the AUMID section below.
-
-**Chromium pixel-click foreground polling restore.** `click({pid, x, y})`
-on a Chromium target falls through to `send_click_synthesized`
-(SendInput + brief foreground swap) because Chromium's input thread filters by
-queue-origin and PostMessage-delivered clicks don't fire DOM events. The
-synchronous restore inside `send_click_synthesized` covers the
-immediate swap; an additional polling guard (same shape as `launch_app`'s
-`FocusRestoreGuard`) catches the **asynchronous** Chromium re-activation
-that can happen as the renderer's input handler processes the click
-(focus().activate() / WebContents::Activate(), 100-500 ms later). The
-guard is gated on `GetWindowThreadProcessId(fg_now) == pid` so user
-Alt-Tabs are respected. The polling guard is asynchronous and best-effort,
-so the tool response is not proof that the previous foreground has already
-been restored.
-
-## Transport
-
-Follow [RUNTIME.md](RUNTIME.md) for CLI and MCP lifecycle ownership.
-On PowerShell, prefer piping JSON to the CLI to avoid argument-quoting errors:
-
-```powershell
-'{"pid":6004,"session":"run-1"}' | cua-driver list_windows
-```
-
-Reuse the same explicit label for a multi-call CLI run and end it afterward.
-For recording or browser preparation, use one persistent MCP connection.
-
-### Intent → tool mapping
-
-If you find yourself reaching for the right column, something has
-gone wrong: re-read "The no-foreground contract" above.
-
-| Intent                             | Use                                                                                             | Don't use                                                                            |
-| ---------------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
-| Open / launch a Win32 app          | `launch_app({path: "C:\\Program Files\\…\\foo.exe"})` or `{name: "foo"}`                        | `Start-Process`, `cmd /c start`, `& "C:\\path\\foo.exe"`                             |
-| Open / launch a UWP / packaged app | `launch_app({aumid: "Microsoft.Foo_8wekyb3d8bbwe!App"})`                                        | `explorer.exe shell:AppsFolder\\<AUMID>`, Start Menu typing                          |
-| Open a URL in the default browser  | `launch_app({urls: ["https://example.com"]})`                                                   | `Start-Process "https://…"`, `explorer.exe ms-edge:…`, `cmd /c start "" "https://…"` |
-| Find a pid                         | `list_apps` or `launch_app`'s return                                                            | `Get-Process`, `tasklist`, Win+S typing                                              |
-| Enumerate an app's windows         | `list_windows({pid})`, or read the `windows` array `launch_app` already returns                | `Get-Process \| Where-Object { $_.MainWindowHandle }`                                |
-| Move or resize one exact window    | `set_window_frame({pid, window_id, x, y, width, height})`                                       | PowerShell Add-Type wrappers, Win+Arrow, or title-bar dragging                       |
-| Click / type / scroll / keys       | `click`, `type_text`, `scroll`, `press_key`, `hotkey`                                           | `SendInput`, `cliclick`-style C# add-types, AutoHotkey scripts                       |
-| Drag / drag-and-drop               | `drag({pid, from_x, from_y, to_x, to_y})`                                                       | `SendInput` with `MOUSEEVENTF_MOVE`, mouse_event                                     |
-| Screenshot                         | `get_window_state` (window) or authorized `get_desktop_state` (desktop)                         | `[System.Windows.Forms.Screen]::CopyFromScreen`, `nircmd savescreenshot`             |
-| Quit an app                        | ask the user first, then `hotkey({pid, keys:["alt","f4"]})`                                     | `taskkill /F`, `Stop-Process -Force`, `Get-Process \| Stop-Process`                  |
-| Hand a file/URL to an app          | `launch_app({urls:[<path>]})` (default app) or `{path: "...exe", additional_arguments:[<file>]}` (specific app) | `& "app.exe" "file"`, `Invoke-Item`, shell associations                              |
-
-### The narrow carve-out
-
-The **only** legitimate use of `SetForegroundWindow` or
-`Start-Process` with a foreground app is when the user **explicitly**
-asked for frontmost state ("bring Edge to the front", "make
-Calculator visible", "I want to see it"). Reaching for it because a
-tool call returned something confusing is wrong: diagnose first.
-
-When a cua-driver surprises you, diagnose cua-driver first:
-
-- **`Posted click to pid X` instead of `Performed UIA Invoke ...`?**
-  The (x,y) UIA hit-test didn't find an `InvokePattern`-bearing
-  element at that pixel inside the target HWND, so it fell through
-  to `PostMessage(WM_LBUTTONDOWN)`. For UWP / WebView2 surfaces,
-  PostMessage silently no-ops: re-snapshot via
-  `get_window_state(pid, window_id)` and use `element_token` so the
-  daemon can invoke the cached UIA element by identity instead of by
-  point.
-- **`stale_element_token` / `No cached UIA state`?** You either
-  skipped `get_window_state` this turn or used a token from an older
-  snapshot. Snapshots are keyed on `(pid, window_id)`: tokens don't carry across windows of
-  the same app. Re-snapshot with the same window_id you're about to
-  click in.
-- **`Invalid window handle (0x80070578)`?** The HWND you passed is
-  stale (window closed, recreated). Re-resolve via `list_windows`.
-- **Large tree or repeated reads?** `get_window_state` returns one compact
-  markdown tree capped at 250 nodes (`full_output:true` restores 5 000 nodes
-  and both forms). Pass `since:<snapshot_id>` for added/changed/removed rows;
-  the focused element is not reported on Windows yet, so an unchanged read
-  says only `no change since …`.
-- **Empty `tree_markdown` / sparse UIA tree?** Some apps populate
-  their UIA tree lazily on first call; retry `get_window_state`
-  once. If still empty, the app has no UIA provider: fall back to an
-  element px action (x,y clicks off the screenshot) on visible content
-  (acceptable for exploration; pair with screenshots).
-- **Empty tree, or a snapshot with no image?** `get_window_state`
-  returns **both** the UIA tree and a screenshot by default: there is no
-  capture mode to pick. If the tree came back empty, the response is
-  `degraded` (no UIA provider: retry once for lazy trees, see the note
-  above); act by **px** off the screenshot in the same response. The
-  `capture_mode` param is **deprecated and ignored**: it's still accepted
-  so old callers don't error, but both the tree and the image come back
-  regardless of what you pass.
-- **`list_windows` returns Win32 windows but misses UWP / WebView2
-  windows?** UIA desktop enumeration may be degraded because a provider
-  is unresponsive. `list_windows` falls back to Win32-only output instead
-  of hanging; run `cua-driver doctor` and retry after the provider
-  recovers.
-- **Need full-display capture?** Use `get_desktop_state` only for a desktop
-  coordinate loop. To verify a specific window, use
-  `get_window_state(pid, window_id)` instead; it works while backgrounded.
-  Desktop actions use
-  `target:{"kind":"desktop","display_id":"primary"}`. Don't reach for
-  `get_desktop_state` as a casual screenshot: it is the desktop capture
-  surface, not window inspection.
-- **`Calc display stuck at 0 after my clicks`?** Almost always
-  means UWP and you're on the PostMessage path. UWP processes
-  pointer input via `Windows.UI.Input`, NOT through HWND message
-  queues: PostMessage(WM_LBUTTONDOWN) gets ignored. Use
-  `element_token` instead of (x,y) for UWP targets.
-
-Only after those are ruled out should you fall through to the
-activate fallback. Always name the focus steal in your response
-("I'll briefly bring Edge to the front because …").
-
-### Self-check pattern
-
-Before every `Bash` call whose command line touches any Windows app
-(launching, opening, clicking, typing, scripting, screenshotting),
-run the self-check:
-
-1. **Does this command foreground the target?** If yes, stop and
-   translate to the cua-driver equivalent from the mapping table.
-2. **Does this command move the user's real cursor?** (`SendInput`,
-   `SetCursorPos` from inline C#, AutoHotkey scripts, `nircmd
-sendmouse`.) If yes, stop; use `click({pid, x, y})` which routes
-   per-HWND via PostMessage / per-element via UIA Invoke and never
-   warps the cursor.
-3. **Does this command bypass cua-driver entirely?** (PowerShell
-   GUI scripts, AutoHotkey, `SendKeys`, AppleScript-equivalent
-   automation tools.) If yes, stop; find the cua-driver tool that
-   does the intent.
-
-If all three are "no," the command is safe. If you can't answer,
-default to stop and ask rather than proceed. A single `Start-Process`
-run by accident steals the user's foreground and kills the trust
-your prior tool calls earned.
-
-## Prerequisites: check before starting
-
-1. **`cua-driver` is on `$PATH`**: `Get-Command cua-driver` or
-   `where.exe cua-driver`. Install location:
-   `%LOCALAPPDATA%\Programs\trycua\cua-driver-rs\bin\cua-driver.exe`,
-   added to the user PATH by the install script.
-   If missing, point the user at:
-   ```powershell
-   irm https://cua.ai/driver/install.ps1 | iex
-   ```
-   and stop.
-2. **The runtime owner must run in an interactive session (Session 1+),
-   NOT Session 0.** This is the daemon for one-shot CLI/service mode and the
-   MCP process for bare stdio MCP. Windows isolates services into Session 0 with no
-   desktop. UIA enumeration, screenshot via PrintWindow, and
-   `IApplicationActivationManager` all silently return empty /
-   timeout in Session 0. Check:
-   ```powershell
-   Get-Process cua-driver | Select Id,SessionId
-   ```
-   `SessionId == 0` is refused before runtime actions. The autostart Scheduled Task uses
-   `LogonType=Interactive` so the daemon lands in the user's logon
-   session. If you started the daemon via SSH-into-Windows, that
-   session is usually Session 0: kick the autostart task instead:
-   ```powershell
-   schtasks /Run /TN cua-driver-serve
-   ```
-3. **Run `cua-driver doctor`**: reports session ID, COM apartment
-   status, UIA desktop-enumeration reachability, install paths,
-   version. If anything reads `false` / `error`, fix that before
-   tool-calling.
-4. **Permissions**: Windows has no TCC equivalent. cua-driver-rs
-   needs:
-   - No admin elevation for normal use (UIA, PostMessage, UWP
-     activation all work from a standard user token).
-   - UAC elevation **only** for `autostart` registration if a
-     system-wide scheduled task is requested (per-user task is
-     non-elevated and the default).
-   - SmartScreen: on a fresh install, Windows Defender SmartScreen
-     may flag the unsigned binary on first run. Click "More info →
-     Run anyway" once.
-
-## Using cua-driver from the shell
-
-Tool names are `snake_case`, management subcommands are
-`kebab-case`: no ambiguity. Tools invoked as `cua-driver
-<tool-name>` with JSON via stdin or positional arg. Management
-subcommands:
-
-- **`cua-driver serve`**: start the persistent daemon used by one-shot CLI
-  calls or by MCP clients that explicitly select it with `--socket`. Bare
-  `cua-driver mcp` owns its runtime directly on Windows.
-  Normally not run manually: the autostart Scheduled Task fires it
-  at every interactive logon. If you stopped it (`Stop-Process`),
-  re-run with `schtasks /Run /TN cua-driver-serve`, not by spawning
-  `cua-driver serve` from SSH (Session 0 problem).
-- **`cua-driver stop`** / **`status`**: daemon lifecycle.
-- **`cua-driver doctor`**: full diagnostics.
-- **`cua-driver list-tools`** / **`describe <tool>`**: tool surface
-  discovery.
-- **`cua-driver autostart {enable|disable|status|kick}`**: manage the
-  Scheduled Task that auto-starts the daemon at logon. `enable`
-  registers it (idempotent: replaces existing). `kick` runs it
-  immediately without waiting for a fresh logon.
-- **`cua-driver recording start|stop|status`**: see `RECORDING.md`.
-  Windows video uses ffmpeg with `gdigrab`; trajectory evidence continues
-  without video when ffmpeg is unavailable.
-
-Over SSH, never use bare `cua-driver mcp`: the direct runtime rejects Session 0. Start the daemon in the interactive user session and run `cua-driver mcp
---socket \\.\pipe\cua-driver` from SSH.
-
-For an ordered action loop, use the named CLI or persistent MCP examples in
-[WORKFLOW.md](WORKFLOW.md), with an observed AUMID, PID, HWND, and fresh token.
-Do not assume the window array is populated immediately on a cold launch.
-
-## The core invariant: snapshot before AND after every action
-
-**Every action MUST be bracketed by `get_window_state(pid, window_id)`**:
-
-- **Before**: the pre-action snapshot resolves the `element_token`
-  you're about to use. Targets from previous turns are stale; the
-  server replaces the snapshot on every read, keyed on
-  `(pid, window_id)`, and lists the replaced ids in
-  `invalidated_snapshot_ids`. Tokens from turn N don't resolve in turn
-  N+1, and targets from window A don't resolve against window B of
-  the same app. Skip this and element actions fail with
-  `stale_element_token`, which names the current snapshots.
-- **After**: the post-action snapshot verifies the action actually
-  landed. Without it you can't tell a silent no-op from a real
-  effect. The UIA tree change (new value, new window, disappeared
-  menu, disabled button, etc.) is your evidence that the action
-  registered. **Especially important on Windows** because the
-  layered click path can return `effect:"unverifiable"` after
-  PostMessage even when the click did nothing (UWP silently no-ops):
-  the action result reports the route, not the task outcome. Only
-  the re-snapshot tells you if the state changed.
-
-## Click semantics on Windows
-
-Two click addressing modes, both gated by `pid`:
-
-### Snapshot-bound element mode (preferred)
-
-```json
-{ "pid": 6004, "element_token": "s0000002a:22" }
-```
-
-Looks up the exact cached UIA element from the named snapshot,
-fires `IUIAutomationInvokePattern::Invoke()` on it directly.
-
-Properties:
-
-- **No mouse cursor moves.** The click is a UIA RPC, not an input
-  event. The user's cursor stays where it is.
-- **No window activates.** UIA Invoke does not foreground the
-  target.
-- **Z-order is irrelevant.** Works on backgrounded, occluded,
-  minimized-but-not-iconic, and cross-virtual-desktop windows.
-- **Cross-process boundaries work.** UIA marshals across the
-  `ApplicationFrameHost.exe` → inner UWP process boundary
-  (CalculatorApp.exe, Notepad-Win11.exe, etc.): this is how it can
-  click Win11 Calc buttons even though they live in a different
-  process from the AppFrame HWND.
-- **Falls back to `PostMessage(WM_LBUTTONDOWN/UP)` to the deepest
-  child HWND** when the cached element doesn't expose
-  `InvokePattern` (most edit fields, custom-drawn widgets,
-  non-actionable elements). The fallback works for plain Win32 but
-  silently no-ops on UWP. Read the closed action `route`:
-  `accessibility` means UIA/MSAA and `synthetic_events` means the
-  targeted event fallback. Do not parse the human-readable text.
-
-This is the right path for **any** "click button N" or "click checkbox Y"
-intent. For a known application-menu hierarchy, prefer `invoke_menu`:
-
-```json
-{ "pid": 6004, "window_id": 459672, "path": ["Window", "Arrange", "Left"] }
-```
-
-It uses `ExpandCollapsePattern` at intermediate hops and `InvokePattern` or
-`SelectionItemPattern` at the leaf, resolving the live UIA hierarchy again
-after every expansion. It refuses ambiguous, missing, or disabled segments and
-never falls back to pixels. Verify the command's semantic effect afterward.
-
-### `(x, y)` mode (element px action / pixel)
-
-```json
-{ "pid": 6004, "window_id": 459672, "x": 446, "y": 671 }
-```
-
-Window-client coordinates (origin at the top-left of the screenshot
-the agent saw). The driver:
-
-1. Converts to screen coords via `ClientToScreen(hwnd, ...)`.
-2. **UIA hit-test in target HWND's subtree**: `ElementFromHandle`
-   resolves the root, `FindAll(TreeScope_Subtree)` enumerates
-   descendants, picks the smallest-area `InvokePattern`-bearing
-   element whose `CurrentBoundingRectangle` contains the screen
-   point, calls `Invoke()`. This is the only path that lands on UWP
-   / WebView2 / DirectComposition surfaces.
-3. **PostMessage fallback**: if step 2 returned false (no Invokable
-   element under the pixel inside `hwnd`, or `hwnd` has no useful
-   UIA tree at all), fires `PostMessage(WM_LBUTTONDOWN/UP)` against
-   the deepest child HWND at the screen point. Covers plain Win32
-   native controls.
-
-Properties:
-
-- **No real cursor movement.** The agent overlay glides + pulses
-  for visual confirmation; the OS cursor is untouched.
-- **No focus steal.** Both UIA Invoke and PostMessage are async per-
-  pid / per-element; the target's window does not activate.
-- **Z-order independent.** UIA hit-test honors the HWND-rooted
-  subtree regardless of what's covering it on screen. PostMessage
-  goes directly to the message queue.
-
-Use this when the agent doesn't have a UIA snapshot in scope (zero-
-shot from a screenshot), or when the element it wants doesn't appear
-in the UIA tree (custom-drawn elements, canvas content, browser
-DOM nodes inside a WebView2 viewport).
-
-### What `(x, y)` mode does NOT solve
-
-Apps with **no useful UIA tree** AND that **ignore `WM_LBUTTONDOWN`**
-on the HWND queue: primarily DirectX / OpenGL / Vulkan-rendered
-surfaces (games, custom renderers). The click chain falls all the
-way through and the click no-ops. For those, the only options are:
-
-- Bring the window to top first (focus steal: ask the user before
-  doing this, and document why), then synthesize input
-- Use the app's keyboard interface via `hotkey` if available
-
-`SendInput` is **not** a silent-fallback option here: it would
-steal focus from whatever the user is doing.
-
-### Right-click and multi-click
-
-`button: "right"` and `count > 1` **skip the UIA Invoke step** and
-go directly through the PostMessage path. Reason: UIA has no clean
-by-coord equivalent of `ShowContextMenu`, and `Invoke()` is single-
-fire by definition. The action result reports `route:"synthetic_events"`
-regardless of the target's UWP-ness: this is expected and
-**will not work for UWP context menus**. To open a UWP context menu,
-prefer `hotkey({pid, keys: ["shift", "f10"]})` against the focused
-UWP element.
-
-## UWP / packaged apps: the AUMID layer
-
-Modern Win11 apps (Calculator, Notepad, Settings, Photos, Edge UI
-chrome, Microsoft Store) are **packaged apps** with an `App User
-Model ID` (AUMID) rather than a plain `.exe` path. The AUMID looks
-like `Microsoft.WindowsCalculator_8wekyb3d8bbwe!App`: package family
-name + `!` + AppId.
-
-Windows architectural quirks that matter:
-
-1. **The `.exe` in `C:\Windows\System32\notepad.exe` is a 7 KB stub
-   that exits immediately on Win11.** It exists for backward
-   compatibility but the real Notepad lives in the
-   `Microsoft.WindowsNotepad` AppX package. `Start-Process notepad`
-   spawns the stub, which exits, which redirects through the AppX
-   broker, which spawns the real process with a different pid. You
-   end up holding a pid that's already gone. `launch_app` handles
-   this transparently: it detects AUMID-looking strings and routes
-   through `IApplicationActivationManager::ActivateApplication`,
-   which returns the **real** packaged-process pid.
-2. **UWP windows are hosted by `ApplicationFrameHost.exe`.** The
-   outer top-level HWND (the one with the title bar, the one
-   `EnumWindows` enumerates) is owned by `ApplicationFrameHost.exe`,
-   pid varies, not the same as the UWP's own pid. The actual UWP
-   content runs in a separate process (e.g. `CalculatorApp.exe`).
-   `list_windows` reports the AppFrame's HWND because that's what
-   `GetWindowRect`, `PostMessage`, `BitBlt` all target. UIA
-   transparently crosses the boundary into the inner process when
-   you walk the tree.
-3. **AUMID resolution by name**: `launch_app({name: "calc"})` will
-   first try a `shell:AppsFolder` lookup, matching against
-   installed-package display names. On a hit it goes through the
-   packaged-app path (real pid). Otherwise it falls back to
-   `ShellExecuteEx`'s PATH search (which hits the stub `.exe`).
-   **Prefer explicit `aumid` when you know it.**
-
-### Known AUMIDs for common Win11 apps
-
-```
-Microsoft.WindowsCalculator_8wekyb3d8bbwe!App
-Microsoft.WindowsNotepad_8wekyb3d8bbwe!App
-Microsoft.MicrosoftEdge.Stable_8wekyb3d8bbwe!App
-Microsoft.WindowsTerminal_8wekyb3d8bbwe!App
-Microsoft.Paint_8wekyb3d8bbwe!App
-Microsoft.WindowsCamera_8wekyb3d8bbwe!App
-Microsoft.WindowsAlarms_8wekyb3d8bbwe!App
-Microsoft.MicrosoftStickyNotes_8wekyb3d8bbwe!App
-windows.immersivecontrolpanel_cw5n1h2txyewy!Microsoft.Windows.ImmersiveControlPanel  # Settings
-```
-
-To find an AUMID at runtime:
-
-```powershell
-Get-StartApps | Where-Object Name -like "*Calculator*"
-```
-
-## Browsers and embedded webviews on Windows
-
-Use the typed, exact-binding workflow in `BROWSER.md` for Chrome and Edge page
-content. Windows has validated trusted background browser clicks when the
-driver runs in an interactive user desktop. A daemon in Session 0 cannot
-provide representative UIA, capture, focus, or browser evidence.
-
-Ref- and coordinate-targeted browser mutations also drive the declared
-session's agent cursor. The adapter converts the live CDP page point into the
-bound window's DPI-aware screen coordinates, pins the overlay above that
-window, and keeps only the selected tab's session cursor visible. This visual
-feedback never moves the physical pointer or changes browser input delivery.
-
-Keep browser chrome and native dialogs on the normal UIA/PX ladder in this
-file. This includes tabs, the address bar, menus, permission prompts,
-downloads, file pickers, and authentication windows. Avoid `Ctrl+L`, tab
-switching shortcuts, `Start-Process`, and shell activation paths when the user
-expects background operation.
-
-WebView2 inside a non-browser host is not automatically equivalent to a
-standalone Edge target. Use typed browser mutation only when
-`get_browser_state` returns an exact binding and `mutation_allowed:true` for
-the selected `(pid, window_id)`. Otherwise use the native UIA/PX route or
-accept the structured refusal. Firefox page mutation is not supported by the
-typed browser tools yet.
-
-## Common failure modes (Windows-specific)
-
-- **`Session 0` daemon**: `cua-driver doctor` reports
-  `SessionId: 0`. UIA enumeration returns empty, screenshot
-  returns blank. Fix: stop the daemon, kick the autostart task with
-  `schtasks /Run /TN cua-driver-serve`.
-- **Stale HWND** (`Invalid window handle 0x80070578`): the window
-  was closed, re-created (e.g. UWP shutdown-on-idle), or moved to
-  a different desktop session. Re-resolve via `list_windows`.
-- **Calc display stuck at "0" after pixel clicks**: the (x,y) UIA
-  hit-test missed and PostMessage fell through (PostMessage is a
-  silent no-op on UWP). Switch to `element_token` mode. Symptom:
-  the action result reports `route:"synthetic_events"` instead of
-  `route:"accessibility"`.
-- **LibreOffice (VCL) `type_text` / `hotkey` reported success but
-  nothing happened**: VCL/SAL apps route accelerators through
-  `TranslateAccelerator` (reads `GetKeyState`, which PostMessage doesn't
-  update) and the Calc/Writer document grid only takes real keystrokes
-  when a cell is in edit mode, so background `WM_CHAR` / key-combos are
-  silently dropped. Two honesty mechanisms now cover this instead of a
-  blind success:
-  - **`hotkey` / `press_key`** (keystroke + key-combo): `delivery_mode:"background"`
-    surfaces a `background_unavailable` error for VCL.
-  - **`type_text`** does a **UIA read-back** and returns the shared
-    `ActionResult`. With an `element_token`, the ValuePattern path returns
-    `effect:"confirmed"` with `evidence:[{"kind":"value_readback"}]` only
-    when the complete expected value is synchronously visible and differs from
-    the prior value. If SetValue succeeded but the provider still exposes the
-    old or no value, the result is `effect:"unverifiable"` with no escalation.
-    This is common for deferred providers such as AccessKit: take a fresh
-    snapshot before retrying because the value may publish only after
-    `type_text` returns and an immediate retry can duplicate text. A pixel
-    escalation is reserved for an Electron/web accessibility echo that does
-    not prove the renderer observed the write. The PostMessage fallback keeps
-    its separate delivery/read-back behavior. Even when the value changes and
-    contains the requested text, the result stays `effect:"unverifiable"` because
-    WM_CHAR does not expose the insertion point. Take a fresh snapshot before
-    retrying. It may recommend foreground when a background insert appears
-    dropped. Passing an `element_token` makes the
-    read-back target that exact element by handle (ValuePattern → TextPattern),
-    independent of foreground focus. Without one, PostMessage verification
-    falls back to system-wide `GetFocusedElement`, which normally resolves only
-    for the foreground app; an unreadable result is therefore not proof of
-    failure.
-    Escalate to `delivery_mode:"foreground"` for both (SendInput Unicode /
-    accelerator). **But** foreground needs the swap to actually land: if the
-    daemon lacks UIAccess and `bring_to_front` returns `landed_on_target:false`
-    (or it reverts before the next call), you can't drive it by input at all:
-    produce the artifact and `launch_app` it (build the `.xlsx` / `.docx` and
-    open it) rather than typing into the GUI.
-- **Edge / Chrome shows tab switching even though I used pid-scoped
-  hotkey**: `Ctrl+Tab` / `Ctrl+1..9` aren't pid-scopable; the
-  receiver activates. Use the windows-per-URL pattern.
-- **`Get-StartApps` returns no AUMID for an app I see in Start
-  Menu**: the app might be a Win32 desktop app, not a packaged
-  app. Use `{path: "..."}` instead of `{aumid}`. (Win11 Calculator
-  IS packaged; Win10 classic Notepad was not.)
-- **`launch_app` returns pid N but `list_windows({pid: N})` returns
-  empty**: UWP cold-launch race: the AppFrame HWND hasn't
-  materialized yet. Re-call `list_windows({pid: N})` after 500ms;
-  for chronic cases, key off the app name in `list_windows({})`
-  output.
-- **Need a screenshot file**: use `get_window_state` with
-  `screenshot_out_file`; the result is PNG. The removed standalone screenshot
-  compatibility inputs (`format` and `quality`) are rejected by the live tool
-  schema. Downsample or convert the saved PNG outside cua-driver if needed.
-
-## Diagnostics
-
-`cua-driver doctor` reports:
-
-- Daemon version and install paths
-- Current session ID (must be ≥1)
-- COM apartment status (STA / MTA / uninitialized)
-- UIA reachability (can we create `CUIAutomation` and enumerate desktop children?)
-- AppX broker reachability (for packaged-app activation)
-- PATH state (is `cua-driver` actually on PATH?)
-- Autostart Scheduled Task status
-
-Run it whenever a tool call returns unexpectedly. Most failures
-trace back to one of these checks reading "false."
-
-`cua-driver autostart status` reports whether the daemon is
-registered to auto-start at logon AND whether it's currently running:
-
-- `not-registered`: Task Scheduler explicitly reported that the named task
-  does not exist. Re-register via `cua-driver autostart enable`.
-- `registered (not running)`: autostart task exists but no daemon
-  process. Kick it with `cua-driver autostart kick`.
-- `registered (running)`: happy path.
-- `permission-denied`: the current process cannot inspect Task Scheduler;
-  registration is unknown. Re-run the status check from a context that can
-  read the task rather than re-registering it blindly.
-- `unknown`: the query failed for another reason. The command exits non-zero
-  and includes the original `schtasks` diagnostic; do not treat this as an
-  absent registration.
-
-## Recording
-
-Windows recording uses ffmpeg with `gdigrab` and writes the same trajectory
-shape described in `RECORDING.md`. Install ffmpeg on `PATH` for MP4 capture.
-When it is unavailable, per-turn state and screenshots continue and
-`last_error` reports the missing dependency.