@bastani/atomic 0.9.39 → 0.9.40-alpha.2

package/dist/builtin/subagents/skills/cua-driver/MACOS.md

@@ -83,8 +83,20 @@ frontmost is true'`). Mutating it is not.
 **Corollary: the AXMenuBar rule.** Do not manually drive a background
 application's `AXMenuBarItem`: the visible macOS menu bar belongs to the
 frontmost app, and command items may be disabled otherwise. Use `invoke_menu`
-for a known application-menu path. It owns the necessary temporary activation,
-resolves every AX level live, and restores the prior app on a best-effort basis. Prefer an in-window
+for a known application-menu path. With the app in the background it resolves
+the whole path without opening a menu and never activates the app; if macOS
+keeps the item unavailable to a background app it refuses with
+`background_unavailable` (then, if the user allows it, `bring_to_front` first).
+One exception: a disabled Save (the ⌘S item) of a document window is saved
+through the app's scripting interface instead, still in the background, and the
+result says so; TextEdit disables Save whenever it is behind the user's window.
+A Save is verified on disk. A background `hotkey` whose combo is a menu item's
+key equivalent (⌘S, ⌘W, ⌘C, ...) in a native app takes the same route, because
+background key events never reach an app's menus; no keys are sent. macOS asks
+the user once to allow Cua Driver to control the app (Automation) the first
+time a scripting route runs; when that is off, the call refuses and nothing is
+saved.
+With the app already in front it resolves every AX level live. Prefer an in-window
 element action when the same command has an ordinary control, and prefer
 `set_window_frame` for exact geometry. Full rationale is in “Navigating native
 menu bars” below.
@@ -109,10 +121,13 @@ is safe even for apps that normally foreground on media-load
 | Enumerate an app's windows            | `list_windows({pid})`: or read the `windows` array `launch_app` already returns       | `osascript 'every window of app …'`                         |
 | Move or resize one exact window       | `set_window_frame({pid, window_id, x, y, width, height})`                              | `osascript` position/size writes or title-bar dragging      |
 | Click / type / scroll / keys          | `click`, `type_text`, `scroll`, `press_key`, `hotkey`                                  | `osascript`, `cliclick`, raw `CGEvent`, `open <url>`        |
-| Drag / drag-and-drop / marquee select | `drag({pid, window_id, from_x, from_y, to_x, to_y, delivery_mode:"foreground"})` (pixel-only and foreground-only: macOS AX has no semantic drag, and there is no background drag) | `cliclick dd:`, `osascript drag`                            |
+| Loop, branch or retry over app actions | `run_script({script})` (sandboxed JavaScript over the `cua` API, one call); `run_steps` for a fixed list | an AppleScript or shell loop around `osascript` or `cua-driver` calls |
+| Drag / drag-and-drop / marquee select | `drag({pid, window_id, from_x, from_y, to_x, to_y})` in the background (pixel-only: macOS AX has no semantic drag). Verify with a screenshot; `delivery_mode:"foreground"` only if nothing changed. A bound browser tab: `browser_pointer` `action:"drag"` | `cliclick dd:`, `osascript drag`, refusing a drag-only step as "needs the window in front" |
+| Hover (reveal a hover-only control)   | Window-targeted `move_cursor`; its result says whether it hovered (AX alternate UI or an Electron page). A web page in Chrome, Edge or Brave: `browser_pointer` hover after `get_browser_state`. If not, look for the same command elsewhere (menu item, context menu, shortcut, help text in `get_window_state`); only `move_cursor({scope:"desktop", delivery_mode:"foreground"})` (moves the user's pointer) or `bring_to_front` reveals it, and only if the user allows it | assuming a window-targeted `move_cursor` hovered when its result says the app sees no hover; a desktop `move_cursor` over a background browser page |
 | Screenshot                            | `get_window_state` (window) or authorized `get_desktop_state` (desktop)                | `screencapture`                                             |
 | Quit an app                           | ask the user first, then `hotkey({pid, keys:["cmd","q"]})`                             | `kill`, `killall`, `pkill`                                  |
 | Hand a file/URL to an app             | `launch_app({bundle_id, urls:[<path>]})`                                               | `open -a <App> <path>`, `open <url>`                        |
+| View a local file in the GUI          | `launch_app({bundle_id, urls:["file:///…"]})`                                          | typing `file://` into the omnibox                           |
 
 ### The narrow carve-out
 
@@ -122,6 +137,11 @@ For authorized foreground input, use the Cua action's
 
 When a cua-driver call surprises you, diagnose cua-driver first:
 
+- **Large tree or repeated reads?** `get_window_state` returns one compact
+  markdown tree capped at 250 nodes; pass `since:"latest"` for a diff
+  (`no change since …; focused element is …` names the focused row on macOS)
+  and `tree_format:"elements"` for structured rows. `full_output:true` restores
+  the old payload (both forms, `_note`, `background_input`, 2 000 nodes).
 - **Empty `tree_markdown`?** `get_window_state` returns **both** the
   AX tree and a screenshot by default: there's nothing to configure and
   no capture mode to pick. An empty tree means the surface isn't AX (a
@@ -158,8 +178,10 @@ response, and only switch _dispatch rung_ on a real signal:
 1. **Re-snapshot and read the tree diff**: a changed `AXValue`, a new
    element, a collapsed menu, a disabled button. If the tree shows the
    change, you're done. When you only need the tree diff and don't need
-   fresh pixels, pass `include_screenshot:false` to skip the grab: a
-   **perf** knob, not a mode flip.
+   fresh pixels, pass `since:"latest"` and `include_screenshot:false` to
+   skip the grab: a **perf** knob, not a mode flip. The window keeps its
+   last screenshot's pixel frame while its size is unchanged, so `x,y`
+   from that screenshot still work afterwards.
 2. **Trust the screenshot and do an element px action** when the tree
    **lies**: the action response carried `effect:"suspected_noop"`, the
    re-snapshot came back `degraded` (empty tree), or the tree looks
@@ -279,14 +301,75 @@ permission prompt the agent loop shouldn't be burning on app launch.
 
 The pixel click is routed through SkyLight's per-pid event path
 (`SLEventPostToPid`), not the system HID stream. The dispatch recipe
-is the backgrounded "noraise" sequence: focus-without-raise
-SLPS event records followed by an off-screen user-activation primer
-and the real click. The target app becomes AppKit-active for event
-routing but its window does **not** rise to the front of the
-z-stack, and macOS's "switch to Space with windows for app" follow
-is suppressed. Full mechanics in
-`Sources/CuaDriverCore/Input/MouseInput.swift` (`clickViaAuthSignedPost`)
-and the companion `FocusWithoutRaise.swift`.
+is the backgrounded "noraise" sequence: a focus-without-raise
+SLPS event record to the target window, followed by an off-screen
+user-activation primer and the real click. The target window becomes
+AppKit-key for event routing, but it does **not** rise to the front of
+the z-stack, macOS's "switch to Space with windows for app" follow is
+suppressed, and the user's front window keeps key status, so their
+typing keeps landing there.
+
+### Menus opened in the background keep the user's keyboard
+
+A popup button, a `<select>`, or a context menu opened in a background
+app takes key focus from the user's front window for as long as the menu
+is open (AppKit's menu tracking steals it). While a background action
+runs, Cua Driver watches for that and hands key focus straight back, so
+the user's window never loses it. The menu stays open and its items are
+chosen with `click` on the `AXMenuItem` (or in one call with
+`set_value` on the popup). Keyboard navigation of such a menu with
+`press_key` is not the route; pick the item by element.
+
+### Apps that activate themselves
+
+Some apps activate themselves in answer to input, even in the background:
+Chrome when `cmd+L` focuses its address bar, GnuCash on a double-click in
+its account tree. No client can stop that. After a background
+action Cua Driver checks WindowServer's front process and, if the target
+made itself frontmost, re-activates the user's app and hands its window
+key focus back within a few tens of milliseconds. Prefer routes that do
+not ask the app to come forward: `set_value` on the address field or
+`browser_navigate` instead of `cmd+L`, and `AXOpen` through
+`double_click` on an element instead of a pixel double-click.
+### Hover in the background
+
+macOS apps track hover and show tooltips (help tags) from the real
+pointer. Pointer-move and enter events posted to a background app reach
+the app but never its tracking areas, so most hover-only controls and all
+native tooltips cannot be revealed in the background. A window-targeted
+`move_cursor` on a background app tries the two routes that leave the
+real pointer, focus and z-order alone. Its result says whether it hovered
+and how:
+
+- `ax_alternate_ui`: the element under the point (or an ancestor) lists
+  `showalternateui`, as sidebar section headers in Finder, Mail, Music and
+  System Settings do. The hover-only controls appear in the next
+  `get_window_state`; hovering elsewhere hides them. The same element can
+  be driven directly with `click` and `action:"show_alternate_ui"` /
+  `"show_default_ui"`.
+- `electron_input`: an Electron page gets `mouseEnter`/`mouseMove` through
+  the app's main process, so page hover effects update. Native tooltips do
+  not show.
+
+For a web page in Chrome, Edge or Brave use `browser_pointer` hover: a
+window-target `move_cursor` there says the page saw no hover and names
+`get_browser_state` as the next call. Otherwise `move_cursor` moves only the
+agent cursor and says so. Look for the same command elsewhere first: a menu
+item, a context menu, a keyboard shortcut, or the control's help text in
+`get_window_state` (toolbar tooltips are usually there as `help=`).
+Otherwise the step needs the user's pointer (`move_cursor` with
+`scope:"desktop"` and `delivery_mode:"foreground"`) or the app in front
+(`bring_to_front`); use either only if the user allows it, and say which.
+
+### Desktop-target pointer input over a background app
+
+`move_cursor`, `click`, `drag` and `scroll` with a desktop target move the
+user's real pointer. When the point lands on a window of an app that is not
+in front, they refuse with `background_unavailable` before sending anything,
+unless the call passes `delivery_mode:"foreground"`. Over a web browser the
+refusal names the browser tools (`get_browser_state`, then
+`browser_pointer`, `browser_click` or `browser_type`) instead of foreground.
+Points over the app in front, or over no window, behave as before.
 
 ### `delivery_mode` on the pointer family (macOS)
 
@@ -300,12 +383,31 @@ surface that only accepts events while frontmost (the canvas/viewport/game
 case below). Unmodified `element_token` (AX) actions remain background-capable
 and hold the no-foreground contract without the flag.
 
-`drag` is the exception: macOS has no background drag. A window-scoped
-`drag` needs `delivery_mode:"foreground"` and `window_id`; without them it
-refuses with `background_unavailable` and sends nothing. The foreground drag
-fronts the exact window, moves the physical pointer along the path, and then
-restores the prior frontmost app. Use it only when a drag is the task, and
-expect the hardware pointer to move.
+`drag` runs in the background too. A window-scoped `drag` (`pid` and
+`window_id`) makes the window key without raising it, posts the press, the
+interpolated drags and the release to that window's process, and hands focus
+back to the user's window. The real pointer does not move. It reaches
+controls that follow the drag events: native sliders and canvases, and
+pointer-event sliders, canvases and boards on web pages in Chrome. The
+result is not driver-verified: check the outcome with a screenshot or the
+tree. Only if nothing changed, retry with `delivery_mode:"foreground"`,
+which fronts the window, moves the physical pointer along the path, and
+restores the prior frontmost app.
+
+A native `<input type="range">` in Chrome does not follow a background drag
+(pointer-event widgets do). Use `set_value` on its slider element, or
+`browser_pointer` `action:"drag"` on a bound tab, which sends the drag
+through CDP `Input.dispatchMouseEvent` with the button held.
+
+A drag that starts a macOS drag-and-drop session (reordering a list by drag
+and drop, moving a file, HTML5 `draggable` content) cannot drop in the
+background: macOS tracks those sessions with the real pointer. The driver
+sees the session start, releases at once, and returns
+`background_unavailable` with `reason: "drag_and_drop_session"`. Look for a
+non-drag route first (Move Up/Down, a menu or context-menu command, a
+shortcut, cut and paste), then foreground if the user allows it. Toolkits that read the hardware pointer (Tk) or drop routed
+mouse events (LibreOffice) refuse the background drag with
+`background_unavailable` before anything is sent.
 
 A foreground window-scoped **pixel** `click`, `double_click`, or
 `right_click` (`x`/`y` with `window_id`) is delivered like a desktop-scope click, not through the per-pid path: Cua Driver
@@ -390,6 +492,16 @@ There is no backgrounded path that reaches these apps today.
   AX-addressable elements (links, buttons, toolbar items), use
   `right_click({pid, element_token})` instead.
 
+### A value write the app does not apply (LibreOffice)
+
+Some apps accept an accessibility value write and ignore it. LibreOffice
+Calc cells do: `set_value` returns success at the AX level and the cell
+keeps its old content. `set_value` re-reads the value, and when it has
+not moved it fails with `code:"write_not_applied"` and
+`effect:"not_applied"`, so a `run_script` stops there. LibreOffice also
+ignores background pointer input, so address it with `window_id` and type
+instead (see "LibreOffice, Numbers and other editors in the background").
+
 ### Known text-input limits (Catalyst + Electron)
 
 On **Catalyst** apps (WhatsApp, Reminders, Notes-via-Catalyst,
@@ -455,6 +567,10 @@ and the refusal says whether any menu window it opened is still on screen. If
 the refusal says the menu may still be open, press `escape` on the window
 before other input.
 
+A `get_window_state` read lists only the menu bar's titles and says
+`menus collapsed`; pass `query` to find an item in any menu, or run it with
+`invoke_menu`. A missed path segment lists the items at that level.
+
 ```bash
 cua-driver invoke_menu \
   '{"pid":844,"window_id":10725,"path":["Window","Move & Resize","Left"]}'
@@ -486,11 +602,11 @@ window, and PID-routes its bounded checkbox-pixel fallback. The same control's
 state must verify after mutation; unsupported appearance, scale, zoom,
 window-size, or toolbar geometry refuses without a click.
 
-Standalone Chromium activates its window when CDP's trusted pointer route is
-used on macOS. The driver therefore returns
-`browser_input_trust_unavailable` before dispatch rather than falsely claiming
-background delivery. Use `input_route:"dom_event"` only when synthetic click
-semantics are acceptable. Embedded Electron has a separately bounded route;
+CDP's trusted pointer route (the `browser_click` and `browser_pointer`
+default) stays in the background on macOS Chrome: it delivers trusted events
+to the bound tab, even an inactive one, without activating or raising the
+browser window. Use `input_route:"dom_event"` only when synthetic events are
+acceptable; controls that check `event.isTrusted` ignore it. Embedded Electron has a separately bounded route;
 do not infer that route for arbitrary WKWebView or Tauri hosts.
 
 For a declared session, typed browser click, type, and pointer mutations animate
@@ -518,8 +634,8 @@ starting point for new browser workflows.
 | `Accessibility permission not granted`                        | TCC not granted                                                                                                   | Stop; tell user to grant in System Settings                                                                                                                                                                                         |
 | `Screen Recording permission not granted`                     | TCC not granted for capture                                                                                       | Screenshots and pixel actions are unavailable. If the task is AX-completable, use `get_window_state({include_screenshot:false})` and `element_token` actions; otherwise stop and ask the user to run `cua-driver permissions grant` |
 | `AX action failed: ... returned -25200` (or -25205, -25206)   | The app may have acted anyway: some AppKit controls return these after a press they performed                     | Re-read with `get_window_state` before retrying; a second press on a checkbox toggles it back. A radio button or checkbox whose AXValue settles on the pressed state is reported as performed, naming the error                     |
-| `set_value refused (file_name_needs_rename)`                  | A file's name as Finder lists it: an AXValue write changes only the list, never the file                          | `click` the item, then with Finder frontmost send `press_key` return, `hotkey` cmd+a, `type_text` the full new name, `press_key` return, each with `scope:"desktop"`; re-read to confirm                                            |
-| `set_value refused (file_name_needs_rename)`                  | A file's name in Finder's Get Info window (AXIdentifier `Name`): an AXValue write changes only the field          | foreground `click` at the field's centre in screenshot pixels (with its `capture_id`), then `hotkey` cmd+a, `type_text` the full new name with its extension, `press_key` return, each with `delivery_mode:"foreground"`; a changed extension asks for confirmation first; check the folder listing |
+| `set_value refused (file_name_needs_rename)`                  | A file's name in Finder's Get Info window (AXIdentifier `Name`): an AXValue write changes only the field          | `set_value` the full new name on the file's name cell in a Finder window that lists it (renamed through Finder in the background and read back from disk) |
+| `Background input refused (no_focused_text_input)`           | Untargeted background `type_text` while the window's focused element is not a text input or an editing surface (spreadsheet grid, document canvas: `AXLayoutArea`, `AXScrollArea`, `AXTable`, `AXCell`, `AXGroup`; not in Finder/Dock); keys would drive the view (Finder type-to-select) | pass the field's `element_token`, or `x,y` to pixel-focus it first; to rename a Finder file, `set_value` its name cell |
 
 ## Example end-to-end task (macOS)
 
@@ -537,4 +653,12 @@ starting point for new browser workflows.
 
 If the user instead asks to navigate _within_ an already-open Finder
 window, use the menu-bar flow from "Navigating native menu bars"
-above (click Go → pick a menu item → re-snapshot → click it).
+above (`invoke_menu` with a path such as `["Go", "Downloads"]`).
+
+### LibreOffice, Numbers and other editors in the background
+
+With `window_id`, `type_text`, `press_key` and `hotkey` post keys to that exact window: the driver makes it the app's key window through in-app key-window records (no raise, no activation, real pointer untouched), confirms that the app's focused window and focused element are in that window, posts, and hands key back. This also covers another window or dialog of the app holding focus, and a LibreOffice that was never activated. When the switch cannot be confirmed the call is refused with `window_not_key` and no key is sent. A second window of the same app (Find & Replace, another document) is no longer `same_pid_keyboard_ambiguity`. A command key whose menu route is refused (item disabled while the app is in the background) falls through to key events in that window. Recipe: pass `window_id` on every call; go to the cell with Name Box navigation (`type_text` `A1\n`), type with `\n` and `\t`, read the cell back; run commands (Paste Special, Format, Save) with `invoke_menu({pid, window_id, path})` rather than clicking the menu bar. A `background_unavailable` refusal here means use `get_window_state` and act by element, not stop.
+
+### Preview: export before resizing
+
+Duplicate is disabled in a background Preview, so work on an export: File > Export (`invoke_menu`) to the destination, then resize or edit the exported copy. Do not resize the original first.

package/dist/builtin/subagents/skills/cua-driver/RUNTIME.md

@@ -98,7 +98,7 @@ Pixel `click` already animates the cursor. Do not precede it with `move_cursor`
 
 The default theme provides session-colored pointer/glow, action marks, and activity animations. Named sessions have stable palette colors. Native hosts own delivery/target badges separately from theme artwork; cursor state is scoped to the private lifecycle.
 
-`set_agent_cursor_enabled` controls idle visibility. `set_agent_cursor_motion` takes `style` (`signature_arc` default, `spring_settle`, `magnetic`, `comet_swoop`, `adaptive`, `classic`), `timing` (`native`, `fitts`, `fixed`), `effects` (`trail`, `glow`, `magnet`, `ripple`, `squish` booleans; null restores the style default), and the arc knobs `start_handle`, `end_handle`, `arc_size`, `arc_flow` (bezier styles) plus `spring` and `turn_radius` (`classic`) where advertised. Choose preinstalled artwork with `set_agent_cursor_theme`; `cursor_id` on input tools is not a theme selector. Never supply inline theme code or arbitrary source paths through agent tools. The trusted local `cursor-theme` workflow owns validation, compilation, preview, installation, and removal.
+`set_agent_cursor_enabled` controls idle visibility. `set_agent_cursor_motion` takes `style` (`signature_arc` default, `spring_settle`, `magnetic`, `comet_swoop`, `adaptive`, `classic`), `timing` (`native`, `fitts`, `fixed`), `effects` (`trail`, `glow`, `magnet`, `ripple`, `squish` as `on`, `off`, or `default`; `default` restores the style default, and an omitted effect keeps its current setting), and the arc knobs `start_handle`, `end_handle`, `arc_size`, `arc_flow` (bezier styles) plus `spring` and `turn_radius` (`classic`) where advertised. Choose preinstalled artwork with `set_agent_cursor_theme`; `cursor_id` on input tools is not a theme selector. Never supply inline theme code or arbitrary source paths through agent tools. The trusted local `cursor-theme` workflow owns validation, compilation, preview, installation, and removal.
 
 Overlay operations need a suitable UI event loop. macOS direct SDK/MCP runtimes without a certified host main-thread adapter can return `facility_unavailable`; do not report that as a successful move. See [EMBEDDING.md](EMBEDDING.md).
 

package/dist/builtin/subagents/skills/cua-driver/SKILL.md

@@ -4,16 +4,26 @@ description: Drive a native GUI app (macOS, Windows, Linux) via the cua-driver C
 hidden: true
 metadata:
     github-path: libs/cua-driver/rust/Skills/cua-driver
-    github-ref: refs/tags/cua-driver-rs-v0.34.0
+    github-ref: refs/tags/cua-driver-rs-v0.36.0
     github-repo: https://github.com/trycua/cua
-    github-tree-sha: bc6d2b59a453e10321e30e58203294fee9d4bc28
+    github-tree-sha: 229cf71d61810fb3fe25771705478985640d93a0
 name: cua-driver
-version: 0.34.0
+version: 0.36.0
 ---
 
 # cua-driver
 
-Operate one exact target, observe its state, act once, and verify the user's postcondition.
+Operate one exact target, observe it narrowly, act by element, and verify the user's postcondition at checkpoints.
+
+## Efficient loop
+
+The MCP server instructions carry the same rules, so they apply even when this file is not opened.
+
+1. Start with a targeted read: `get_window_state({pid, window_id, query:"Save"})` returns about 2K chars where a full snapshot is about 42K. Bound large trees with `max_elements` / `max_depth`. Widen only if the target is missing.
+2. Act by `element_token`: row `[N]` of a read is `<snapshot_id>:N` (for example `"s0000002a:11"`, not `"11"`). Use pixels only for surfaces missing from the tree.
+3. Act through `run_steps` with `observe:{}`, even for a single step. It runs the steps, stops at the first failure, and returns only what changed since your last read, so you do not need a separate read after acting; see [Workflow](WORKFLOW.md#batch-known-actions). When the flow needs logic between actions (a loop, a branch on what the app shows, a retry), write it as one `run_script` instead; see [Workflow](WORKFLOW.md#script-a-flow).
+4. To re-read a window, pass `since:"latest"`: you get the changed rows, not the whole tree. To just look, use `include_accessibility_tree:false` where advertised (it keeps the current tokens valid). Do not use `max_elements:1` or a junk `query` to get a picture.
+5. Verify at checkpoints (after a meaningful state change, before finishing), not after every action. One `verify_state` or one targeted `get_window_state` is usually enough.
 
 ## Act
 
@@ -21,12 +31,16 @@ Operate one exact target, observe its state, act once, and verify the user's pos
 | ----------------------------------------- | ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
 | Check installation and capabilities       | `cua-driver --version`, `status`, `doctor`, `describe <tool>`; MCP `tools/list`                       | [Runtime](RUNTIME.md)                                 |
 | Find or open the requested app            | `list_apps`, `list_windows`, `launch_app`                                                             | Current platform guide below                          |
-| Observe one window                        | `get_window_state({pid, window_id})`                                                                  | [Workflow](WORKFLOW.md)                               |
+| View a local file in the GUI              | `launch_app({bundle_id, urls:["file:///…"]})` opens it in that app                                    | Current platform guide below                          |
+| Observe one window                        | `get_window_state({pid, window_id, query})`                                                           | [Workflow](WORKFLOW.md)                               |
 | Act on a control                          | `click` / `type_text` with a fresh `element_token` and exact window target                            | [Workflow](WORKFLOW.md)                               |
+| Act and see what changed in one call      | `run_steps({steps:[{tool,args},...], observe:{}})`; stops at the first failure                        | [Workflow](WORKFLOW.md#batch-known-actions)           |
+| Loop, branch or retry in one call         | `run_script({script})`: JavaScript over the `cua` API, sandboxed, time and call limits                | [Workflow](WORKFLOW.md#script-a-flow)                 |
 | Use pixels when semantics cannot reach it | Fresh target screenshot, then `x,y` on the same target                                                | [Workflow](WORKFLOW.md)                               |
 | Verify the outcome                        | `verify_state({pid, window_id, expect})` or a fresh snapshot read by the agent                        | [Workflow](WORKFLOW.md)                               |
 | Operate the authorized desktop            | `get_desktop_state` → input with `target:{kind:"desktop",display_id:"primary"}` → `get_desktop_state` | [Workflow](WORKFLOW.md), [Linux](LINUX.md) on Wayland |
-| Drive supported browser page content      | `get_browser_state` → typed browser action → fresh state                                              | [Browser](BROWSER.md)                                 |
+| Drive supported browser page content      | `get_browser_state` → `browser_prepare` from `detail.next_call` if refused → typed browser action      | [Browser](BROWSER.md)                                 |
+| Browser tools refuse in the background   | Not blocked: fill the page through `get_window_state` + `set_value`/click; if the first read shows only browser chrome, read again. BLOCKED only after that fails | [Browser](BROWSER.md) |
 | Record an explicitly requested run        | `start_recording` → actions → `stop_recording`; verify artifacts                                      | [Recording](RECORDING.md)                             |
 | Finish                                    | Stop after proof; `end_session` for this run, not `cua-driver stop` on a shared service               | [Runtime](RUNTIME.md)                                 |
 
@@ -39,8 +53,8 @@ Check the installed version and advertised schema before using unfamiliar parame
 ## Rules
 
 1. Select the exact target on each action. A session is lifecycle metadata, not capture scope or permission authority.
-2. Observe before input and verify after it. `effect:"unverifiable"` and a successful exit are not task success; never replay a partial, canceled, or unknown action blindly.
-3. Use returned tokens, never invented indices. A fresh snapshot replaces prior element handles and lists them in `invalidated_snapshot_ids`; act with `element_token`.
+2. Observe before input and verify the outcome at checkpoints, not after every action. `effect:"unverifiable"` and a successful exit are not task success; never replay a partial, canceled, or unknown action blindly.
+3. Use returned tokens, never invented indices. A fresh snapshot replaces prior element handles and lists them in `invalidated_snapshot_ids`; act with `element_token`. Reads are bounded and return one tree form by default; follow up with `since:"latest"` instead of re-reading a large window.
 4. Keep background window actions non-interfering. Foreground delivery and desktop input require authorization for visible control; an unavailable route is not permission to escalate.
 5. Never infer pixels from a missing image, a different window, or an unaccounted-for resized preview. Capture failure and an empty accessibility tree are different failures.
 6. Keep one controller for a shared desktop. Distinct sessions/cursors do not isolate focus, keyboard input, application state, or snapshot caches.
@@ -48,15 +62,17 @@ Check the installed version and advertised schema before using unfamiliar parame
 
 ## Failure map
 
-| Symptom                                                   | Next step                                                                             |
-| --------------------------------------------------------- | ------------------------------------------------------------------------------------- |
-| Missing binary, mismatched daemon, unknown tool/field     | [Runtime preflight](RUNTIME.md#preflight-and-transport)                               |
-| Stale token or ambiguous window                           | Refresh `list_windows` / `get_window_state`; choose the intended live target          |
-| Large or sparse tree                                      | [Bounded observation](WORKFLOW.md#observe)                                            |
-| `surface_identity_unproven` or screenshot permission wait | [Wayland capture recovery](LINUX.md#capture-recovery)                                 |
-| `background_unavailable`                                  | Verify current state; ask before foreground/desktop control if not already authorized |
-| Text did not visibly change                               | Reobserve before retrying; [text and value semantics](WORKFLOW.md#act-once)           |
-| Browser setup, binding, or ref refused                    | [Browser recovery](BROWSER.md#recovery-rules)                                         |
+| Symptom                                                   | Next step                                                                                                                                                           |
+| --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Missing binary, mismatched daemon, unknown tool/field     | [Runtime preflight](RUNTIME.md#preflight-and-transport)                                                                                                             |
+| Stale token or ambiguous window                           | Refresh `list_windows` / `get_window_state`; choose the intended live target                                                                                        |
+| Large, truncated, or sparse tree; repeated reads          | [Bounded observation and `since` diffs](WORKFLOW.md#observe)                                                                                                        |
+| `surface_identity_unproven` or screenshot permission wait | [Wayland capture recovery](LINUX.md#capture-recovery)                                                                                                               |
+| `background_unavailable`                                  | Not a stop: that input route is closed, the task is not. Read `get_window_state` and act by element (`element_token`, `invoke_menu`); foreground only if authorized |
+| Web page in a browser                                     | Refs from `get_browser_state` with `browser_click` / `browser_type`; pixel clicks only when no ref exists                                                           |
+| LibreOffice, Preview and other macOS editors              | [macOS recipes](MACOS.md#libreoffice-numbers-and-other-editors-in-the-background)                                                                                   |
+| Text did not visibly change                               | Reobserve before retrying; [text and value semantics](WORKFLOW.md#act-once)                                                                                         |
+| Browser setup, binding, or ref refused                    | [Browser recovery](BROWSER.md#recovery-rules)                                                                                                                       |
 
 ## Consult recent Cua activity only for continuation
 

package/dist/builtin/subagents/skills/cua-driver/WINDOWS.md

@@ -376,6 +376,11 @@ When a cua-driver surprises you, diagnose cua-driver first:
   click in.
 - **`Invalid window handle (0x80070578)`?** The HWND you passed is
   stale (window closed, recreated). Re-resolve via `list_windows`.
+- **Large tree or repeated reads?** `get_window_state` returns one compact
+  markdown tree capped at 250 nodes (`full_output:true` restores 5 000 nodes
+  and both forms). Pass `since:<snapshot_id>` for added/changed/removed rows;
+  the focused element is not reported on Windows yet, so an unchanged read
+  says only `no change since …`.
 - **Empty `tree_markdown` / sparse UIA tree?** Some apps populate
   their UIA tree lazily on first call; retry `get_window_state`
   once. If still empty, the app has no UIA provider: fall back to an

package/dist/builtin/subagents/skills/cua-driver/WORKFLOW.md

@@ -35,14 +35,17 @@ These objects are argument fragments for tools advertising `target`, not standal
 
 ## Observe
 
-`get_window_state({pid, window_id})` requests the accessibility tree and a grounding screenshot by default. Check what actually came back: permission, backing-store, or surface-identity failures can leave usable tree data without an image. `screenshot_error` and `screenshot_frame_valid:false` are not empty-tree signals.
+`get_window_state({pid, window_id})` requests the accessibility tree and a grounding screenshot by default. Start with `query` (or `max_elements`) instead of a full read, and verify at checkpoints rather than after every action. Check what actually came back: permission, backing-store, or surface-identity failures can leave usable tree data without an image. `screenshot_error` and `screenshot_frame_valid:false` are not empty-tree signals.
 
-Prefer `structuredContent.elements` in MCP (the CLI prints structured fields directly) over parsing `tree_markdown`. Rows may contain `element_token`, role, label, value, actions, parent, depth, enabled/selected state, `frame` (screen coordinates, the space of `scope:"desktop"` actions), and `screenshot_frame` (pixels of the screenshot in the same response, the space of window-local pointer `x`/`y`). Missing fields are unknown.
+By default the tree comes back **once**, as compact `tree_markdown` (`tree_format:"markdown"`). A row `[N]` is addressed with `element_token` `<snapshot_id>:N`; the `snapshot_id` is in the response header and in `structuredContent`. Use `tree_format:"elements"` for the structured `elements` array when you need explicit tokens, frames, parents, or values, or `"both"` for the old double payload (about twice the size). `full_output:true` restores the previous full response, including the platform's walk limits. In `elements`, rows may contain `element_token`, role, label, value, actions, parent, depth, enabled/selected state, `frame` (screen coordinates, the space of `scope:"desktop"` actions), and `screenshot_frame` (pixels of the screenshot in the same response, the space of window-local pointer `x`/`y`). Missing fields are unknown.
 
 - Use `query` to project matching rows plus ancestors without renumbering their indices.
-- Use `max_elements` / `max_depth` to bound the walk, and compare returned/total counts. Truncation does not prove absence.
-- Use `include_screenshot:false` only when tree-only observation is enough; it cannot ground a pixel action.
-- Where advertised, `include_accessibility_tree:false` requests capture without a tree walk. Check the installed schema first.
+- A read walks at most 250 nodes by default. When it stops there the response says `Tree truncated at max_elements=…` (`truncated:true`, `truncation_hint`). Pass a larger `max_elements`, or narrow with `query` / `max_depth`. Truncation does not prove absence.
+- After the first read, pass `since:"latest"` (or `since:<snapshot_id>`) to get only what changed. `latest` is this window's most recent read with the same `query`, `max_elements` and `max_depth`. You get: `+` added, `~` changed, `-` removed rows (removed ids are the old snapshot's), or `no change since …; focused element is …` (focus is reported on macOS only). The response carries a new `snapshot_id`: use it in tokens (`<new id>:N`). Rows not listed keep their `[N]` unless a `reindexed:` line says otherwise. An unknown, expired, other-window, or differently-scoped `since` returns a full read, and `since_status` says why. A diff reads the rendered tree rows only; it does not prove the screenshot is unchanged.
+- `_note`, `background_input`, and (Linux) `frame_note` are omitted unless `verbose:true`; a degraded snapshot keeps `background_input`.
+- `include_screenshot:false` returns the tree only. On macOS the window keeps the pixel frame of its last screenshot while its size is unchanged, so `x,y` read off that screenshot still work; elsewhere, and after a resize, take a new screenshot before a pixel action.
+- Where advertised, `include_accessibility_tree:false` requests capture without a tree walk. It keeps the current snapshot's rows and tokens valid. `display_only:true` with it is for live previews: it also leaves your pixel frame alone, and its image cannot ground actions. Use it when you only need to look; do not use `max_elements:1` or a `query` that matches nothing for that, because those replace the snapshot with an almost empty one.
+- Row numbers belong to one read. A narrower re-read (smaller `max_elements`, `max_depth`, another `query`) can stop before a row you saw earlier, and `<new snapshot_id>:N` for that row is then out of range. Use rows printed in the read whose `snapshot_id` you use.
 - `capture_mode` is deprecated and ignored. Do not change configuration to repair a sparse tree.
 - Use `screenshot_out_file` to save a PNG instead of inlining it, then actually read the image. Use an absolute, run-scoped output path.
 
@@ -50,7 +53,7 @@ The accessibility model may lag or disagree with rendered state: Electron text s
 
 ## Act once
 
-Use an opaque `element_token` from the latest snapshot of the intended window. Do not derive or edit tokens. A later snapshot can invalidate a pending action, including when another agent observes the same window.
+Use the `element_token` `<snapshot_id>:N` for row `[N]` of the latest snapshot of the intended window, for example `"s0000002a:11"`. A bare `"11"` is accepted only together with `window_id` and then means row 11 of that window's current snapshot. Do not mix a row number from one read with the `snapshot_id` of another. A later snapshot can invalidate a pending action, including when another agent observes the same window.
 
 Example CLI window action, with IDs and token replaced from the preceding response:
 
@@ -66,7 +69,7 @@ cua-driver click '{"target":{"kind":"window","pid":844,"window_id":10725},"eleme
 | Focus a visible field and type  | `type_text` with `x,y,text`, when that form is advertised                      |
 | Replace an exposed native value | `set_value` with `pid,element_token,value` (its own schema, no `target`)       |
 | Send a key or combination       | `press_key` with `key`, or `hotkey` with `keys`                                |
-| Scroll / drag                   | Inspect `describe scroll` / `describe drag`; units and supported shapes matter |
+| Scroll / drag                   | `scroll` takes `direction` and `amount` (wheel notches); signed `dx`/`dy` is converted (positive `dy` is down; up to 50 is notches, more is pixels at 100 px per notch). Inspect `describe drag` |
 
 Text insertion and value replacement are different intents. Setting a field does not prove a form submission, navigation, or rename committed. Open a collapsed search/input control and reobserve before typing into it; one focus-click may not both open and focus it. Inspect the existing value/selection before replacing content.
 
@@ -74,13 +77,48 @@ If a text action returns `unverifiable`, take a fresh snapshot before retrying.
 
 Keep `delivery_mode:"background"` as the default for window input. The route may use accessibility hit-testing even when addressed by pixels: pixel coordinates do not promise physical pointer delivery. Read the returned `route` instead of inferring it from the tool name.
 
+## Batch known actions
+
+`run_steps` is the default way to act. When the next action or actions are decided, send them as one `run_steps` call with `observe:{}` instead of one call per action followed by a read. Even a single step pays off: the action and the look at its result become one call. The batch runs the same tools in order, stops at the first failure, and returns per-step status plus at most one bounded observation. Typical fit: fill several fields, then press a button, then read the result. Do not batch across a point where the answer decides the next step, or when a step reshuffles the window and invalidates element tokens used by later steps (use pixel targets after it, or split the batch there).
+
+```bash
+cua-driver run_steps '{"session":"run-1","steps":[
+  {"tool":"set_value","args":{"pid":844,"element_token":"s0000002a:14","value":"Ada"}},
+  {"tool":"set_value","args":{"pid":844,"element_token":"s0000002a:15","value":"Lovelace"}},
+  {"tool":"click","args":{"target":{"kind":"window","pid":844,"window_id":10725},"element_token":"s0000002a:21"}},
+  {"tool":"press_key","args":{"pid":844,"key":"return"}}
+ ],"delay_ms":100,"observe":{}}'
+```
+
+- `tool` is one of `click`, `double_click`, `right_click`, `set_value`, `type_text`, `press_key`, `hotkey`, `scroll`, `drag`, `move_cursor` (`press` and `type` are accepted for `press_key` and `type_text`, `triple_click` is a click with `count:3`, and a key name such as `down` as the tool is a `press_key`); `args` are exactly that tool's arguments. Observation tools do not run inside a batch: a read between actions is skipped (it could not change the later steps), a `get_window_state` as the last step is the observation (merged with `observe`), and a trailing `zoom` runs after it. Run `describe run_steps` and `describe <tool>` for schemas. Up to 64 steps. `delivery_mode` on `run_steps` is the default for steps that set none.
+- Name the window once: `pid` and `window_id` (or `app`/`window`) on `run_steps` itself are the default for every step that names none.
+- A step whose `element_token` went stale because an earlier step re-rendered the window is retried once on the element with the same role and label in a fresh read, when exactly one matches; the step's line says so.
+- Every step is validated before the first runs, so a malformed step changes nothing. Each step then passes the same session, permission, capability-manifest and approval checks as a direct call; a batch grants nothing a single call lacks, and a refused step ends the batch like any other failure.
+- A batch has one session. Set `session` on `run_steps`; a step may repeat it but not name another.
+- `observe` is optional and reads once, after the last executed step (also after a failure). Pass `true` or `get_window_state` arguments; `pid`/`window_id` come from the last step that names both. Defaults: `since:"latest"` (only what changed since your last read of that window with the same view; a full read if there is none), `include_screenshot:false`, `max_elements:250`. Pass `include_screenshot:true` to see the window, or `since:null` for a full read. Omit `observe` to read nothing.
+- Read `steps[].ok` and `failed_step`. Steps before a failure did run and are not rolled back; steps after it did not. Observe before repairing, as for a single `unverifiable` action. When a step fails because another window of the app holds focus (a dialog opened mid-batch), the end read is of that window, even without `observe`; when a pixel step names a window with no screenshot yet, the end read includes its screenshot. A step refused because its app ignores background input (Electron, LibreOffice) says how to re-send it; set `foreground_fallback:true` on `run_steps` to retry such steps once in the foreground (the window is activated and the pointer may move).
+
+## Script a flow
+
+Use `run_script` when the flow needs logic between actions: a loop over rows, a branch on what the app shows, a retry, or a value read in one window and typed into another. The script is JavaScript, the body of an async function; it runs in ONE call and only its `return` value, console output and a per-call log come back. Use `run_steps` for a fixed list of steps you have already decided, and plain calls when the next step needs your judgement.
+
+```bash
+cua-driver run_script '{"session":"run-1","script":"const app = await cua.getApp(\"Calculator\"); for (const key of [\"7\", \"multiply\", \"6\", \"equals\"]) await app.click({role: \"button\", name: key}); const [display] = await app.query({role: \"statictext\"}); return display.value;"}'
+```
+
+- The only API is `cua`: `getApp`, `launch`, `listApps`, `listWindows`, `sleep`, and on an app `getState`, `query`, `waitFor`, `verify` and the actions (`click`, `typeText`, `pressKey`, `setValue`, `scroll`, `drag`, ...). A target is `{role, name, nth?}` (found fresh, waits up to 3 s), an `element_token`, or `[x, y]` window pixels. `describe run_script` lists the full API.
+- A failed driver call throws; catch it to recover. An uncaught error names the script line, the call and the reason.
+- Every driver call passes the same session, permission, capability-manifest and approval checks as a direct call, so a script grants nothing the same direct calls lack. It cannot call `run_steps`, `run_script`, `set_config` or tools outside its allowlist.
+- Sandbox: QuickJS with no filesystem, network, process, timers or environment. Limits: `timeout_ms` wall time (default 30000, max 120000), `max_calls` driver calls (default 100, max 500), a 32 MiB heap, a bounded stack, and a 64 KiB script. The script stops when the request is cancelled.
+- `run_script` is on by default. Whoever runs the driver can turn it off with `CUA_DRIVER_DISABLE_RUN_SCRIPT=1` or `"disable_run_script": true` in `~/.cua-driver/config.json` (the older `CUA_DRIVER_EXPERIMENTAL_SCRIPT=0` and `"experimental_script": false` also work). `set_config` cannot change it. When it is off, it is not in `tools/list`.
+
 ## Pixel coordinates
 
 Ground window actions on the PNG from that exact `get_window_state`; ground desktop actions on `get_desktop_state`. Origins are top-left, increasing downward. The driver handles its own window capture scaling; do not add window offsets to window-local input.
 
 The harness may downsample the displayed preview independently of the returned PNG. Use the returned dimensions and the original file. If measuring on a resized preview, account for its exact scale in both axes; do not assume the preview is native resolution. Do not guess from accessibility frames or another app's geometry.
 
-After movement, resize, navigation, or a competing desktop interaction, reobserve. When using `zoom`, read its schema and preserve the `from_zoom` mapping on the supported follow-up action. A manually cropped image requires its crop offset; an untracked crop is not an action coordinate source.
+After movement, resize, navigation, or a competing desktop interaction, reobserve. `zoom` takes a region as corners, `x1,y1` (top-left) to `x2,y2` (bottom-right); `x,y,width,height` is converted. Preserve the `from_zoom` mapping on the supported follow-up action. A manually cropped image requires its crop offset; an untracked crop is not an action coordinate source.
 
 For tiny targets, inspect at full resolution or annotate a copy without changing its dimensions. Keep the raw evidence unmodified. `debug_image_out`, where advertised, captures an action diagnostic; it is not a pre-action approval step.