@agent-sh/computer-use-linux 0.7.32 → 0.7.34

package/README.md

@@ -37,7 +37,7 @@ computer-use-linux setup-window-targeting     # install GNOME Wayland window-con
 
 > **Plugin-only or Pi-only installs:** The Claude Code / Codex plugin launcher and `pi install` manage private cached binaries and do not add `computer-use-linux` to `PATH`. Either run the MCP `doctor` / `setup_accessibility` / `setup_window_targeting` tools (or `computer_use_linux_doctor` in Pi) directly from your agent, or also install the CLI above when you want shell subcommands on `PATH`.
 
-The Rust crate is published as [`computer-use-linux`](https://crates.io/crates/computer-use-linux) and the npm package as [`@agent-sh/computer-use-linux`](https://www.npmjs.com/package/@agent-sh/computer-use-linux). Prebuilt `x86_64` and `aarch64` Linux binaries ship with the [latest release](https://github.com/agent-sh/computer-use-linux/releases/latest).
+The Rust crate is published as [`computer-use-linux`](https://crates.io/crates/computer-use-linux) and the npm package as [`@agent-sh/computer-use-linux`](https://www.npmjs.com/package/@agent-sh/computer-use-linux). Prebuilt `x86_64` and `aarch64` Linux binaries ship with the [latest release](https://github.com/agent-sh/computer-use-linux/releases/latest). They need glibc 2.28 or newer: RHEL 8 and its rebuilds, Debian 10, Ubuntu 20.04, and anything later. On an older system, build from source with `cargo install computer-use-linux`.
 
 > Need an **isolated** desktop where the agent never touches your real mouse, keyboard, or focused window? Use the sibling project [**`agent-workspace-linux`**](https://github.com/agent-sh/agent-workspace-linux) alongside this one.
 
@@ -48,7 +48,7 @@ The Rust crate is published as [`computer-use-linux`](https://crates.io/crates/c
 Most computer-use MCP servers are macOS-only (they lean on AppKit, AXUIElement, CGEvent). The few that target Linux either drive `xdotool` against an X11 root window or shell out to OCR over screenshots. Five things set this one apart:
 
 - **Wayland actually works.** Pointer actions use `/dev/uinput` or `org.freedesktop.portal.RemoteDesktop` on Wayland, with `ydotool` / `ydotoold` as the fallback. Literal text prefers KDE clipboard integration on Plasma, `wtype` on compatible Wayland compositors, and portal keysyms on other Wayland sessions, including GNOME with ydotool installed. Portal text startup or conversion errors fail the call without replaying text through ydotool. Raw ydotool text accepts printable ASCII, tab and newline only, and its physical US key positions can produce different text under the active layout. Screenshots use the GNOME Shell DBus screenshot method when present, `org.freedesktop.portal.Screenshot` otherwise, then on a native X11 session a root-window `GetImage` (device pixels, no toolkit scaling), and finally spawn `gnome-screenshot` for background/systemd contexts where the DBus paths are denied.
-- **Window targeting is compositor-aware.** The window registry tries GNOME Shell extension, GNOME Shell Introspect, COSMIC Wayland helper, KWin DBus scripting, Hyprland `hyprctl`, the niri IPC socket, i3 IPC, labwc's `labwc.sock`, and generic X11/EWMH in order, then reports exactly which backend won or why each backend failed. Terminal windows are enriched with their controlling TTY and foreground process so agents can target "the terminal running `pytest`" directly.
+- **Window targeting is compositor-aware.** The window registry tries GNOME Shell extension, GNOME Shell Introspect, COSMIC Wayland helper, KWin DBus scripting, Hyprland `hyprctl`, the niri IPC socket, i3 IPC, labwc's `labwc.sock`, and generic X11/EWMH in order, then reports exactly which backend won or why each backend failed. Terminal windows are enriched with their controlling TTY and foreground process so agents can target "the terminal running `pytest`" directly. Every `window_id` fits JavaScript's exact integer range (at most 2^53-1), so JSON clients can pass it back unchanged; a larger value is rejected with a prompt to re-run `list_windows`.
 - **Semantic selectors first, pixel coordinates when needed.** Tools like `click`, `perform_action`, and `set_value` accept `role` / `name` / `text` / `states` selectors or `element_index` values backed by AT-SPI, and every mutating tool can return a post-action screenshot and updated accessibility tree in the same round trip. Pixel coordinates remain available for rendering-only surfaces (canvas, games, X clients without ATK).
 - **Visible to humans, invisible to screenshots.** While an agent acts, `computer-use-linux-indicator` renders a click-through overlay with a per-agent cursor glide, click ripple, keycaps/typing banner, screen-edge glow, and status pill — and hides automatically under a shared capture lock before any screenshot is read.
 - **One JSON readiness report.** `computer-use-linux doctor` returns a structured document covering platform, portals, AT-SPI, windowing, input, and a `readiness` summary with explicit blockers and a recommended next step. MCP hosts can render or surface that to the user without parsing prose.
@@ -80,6 +80,8 @@ The crate was extracted from [`codex-desktop-linux`](https://github.com/avifenes
 | **Opt-in host tools** | `complete_interaction` | Parameter-free desktop completion notification (`notify-send`); registered only when `COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1`. |
 | | `run_shell` | Bounded non-login `/bin/sh -c` execution; registered only when `COMPUTER_USE_LINUX_ENABLE_SHELL=1` (absent by default). |
 
+`click`, `scroll`, `perform_action`, and `set_value` take an optional app target (`pid`, `window_id`, `app_id`, `wm_class`, `window_title`). With one, an `element_index` or selector from another app's `get_app_state` snapshot is refused; call `get_app_state` for the target app first. When the window backend reports no pid for a target window (i3, sway, COSMIC), the snapshot must have been taken for that window, or the call is refused. `perform_action` and `set_value` do not focus the target window. An `element_identifier` is refused when its owner on the accessibility bus is a different pid than the target; it is not checked when either pid is unknown.
+
 ### Post-action observation in one round trip
 
 All mutating tools (`click`, `drag`, `scroll`, `press_key`, `type_text`, `perform_action`, `set_value`, `activate_window`, `move_window`, and `resize_window`) accept optional post-action observation parameters so an action and its verification complete in a single MCP tool call:
@@ -119,7 +121,7 @@ the origin. A missing window target is rejected.
 
 **X11 text fidelity:** `xdotool`/XTEST can silently substitute symbols under non-US XKB layouts (reported with Brazilian `br`/ABNT2) or changing keymap state, even when `type_text` returns `ok: true`. Every successful xdotool text dispatch includes a layout warning. XKB layout replacement events, persistent mapping/group changes, and keyboard focus/grab transitions detected across a call return an error after dispatch, without replaying the text. Keyboard grabs can divert characters without changing the polled focused-window ID. If monitoring cannot start, xdotool still dispatches and the response explicitly warns that layout-change and keyboard interception detection were unavailable. PointerRoot focus is unsupported because pointer motion can redirect input without a detectable focus event. This also detects a `setxkbmap` layout changed and restored during one call. Temporary raw remaps or group switches restored within a call and dropped text with a stable map remain unverified. Re-observe the actual text before pressing Enter, submitting it, or treating it as delivered. Use `include_tree: true` for AT-SPI readback when the application exposes the text, or `screenshot_after: true` to inspect it. These observations do not automatically compare the field to the requested text. Do not retry blindly.
 
-Targeted `press_key`/`type_text` results append focused-element feedback from AT-SPI (role, name, editable) and warn when a concrete non-terminal focus is not editable. Terminals accept keyboard input without an EditableText interface; re-observe the contents to confirm input. Click/screenshot/input results warn when the target window or coordinate is partially or fully off-screen. `get_app_state` returns a compact readiness block by default; pass `verbose: true` for the full diagnostics report. It also reports `tree_scoped` (false when no app target narrowed the AT-SPI tree, with a warning in `message`) and `accessibility_tree_truncated` (true when the node, depth, read, or time budget stopped traversal with unread elements left). When a result would pass 32,000 bytes, the tree comes back as its leading nodes plus `accessibility_tree_spill` (`path`, `node_count`, `bytes`, `inline_node_count`). `path` is a private JSON file holding the whole tree with the same `element_index` values, in `$XDG_RUNTIME_DIR/computer-use-linux/trees` when that runtime directory is private, otherwise under `~/.cache` (directories 0700, file 0600). Search the file instead of requesting the tree again; hosts that cannot read files page through the same tree with `get_tree_page`. The file lasts until the next snapshot or until the server exits; after a crash it is swept at the next start or after an hour.
+Targeted `press_key`/`type_text` results append focused-element feedback from AT-SPI (role, name, editable) and warn when a concrete non-terminal focus is not editable. Terminals accept keyboard input without an EditableText interface; re-observe the contents to confirm input. Click/screenshot/input results warn when the target window or coordinate is partially or fully off-screen. `get_app_state` returns a compact readiness block by default; pass `verbose: true` for the full diagnostics report. Tree nodes use a compact encoding: null, empty, zero and false fields are left out, containers do not carry the U+FFFC placeholder text Chromium gives them, and `actions` lists action names (pass one as `perform_action`'s `action`). `verbose: true` returns the full encoding, with every field and action descriptions and keybindings. `interactive_only: true` returns only nodes with an action, editable text or a value, plus headings, status and live regions, windows and dialogs; containers are collapsed, so `parent_index` names the nearest kept ancestor. Its `element_index` values are the same as in the full tree. When `title` or `window_id` names one of several windows under one AT-SPI application root (Chromium puts every browser window under one), the tree covers that window plus the app's popups and dialogs, and drops its other named windows; `message` says so. It also reports `tree_scoped` (false when no app target narrowed the AT-SPI tree, with a warning in `message`) and `accessibility_tree_truncated` (true when the node, depth, read, or time budget stopped traversal with unread elements left). When a result would pass 32,000 bytes, the tree comes back as its leading nodes plus `accessibility_tree_spill` (`path`, `node_count`, `bytes`, `inline_node_count`). `path` is a private JSON file holding the whole tree with the same `element_index` values, in `$XDG_RUNTIME_DIR/computer-use-linux/trees` when that runtime directory is private, otherwise under `~/.cache` (directories 0700, file 0600). Search the file instead of requesting the tree again; hosts that cannot read files page through the same tree with `get_tree_page`. The file lasts until the next snapshot or until the server exits; after a crash it is swept at the next start or after an hour.
 
 An `app_id` may be a full desktop ID or its final dot-separated segment; matching is case-insensitive, ambiguous aliases are rejected, and an unresolved explicit `app_id` returns an empty scoped tree rather than the desktop.
 
@@ -177,7 +179,7 @@ Validated manually on Ubuntu 25.10 (GNOME Shell 50.1, Wayland). niri window list
 | --- | --- | --- |
 | GNOME Wayland | GNOME Shell extension first, `org.gnome.Shell.Introspect` fallback | Full target. The extension provides exact window activation when GNOME blocks native introspection; Introspect can list windows and verify the current app; activating a window requires the extension. |
 | GNOME X11 | `org.gnome.Shell.Introspect`, then generic X11/EWMH | AT-SPI works; keyboard input prefers `xdotool`/XTEST so the live XKB layout resolves keys correctly. Scroll uses xdotool wheel buttons so GTK 3 does not drop the event after a pointer warp. |
-| KDE Plasma / KWin | temporary KWin DBus scripting | Lists and focuses windows through Plasma 5 or 6 `org.kde.KWin` scripting APIs when the session bus exposes them. |
+| KDE Plasma / KWin | temporary KWin DBus scripting | Lists and focuses windows through Plasma 5 or 6 `org.kde.KWin` scripting APIs when the session bus exposes them. IDs are derived from the KWin window UUID, stay the same for a window across calls, and fit JavaScript integers. |
 | Hyprland | `hyprctl clients -j` and `hyprctl dispatch focuswindow` | Requires `hyprctl` in the desktop session. |
 | niri | `niri msg` with a direct JSON IPC fallback | Lists toplevels and focuses exact window IDs. Export the session's `NIRI_SOCKET`; otherwise discovery requires an unambiguous socket matching `WAYLAND_DISPLAY`. The fallback works without the `niri` binary. Missing positions remain `null`; bounds are omitted if output scaling is unknown or mixed. |
 | i3 | `i3-msg`; optional `xprop` for PID hydration | Lists and focuses i3 windows over the active i3 IPC socket. |
@@ -479,7 +481,9 @@ Most setups need none of these — `doctor` and the installers pick sensible def
 | `YDOTOOL_SOCKET` | Explicit ydotool socket path. When unset, only an owner-only `.ydotool_socket` in the user's private runtime directory is accepted; the legacy shared `/tmp/.ydotool_socket` is never selected automatically. Setting this variable is an operator trust override for custom layouts. |
 | `COMPUTER_USE_LINUX_FORCE_XDOTOOL_KEYBOARD` | Prefer `xdotool`/XTEST keyboard input when `DISPLAY` is available. `COMPUTER_USE_LINUX_FORCE_YDOTOOL_KEYBOARD=1` takes precedence. |
 | `COMPUTER_USE_LINUX_XDOTOOL_TYPE_DELAY_MS` | Per-character delay for `xdotool type` in milliseconds (default `12`). `0` is faster but can deliver characters out of order on some X servers. |
-| `COMPUTER_USE_LINUX_SCREENSHOT_BACKEND` | Force a single screenshot backend, skipping the fallback chain. Accepts `gnome-shell`, `portal`, `x11`, or `gnome-screenshot`. `x11` works only on a native X11 session. Pin `gnome-screenshot` for background/systemd contexts where the GNOME Shell and portal DBus paths are denied. |
+| `COMPUTER_USE_LINUX_SCREENSHOT_BACKEND` | Force a single screenshot backend, skipping the fallback chain. Accepts `gnome-shell`, `portal`, `x11`, `gnome-screenshot`, or `pixelflux`. `x11` works only on a native X11 session. `pixelflux` needs `PIXELFLUX_CU` and `PIXELFLUX_CU_TOKEN`. Pin `gnome-screenshot` for background/systemd contexts where the GNOME Shell and portal DBus paths are denied. |
+| `PIXELFLUX_CU` | Loopback port of a Selkies pixelflux computer-use server (`5000` or `127.0.0.1:5000`). With `PIXELFLUX_CU_TOKEN`, screenshots try pixelflux before the other backends, and click, scroll, drag, press_key, and type_text go there without a portal prompt. A closed port falls through. A server error does not. Anything other than loopback is ignored. |
+| `PIXELFLUX_CU_TOKEN` | Bearer token for `PIXELFLUX_CU`. Never logged. Both variables must be set. |
 | `COMPUTER_USE_LINUX_ENABLE_SHELL` | Set exactly to `1` before starting the MCP server to register the destructive `run_shell` tool. Unset by default. Do not enable for untrusted or unattended MCP hosts. |
 
 **Build-time identity overrides** (set while compiling a downstream embedded
@@ -507,6 +511,7 @@ files.
 - **Limited accessibility feedback** — when exact window focus was verified before input but AT-SPI reports no focused element, the result describes the missing feedback without claiming the input was lost. It still does not verify the field contents. A complete scoped tree containing only application/window containers adds an Electron/Chromium `--force-renderer-accessibility` hint; a truncated tree instead retains its traversal-limit guidance.
 - **Focused containers**: Chromium can mark a `document web` focused alongside its contenteditable editor. The focus probe keeps searching that document and reports the editor; a document that holds focus itself still gets a warning, which no longer claims the text was lost. An editor can carry the `editable` state without the EditableText interface (`supports_editable_text: false`): `type_text` can reach it, `set_value` cannot. Verify its text or a screenshot before submitting or retrying.
 - **Large embedded browser trees**: an app-scoped snapshot includes embedded browser documents, which can use up the node budget before the app's own controls. When `accessibility_tree_truncated` is true, a missing editor proves nothing: raise `max_nodes` (default 1000, hard cap 10000) and, if needed, `max_depth` (hard cap 64). A busy renderer may give only a few thousand nodes before the deadline. A snapshot stops reading at 10 seconds and returns what it read, marked truncated; if the editor is still missing, hide the embedded browser pane for the snapshot.
+- **Selkies pixelflux**: when `PIXELFLUX_CU` and `PIXELFLUX_CU_TOKEN` are set, screenshots and pointer/keyboard actions use the container's loopback computer-use server first. Element clicks scale AT-SPI bounds by the latest screenshot size over the logical layout (`wlr-output-management` on labwc, the existing desktop layout elsewhere). Coordinate clicks use screenshot pixels. A connection failure falls through to the backends below; an HTTP error does not.
 - **Wayland coordinate input** — coordinate clicks, drags, and targeted scrolls require the absolute uinput pointer or an approved RemoteDesktop portal session. Targeted scroll positions through uinput before sending only wheel events through ydotool. If accurate positioning is unavailable, the call returns `ok: false` before sending coordinate input. The absolute pointer still requires a successful full-desktop capture when first created; an existing device does not need another capture. Scrolling without a point or window/element target keeps using the current pointer position. Portal keyboard input preserves the receiving window or app target before consent, then restores and verifies its focus after consent. If focus cannot be verified, no text or key chord is sent. Untargeted portal keyboard calls therefore require working window introspection. GNOME Introspect only verifies an already focused app; if consent changes focus, refocus the app manually or install the Shell extension.
 - **Native X11 coordinate clicks.** Eligible native X11 sessions use one supervised `xdotool mousemove -- X Y click --repeat N BUTTON` command for left, middle, and right clicks; native X11 does not initialize the system-wide uinput device. Missing or failed xdotool commands return an error without routing input to another display. `COMPUTER_USE_LINUX_FORCE_YDOTOOL_POINTER=1` skips this xdotool path.
 - **Native X11 scroll.** The same sessions send one `xdotool` command, `mousemove -- X Y click --delay 12 --repeat N BUTTON` when a point is set, or `click` alone when it is not. Buttons are 4 up, 5 down, 6 left, and 7 right, and `N` is the same notch count ydotool would pass to `mousemove --wheel` (five per page). ydotool's absolute move warps through (0, 0); GTK 3 then drops the single following wheel event because re-entry resets its XI2 scroll valuators. XTEST has no scroll valuators, so GTK takes the button path. Missing or failed xdotool commands return an error without a system-wide fallback. `COMPUTER_USE_LINUX_FORCE_YDOTOOL_POINTER=1` skips this path.

package/npm/README.md

@@ -138,6 +138,7 @@ does not yet forward the flag or include the tool in its catalog.
   disabling other clients or restoring an old value. An app launched during a
   reset may still need restarting, so this is not a complete GNOME toggle fix.
 - **COSMIC Wayland**: Pointer coordinates use per-monitor RemoteDesktop portal streams. Approve the consent dialog before continuing. Available EIS connections wait for device readiness before the first input event. Window bounds support targeted screenshots and relative input, and window IDs fit JavaScript's exact integer range. The toplevel protocol does not expose PID.
+- **Window IDs**: Every backend's `window_id` fits JavaScript's exact integer range (at most 2^53-1), including KWin, so JSON clients can pass it back unchanged. A larger value is rejected with a prompt to re-run `list_windows`.
 - **Portal keyboard focus**: The receiving window or app target is preserved before consent and its focus is restored and verified afterward. If verification fails, no text or key chord is sent. Untargeted portal keyboard calls require working window introspection. GNOME Introspect can only verify an already focused app; refocus it manually after consent if needed.
 - **GNOME Wayland portal persistence**: On GNOME Wayland the remote-control portal asks again for every new server
   process. Set `COMPUTER_USE_LINUX_PERSIST_REMOTE_DESKTOP=1` in that process's
@@ -152,6 +153,14 @@ does not yet forward the flag or include the tool in its catalog.
 
 ## Input & coordinate contracts
 
+- **Element targets**: `click`, `scroll`, `perform_action`, and `set_value`
+  accept an optional app target (`pid`, `window_id`, `app_id`, `wm_class`,
+  `window_title`). With one, an `element_index` or selector from another app's
+  `get_app_state` snapshot is refused. When the window backend reports no pid
+  for the target window (i3, sway, COSMIC), the snapshot must have been taken
+  for that window. `perform_action` and `set_value` do not focus the window;
+  an `element_identifier` is refused only when its bus owner is known and is
+  not the target pid.
 - **Click vs. semantic actions**: Plain left `click` by element index or selector prefers a native AT-SPI
   `click`, `press`, or `toggle` action over toolkit bounds,
   avoiding pointer-coordinate conversion when that action is available. Explicit
@@ -168,6 +177,7 @@ does not yet forward the flag or include the tool in its catalog.
   screenshot crop origin and divide preview coordinates by screenshot `scale`.
   Do not use widget-local or raw GDK surface coordinates. A window target is
   required.
+- **Selkies containers**: set `PIXELFLUX_CU` and `PIXELFLUX_CU_TOKEN` so screenshots and input use the container's pixelflux computer-use server. No portal prompt. A closed port falls through; a server error does not.
 - **X11 layout risk**: xdotool/XTEST can silently substitute symbols under non-US XKB layouts (reported with Brazilian `br`/ABNT2) or changing keymap state, even with `ok: true`. Successful xdotool text dispatches include a layout warning. XKB layout replacement events, persistent mapping/group changes, and keyboard focus/grab transitions detected across a call return an error after dispatch, without replaying the text. Keyboard grabs can divert characters without changing the polled focused-window ID. If monitoring cannot start, xdotool still dispatches and the response explicitly warns that layout-change and keyboard interception detection were unavailable. This also detects a `setxkbmap` layout changed and restored during one call. Temporary raw remaps or group switches restored within a call and dropped text with a stable map remain unverified. Inspect the actual text before pressing Enter, submitting it, or treating it as delivered. Use `include_tree: true` when AT-SPI exposes the text, or `screenshot_after: true`; neither automatically compares the field with the request. Do not retry blindly.
 - **X11 text input**: X11 text input sends LF and CRLF line endings as Return. Focused terminals
   accept keyboard input even when AT-SPI does not expose EditableText. Focus
@@ -196,7 +206,8 @@ does not yet forward the flag or include the tool in its catalog.
   a document that holds focus itself still gets a warning, which no longer
   claims the text was lost. An editor can carry the `editable` state without the
   EditableText interface (`supports_editable_text: false`): `type_text` can
-  reach it, `set_value` cannot. Verify its text or a screenshot before
+  reach it, `set_value` cannot. Compact nodes leave a false
+  `supports_editable_text` out. Verify its text or a screenshot before
   submitting or retrying. Embedded browser documents can use up an app-scoped
   snapshot's node budget before the app's own controls; when
   `accessibility_tree_truncated` is true, a missing editor proves nothing. Raise
@@ -213,3 +224,14 @@ does not yet forward the flag or include the tool in its catalog.
   private, otherwise `~/.cache`). Search it instead of requesting the tree
   again; hosts that cannot read files page through it with `get_tree_page`. The
   file lasts until the next snapshot or until the server exits.
+
+  Tree nodes use a compact encoding: null, empty, zero and false fields are
+  left out, containers do not carry the U+FFFC placeholder text Chromium gives
+  them, and `actions` lists action names. `verbose: true` returns the full
+  encoding with action descriptions and keybindings. `interactive_only: true`
+  returns only nodes with an action, editable text or a value, plus headings,
+  status and live regions, windows and dialogs, with containers collapsed and
+  the same `element_index` values as the full tree. When `title` or
+  `window_id` names one of several windows under one AT-SPI application root,
+  as in Chromium, the tree covers that window plus the app's popups and
+  dialogs, and drops its other named windows.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@agent-sh/computer-use-linux",
-  "version": "0.7.32",
+  "version": "0.7.34",
   "description": "Native Linux desktop control for AI agents (MCP server, Pi tools, and CLI): semantic AT-SPI accessibility trees, screenshots, compositor window targeting, Wayland/X11 input, and an on-screen activity indicator.",
   "license": "MIT",
   "type": "commonjs",

package/pi/extension/generated-tools.ts

@@ -7,9 +7,9 @@ export interface GeneratedMcpToolDefinition {
 	annotations: Record<string, unknown>;
 }
 
-export const GENERATED_SERVER_VERSION = "0.7.32";
-export const GENERATED_TOOL_CATALOG_HASH = "379903a7c526e5517b75cdc50ac9319aaadf0df01441bd96a87ab46787d26ae7";
-export const GENERATED_SHELL_TOOL_CATALOG_HASH = "b08763dff65cdc5388c5202df72e0399f6809a8f8144fa7d37281df5feea2308";
+export const GENERATED_SERVER_VERSION = "0.7.34";
+export const GENERATED_TOOL_CATALOG_HASH = "88f353a580174a633cfc2a501dc19aca3913d1ec317d33fb38e8ac1d2bcd40d4";
+export const GENERATED_SHELL_TOOL_CATALOG_HASH = "52d70dfdec115799ca570e3419ccddecdd3d170d36492903fc8796fdb14037a9";
 export const GENERATED_MCP_TOOLS =
 [
   {
@@ -162,6 +162,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "window_id": {
           "default": null,
+          "maximum": 9007199254740991,
           "minimum": 0,
           "type": [
             "integer",
@@ -365,6 +366,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "window_id": {
           "default": null,
+          "maximum": 9007199254740991,
           "minimum": 0,
           "type": [
             "integer",
@@ -565,7 +567,7 @@ export const GENERATED_MCP_TOOLS =
       "openWorldHint": true,
       "readOnlyHint": true
     },
-    "description": "Start an app use session if needed, then get a size-bounded screenshot and accessibility state for a Linux app. Scope the accessibility tree with app_name_or_bundle_identifier or a window_id/pid/app_id/wm_class/title target; omitting a target returns the whole desktop tree and can flood context. A tree too large to return inline is written whole to the private file in accessibility_tree_spill. Screenshot results include coordinate_width, coordinate_height, scale, format, and quality when the returned image is downscaled or compressed; callers can request jpeg/quality for compression before resizing.",
+    "description": "Start an app use session if needed, then get a size-bounded screenshot and accessibility state for a Linux app. Scope the accessibility tree with app_name_or_bundle_identifier or a window_id/pid/app_id/wm_class/title target; omitting a target returns the whole desktop tree and can flood context. Nodes are compact (empty fields left out, actions by name; verbose=true returns every field). Pass interactive_only=true for only actionable nodes, headings and status regions, with the same element_index values. A title or window_id target narrows an app with several windows under one AT-SPI root (Chromium) to that window, keeping the app's popups and dialogs. A tree too large to return inline is written whole to the private file in accessibility_tree_spill. Screenshot results include coordinate_width, coordinate_height, scale, format, and quality when the returned image is downscaled or compressed; callers can request jpeg/quality for compression before resizing.",
     "inputSchema": {
       "$defs": {
         "ScreenshotOutputFormat": {
@@ -614,6 +616,14 @@ export const GENERATED_MCP_TOOLS =
             "null"
           ]
         },
+        "interactive_only": {
+          "default": null,
+          "description": "Return only nodes that can be acted on (an action, editable text, or a\nvalue), plus headings, status/live regions, windows and dialogs.\nContainers are collapsed: each node's parent_index points at its\nnearest kept ancestor. element_index values are the same as in the full\ntree, so they work with click, perform_action and set_value.",
+          "type": [
+            "boolean",
+            "null"
+          ]
+        },
         "max_bytes": {
           "default": null,
           "description": "Maximum returned screenshot image bytes before base64 (default 2 MiB, hard-capped).",
@@ -730,7 +740,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "verbose": {
           "default": null,
-          "description": "Include the full diagnostics report (large). Default false: only the\ncompact readiness block is returned.",
+          "description": "Include the full diagnostics report (large) and return tree nodes in the\nfull encoding, with null and empty fields and action descriptions and\nkeybindings. Default false: only the compact readiness block, and\ncompact nodes.",
           "type": [
             "boolean",
             "null"
@@ -739,6 +749,7 @@ export const GENERATED_MCP_TOOLS =
         "window_id": {
           "default": null,
           "description": "Compositor window id. Also scopes the accessibility tree to that window's\napplication when possible.",
+          "maximum": 9007199254740991,
           "minimum": 0,
           "type": [
             "integer",
@@ -961,6 +972,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "window_id": {
           "default": null,
+          "maximum": 9007199254740991,
           "minimum": 0,
           "type": [
             "integer",
@@ -1000,7 +1012,7 @@ export const GENERATED_MCP_TOOLS =
       "openWorldHint": true,
       "readOnlyHint": false
     },
-    "description": "Invoke an accessibility action exposed by an element selected by index, identifier, or semantic selector. Defaults to the primary action unless action is provided. A stale semantic target can recover once with verified process/window ownership. Recovery invalidates cached indices; call get_app_state before further element_index actions. Pass screenshot_after (true, \"screen\", or \"window\") or include_tree=true to return post-action observation in the same result.",
+    "description": "Invoke an accessibility action exposed by an element selected by index, identifier, or semantic selector. Defaults to the primary action unless action is provided. A stale semantic target can recover once with verified process/window ownership. Recovery invalidates cached indices; call get_app_state before further element_index actions. Pass the target app (pid, window_id, app_id, wm_class, or window_title, as for click) to refuse an element_index or selector from another app's or window's get_app_state snapshot, and an element_identifier owned by another pid; the window is not focused. Pass screenshot_after (true, \"screen\", or \"window\") or include_tree=true to return post-action observation in the same result.",
     "inputSchema": {
       "$defs": {
         "ScreenshotOutputFormat": {
@@ -1020,6 +1032,13 @@ export const GENERATED_MCP_TOOLS =
             "null"
           ]
         },
+        "app_id": {
+          "default": null,
+          "type": [
+            "string",
+            "null"
+          ]
+        },
         "element_identifier": {
           "default": null,
           "type": [
@@ -1084,6 +1103,14 @@ export const GENERATED_MCP_TOOLS =
             "null"
           ]
         },
+        "pid": {
+          "default": null,
+          "minimum": 0,
+          "type": [
+            "integer",
+            "null"
+          ]
+        },
         "quality": {
           "description": "JPEG quality from 1 to 95 (default 80) for post-action screenshots. Ignored for png.",
           "maximum": 95,
@@ -1139,6 +1166,29 @@ export const GENERATED_MCP_TOOLS =
             "string",
             "null"
           ]
+        },
+        "window_id": {
+          "default": null,
+          "maximum": 9007199254740991,
+          "minimum": 0,
+          "type": [
+            "integer",
+            "null"
+          ]
+        },
+        "window_title": {
+          "default": null,
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "wm_class": {
+          "default": null,
+          "type": [
+            "string",
+            "null"
+          ]
         }
       },
       "type": "object"
@@ -1298,6 +1348,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "window_id": {
           "default": null,
+          "maximum": 9007199254740991,
           "minimum": 0,
           "type": [
             "integer",
@@ -1479,6 +1530,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "window_id": {
           "default": null,
+          "maximum": 9007199254740991,
           "minimum": 0,
           "type": [
             "integer",
@@ -1619,6 +1671,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "window_id": {
           "default": null,
+          "maximum": 9007199254740991,
           "minimum": 0,
           "type": [
             "integer",
@@ -1780,6 +1833,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "window_id": {
           "default": null,
+          "maximum": 9007199254740991,
           "minimum": 0,
           "type": [
             "integer",
@@ -1831,7 +1885,7 @@ export const GENERATED_MCP_TOOLS =
       "openWorldHint": true,
       "readOnlyHint": false
     },
-    "description": "Set the value of a settable accessibility element selected by index, identifier, or semantic selector. Editable-text writes are verified by exact readback; password fields are explicitly unverifiable. Pass screenshot_after (true, \"screen\", or \"window\") or include_tree=true to return post-action observation in the same result.",
+    "description": "Set the value of a settable accessibility element selected by index, identifier, or semantic selector. Editable-text writes are verified by exact readback; password fields are explicitly unverifiable. Pass the target app (pid, window_id, app_id, wm_class, or window_title, as for click) to refuse an element_index or selector from another app's or window's get_app_state snapshot, and an element_identifier owned by another pid; the window is not focused. Pass screenshot_after (true, \"screen\", or \"window\") or include_tree=true to return post-action observation in the same result.",
     "inputSchema": {
       "$defs": {
         "ScreenshotOutputFormat": {
@@ -1844,6 +1898,13 @@ export const GENERATED_MCP_TOOLS =
       },
       "$schema": "https://json-schema.org/draft/2020-12/schema",
       "properties": {
+        "app_id": {
+          "default": null,
+          "type": [
+            "string",
+            "null"
+          ]
+        },
         "element_identifier": {
           "default": null,
           "type": [
@@ -1908,6 +1969,14 @@ export const GENERATED_MCP_TOOLS =
             "null"
           ]
         },
+        "pid": {
+          "default": null,
+          "minimum": 0,
+          "type": [
+            "integer",
+            "null"
+          ]
+        },
         "quality": {
           "description": "JPEG quality from 1 to 95 (default 80) for post-action screenshots. Ignored for png.",
           "maximum": 95,
@@ -1966,6 +2035,29 @@ export const GENERATED_MCP_TOOLS =
         },
         "value": {
           "type": "string"
+        },
+        "window_id": {
+          "default": null,
+          "maximum": 9007199254740991,
+          "minimum": 0,
+          "type": [
+            "integer",
+            "null"
+          ]
+        },
+        "window_title": {
+          "default": null,
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "wm_class": {
+          "default": null,
+          "type": [
+            "string",
+            "null"
+          ]
         }
       },
       "required": [
@@ -2156,6 +2248,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "window_id": {
           "default": null,
+          "maximum": 9007199254740991,
           "minimum": 0,
           "type": [
             "integer",

package/skills/computer-use-linux/SKILL.md

@@ -138,14 +138,14 @@ Pi native tools skip this MCP `command` config; see [Pi setup](references/pi-set
 ## Procedure
 
 1. In Pi, call `computer_use_linux_tools` with the exact tools or capability you need. Enabled tools use the `computer_use_linux_<name>` prefix, appear starting on the next model turn, and remain active for the session.
-2. Begin every desktop-control turn with `get_app_state`, scoped to the app you are working in: pass `app_name_or_bundle_identifier` or a window target (`window_id`, `pid`, `app_id`, `wm_class`, `title`). An `app_id` may be a full desktop ID or its final dot-separated segment; matching is case-insensitive, ambiguous aliases are rejected, and an unresolved explicit `app_id` returns an empty scoped tree rather than the desktop. Without a target the result is the whole desktop AT-SPI tree, `tree_scoped` is `false`, and `message` warns; that can flood context. Use `include_screenshot: false` when the accessibility tree is sufficient. If `accessibility_tree_truncated` is `true`, the tree is incomplete: scope to a narrower target and raise `max_nodes` or `max_depth` (hard caps 10000 and 64) rather than lowering them. A tree too large to return inline comes back as its leading nodes plus `accessibility_tree_spill.path`, a private JSON file with the whole tree and the same `element_index` values: search it (for a role, name or text) instead of requesting the tree again, or page through it with `get_tree_page` when you cannot read files. The compact readiness block identifies missing setup.
+2. Begin every desktop-control turn with `get_app_state`, scoped to the app you are working in: pass `app_name_or_bundle_identifier` or a window target (`window_id`, `pid`, `app_id`, `wm_class`, `title`). An `app_id` may be a full desktop ID or its final dot-separated segment; matching is case-insensitive, ambiguous aliases are rejected, and an unresolved explicit `app_id` returns an empty scoped tree rather than the desktop. Without a target the result is the whole desktop AT-SPI tree, `tree_scoped` is `false`, and `message` warns; that can flood context. Use `include_screenshot: false` when the accessibility tree is sufficient. If `accessibility_tree_truncated` is `true`, the tree is incomplete: scope to a narrower target and raise `max_nodes` or `max_depth` (hard caps 10000 and 64) rather than lowering them. A tree too large to return inline comes back as its leading nodes plus `accessibility_tree_spill.path`, a private JSON file with the whole tree and the same `element_index` values: search it (for a role, name or text) instead of requesting the tree again, or page through it with `get_tree_page` when you cannot read files. Nodes are compact: null, empty, zero and false fields are left out and `actions` lists action names; `verbose: true` returns every field. To find controls in a large tree, pass `interactive_only: true`: it keeps nodes with an action, editable text or a value, plus headings, status and live regions, windows and dialogs, collapses containers, and keeps the same `element_index` values. A `title` or `window_id` target narrows a Chromium-style app with several windows under one AT-SPI root to that window, keeping the app's popups and dialogs. The compact readiness block identifies missing setup.
 3. Use `doctor` only when you need the full diagnostic report.
 4. If `can_build_accessibility_tree` is false, run `setup_accessibility` and restart the target app.
 5. If `can_query_windows` is false on GNOME Wayland, run `setup_window_targeting` and ask the user to log out and back in if setup says the shell extension needs a reload.
 6. Before targeted input, call `list_windows` or `focused_window` and verify the intended window by title, app id, pid, or wm class.
-7. Prefer semantic targeting from `get_app_state`: use element indices or role/name/text/states selectors. Automatic recovery of a dead semantic target requires verified PID/window ownership and invalidates cached indices; call `get_app_state` again before further index-based actions.
+7. Prefer semantic targeting from `get_app_state`: use element indices or role/name/text/states selectors. Automatic recovery of a dead semantic target requires verified PID/window ownership and invalidates cached indices; call `get_app_state` again before further index-based actions. When several apps are in play, pass the app target (`pid`, `window_id`, `app_id`, `wm_class`, `window_title`) to `click`, `scroll`, `perform_action`, and `set_value`: an index or selector from another app's snapshot is then refused instead of acting on that app. Where the window backend reports no pid (i3, sway, COSMIC), call `get_app_state` for the same window first or pass `pid`, or the call is refused.
 8. Use coordinates only when the UI surface has no useful accessibility tree.
-9. For text input, prefer `type_text` with a target selector (`window_id`, `pid`, `app_id`, `wm_class`, `title`, `tty`, `terminal_pid`, `terminal_command`, or `terminal_cwd`) rather than relying on current focus.
+9. For text input, prefer `type_text` with a target selector (`window_id`, `pid`, `app_id`, `wm_class`, `title`, `tty`, `terminal_pid`, `terminal_command`, or `terminal_cwd`) rather than relying on current focus. A `window_id` is valid only as returned by a recent `list_windows`; if a call rejects it as outside 0 to 2^53-1 or matches no window, re-run `list_windows`.
 10. After mutating actions (`click`, `drag`, `scroll`, `press_key`, `type_text`, `perform_action`, `set_value`, `activate_window`, `move_window`, `resize_window`), either pass `screenshot_after: true | "screen" | "window"` (with optional `format`, `quality`, `scale`, `max_width`, `max_height`, `max_bytes`) and/or `include_tree: true` on the action call itself to observe the result in the same tool call, or re-check state with `get_app_state`, `focused_window`, or an app-specific readback.
 11. For `set_value`, `verified: true` is the only exact-content confirmation. `verified: false` requires fresh observation; password fields are deliberately unverifiable. The field is omitted for numeric values and early failures.
 
@@ -172,7 +172,7 @@ Chromium can mark a `document web` focused alongside its contenteditable editor.
 The focus probe keeps searching that document and reports the editor; a document
 that holds focus itself still gets a warning, which no longer claims the text
 was lost. An editor can carry the `editable` state without the EditableText
-interface (`supports_editable_text: false`): `type_text` can reach it,
+interface (`supports_editable_text` false or absent): `type_text` can reach it,
 `set_value` cannot. Verify its text or a screenshot before submitting or
 retrying. Embedded browser documents can use up an app-scoped snapshot's node
 budget before the app's own controls; when `accessibility_tree_truncated` is
@@ -212,6 +212,7 @@ directly to a window-relative click.
 
 - Already-running GTK, Qt, and Electron apps may need a restart after AT-SPI is enabled.
 - GNOME may show a portal prompt on the first screenshot or `get_app_state` call with screenshots enabled.
+- Inside a Selkies container, `PIXELFLUX_CU` plus `PIXELFLUX_CU_TOKEN` sends screenshots and input to pixelflux on loopback, with no portal prompt. A closed port falls through. A server error does not, so do not retry that call through another backend.
 - On GNOME Wayland, the remote-control portal asks on each new `computer-use-linux` process. `COMPUTER_USE_LINUX_PERSIST_REMOTE_DESKTOP=1` in the server environment asks the portal to remember the grant (GNOME starts with the remember box checked). Later processes reuse a single-use restore token stored mode 0600 in the user state directory. Unset is the default and prompts every time. The flag does nothing when ydotool, not the portal, is the input backend.
 - Desktop input is stateful. Avoid concurrent tool calls against this MCP server.
 - Pi serializes the native Computer Use tools and keeps one process for the session. If that process exits, do not replay an ambiguous mutating call; obtain a fresh `get_app_state` before another element-based action.