@agent-sh/computer-use-linux 0.5.0 → 0.7.0

package/README.md

@@ -48,7 +48,7 @@ MCP tools exposed by the server:
 - `list_apps` — running desktop apps visible to the AT-SPI registry
 - `list_windows` — compositor windows with title, app id, wm_class, focus state, client type (Wayland/X11), and bounds
 - `focused_window` — the window currently holding keyboard focus
-- `get_app_state` — combined screenshot + accessibility tree for a chosen app, with element indices that the input tools accept
+- `get_app_state` — combined screenshot + accessibility tree for a chosen app, with element indices that the input tools accept. Scope it with `app_name_or_bundle_identifier` or a window target; an unscoped call returns the whole desktop tree, reports `tree_scoped: false`, and warns
 - `screenshot` — capture the screen as a bounded PNG or JPEG image; can target a window, which is raised to the front and cropped to just that window
 
 Screenshot payloads are size-bounded by default before they are returned to the MCP host: max 1920 px width/height and 2 MiB image bytes, with hard caps even when callers request more. Agents that need more detail can pass `max_width`, `max_height`, `max_bytes`, `scale`, `format: "jpeg"`, or `quality`, preferably with a window target or crop. PNG remains the default; JPEG lets callers trade lossless pixels for a smaller payload before the byte cap forces further resizing. Returned screenshot metadata includes `coordinate_width`, `coordinate_height`, `scale`, `format`, and `quality` so callers can convert from a downscaled preview to desktop coordinate pixels.
@@ -61,7 +61,24 @@ Screenshot payloads are size-bounded by default before they are returned to the
 - `press_key` — keys / chords; can focus a window or terminal first
 - `type_text` — literal text input, optionally targeted at a window or terminal
 
-Targeted `press_key`/`type_text` results append focused-element feedback from AT-SPI (role, name, editable) and warn when no editable element holds focus. Click/screenshot/input results warn when the target window or coordinate is partially or fully off-screen. `get_app_state` returns a compact readiness block by default; pass `verbose: true` for the full diagnostics report.
+For a plain left `click` by element index or selector, a recognized native
+AT-SPI `click`, `press`, or `toggle` action takes precedence
+over bounds. Entry `activate` and slider `jump` must be requested explicitly
+with `perform_action`; they are never a substitute for a pointer click, even
+when bounds are missing.
+This avoids pointer conversion for GTK3 HiDPI
+extents and GTK4 zero-origin bounds when the element exposes such an action.
+The preference does not substitute an arbitrary action name for a coordinate
+click. Explicit `x`/`y`, right clicks, and double/multiple clicks retain pointer
+semantics. Re-check application state after either kind of activation.
+
+For coordinate `click` or `scroll` with `relative: true`, select a target window
+and measure from its clipped screenshot crop origin. Divide preview `x` and
+`y` by the returned screenshot `scale` before passing them. These are not raw
+GDK surface or widget-local coordinates; decorations and clipping can change
+the origin. A missing window target is rejected.
+
+Targeted `press_key`/`type_text` results append focused-element feedback from AT-SPI (role, name, editable) and warn when no editable element holds focus. Click/screenshot/input results warn when the target window or coordinate is partially or fully off-screen. `get_app_state` returns a compact readiness block by default; pass `verbose: true` for the full diagnostics report. It also reports `tree_scoped` (false when no app target narrowed the AT-SPI tree, with a warning in `message`) and `accessibility_tree_truncated` (true when the node, depth, or read budget stopped traversal with unread elements left).
 
 **Semantic actions**
 
@@ -75,6 +92,8 @@ Targeted `press_key`/`type_text` results append focused-element feedback from AT
 
 **Conditional host execution**
 
+- `complete_interaction` - optional desktop completion notification, registered only with `COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1`. Repeated calls can create repeated notifications; it does not provide desktop exclusivity.
+
 - `run_shell` — same-user `/bin/sh -c` execution without login-profile loading, registered only when the server operator starts the MCP process with `COMPUTER_USE_LINUX_ENABLE_SHELL=1`. It is deliberately absent by default and is not a sandbox.
 
 ### MCP safety contract
@@ -99,6 +118,7 @@ The binary also exposes the same capabilities from the CLI for scripting and deb
 computer-use-linux mcp                                  # stdio MCP server
 computer-use-linux doctor                               # JSON readiness report
 computer-use-linux setup                                # enable AT-SPI
+computer-use-linux guard-accessibility                  # explicit foreground GNOME accessibility guard
 computer-use-linux setup-window-targeting               # install GNOME Shell extension
 computer-use-linux apps
 computer-use-linux state [APP_NAME]
@@ -358,6 +378,7 @@ Most setups need none of these — `doctor` and the installers pick sensible def
 
 | Variable | Effect |
 | --- | --- |
+| `COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE` | Set exactly to `1` to expose the optional `complete_interaction` notification tool. Requires `notify-send` and a desktop notification service; disabled by default. |
 | `COMPUTER_USE_LINUX_COSMIC_HELPER` | Path to the `computer-use-linux-cosmic` helper when it isn't next to the binary or on `PATH`. |
 | `CU_DISABLE_ABS_POINTER` | Disable the uinput absolute pointer and click through `ydotool` instead for setups where the abs-pointer device misbehaves. |
 | `COMPUTER_USE_LINUX_FORCE_PORTAL_POINTER` / `…_KEYBOARD` | Always route pointer / keyboard through the RemoteDesktop portal on Wayland, skipping auto-detection. |
@@ -408,6 +429,44 @@ If you're running this on a shared workstation, set `ydotoold`'s socket permissi
 
 ## Troubleshooting
 
+To receive an explicit completion cue, start the MCP server with
+`COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1`. This exposes `complete_interaction`,
+a parameter-free tool the agent calls once after finishing its desktop work.
+It submits a notification through `notify-send` with a two-second execution
+limit and bounded process cleanup. Missing services, errors, or timeouts return
+`cue: "skipped"`; notification settings may suppress a submitted cue. This does
+not reserve the desktop or prove that other clients have stopped sending input.
+No sound or additional desktop settings are enabled by this option.
+This option currently applies only to directly spawned MCP hosts. The native Pi
+extension does not forward the flag or include this optional tool in its catalog.
+
+`setup` and `setup_accessibility` write and read back GNOME's
+`org.gnome.desktop.interface toolkit-accessibility` setting, even if AT-SPI
+is already enabled at runtime. A runtime-only success is reported as a warning:
+newly launched GTK apps may still have no tree. Enabling the saved key may
+require restarting target apps. Other accessibility tools can change that key
+later; setup does not continuously override user settings.
+
+For an explicit foreground guard while using desktop automation, run:
+
+```bash
+computer-use-linux guard-accessibility
+```
+
+The guard registers a passive AT-SPI window-activation listener and watches
+GNOME's saved `toolkit-accessibility` key. It re-enables the key after a reset
+and verifies it by readback, with periodic checks as well as change monitoring.
+This affects all applications using the current user's GNOME setting, not just
+the target app. It does not start a screen reader or change focus.
+
+`mcp`, `setup`, `setup_accessibility`, and `get_app_state` never start this guard.
+Stop it with Ctrl-C or SIGTERM **before** intentionally disabling accessibility.
+Stopping ends setting writes and removes its listener without disabling other
+accessibility clients or restoring an old saved value. A reset and reassertion
+are not atomic: an app launched in that interval may still need restarting.
+This is an opt-in mitigation, not a guarantee that every GNOME toggle sequence
+preserves application accessibility trees.
+
 `computer-use-linux doctor` is the source of truth. Common failure modes and fixes:
 
 - **`accessibility.at_spi_bus.ok = false`** — AT-SPI registry isn't running or the toolkit bridge is off. Fix: `computer-use-linux setup` (or call the `setup_accessibility` MCP tool). Restart the apps you want to drive.
@@ -419,9 +478,30 @@ If you're running this on a shared workstation, set `ydotoold`'s socket permissi
 - **KWin / Hyprland / i3 / COSMIC / X11 windowing is unavailable** — check `doctor.windowing.backends`. KWin needs session-bus scripting; Hyprland needs `hyprctl`; i3 needs `i3-msg` and its IPC socket; generic X11 needs `wmctrl` and `xprop`. COSMIC needs `computer-use-linux-cosmic`, which the standard installers provide automatically; if you copied binaries by hand, copy the helper too or set `COMPUTER_USE_LINUX_COSMIC_HELPER`.
 - **Screenshots return black frames on multi-monitor setups** — known portal / compositor edge case. Use `get_app_state` with `include_screenshot: false` and rely on AT-SPI until the portal backend is healthy.
 - **`type_text` types into the wrong window** — pass an explicit target (`window_id`, `pid`, `wm_class`, `title`, or for terminals `tty` / `terminal_pid` / `terminal_command` / `terminal_cwd`). Without a target, input goes to whatever window currently has compositor focus.
+- **Wayland pointer actions miss an unfocused window** — injected pointer input is subject to the compositor's input-focus rules. Call `activate_window` for the target before `click`, `drag`, or coordinate `scroll`; a pointer can land at the requested coordinate without the unfocused surface receiving the action.
 
 If `doctor` is green and a specific tool still misbehaves, file an issue with the JSON output of `doctor` and the failing tool's request payload.
 
+For coordinate calibration, launch [the GTK4 probe](examples/coordinate_probe.py)
+in your test desktop and take a targeted screenshot. Choose the center of its
+green 10x10 square from that screenshot, convert by `scale`, and click relative
+to the same target window. The probe's delivered-event `hit: true` is the
+acceptance condition. Do not pass its widget-local `(85, 85)` directly as a
+window-relative click: margins and decorations belong to different spaces.
+
+[The semantic-click regression](scripts/semantic_click_test.py) runs against a
+built binary in an isolated graphical display and checks actual GTK3 button
+activation through MCP at scales 1 and 2. This does not establish correctness
+of Mutter 46 EWMH move/resize, all mixed-output layouts, or GNOME 50.4 pointer
+clicks.
+
+Native Wayland qualification on GNOME 50.1 at 133.3% display scale used the
+same green target for both modes: crop-relative `(122, 145)` and desktop
+`(2140, 193)` both delivered widget coordinates approximately `(84.81, 84.78)`
+with `hit: true` through uinput. These are one capture's measured coordinates,
+not reusable offsets. Raw surface coordinates must still be transformed before
+comparing them with a widget-local target.
+
 ## Related
 
 - [agent-workspace-linux](https://github.com/agent-sh/agent-workspace-linux) — the sibling MCP that gives an agent its **own** isolated Linux desktop (a hidden Xvfb display with its own apps and browser) instead of driving yours. It is the inverse of this project: `computer-use-linux` automates the desktop you are already on; `agent-workspace-linux` sandboxes the agent in a separate one. Use them together.

package/npm/README.md

@@ -9,6 +9,17 @@ mutating and can change real application state. The MCP tool list includes
 `ToolAnnotations` so hosts can distinguish read-only observation from mutating
 desktop actions.
 
+Plain left `click` by element index or selector prefers a native AT-SPI
+`click`, `press`, or `toggle` action over toolkit bounds,
+avoiding pointer-coordinate conversion when that action is available. Explicit
+`activate`/`jump` requests belong in `perform_action`, not `click`, even when
+bounds are missing. Explicit
+`x`/`y`, right clicks, and multi-clicks retain pointer semantics. For coordinate
+`click` or `scroll` with `relative: true`, use the clipped target-window
+screenshot crop origin and divide preview coordinates by screenshot `scale`.
+Do not use widget-local or raw GDK surface coordinates. A window target is
+required.
+
 ```bash
 npm install -g @agent-sh/computer-use-linux
 computer-use-linux doctor
@@ -20,6 +31,28 @@ hermes mcp test computer-use-linux
 hermes mcp configure computer-use-linux
 ```
 
+For an optional MCP completion notification, set
+`COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1` in the server environment and have the
+agent call `complete_interaction` after its desktop work. `notify-send` must be
+installed with an available desktop notification service. The cue is best effort
+and does not provide exclusive desktop ownership.
+This cue is available only to directly spawned MCP hosts; the native Pi extension
+does not yet forward the flag or include the tool in its catalog.
+
+If accessibility is disabled, run `computer-use-linux setup`. Setup writes and
+reads back GNOME's `toolkit-accessibility` setting and warns if only runtime
+accessibility is available. Restart target apps if their trees remain empty.
+
+To explicitly keep that saved setting enabled during desktop automation, run
+`computer-use-linux guard-accessibility` in a foreground terminal. It registers
+a passive AT-SPI window-activation listener and reasserts the current user's
+GNOME `toolkit-accessibility` key after resets, verifying each write by readback.
+This affects other apps using the same user setting. MCP, setup, and
+`get_app_state` never start it automatically. Stop with Ctrl-C or SIGTERM before
+disabling accessibility; stopping removes its listener and ends writes without
+disabling other clients or restoring an old value. An app launched during a
+reset may still need restarting, so this is not a complete GNOME toggle fix.
+
 The generated Hermes config should look like this:
 
 ```yaml

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@agent-sh/computer-use-linux",
-  "version": "0.5.0",
+  "version": "0.7.0",
   "description": "Linux desktop-control MCP server: AT-SPI accessibility trees, Wayland/X11 input, screenshots, and compositor window targeting.",
   "license": "MIT",
   "type": "commonjs",

package/pi/extension/generated-tools.ts

@@ -7,9 +7,9 @@ export interface GeneratedMcpToolDefinition {
 	annotations: Record<string, unknown>;
 }
 
-export const GENERATED_SERVER_VERSION = "0.5.0";
-export const GENERATED_TOOL_CATALOG_HASH = "3dd52d53c100e240fe2789dd1818ef80fd30202331c0aca2718212e2a79e47ff";
-export const GENERATED_SHELL_TOOL_CATALOG_HASH = "13406c87ad78dbdea7e5480e3e482be14c97687fa92320fde8e16f2efbbd3339";
+export const GENERATED_SERVER_VERSION = "0.7.0";
+export const GENERATED_TOOL_CATALOG_HASH = "c0c1e9d0e200637ff286baec47b6ee8a6798c7e17f29a9cb71a62a8c292a47ed";
+export const GENERATED_SHELL_TOOL_CATALOG_HASH = "f2ebf4164a8a92893933b674bf761430251dce068d7944ce363c99eb88d316ff";
 export const GENERATED_MCP_TOOLS =
 [
   {
@@ -102,7 +102,7 @@ export const GENERATED_MCP_TOOLS =
       "openWorldHint": true,
       "readOnlyHint": false
     },
-    "description": "Click an element by index, semantic selector, or desktop coordinate pixels from screenshot metadata.",
+    "description": "Click an element by index, semantic selector, or desktop coordinate pixels from screenshot metadata. Plain left activation prefers a native AT-SPI click/press/toggle action, avoiding toolkit coordinate scaling. Entry activate and slider jump actions are not substituted for pointer clicks. Explicit coordinates, right clicks, and multi-clicks retain pointer semantics.",
     "inputSchema": {
       "$schema": "https://json-schema.org/draft/2020-12/schema",
       "properties": {
@@ -153,7 +153,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "relative": {
           "default": null,
-          "description": "Interpret `x`/`y` as relative to the targeted window's top-left corner\n(the same coordinate space as a window-cropped `screenshot`). Requires a\nwindow target; ignored otherwise.",
+          "description": "Interpret `x`/`y` from the clipped window screenshot crop origin, in\ncoordinate pixels before preview resizing. Divide preview pixels by its\nscale first. This is not a toolkit widget or raw GDK surface origin.\nRequires a window target; missing targets are rejected.",
           "type": [
             "boolean",
             "null"
@@ -298,7 +298,7 @@ export const GENERATED_MCP_TOOLS =
       "openWorldHint": true,
       "readOnlyHint": true
     },
-    "description": "Start an app use session if needed, then get a size-bounded screenshot and accessibility state for a Linux app. Screenshot results include coordinate_width, coordinate_height, scale, format, and quality when the returned image is downscaled or compressed; callers can request jpeg/quality for compression before resizing.",
+    "description": "Start an app use session if needed, then get a size-bounded screenshot and accessibility state for a Linux app. Scope the accessibility tree with app_name_or_bundle_identifier or a window_id/pid/app_id/wm_class/title target; omitting a target returns the whole desktop tree and can flood context. Screenshot results include coordinate_width, coordinate_height, scale, format, and quality when the returned image is downscaled or compressed; callers can request jpeg/quality for compression before resizing.",
     "inputSchema": {
       "$defs": {
         "ScreenshotOutputFormat": {
@@ -313,6 +313,7 @@ export const GENERATED_MCP_TOOLS =
       "properties": {
         "app_id": {
           "default": null,
+          "description": "Application id. Also scopes the accessibility tree when it matches an\nAT-SPI root.",
           "type": [
             "string",
             "null"
@@ -320,6 +321,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "app_name_or_bundle_identifier": {
           "default": null,
+          "description": "App name or AT-SPI id that limits the accessibility tree. Omit only when\nyou need the whole desktop tree; unscoped results can flood context.",
           "type": [
             "string",
             "null"
@@ -339,6 +341,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "include_screenshot": {
           "default": null,
+          "description": "Include a size-bounded screenshot (default true). Set false when the\naccessibility tree is enough.",
           "type": [
             "boolean",
             "null"
@@ -391,6 +394,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "pid": {
           "default": null,
+          "description": "Process id. Also scopes the accessibility tree to that process when it\nexposes AT-SPI.",
           "minimum": 0,
           "type": [
             "integer",
@@ -418,6 +422,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "terminal_command": {
           "default": null,
+          "description": "Terminal command substring. Resolves a window target and scopes the tree\nwhen possible.",
           "type": [
             "string",
             "null"
@@ -425,6 +430,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "terminal_cwd": {
           "default": null,
+          "description": "Terminal working directory. Resolves a window target and scopes the tree\nwhen possible.",
           "type": [
             "string",
             "null"
@@ -432,6 +438,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "terminal_pid": {
           "default": null,
+          "description": "Terminal emulator pid. Resolves a window target and scopes the tree when\npossible.",
           "minimum": 0,
           "type": [
             "integer",
@@ -440,6 +447,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "title": {
           "default": null,
+          "description": "Window title substring. Also scopes the accessibility tree when it\nmatches an AT-SPI root.",
           "type": [
             "string",
             "null"
@@ -447,6 +455,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "tty": {
           "default": null,
+          "description": "Terminal tty device (for example /dev/pts/3). Resolves a window target\nand scopes the tree when possible.",
           "type": [
             "string",
             "null"
@@ -462,6 +471,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "window_id": {
           "default": null,
+          "description": "Compositor window id. Also scopes the accessibility tree to that window's\napplication when possible.",
           "minimum": 0,
           "type": [
             "integer",
@@ -470,6 +480,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "wm_class": {
           "default": null,
+          "description": "Window manager class. Also scopes the accessibility tree when it matches\nan AT-SPI root.",
           "type": [
             "string",
             "null"
@@ -1043,7 +1054,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "relative": {
           "default": null,
-          "description": "Interpret `x`/`y` as relative to the targeted window's top-left corner\n(the same coordinate space as a window-cropped `screenshot`). Requires a\nwindow target; ignored otherwise.",
+          "description": "Interpret `x`/`y` from the clipped window screenshot crop origin, in\ncoordinate pixels before preview resizing. Divide preview pixels by its\nscale first. This is not a toolkit widget or raw GDK surface origin.\nRequires a window target; missing targets are rejected.",
           "type": [
             "boolean",
             "null"

package/skills/computer-use-linux/SKILL.md

@@ -24,13 +24,23 @@ Do not use this for remote browsers, websites, or headless automation when a bro
 
 ## Install
 
-Pi users need only the package:
+Pick the install that matches how you will run this skill. You can use both.
+
+### Pi native tools
 
 ```bash
 pi install npm:@agent-sh/computer-use-linux
 ```
 
-Preferred install:
+This enables Pi's `computer_use_linux_*` tools. It does not put
+`computer-use-linux` on `PATH`. Shell commands in this skill (`doctor`,
+`setup`, `setup-window-targeting`, `guard-accessibility`, the MCP `command`
+config, and Verification) need the CLI install below.
+
+### Shell CLI / MCP server
+
+Use this when you need `computer-use-linux` on `PATH` for the commands in this
+skill.
 
 ```bash
 npm install -g @agent-sh/computer-use-linux
@@ -56,6 +66,31 @@ If `doctor` selects ydotool as the input backend, also enable its per-user daemo
 
 On GNOME Wayland, log out and back in after `setup-window-targeting` if the GNOME Shell extension was newly installed.
 
+For MCP hosts with `COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1`, call the optional
+`complete_interaction` tool once after finishing desktop interaction. A skipped
+cue is not a task failure. This notification does not guarantee exclusive
+desktop ownership or that other clients have stopped sending input.
+This applies only to directly spawned MCP hosts, not the native Pi extension.
+
+`setup_accessibility` verifies the saved GNOME `toolkit-accessibility` key
+separately from runtime AT-SPI. Inspect its warning and readback before assuming
+new apps can expose trees. Other accessibility tools may change the key later;
+setup does not hold it enabled continuously.
+
+### Optional foreground accessibility guard
+
+Skip unless: the user explicitly wants GNOME's saved `toolkit-accessibility`
+setting kept enabled while desktop automation runs.
+
+Run `computer-use-linux guard-accessibility` in a foreground terminal. It
+registers a passive AT-SPI window-activation listener and watches/reasserts the
+saved key with readback. The setting affects all apps for the current user.
+`mcp`, setup, and `get_app_state` never start this guard automatically.
+Stop with Ctrl-C or SIGTERM before intentionally disabling accessibility.
+Stopping ends writes and removes its listener without disabling other clients
+or restoring a previous saved value. Apps launched during a reset/reassertion
+race may still need restarting; do not claim a complete GNOME toggle fix.
+
 ## Configure Your Agent
 
 The `computer-use-linux` binary is an MCP server. Configure it as a stdio MCP server in your agent of choice:
@@ -68,6 +103,7 @@ The `computer-use-linux` binary is an MCP server. Configure it as a stdio MCP se
 ```
 
 If the binary is not on `PATH`, use the absolute path (typically `~/.local/bin/computer-use-linux` or the npm global bin directory).
+Pi native tools skip this MCP `command` config; see [Pi setup](references/pi-setup.md).
 
 ### Host-specific guides
 
@@ -77,7 +113,7 @@ If the binary is not on `PATH`, use the absolute path (typically `~/.local/bin/c
 ## Procedure
 
 1. In Pi, call `computer_use_linux_tools` with the exact tools or capability you need. Enabled tools use the `computer_use_linux_<name>` prefix, appear starting on the next model turn, and remain active for the session.
-2. Begin every desktop-control turn with `get_app_state`; use `include_screenshot: false` when the accessibility tree is sufficient. Its compact readiness block identifies missing setup.
+2. Begin every desktop-control turn with `get_app_state`, scoped to the app you are working in: pass `app_name_or_bundle_identifier` or a window target (`window_id`, `pid`, `app_id`, `wm_class`, `title`). Without a target the result is the whole desktop AT-SPI tree, `tree_scoped` is `false`, and `message` warns; that can flood context. Use `include_screenshot: false` when the accessibility tree is sufficient. If `accessibility_tree_truncated` is `true`, the tree is incomplete: scope to a narrower target and raise `max_nodes` or `max_depth` (hard caps 2000 and 64) rather than lowering them. The compact readiness block identifies missing setup.
 3. Use `doctor` only when you need the full diagnostic report.
 4. If `can_build_accessibility_tree` is false, run `setup_accessibility` and restart the target app.
 5. If `can_query_windows` is false on GNOME Wayland, run `setup_window_targeting` and ask the user to log out and back in if setup says the shell extension needs a reload.
@@ -87,6 +123,26 @@ If the binary is not on `PATH`, use the absolute path (typically `~/.local/bin/c
 9. For text input, prefer `type_text` with a target selector (`window_id`, `pid`, `app_id`, `wm_class`, `title`, `tty`, `terminal_pid`, `terminal_command`, or `terminal_cwd`) rather than relying on current focus.
 10. After mutating actions, re-check state with `get_app_state`, `focused_window`, or an app-specific readback.
 
+Plain left element/index/selector `click` prefers native AT-SPI `click`,
+`press`, or `toggle` over toolkit bounds, avoiding coordinate
+conversion when available. This preference does not replace a coordinate click
+with an arbitrary action name. Explicit `x`/`y`, right clicks, and double/multiple
+clicks retain pointer semantics.
+Use `perform_action` explicitly for entry `activate` or slider `jump`; `click`
+never substitutes those actions, including when bounds are unavailable.
+
+### Screenshot-relative coordinates
+
+Skip unless: a coordinate `click` or `scroll` uses `relative: true`.
+
+Select a target window and use its clipped screenshot crop origin. Divide
+preview `x`/`y` by screenshot `scale` first. Widget-local and raw GDK surface
+coordinates are not interchangeable with that origin; missing window targets
+are rejected. For calibration, use the repository's
+`examples/coordinate_probe.py`: select the green square from the screenshot
+and require a delivered-event `hit: true`. Do not pass widget-local `(85, 85)`
+directly to a window-relative click.
+
 ## Pitfalls
 
 - Already-running GTK, Qt, and Electron apps may need a restart after AT-SPI is enabled.
@@ -100,6 +156,10 @@ If the binary is not on `PATH`, use the absolute path (typically `~/.local/bin/c
 
 ## Verification
 
+Pi-only installs: enable and call `computer_use_linux_doctor` as in
+[Pi setup](references/pi-setup.md). Shell `computer-use-linux doctor` needs
+the CLI on `PATH`.
+
 Run:
 
 ```bash

package/skills/computer-use-linux/references/pi-setup.md

@@ -14,7 +14,10 @@ pi install npm:@agent-sh/computer-use-linux
 ```
 
 Restart Pi or run `/reload`. No separate MCP adapter or MCP configuration is
-required.
+required. This does not put `computer-use-linux` on `PATH`. For shell `doctor`,
+`setup`, or MCP hosts, also install the CLI with
+`npm install -g @agent-sh/computer-use-linux` or
+`cargo install computer-use-linux`.
 
 The integration is designed for current Pi releases with additive dynamic tool
 loading. Update Pi and installed packages when needed:
@@ -59,8 +62,11 @@ session so `get_app_state` element indices and portal sessions remain valid.
 ## Safe operating loop
 
 1. Enable `get_app_state`, `list_windows`, and any likely action tools.
-2. Call `computer_use_linux_get_app_state`, using
-   `include_screenshot: false` when accessibility data is enough.
+2. Call `computer_use_linux_get_app_state` scoped to the target app
+   (`app_name_or_bundle_identifier`, or `window_id`/`pid`/`app_id`/`wm_class`/
+   `title`), using `include_screenshot: false` when accessibility data is
+   enough. An unscoped call returns the whole desktop tree, reports
+   `tree_scoped: false`, and warns; that can exhaust a small context window.
 3. Inspect the returned readiness block; enable/call `doctor` only for full
    diagnostics.
 4. Identify the target with `computer_use_linux_list_windows` or