@agent-sh/computer-use-linux 0.6.0 → 0.7.0

package/README.md

@@ -48,7 +48,7 @@ MCP tools exposed by the server:
 - `list_apps` — running desktop apps visible to the AT-SPI registry
 - `list_windows` — compositor windows with title, app id, wm_class, focus state, client type (Wayland/X11), and bounds
 - `focused_window` — the window currently holding keyboard focus
-- `get_app_state` — combined screenshot + accessibility tree for a chosen app, with element indices that the input tools accept
+- `get_app_state` — combined screenshot + accessibility tree for a chosen app, with element indices that the input tools accept. Scope it with `app_name_or_bundle_identifier` or a window target; an unscoped call returns the whole desktop tree, reports `tree_scoped: false`, and warns
 - `screenshot` — capture the screen as a bounded PNG or JPEG image; can target a window, which is raised to the front and cropped to just that window
 
 Screenshot payloads are size-bounded by default before they are returned to the MCP host: max 1920 px width/height and 2 MiB image bytes, with hard caps even when callers request more. Agents that need more detail can pass `max_width`, `max_height`, `max_bytes`, `scale`, `format: "jpeg"`, or `quality`, preferably with a window target or crop. PNG remains the default; JPEG lets callers trade lossless pixels for a smaller payload before the byte cap forces further resizing. Returned screenshot metadata includes `coordinate_width`, `coordinate_height`, `scale`, `format`, and `quality` so callers can convert from a downscaled preview to desktop coordinate pixels.
@@ -78,7 +78,7 @@ and measure from its clipped screenshot crop origin. Divide preview `x` and
 GDK surface or widget-local coordinates; decorations and clipping can change
 the origin. A missing window target is rejected.
 
-Targeted `press_key`/`type_text` results append focused-element feedback from AT-SPI (role, name, editable) and warn when no editable element holds focus. Click/screenshot/input results warn when the target window or coordinate is partially or fully off-screen. `get_app_state` returns a compact readiness block by default; pass `verbose: true` for the full diagnostics report.
+Targeted `press_key`/`type_text` results append focused-element feedback from AT-SPI (role, name, editable) and warn when no editable element holds focus. Click/screenshot/input results warn when the target window or coordinate is partially or fully off-screen. `get_app_state` returns a compact readiness block by default; pass `verbose: true` for the full diagnostics report. It also reports `tree_scoped` (false when no app target narrowed the AT-SPI tree, with a warning in `message`) and `accessibility_tree_truncated` (true when the node, depth, or read budget stopped traversal with unread elements left).
 
 **Semantic actions**
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@agent-sh/computer-use-linux",
-  "version": "0.6.0",
+  "version": "0.7.0",
   "description": "Linux desktop-control MCP server: AT-SPI accessibility trees, Wayland/X11 input, screenshots, and compositor window targeting.",
   "license": "MIT",
   "type": "commonjs",

package/pi/extension/generated-tools.ts

@@ -7,9 +7,9 @@ export interface GeneratedMcpToolDefinition {
 	annotations: Record<string, unknown>;
 }
 
-export const GENERATED_SERVER_VERSION = "0.6.0";
-export const GENERATED_TOOL_CATALOG_HASH = "9107297df765ce3b907540db172cc7823b7664586255db1064a40a44483d6fd6";
-export const GENERATED_SHELL_TOOL_CATALOG_HASH = "e0bc41db617ea6d10d812cbddce6ef78036e44251360f5fac9c207dd0d5ce392";
+export const GENERATED_SERVER_VERSION = "0.7.0";
+export const GENERATED_TOOL_CATALOG_HASH = "c0c1e9d0e200637ff286baec47b6ee8a6798c7e17f29a9cb71a62a8c292a47ed";
+export const GENERATED_SHELL_TOOL_CATALOG_HASH = "f2ebf4164a8a92893933b674bf761430251dce068d7944ce363c99eb88d316ff";
 export const GENERATED_MCP_TOOLS =
 [
   {
@@ -298,7 +298,7 @@ export const GENERATED_MCP_TOOLS =
       "openWorldHint": true,
       "readOnlyHint": true
     },
-    "description": "Start an app use session if needed, then get a size-bounded screenshot and accessibility state for a Linux app. Screenshot results include coordinate_width, coordinate_height, scale, format, and quality when the returned image is downscaled or compressed; callers can request jpeg/quality for compression before resizing.",
+    "description": "Start an app use session if needed, then get a size-bounded screenshot and accessibility state for a Linux app. Scope the accessibility tree with app_name_or_bundle_identifier or a window_id/pid/app_id/wm_class/title target; omitting a target returns the whole desktop tree and can flood context. Screenshot results include coordinate_width, coordinate_height, scale, format, and quality when the returned image is downscaled or compressed; callers can request jpeg/quality for compression before resizing.",
     "inputSchema": {
       "$defs": {
         "ScreenshotOutputFormat": {
@@ -313,6 +313,7 @@ export const GENERATED_MCP_TOOLS =
       "properties": {
         "app_id": {
           "default": null,
+          "description": "Application id. Also scopes the accessibility tree when it matches an\nAT-SPI root.",
           "type": [
             "string",
             "null"
@@ -320,6 +321,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "app_name_or_bundle_identifier": {
           "default": null,
+          "description": "App name or AT-SPI id that limits the accessibility tree. Omit only when\nyou need the whole desktop tree; unscoped results can flood context.",
           "type": [
             "string",
             "null"
@@ -339,6 +341,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "include_screenshot": {
           "default": null,
+          "description": "Include a size-bounded screenshot (default true). Set false when the\naccessibility tree is enough.",
           "type": [
             "boolean",
             "null"
@@ -391,6 +394,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "pid": {
           "default": null,
+          "description": "Process id. Also scopes the accessibility tree to that process when it\nexposes AT-SPI.",
           "minimum": 0,
           "type": [
             "integer",
@@ -418,6 +422,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "terminal_command": {
           "default": null,
+          "description": "Terminal command substring. Resolves a window target and scopes the tree\nwhen possible.",
           "type": [
             "string",
             "null"
@@ -425,6 +430,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "terminal_cwd": {
           "default": null,
+          "description": "Terminal working directory. Resolves a window target and scopes the tree\nwhen possible.",
           "type": [
             "string",
             "null"
@@ -432,6 +438,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "terminal_pid": {
           "default": null,
+          "description": "Terminal emulator pid. Resolves a window target and scopes the tree when\npossible.",
           "minimum": 0,
           "type": [
             "integer",
@@ -440,6 +447,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "title": {
           "default": null,
+          "description": "Window title substring. Also scopes the accessibility tree when it\nmatches an AT-SPI root.",
           "type": [
             "string",
             "null"
@@ -447,6 +455,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "tty": {
           "default": null,
+          "description": "Terminal tty device (for example /dev/pts/3). Resolves a window target\nand scopes the tree when possible.",
           "type": [
             "string",
             "null"
@@ -462,6 +471,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "window_id": {
           "default": null,
+          "description": "Compositor window id. Also scopes the accessibility tree to that window's\napplication when possible.",
           "minimum": 0,
           "type": [
             "integer",
@@ -470,6 +480,7 @@ export const GENERATED_MCP_TOOLS =
         },
         "wm_class": {
           "default": null,
+          "description": "Window manager class. Also scopes the accessibility tree when it matches\nan AT-SPI root.",
           "type": [
             "string",
             "null"

package/skills/computer-use-linux/SKILL.md

@@ -24,13 +24,23 @@ Do not use this for remote browsers, websites, or headless automation when a bro
 
 ## Install
 
-Pi users need only the package:
+Pick the install that matches how you will run this skill. You can use both.
+
+### Pi native tools
 
 ```bash
 pi install npm:@agent-sh/computer-use-linux
 ```
 
-Preferred install:
+This enables Pi's `computer_use_linux_*` tools. It does not put
+`computer-use-linux` on `PATH`. Shell commands in this skill (`doctor`,
+`setup`, `setup-window-targeting`, `guard-accessibility`, the MCP `command`
+config, and Verification) need the CLI install below.
+
+### Shell CLI / MCP server
+
+Use this when you need `computer-use-linux` on `PATH` for the commands in this
+skill.
 
 ```bash
 npm install -g @agent-sh/computer-use-linux
@@ -93,6 +103,7 @@ The `computer-use-linux` binary is an MCP server. Configure it as a stdio MCP se
 ```
 
 If the binary is not on `PATH`, use the absolute path (typically `~/.local/bin/computer-use-linux` or the npm global bin directory).
+Pi native tools skip this MCP `command` config; see [Pi setup](references/pi-setup.md).
 
 ### Host-specific guides
 
@@ -102,7 +113,7 @@ If the binary is not on `PATH`, use the absolute path (typically `~/.local/bin/c
 ## Procedure
 
 1. In Pi, call `computer_use_linux_tools` with the exact tools or capability you need. Enabled tools use the `computer_use_linux_<name>` prefix, appear starting on the next model turn, and remain active for the session.
-2. Begin every desktop-control turn with `get_app_state`; use `include_screenshot: false` when the accessibility tree is sufficient. Its compact readiness block identifies missing setup.
+2. Begin every desktop-control turn with `get_app_state`, scoped to the app you are working in: pass `app_name_or_bundle_identifier` or a window target (`window_id`, `pid`, `app_id`, `wm_class`, `title`). Without a target the result is the whole desktop AT-SPI tree, `tree_scoped` is `false`, and `message` warns; that can flood context. Use `include_screenshot: false` when the accessibility tree is sufficient. If `accessibility_tree_truncated` is `true`, the tree is incomplete: scope to a narrower target and raise `max_nodes` or `max_depth` (hard caps 2000 and 64) rather than lowering them. The compact readiness block identifies missing setup.
 3. Use `doctor` only when you need the full diagnostic report.
 4. If `can_build_accessibility_tree` is false, run `setup_accessibility` and restart the target app.
 5. If `can_query_windows` is false on GNOME Wayland, run `setup_window_targeting` and ask the user to log out and back in if setup says the shell extension needs a reload.
@@ -145,6 +156,10 @@ directly to a window-relative click.
 
 ## Verification
 
+Pi-only installs: enable and call `computer_use_linux_doctor` as in
+[Pi setup](references/pi-setup.md). Shell `computer-use-linux doctor` needs
+the CLI on `PATH`.
+
 Run:
 
 ```bash

package/skills/computer-use-linux/references/pi-setup.md

@@ -14,7 +14,10 @@ pi install npm:@agent-sh/computer-use-linux
 ```
 
 Restart Pi or run `/reload`. No separate MCP adapter or MCP configuration is
-required.
+required. This does not put `computer-use-linux` on `PATH`. For shell `doctor`,
+`setup`, or MCP hosts, also install the CLI with
+`npm install -g @agent-sh/computer-use-linux` or
+`cargo install computer-use-linux`.
 
 The integration is designed for current Pi releases with additive dynamic tool
 loading. Update Pi and installed packages when needed:
@@ -59,8 +62,11 @@ session so `get_app_state` element indices and portal sessions remain valid.
 ## Safe operating loop
 
 1. Enable `get_app_state`, `list_windows`, and any likely action tools.
-2. Call `computer_use_linux_get_app_state`, using
-   `include_screenshot: false` when accessibility data is enough.
+2. Call `computer_use_linux_get_app_state` scoped to the target app
+   (`app_name_or_bundle_identifier`, or `window_id`/`pid`/`app_id`/`wm_class`/
+   `title`), using `include_screenshot: false` when accessibility data is
+   enough. An unscoped call returns the whole desktop tree, reports
+   `tree_scoped: false`, and warns; that can exhaust a small context window.
 3. Inspect the returned readiness block; enable/call `doctor` only for full
    diagnostics.
 4. Identify the target with `computer_use_linux_list_windows` or