@agent-sh/computer-use-linux 0.7.24 → 0.7.25

package/README.md

@@ -106,7 +106,9 @@ The preference does not substitute an arbitrary action name for a coordinate
 click. Explicit `x`/`y`, right clicks, and double/multiple clicks retain pointer
 semantics. Re-check application state after either kind of activation.
 
-On KDE Wayland, element clicks that need pointer dispatch match the cached AT-SPI frame to a coherent KWin window/output snapshot, then map the target separately to physical capture pixels (uinput) and logical desktop coordinates (portal). Client-local and decorated frames, actual decoration offsets, and nonzero or negative output origins are supported when the geometry and uniform output scale can be verified. Ambiguous windows, mixed or unknown scales, overlapping outputs, or stale pointer geometry fail before input; explicit coordinates keep their existing behavior.
+On KDE Wayland, element clicks that need pointer dispatch match the cached AT-SPI frame to a coherent KWin window/output snapshot, then map the target separately to physical capture pixels (uinput) and logical desktop coordinates (portal). Client-local and decorated frames, actual decoration offsets, and nonzero or negative output origins are supported when the geometry and uniform output scale can be verified. Ambiguous windows, mixed or unknown scales, overlapping outputs, or stale pointer geometry fail before input.
+
+Explicit `x`/`y` defaults to `coordinate_space: "capture"` (raw screenshot pixels). To click a global AT-SPI screen point, pass `coordinate_space: "atspi"` together with its `element_index` or semantic selector; KDE Wayland then applies the same native KWin decoration and scale conversion as an element-bounds click, including when the AT-SPI window frame is client-local. This mode requires both coordinates and element identity and cannot be combined with `relative: true`. Other calls keep their existing coordinate behavior.
 
 For coordinate `click` or `scroll` with `relative: true`, select a target window
 and measure from its clipped screenshot crop origin. Divide preview `x` and

package/npm/README.md

@@ -151,7 +151,13 @@ does not yet forward the flag or include the tool in its catalog.
   avoiding pointer-coordinate conversion when that action is available. Explicit
   `activate`/`jump` requests belong in `perform_action`, not `click`, even when
   bounds are missing. Explicit
-  `x`/`y`, right clicks, and multi-clicks retain pointer semantics. For coordinate
+  `x`/`y`, right clicks, and multi-clicks retain pointer semantics. Explicit
+  coordinates default to raw screenshot pixels (`coordinate_space: "capture"`).
+  On KDE Wayland, `coordinate_space: "atspi"` with `element_index` or a semantic
+  selector converts a global AT-SPI screen point through the element's KWin window,
+  decoration, output origin, and display scale, even with a client-local AT-SPI
+  window frame. It requires both `x`/`y` and element identity
+  and cannot be combined with `relative: true`. For coordinate
   `click` or `scroll` with `relative: true`, use the clipped target-window
   screenshot crop origin and divide preview coordinates by screenshot `scale`.
   Do not use widget-local or raw GDK surface coordinates. A window target is

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@agent-sh/computer-use-linux",
-  "version": "0.7.24",
+  "version": "0.7.25",
   "description": "Native Linux desktop control for AI agents (MCP server, Pi tools, and CLI): semantic AT-SPI accessibility trees, screenshots, compositor window targeting, Wayland/X11 input, and an on-screen activity indicator.",
   "license": "MIT",
   "type": "commonjs",

package/pi/extension/generated-tools.ts

@@ -7,9 +7,9 @@ export interface GeneratedMcpToolDefinition {
 	annotations: Record<string, unknown>;
 }
 
-export const GENERATED_SERVER_VERSION = "0.7.24";
-export const GENERATED_TOOL_CATALOG_HASH = "dee39a43a4d642b61b861f4aa90cdabc9a416befbc10d463a005bcbb892eaf14";
-export const GENERATED_SHELL_TOOL_CATALOG_HASH = "b78f54e50d3a2d615af92ee2a2373599684f02a802b06f1226981f985e34f456";
+export const GENERATED_SERVER_VERSION = "0.7.25";
+export const GENERATED_TOOL_CATALOG_HASH = "ca02e39b8ba12b0f3774af084e67a8401bdd0d3777a82af05da9a7579fa9e605";
+export const GENERATED_SHELL_TOOL_CATALOG_HASH = "b0ae29440000015c1735b1da5744fbaccb505202e49de40b565800130e6dc48d";
 export const GENERATED_MCP_TOOLS =
 [
   {
@@ -187,9 +187,16 @@ export const GENERATED_MCP_TOOLS =
       "openWorldHint": true,
       "readOnlyHint": false
     },
-    "description": "Click an element by index, semantic selector, or desktop coordinate pixels from screenshot metadata. Plain left activation prefers a native AT-SPI click/press/toggle action, avoiding toolkit coordinate scaling. Entry activate and slider jump actions are not substituted for pointer clicks. Explicit coordinates, right clicks, and multi-clicks retain pointer semantics. Pass screenshot_after (true, \"screen\", or \"window\") or include_tree=true to return post-action observation in the same result.",
+    "description": "Click an element by index, semantic selector, or desktop coordinate pixels from screenshot metadata. Plain left activation prefers a native AT-SPI click/press/toggle action, avoiding toolkit coordinate scaling. Entry activate and slider jump actions are not substituted for pointer clicks. Explicit coordinates, right clicks, and multi-clicks retain pointer semantics. Explicit x/y default to raw screenshot capture pixels; coordinate_space=\"atspi\" with element_index or a semantic selector maps a global AT-SPI screen point through the element's KWin window and output geometry on KDE Wayland. Pass screenshot_after (true, \"screen\", or \"window\") or include_tree=true to return post-action observation in the same result.",
     "inputSchema": {
       "$defs": {
+        "ClickCoordinateSpace": {
+          "enum": [
+            "capture",
+            "atspi"
+          ],
+          "type": "string"
+        },
         "ScreenshotOutputFormat": {
           "enum": [
             "png",
@@ -222,6 +229,11 @@ export const GENERATED_MCP_TOOLS =
             "null"
           ]
         },
+        "coordinate_space": {
+          "$ref": "#/$defs/ClickCoordinateSpace",
+          "default": "capture",
+          "description": "Pixel space for explicit x/y: raw screenshot capture pixels (default),\nor global AT-SPI screen coordinates mapped through the element's KWin window."
+        },
         "element_index": {
           "default": null,
           "minimum": 0,

package/skills/computer-use-linux/SKILL.md

@@ -168,7 +168,13 @@ Plain left element/index/selector `click` prefers native AT-SPI `click`,
 `press`, or `toggle` over toolkit bounds, avoiding coordinate
 conversion when available. This preference does not replace a coordinate click
 with an arbitrary action name. Explicit `x`/`y`, right clicks, and double/multiple
-clicks retain pointer semantics.
+clicks retain pointer semantics. Explicit coordinates default to raw screenshot
+pixels (`coordinate_space: "capture"`). On KDE Wayland, pass
+`coordinate_space: "atspi"` with `element_index` or a semantic selector to map
+a global AT-SPI screen point through the element's KWin window, decoration,
+output origin, and display scale, even with a client-local AT-SPI window frame.
+This requires both `x`/`y` and element identity; do not
+combine it with `relative: true` or use it for ordinary screenshot pixels.
 Use `perform_action` explicitly for entry `activate` or slider `jump`; `click`
 never substitutes those actions, including when bounds are unavailable.