@agent-sh/computer-use-linux 0.7.36 → 0.7.37
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -9,8 +9,6 @@
|
|
|
9
9
|
</p>
|
|
10
10
|
</div>
|
|
11
11
|
|
|
12
|
-
> Running open models for your company? [**Tiyuvta**](https://tiyuvta.ai/services/) helps with model choice, deployment and optimization, and fine-tuning on your hardware or cloud account.
|
|
13
|
-
|
|
14
12
|
<p align="center">
|
|
15
13
|
<a href="https://github.com/sponsors/avifenesh"><img src="https://img.shields.io/badge/Sponsor-%E2%9D%A4-db61a2?style=for-the-badge&logo=githubsponsors&logoColor=white" alt="Sponsor"></a>
|
|
16
14
|
</p>
|
|
@@ -85,7 +83,7 @@ The crate was extracted from [`codex-desktop-linux`](https://github.com/avifenes
|
|
|
85
83
|
| **Opt-in host tools** | `complete_interaction` | Parameter-free desktop completion notification (`notify-send`); registered only when `COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1`. |
|
|
86
84
|
| | `run_shell` | Bounded non-login `/bin/sh -c` execution; registered only when `COMPUTER_USE_LINUX_ENABLE_SHELL=1` (absent by default). |
|
|
87
85
|
|
|
88
|
-
`click`, `scroll`, `perform_action`, and `set_value` take an optional app target (`pid`, `window_id`, `app_id`, `wm_class`, `window_title`). With one, an `element_index` or selector from another app's `get_app_state` snapshot is refused; call `get_app_state` for the target app first. When the window backend reports no pid for a target window (i3
|
|
86
|
+
`click`, `scroll`, `perform_action`, and `set_value` take an optional app target (`pid`, `window_id`, `app_id`, `wm_class`, `window_title`). With one, an `element_index` or selector from another app's `get_app_state` snapshot is refused; call `get_app_state` for the target app first. When the window backend reports no pid for a target window (i3 without `xprop`, COSMIC), the snapshot must have been taken for that window, or the call is refused. `perform_action` and `set_value` do not focus the target window. An `element_identifier` is refused when its owner on the accessibility bus is a different pid than the target; it is not checked when either pid is unknown.
|
|
89
87
|
|
|
90
88
|
### Post-action observation in one round trip
|
|
91
89
|
|
|
@@ -187,7 +185,7 @@ Validated manually on Ubuntu 25.10 (GNOME Shell 50.1, Wayland). niri window list
|
|
|
187
185
|
| KDE Plasma / KWin | temporary KWin DBus scripting | Lists and focuses windows through Plasma 5 or 6 `org.kde.KWin` scripting APIs when the session bus exposes them. IDs are derived from the KWin window UUID, stay the same for a window across calls, and fit JavaScript integers. |
|
|
188
186
|
| Hyprland | `hyprctl clients -j` and `hyprctl dispatch focuswindow` | Requires `hyprctl` in the desktop session. With `COMPUTER_USE_LINUX_HYPRLAND_BACKGROUND=1`, targeted keyboard input uses `send_key_state` and targeted capture uses `grim -T`, so neither focuses the window. |
|
|
189
187
|
| niri | `niri msg` with a direct JSON IPC fallback | Lists toplevels and focuses exact window IDs. Export the session's `NIRI_SOCKET`; otherwise discovery requires an unambiguous socket matching `WAYLAND_DISPLAY`. The fallback works without the `niri` binary. Missing positions remain `null`; bounds are omitted if output scaling is unknown or mixed. |
|
|
190
|
-
| i3 | `i3-msg`; optional `xprop` for PID hydration | Lists and focuses
|
|
188
|
+
| i3, Sway | `i3-msg`; on i3, optional `xprop` for PID hydration | Lists and focuses windows over the active i3 or Sway IPC socket. On Sway this covers native Wayland and Xwayland windows, with pids from the tree. X11 windows keep their X11 id; a native Wayland window on Sway gets 2^32 plus its container id, so the two never collide. |
|
|
191
189
|
| labwc with Selkies IPC | `$XDG_RUNTIME_DIR/labwc.sock` | Requires the Selkies IPC patch; stock labwc has no such socket. Lists windows and focuses unique app-id/title matches through `wlr-foreign-toplevel-management`. IDs include pid, app id and title, fit JavaScript integers, and change when the title changes. Ambiguous windows are rejected. Bounds are omitted because the IPC does not expose output scaling. |
|
|
192
190
|
| COSMIC Wayland | `computer-use-linux-cosmic` helper | Installed automatically by `./install.sh`, `cargo install`, and npm. For custom/manual layouts, put the helper next to the main binary, on `PATH`, or point `COMPUTER_USE_LINUX_COSMIC_HELPER` at it. The on-screen indicator was validated here (three outputs, mixed fractional scaling). |
|
|
193
191
|
| Other wlroots compositors (stock labwc, phoc, river-classic) | `zwlr_foreign_toplevel_manager_v1` on `WAYLAND_DISPLAY` | Lists toplevels with title, app id, and activated/minimized state, and focuses unique app-id/title matches. The protocol reports no pids or bounds. IDs derive from the app id and title, fit JavaScript integers, and change when the title changes; windows sharing both are rejected for exact focus. On Sway it answers only when the i3 backend does not: when `i3-msg` is missing, cannot reach the IPC socket, or lists no windows. Compositors that do not advertise the protocol, such as Cage, are not supported. |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agent-sh/computer-use-linux",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.37",
|
|
4
4
|
"description": "Native Linux desktop control for AI agents (MCP server, Pi tools, and CLI): semantic AT-SPI accessibility trees, screenshots, compositor window targeting, Wayland/X11 input, and an on-screen activity indicator.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "commonjs",
|
|
@@ -7,7 +7,7 @@ export interface GeneratedMcpToolDefinition {
|
|
|
7
7
|
annotations: Record<string, unknown>;
|
|
8
8
|
}
|
|
9
9
|
|
|
10
|
-
export const GENERATED_SERVER_VERSION = "0.7.
|
|
10
|
+
export const GENERATED_SERVER_VERSION = "0.7.37";
|
|
11
11
|
export const GENERATED_TOOL_CATALOG_HASH = "626768053c7e97574226d8d8b382ced55272e50273dc6fe72a81d130b20fdb7a";
|
|
12
12
|
export const GENERATED_SHELL_TOOL_CATALOG_HASH = "628cbd7dc623cfcf7ff2dc18c69960f4c964cf1566045d796e89a76ba7bba9e2";
|
|
13
13
|
export const GENERATED_MCP_TOOLS =
|
|
@@ -144,7 +144,7 @@ Pi native tools skip this MCP `command` config; see [Pi setup](references/pi-set
|
|
|
144
144
|
4. If `can_build_accessibility_tree` is false, run `setup_accessibility` and restart the target app.
|
|
145
145
|
5. If `can_query_windows` is false on GNOME Wayland, run `setup_window_targeting` and ask the user to log out and back in if setup says the shell extension needs a reload.
|
|
146
146
|
6. Before targeted input, call `list_windows` or `focused_window` and verify the intended window by title, app id, pid, or wm class.
|
|
147
|
-
7. Prefer semantic targeting from `get_app_state`: use element indices or role/name/text/states selectors. Automatic recovery of a dead semantic target requires verified PID/window ownership and invalidates cached indices; call `get_app_state` again before further index-based actions. When several apps are in play, pass the app target (`pid`, `window_id`, `app_id`, `wm_class`, `window_title`) to `click`, `scroll`, `perform_action`, and `set_value`: an index or selector from another app's snapshot is then refused instead of acting on that app. Where the window backend reports no pid (i3
|
|
147
|
+
7. Prefer semantic targeting from `get_app_state`: use element indices or role/name/text/states selectors. Automatic recovery of a dead semantic target requires verified PID/window ownership and invalidates cached indices; call `get_app_state` again before further index-based actions. When several apps are in play, pass the app target (`pid`, `window_id`, `app_id`, `wm_class`, `window_title`) to `click`, `scroll`, `perform_action`, and `set_value`: an index or selector from another app's snapshot is then refused instead of acting on that app. Where the window backend reports no pid (i3 without `xprop`, COSMIC), call `get_app_state` for the same window first or pass `pid`, or the call is refused.
|
|
148
148
|
8. Use coordinates only when the UI surface has no useful accessibility tree.
|
|
149
149
|
9. For text input, prefer `type_text` with a target selector (`window_id`, `pid`, `app_id`, `wm_class`, `title`, `tty`, `terminal_pid`, `terminal_command`, or `terminal_cwd`) rather than relying on current focus. A `window_id` is valid only as returned by a recent `list_windows`; if a call rejects it as outside 0 to 2^53-1 or matches no window, re-run `list_windows`.
|
|
150
150
|
10. After mutating actions (`click`, `drag`, `scroll`, `press_key`, `type_text`, `perform_action`, `set_value`, `activate_window`, `move_window`, `resize_window`), either pass `screenshot_after: true | "screen" | "window"` (with optional `format`, `quality`, `scale`, `max_width`, `max_height`, `max_bytes`) and/or `include_tree: true` on the action call itself to observe the result in the same tool call, or re-check state with `get_app_state`, `focused_window`, or an app-specific readback.
|