@agent-sh/computer-use-linux 0.7.55 → 0.7.56
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/package.json +1 -1
- package/pi/extension/generated-tools.ts +3 -3
package/README.md
CHANGED
|
@@ -205,7 +205,7 @@ Validated manually on Ubuntu 25.10 (GNOME Shell 50.1, Wayland). niri window list
|
|
|
205
205
|
| KDE Plasma / KWin | temporary KWin DBus scripting | Lists and focuses windows through Plasma 5 or 6 `org.kde.KWin` scripting APIs when the session bus exposes them. IDs are derived from the KWin window UUID, stay the same for a window across calls, and fit JavaScript integers. |
|
|
206
206
|
| Hyprland | `hyprctl clients -j`; focus through the Lua `hl.dsp.focus` dispatcher, then `focuswindow` | Requires `hyprctl` in the desktop session. With `COMPUTER_USE_LINUX_HYPRLAND_BACKGROUND=1`, targeted keyboard input uses `send_key_state` and targeted capture uses `grim -T`, so neither focuses the window. |
|
|
207
207
|
| niri | `niri msg` with a direct JSON IPC fallback | Lists toplevels and focuses exact window IDs. When the host does not pass `NIRI_SOCKET`, the server reads it from its parent processes or the systemd user environment; without it, discovery requires an unambiguous socket matching `WAYLAND_DISPLAY`. The fallback works without the `niri` binary. Missing positions remain `null`; bounds are omitted if output scaling is unknown or mixed. |
|
|
208
|
-
| Umbriel | `umbriel windows --json` and `umbriel msg window-focus` | Lists toplevels with pids and focuses exact window IDs through the `umbriel` CLI, which needs `UMBRIEL_SOCKET` (passed by the host, read from the session's environment, or found as `umbriel-$WAYLAND_DISPLAY.sock` in `XDG_RUNTIME_DIR`). IDs are the first 52 bits of Umbriel's window id, so they fit JavaScript integers.
|
|
208
|
+
| Umbriel | `umbriel windows --json` and `umbriel msg window-focus` | Lists toplevels with pids and focuses exact window IDs through the `umbriel` CLI, which needs `UMBRIEL_SOCKET` (passed by the host, read from the session's environment, or found as `umbriel-$WAYLAND_DISPLAY.sock` in `XDG_RUNTIME_DIR`). IDs are the first 52 bits of Umbriel's window id, so they fit JavaScript integers. Windows on a workspace that is not shown are `hidden`. Screenshots use `dms screenshot all` when the DMS shell is installed, so every output is captured; without it the portal captures one output and uinput pointer coordinates are placed inside the whole layout. Bounds are scaled to capture pixels with `umbriel outputs --json`: always with one enabled output, and with several only while captures cover every output, so they are left out in the portal fallback and with mixed output scales. Validated on Umbriel 0.1.0 with two stacked 2880x1800 outputs at scale 1.8. |
|
|
209
209
|
| i3, Sway | `i3-msg`, or `swaymsg` when i3 is not installed; on i3, optional `xprop` for PID hydration | Lists and focuses windows over the active i3 or Sway IPC socket. On Sway this covers native Wayland and Xwayland windows, with pids from the tree, and bounds scaled to capture pixels when every output has the same scale (left out with mixed scales). The socket is the first of `I3SOCK` and `SWAYSOCK` that names a socket file, then the newest i3 socket, then the newest Sway socket in `XDG_RUNTIME_DIR`. X11 windows keep their X11 id; a native Wayland window on Sway gets 2^32 plus its container id, so the two never collide. |
|
|
210
210
|
| labwc with Selkies IPC | `$XDG_RUNTIME_DIR/labwc.sock` | Requires the Selkies IPC patch; stock labwc has no such socket. Lists windows and focuses unique app-id/title matches through `wlr-foreign-toplevel-management`. IDs include pid, app id and title, fit JavaScript integers, and change when the title changes. Ambiguous windows are rejected. Bounds are omitted because the IPC does not expose output scaling. |
|
|
211
211
|
| COSMIC Wayland | `computer-use-linux-cosmic` helper | Installed automatically by `./install.sh`, `cargo install`, and npm. For custom/manual layouts, put the helper next to the main binary, on `PATH`, or point `COMPUTER_USE_LINUX_COSMIC_HELPER` at it. The on-screen indicator was validated here (three outputs, mixed fractional scaling). |
|
|
@@ -554,7 +554,7 @@ files.
|
|
|
554
554
|
- **Native X11 transport.** Native X11 queries (window origins, root screenshots, `doctor`) connect and read replies under one shared 2-second deadline enforced in the transport itself, including the handshake and every reply. A stalled X server closes the query connection instead of leaving a blocked worker behind. Unix sockets, literal IP addresses, and `localhost` need no resolver. A remote hostname in `DISPLAY` needs `getent ahosts` for bounded name resolution; if it is missing or fails, the X11 route is reported as unavailable.
|
|
555
555
|
- **Window registry**: `list_windows`, `focused_window`, `activate_window`, `press_key`, and `type_text` share a backend registry. It tries GNOME extension, GNOME Introspect, COSMIC helper, KWin scripting, Hyprland `hyprctl`, the niri IPC socket, the `umbriel` CLI, i3 IPC, labwc's `labwc.sock`, wlr-foreign-toplevel-management, and generic X11/EWMH in that order, skipping empty or failed backends so another compositor backend can answer.
|
|
556
556
|
- **niri backend**: prefers `niri msg`, falling back to direct JSON IPC over the same session socket. `is_minimized` maps to `hidden`. Bounds use `layout.window_size` (or `tile_size`) and, when available, the tile position plus the window's offset and the output's logical origin. Uniform output scaling converts these to device pixels; unknown or mixed scaling omits bounds. Missing window positions stay `null`, so listing and focusing still work without enabling coordinate targeting. An accepted focus action is verified by querying the focused window again.
|
|
557
|
-
- **Umbriel backend**: runs the `umbriel` CLI. `active` maps to `focused` (Umbriel's own `focused` marks one window per workspace), and a minimized window or one on a workspace that is not shown maps to `hidden`. With one enabled output, logical window geometry is scaled by its scale from its origin. With several
|
|
557
|
+
- **Umbriel backend**: runs the `umbriel` CLI. `active` maps to `focused` (Umbriel's own `focused` marks one window per workspace), and a minimized window or one on a workspace that is not shown maps to `hidden`. xdg-desktop-portal-umbriel answers a non-interactive screenshot with its first output by name, so the server captures with `dms screenshot all` when it can; when it cannot and the capture is one output of several, the uinput pointer is sized to the whole layout and offset by that output's position. A capture that matches neither the whole layout nor the first output, within a pixel per output, creates no uinput device, so the other pointer routes are tried instead of scaling every point. `doctor` lists the route as `dms` in `capabilities.screenshot`. With one enabled output, logical window geometry is scaled by its scale from its origin. With several, geometry is scaled from the layout's top-left corner only while the capture covers every output: the size of the latest capture decides, and before the first capture, whether `dms` is on `PATH` with no other screenshot backend forced. Otherwise bounds are omitted, as they are with mixed output scales; listing and exact focus still work. Because a target is resolved before its capture is taken, the server reads an Umbriel window's bounds again for the frame in hand before it crops or maps window-relative coordinates, so bounds for one capture space are never applied to another.
|
|
558
558
|
- **GNOME extension fallback**: recent GNOME builds deny `org.gnome.Shell.Introspect.GetWindows` to non-blessed clients. The bundled Shell extension exposes window data and exact activation under `dev.avifenesh.ComputerUseLinux.WindowControl`. `setup-window-targeting` and `install.sh` install it in `$XDG_DATA_HOME/gnome-shell/extensions` (`~/.local/share/gnome-shell/extensions` when `XDG_DATA_HOME` is unset or empty), where GNOME Shell loads user extensions. When the Shell already loaded a user copy, `setup-window-targeting` updates the copy the Shell reports. When the extension is already enabled, both ask for a new login after an upgrade that GNOME Shell would run: a changed `extension.js`, or a changed `metadata.json` field the Shell acts on, such as `shell-version`. The Shell only checks that `name` and `description` are present, and only displays them, `url`, `version-name` and `donations`, so a change to those alone needs none. `install.sh` checks the four string fields and asks for a login when `donations` changes.
|
|
559
559
|
- **COSMIC helper**: `computer-use-linux-cosmic` talks to COSMIC toplevel protocols and is resolved from `COMPUTER_USE_LINUX_COSMIC_HELPER`, next to the running binary, or from `PATH`. COSMIC pointer coordinates use per-monitor RemoteDesktop portal streams because its uinput absolute axes map onto the active monitor. Approve the portal dialog before continuing. When ConnectToEIS is available, COSMIC input waits for resumed EIS devices before sending the first event, so portal startup cannot discard the text prefix. The helper returns logical window bounds from COSMIC geometry and xdg-output events; targeted screenshots and relative clicks map these into capture pixels. Window IDs fit JavaScript's exact integer range. PID remains optional because the toplevel protocol does not expose it. Like `computer-use-linux`, the helper checks its command line first: `-h` or `--help` prints help and exits 0, and a command, flag or argument it does not take exits 2 before it connects to the compositor.
|
|
560
560
|
- **Indicator overlay**: the server sends one JSON datagram per action (`{"agent","tool","x","y","space","keys","text"}`, where `x`/`y` are screenshot pixels and `space` is the capture size; or `{"capture":"begin"}`/`{"capture":"end"}` around captures; `begin` is answered with `shown` or `empty` when the sender's socket is bound) to `$XDG_RUNTIME_DIR/computer-use-linux-indicator.sock`. `computer-use-linux-indicator` (resolved like the COSMIC helper, via `COMPUTER_USE_LINUX_INDICATOR_BIN`) draws nothing while any server holds a shared `flock` on `computer-use-linux-indicator.capture` next to the socket (held for the length of each capture and released by the kernel if a server dies), and otherwise draws on small `wlr-layer-shell` surfaces with [`tiny-skia`](https://crates.io/crates/tiny-skia) into shared-memory buffers, honouring fractional scaling through `wp_fractional_scale_v1` and `wp_viewporter`. The cursor surface moves by changing its margins, so a glide redraws nothing. On GNOME the helper relays events to the verified Shell extension, which draws non-interactive actors. The helper still owns the shared socket and probes the capture lock. If the helper exits, the extension hides its actors when its D-Bus connection disappears. Captures also hide residual GNOME actors when no helper is listening. The indicator takes no arguments: `-h` or `--help` prints help, and any other argument exits 2 before it takes a lock or connects to the compositor.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agent-sh/computer-use-linux",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.56",
|
|
4
4
|
"description": "Native Linux desktop control for AI agents (MCP server, Pi tools, and CLI): semantic AT-SPI accessibility trees, screenshots, compositor window targeting, Wayland/X11 input, and an on-screen activity indicator.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "commonjs",
|
|
@@ -7,9 +7,9 @@ export interface GeneratedMcpToolDefinition {
|
|
|
7
7
|
annotations: Record<string, unknown>;
|
|
8
8
|
}
|
|
9
9
|
|
|
10
|
-
export const GENERATED_SERVER_VERSION = "0.7.
|
|
11
|
-
export const GENERATED_TOOL_CATALOG_HASH = "
|
|
12
|
-
export const GENERATED_SHELL_TOOL_CATALOG_HASH = "
|
|
10
|
+
export const GENERATED_SERVER_VERSION = "0.7.56";
|
|
11
|
+
export const GENERATED_TOOL_CATALOG_HASH = "88fdbedb6fab2e9bde4236382de77f35d32542894a77d7ec3ff9a3fde53078ba";
|
|
12
|
+
export const GENERATED_SHELL_TOOL_CATALOG_HASH = "dbc5fec64bafdd29cb7dc22464241e3583e59dbfffe052c02493fc9632b2ed8e";
|
|
13
13
|
export const GENERATED_MCP_TOOLS =
|
|
14
14
|
[
|
|
15
15
|
{
|