@agent-sh/computer-use-linux 0.7.42 → 0.7.43
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -124,7 +124,7 @@ the origin. A missing window target is rejected.
|
|
|
124
124
|
|
|
125
125
|
**X11 text fidelity:** `xdotool`/XTEST can silently substitute symbols under non-US XKB layouts (reported with Brazilian `br`/ABNT2) or changing keymap state, even when `type_text` returns `ok: true`. Every successful xdotool text dispatch includes a layout warning. XKB layout replacement events, persistent mapping/group changes, and keyboard focus/grab transitions detected across a call return an error after dispatch, without replaying the text. Keyboard grabs can divert characters without changing the polled focused-window ID. If monitoring cannot start, xdotool still dispatches and the response explicitly warns that layout-change and keyboard interception detection were unavailable. PointerRoot focus is unsupported because pointer motion can redirect input without a detectable focus event. This also detects a `setxkbmap` layout changed and restored during one call. Temporary raw remaps or group switches restored within a call and dropped text with a stable map remain unverified. Re-observe the actual text before pressing Enter, submitting it, or treating it as delivered. Use `include_tree: true` for AT-SPI readback when the application exposes the text, or `screenshot_after: true` to inspect it. These observations do not automatically compare the field to the requested text. Do not retry blindly.
|
|
126
126
|
|
|
127
|
-
Targeted `press_key`/`type_text` results append focused-element feedback from AT-SPI (role, name, editable) and warn when a concrete non-terminal focus is not editable. Terminals accept keyboard input without an EditableText interface; re-observe the contents to confirm input. Click/screenshot/input results warn when the target window or coordinate is partially or fully off-screen. A click or
|
|
127
|
+
Targeted `press_key`/`type_text` results append focused-element feedback from AT-SPI (role, name, editable) and warn when a concrete non-terminal focus is not editable. Terminals accept keyboard input without an EditableText interface; re-observe the contents to confirm input. Click/screenshot/input results warn when the target window or coordinate is partially or fully off-screen. A click, scroll or drag coordinate between monitors, where monitors of different sizes or offset rows leave part of the screenshot uncovered, gets a warning too: the server reads the monitor layout once per screenshot (from the portal session, KWin, the GNOME Shell extension, Hyprland, kscreen, Sway, COSMIC, niri or wlr-output-management, and from RandR on X11 only) and checks the point in screenshot pixels. Without a layout that fits the screenshot, only its outer rectangle is checked. The warning says what the route did. The portal sends nothing. X11 keeps the pointer on the screen, so a point off the screenshot lands at an edge. On X11 and the uinput pointer, a point between monitors lands at a monitor edge or on no visible pixel. A failed X11 click or scroll gets no warning, since nothing was sent. A failed drag gets none either, though it may have pressed and released before it failed. A drag checks its start and end points, and each warning names the endpoint. On the uinput pointer, a drag result also says where an off-screen endpoint was clamped to, as a click result does. Points along the path are not checked: the portal moves them onto the nearest shared monitor, and on X11 and uinput the drop goes to the end point. `get_app_state` returns a compact readiness block by default; pass `verbose: true` for the full diagnostics report. Tree nodes use a compact encoding: null, empty, zero and false fields are left out, containers do not carry the U+FFFC placeholder text Chromium gives them, and `actions` lists action names (pass one as `perform_action`'s `action`). `verbose: true` returns the full encoding, with every field and action descriptions and keybindings. `interactive_only: true` returns only nodes with an action, editable text or a value, plus headings, status and live regions, windows and dialogs; containers are collapsed, so `parent_index` names the nearest kept ancestor. Its `element_index` values are the same as in the full tree. When `title` or `window_id` names one of several windows under one AT-SPI application root (Chromium puts every browser window under one), the tree covers that window plus the app's popups and dialogs, and drops its other named windows; `message` says so. It also reports `tree_scoped` (false when no app target narrowed the AT-SPI tree, with a warning in `message`) and `accessibility_tree_truncated` (true when the node, depth, read, or time budget stopped traversal with unread elements left). When a result would pass 32,000 bytes, the tree comes back as its leading nodes plus `accessibility_tree_spill` (`path`, `node_count`, `bytes`, `inline_node_count`). `path` is a private JSON file holding the whole tree with the same `element_index` values, in `$XDG_RUNTIME_DIR/computer-use-linux/trees` when that runtime directory is private, otherwise under `~/.cache` (directories 0700, file 0600). Search the file instead of requesting the tree again; hosts that cannot read files page through the same tree with `get_tree_page`. The file lasts until the next snapshot or until the server exits; after a crash it is swept at the next start or after an hour.
|
|
128
128
|
|
|
129
129
|
An `app_id` may be a full desktop ID or its final dot-separated segment; matching is case-insensitive, ambiguous aliases are rejected, and an unresolved explicit `app_id` returns an empty scoped tree rather than the desktop.
|
|
130
130
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agent-sh/computer-use-linux",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.43",
|
|
4
4
|
"description": "Native Linux desktop control for AI agents (MCP server, Pi tools, and CLI): semantic AT-SPI accessibility trees, screenshots, compositor window targeting, Wayland/X11 input, and an on-screen activity indicator.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "commonjs",
|
|
@@ -7,7 +7,7 @@ export interface GeneratedMcpToolDefinition {
|
|
|
7
7
|
annotations: Record<string, unknown>;
|
|
8
8
|
}
|
|
9
9
|
|
|
10
|
-
export const GENERATED_SERVER_VERSION = "0.7.
|
|
10
|
+
export const GENERATED_SERVER_VERSION = "0.7.43";
|
|
11
11
|
export const GENERATED_TOOL_CATALOG_HASH = "c3da735e9c17d0bd15294e604abd28a6008e67d250f6db226da05e3f9504f68f";
|
|
12
12
|
export const GENERATED_SHELL_TOOL_CATALOG_HASH = "96bd879e7d29ef3f3193d49ef6f86b0d561b2ac9c7eeeece326c1637e1c6428f";
|
|
13
13
|
export const GENERATED_MCP_TOOLS =
|
|
@@ -196,11 +196,11 @@ This requires both `x`/`y` and element identity; do not
|
|
|
196
196
|
combine it with `relative: true` or use it for ordinary screenshot pixels.
|
|
197
197
|
Use `perform_action` explicitly for entry `activate` or slider `jump`; `click`
|
|
198
198
|
never substitutes those actions, including when bounds are unavailable.
|
|
199
|
-
A coordinate `click` or `
|
|
200
|
-
screenshot or between monitors, where no monitor shows that part of the
|
|
201
|
-
screenshot.
|
|
202
|
-
|
|
203
|
-
before retrying.
|
|
199
|
+
A coordinate `click`, `scroll` or `drag` result warns when the point is off
|
|
200
|
+
the screenshot or between monitors, where no monitor shows that part of the
|
|
201
|
+
screenshot. A `drag` warning names its start or end coordinate. The result
|
|
202
|
+
says whether the input was sent, and the warning says where it may have
|
|
203
|
+
landed; pick a point on a monitor and check the screen before retrying.
|
|
204
204
|
|
|
205
205
|
### Screenshot-relative coordinates
|
|
206
206
|
|