@agent-sh/computer-use-linux 0.7.48 → 0.7.49
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -9,6 +9,8 @@
|
|
|
9
9
|
</p>
|
|
10
10
|
</div>
|
|
11
11
|
|
|
12
|
+
> Running open models for your company? [**Tiyuvta**](https://tiyuvta.ai/services/) helps with model choice, deployment and optimization, and fine-tuning on your hardware or cloud account.
|
|
13
|
+
|
|
12
14
|
<p align="center">
|
|
13
15
|
<a href="https://github.com/sponsors/avifenesh"><img src="https://img.shields.io/badge/Sponsor-%E2%9D%A4-db61a2?style=for-the-badge&logo=githubsponsors&logoColor=white" alt="Sponsor"></a>
|
|
14
16
|
</p>
|
|
@@ -83,9 +85,9 @@ The crate was extracted from [`codex-desktop-linux`](https://github.com/avifenes
|
|
|
83
85
|
| **Opt-in host tools** | `complete_interaction` | Parameter-free desktop completion notification (`notify-send`); registered only when `COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1`. |
|
|
84
86
|
| | `run_shell` | Bounded non-login `/bin/sh -c` execution; registered only when `COMPUTER_USE_LINUX_ENABLE_SHELL=1` (absent by default). |
|
|
85
87
|
|
|
86
|
-
Every tree result carries a `snapshot_id`: `get_app_state`, an `include_tree` observation, and `get_tree_page`. The server caches one tree, and each new tree replaces it. Pass the `snapshot_id` with an `element_index` or selector to `click`, `scroll`, `perform_action` or `set_value`, and to `get_tree_page`: if a later tree replaced that snapshot, the call is refused before any window is raised or input is sent, since the index could name another element by then. A full `x`/`y` point does not come from the tree, so it is not checked. Without `snapshot_id`, the element resolves in the latest tree. An `element_identifier` names the AT-SPI object itself and is not checked against a snapshot. Ids differ between server processes, so an id from before a restart is refused.
|
|
88
|
+
Every tree result carries a `snapshot_id`: `get_app_state`, an `include_tree` observation, and `get_tree_page`. The server caches one tree, and each new tree replaces it. Pass the `snapshot_id` with an `element_index` (or a selector, except on `scroll`) to `click`, `scroll`, `perform_action` or `set_value`, and to `get_tree_page`: if a later tree replaced that snapshot, the call is refused before any window is raised or input is sent, since the index could name another element by then. A full capture-space `x`/`y` point does not come from the tree, so it is not checked. A `coordinate_space: "atspi"` click maps through its element, so it is checked. Without `snapshot_id`, the element resolves in the latest tree. An `element_identifier` names the AT-SPI object itself and is not checked against a snapshot. Ids differ between server processes, so an id from before a restart is refused.
|
|
87
89
|
|
|
88
|
-
`click`, `scroll`, `perform_action`, and `set_value` take an optional app target (`pid`, `window_id`, `app_id`, `wm_class`, `window_title`). With one, an `element_index` or selector from another app's `get_app_state` snapshot is refused; call `get_app_state` for the target app first. When the window backend reports no pid for a target window (i3 without `xprop`, COSMIC, other wlroots compositors), the snapshot must have been taken for that window, or the call is refused. `perform_action` and `set_value` do not focus the target window. An `element_identifier` is refused when its owner on the accessibility bus is a different pid than the target; it is not checked when either pid is unknown.
|
|
90
|
+
`click`, `scroll`, `perform_action`, and `set_value` take an optional app target (`pid`, `window_id`, `app_id`, `wm_class`, `window_title`). With one, an `element_index` (or a selector, where the tool takes one) from another app's `get_app_state` snapshot is refused; call `get_app_state` for the target app first. When the window backend reports no pid for a target window (i3 without `xprop`, COSMIC, other wlroots compositors), the snapshot must have been taken for that window, or the call is refused. `perform_action` and `set_value` do not focus the target window. An `element_identifier` is refused when its owner on the accessibility bus is a different pid than the target; it is not checked when either pid is unknown.
|
|
89
91
|
|
|
90
92
|
### Post-action observation in one round trip
|
|
91
93
|
|
package/npm/README.md
CHANGED
|
@@ -166,14 +166,15 @@ does not yet forward the flag or include the tool in its catalog.
|
|
|
166
166
|
- **Snapshot ids**: `get_app_state`, `include_tree` observations and
|
|
167
167
|
`get_tree_page` return the `snapshot_id` of their tree. The server caches
|
|
168
168
|
one tree, and each new tree replaces it. Pass `snapshot_id` with an
|
|
169
|
-
`element_index` or selector to `click`, `scroll`,
|
|
170
|
-
`set_value`, and to `get_tree_page`: a call naming a
|
|
171
|
-
refused before any input. Without it, the element
|
|
172
|
-
tree.
|
|
169
|
+
`element_index` (or a selector, except on `scroll`) to `click`, `scroll`,
|
|
170
|
+
`perform_action` or `set_value`, and to `get_tree_page`: a call naming a
|
|
171
|
+
replaced snapshot is refused before any input. Without it, the element
|
|
172
|
+
resolves in the latest tree. A full capture-space `x`/`y` point is not
|
|
173
|
+
checked; a `coordinate_space: "atspi"` click is checked.
|
|
173
174
|
- **Element targets**: `click`, `scroll`, `perform_action`, and `set_value`
|
|
174
175
|
accept an optional app target (`pid`, `window_id`, `app_id`, `wm_class`,
|
|
175
|
-
`window_title`). With one, an `element_index` or selector
|
|
176
|
-
`get_app_state` snapshot is refused. When the window backend reports no pid
|
|
176
|
+
`window_title`). With one, an `element_index` (or a selector, where the
|
|
177
|
+
tool takes one) from another app's `get_app_state` snapshot is refused. When the window backend reports no pid
|
|
177
178
|
for the target window (i3 without `xprop`, COSMIC, other wlroots compositors),
|
|
178
179
|
the snapshot must have been taken for that window. `perform_action` and
|
|
179
180
|
`set_value` do not focus the window; an `element_identifier` is refused only
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agent-sh/computer-use-linux",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.49",
|
|
4
4
|
"description": "Native Linux desktop control for AI agents (MCP server, Pi tools, and CLI): semantic AT-SPI accessibility trees, screenshots, compositor window targeting, Wayland/X11 input, and an on-screen activity indicator.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "commonjs",
|
|
@@ -7,9 +7,9 @@ export interface GeneratedMcpToolDefinition {
|
|
|
7
7
|
annotations: Record<string, unknown>;
|
|
8
8
|
}
|
|
9
9
|
|
|
10
|
-
export const GENERATED_SERVER_VERSION = "0.7.
|
|
11
|
-
export const GENERATED_TOOL_CATALOG_HASH = "
|
|
12
|
-
export const GENERATED_SHELL_TOOL_CATALOG_HASH = "
|
|
10
|
+
export const GENERATED_SERVER_VERSION = "0.7.49";
|
|
11
|
+
export const GENERATED_TOOL_CATALOG_HASH = "cffd6953843133dfee35c6c5bfc4f98e177798a31cc05e711a18d6b96ae7d2e4";
|
|
12
|
+
export const GENERATED_SHELL_TOOL_CATALOG_HASH = "1b9f83b4ce6e5b07b25eae206dae6af7a31dda69fd073ad55d50f03bbd4bdb66";
|
|
13
13
|
export const GENERATED_MCP_TOOLS =
|
|
14
14
|
[
|
|
15
15
|
{
|
|
@@ -146,7 +146,7 @@ Pi native tools skip this MCP `command` config; see [Pi setup](references/pi-set
|
|
|
146
146
|
4. If `can_build_accessibility_tree` is false, run `setup_accessibility` and restart the target app.
|
|
147
147
|
5. If `can_query_windows` or `can_focus_windows` is false on GNOME Wayland, run `setup_window_targeting` and ask the user to log out and back in if setup says the shell extension needs a reload.
|
|
148
148
|
6. Before targeted input, call `list_windows` or `focused_window` and verify the intended window by title, app id, pid, or wm class.
|
|
149
|
-
7. Prefer semantic targeting from `get_app_state`: use element indices or role/name/text/states selectors, and pass the `snapshot_id` of the result they came from. Every `get_app_state` and `include_tree` result replaces the one cached tree; a call that names a replaced snapshot is refused before any input, so take the indices and `snapshot_id` from the newest result. Automatic recovery of a dead semantic target requires verified PID/window ownership and invalidates cached indices; call `get_app_state` again before further index-based actions. When several apps are in play, pass the app target (`pid`, `window_id`, `app_id`, `wm_class`, `window_title`) to `click`, `scroll`, `perform_action`, and `set_value`: an index or selector from another app's snapshot is then refused instead of acting on that app. On `click` and `scroll` the target also raises and focuses that window, and nothing is sent when focus cannot be verified; on GNOME Wayland, raising a window needs the Shell extension. The title field there is `window_title`. Where the window backend reports no pid (i3 without `xprop`, COSMIC, and wlr-foreign-toplevel on stock labwc and other wlroots compositors), call `get_app_state` for the same window first or pass `pid`, or the call is refused.
|
|
149
|
+
7. Prefer semantic targeting from `get_app_state`: use element indices or role/name/text/states selectors, and pass the `snapshot_id` of the result they came from. Every `get_app_state` and `include_tree` result replaces the one cached tree; a call that names a replaced snapshot is refused before any input, so take the indices and `snapshot_id` from the newest result. Automatic recovery of a dead semantic target requires verified PID/window ownership and invalidates cached indices; call `get_app_state` again before further index-based actions. When several apps are in play, pass the app target (`pid`, `window_id`, `app_id`, `wm_class`, `window_title`) to `click`, `scroll`, `perform_action`, and `set_value`: an index (or a selector, where the tool takes one) from another app's snapshot is then refused instead of acting on that app. On `click` and `scroll` the target also raises and focuses that window, and nothing is sent when focus cannot be verified; on GNOME Wayland, raising a window needs the Shell extension. The title field there is `window_title`. Where the window backend reports no pid (i3 without `xprop`, COSMIC, and wlr-foreign-toplevel on stock labwc and other wlroots compositors), call `get_app_state` for the same window first or pass `pid`, or the call is refused.
|
|
150
150
|
8. Use coordinates only when the UI surface has no useful accessibility tree.
|
|
151
151
|
9. For text input, prefer `type_text` with a target selector (`window_id`, `pid`, `app_id`, `wm_class`, `title`, `tty`, `terminal_pid`, `terminal_command`, or `terminal_cwd`) rather than relying on current focus. A `window_id` is valid only as returned by a recent `list_windows`; if a call rejects it as outside 0 to 2^53-1 or matches no window, re-run `list_windows`.
|
|
152
152
|
10. After mutating actions (`click`, `drag`, `scroll`, `press_key`, `type_text`, `perform_action`, `set_value`, `activate_window`, `move_window`, `resize_window`), either pass `screenshot_after: true | "screen" | "window"` (with optional `format`, `quality`, `scale`, `max_width`, `max_height`, `max_bytes`) and/or `include_tree: true` on the action call itself to observe the result in the same tool call, or re-check state with `get_app_state`, `focused_window`, or an app-specific readback. A failed action (`ok: false`) returns no post-action screenshot or tree, so call `get_app_state`; `set_value` still observes when `verified` is `false`.
|