@agent-sh/computer-use-linux 0.2.1 → 0.2.3

package/README.md

@@ -1,12 +1,22 @@
-# computer-use-linux
+<div align="center">
+  <h1>computer-use-linux</h1>
+  <p><strong>Linux desktop control for MCP hosts.</strong></p>
+  <p>
+    <a href="https://github.com/agent-sh/computer-use-linux/actions/workflows/ci.yml"><img src="https://github.com/agent-sh/computer-use-linux/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
+    <a href="https://crates.io/crates/computer-use-linux"><img src="https://img.shields.io/crates/v/computer-use-linux.svg" alt="crates.io"></a>
+    <a href="https://www.npmjs.com/package/@agent-sh/computer-use-linux"><img src="https://img.shields.io/npm/v/@agent-sh/computer-use-linux.svg" alt="npm"></a>
+    <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
+  </p>
+</div>
+
+Linux desktop control for any MCP host: AT-SPI accessibility trees, portal screenshots, Wayland/X11 input, and multi-compositor window targeting for GNOME, KDE/KWin, Hyprland, i3, and COSMIC.
 
-Linux desktop control for any MCP host — AT-SPI accessibility trees, portal screenshots, Wayland/X11 input, and multi-compositor window targeting for GNOME, KDE/KWin, Hyprland, i3, and COSMIC.
-
-[![CI](https://github.com/avifenesh/computer-use-linux/actions/workflows/ci.yml/badge.svg)](https://github.com/avifenesh/computer-use-linux/actions/workflows/ci.yml)
-[![crates.io](https://img.shields.io/crates/v/computer-use-linux.svg)](https://crates.io/crates/computer-use-linux)
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+```bash
+npm install -g @agent-sh/computer-use-linux@0.2.1
+computer-use-linux doctor | jq .readiness
+```
 
-Current release: [`v0.2.1`](https://github.com/avifenesh/computer-use-linux/releases/tag/v0.2.1). The Rust crate is published as [`computer-use-linux`](https://crates.io/crates/computer-use-linux), and the npm wrapper is published as [`@agent-sh/computer-use-linux`](https://www.npmjs.com/package/@agent-sh/computer-use-linux).
+Current release: [`v0.2.1`](https://github.com/agent-sh/computer-use-linux/releases/tag/v0.2.1). The Rust crate is published as [`computer-use-linux`](https://crates.io/crates/computer-use-linux), and the npm wrapper is published as [`@agent-sh/computer-use-linux`](https://www.npmjs.com/package/@agent-sh/computer-use-linux).
 
 ## What this is
 
@@ -102,7 +112,7 @@ COSMIC users do not need a second package or a separate helper install when usin
 Installs system packages on Debian/Ubuntu, Fedora/RHEL-like, or Arch-like distros; installs Rust if needed; builds both release binaries; installs them to `~/.local/bin`; enables `ydotoold` as a user service; enables GNOME AT-SPI settings when running under GNOME; and installs the bundled GNOME Shell extension on GNOME Wayland.
 
 ```bash
-git clone https://github.com/avifenesh/computer-use-linux
+git clone https://github.com/agent-sh/computer-use-linux
 cd computer-use-linux
 ./install.sh
 # log out and back in if the GNOME extension was newly installed
@@ -121,7 +131,7 @@ computer-use-linux doctor
 For unreleased changes from `main`, install directly from Git:
 
 ```bash
-cargo install --git https://github.com/avifenesh/computer-use-linux
+cargo install --git https://github.com/agent-sh/computer-use-linux
 ```
 
 Then, as needed:
@@ -148,15 +158,15 @@ You will still need `ydotoold` running and AT-SPI enabled (run `computer-use-lin
 
 Linux x86_64 / aarch64 builds are published with each tag. Each binary ships a `.sha256` next to it.
 
-- Release: <https://github.com/avifenesh/computer-use-linux/releases/tag/v0.2.1>
+- Release: <https://github.com/agent-sh/computer-use-linux/releases/tag/v0.2.1>
 
 ```bash
 target=x86_64-unknown-linux-gnu
 version=v0.2.1
 for binary in computer-use-linux computer-use-linux-cosmic; do
   asset="$binary-$target"
-  curl -L -O "https://github.com/avifenesh/computer-use-linux/releases/download/$version/$asset"
-  curl -L -O "https://github.com/avifenesh/computer-use-linux/releases/download/$version/$asset.sha256"
+  curl -L -O "https://github.com/agent-sh/computer-use-linux/releases/download/$version/$asset"
+  curl -L -O "https://github.com/agent-sh/computer-use-linux/releases/download/$version/$asset.sha256"
   sha256sum -c "$asset.sha256"
   install -m 0755 "$asset" "$HOME/.local/bin/$binary"
 done
@@ -191,29 +201,51 @@ Restart Claude Desktop. The 15 tools should appear in the tools list.
 
 ### Hermes Agent
 
-If `computer-use-linux` is on your `PATH`, let Hermes discover it:
+Install the companion Hermes skill so Hermes has the desktop-specific runbook:
 
 ```bash
-hermes mcp add computer-use-linux --command computer-use-linux --args mcp
-hermes mcp test computer-use-linux
+hermes skills tap add agent-sh/computer-use-linux
+hermes skills install agent-sh/computer-use-linux/computer-use-linux
 ```
 
-Press Enter at the "Enable all tools?" prompt to expose all 15 tools. Hermes registers them as `mcp_computer_use_linux_<tool>` and creates the `mcp-computer-use-linux` runtime toolset.
+The skill is optional but recommended for Hermes users. It teaches Hermes how to install, configure, verify, and call the Linux desktop MCP safely. It follows the same `skills/<name>/SKILL.md` tap layout used by Hermes community skills.
 
-If you installed the binary somewhere that is not on `PATH`, pass the absolute path as `--command`.
+Then add the stdio MCP server:
+
+```bash
+hermes mcp add computer-use-linux --command computer-use-linux --args mcp
+hermes mcp test computer-use-linux
+hermes mcp configure computer-use-linux
+```
 
-You can also edit `~/.hermes/config.yaml` directly:
+`configure` opens Hermes' tool-selection UI for the server. The generated config should look like this:
 
 ```yaml
 mcp_servers:
   computer-use-linux:
     command: computer-use-linux
     args: ["mcp"]
+    timeout: 120
+    connect_timeout: 30
 
 # Optional: expose the tools to subagents as well.
 inherit_mcp_toolsets: true
 ```
 
+If you installed the binary somewhere that is not on `PATH`, pass the absolute path as `--command`.
+
+Restart Hermes after editing the config. Hermes registers the tools as `mcp_computer_use_linux_<tool>` and creates the `mcp-computer-use-linux` runtime toolset.
+
+You can verify both sides before asking Hermes to use the desktop:
+
+```bash
+computer-use-linux doctor | jq .readiness
+hermes skills inspect agent-sh/computer-use-linux/computer-use-linux
+hermes chat --toolsets mcp-computer-use-linux -q "List the current desktop windows."
+```
+
+For one-off installs without adding the tap first, Hermes also accepts `hermes skills install agent-sh/computer-use-linux/skills/computer-use-linux`.
+
 ### Generic MCP client
 
 Spawn the binary with `["mcp"]` as the argv tail. It speaks JSON-RPC over stdio per the rmcp 2024-11-05 protocol; capability discovery happens through `tools/list` and the `doctor` tool. The server normally needs no MCP-specific configuration, but desktop runtime environment still matters (`DBUS_SESSION_BUS_ADDRESS`, `XDG_RUNTIME_DIR`, portals, AT-SPI, `ydotoold`, and optionally `COMPUTER_USE_LINUX_COSMIC_HELPER`).
@@ -284,6 +316,10 @@ If you're running this on a shared workstation, set `ydotoold`'s socket permissi
 
 If `doctor` is green and a specific tool still misbehaves, file an issue with the JSON output of `doctor` and the failing tool's request payload.
 
+## Contributing
+
+Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for the local development workflow, CI gates, and PR expectations. Report security vulnerabilities through [SECURITY.md](SECURITY.md), not public issues.
+
 ## Credits
 
 Extracted from [`codex-desktop-linux`](https://github.com/avifenesh/codex-desktop-linux), the Linux distribution of Codex Desktop, which continues to ship this same binary as a bundled plugin. Maintained by [Avi Fenesh](https://github.com/avifenesh).
@@ -301,8 +337,8 @@ Built on top of:
 Publishing is tag-driven from GitHub Actions. The repository needs these Actions secrets:
 
 ```bash
-gh secret set CARGO_REGISTRY_TOKEN -R avifenesh/computer-use-linux
-gh secret set NPM_TOKEN -R avifenesh/computer-use-linux
+gh secret set CARGO_REGISTRY_TOKEN -R agent-sh/computer-use-linux
+gh secret set NPM_TOKEN -R agent-sh/computer-use-linux
 ```
 
 Then bump `Cargo.toml` and `package.json` together, update `CHANGELOG.md`, and push a `vX.Y.Z` tag. CI runs the full Rust and MCP safety gates, builds release assets for both architectures, publishes `computer-use-linux` to crates.io, and publishes the npm wrapper after the GitHub release binaries are available.

package/npm/README.md

@@ -12,8 +12,22 @@ desktop actions.
 ```bash
 npm install -g @agent-sh/computer-use-linux@0.2.1
 computer-use-linux doctor
+hermes skills tap add agent-sh/computer-use-linux
+hermes skills install agent-sh/computer-use-linux/computer-use-linux
 hermes mcp add computer-use-linux --command computer-use-linux --args mcp
 hermes mcp test computer-use-linux
+hermes mcp configure computer-use-linux
+```
+
+The generated Hermes config should look like this:
+
+```yaml
+mcp_servers:
+  computer-use-linux:
+    command: computer-use-linux
+    args: ["mcp"]
+    timeout: 120
+    connect_timeout: 30
 ```
 
 The package downloads the matching Linux x86_64 or aarch64 binary from the

package/npm/install.js

@@ -102,7 +102,7 @@ async function main() {
   const cosmicAsset = `computer-use-linux-cosmic-${targetArch}-unknown-linux-gnu`;
   const baseUrl =
     process.env.COMPUTER_USE_LINUX_DOWNLOAD_BASE ||
-    `https://github.com/avifenesh/computer-use-linux/releases/download/v${pkg.version}`;
+    `https://github.com/agent-sh/computer-use-linux/releases/download/v${pkg.version}`;
   const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'computer-use-linux-'));
   const tmpBinary = path.join(tmpDir, asset);
   const tmpSha = path.join(tmpDir, `${asset}.sha256`);

package/package.json

@@ -1,16 +1,16 @@
 {
   "name": "@agent-sh/computer-use-linux",
-  "version": "0.2.1",
+  "version": "0.2.3",
   "description": "Linux desktop-control MCP server: AT-SPI accessibility trees, Wayland/X11 input, screenshots, and compositor window targeting.",
   "license": "MIT",
   "type": "commonjs",
-  "homepage": "https://github.com/avifenesh/computer-use-linux#readme",
+  "homepage": "https://github.com/agent-sh/computer-use-linux#readme",
   "repository": {
     "type": "git",
-    "url": "git+https://github.com/avifenesh/computer-use-linux.git"
+    "url": "git+https://github.com/agent-sh/computer-use-linux.git"
   },
   "bugs": {
-    "url": "https://github.com/avifenesh/computer-use-linux/issues"
+    "url": "https://github.com/agent-sh/computer-use-linux/issues"
   },
   "keywords": [
     "mcp",
@@ -34,6 +34,7 @@
   "files": [
     "LICENSE",
     "README.md",
+    "skills/computer-use-linux/SKILL.md",
     "npm/README.md",
     "npm/bin/computer-use-linux.js",
     "npm/install.js"

package/skills/computer-use-linux/SKILL.md

@@ -0,0 +1,125 @@
+---
+name: computer-use-linux
+description: "Use when Hermes needs Linux desktop observation or control through the computer-use-linux MCP server."
+version: 0.2.1
+author: agent-sh
+license: MIT
+platforms: [linux]
+---
+
+# computer-use-linux
+
+Use `computer-use-linux` when Hermes needs to observe or operate a local Linux desktop through MCP: inspect the accessibility tree, list/focus windows, take screenshots, click, scroll, type, press keys, or invoke AT-SPI actions.
+
+## When to Use
+
+Use this skill when:
+- The user wants Hermes to control a Linux GUI app.
+- You need desktop state from AT-SPI, screenshots, or compositor window metadata.
+- You are configuring the `computer-use-linux` MCP server for Hermes.
+- A desktop action needs target-aware input instead of blind shell commands.
+
+Do not use this for remote browsers, websites, or headless automation when a browser-specific tool is available. Do not assume desktop actions are safe just because the MCP connection works.
+
+## Install
+
+Preferred install for Hermes users:
+
+```bash
+npm install -g @agent-sh/computer-use-linux@0.2.1
+computer-use-linux doctor | jq .readiness
+```
+
+Rust users can install the same server from crates.io:
+
+```bash
+cargo install computer-use-linux --version 0.2.1
+computer-use-linux doctor | jq .readiness
+```
+
+If `doctor` reports missing input or accessibility support, run:
+
+```bash
+computer-use-linux setup
+systemctl --user enable --now ydotoold
+computer-use-linux setup-window-targeting
+computer-use-linux doctor | jq .readiness
+```
+
+On GNOME Wayland, log out and back in after `setup-window-targeting` if the GNOME Shell extension was newly installed.
+
+## Configure Hermes
+
+Add the server with the Hermes MCP CLI:
+
+```bash
+hermes mcp add computer-use-linux --command computer-use-linux --args mcp
+hermes mcp test computer-use-linux
+hermes mcp configure computer-use-linux
+```
+
+`configure` opens Hermes' tool-selection UI for this MCP server.
+
+The generated config should look like this:
+
+```yaml
+mcp_servers:
+  computer-use-linux:
+    command: computer-use-linux
+    args: ["mcp"]
+    timeout: 120
+    connect_timeout: 30
+```
+
+If the binary is not on `PATH`, pass the absolute path to `--command`.
+
+Hermes registers tools using the `mcp_<server>_<tool>` pattern. With this config, tool names are prefixed as `mcp_computer_use_linux_`, for example:
+
+| MCP tool | Hermes tool name |
+| --- | --- |
+| `doctor` | `mcp_computer_use_linux_doctor` |
+| `get_app_state` | `mcp_computer_use_linux_get_app_state` |
+| `list_windows` | `mcp_computer_use_linux_list_windows` |
+| `click` | `mcp_computer_use_linux_click` |
+| `type_text` | `mcp_computer_use_linux_type_text` |
+
+Restart Hermes after changing MCP config.
+
+## Procedure
+
+1. Start every desktop-control session with `doctor`.
+2. If `can_build_accessibility_tree` is false, run `setup` and restart the target app.
+3. If `can_query_windows` is false on GNOME Wayland, run `setup-window-targeting` and ask the user to log out and back in if setup says the shell extension needs a reload.
+4. Before targeted input, call `list_windows` or `focused_window` and verify the intended window by title, app id, pid, or wm class.
+5. Prefer semantic targeting from `get_app_state`: use element indices or role/name/text/states selectors.
+6. Use coordinates only when the UI surface has no useful accessibility tree.
+7. For text input, prefer `type_text` with a target selector (`window_id`, `pid`, `app_id`, `wm_class`, `title`, `tty`, `terminal_pid`, `terminal_command`, or `terminal_cwd`) rather than relying on current focus.
+8. After mutating actions, re-check state with `get_app_state`, `focused_window`, or an app-specific readback.
+
+## Pitfalls
+
+- Already-running GTK, Qt, and Electron apps may need a restart after AT-SPI is enabled.
+- GNOME may show a portal prompt on the first screenshot or `get_app_state` call with screenshots enabled.
+- Desktop input is stateful. Avoid concurrent tool calls against this MCP server.
+- `click`, `drag`, `press_key`, `type_text`, `perform_action`, and `set_value` can change real application state.
+- `ydotoold` should run as a per-user service with its socket under `/run/user/$UID`, not as a system-wide service.
+- On COSMIC, the standard npm, Cargo, and install-script paths install the `computer-use-linux-cosmic` helper automatically. Manual binary installs must copy both binaries.
+
+## Verification
+
+Run:
+
+```bash
+computer-use-linux doctor | jq .readiness
+hermes chat --toolsets mcp-computer-use-linux -q "List the current desktop windows."
+```
+
+Ready output should have:
+
+- `can_register_mcp_tools: true`
+- `can_build_accessibility_tree: true`
+- `can_query_windows: true`
+- `can_send_development_input: true`
+- `blockers: []`
+
+If Hermes does not expose the tools, check startup logs for MCP discovery errors and confirm the server name in `config.yaml` is exactly `computer-use-linux`.