@aryacode/dsh-plugin-arya-computer-use 0.0.0-stage → 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 KonnectSuite
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/NOTICE.md

@@ -0,0 +1,18 @@
+# Third-party notices
+
+This plugin drives the desktop through two Apache-2.0 engines by Vercel Labs,
+consumed as pinned npm dependencies (not vendored):
+
+- **agent-desktop** — https://github.com/vercel-labs/agent-desktop (Apache-2.0),
+  npm `agent-desktop@1.2.0`. Native accessibility-tree computer use
+  (Windows UIA / macOS AX / Linux AT-SPI). Mirror used for reference and
+  upstream sync: https://github.com/KonnectSuite/dsh-arya-agent-desktop.
+- **agent-browser** — https://github.com/vercel-labs/agent-browser (Apache-2.0),
+  npm `agent-browser@0.41.2`. Web lane (CDP browser automation).
+
+The takeover helper design is adapted from **dsh-computer-use-win**
+(https://github.com/Yu-tao-Li/dsh-computer-use-win, MIT, derived from
+cgissing/windows-computer-use).
+
+All engine binaries remain under their original Apache-2.0 terms; this
+plugin's MIT license covers only KonnectSuite's original code in this repo.

package/README.md

@@ -1,3 +1,109 @@
-# Temporary Holding Version
+# Arya Computer Use
+
+Computer use for [DeepSeek Harness / AryaCode](https://github.com/KonnectSuite): drive Windows desktop apps through **native accessibility refs** (UIA) instead of pixel coordinates, with a **takeover overlay** that frames the window being driven, shows what the agent is doing, keeps a synthetic ghost cursor where it acts (your own pointer is never followed), and stops on a global Esc. The agent calls itself **Arya** in that overlay by default, and the name is a setting.
+
+Replaces [dsh-computer-use-win](https://github.com/Yu-tao-Li/dsh-computer-use-win) (wincu) completely — see [docs/PLAN.md](docs/PLAN.md) §7 for the retirement path.
+
+## Why
+
+| wincu complaint | Arya Computer Use |
+| --- | --- |
+| Slow (PowerShell + UIA backend) | Native Rust engine: snapshot ~117 ms, ref action ~197 ms on a Windows devbox |
+| Pixel alignment off | Ref-based semantic UIA — coordinates are never the targeting mechanism; identical twins are refused, not mis-clicked |
+| Chinese UI/docs | English throughout |
+| (kept) DSH integration + setup UI | Settings page, chat step rows, takeover overlay, Esc kill, tool presets |
+
+## Engines (not vendored — pinned npm dependencies)
+
+- [`agent-desktop`](https://github.com/vercel-labs/agent-desktop) `1.2.0` (Apache-2.0) — accessibility-tree computer use CLI, native Windows binaries on npm. Reference mirror: [KonnectSuite/dsh-arya-agent-desktop](https://github.com/KonnectSuite/dsh-arya-agent-desktop).
+- [`agent-browser`](https://github.com/vercel-labs/agent-browser) `0.41.2` (Apache-2.0) — web lane via CDP (`agent-browser mcp`).
+
+## Requirements
+
+- **DSH host must run elevated (Run as administrator).** The engine's interaction lease lives at `C:\ProgramData\agent-desktop\<user-SID>\interaction.lock` with a protected DACL `{SYSTEM, Administrators, token user}`; a non-elevated medium-IL token fails validation (`lease_directory_untrusted`). The adapter surfaces this as an explicit "run AryaCode as administrator" hint.
+- Accessibility permission (`desktop_status` reports `accessibility: granted`).
+- Engine state root is the engine default `~/.agent-desktop` — do **not** point `AGENT_DESKTOP_HOME` at `%LOCALAPPDATA%` (inherited AppContainer ACEs fail the engine's private-DACL check).
+
+## Status
+
+**Phases 0–2 complete (Phase 0 accepted 2026-11-06; takeover UI verified live 2026-11-07).**
+
+| Phase | Deliverable | Status |
+| --- | --- | --- |
+| 0 | Evaluation spike: raw `agent-desktop mcp` row, drive Notepad/Explorer/Chrome — speed ✓, readable trees ✓, refs-not-pixels ✓ | ✅ accepted, spike removed |
+| 1 | Adapter MCP server: Lite 16 tools, per-conversation session lifecycle, headed toggle + foreground promotion, screenshot→file path, WinRT OCR, skill | ✅ built, installed, driven in-host |
+| 2 | Takeover UI: window rim + banner + Esc kill, configurable ghost cursor, settings page, chat step rows, driver lock, English UI | ✅ built + verified live |
+| 3 | Web lane (`arya_browser` row), sign-in profiles, attach-to-your-Chrome, **wincu retirement** | planned |
+| 4 | Lazy preset, notifications/clipboard/trace, npm publish | planned |
+
+Lite 16 tool surface: `snapshot, find, get, is, click, set_value, type, press, scroll, list_windows, launch, wait, screenshot, ocr, batch, tools` (superset of wincu Lite 12; `set_value` fills a field through the accessibility value pattern without touching the pointer; `type` follows the input lane (background by default, real keyboard only when that conversation opts in); `tools` manages headed/overlay/session state).
+
+Full plan: [docs/PLAN.md](docs/PLAN.md).
+
+## Layout
+
+```
+package.json          Cordis bundle manifest (dsh.bundle.patch) + pinned engine deps
+cordis.patch.yml      rows: arya-computer-use (Host), mcp-arya-desktop, mcp-arya-browser
+index.js              Host plugin entry (settings, driver lock, takeover mgmt, status route)
+client.js             Settings page + chat step rows (client plugin)
+mcp/adapter-server.mjs  adapter MCP server (Lite 16 tool surface)
+scripts/windows-takeover.cs  window rim + banner + progress + ghost cursor + global Esc
+scripts/ocr.ps1       WinRT OCR helper (Phase 1)
+skills/arya-desktop/  observe-act loop guide (Phase 1)
+locale/ icon.svg      Plugin Manager display metadata
+docs/PLAN.md          complete v1 plan
+```
+
+## Install
+
+### From the plugin market (recommended)
+
+1. In AryaCode: **Plugins → Arya Plugin Market → add a source** and paste
+   `https://cdn.jsdelivr.net/gh/KonnectSuite/dsh-arya-computer-use@main/index.json`
+2. Install **Arya Computer Use** from the list, then restart AryaCode.
+
+The published package is `@aryacode/dsh-plugin-arya-computer-use` (npm, public). Everything it needs ships in the tarball: the desktop adapter, the browser launcher, the takeover overlay sources and the skill. The two MCP lanes are mounted **in-process** by `index.js` (`ctx.plugin(<bridge>, …)`), so there are no absolute paths in the bundle patch and no `npx` child process — a 1.0.0 install works the same on any machine.
+
+### From a checkout (development)
+
+`plugin_manager` → `install_bundle` pointing at this directory. The bundle patch is a single row (`arya-computer-use`); the MCP lanes still mount in-process, so a checkout install and a registry install behave identically.
+
+## Settings (Settings → Plugins → Computer Use)
+
+| Setting | Default | Meaning |
+| --- | --- | --- |
+| Frame the app window | on | Rim only the driven window plus a banner; monitor edges are just the fallback before a target is known |
+| Show a ghost cursor | on | Draw a synthetic cursor where the agent acts; your pointer is never followed |
+| Agent name | `Arya` | Drives the ghost-cursor tag and the progress-panel header (`Arya · Step 3 · acting`), 1–16 characters |
+| Banner text | `Arya is controlling your computer` | One line, 1–120 characters |
+| Real mouse and keyboard | off | Opted-in headed input for one conversation; the background lane never moves the pointer |
+| One conversation drives at a time | on | Refuses desktop tools in other conversations while a drive is live |
+| Stop shortcut | Esc | Global kill switch; synthetic keystrokes are ignored, so it needs a real keypress |
+| Tool preset | `lite` | `lite` lists all 16 desktop tools; `lazy` starts with tools/snapshot/click and reveals the rest on demand |
+| Tool surface (web) | `core` | agent-browser tool profile: core 29, network 9, state 27, debug 40, tabs 13, react 8, mobile 15, webmcp 4, all 156 |
+| Chrome profile / Session | – / `arya` | Reuse a Chrome login state and name the browser session |
+| Connect to my browser | – | `--cdp` port or URL to attach to a Chrome you already have open |
+| Saved sign-in state | – | Absolute path to a cookies+localStorage JSON to reuse |
+| Show the browser window | off | `--headed` for the web lane |
+
+## Web lane
+
+The second MCP row (`arya_browser`, server `agent-browser`) runs through `mcp/browser-launcher.mjs`, a transparent stdio pipe that reads the control file and spawns the real server with the settings applied:
+
+```
+node mcp/browser-launcher.mjs          # → agent-browser mcp --tools core [--profile X] [--session Y] [--state Z] [--cdp ...] [--headed]
+```
+
+So sign-in reuse and attaching to your own Chrome are settings, not hand-edited rows. Web-lane changes land on the next MCP restart; the settings page says so.
+
+## Tool presets
+
+- `lite` (default) — all 16 desktop tools listed up front.
+- `lazy` — starts with `tools`, `snapshot`, `click`; everything else appears after `tools {operation:"load", names:[...]}`. The adapter advertises `tools.listChanged` and emits `notifications/tools/list_changed`, which DSH's MCP client re-syncs (`listChanged.onChanged → refreshTools`), so the new tools become callable immediately.
+
+## License
+
+MIT (KonnectSuite original code). Engines stay Apache-2.0 — see [NOTICE.md](NOTICE.md).
+
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.