@beeos-ai/cli 1.0.24 → 1.1.0

package/README.md

@@ -14,14 +14,32 @@ npm install -g @beeos-ai/cli
 
 Pure JavaScript — runs on any platform with Node.js 18+. No native compilation needed.
 
+## What's new in 1.1.0
+
+- **Multi-OS desktop streaming** for the OpenClaw agent: macOS Screen
+  Sharing, Linux TigerVNC and Windows TightVNC are now first-class
+  cold-start targets. `beeos init` configures the host's VNC password,
+  generates `~/.beeos/vnc.password`, and stages a `vnc-bridge` sidecar
+  in one shot.
+- `beeos device attach --self` (experimental) — tags the persisted
+  `DeviceEntry.backend` with the host's native backend (`macos` /
+  `linux` / `windows`) instead of the historical `adb`. Forward-
+  compatible with the upcoming Phase-4 self-attach lifecycle.
+- `BEEOS_NO_DESKTOP=1` opt-out — single chokepoint at the desktop
+  pipeline. When set, no probe, no cold-start, no UAC / osascript
+  prompt, no TightVNC / TigerVNC / Screen Sharing detection. Useful
+  for CI runners and headless servers.
+- Linux fleet auto-enable prompt is now offered on Linux too (was
+  darwin-only).
+
 ## Commands
 
 ### Device agent (Android)
 
 ```bash
 # Attach a USB-connected Android device as an AI agent (launches
-# device-agent + scrcpy-bridge in parallel — the latter powers WebRTC
-# video / control)
+# device-agent + device-mcp-server + scrcpy-bridge in parallel —
+# the latter powers WebRTC video / control)
 beeos device attach
 
 # Attach a specific device
@@ -30,47 +48,58 @@ beeos device attach --serial SERIAL_NUMBER
 # Attach all connected devices
 beeos device attach --all
 
-# Skip the scrcpy-bridge sidecar; run only device-agent (useful for
-# iterating on ACP / Python code without the video stream)
+# Skip the scrcpy-bridge sidecar; run only device-agent + MCP server
+# (useful for iterating on tools / ACP without the video stream)
 beeos device attach --no-video
 
-# List attached devices (shows both device-agent and scrcpy-bridge PIDs)
+# Attach the host machine itself (experimental, multi-OS) — tags the
+# persisted backend as macos / linux / windows depending on the host
+beeos device attach --self
+
+# List attached devices
 beeos device list --local
 
-# Detach a device (stops device-agent AND scrcpy-bridge)
+# Detach a device (stops every sibling process spawned for that device)
 beeos device detach --serial SERIAL_NUMBER
 
 # Send a command to a device
 beeos device exec "open Chrome"
 
-# Upgrade device-agent to latest version
+# Upgrade device-agent + device-mcp-server to latest version
 beeos device upgrade
 ```
 
 #### What `attach` does
 
-Under the hood `beeos device attach` orchestrates two sibling processes
-for each device:
-
-1. **device-agent** (Python, ACP over WebSocket) — AI chat, tool
-   execution, file operations. Binds the phone to your BeeOS account
-   and writes identity to `~/.beeos/identity/device-<serial>.key.json`.
-2. **scrcpy-bridge** (Rust, WebRTC) — mirrors screen + audio and
-   forwards touch / key input. Shares the **same** `.key.json` identity
-   file (both `BRIDGE_PRIVATE_KEY_FILE` and `BRIDGE_PUBLIC_KEY_FILE`
-   are set to it) and authenticates to EMQX via Agent Gateway's
-   `/api/v1/device/bootstrap` endpoint — no static MQTT tokens.
-
-Both processes are tracked in `~/.beeos/devices.json`; `detach` and
-`list` operate on the pair atomically. If the scrcpy-bridge binary
-isn't yet installed on your machine, it is downloaded on first attach
-from GitHub Releases into `~/.beeos/bin/` (set
-`$BEEOS_SCRCPY_BRIDGE_BIN` to point at a pre-built binary to skip the
-download, or `$BEEOS_SCRCPY_BRIDGE_RELEASE_URL` for an internal
-mirror).
+Under the hood `beeos device attach` orchestrates **three sibling
+processes** for each Android device:
+
+1. **device-agent** (TypeScript, Node 20 — `@beeos-ai/device-agent`)
+   — Plan-Act-Reflect agent loop, ACP chat, and orchestration. Binds
+   the phone to your BeeOS account and writes identity to
+   `~/.beeos/identity/device-<serial>.key.json`.
+2. **device-mcp-server** (TypeScript, Node 20 —
+   `@beeos-ai/device-mcp-server`) — Fastify HTTP MCP server backing
+   `screenshot` / `tap` / `swipe` / `install_apk` / `execute_command`
+   and ~14 more tools. The agent talks to it on a private loopback
+   port allocated per device (default range starts at `127.0.0.1:7900`).
+3. **scrcpy-bridge** (Rust, WebRTC) — mirrors screen + audio and
+   forwards touch / key input. Shares the **same** `.key.json`
+   identity file the agent uses, and authenticates to EMQX via Agent
+   Gateway's `/api/v1/device/bootstrap` endpoint — no static MQTT
+   tokens.
+
+The three processes are independent — one crashing does not take down
+the others. All three are tracked in `~/.beeos/devices.json`; `detach`
+and `list` operate on the trio atomically. If `device-agent` /
+`device-mcp-server` aren't yet installed, the CLI runs
+`npm i -g @beeos-ai/device-agent @beeos-ai/device-mcp-server` on first
+use. If the scrcpy-bridge binary is missing, it is downloaded on first
+attach from GitHub Releases into `~/.beeos/bin/`. Override paths via
+`BEEOS_DEVICE_AGENT_BIN`, `BEEOS_SCRCPY_BRIDGE_BIN`, etc.
 
 When running against a local dev control plane (e.g. `api_url =
-http://localhost:9080`), the CLI automatically points scrcpy-bridge at
+http://localhost:9080`), the CLI automatically points the bridge at
 `http://localhost:8083` unless `platform.agent_gateway_url` is
 explicitly set to something non-default. Set
 `BEEOS_AGENT_GATEWAY_URL` to override for one-off experiments without
@@ -92,15 +121,47 @@ beeos status
 beeos update openclaw
 ```
 
+#### Multi-OS desktop streaming
+
+Starting in 1.1.0, `beeos init` (and `beeos start openclaw`) drives a
+host-aware desktop cold-start so the BeeOS dashboard's desktop view
+works on all three major operating systems:
+
+| OS      | What gets configured                                                                                | Auth dialog            |
+| ------- | --------------------------------------------------------------------------------------------------- | ---------------------- |
+| macOS   | `com.apple.screensharing` enabled; `ARD_AllLocalUsers=NO`; VNC password written to plist + `~/.beeos/vnc.password` | osascript admin prompt |
+| Linux   | `~/.vnc/passwd` generated via `tigervncpasswd -f`; `vncserver :1 -localhost yes` started; mirrored to `~/.beeos/vnc.password` | none (no sudo)         |
+| Windows | TightVNC registry password (REG_BINARY DES) + `LoopbackOnly=1`; `tvnserver` restarted; mirrored to `~/.beeos/vnc.password` | UAC consent dialog     |
+
+The flow is **idempotent**: re-running `beeos init` on an
+already-configured host is a no-op. If the cold-start can't run (no
+TigerVNC on Linux, no TightVNC on Windows, user declined the prompt,
+…) the bind still succeeds and OpenClaw works headlessly — only the
+dashboard's desktop view is unavailable.
+
+To opt out entirely, set `BEEOS_NO_DESKTOP=1` before running
+`beeos init`. The desktop pipeline returns null at its single
+chokepoint (`getDesktopPipeline()`), so no probe, no prompt, no
+cold-start fires. Useful for CI runners and pure headless servers.
+
 ## Requirements
 
 - **Node.js 18+**
-- **Android devices**: `adb` (Android Debug Bridge) must be in your PATH
-- **device-agent**: Python 3.11+ or [uv](https://astral.sh/uv) (auto-installed on first use)
+- **Android devices**: `adb` (Android Debug Bridge) on PATH (the CLI
+  can install Google's official platform-tools on first use)
+- **device-agent + device-mcp-server**: TypeScript / Node 20+. The
+  CLI lazy-installs `@beeos-ai/device-agent` and
+  `@beeos-ai/device-mcp-server` on first `device attach` (~20 MB
+  one-time download). No Python required.
 - **scrcpy-bridge** (video streaming): auto-downloaded from GitHub
   Releases on first `device attach`. Supported targets: darwin
-  arm64/x64, linux arm64/x64, windows x64. On unsupported platforms
-  attach still succeeds — only video is skipped.
+  arm64/x64, linux arm64/x64, windows x64/arm64. On unsupported
+  platforms attach still succeeds — only video is skipped.
+- **Desktop streaming** (optional, OpenClaw): a VNC server on the
+  host. macOS ships Screen Sharing built-in; Linux needs `apt install
+  tigervnc-standalone-server` (or `dnf install tigervnc-server` /
+  `pacman -S tigervnc`); Windows needs `winget install
+  GlavSoft.TightVNC`.
 
 ## Configuration
 
@@ -118,9 +179,24 @@ http_enabled = true
 http_port = 9090
 ```
 
-`agent_gateway_url` is consumed by the scrcpy-bridge sidecar for
-bootstrap (Ed25519-signed `/api/v1/device/bootstrap` calls). Set it
-alongside `api_url` when pointing at a non-default control plane.
+`agent_gateway_url` is consumed by the scrcpy-bridge / vnc-bridge
+sidecars for bootstrap (Ed25519-signed `/api/v1/device/bootstrap`
+calls). Set it alongside `api_url` when pointing at a non-default
+control plane.
+
+### Environment variable overrides
+
+| Var                                | Effect                                                                                  |
+| ---------------------------------- | --------------------------------------------------------------------------------------- |
+| `BEEOS_NO_DESKTOP=1`               | Skip every desktop-streaming codepath. Honoured at `getDesktopPipeline()` and forwarded by `install.sh` / `install.ps1`. |
+| `BEEOS_VNC_PASSWORD`               | Per-shell override for the VNC viewer password. Tried before `~/.beeos/vnc.password`.    |
+| `BEEOS_AGENT_GATEWAY_URL`          | Override the Agent Gateway URL without editing `config.toml`.                            |
+| `BEEOS_DEVICE_AGENT_BIN`           | Absolute path to a pre-installed `device-agent` binary / JS entry.                       |
+| `BEEOS_SCRCPY_BRIDGE_BIN`          | Absolute path to a pre-built `scrcpy-bridge` binary.                                     |
+| `BEEOS_VNC_BRIDGE_BIN`             | Absolute path to a pre-built `vnc-bridge` binary.                                        |
+| `BEEOS_*_RELEASE_URL`              | Mirror URL for the matching cargo-dist sidecar (corporate proxies / staging).            |
+| `BEEOS_NO_FLEET_AUTOSTART=1`       | Suppress the interactive `device-agent fleet enable` prompt on macOS / Linux.            |
+| `BEEOS_INSTALL_DRY_RUN=1`          | Used by `install.sh` / `install.ps1` smoke tests; exits before any state mutation.       |
 
 ## Troubleshooting
 
@@ -148,3 +224,9 @@ debugging locally:
 In production, we rely on the TURN relay path; the mDNS issue is
 only a concern when running Chrome + scrcpy-bridge on the same host
 behind restrictive local firewalls.
+
+### Desktop view stuck / black for OpenClaw
+
+Run `beeos doctor` — it surfaces per-OS hints (TCC permission on
+macOS, `~/.vnc/xstartup` on Linux, `Get-Service tvnserver` on
+Windows) when the local `vnc-bridge-openclaw` service is unhealthy.