@zhihand/mcp 0.12.0 → 0.12.1

package/README.md

@@ -0,0 +1,288 @@
+# @zhihand/mcp
+
+ZhiHand MCP Server — let AI agents see and control your phone.
+
+Version: `0.12.1`
+
+## What is this?
+
+`@zhihand/mcp` is the core integration layer for ZhiHand. It provides an [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) server that exposes phone control tools to any compatible AI agent, including:
+
+- **Claude Code**
+- **Codex CLI**
+- **Gemini CLI**
+- **OpenClaw**
+
+One npm package, two entry points:
+
+| Entry | Purpose |
+|---|---|
+| `zhihand serve` | MCP Server (stdio) — used by Claude Code, Codex, Gemini CLI |
+| `zhihand.openclaw` | OpenClaw Plugin entry — thin wrapper calling the same core |
+
+## Requirements
+
+- **Node.js >= 22**
+- A **ZhiHand mobile app** (Android or iOS) installed on your phone
+
+## Installation
+
+```bash
+npm install -g @zhihand/mcp
+```
+
+Or use directly with `npx`:
+
+```bash
+npx @zhihand/mcp serve
+```
+
+## Quick Start
+
+### 1. Pair your phone
+
+```bash
+zhihand setup
+```
+
+This runs the full interactive setup:
+
+1. Registers as a plugin with the ZhiHand server
+2. Creates a pairing session and displays a QR code in the terminal
+3. Waits for you to scan the QR code with the ZhiHand mobile app
+4. Saves credentials to `~/.zhihand/credentials.json`
+5. Detects installed CLI tools (Claude Code, Codex, Gemini CLI, OpenClaw)
+6. Prints the MCP configuration snippet for your tools
+
+### 2. Configure your AI tool
+
+Add the ZhiHand MCP server to your tool's configuration:
+
+**Claude Code** — Add to `.mcp.json` in your project root, or run:
+
+```bash
+claude mcp add zhihand -- zhihand serve
+```
+
+Or manually create/edit `.mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "zhihand": {
+      "command": "zhihand",
+      "args": ["serve"]
+    }
+  }
+}
+```
+
+**Codex CLI** — Add to your MCP config:
+
+```json
+{
+  "mcpServers": {
+    "zhihand": {
+      "command": "zhihand",
+      "args": ["serve"]
+    }
+  }
+}
+```
+
+**Gemini CLI** — Add to `~/.gemini/settings.json`:
+
+```json
+{
+  "mcpServers": {
+    "zhihand": {
+      "command": "zhihand",
+      "args": ["serve"]
+    }
+  }
+}
+```
+
+**OpenClaw** — Install the plugin directly:
+
+```bash
+openclaw plugins install @zhihand/mcp
+```
+
+### 3. Start using it
+
+Once configured, your AI agent can use ZhiHand tools directly. For example, in Claude Code:
+
+```
+> Take a screenshot of my phone
+> Tap on the Settings icon
+> Type "hello world" into the search box
+> Scroll down to find the About section
+```
+
+## CLI Commands
+
+```
+zhihand serve              Start MCP Server (stdio mode, called by AI tools)
+zhihand setup              Interactive setup: pair + detect tools + print config
+zhihand pair               Pair with a phone (QR code in terminal)
+zhihand status             Show current pairing status and device info
+zhihand detect             List detected CLI tools and their login status
+zhihand --help             Show help
+```
+
+### Options
+
+| Option | Description |
+|---|---|
+| `--device <name>` | Use a specific paired device (if you have multiple) |
+| `-h, --help` | Show help |
+
+### Environment Variables
+
+| Variable | Description |
+|---|---|
+| `ZHIHAND_DEVICE` | Default device name (same as `--device`) |
+| `ZHIHAND_CLI` | Override CLI tool selection for mobile-initiated tasks |
+
+## MCP Tools
+
+The server exposes three tools to AI agents:
+
+### `zhihand_control`
+
+The main phone control tool. Supports these actions:
+
+| Action | Parameters | Description |
+|---|---|---|
+| `click` | `xRatio`, `yRatio` | Tap at normalized coordinates [0,1] |
+| `doubleclick` | `xRatio`, `yRatio` | Double-tap |
+| `rightclick` | `xRatio`, `yRatio` | Right-click (long press) |
+| `middleclick` | `xRatio`, `yRatio` | Middle-click |
+| `type` | `text` | Type text into the focused field |
+| `swipe` | `startXRatio`, `startYRatio`, `endXRatio`, `endYRatio` | Swipe gesture |
+| `scroll` | `xRatio`, `yRatio`, `direction`, `amount` | Scroll up/down/left/right |
+| `keycombo` | `keys` | Key combination (e.g. `"ctrl+c"`, `"alt+tab"`) |
+| `clipboard` | `clipboardAction` (`get`/`set`), `text` | Read or write clipboard |
+| `wait` | `durationMs` | Wait (local sleep, no server round-trip) |
+| `screenshot` | — | Capture screen immediately |
+
+Coordinates use **normalized ratios** (0.0 to 1.0), where `(0, 0)` is the top-left corner and `(1, 1)` is the bottom-right. This works across any screen resolution.
+
+Every action returns a text summary and a screenshot of the result.
+
+### `zhihand_screenshot`
+
+Capture the current phone screen without performing any action. Returns an image.
+
+No parameters required.
+
+### `zhihand_pair`
+
+Pair with a phone device. Returns a QR code and pairing URL.
+
+| Parameter | Type | Description |
+|---|---|---|
+| `forceNew` | `boolean` | Force new pairing even if already paired (default: `false`) |
+
+## How It Works
+
+```
+AI Agent ←stdio→ zhihand serve (MCP Server)
+                       │
+                       ├── POST /v1/plugins           Register plugin
+                       ├── POST /v1/pairing/sessions   Create pairing
+                       ├── POST /v1/credentials/{id}/commands   Send command
+                       ├── GET  /v1/credentials/{id}/commands/{cid}   Poll ACK
+                       ├── SSE  /v1/credentials/{id}/events?topic=commands   Real-time ACK
+                       └── GET  /v1/credentials/{id}/screen   Fetch screenshot (JPEG)
+                       │
+                  ZhiHand Server
+                       │
+                  ZhiHand Mobile App
+```
+
+1. AI agent calls a tool (e.g. `zhihand_control` with `action: "click"`)
+2. MCP Server translates to a device command and enqueues it via the ZhiHand API
+3. Mobile app picks up the command, executes it, and sends an ACK
+4. MCP Server receives the ACK (via SSE or polling fallback)
+5. MCP Server fetches a fresh screenshot and returns it to the AI agent
+
+Screenshots are transferred as raw JPEG binary and only base64-encoded at the LLM API boundary, minimizing bandwidth.
+
+## Credential Storage
+
+Pairing credentials are stored at:
+
+```
+~/.zhihand/
+├── credentials.json    # Device credentials (credentialId, controllerToken, endpoint)
+└── state.json          # Current pairing session state
+```
+
+You can manage multiple devices. The `credentials.json` file stores a `default` device name and a `devices` map:
+
+```json
+{
+  "default": "mcp-myhost",
+  "devices": {
+    "mcp-myhost": {
+      "credentialId": "cred_abc123",
+      "controllerToken": "tok_...",
+      "endpoint": "https://api.zhihand.com",
+      "deviceName": "mcp-myhost",
+      "pairedAt": "2026-04-01T00:00:00.000Z"
+    }
+  }
+}
+```
+
+## Architecture
+
+```
+packages/mcp/
+├── bin/
+│   ├── zhihand              # Main CLI entry (serve/setup/pair/status/detect)
+│   └── zhihand.openclaw     # OpenClaw plugin entry
+├── src/
+│   ├── index.ts             # MCP Server (stdio transport)
+│   ├── openclaw.adapter.ts  # OpenClaw Plugin adapter (thin wrapper)
+│   ├── core/
+│   │   ├── config.ts        # Credential & config management (~/.zhihand/)
+│   │   ├── command.ts       # Command creation, enqueue, ACK formatting
+│   │   ├── screenshot.ts    # Binary screenshot fetch (JPEG)
+│   │   ├── sse.ts           # SSE client + hybrid ACK (SSE push + polling fallback)
+│   │   └── pair.ts          # Plugin registration + device pairing flow
+│   ├── tools/
+│   │   ├── schemas.ts       # Zod parameter schemas
+│   │   ├── control.ts       # zhihand_control handler
+│   │   ├── screenshot.ts    # zhihand_screenshot handler
+│   │   └── pair.ts          # zhihand_pair handler
+│   └── cli/
+│       ├── detect.ts        # CLI tool detection (Claude Code, Codex, Gemini, OpenClaw)
+│       ├── spawn.ts         # CLI process spawning (for mobile-initiated tasks)
+│       └── openclaw.ts      # OpenClaw auto-detect & plugin install
+├── dist/                    # Compiled JavaScript (shipped in npm package)
+├── package.json
+└── tsconfig.json
+```
+
+## Development
+
+```bash
+# Install dependencies
+npm install
+
+# Build (compiles TypeScript to dist/)
+npm run build
+
+# Run in development mode (uses --experimental-strip-types)
+npm run dev
+
+# Run tests
+npm test
+```
+
+## License
+
+MIT

package/bin/zhihand

@@ -1,12 +1,12 @@
-#!/usr/bin/env node --experimental-strip-types
+#!/usr/bin/env node
 
 import os from "node:os";
 import { parseArgs } from "node:util";
-import { startStdioServer } from "../src/index.ts";
-import { detectCLITools, formatDetectedTools } from "../src/cli/detect.ts";
-import { detectAndSetupOpenClaw } from "../src/cli/openclaw.ts";
-import { loadDefaultCredential } from "../src/core/config.ts";
-import { executePairing } from "../src/core/pair.ts";
+import { startStdioServer } from "../dist/index.js";
+import { detectCLITools, formatDetectedTools } from "../dist/cli/detect.js";
+import { detectAndSetupOpenClaw } from "../dist/cli/openclaw.js";
+import { loadDefaultCredential } from "../dist/core/config.js";
+import { executePairing } from "../dist/core/pair.js";
 
 const DEFAULT_ENDPOINT = "https://api.zhihand.com";
 

package/bin/zhihand.openclaw

@@ -1,11 +1,11 @@
-#!/usr/bin/env node --experimental-strip-types
+#!/usr/bin/env node
 
 /**
  * OpenClaw Plugin entry point.
  * This script is invoked by OpenClaw when the zhihand plugin is loaded.
  * It bridges the OpenClaw Plugin API to MCP core logic via the adapter.
  */
-import { registerOpenClawTools } from "../src/openclaw.adapter.ts";
+import { registerOpenClawTools } from "../dist/openclaw.adapter.js";
 
 // OpenClaw injects the plugin API as the default export's argument
 export default function activate(api) {

package/dist/cli/detect.d.ts

@@ -0,0 +1,10 @@
+export interface CLITool {
+    name: "claudecode" | "codex" | "gemini" | "openclaw";
+    command: string;
+    version: string;
+    loggedIn: boolean;
+    priority: number;
+}
+export declare function detectCLITools(): Promise<CLITool[]>;
+export declare function detectBestCLI(): Promise<CLITool | null>;
+export declare function formatDetectedTools(tools: CLITool[]): string;

package/dist/cli/detect.js

@@ -0,0 +1,75 @@
+import { execSync } from "node:child_process";
+function tryExec(cmd) {
+    try {
+        return execSync(cmd, { encoding: "utf8", timeout: 5000, stdio: ["pipe", "pipe", "pipe"] }).trim();
+    }
+    catch {
+        return null;
+    }
+}
+function isCommandAvailable(cmd) {
+    return tryExec(`which ${cmd}`) !== null;
+}
+async function detectClaudeCode() {
+    if (!isCommandAvailable("claude"))
+        return null;
+    const version = tryExec("claude --version") ?? "unknown";
+    // Check login: claude has config in ~/.claude/
+    const loggedIn = tryExec("ls ~/.claude/settings.json") !== null;
+    return { name: "claudecode", command: "claude", version, loggedIn, priority: 1 };
+}
+async function detectCodex() {
+    if (!isCommandAvailable("codex"))
+        return null;
+    const version = tryExec("codex --version") ?? "unknown";
+    // Check login: OPENAI_API_KEY env var or config
+    const loggedIn = !!process.env.OPENAI_API_KEY || tryExec("ls ~/.codex/") !== null;
+    return { name: "codex", command: "codex", version, loggedIn, priority: 2 };
+}
+async function detectGemini() {
+    if (!isCommandAvailable("gemini"))
+        return null;
+    const version = tryExec("gemini --version") ?? "unknown";
+    // Check login: Google Cloud auth
+    const loggedIn = tryExec("gemini auth status") !== null;
+    return { name: "gemini", command: "gemini", version, loggedIn, priority: 3 };
+}
+async function detectOpenClaw() {
+    if (!isCommandAvailable("openclaw"))
+        return null;
+    const version = tryExec("openclaw --version") ?? "unknown";
+    const loggedIn = tryExec("ls ~/.openclaw/openclaw.json") !== null;
+    return { name: "openclaw", command: "openclaw", version, loggedIn, priority: 4 };
+}
+export async function detectCLITools() {
+    const results = await Promise.allSettled([
+        detectClaudeCode(),
+        detectCodex(),
+        detectGemini(),
+        detectOpenClaw(),
+    ]);
+    return results
+        .filter((r) => r.status === "fulfilled")
+        .map((r) => r.value)
+        .filter((t) => t !== null)
+        .sort((a, b) => a.priority - b.priority);
+}
+export async function detectBestCLI() {
+    const cliOverride = process.env.ZHIHAND_CLI;
+    const tools = await detectCLITools();
+    if (cliOverride) {
+        const match = tools.find((t) => t.name === cliOverride || t.command === cliOverride);
+        if (match)
+            return match;
+    }
+    // Return best available tool (logged in + highest priority)
+    return tools.find((t) => t.loggedIn) ?? tools[0] ?? null;
+}
+export function formatDetectedTools(tools) {
+    if (tools.length === 0)
+        return "No CLI tools detected.";
+    return [
+        "Detected CLI tools:",
+        ...tools.map((t) => `  ${t.loggedIn ? "✓" : "✗"} ${t.name} (${t.command} ${t.version})${t.loggedIn ? "" : " — not logged in"}`),
+    ].join("\n");
+}

package/dist/cli/openclaw.d.ts

@@ -0,0 +1,6 @@
+export declare function isZhiHandPluginInstalled(): Promise<boolean>;
+export declare function installZhiHandPlugin(options?: {
+    timeoutMs?: number;
+    autoConfirm?: boolean;
+}): Promise<boolean>;
+export declare function detectAndSetupOpenClaw(): Promise<void>;

package/dist/cli/openclaw.js

@@ -0,0 +1,47 @@
+import { execSync } from "node:child_process";
+function tryExec(cmd) {
+    try {
+        return execSync(cmd, { encoding: "utf8", timeout: 30_000, stdio: ["pipe", "pipe", "pipe"] }).trim();
+    }
+    catch {
+        return null;
+    }
+}
+function isCommandAvailable(cmd) {
+    return tryExec(`which ${cmd}`) !== null;
+}
+export async function isZhiHandPluginInstalled() {
+    const output = tryExec("openclaw plugins list");
+    if (!output)
+        return false;
+    return output.includes("zhihand") || output.includes("@zhihand/mcp");
+}
+export async function installZhiHandPlugin(options = {}) {
+    const timeout = options.timeoutMs ?? 30_000;
+    try {
+        execSync("openclaw plugins install @zhihand/mcp", {
+            encoding: "utf8",
+            timeout,
+            stdio: options.autoConfirm ? ["pipe", "pipe", "pipe"] : "inherit",
+        });
+        return true;
+    }
+    catch {
+        return false;
+    }
+}
+export async function detectAndSetupOpenClaw() {
+    if (!isCommandAvailable("openclaw"))
+        return;
+    const pluginInstalled = await isZhiHandPluginInstalled();
+    if (pluginInstalled)
+        return;
+    process.stderr.write("[zhihand] Detected OpenClaw without ZhiHand plugin. Installing...\n");
+    const success = await installZhiHandPlugin({ timeoutMs: 30_000, autoConfirm: true });
+    if (success) {
+        process.stderr.write("[zhihand] ZhiHand plugin installed to OpenClaw.\n");
+    }
+    else {
+        process.stderr.write("[zhihand] Failed to install ZhiHand plugin to OpenClaw.\n");
+    }
+}

package/dist/cli/spawn.d.ts

@@ -0,0 +1,2 @@
+import type { CLITool } from "./detect.ts";
+export declare function spawnCLITask(tool: CLITool, prompt: string): Promise<string>;

package/dist/cli/spawn.js

@@ -0,0 +1,31 @@
+import { execSync } from "node:child_process";
+function shellEscape(s) {
+    return `'${s.replace(/'/g, "'\\''")}'`;
+}
+export async function spawnCLITask(tool, prompt) {
+    const escaped = shellEscape(prompt);
+    switch (tool.name) {
+        case "claudecode":
+            return execSync(`${tool.command} -p ${escaped} --output-format json`, {
+                encoding: "utf8",
+                timeout: 300_000,
+            });
+        case "codex":
+            return execSync(`${tool.command} -q ${escaped} --json`, {
+                encoding: "utf8",
+                timeout: 300_000,
+            });
+        case "gemini":
+            return execSync(`${tool.command} -p ${escaped}`, {
+                encoding: "utf8",
+                timeout: 300_000,
+            });
+        case "openclaw":
+            return execSync(`${tool.command} run ${escaped}`, {
+                encoding: "utf8",
+                timeout: 300_000,
+            });
+        default:
+            throw new Error(`Unsupported CLI tool: ${tool.name}`);
+    }
+}

package/dist/core/command.d.ts

@@ -0,0 +1,41 @@
+import type { ZhiHandConfig } from "./config.ts";
+export type ScrollDirection = "up" | "down" | "left" | "right";
+export type ClipboardAction = "get" | "set";
+export interface ControlParams {
+    action: string;
+    xRatio?: number;
+    yRatio?: number;
+    text?: string;
+    direction?: ScrollDirection;
+    amount?: number;
+    keys?: string;
+    clipboardAction?: ClipboardAction;
+    durationMs?: number;
+    startXRatio?: number;
+    startYRatio?: number;
+    endXRatio?: number;
+    endYRatio?: number;
+}
+export interface QueuedControlCommand {
+    type: string;
+    payload?: Record<string, unknown>;
+    messageId?: number;
+}
+export interface QueuedCommandRecord {
+    id: string;
+    credential_id: string;
+    status: string;
+    command: QueuedControlCommand;
+    created_at: string;
+    acked_at?: string;
+    ack_status?: string;
+    ack_result?: Record<string, unknown>;
+}
+export interface WaitForCommandAckResult {
+    acked: boolean;
+    command?: QueuedCommandRecord;
+}
+export declare function createControlCommand(params: ControlParams): QueuedControlCommand;
+export declare function enqueueCommand(config: ZhiHandConfig, command: QueuedControlCommand): Promise<QueuedCommandRecord>;
+export declare function getCommand(config: ZhiHandConfig, commandId: string): Promise<QueuedCommandRecord>;
+export declare function formatAckSummary(action: string, result: WaitForCommandAckResult): string;

package/dist/core/command.js

@@ -0,0 +1,84 @@
+let messageCounter = 0;
+function nextMessageId() {
+    messageCounter = (messageCounter + 1) % 1000;
+    return (Date.now() * 1000) + messageCounter;
+}
+export function createControlCommand(params) {
+    switch (params.action) {
+        case "click":
+            return { type: "receive_click", payload: { x: params.xRatio, y: params.yRatio } };
+        case "doubleclick":
+            return { type: "receive_doubleclick", payload: { x: params.xRatio, y: params.yRatio } };
+        case "rightclick":
+            return { type: "receive_rightclick", payload: { x: params.xRatio, y: params.yRatio } };
+        case "middleclick":
+            return { type: "receive_middleclick", payload: { x: params.xRatio, y: params.yRatio } };
+        case "type":
+            return { type: "receive_type", payload: { text: params.text } };
+        case "swipe":
+            return {
+                type: "receive_swipe",
+                payload: {
+                    startX: params.startXRatio,
+                    startY: params.startYRatio,
+                    endX: params.endXRatio,
+                    endY: params.endYRatio,
+                },
+            };
+        case "scroll":
+            return {
+                type: "receive_scroll",
+                payload: {
+                    x: params.xRatio,
+                    y: params.yRatio,
+                    direction: params.direction,
+                    amount: params.amount ?? 3,
+                },
+            };
+        case "keycombo":
+            return { type: "receive_keycombo", payload: { keys: params.keys } };
+        case "clipboard":
+            return {
+                type: "receive_clipboard",
+                payload: { action: params.clipboardAction, text: params.text },
+            };
+        case "screenshot":
+            return { type: "receive_screenshot", payload: {} };
+        default:
+            throw new Error(`Unsupported action: ${params.action}`);
+    }
+}
+export async function enqueueCommand(config, command) {
+    const response = await fetch(`${config.controlPlaneEndpoint}/v1/credentials/${encodeURIComponent(config.credentialId)}/commands`, {
+        method: "POST",
+        headers: {
+            "Content-Type": "application/json",
+            "x-zhihand-controller-token": config.controllerToken,
+        },
+        body: JSON.stringify({
+            command: { ...command, message_id: command.messageId ?? nextMessageId() },
+        }),
+    });
+    if (!response.ok) {
+        throw new Error(`Enqueue command failed: ${response.status}`);
+    }
+    const payload = (await response.json());
+    return payload.command;
+}
+export async function getCommand(config, commandId) {
+    const response = await fetch(`${config.controlPlaneEndpoint}/v1/credentials/${encodeURIComponent(config.credentialId)}/commands/${encodeURIComponent(commandId)}`, {
+        headers: { "x-zhihand-controller-token": config.controllerToken },
+    });
+    if (!response.ok) {
+        throw new Error(`Get command failed: ${response.status}`);
+    }
+    const payload = (await response.json());
+    return payload.command;
+}
+export function formatAckSummary(action, result) {
+    if (!result.acked) {
+        return `Sent ${action}, waiting for ACK (timed out).`;
+    }
+    const ackStatus = result.command?.ack_status ?? "ok";
+    return `Sent ${action}. ACK: ${ackStatus}`;
+}

package/dist/core/config.d.ts

@@ -0,0 +1,26 @@
+export interface DeviceCredential {
+    credentialId: string;
+    controllerToken: string;
+    endpoint: string;
+    deviceName?: string;
+    pairedAt?: string;
+}
+export interface CredentialStore {
+    default: string;
+    devices: Record<string, DeviceCredential>;
+}
+export interface ZhiHandConfig {
+    controlPlaneEndpoint: string;
+    credentialId: string;
+    controllerToken: string;
+    edgeId?: string;
+    timeoutMs?: number;
+}
+export declare function resolveZhiHandDir(): string;
+export declare function ensureZhiHandDir(): void;
+export declare function loadCredentialStore(): CredentialStore | null;
+export declare function loadDefaultCredential(): DeviceCredential | null;
+export declare function saveCredential(name: string, cred: DeviceCredential, setDefault?: boolean): void;
+export declare function resolveConfig(deviceName?: string): ZhiHandConfig;
+export declare function loadState<T = unknown>(): T | null;
+export declare function saveState(state: unknown): void;