shellx-ai 1.1.1 → 1.1.2

package/README.md

@@ -1,666 +1,162 @@
-# ShellX AI
-
-<div align="center">
-
-  **TypeScript library for Android device automation via WebSocket**
-
-  [![npm version](https://badge.fury.io/js/shellx-ai.svg)](https://www.npmjs.org/package/shellx-ai)
-  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-  [![Node.js Version](https://img.shields.io/node/v/shellx-ai.svg)](https://nodejs.org)
-  [![TypeScript](https://img.shields.io/badge/TypeScript-5.8-blue.svg)](https://www.typescriptlang.org/)
-
-  [Features](#-features) • [Installation](#-installation) • [Quick Start](#-quick-start) • [API Reference](#-api-reference) • [Examples](#-examples)
-
-</div>
-
----
-
-## Summary
-
-ShellX AI is a TypeScript library that enables programmatic control of Android devices through a WebSocket connection. It provides a simple, type-safe API for:
-
-- **UI Automation** - Click, input, swipe, press keys, wait for elements
-- **Element Finding** - Find UI elements by text, ID, class, or coordinates
-- **Shell Commands** - Execute shell commands and get output
-- **Device Info** - Get screen info, app info, screenshots
-- **Batch Operations** - Execute multiple actions in sequence
-
----
-
-## Features
-
-- 🎯 **Simple API** - Intuitive methods for common automation tasks
-- 🔄 **Auto Retry** - Built-in retry logic for robust operations
-- 📝 **Type-Safe** - Full TypeScript support with comprehensive types
-- 🌍 **Cross-Platform** - Works in Node.js and browser environments
-- 📸 **Screenshots** - Capture screenshots with customizable options
-- 🔧 **Shell Commands** - Execute commands with real-time output
-- 🛠️ **Batch Actions** - Chain multiple operations together
-
----
-
-## Installation
-
-```bash
-npm install shellx-ai
-```
-
-### Requirements
-
-- Node.js >= 14.0.0
-- TypeScript >= 4.0.0 (recommended)
-- Android device connected to ShellX service
-
-### Optional Dependencies
-
-For Node.js environment, install ws:
-
-```bash
-npm install ws  # Optional, for Node.js WebSocket support
-```
-
-For browser environments, no additional dependencies needed.
-
----
-
-## Quick Start
-
-### 1. Basic Example
-
-```typescript
-import { ShellX } from 'shellx-ai';
-
-// Create ShellX instance
-const shellx = new ShellX({
-  deviceId: 'your-device-id'  // Replace with your device ID
-});
-
-// Wait for connection
-await shellx.ready();
-
-// Click element by text
-await shellx.click('Settings');
-
-// Get screen info
-const screen = await shellx.getScreenInfo();
-console.log(`Screen: ${screen.width}x${screen.height}`);
-
-// Execute shell command
-const result = await shellx.command('getprop ro.build.version.release');
-console.log('Android version:', result.output);
-```
-
-### 2. Using Environment Variables
-
-Create a `.env` file:
-
-```env
-DEVICE_ID=your-device-id
-```
-
-Then use it in your code:
-
-```typescript
-import { ShellX } from 'shellx-ai';
-import dotenv from 'dotenv';
-
-dotenv.config();
-
-const shellx = new ShellX({
-  deviceId: process.env.DEVICE_ID
-});
-
-await shellx.ready();
-// Start automating...
-```
-
-### 3. With Connection Events
-
-```typescript
-const shellx = new ShellX({
-  deviceId: 'your-device-id',
-  onOpen: () => console.log('✅ Connected'),
-  onClose: () => console.log('❌ Disconnected'),
-  onError: (error) => console.error('⚠️ Error:', error)
-});
-
-await shellx.ready();
-```
-
----
-
-## API Reference
-
-### ShellX Class
-
-The main class for Android automation.
-
-#### Constructor
-
-```typescript
-new ShellX(options: ShellXOptions)
-```
-
-**Options:**
-
-| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| `deviceId` | `string` | Yes | - | Device ID (UUID or identifier) |
-| `timeout` | `number` | No | `5000` | Connection timeout in ms |
-| `reconnect` | `boolean` | No | `true` | Enable auto-reconnect |
-| `reconnectMaxAttempts` | `number` | No | `5` | Max reconnect attempts |
-| `logLevel` | `LogLevel` | No | `INFO` | Log level (0=NONE, 1=ERROR, 2=WARN, 3=INFO, 4=DEBUG) |
-| `onOpen` | `() => void` | No | - | Callback when connection opens |
-| `onClose` | `() => void` | No | - | Callback when connection closes |
-| `onError` | `(error?) => void` | No | - | Callback on error |
-| `onMessage` | `(msg) => void` | No | - | Callback on message |
-
-#### Methods
-
-##### Connection
-
-| Method | Description | Returns |
-|--------|-------------|---------|
-| `ready()` | Wait for connection to be ready | `Promise<void>` |
-| `getClient()` | Get underlying ConnectionClient | `ConnectionClient` |
-
-##### UI Actions
-
-| Method | Description | Example |
-|--------|-------------|---------|
-| `click(selector)` | Click element | `await shellx.click('Submit')` |
-| `input(data)` | Input text | `await shellx.input({ text: 'Hello' })` |
-| `swipe(data)` | Swipe gesture | `await shellx.swipe({ fromX: 500, fromY: 1000, toX: 500, toY: 500 })` |
-| `press(key)` | Press key | `await shellx.press('BACK')` |
-| `wait(selector)` | Wait for element | `await shellx.wait('Loading')` |
-| `find(selector)` | Find elements | `await shellx.find('Button', { multiple: true })` |
-
-##### Device Operations
-
-| Method | Description | Example |
-|--------|-------------|---------|
-| `command(cmd)` | Execute shell command | `await shellx.command('ls -la')` |
-| `getScreenInfo()` | Get screen info | `await shellx.getScreenInfo()` |
-| `takeScreenshot()` | Take screenshot | `await shellx.takeScreenshot({ format: 'png' })` |
-| `getAppInfo(pkg)` | Get app info | `await shellx.getAppInfo('com.example.app')` |
-| `getAppList()` | Get app list | `await shellx.getAppList()` |
-| `clipboard(data)` | Clipboard operations | `await shellx.clipboard({ text: 'Hello' })` |
-
-##### Advanced
-
-| Method | Description | Example |
-|--------|-------------|---------|
-| `executeActions(actions)` | Execute multiple actions | `await shellx.executeActions([{ text: 'OK' }, { cmd: 'ls' }])` |
-| `sendRawMessage(msg)` | Send raw WebSocket message | `await shellx.sendRawMessage({ screenInfo: {} })` |
-
----
-
-## Type Definitions
-
-### Click
-
-Click on an element by text, ID, or coordinates.
-
-**Simplified form (click by text):**
-```typescript
-await shellx.click('Submit');
-await shellx.click('Submit', { clickType: 'long' });
-```
-
-**Full form:**
-```typescript
-await shellx.click({
-  text: 'Submit',           // Click by text
-  elementId: 'btn123',      // Or by element ID
-  resourceId: 'submit_btn', // Or by resource ID
-  x: 500,                   // Or by coordinates
-  y: 1000,
-  clickType: 'single',      // 'single' | 'double' | 'long' | 'normal'
-  timeout: 5000
-});
-```
-
-### Input
-
-Input text into a field.
-
-```typescript
-await shellx.input({
-  elementId: 'field123',
-  text: 'Hello World',
-  clear: true,              // Clear field before input
-  hideKeyboard: true        // Hide keyboard after input
-});
-```
-
-### Swipe
-
-Perform swipe gesture.
-
-```typescript
-await shellx.swipe({
-  fromX: 500,
-  fromY: 1000,
-  toX: 500,
-  toY: 500,
-  duration: 800            // Duration in ms
-});
-```
-
-### Press
-
-Press hardware key.
-
-**Simplified:**
-```typescript
-await shellx.press('BACK');
-await shellx.press('HOME');
-await shellx.press('MENU', { longPress: true });
-```
-
-**Full:**
-```typescript
-await shellx.press({
-  key: 'BACK',
-  longPress: false
-});
-```
-
-### Wait
-
-Wait for an element condition.
-
-**Simplified:**
-```typescript
-await shellx.wait('Submit');
-await shellx.wait('Loading', { timeout: 10000 });
-```
-
-**Full:**
-```typescript
-await shellx.wait({
-  text: 'Submit',
-  condition: 'visible',      // 'visible' | 'gone' | 'enabled'
-  timeout: 5000
-});
-```
-
-### Find
-
-Find UI elements.
-
-**Simplified:**
-```typescript
-const result = await shellx.find('Button');
-const all = await shellx.find('Button', { multiple: true, maxResults: 10 });
-```
-
-**Full:**
-```typescript
-const result = await shellx.find({
-  text: 'Submit',
-  multiple: false,
-  maxResults: 100,
-  pressClick: true          // Auto-click after find
-});
-```
-
-### Command
-
-Execute shell command.
-
-**Simplified:**
-```typescript
-const result = await shellx.command('ls -la');
-const result = await shellx.command('ls -la', { timeout: 5000 });
-```
-
-**Full:**
-```typescript
-const result = await shellx.command({
-  cmd: 'ls -la',
-  timeout: 10000,
-  wait: 1000               // Wait after command
-});
-```
-
-### Screenshot
-
-Take screenshot.
-
-```typescript
-const screenshot = await shellx.takeScreenshot({
-  format: 'png',            // 'png' | 'jpeg'
-  quality: 100,             // 0-100 for JPEG
-  saveToFile: true
-});
-```
-
-### Clipboard
-
-Clipboard operations.
-
-```typescript
-// Get clipboard
-const result = await shellx.clipboard({ get: true });
-
-// Set clipboard
-await shellx.clipboard({ text: 'Hello' });
-
-// Paste clipboard
-await shellx.clipboard({ paste: true });
-```
-
-### ExecuteActions (Batch)
-
-Execute multiple actions in sequence.
-
-```typescript
-const result = await shellx.executeActions([
-  { text: 'Settings' },                    // Click
-  { cmd: 'ls -la' },                       // Command
-  { fromX: 500, fromY: 1000, toX: 500, toY: 500 },  // Swipe
-  { key: 'BACK' }                          // Press
-]);
-
-console.log(`Success: ${result.successCount}/${result.results.length}`);
-```
-
----
-
-## Complete Working Example
-
-Here's a complete example that you can run directly:
-
-```typescript
-// File: example.ts
-import { ShellX } from 'shellx-ai';
-
-async function main() {
-  // 1. Create ShellX instance
-  const shellx = new ShellX({
-    deviceId: process.env.DEVICE_ID || 'your-device-id',
-    onOpen: () => console.log('✅ Connected!')
-  });
-
-  try {
-    // 2. Wait for connection
-    await shellx.ready();
-    console.log('📱 Connected to device');
-
-    // 3. Get screen info
-    const screen = await shellx.getScreenInfo();
-    console.log(`Screen: ${screen.width}x${screen.height}`);
-
-    // 4. Click element
-    await shellx.click('Settings');
-    console.log('✅ Clicked Settings');
-
-    // 5. Press back
-    await shellx.press('BACK');
-    console.log('✅ Pressed BACK');
-
-    // 6. Execute command
-    const result = await shellx.command('getprop ro.build.version.release');
-    console.log(`Android: ${result.output.trim()}`);
-
-    // 7. Take screenshot
-    const screenshot = await shellx.takeScreenshot({ format: 'png' });
-    console.log(`Screenshot: ${screenshot.imagePath}`);
-
-    // 8. Batch operations
-    const batch = await shellx.executeActions([
-      { text: 'Settings' },
-      { cmd: 'wm size' }
-    ]);
-    console.log(`Batch: ${batch.successCount}/${batch.results.length} successful`);
-
-    console.log('🎉 All operations completed!');
-  } catch (error) {
-    console.error('❌ Error:', error);
-  }
-}
-
-main();
-```
-
-Compile and run:
-
-```bash
-# Install dependencies
-npm install shellx-ai dotenv
-
-# Create .env file
-echo "DEVICE_ID=your-device-id" > .env
-
-# Run with tsx
-npx tsx example.ts
-```
-
----
-
-## Response Types
-
-All operations return a result object with the following structure:
-
-```typescript
-interface Result {
-  success: boolean;      // Operation success status
-  error?: string;        // Error message if failed
-  duration: number;      // Operation duration in ms
-  timestamp: number;     // Operation timestamp
-  // ... additional fields specific to each operation
-}
-```
-
-### CommandResult
-
-```typescript
-{
-  success: boolean;
-  output: string;        // Command output
-  error?: string;
-  exitCode?: number;
-  duration: number;
-  cmd: string;
-  timestamp: number;
-}
-```
-
-### FindResult
-
-```typescript
-{
-  success: boolean;
-  found: boolean;
-  count: number;         // Number of elements found
-  elements: Array<{
-    id: string;
-    text: string;
-    class: string;
-    left: number;
-    top: number;
-    right: number;
-    bottom: number;
-    visible: boolean;
-    clickable: boolean;
-  }>;
-  duration: number;
-  timestamp: number;
-}
-```
-
-### ScreenInfoResult
-
-```typescript
-{
-  success: boolean;
-  width: number;
-  height: number;
-  density: number;
-  screenOn: boolean;
-  screenUnlocked: boolean;
-  foregroundApp?: string;
-  foregroundActivity?: string;
-  model?: string;
-  androidVersion?: string;
-  manufacturer?: string;
-  duration: number;
-  timestamp: number;
-}
-```
-
----
-
-## Logging
-
-Control logging level:
-
-```typescript
-const shellx = new ShellX({
-  deviceId: 'your-device-id',
-  logLevel: 3  // 0=NONE, 1=ERROR, 2=WARN, 3=INFO, 4=DEBUG
-});
-
-// Or at runtime
-shellx.setLogLevel(4);         // Enable debug logging
-shellx.enableDebugLogging();   // Shortcut
-shellx.disableLogging();       // Disable all logging
-```
-
----
-
-## Error Handling
-
-```typescript
-try {
-  await shellx.click('Settings');
-} catch (error) {
-  if (error instanceof Error) {
-    console.error('Click failed:', error.message);
-  }
-}
-
-// Or check result
-const result = await shellx.click('Settings');
-if (!result.success) {
-  console.error('Failed:', result.error);
-}
-```
-
----
-
-## Advanced Usage
-
-### Wait for Multiple Elements
-
-```typescript
-const result = await shellx.waitAnyElement([
-  { text: 'OK' },
-  { text: 'Confirm' },
-  { text: 'Submit' }
-], 10000);
-```
-
-### Scroll to Find Element
-
-```typescript
-const element = await shellx.scrollToFindElement(
-  { text: 'Target Item' },
-  5,      // max scroll attempts
-  'down'  // direction
-);
-```
-
-### Find with Retry
-
-```typescript
-const element = await shellx.findElementWithRetry(
-  { text: 'Submit', visible: true },
-  3,      // max retries
-  1000    // retry delay
-);
-```
-
----
-
-## Platform Support
-
-### Node.js
-
-```bash
-npm install shellx-ai ws
-```
-
-```typescript
-import { ShellX } from 'shellx-ai';
-const shellx = new ShellX({ deviceId: 'device-id' });
-```
-
-### Browser
-
-```bash
-npm install shellx-ai
-```
-
-```typescript
-import { ShellX } from 'shellx-ai';
-const shellx = new ShellX({ deviceId: 'device-id' });
-```
-
----
-
-## Troubleshooting
-
-### Connection Issues
-
-```typescript
-const shellx = new ShellX({
-  deviceId: 'your-device-id',
-  timeout: 10000,           // Increase timeout
-  reconnect: true,          // Enable auto-reconnect
-  reconnectMaxAttempts: 10, // More attempts
-  onError: (error) => console.error('Connection error:', error)
-});
-```
-
-### Element Not Found
-
-```typescript
-// Wait for element first
-await shellx.wait('Submit', { timeout: 10000 });
-
-// Then click
-await shellx.click('Submit');
-```
-
-### Command Timeout
-
-```typescript
-const result = await shellx.command({
-  cmd: 'long-running-command',
-  timeout: 30000  // 30 seconds
-});
-```
-
----
-
-## See Also
-
-- [API-GUIDE.md](./API-GUIDE.md) - Detailed API guide
-- [API-QUICK-REFERENCE.md](./API-QUICK-REFERENCE.md) - Quick reference
-- [example/](./example/) - Complete working examples
-
----
-
-## License
-
-MIT License - see [LICENSE](LICENSE) file.
-
----
-
-## Support
-
-- 📚 [Documentation](https://github.com/10cl/shellx)
-- 🐛 [Issues](https://github.com/10cl/shellx/issues)
-- 💬 [Discussions](https://github.com/10cl/shellx/discussions)
+# ShellX-AI 架构文档
+
+## 概述
+
+ShellX-AI 是一个 Android 设备自动化控制库，提供 WebSocket 通信和 HTTP API 支持。
+
+## 模块结构
+
+```
+shellx-ai/              # 核心库
+├── src/
+│   ├── index.ts       # 入口，导出 ConnectionClient 和 ShellX
+│   ├── shellx.ts      # 高级 API
+│   ├── protocol.ts    # 协议定义
+│   └── data/          # 数据操作类型
+│       └── types.ts   # 日历、联系人等数据类型
+
+phone-agent/           # Phone Agent
+├── src/
+│   ├── index.ts       # Agent 初始化
+│   └── tools.ts       # 工具定义
+
+shellx-cli/            # HTTP API 服务器
+├── src/
+│   ├── server/        # HTTP 路由
+│   └── cli-core.ts    # CLI 核心
+
+shellx-cli-v2/         # CLI v2 (pi-tui)
+├── src/
+│   └── agent.ts       # Agent 包装器
+```
+
+## 架构层次
+
+```
+┌─────────────────────────────────────────────────────┐
+│              Phone Agent (phone-agent)              │
+│         - 工具定义和参数验证                         │
+│         - LLM 意图转换                               │
+└─────────────────────────────────────────────────────┘
+                         ▼
+┌─────────────────────────────────────────────────────┐
+│              ShellX-AI (shellx-ai)                  │
+│  - ShellX: 高级 API (推荐)                          │
+│  - ConnectionClient: WebSocket 原始 API            │
+└─────────────────────────────────────────────────────┘
+                         ▼
+┌─────────────────────────────────────────────────────┐
+│              Android 设备端                         │
+│         (ShellX 服务 + OpenClaw)                    │
+└─────────────────────────────────────────────────────┘
+```
+
+## 核心功能
+
+### ShellX API
+
+```typescript
+// 连接管理
+const shellx = new ShellX({ deviceId, timeout: 5000 });
+
+// UI 操作
+await shellx.click(selector);
+await shellx.input(selector, text);
+await shellx.swipe(from, to);
+
+// 数据操作
+await shellx.getCalendarList();
+await shellx.getCalendarEvents({ limit: 10 });
+await shellx.addCalendarEvent({ title, startTime, endTime });
+```
+
+### Phone Agent
+
+```typescript
+import { PhoneAgent } from '@shellx/phone-agent';
+
+const agent = new PhoneAgent({
+  apiKey,
+  baseUrl,
+  model,
+  shellx,
+  maxSteps: 10,
+});
+
+await agent.run("帮我添加一个明天下午3点的会议");
+```
+
+### HTTP API
+
+```bash
+# 获取日历事件
+GET /api/data/calendar/events?limit=10
+
+# 添加日历事件
+POST /api/data/calendar/add
+{
+  "title": "会议",
+  "startTime": 1740990000000,
+  "endTime": 1740997200000
+}
+```
+
+## 协议
+
+### WebSocket (dataRequest)
+
+```typescript
+// 请求
+{
+  dataRequest: {
+    requestType: "getCalendarEvents",
+    params: { limit: 10 }
+  }
+}
+
+// 响应
+{
+  dataRequest: {
+    requestType: "getCalendarEvents",
+    result: { success: true, events: [...] }
+  }
+}
+```
+
+## 技术栈
+
+- **TypeScript** - 类型安全
+- **WebSocket** - 设备通信
+- **CBOR** - 消息编码
+- **@sinclair/typebox** - 参数验证
+- **@mariozechner/pi-agent-core** - Agent 框架
+- **Ink** - TUI 组件库
+
+## 环境变量
+
+| 变量名 | 用途 | 默认值 |
+|--------|------|--------|
+| `SHELLX_DEVICE_ID` | 设备 ID | - |
+| `SHELLX_SERVER_PORT` | HTTP 端口 | 8080 |
+| `SHELLX_LOG_LEVEL` | 日志级别 | INFO |
+
+## 快速开始
+
+```bash
+# 安装依赖
+npm install
+
+# 构建
+npm run build
+
+# 启动设备端服务
+# (需要单独安装 Android 端服务)
+
+# 运行 Phone Agent
+npm run build:phone-agent
+```
+
+## 文档
+
+- [API 文档](./API中文版.md) - 完整的 API 参考
+- [架构设计文档](./架构设计文档.md) - 详细的架构说明