shellx-ai 1.1.0 → 1.1.2

package/README.md

@@ -1,586 +1,162 @@
-# ShellX AI
-
-<div align="center">
-
-  **A powerful TypeScript library for Android device automation and UI control**
-
-  [![npm version](https://badge.fury.io/js/shellx-ai.svg)](https://www.npmjs.org/package/shellx-ai)
-  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-  [![Node.js Version](https://img.shields.io/node/v/shellx-ai.svg)](https://nodejs.org)
-  [![TypeScript](https://img.shields.io/badge/TypeScript-5.8-blue.svg)](https://www.typescriptlang.org/)
-  [![Code Style](https://img.shields.io/badge/code_style-prettier-ff69b4.svg)](https://prettier.io)
-
-  [Features](#-features) • [Quick Start](#-quick-start) • [API Guide](#-api-guide) • [API Reference](#-api-reference) • [Examples](#-examples) • [Contributing](#-contributing)
-
-</div>
-
----
-
-## 🎯 Which API Should I Use?
-
-ShellX provides two API levels. Choose the one that matches your needs:
-
-### 🟢 ShellX (High-Level API) - **Recommended for 95% of users**
-
-Perfect for most automation tasks:
-- ✅ Simple, intuitive methods
-- ✅ Automatic error handling & retry
-- ✅ Type-safe with TypeScript
-- ✅ Less code to write
-
-```typescript
-import { ShellX } from 'shellx-ai';
-
-const shellx = new ShellX({ deviceId: 'your-device-id' });
-await shellx.connect();
-
-// Easy to use!
-await shellx.click('Submit');
-await shellx.input({ text: 'Hello World' });
-await shellx.swipe({ fromX: 500, fromY: 1000, toX: 500, toY: 500 });
-```
-
-### 🔵 ConnectionClient (Low-Level API) - Advanced users only
-
-For special cases requiring direct WebSocket access:
-- Custom protocol implementations
-- Debugging WebSocket communication
-- Non-standard operations
-
-**⚠️ Warning:** Steeper learning curve. Most users don't need this.
-
-```typescript
-import { ConnectionClient } from 'shellx-ai';
-
-const client = new ConnectionClient('device-id');
-await client.ensureConnected();
-
-// Low-level protocol access
-await client.sendMessage({ click: { elementId: 'element123' } });
-```
-
-**📚 See [API-GUIDE.md](./API-GUIDE.md) for detailed comparison and examples.**
-
----
-
-## ✨ Features
-
-ShellX AI provides a comprehensive suite of tools for Android device automation:
-
-- 🎯 **Smart UI Automation** - Advanced element finding with retry logic and multiple selector strategies
-- 🔧 **Shell Command Execution** - Execute shell commands with real-time output monitoring
-- 📸 **Screen Operations** - Screenshots, screen info, and visual element capture
-- 🔄 **Automatic Retry Logic** - Built-in retry mechanism for robust operations
-- 🛠️ **Modular Architecture** - Clean, maintainable code structure with separation of concerns
-- 📝 **Type-Safe API** - Full TypeScript support with comprehensive type definitions
-- 🌍 **Global Optimization** - Automatic domain selection based on user location
-- 🧪 **Well-Tested** - Comprehensive test coverage for reliability
-
-## 📦 Installation
-
-```bash
-npm install shellx-ai
-```
-
-### Requirements
-
-- **Node.js**: >= 14.0.0
-- **TypeScript**: >= 4.0.0 (recommended)
-
-## 🚀 Quick Start
-
-### Basic Usage
-
-```typescript
-import { ShellX } from 'shellx-ai';
-
-// Create ShellX instance
-const shellx = new ShellX({
-  deviceId: 'your-device-id',
-  onOpen: () => console.log('Connected!'),
-  onMessage: (message) => console.log('Message:', message)
-});
-
-// Wait for connection to be ready
-await shellx.ready();
-
-// Execute shell command
-const result = await shellx.command({ cmd: 'getprop ro.build.version.release' });
-console.log('Android version:', result.output);
-
-// Click UI element (simplified API)
-await shellx.click('Settings');
-
-// Or use full API
-await shellx.click({ text: 'Settings', clickType: 'single' });
-
-// Find elements
-const elements = await shellx.find({
-  text: 'Submit',
-  multiple: true,
-  maxResults: 10
-});
-
-console.log(`Found ${elements.count} elements`);
-```
-
-### With Connection Callbacks
-
-```typescript
-import { ShellX } from 'shellx-ai';
-
-const shellx = new ShellX({
-  deviceId: 'your-device-id',
-  timeout: 5000,
-  reconnect: true,
-  reconnectMaxAttempts: 5,
-  onOpen: () => console.log('✅ Connected'),
-  onClose: () => console.log('❌ Disconnected'),
-  onError: (error) => console.error('⚠️ Error:', error),
-  onMessage: (message) => console.log('📨 Message:', message)
-});
-
-// Wait for connection
-await shellx.ready();
-
-// Ready to use!
-await shellx.command('pm list packages');
-```
-
-## 🎯 API Reference
-
-### Core Classes
-
-#### `ShellX`
-
-Main class providing high-level automation utilities.
-
-**Constructor:**
-```typescript
-constructor(options: ShellXOptions)
-```
-
-**ShellXOptions:**
-```typescript
-interface ShellXOptions {
-  deviceId: string;                      // Required: Device ID
-  timeout?: number;                      // Connection timeout (default: 5000ms)
-  reconnect?: boolean;                   // Enable auto-reconnect (default: true)
-  reconnectMaxAttempts?: number;         // Max reconnect attempts (default: 5)
-  reconnectInterval?: number;            // Reconnect interval (default: 1000ms)
-  pingInterval?: number;                 // Ping interval (default: 2000ms)
-  onOpen?: () => void;                   // Connection opened callback
-  onClose?: () => void;                  // Connection closed callback
-  onError?: (error?: Event) => void;     // Error callback
-  onReconnectFailed?: () => void;        // Reconnect failed callback
-  onMessage?: (message: WsServer) => void; // Message callback
-}
-```
-
-**Methods:**
-
-| Method | Description | Return Type |
-|--------|-------------|-------------|
-| `ready()` | Wait for connection to be ready | `Promise<void>` |
-| `click(data)` | Click element by ID, coordinates, or selector | `Promise<ActionResult>` |
-| `input(data)` | Input text into element | `Promise<ActionResult>` |
-| `swipe(data)` | Perform swipe gesture | `Promise<ActionResult>` |
-| `pressKey(data)` | Press hardware key | `Promise<ActionResult>` |
-| `wait(data)` | Wait for element condition | `Promise<ActionResult>` |
-| `find(data)` | Find UI elements | `Promise<FindResult>` |
-| `clipboard(data)` | Clipboard operations (get/set/paste) | `Promise<ActionResult>` |
-| `takeScreenshot(data)` | Capture screenshot | `Promise<ActionResult>` |
-| `getScreenInfo()` | Get screen information | `Promise<ScreenInfoResponse>` |
-| `executeActions(actions)` | Execute multiple actions in sequence | `Promise<ActionResult[]>` |
-
-**Element Finder Methods:**
-
-| Method | Description | Return Type |
-|--------|-------------|-------------|
-| `findElementWithRetry(selector, maxRetries, retryDelay)` | Find single element with retry | `Promise<UIElement \| null>` |
-| `findElementsWithRetry(selector, maxRetries, retryDelay, options)` | Find multiple elements | `Promise<UIElement[]>` |
-| `waitForAnyElement(selectors, timeout)` | Wait for any element to appear | `Promise<{element, selectorIndex} \| null>` |
-| `scrollToFindElement(selector, maxScrolls, direction)` | Scroll to find element | `Promise<UIElement \| null>` |
-
-**Shell Command Methods:**
-
-| Method | Description | Return Type |
-|--------|-------------|-------------|
-| `executeShellCommand(command, options)` | Execute shell command with monitoring | `Promise<ShellCommandResult>` |
-| `executeSimpleShellCommand(command, options)` | Execute simple shell command | `Promise<ShellCommandResult>` |
-| `executeShellCommands(commands, options)` | Execute multiple commands | `Promise<ShellCommandResult[]>` |
-| `adbCommand(command, options)` | Execute ADB command | `Promise<ShellCommandResult>` |
-| `executeCode(code, context, timeout)` | Execute code in sandboxed environment | `Promise<any>` |
-
-**Device Info Methods:**
-
-| Method | Description | Return Type |
-|--------|-------------|-------------|
-| `getDeviceInfo()` | Get comprehensive device information | `Promise<ShellCommandResult[]>` |
-| `getDeviceModel()` | Get device model | `Promise<string \| undefined>` |
-| `getAndroidVersion()` | Get Android version | `Promise<string \| undefined>` |
-| `getScreenSize()` | Get screen size | `Promise<string \| undefined>` |
-| `getBatteryInfo()` | Get battery information | `Promise<BatteryInfo \| undefined>` |
-| `getDeviceInfoSummary()` | Get device information summary | `Promise<DeviceInfo>` |
-
-**Navigation Methods:**
-
-| Method | Description | Return Type |
-|--------|-------------|-------------|
-| `navigateByPath(textPath)` | Navigate using text path | `Promise<boolean>` |
-| `clickByText(text, exact)` | Click element by text | `Promise<boolean>` |
-| `inputText(selector, text, options)` | Input text into field | `Promise<boolean>` |
-
-### Type Definitions
-
-#### `Click`
-
-```typescript
-interface Click {
-  targetElementId?: string;
-  targetResourceId?: string;
-  targetText?: string;
-  targetClass?: string;
-  targetX?: number;
-  targetY?: number;
-  clickType?: 'single' | 'double' | 'long';
-  wait?: number;
-  retry?: number;
-}
-```
-
-#### `Input`
-
-```typescript
-interface Input {
-  targetElementId?: string;
-  targetResourceId?: string;
-  targetText?: string;
-  targetClass?: string;
-  text: string;
-  clear?: boolean;
-  hideKeyboard?: boolean;
-  wait?: number;
-  retry?: number;
-}
-```
-
-#### `Swipe`
-
-```typescript
-interface Swipe {
-  fromX: number;
-  fromY: number;
-  toX: number;
-  toY: number;
-  duration?: number;
-  wait?: number;
-  retry?: number;
-}
-```
-
-#### `Command`
-
-```typescript
-interface Command {
-  cmd: string;
-  timeout?: number;
-  wait?: number;
-  retry?: number;
-}
-```
-
-#### `ActionResult`
-
-```typescript
-interface ActionResult {
-  success: boolean;
-  data?: any;
-  error?: string;
-  duration: number;
-}
-```
-
-## 📚 Examples
-
-### Example 1: UI Automation Workflow
-
-```typescript
-import { createShellXWithShellMonitoring } from 'shellx-ai';
-
-async function automateApp() {
-  const shellx = await createShellXWithShellMonitoring({
-    deviceId: process.env.SHELLX_DEVICE_ID
-  });
-
-  try {
-    // Navigate to settings
-    await shellx.clickByText('Settings');
-    await shellx.wait({ targetText: 'Accounts', condition: 'visible', timeout: 5000 });
-
-    // Click on account item
-    await shellx.click({ targetText: 'Accounts' });
-
-    // Input text
-    await shellx.input({
-      targetResourceId: 'username_field',
-      text: 'user@example.com',
-      clear: true
-    });
-
-    // Take screenshot
-    await shellx.takeScreenshot({
-      format: 'png',
-      quality: 90,
-      saveToFile: true
-    });
-
-    console.log('Automation completed successfully!');
-  } catch (error) {
-    console.error('Automation failed:', error);
-  }
-}
-
-automateApp();
-```
-
-### Example 2: Batch Command Execution
-
-```typescript
-import { createShellX } from 'shellx-ai';
-import ConnectionTaskClient from 'shellx-ai';
-
-async function executeDeviceCommands() {
-  const client = new ConnectionTaskClient('device-id');
-  await client.waitForInitialization();
-  const shellx = createShellX(client);
-
-  // Execute multiple commands
-  const commands = [
-    { command: 'pm list packages', title: 'List packages' },
-    { command: 'dumpsys battery', title: 'Get battery info' },
-    { command: 'wm size', title: 'Get screen size' }
-  ];
-
-  const results = await shellx.executeShellCommands(commands, {
-    continueOnError: true,
-    timeout: 10000
-  });
-
-  results.forEach((result, index) => {
-    console.log(`\n${commands[index].title}:`);
-    console.log(`Success: ${result.success}`);
-    console.log(`Output: ${result.output}`);
-    if (result.error) {
-      console.error(`Error: ${result.error}`);
-    }
-  });
-}
-
-executeDeviceCommands();
-```
-
-### Example 3: Advanced Element Finding
-
-```typescript
-async function findAndInteract() {
-  const shellx = await createShellXWithShellMonitoring({
-    deviceId: 'device-id'
-  });
-
-  // Find element with retry
-  const element = await shellx.findElementWithRetry(
-    { text: 'Submit Button', visible: true, clickable: true },
-    5,  // max retries
-    1000  // retry delay
-  );
-
-  if (element) {
-    console.log('Found element:', element.elementId);
-    shellx.printElementInfo(element);
-  } else {
-    console.log('Element not found');
-
-    // Try scrolling to find it
-    const foundElement = await shellx.scrollToFindElement(
-      { text: 'Submit Button' },
-      5,
-      'down'
-    );
-
-    if (foundElement) {
-      console.log('Found after scrolling!');
-      await shellx.click({ targetElementId: foundElement.elementId });
-    }
-  }
-}
-
-findAndInteract();
-```
-
-### Example 4: Action Sequence
-
-```typescript
-async function executeActionSequence() {
-  const shellx = await createShellXWithShellMonitoring({
-    deviceId: 'device-id'
-  });
-
-  // Execute multiple actions
-  const actions = [
-    // Click settings
-    { targetText: 'Settings' },
-
-    // Wait for menu
-    { targetText: 'Network', condition: 'visible', timeout: 5000 },
-
-    // Click network
-    { targetText: 'Network' },
-
-    // Execute command
-    { cmd: 'dumpsys wifi' },
-
-    // Take screenshot
-    { format: 'png' }
-  ];
-
-  const results = await shellx.executeActions(actions);
-
-  results.forEach((result, index) => {
-    const status = result.success ? '✅' : '❌';
-    console.log(`Action ${index + 1}: ${status}`);
-    if (!result.success) {
-      console.error(`  Error: ${result.error}`);
-    }
-  });
-}
-
-executeActionSequence();
-```
-
-## 🛠️ Development
-
-### Setup
-
-```bash
-# Clone repository
-git clone https://github.com/10cl/shellx.git
-cd shellx
-
-# Install dependencies
-npm install
-
-# Build project
-npm run build
-
-# Run tests
-npm test
-```
-
-### Available Scripts
-
-| Script | Description |
-|--------|-------------|
-| `npm run build` | Compile TypeScript to JavaScript |
-| `npm run build:watch` | Watch mode for compilation |
-| `npm run type-check` | Type check without emitting files |
-| `npm run lint` | Run ESLint |
-| `npm run lint:fix` | Fix ESLint errors automatically |
-| `npm run format` | Format code with Prettier |
-| `npm run format:check` | Check code formatting |
-| `npm test` | Run tests |
-| `npm run test:coverage` | Run tests with coverage |
-| `npm run clean` | Remove dist directory |
-
-### Project Structure
-
-```
-shellx-ai/
-├── src/
-│   ├── automation/
-│   │   ├── element-finder.ts       # Element finding with retry logic
-│   │   ├── ui-action-handler.ts    # UI action execution
-│   │   └── device-info-helper.ts   # Device information retrieval
-│   ├── shell/
-│   │   ├── output-buffer.ts         # Shell output buffering
-│   │   └── shell-command-executor.ts # Shell command execution
-│   ├── utils/
-│   │   └── retry-helper.ts          # Generic retry mechanism
-│   ├── index.ts                     # Main connection client
-│   ├── shellx.ts                    # Main ShellX class
-│   ├── protocol.ts                  # Protocol type definitions
-│   ├── types.ts                     # Simplified type definitions
-│   ├── domain-manager.ts            # Domain management
-│   └── utils.ts                     # Utility functions
-├── dist/                            # Compiled output
-├── package.json
-├── tsconfig.json
-├── jest.config.js
-├── .eslintrc.json
-├── .prettierrc.json
-└── README.md
-```
-
-## 🧪 Testing
-
-```bash
-# Run all tests
-npm test
-
-# Run tests in watch mode
-npm run test:watch
-
-# Generate coverage report
-npm run test:coverage
-
-# Type checking
-npm run type-check
-
-# Linting
-npm run lint
-```
-
-## 🤝 Contributing
-
-We welcome contributions! Please follow these steps:
-
-1. Fork the repository
-2. Create your feature branch (`git checkout -b feature/amazing-feature`)
-3. Commit your changes (`git commit -m 'Add amazing feature'`)
-4. Push to the branch (`git push origin feature/amazing-feature`)
-5. Open a Pull Request
-
-### Code Style
-
-- Use TypeScript for all new features
-- Follow ESLint rules (`npm run lint`)
-- Format code with Prettier (`npm run format`)
-- Add JSDoc comments for public APIs
-- Write tests for new functionality
-- Ensure all tests pass before submitting
-
-## 📝 License
-
-This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
-
-## 🆘 Support
-
-- 📚 [Documentation](https://github.com/10cl/shellx/wiki)
-- 🐛 [Issue Tracker](https://github.com/10cl/shellx/issues)
-- 💬 [Discussions](https://github.com/10cl/shellx/discussions)
-
-## 🌟 Acknowledgments
-
-- Built with [TypeScript](https://www.typescriptlang.org/)
-- Powered by [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket)
-- Tested with [Jest](https://jestjs.io/)
-
----
-
-<div align="center">
-
-  **Built with ❤️ for the automation community**
-
-  [⬆ Back to Top](#shellx-ai)
-
-</div>
+# ShellX-AI 架构文档
+
+## 概述
+
+ShellX-AI 是一个 Android 设备自动化控制库，提供 WebSocket 通信和 HTTP API 支持。
+
+## 模块结构
+
+```
+shellx-ai/              # 核心库
+├── src/
+│   ├── index.ts       # 入口，导出 ConnectionClient 和 ShellX
+│   ├── shellx.ts      # 高级 API
+│   ├── protocol.ts    # 协议定义
+│   └── data/          # 数据操作类型
+│       └── types.ts   # 日历、联系人等数据类型
+
+phone-agent/           # Phone Agent
+├── src/
+│   ├── index.ts       # Agent 初始化
+│   └── tools.ts       # 工具定义
+
+shellx-cli/            # HTTP API 服务器
+├── src/
+│   ├── server/        # HTTP 路由
+│   └── cli-core.ts    # CLI 核心
+
+shellx-cli-v2/         # CLI v2 (pi-tui)
+├── src/
+│   └── agent.ts       # Agent 包装器
+```
+
+## 架构层次
+
+```
+┌─────────────────────────────────────────────────────┐
+│              Phone Agent (phone-agent)              │
+│         - 工具定义和参数验证                         │
+│         - LLM 意图转换                               │
+└─────────────────────────────────────────────────────┘
+                         ▼
+┌─────────────────────────────────────────────────────┐
+│              ShellX-AI (shellx-ai)                  │
+│  - ShellX: 高级 API (推荐)                          │
+│  - ConnectionClient: WebSocket 原始 API            │
+└─────────────────────────────────────────────────────┘
+                         ▼
+┌─────────────────────────────────────────────────────┐
+│              Android 设备端                         │
+│         (ShellX 服务 + OpenClaw)                    │
+└─────────────────────────────────────────────────────┘
+```
+
+## 核心功能
+
+### ShellX API
+
+```typescript
+// 连接管理
+const shellx = new ShellX({ deviceId, timeout: 5000 });
+
+// UI 操作
+await shellx.click(selector);
+await shellx.input(selector, text);
+await shellx.swipe(from, to);
+
+// 数据操作
+await shellx.getCalendarList();
+await shellx.getCalendarEvents({ limit: 10 });
+await shellx.addCalendarEvent({ title, startTime, endTime });
+```
+
+### Phone Agent
+
+```typescript
+import { PhoneAgent } from '@shellx/phone-agent';
+
+const agent = new PhoneAgent({
+  apiKey,
+  baseUrl,
+  model,
+  shellx,
+  maxSteps: 10,
+});
+
+await agent.run("帮我添加一个明天下午3点的会议");
+```
+
+### HTTP API
+
+```bash
+# 获取日历事件
+GET /api/data/calendar/events?limit=10
+
+# 添加日历事件
+POST /api/data/calendar/add
+{
+  "title": "会议",
+  "startTime": 1740990000000,
+  "endTime": 1740997200000
+}
+```
+
+## 协议
+
+### WebSocket (dataRequest)
+
+```typescript
+// 请求
+{
+  dataRequest: {
+    requestType: "getCalendarEvents",
+    params: { limit: 10 }
+  }
+}
+
+// 响应
+{
+  dataRequest: {
+    requestType: "getCalendarEvents",
+    result: { success: true, events: [...] }
+  }
+}
+```
+
+## 技术栈
+
+- **TypeScript** - 类型安全
+- **WebSocket** - 设备通信
+- **CBOR** - 消息编码
+- **@sinclair/typebox** - 参数验证
+- **@mariozechner/pi-agent-core** - Agent 框架
+- **Ink** - TUI 组件库
+
+## 环境变量
+
+| 变量名 | 用途 | 默认值 |
+|--------|------|--------|
+| `SHELLX_DEVICE_ID` | 设备 ID | - |
+| `SHELLX_SERVER_PORT` | HTTP 端口 | 8080 |
+| `SHELLX_LOG_LEVEL` | 日志级别 | INFO |
+
+## 快速开始
+
+```bash
+# 安装依赖
+npm install
+
+# 构建
+npm run build
+
+# 启动设备端服务
+# (需要单独安装 Android 端服务)
+
+# 运行 Phone Agent
+npm run build:phone-agent
+```
+
+## 文档
+
+- [API 文档](./API中文版.md) - 完整的 API 参考
+- [架构设计文档](./架构设计文档.md) - 详细的架构说明