shellx-ai 1.1.1 → 1.1.3

package/README.md

@@ -1,666 +1,201 @@
-# ShellX AI
-
-<div align="center">
-
-  **TypeScript library for Android device automation via WebSocket**
-
-  [![npm version](https://badge.fury.io/js/shellx-ai.svg)](https://www.npmjs.org/package/shellx-ai)
-  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-  [![Node.js Version](https://img.shields.io/node/v/shellx-ai.svg)](https://nodejs.org)
-  [![TypeScript](https://img.shields.io/badge/TypeScript-5.8-blue.svg)](https://www.typescriptlang.org/)
-
-  [Features](#-features) • [Installation](#-installation) • [Quick Start](#-quick-start) • [API Reference](#-api-reference) • [Examples](#-examples)
-
-</div>
-
----
-
-## Summary
-
-ShellX AI is a TypeScript library that enables programmatic control of Android devices through a WebSocket connection. It provides a simple, type-safe API for:
-
-- **UI Automation** - Click, input, swipe, press keys, wait for elements
-- **Element Finding** - Find UI elements by text, ID, class, or coordinates
-- **Shell Commands** - Execute shell commands and get output
-- **Device Info** - Get screen info, app info, screenshots
-- **Batch Operations** - Execute multiple actions in sequence
-
----
-
-## Features
-
-- 🎯 **Simple API** - Intuitive methods for common automation tasks
-- 🔄 **Auto Retry** - Built-in retry logic for robust operations
-- 📝 **Type-Safe** - Full TypeScript support with comprehensive types
-- 🌍 **Cross-Platform** - Works in Node.js and browser environments
-- 📸 **Screenshots** - Capture screenshots with customizable options
-- 🔧 **Shell Commands** - Execute commands with real-time output
-- 🛠️ **Batch Actions** - Chain multiple operations together
-
----
-
-## Installation
-
-```bash
-npm install shellx-ai
-```
-
-### Requirements
-
-- Node.js >= 14.0.0
-- TypeScript >= 4.0.0 (recommended)
-- Android device connected to ShellX service
-
-### Optional Dependencies
-
-For Node.js environment, install ws:
-
-```bash
-npm install ws  # Optional, for Node.js WebSocket support
-```
-
-For browser environments, no additional dependencies needed.
-
----
-
-## Quick Start
-
-### 1. Basic Example
-
-```typescript
-import { ShellX } from 'shellx-ai';
-
-// Create ShellX instance
-const shellx = new ShellX({
-  deviceId: 'your-device-id'  // Replace with your device ID
-});
-
-// Wait for connection
-await shellx.ready();
-
-// Click element by text
-await shellx.click('Settings');
-
-// Get screen info
-const screen = await shellx.getScreenInfo();
-console.log(`Screen: ${screen.width}x${screen.height}`);
-
-// Execute shell command
-const result = await shellx.command('getprop ro.build.version.release');
-console.log('Android version:', result.output);
-```
-
-### 2. Using Environment Variables
-
-Create a `.env` file:
-
-```env
-DEVICE_ID=your-device-id
-```
-
-Then use it in your code:
-
-```typescript
-import { ShellX } from 'shellx-ai';
-import dotenv from 'dotenv';
-
-dotenv.config();
-
-const shellx = new ShellX({
-  deviceId: process.env.DEVICE_ID
-});
-
-await shellx.ready();
-// Start automating...
-```
-
-### 3. With Connection Events
-
-```typescript
-const shellx = new ShellX({
-  deviceId: 'your-device-id',
-  onOpen: () => console.log('✅ Connected'),
-  onClose: () => console.log('❌ Disconnected'),
-  onError: (error) => console.error('⚠️ Error:', error)
-});
-
-await shellx.ready();
-```
-
----
-
-## API Reference
-
-### ShellX Class
-
-The main class for Android automation.
-
-#### Constructor
-
-```typescript
-new ShellX(options: ShellXOptions)
-```
-
-**Options:**
-
-| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| `deviceId` | `string` | Yes | - | Device ID (UUID or identifier) |
-| `timeout` | `number` | No | `5000` | Connection timeout in ms |
-| `reconnect` | `boolean` | No | `true` | Enable auto-reconnect |
-| `reconnectMaxAttempts` | `number` | No | `5` | Max reconnect attempts |
-| `logLevel` | `LogLevel` | No | `INFO` | Log level (0=NONE, 1=ERROR, 2=WARN, 3=INFO, 4=DEBUG) |
-| `onOpen` | `() => void` | No | - | Callback when connection opens |
-| `onClose` | `() => void` | No | - | Callback when connection closes |
-| `onError` | `(error?) => void` | No | - | Callback on error |
-| `onMessage` | `(msg) => void` | No | - | Callback on message |
-
-#### Methods
-
-##### Connection
-
-| Method | Description | Returns |
-|--------|-------------|---------|
-| `ready()` | Wait for connection to be ready | `Promise<void>` |
-| `getClient()` | Get underlying ConnectionClient | `ConnectionClient` |
-
-##### UI Actions
-
-| Method | Description | Example |
-|--------|-------------|---------|
-| `click(selector)` | Click element | `await shellx.click('Submit')` |
-| `input(data)` | Input text | `await shellx.input({ text: 'Hello' })` |
-| `swipe(data)` | Swipe gesture | `await shellx.swipe({ fromX: 500, fromY: 1000, toX: 500, toY: 500 })` |
-| `press(key)` | Press key | `await shellx.press('BACK')` |
-| `wait(selector)` | Wait for element | `await shellx.wait('Loading')` |
-| `find(selector)` | Find elements | `await shellx.find('Button', { multiple: true })` |
-
-##### Device Operations
-
-| Method | Description | Example |
-|--------|-------------|---------|
-| `command(cmd)` | Execute shell command | `await shellx.command('ls -la')` |
-| `getScreenInfo()` | Get screen info | `await shellx.getScreenInfo()` |
-| `takeScreenshot()` | Take screenshot | `await shellx.takeScreenshot({ format: 'png' })` |
-| `getAppInfo(pkg)` | Get app info | `await shellx.getAppInfo('com.example.app')` |
-| `getAppList()` | Get app list | `await shellx.getAppList()` |
-| `clipboard(data)` | Clipboard operations | `await shellx.clipboard({ text: 'Hello' })` |
-
-##### Advanced
-
-| Method | Description | Example |
-|--------|-------------|---------|
-| `executeActions(actions)` | Execute multiple actions | `await shellx.executeActions([{ text: 'OK' }, { cmd: 'ls' }])` |
-| `sendRawMessage(msg)` | Send raw WebSocket message | `await shellx.sendRawMessage({ screenInfo: {} })` |
-
----
-
-## Type Definitions
-
-### Click
-
-Click on an element by text, ID, or coordinates.
-
-**Simplified form (click by text):**
-```typescript
-await shellx.click('Submit');
-await shellx.click('Submit', { clickType: 'long' });
-```
-
-**Full form:**
-```typescript
-await shellx.click({
-  text: 'Submit',           // Click by text
-  elementId: 'btn123',      // Or by element ID
-  resourceId: 'submit_btn', // Or by resource ID
-  x: 500,                   // Or by coordinates
-  y: 1000,
-  clickType: 'single',      // 'single' | 'double' | 'long' | 'normal'
-  timeout: 5000
-});
-```
-
-### Input
-
-Input text into a field.
-
-```typescript
-await shellx.input({
-  elementId: 'field123',
-  text: 'Hello World',
-  clear: true,              // Clear field before input
-  hideKeyboard: true        // Hide keyboard after input
-});
-```
-
-### Swipe
-
-Perform swipe gesture.
-
-```typescript
-await shellx.swipe({
-  fromX: 500,
-  fromY: 1000,
-  toX: 500,
-  toY: 500,
-  duration: 800            // Duration in ms
-});
-```
-
-### Press
-
-Press hardware key.
-
-**Simplified:**
-```typescript
-await shellx.press('BACK');
-await shellx.press('HOME');
-await shellx.press('MENU', { longPress: true });
-```
-
-**Full:**
-```typescript
-await shellx.press({
-  key: 'BACK',
-  longPress: false
-});
-```
-
-### Wait
-
-Wait for an element condition.
-
-**Simplified:**
-```typescript
-await shellx.wait('Submit');
-await shellx.wait('Loading', { timeout: 10000 });
-```
-
-**Full:**
-```typescript
-await shellx.wait({
-  text: 'Submit',
-  condition: 'visible',      // 'visible' | 'gone' | 'enabled'
-  timeout: 5000
-});
-```
-
-### Find
-
-Find UI elements.
-
-**Simplified:**
-```typescript
-const result = await shellx.find('Button');
-const all = await shellx.find('Button', { multiple: true, maxResults: 10 });
-```
-
-**Full:**
-```typescript
-const result = await shellx.find({
-  text: 'Submit',
-  multiple: false,
-  maxResults: 100,
-  pressClick: true          // Auto-click after find
-});
-```
-
-### Command
-
-Execute shell command.
-
-**Simplified:**
-```typescript
-const result = await shellx.command('ls -la');
-const result = await shellx.command('ls -la', { timeout: 5000 });
-```
-
-**Full:**
-```typescript
-const result = await shellx.command({
-  cmd: 'ls -la',
-  timeout: 10000,
-  wait: 1000               // Wait after command
-});
-```
-
-### Screenshot
-
-Take screenshot.
-
-```typescript
-const screenshot = await shellx.takeScreenshot({
-  format: 'png',            // 'png' | 'jpeg'
-  quality: 100,             // 0-100 for JPEG
-  saveToFile: true
-});
-```
-
-### Clipboard
-
-Clipboard operations.
-
-```typescript
-// Get clipboard
-const result = await shellx.clipboard({ get: true });
-
-// Set clipboard
-await shellx.clipboard({ text: 'Hello' });
-
-// Paste clipboard
-await shellx.clipboard({ paste: true });
-```
-
-### ExecuteActions (Batch)
-
-Execute multiple actions in sequence.
-
-```typescript
-const result = await shellx.executeActions([
-  { text: 'Settings' },                    // Click
-  { cmd: 'ls -la' },                       // Command
-  { fromX: 500, fromY: 1000, toX: 500, toY: 500 },  // Swipe
-  { key: 'BACK' }                          // Press
-]);
-
-console.log(`Success: ${result.successCount}/${result.results.length}`);
-```
-
----
-
-## Complete Working Example
-
-Here's a complete example that you can run directly:
-
-```typescript
-// File: example.ts
-import { ShellX } from 'shellx-ai';
-
-async function main() {
-  // 1. Create ShellX instance
-  const shellx = new ShellX({
-    deviceId: process.env.DEVICE_ID || 'your-device-id',
-    onOpen: () => console.log('✅ Connected!')
-  });
-
-  try {
-    // 2. Wait for connection
-    await shellx.ready();
-    console.log('📱 Connected to device');
-
-    // 3. Get screen info
-    const screen = await shellx.getScreenInfo();
-    console.log(`Screen: ${screen.width}x${screen.height}`);
-
-    // 4. Click element
-    await shellx.click('Settings');
-    console.log('✅ Clicked Settings');
-
-    // 5. Press back
-    await shellx.press('BACK');
-    console.log('✅ Pressed BACK');
-
-    // 6. Execute command
-    const result = await shellx.command('getprop ro.build.version.release');
-    console.log(`Android: ${result.output.trim()}`);
-
-    // 7. Take screenshot
-    const screenshot = await shellx.takeScreenshot({ format: 'png' });
-    console.log(`Screenshot: ${screenshot.imagePath}`);
-
-    // 8. Batch operations
-    const batch = await shellx.executeActions([
-      { text: 'Settings' },
-      { cmd: 'wm size' }
-    ]);
-    console.log(`Batch: ${batch.successCount}/${batch.results.length} successful`);
-
-    console.log('🎉 All operations completed!');
-  } catch (error) {
-    console.error('❌ Error:', error);
-  }
-}
-
-main();
-```
-
-Compile and run:
-
-```bash
-# Install dependencies
-npm install shellx-ai dotenv
-
-# Create .env file
-echo "DEVICE_ID=your-device-id" > .env
-
-# Run with tsx
-npx tsx example.ts
-```
-
----
-
-## Response Types
-
-All operations return a result object with the following structure:
-
-```typescript
-interface Result {
-  success: boolean;      // Operation success status
-  error?: string;        // Error message if failed
-  duration: number;      // Operation duration in ms
-  timestamp: number;     // Operation timestamp
-  // ... additional fields specific to each operation
-}
-```
-
-### CommandResult
-
-```typescript
-{
-  success: boolean;
-  output: string;        // Command output
-  error?: string;
-  exitCode?: number;
-  duration: number;
-  cmd: string;
-  timestamp: number;
-}
-```
-
-### FindResult
-
-```typescript
-{
-  success: boolean;
-  found: boolean;
-  count: number;         // Number of elements found
-  elements: Array<{
-    id: string;
-    text: string;
-    class: string;
-    left: number;
-    top: number;
-    right: number;
-    bottom: number;
-    visible: boolean;
-    clickable: boolean;
-  }>;
-  duration: number;
-  timestamp: number;
-}
-```
-
-### ScreenInfoResult
-
-```typescript
-{
-  success: boolean;
-  width: number;
-  height: number;
-  density: number;
-  screenOn: boolean;
-  screenUnlocked: boolean;
-  foregroundApp?: string;
-  foregroundActivity?: string;
-  model?: string;
-  androidVersion?: string;
-  manufacturer?: string;
-  duration: number;
-  timestamp: number;
-}
-```
-
----
-
-## Logging
-
-Control logging level:
-
-```typescript
-const shellx = new ShellX({
-  deviceId: 'your-device-id',
-  logLevel: 3  // 0=NONE, 1=ERROR, 2=WARN, 3=INFO, 4=DEBUG
-});
-
-// Or at runtime
-shellx.setLogLevel(4);         // Enable debug logging
-shellx.enableDebugLogging();   // Shortcut
-shellx.disableLogging();       // Disable all logging
-```
-
----
-
-## Error Handling
-
-```typescript
-try {
-  await shellx.click('Settings');
-} catch (error) {
-  if (error instanceof Error) {
-    console.error('Click failed:', error.message);
-  }
-}
-
-// Or check result
-const result = await shellx.click('Settings');
-if (!result.success) {
-  console.error('Failed:', result.error);
-}
-```
-
----
-
-## Advanced Usage
-
-### Wait for Multiple Elements
-
-```typescript
-const result = await shellx.waitAnyElement([
-  { text: 'OK' },
-  { text: 'Confirm' },
-  { text: 'Submit' }
-], 10000);
-```
-
-### Scroll to Find Element
-
-```typescript
-const element = await shellx.scrollToFindElement(
-  { text: 'Target Item' },
-  5,      // max scroll attempts
-  'down'  // direction
-);
-```
-
-### Find with Retry
-
-```typescript
-const element = await shellx.findElementWithRetry(
-  { text: 'Submit', visible: true },
-  3,      // max retries
-  1000    // retry delay
-);
-```
-
----
-
-## Platform Support
-
-### Node.js
-
-```bash
-npm install shellx-ai ws
-```
-
-```typescript
-import { ShellX } from 'shellx-ai';
-const shellx = new ShellX({ deviceId: 'device-id' });
-```
-
-### Browser
-
-```bash
-npm install shellx-ai
-```
-
-```typescript
-import { ShellX } from 'shellx-ai';
-const shellx = new ShellX({ deviceId: 'device-id' });
-```
-
----
-
-## Troubleshooting
-
-### Connection Issues
-
-```typescript
-const shellx = new ShellX({
-  deviceId: 'your-device-id',
-  timeout: 10000,           // Increase timeout
-  reconnect: true,          // Enable auto-reconnect
-  reconnectMaxAttempts: 10, // More attempts
-  onError: (error) => console.error('Connection error:', error)
-});
-```
-
-### Element Not Found
-
-```typescript
-// Wait for element first
-await shellx.wait('Submit', { timeout: 10000 });
-
-// Then click
-await shellx.click('Submit');
-```
-
-### Command Timeout
-
-```typescript
-const result = await shellx.command({
-  cmd: 'long-running-command',
-  timeout: 30000  // 30 seconds
-});
-```
-
----
-
-## See Also
-
-- [API-GUIDE.md](./API-GUIDE.md) - Detailed API guide
-- [API-QUICK-REFERENCE.md](./API-QUICK-REFERENCE.md) - Quick reference
-- [example/](./example/) - Complete working examples
-
----
-
-## License
-
-MIT License - see [LICENSE](LICENSE) file.
-
----
-
-## Support
-
-- 📚 [Documentation](https://github.com/10cl/shellx)
-- 🐛 [Issues](https://github.com/10cl/shellx/issues)
-- 💬 [Discussions](https://github.com/10cl/shellx/discussions)
+# ShellX-AI 数据采集与监控系统
+
+一个完整的数据采集、传输和可视化监控系统，使用 ShellX-AI SDK 进行设备数据采集，Node.js 后端接收和存储数据，Web 界面实时展示。
+
+## 📋 功能特性
+
+- ✅ 使用 ShellX-AI SDK 自动采集设备屏幕数据
+- ✅ 实时提取数值数据（点赞数、评论数、分享数等）
+- ✅ 实时提取文本数据
+- ✅ Node.js 后端接收和存储数据
+- ✅ Web 界面实时展示和监控
+- ✅ 历史数据记录和趋势分析
+- ✅ 统计指标计算
+
+## 🚀 快速开始
+
+### 1. 安装依赖
+
+```bash
+npm install
+```
+
+### 2. 启动系统
+
+**方式一：分别启动**
+
+```bash
+# 终端1：启动后端服务器
+npm run server
+
+# 终端2：启动数据采集
+npm run collect
+```
+
+**方式二：同时启动（推荐）**
+
+```bash
+npm run dev
+```
+
+### 3. 访问监控面板
+
+打开浏览器访问：`http://localhost:3000`
+
+## 📁 项目结构
+
+```
+.
+├── collect_douyin_data.ts    # 数据采集脚本（使用 ShellX-AI SDK）
+├── server.ts                 # Node.js 后端服务器
+├── package.json              # 项目配置和依赖
+├── README.md                 # 项目说明文档
+└── public/
+    └── index.html           # 数据展示页面
+```
+
+## 🔧 配置说明
+
+### 修改采集间隔
+
+在 `collect_douyin_data.ts` 中修改：
+
+```typescript
+// 每5秒采集一次
+setInterval(async () => {
+  try {
+    await collectVideoData();
+  } catch (error) {
+    console.error('采集出错:', error.message);
+  }
+}, 5000); // 修改这个值（毫秒）
+```
+
+### 修改后端端口
+
+在 `server.ts` 中修改：
+
+```typescript
+const PORT = 3000; // 修改为你想要的端口
+```
+
+### 修改API地址
+
+在 `collect_douyin_data.ts` 中修改：
+
+```typescript
+const API_URL = 'http://localhost:3000/api/data';
+```
+
+## 📊 数据格式
+
+### 发送到后端的数据格式
+
+```typescript
+{
+  timestamp: string;           // 时间戳
+  app: string;                 // 当前应用
+  screen: {                    // 屏幕信息
+    width: number;
+    height: number;
+  };
+  metrics: {
+    numericData: Array<{       // 数值数据
+      label: string;
+      value: number;
+      raw: string;
+    }>;
+    textData: Array<{          // 文本数据
+      text: string;
+      id: string;
+    }>;
+  };
+  elementCount: number;        // 元素总数
+}
+```
+
+## 🌐 API 接口
+
+### POST /api/data
+接收采集的数据
+
+**请求体：** 数据采集对象
+
+**响应：** `{ success: true, message: 'Data received' }`
+
+### GET /api/data
+获取所有数据
+
+**响应：** 完整的数据存储对象
+
+### GET /api/history?limit=50
+获取历史数据
+
+**参数：**
+- `limit`: 返回的记录数（默认50）
+
+**响应：** 历史数据数组
+
+### GET /api/metrics
+获取统计指标
+
+**响应：** 统计指标对象
+
+### POST /api/clear
+清空所有数据
+
+**响应：** `{ success: true, message: 'Data cleared' }`
+
+## 📱 支持的应用
+
+当前脚本针对抖音（`com.ss.android.ugc.aweme`）进行了优化，但可以轻松适配其他应用：
+
+1. 修改 `collect_douyin_data.ts` 中的数据提取逻辑
+2. 根据目标应用的 UI 结构调整过滤条件
+
+## 🔍 数据采集原理
+
+1. **获取设备信息**：使用 `shellx.getScreenInfo()` 获取屏幕状态
+2. **获取UI元素**：使用 `shellx.findElementsWithRetry()` 获取屏幕所有元素
+3. **数据过滤**：
+   - 数值数据：匹配数字格式（支持"1.2万"、"10w"等）
+   - 文本数据：过滤出有意义的长文本
+4. **数据传输**：通过 HTTP POST 发送到后端
+5. **数据展示**：Web 界面实时更新和可视化
+
+## 🛠️ 技术栈
+
+- **数据采集**: ShellX-AI SDK
+- **后端**: Node.js + Express + TypeScript
+- **前端**: HTML + CSS + JavaScript
+- **数据传输**: REST API + JSON
+
+## 📝 注意事项
+
+1. 确保设备已连接并授权 ShellX-AI
+2. 后端服务器必须先启动才能接收数据
+3. 数据采集频率不宜过高，避免对设备性能产生影响
+4. 根据实际需求调整数据过滤和提取逻辑
+
+## 🚨 故障排除
+
+### 数据无法显示
+
+1. 检查后端服务器是否正常运行
+2. 检查采集脚本是否正常启动
+3. 检查浏览器控制台是否有错误信息
+4. 确认 API 地址配置正确
+
+### 采集脚本报错
+
+1. 检查设备连接状态
+2. 确认 ShellX-AI SDK 已正确安装
+3. 查看错误日志定位问题
+
+## 📄 许可证
+
+MIT License
+
+## 🤝 贡献
+
+欢迎提交 Issue 和 Pull Request！