shellx-ai 1.0.12 → 1.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 10cl <notice@toscl.com>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,586 @@
+# ShellX AI
+
+<div align="center">
+
+  **A powerful TypeScript library for Android device automation and UI control**
+
+  [![npm version](https://badge.fury.io/js/shellx-ai.svg)](https://www.npmjs.org/package/shellx-ai)
+  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+  [![Node.js Version](https://img.shields.io/node/v/shellx-ai.svg)](https://nodejs.org)
+  [![TypeScript](https://img.shields.io/badge/TypeScript-5.8-blue.svg)](https://www.typescriptlang.org/)
+  [![Code Style](https://img.shields.io/badge/code_style-prettier-ff69b4.svg)](https://prettier.io)
+
+  [Features](#-features) • [Quick Start](#-quick-start) • [API Guide](#-api-guide) • [API Reference](#-api-reference) • [Examples](#-examples) • [Contributing](#-contributing)
+
+</div>
+
+---
+
+## 🎯 Which API Should I Use?
+
+ShellX provides two API levels. Choose the one that matches your needs:
+
+### 🟢 ShellX (High-Level API) - **Recommended for 95% of users**
+
+Perfect for most automation tasks:
+- ✅ Simple, intuitive methods
+- ✅ Automatic error handling & retry
+- ✅ Type-safe with TypeScript
+- ✅ Less code to write
+
+```typescript
+import { ShellX } from 'shellx-ai';
+
+const shellx = new ShellX({ deviceId: 'your-device-id' });
+await shellx.connect();
+
+// Easy to use!
+await shellx.click('Submit');
+await shellx.input({ text: 'Hello World' });
+await shellx.swipe({ fromX: 500, fromY: 1000, toX: 500, toY: 500 });
+```
+
+### 🔵 ConnectionClient (Low-Level API) - Advanced users only
+
+For special cases requiring direct WebSocket access:
+- Custom protocol implementations
+- Debugging WebSocket communication
+- Non-standard operations
+
+**⚠️ Warning:** Steeper learning curve. Most users don't need this.
+
+```typescript
+import { ConnectionClient } from 'shellx-ai';
+
+const client = new ConnectionClient('device-id');
+await client.ensureConnected();
+
+// Low-level protocol access
+await client.sendMessage({ click: { elementId: 'element123' } });
+```
+
+**📚 See [API-GUIDE.md](./API-GUIDE.md) for detailed comparison and examples.**
+
+---
+
+## ✨ Features
+
+ShellX AI provides a comprehensive suite of tools for Android device automation:
+
+- 🎯 **Smart UI Automation** - Advanced element finding with retry logic and multiple selector strategies
+- 🔧 **Shell Command Execution** - Execute shell commands with real-time output monitoring
+- 📸 **Screen Operations** - Screenshots, screen info, and visual element capture
+- 🔄 **Automatic Retry Logic** - Built-in retry mechanism for robust operations
+- 🛠️ **Modular Architecture** - Clean, maintainable code structure with separation of concerns
+- 📝 **Type-Safe API** - Full TypeScript support with comprehensive type definitions
+- 🌍 **Global Optimization** - Automatic domain selection based on user location
+- 🧪 **Well-Tested** - Comprehensive test coverage for reliability
+
+## 📦 Installation
+
+```bash
+npm install shellx-ai
+```
+
+### Requirements
+
+- **Node.js**: >= 14.0.0
+- **TypeScript**: >= 4.0.0 (recommended)
+
+## 🚀 Quick Start
+
+### Basic Usage
+
+```typescript
+import { ShellX } from 'shellx-ai';
+
+// Create ShellX instance
+const shellx = new ShellX({
+  deviceId: 'your-device-id',
+  onOpen: () => console.log('Connected!'),
+  onMessage: (message) => console.log('Message:', message)
+});
+
+// Wait for connection to be ready
+await shellx.ready();
+
+// Execute shell command
+const result = await shellx.command({ cmd: 'getprop ro.build.version.release' });
+console.log('Android version:', result.output);
+
+// Click UI element (simplified API)
+await shellx.click('Settings');
+
+// Or use full API
+await shellx.click({ text: 'Settings', clickType: 'single' });
+
+// Find elements
+const elements = await shellx.find({
+  text: 'Submit',
+  multiple: true,
+  maxResults: 10
+});
+
+console.log(`Found ${elements.count} elements`);
+```
+
+### With Connection Callbacks
+
+```typescript
+import { ShellX } from 'shellx-ai';
+
+const shellx = new ShellX({
+  deviceId: 'your-device-id',
+  timeout: 5000,
+  reconnect: true,
+  reconnectMaxAttempts: 5,
+  onOpen: () => console.log('✅ Connected'),
+  onClose: () => console.log('❌ Disconnected'),
+  onError: (error) => console.error('⚠️ Error:', error),
+  onMessage: (message) => console.log('📨 Message:', message)
+});
+
+// Wait for connection
+await shellx.ready();
+
+// Ready to use!
+await shellx.command('pm list packages');
+```
+
+## 🎯 API Reference
+
+### Core Classes
+
+#### `ShellX`
+
+Main class providing high-level automation utilities.
+
+**Constructor:**
+```typescript
+constructor(options: ShellXOptions)
+```
+
+**ShellXOptions:**
+```typescript
+interface ShellXOptions {
+  deviceId: string;                      // Required: Device ID
+  timeout?: number;                      // Connection timeout (default: 5000ms)
+  reconnect?: boolean;                   // Enable auto-reconnect (default: true)
+  reconnectMaxAttempts?: number;         // Max reconnect attempts (default: 5)
+  reconnectInterval?: number;            // Reconnect interval (default: 1000ms)
+  pingInterval?: number;                 // Ping interval (default: 2000ms)
+  onOpen?: () => void;                   // Connection opened callback
+  onClose?: () => void;                  // Connection closed callback
+  onError?: (error?: Event) => void;     // Error callback
+  onReconnectFailed?: () => void;        // Reconnect failed callback
+  onMessage?: (message: WsServer) => void; // Message callback
+}
+```
+
+**Methods:**
+
+| Method | Description | Return Type |
+|--------|-------------|-------------|
+| `ready()` | Wait for connection to be ready | `Promise<void>` |
+| `click(data)` | Click element by ID, coordinates, or selector | `Promise<ActionResult>` |
+| `input(data)` | Input text into element | `Promise<ActionResult>` |
+| `swipe(data)` | Perform swipe gesture | `Promise<ActionResult>` |
+| `pressKey(data)` | Press hardware key | `Promise<ActionResult>` |
+| `wait(data)` | Wait for element condition | `Promise<ActionResult>` |
+| `find(data)` | Find UI elements | `Promise<FindResult>` |
+| `clipboard(data)` | Clipboard operations (get/set/paste) | `Promise<ActionResult>` |
+| `takeScreenshot(data)` | Capture screenshot | `Promise<ActionResult>` |
+| `getScreenInfo()` | Get screen information | `Promise<ScreenInfoResponse>` |
+| `executeActions(actions)` | Execute multiple actions in sequence | `Promise<ActionResult[]>` |
+
+**Element Finder Methods:**
+
+| Method | Description | Return Type |
+|--------|-------------|-------------|
+| `findElementWithRetry(selector, maxRetries, retryDelay)` | Find single element with retry | `Promise<UIElement \| null>` |
+| `findElementsWithRetry(selector, maxRetries, retryDelay, options)` | Find multiple elements | `Promise<UIElement[]>` |
+| `waitForAnyElement(selectors, timeout)` | Wait for any element to appear | `Promise<{element, selectorIndex} \| null>` |
+| `scrollToFindElement(selector, maxScrolls, direction)` | Scroll to find element | `Promise<UIElement \| null>` |
+
+**Shell Command Methods:**
+
+| Method | Description | Return Type |
+|--------|-------------|-------------|
+| `executeShellCommand(command, options)` | Execute shell command with monitoring | `Promise<ShellCommandResult>` |
+| `executeSimpleShellCommand(command, options)` | Execute simple shell command | `Promise<ShellCommandResult>` |
+| `executeShellCommands(commands, options)` | Execute multiple commands | `Promise<ShellCommandResult[]>` |
+| `adbCommand(command, options)` | Execute ADB command | `Promise<ShellCommandResult>` |
+| `executeCode(code, context, timeout)` | Execute code in sandboxed environment | `Promise<any>` |
+
+**Device Info Methods:**
+
+| Method | Description | Return Type |
+|--------|-------------|-------------|
+| `getDeviceInfo()` | Get comprehensive device information | `Promise<ShellCommandResult[]>` |
+| `getDeviceModel()` | Get device model | `Promise<string \| undefined>` |
+| `getAndroidVersion()` | Get Android version | `Promise<string \| undefined>` |
+| `getScreenSize()` | Get screen size | `Promise<string \| undefined>` |
+| `getBatteryInfo()` | Get battery information | `Promise<BatteryInfo \| undefined>` |
+| `getDeviceInfoSummary()` | Get device information summary | `Promise<DeviceInfo>` |
+
+**Navigation Methods:**
+
+| Method | Description | Return Type |
+|--------|-------------|-------------|
+| `navigateByPath(textPath)` | Navigate using text path | `Promise<boolean>` |
+| `clickByText(text, exact)` | Click element by text | `Promise<boolean>` |
+| `inputText(selector, text, options)` | Input text into field | `Promise<boolean>` |
+
+### Type Definitions
+
+#### `Click`
+
+```typescript
+interface Click {
+  targetElementId?: string;
+  targetResourceId?: string;
+  targetText?: string;
+  targetClass?: string;
+  targetX?: number;
+  targetY?: number;
+  clickType?: 'single' | 'double' | 'long';
+  wait?: number;
+  retry?: number;
+}
+```
+
+#### `Input`
+
+```typescript
+interface Input {
+  targetElementId?: string;
+  targetResourceId?: string;
+  targetText?: string;
+  targetClass?: string;
+  text: string;
+  clear?: boolean;
+  hideKeyboard?: boolean;
+  wait?: number;
+  retry?: number;
+}
+```
+
+#### `Swipe`
+
+```typescript
+interface Swipe {
+  fromX: number;
+  fromY: number;
+  toX: number;
+  toY: number;
+  duration?: number;
+  wait?: number;
+  retry?: number;
+}
+```
+
+#### `Command`
+
+```typescript
+interface Command {
+  cmd: string;
+  timeout?: number;
+  wait?: number;
+  retry?: number;
+}
+```
+
+#### `ActionResult`
+
+```typescript
+interface ActionResult {
+  success: boolean;
+  data?: any;
+  error?: string;
+  duration: number;
+}
+```
+
+## 📚 Examples
+
+### Example 1: UI Automation Workflow
+
+```typescript
+import { createShellXWithShellMonitoring } from 'shellx-ai';
+
+async function automateApp() {
+  const shellx = await createShellXWithShellMonitoring({
+    deviceId: process.env.SHELLX_DEVICE_ID
+  });
+
+  try {
+    // Navigate to settings
+    await shellx.clickByText('Settings');
+    await shellx.wait({ targetText: 'Accounts', condition: 'visible', timeout: 5000 });
+
+    // Click on account item
+    await shellx.click({ targetText: 'Accounts' });
+
+    // Input text
+    await shellx.input({
+      targetResourceId: 'username_field',
+      text: 'user@example.com',
+      clear: true
+    });
+
+    // Take screenshot
+    await shellx.takeScreenshot({
+      format: 'png',
+      quality: 90,
+      saveToFile: true
+    });
+
+    console.log('Automation completed successfully!');
+  } catch (error) {
+    console.error('Automation failed:', error);
+  }
+}
+
+automateApp();
+```
+
+### Example 2: Batch Command Execution
+
+```typescript
+import { createShellX } from 'shellx-ai';
+import ConnectionTaskClient from 'shellx-ai';
+
+async function executeDeviceCommands() {
+  const client = new ConnectionTaskClient('device-id');
+  await client.waitForInitialization();
+  const shellx = createShellX(client);
+
+  // Execute multiple commands
+  const commands = [
+    { command: 'pm list packages', title: 'List packages' },
+    { command: 'dumpsys battery', title: 'Get battery info' },
+    { command: 'wm size', title: 'Get screen size' }
+  ];
+
+  const results = await shellx.executeShellCommands(commands, {
+    continueOnError: true,
+    timeout: 10000
+  });
+
+  results.forEach((result, index) => {
+    console.log(`\n${commands[index].title}:`);
+    console.log(`Success: ${result.success}`);
+    console.log(`Output: ${result.output}`);
+    if (result.error) {
+      console.error(`Error: ${result.error}`);
+    }
+  });
+}
+
+executeDeviceCommands();
+```
+
+### Example 3: Advanced Element Finding
+
+```typescript
+async function findAndInteract() {
+  const shellx = await createShellXWithShellMonitoring({
+    deviceId: 'device-id'
+  });
+
+  // Find element with retry
+  const element = await shellx.findElementWithRetry(
+    { text: 'Submit Button', visible: true, clickable: true },
+    5,  // max retries
+    1000  // retry delay
+  );
+
+  if (element) {
+    console.log('Found element:', element.elementId);
+    shellx.printElementInfo(element);
+  } else {
+    console.log('Element not found');
+
+    // Try scrolling to find it
+    const foundElement = await shellx.scrollToFindElement(
+      { text: 'Submit Button' },
+      5,
+      'down'
+    );
+
+    if (foundElement) {
+      console.log('Found after scrolling!');
+      await shellx.click({ targetElementId: foundElement.elementId });
+    }
+  }
+}
+
+findAndInteract();
+```
+
+### Example 4: Action Sequence
+
+```typescript
+async function executeActionSequence() {
+  const shellx = await createShellXWithShellMonitoring({
+    deviceId: 'device-id'
+  });
+
+  // Execute multiple actions
+  const actions = [
+    // Click settings
+    { targetText: 'Settings' },
+
+    // Wait for menu
+    { targetText: 'Network', condition: 'visible', timeout: 5000 },
+
+    // Click network
+    { targetText: 'Network' },
+
+    // Execute command
+    { cmd: 'dumpsys wifi' },
+
+    // Take screenshot
+    { format: 'png' }
+  ];
+
+  const results = await shellx.executeActions(actions);
+
+  results.forEach((result, index) => {
+    const status = result.success ? '✅' : '❌';
+    console.log(`Action ${index + 1}: ${status}`);
+    if (!result.success) {
+      console.error(`  Error: ${result.error}`);
+    }
+  });
+}
+
+executeActionSequence();
+```
+
+## 🛠️ Development
+
+### Setup
+
+```bash
+# Clone repository
+git clone https://github.com/10cl/shellx.git
+cd shellx
+
+# Install dependencies
+npm install
+
+# Build project
+npm run build
+
+# Run tests
+npm test
+```
+
+### Available Scripts
+
+| Script | Description |
+|--------|-------------|
+| `npm run build` | Compile TypeScript to JavaScript |
+| `npm run build:watch` | Watch mode for compilation |
+| `npm run type-check` | Type check without emitting files |
+| `npm run lint` | Run ESLint |
+| `npm run lint:fix` | Fix ESLint errors automatically |
+| `npm run format` | Format code with Prettier |
+| `npm run format:check` | Check code formatting |
+| `npm test` | Run tests |
+| `npm run test:coverage` | Run tests with coverage |
+| `npm run clean` | Remove dist directory |
+
+### Project Structure
+
+```
+shellx-ai/
+├── src/
+│   ├── automation/
+│   │   ├── element-finder.ts       # Element finding with retry logic
+│   │   ├── ui-action-handler.ts    # UI action execution
+│   │   └── device-info-helper.ts   # Device information retrieval
+│   ├── shell/
+│   │   ├── output-buffer.ts         # Shell output buffering
+│   │   └── shell-command-executor.ts # Shell command execution
+│   ├── utils/
+│   │   └── retry-helper.ts          # Generic retry mechanism
+│   ├── index.ts                     # Main connection client
+│   ├── shellx.ts                    # Main ShellX class
+│   ├── protocol.ts                  # Protocol type definitions
+│   ├── types.ts                     # Simplified type definitions
+│   ├── domain-manager.ts            # Domain management
+│   └── utils.ts                     # Utility functions
+├── dist/                            # Compiled output
+├── package.json
+├── tsconfig.json
+├── jest.config.js
+├── .eslintrc.json
+├── .prettierrc.json
+└── README.md
+```
+
+## 🧪 Testing
+
+```bash
+# Run all tests
+npm test
+
+# Run tests in watch mode
+npm run test:watch
+
+# Generate coverage report
+npm run test:coverage
+
+# Type checking
+npm run type-check
+
+# Linting
+npm run lint
+```
+
+## 🤝 Contributing
+
+We welcome contributions! Please follow these steps:
+
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
+
+### Code Style
+
+- Use TypeScript for all new features
+- Follow ESLint rules (`npm run lint`)
+- Format code with Prettier (`npm run format`)
+- Add JSDoc comments for public APIs
+- Write tests for new functionality
+- Ensure all tests pass before submitting
+
+## 📝 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## 🆘 Support
+
+- 📚 [Documentation](https://github.com/10cl/shellx/wiki)
+- 🐛 [Issue Tracker](https://github.com/10cl/shellx/issues)
+- 💬 [Discussions](https://github.com/10cl/shellx/discussions)
+
+## 🌟 Acknowledgments
+
+- Built with [TypeScript](https://www.typescriptlang.org/)
+- Powered by [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket)
+- Tested with [Jest](https://jestjs.io/)
+
+---
+
+<div align="center">
+
+  **Built with ❤️ for the automation community**
+
+  [⬆ Back to Top](#shellx-ai)
+
+</div>