rird 2.1.231 → 2.3.0

package/docs/ARCHITECTURE.md

@@ -0,0 +1,768 @@
+# RIRD CLI Architecture
+
+Technical architecture and system design of RIRD AI Agent.
+
+## System Architecture Overview
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                      User Interface Layer                    │
+│  Terminal UI (TUI) | API Server | CLI Commands              │
+└────────────────────────────┬────────────────────────────────┘
+                             │
+┌────────────────────────────▼────────────────────────────────┐
+│                      Agent Control Layer                     │
+│  Command Router | State Management | Session Handler        │
+└────────────────────────────┬────────────────────────────────┘
+                             │
+┌────────────────────────────▼────────────────────────────────┐
+│                   Execution Engine Layer                     │
+│  Task Processing | Planning | Tool Calling | Response Gen   │
+└────────────────────────────┬────────────────────────────────┘
+                             │
+┌─────────────────────────────────────────────────────────────┐
+│                    Integration Layer                         │
+│  ┌──────────────┐  ┌──────────────┐  ┌─────────────────┐    │
+│  │  LLM Proxy   │  │  MCP Servers │  │  File System    │    │
+│  │  (Anthropic, │  │  (Browser,   │  │  (Read/Write)   │    │
+│  │   OpenAI,    │  │   Computer   │  │                 │    │
+│  │   Google,    │  │   Use)       │  │                 │    │
+│  │   etc)       │  │              │  │                 │    │
+│  └──────────────┘  └──────────────┘  └─────────────────┘    │
+└─────────────────────────────────────────────────────────────┘
+                             │
+┌─────────────────────────────────────────────────────────────┐
+│                    Foundation Layer                          │
+│  Configuration | License | Logging | Error Handling         │
+└─────────────────────────────────────────────────────────────┘
+```
+
+## Component Responsibilities
+
+### User Interface Layer
+
+**Terminal UI (TUI)**
+- Interactive conversation interface
+- Real-time task progress display
+- Keyboard navigation and shortcuts
+- Session management (save/resume)
+- Syntax highlighting for code
+
+**API Server**
+- REST endpoints for remote task submission
+- Bearer token authentication
+- WebSocket support for streaming
+- Health checks and status reporting
+
+**CLI Commands**
+- One-shot task execution
+- License management (activate, status)
+- System information (version, health)
+- Configuration management
+- Update/upgrade functionality
+
+### Agent Control Layer
+
+**Command Router**
+- Parses user input and shell arguments
+- Routes to appropriate handler
+- Validates command syntax
+- Handles special commands (--version, --help)
+
+**State Management**
+- Maintains conversation history
+- Tracks task progress and results
+- Manages session persistence
+- Handles interrupts and signals
+
+**Session Handler**
+- Creates new sessions
+- Loads previous conversations
+- Saves state to disk
+- Implements session recovery
+
+### Execution Engine Layer
+
+**Task Processing**
+- Breaks down user tasks into steps
+- Manages task lifecycle (queue, execute, complete)
+- Implements retry logic with exponential backoff
+- Tracks execution metrics and timings
+
+**Planning**
+- Analyzes user intent and requirements
+- Generates execution plan
+- Determines required tools and models
+- Estimates resource usage
+
+**Tool Calling**
+- Invokes appropriate tools based on plan
+- Manages tool execution and result handling
+- Implements tool chaining (output of one tool feeds into another)
+- Handles tool errors with fallbacks
+
+**Response Generation**
+- Synthesizes results from tool calls
+- Generates human-readable explanations
+- Formats output (text, JSON, code)
+- Implements streaming for large results
+
+### Integration Layer
+
+**LLM Proxy**
+- Communicates with remote LLM servers
+- Handles authentication and API keys
+- Implements retry logic for transient failures
+- Caches model responses
+- Streams tokens for real-time display
+
+**MCP Servers**
+- Browser automation (Playwright/Puppeteer)
+- Computer use (mouse, keyboard, screenshots)
+- File system operations (read, write, edit)
+- Web scraping and extraction
+- Image analysis and OCR
+
+**File System**
+- Read files for analysis
+- Write generated code
+- Edit existing files
+- Manage project structure
+
+### Foundation Layer
+
+**Configuration**
+- Loads YAML/JSON configuration files
+- Validates configuration schema
+- Provides sensible defaults
+- Environment variable overrides
+
+**License Management**
+- Validates license key format
+- Communicates with license server
+- Manages device fingerprinting
+- Implements 24-hour cache for offline use
+
+**Logging**
+- Structured logging (JSON or text)
+- Configurable log levels (debug, info, warn, error)
+- File and console output
+- Log rotation and cleanup
+
+**Error Handling**
+- Graceful degradation
+- User-friendly error messages
+- Error context and debugging info
+- Recovery suggestions
+
+## Data Flow Diagrams
+
+### Task Execution Flow
+
+```
+User Input
+    │
+    ▼
+┌─────────────────────────┐
+│ Parse & Validate Input  │
+└──────────┬──────────────┘
+           │
+           ▼
+┌─────────────────────────┐
+│ License Check           │
+│ (If not cached)         │
+└──────────┬──────────────┘
+           │
+           ▼
+┌─────────────────────────┐
+│ Load Configuration      │
+│ & Environment Variables │
+└──────────┬──────────────┘
+           │
+           ▼
+┌─────────────────────────┐
+│ Initialize Session      │
+│ (Load history if resume)│
+└──────────┬──────────────┘
+           │
+           ▼
+┌─────────────────────────┐
+│ Send to LLM             │
+│ (Planning phase)        │
+└──────────┬──────────────┘
+           │
+           ▼
+┌─────────────────────────┐
+│ Tool Selection Phase    │
+│ (Analyze LLM response)  │
+└──────────┬──────────────┘
+           │
+           ▼
+┌─────────────────────────┐
+│ Execute Tools via MCP   │
+│ (Browser, File I/O,     │
+│  Computer use, etc)     │
+└──────────┬──────────────┘
+           │
+           ▼
+┌─────────────────────────┐
+│ Collect Tool Results    │
+│ (Process output)        │
+└──────────┬──────────────┘
+           │
+           ▼
+┌─────────────────────────┐
+│ Feed Results to LLM     │
+│ (Reasoning phase)       │
+└──────────┬──────────────┘
+           │
+           ▼
+┌─────────────────────────┐
+│ Generate Response       │
+│ (Save to session)       │
+└──────────┬──────────────┘
+           │
+           ▼
+Output to User
+```
+
+### LLM Communication Flow
+
+```
+RIRD Agent
+    │
+    ├─── Remote URL ────────┐
+    │                       ▼
+    │        ┌──────────────────────┐
+    │        │ License Validation    │
+    │        │ (Include license key) │
+    │        └──────────┬───────────┘
+    │                   │
+    │                   ▼
+    │        ┌──────────────────────┐
+    │        │ Model Selection       │
+    │        │ (main/fast/vision)    │
+    │        └──────────┬───────────┘
+    │                   │
+    │                   ▼
+    │        ┌──────────────────────┐
+    │        │ Prepare Prompt       │
+    │        │ (Context + Tools)    │
+    │        └──────────┬───────────┘
+    │                   │
+    │                   ▼
+    │        ┌──────────────────────┐
+    │        │ Send to LLM           │
+    │        │ (via HTTPS)           │
+    │        └──────────┬───────────┘
+    │                   │
+    │                   ▼
+    │        ┌──────────────────────┐
+    │        │ Stream Tokens         │
+    │        │ (Real-time display)   │
+    │        └──────────┬───────────┘
+    │                   │
+    │                   ▼
+    │        ┌──────────────────────┐
+    │        │ Parse Response        │
+    │        │ (Extract tool calls)  │
+    │        └──────────┬───────────┘
+    │                   │
+    └───────────────────┘
+                   │
+                   ▼
+         Execute Tool Calls
+```
+
+## Security Model
+
+### Authentication & Authorization
+
+**License Key Authentication**
+- Token format: `rird_` prefix + 32+ characters
+- Stored in: `~/.rird/license.key` (user-readable, must be protected)
+- Validated: On-demand with 24-hour cache
+- Device-bound: SHA256 fingerprint prevents key sharing
+
+**Device Fingerprinting**
+```
+fingerprint = SHA256(
+  hostname +
+  platform +
+  architecture +
+  total_memory_bytes +
+  cpu_count
+)
+```
+
+Benefits:
+- Prevents unauthorized device use
+- Survives system updates (hostname typically stable)
+- User-local computation only
+- No personal data transmission
+
+**API Key Management**
+- Separate from license key
+- Can be rotated without reinstalling
+- Stored in config file (should be protected via file permissions)
+- Environment variable override for CI/CD safety
+
+### Network Security
+
+**HTTPS Enforcement**
+- All API communication over HTTPS
+- Certificate validation enabled
+- TLS 1.2+ required
+- Cipher suite validation
+
+**Request Signing** (if applicable)
+- Timestamp-based to prevent replay
+- HMAC-SHA256 for integrity
+- Rate limiting on server-side
+
+**Proxy Support**
+- HTTP/HTTPS proxy support
+- Authentication headers passed through
+- Certificate validation preserved
+
+### Data Privacy
+
+**Local-First Architecture**
+- Conversations stored locally in `~/.rird/engine/sessions/`
+- Configuration kept local (not synced)
+- License cache encrypted with device fingerprint
+- No telemetry without explicit consent
+
+**Information Shared with Servers**
+1. **License Validation**:
+   - License key
+   - Device fingerprint (SHA256 hash only)
+   - Version number
+   - Platform/architecture
+
+2. **LLM Requests**:
+   - Task description
+   - Required tool specifications
+   - Previous conversation context
+   - File contents (if user provides)
+
+3. **NOT Shared**:
+   - File system structure (only file contents explicitly provided)
+   - Sensitive credentials in ~/.rird/
+   - Local environment variables (unless explicitly used in task)
+   - Browsing history or local data
+
+**Data Retention**
+- User data retained for billing/support purposes only
+- Can request deletion via support@rird.ai
+- GDPR/CCPA compliant
+- Automatic logs deletion after 90 days
+
+## Anti-Detection Mechanisms
+
+RIRD implements 12+ techniques to avoid bot detection:
+
+### 1. Fingerprint Spoofing
+```javascript
+// Spoof canvas fingerprint
+canvas.toDataURL = () => {
+  // Return modified fingerprint
+  return "data:image/png;base64,..."
+}
+
+// Spoof WebGL
+overrideWebGL()
+```
+
+### 2. Humanized Timing
+```python
+# Add random jitter to all timings
+base_delay = 1000  # 1 second
+jitter = random(0, 500)  # 0-500ms variance
+actual_delay = base_delay + jitter
+time.sleep(actual_delay / 1000)
+```
+
+### 3. Mouse Movement Simulation
+```python
+# Curved trajectory with acceleration/deceleration
+start = (100, 100)
+end = (500, 500)
+for t in smooth_curve(0, 1, steps=50):
+  x = interpolate(start[0], end[0], t)
+  y = interpolate(start[1], end[1], t)
+  move_mouse(x, y)
+  sleep(random(10, 50))
+```
+
+### 4. User-Agent Rotation
+```python
+user_agents = [
+  "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
+  "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36",
+  # ... more variations
+]
+random_agent = random.choice(user_agents)
+```
+
+### 5. Header Manipulation
+```python
+headers = {
+  'User-Agent': random_user_agent(),
+  'Accept-Language': 'en-US,en;q=0.9',
+  'Referer': 'https://google.com',
+  'DNT': '1',
+  'Connection': 'keep-alive'
+}
+```
+
+### 6. Cookie Management
+- Persist cookies across requests
+- Rotate cookies periodically
+- Simulate realistic cookie aging
+- Handle cookie domains properly
+
+### 7. Referrer Spoofing
+- Set realistic referrers
+- Rotate between common sources (Google, direct, etc.)
+- Respect rel="noreferrer" attributes
+
+### 8. Screen Resolution Tricks
+```python
+# Vary reported screen dimensions
+resolutions = [
+  (1920, 1080),  # 1080p
+  (1366, 768),   # HD
+  (2560, 1440),  # 2K
+]
+random_res = random.choice(resolutions)
+set_screen_resolution(random_res)
+```
+
+### 9. Proxy Rotation
+- Support HTTP/HTTPS proxies
+- Rotate through proxy pool
+- Fallback to direct if proxy fails
+- Geographic IP spoofing support
+
+### 10. WebDriver Detection Evasion
+```python
+# Hide webdriver property
+driver.execute_script(
+  "Object.defineProperty(navigator, 'webdriver', "
+  "{get: () => false})"
+)
+```
+
+### 11. CDP Detection Bypass
+```python
+# Chrome DevTools Protocol detection avoidance
+# - Disable --enable-automation flag
+# - Use --disable-blink-features=AutomationControlled
+# - Override cdp property
+```
+
+### 12. Headless Detection Mitigation
+```python
+# Headless-specific detections
+chrome_args = [
+  '--disable-web-resources',
+  '--disable-component-extensions-with-background-pages',
+  '--disable-background-networking'
+]
+```
+
+## Configuration Management
+
+### Configuration Hierarchy (Priority Order)
+
+```
+1. Command-line Arguments (highest priority)
+   └─ rird --config /custom/path.yaml <task>
+
+2. Environment Variables
+   └─ export RIRD_HOME=/custom/rird
+
+3. Config File (YAML or JSON)
+   └─ ~/.rird/engine/config.yaml
+
+4. Default Values (lowest priority)
+   └─ Built-in defaults in binary
+```
+
+### Configuration File Structure
+
+```yaml
+# ~/.rird/engine/config.yaml
+
+app:
+  name: rird
+  version: 2.1.239
+  debug: false
+
+# LLM Configuration
+llm:
+  mode: remote  # 'remote' or 'local'
+  remote_url: https://rird.ai/api/llm
+
+  models:
+    main: 0000/ui-tars-1.5-7b:latest
+    fast: 0000/ui-tars-1.5-7b:latest
+    vision: 0000/ui-tars-1.5-7b:latest
+
+  parameters:
+    temperature: 0.7
+    top_p: 0.9
+    max_tokens: 4096
+    timeout: 30000
+
+# Browser Configuration
+browser:
+  enabled: true
+  headless: true
+  timeout: 30000
+  stealth_mode: true
+  viewport:
+    width: 1920
+    height: 1080
+  proxy: null
+
+# MCP Servers
+mcp:
+  enabled: true
+  servers:
+    - name: stealth-browser-mcp
+      type: browser
+      enabled: true
+    - name: computer-use
+      type: system
+      enabled: true
+
+# Execution Control
+execution:
+  max_concurrent_tasks: 1
+  task_timeout: 300000  # 5 minutes
+  max_retries: 3
+  retry_delay: 1000
+
+  circuit_breaker:
+    enabled: true
+    failure_threshold: 5
+    reset_timeout: 60000
+
+# Logging
+logging:
+  level: info  # debug, info, warn, error
+  format: json  # json or text
+  file: ~/.rird/logs/rird.log
+  max_size: 104857600  # 100MB
+  max_files: 10
+
+# Security
+security:
+  ssl_verify: true
+  insecure_skip_verify: false  # Debug only
+
+# Storage
+storage:
+  sessions_dir: ~/.rird/sessions
+  cache_dir: ~/.rird/cache
+  logs_dir: ~/.rird/logs
+```
+
+## Error Handling Strategy
+
+### Error Classification
+
+**Fatal Errors** (Exit with code 1)
+- License invalid or expired
+- Configuration file corrupted
+- Python environment missing
+- Network completely unavailable
+
+**Recoverable Errors** (Retry with backoff)
+- Temporary network timeout
+- API rate limit (429)
+- Service temporarily unavailable (503)
+- Tool execution failure
+
+**User Errors** (Exit with code 2)
+- Invalid arguments
+- Task description too vague
+- Unsupported file format
+- Missing required input
+
+### Retry Strategy
+
+```python
+def execute_with_retry(fn, max_retries=3, initial_delay=1):
+  """Execute with exponential backoff retry."""
+  for attempt in range(max_retries):
+    try:
+      return fn()
+    except RecoverableError as e:
+      if attempt >= max_retries - 1:
+        raise
+
+      delay = initial_delay * (2 ** attempt)
+      jitter = random(0, delay * 0.1)
+      total_delay = delay + jitter
+
+      logger.warn(
+        f"Retry {attempt + 1}/{max_retries} after {total_delay}ms: {e}"
+      )
+      sleep(total_delay / 1000)
+```
+
+### Error Messages
+
+Clear, actionable error messages with:
+- What went wrong
+- Why it happened
+- How to fix it
+- Where to get help
+
+**Example**:
+```
+Error: Invalid license key
+
+The license key you provided is not in the correct format.
+
+Expected format: rird_<32+ characters>
+
+Fix:
+1. Go to https://rird.ai/dashboard
+2. Copy your license key (starts with 'rird_')
+3. Run: rird activate YOUR_LICENSE_KEY
+
+Need help? Email support@rird.ai
+```
+
+## Performance Characteristics
+
+### Time Complexity
+
+| Operation | Time | Notes |
+|-----------|------|-------|
+| License validation (cached) | 0ms | Local file read |
+| License validation (online) | 100-500ms | Network request |
+| Config load and parse | 5-20ms | YAML parsing |
+| Task initialization | 50-200ms | MCP server startup |
+| LLM request | 1-10s | Depends on task complexity |
+| Tool execution | Variable | Depends on tool (browser 5-30s) |
+
+### Space Complexity
+
+| Component | Memory | Notes |
+|-----------|--------|-------|
+| RIRD binary | 156MB | Standalone executable |
+| Runtime overhead | 50-100MB | Node.js/Python process |
+| Session history | 1-50MB | Depends on conversation length |
+| Cache | 100-500MB | Configurable |
+| Total | 300-700MB | Typical usage |
+
+## Concurrency Model
+
+**Single-threaded by design**
+- One task executed at a time
+- Prevents race conditions in file I/O
+- Simplified state management
+- Clear execution ordering
+
+**Future: Parallel Execution**
+```bash
+rird --parallel 4 batch-process-files
+```
+Would process 4 files concurrently with separate agent instances.
+
+## Testing and Quality
+
+### Test Pyramid
+
+```
+        /\
+       /  \ Unit Tests (60%)
+      /────\
+     /      \ Integration Tests (25%)
+    /────────\
+   /          \ E2E Tests (15%)
+  /────────────\
+ / Smoke Tests  \
+/────────────────\
+```
+
+### Coverage Goals
+- Unit: 80%+ coverage
+- Integration: Key workflows
+- E2E: Critical user paths
+- Performance: Benchmarks tracked
+
+## Deployment Architecture
+
+### Single Machine
+```
+User -> RIRD Binary -> LLM API -> Result
+```
+
+### Server Mode (Cloud)
+```
+Clients -> API Server (rird serve)
+          /  |  \
+    Task Q  Worker  Status DB
+         |
+    LLM API
+```
+
+### Kubernetes (Enterprise)
+```
+Load Balancer
+      |
+  +---+---+
+  |   |   |
+Pod Pod Pod  <- RIRD containers (scaled)
+  |   |   |
+  +---+---+
+      |
+  Task Queue (Redis)
+      |
+  LLM API
+```
+
+## Monitoring and Observability
+
+### Metrics Collected
+- Task execution time
+- Success/failure rate
+- Tool usage statistics
+- LLM token consumption
+- Error frequencies
+
+### Health Checks
+```bash
+rird health-check
+```
+
+Verifies:
+- License validity
+- Python environment
+- MCP server status
+- Network connectivity
+- Disk space availability
+
+### Logging Levels
+
+| Level | Use Case | Example |
+|-------|----------|---------|
+| ERROR | Fatal issues | License validation failed |
+| WARN | Concerning but recoverable | Retry attempt 2 of 3 |
+| INFO | Normal operations | Task started, tool called |
+| DEBUG | Detailed troubleshooting | Full request/response bodies |
+
+## Future Architecture Enhancements
+
+1. **Plugin System**: Allow third-party tool extensions
+2. **Distributed Execution**: Process tasks across multiple machines
+3. **Real-time Collaboration**: Multiple users on shared sessions
+4. **Advanced Caching**: Persistent learned patterns
+5. **Custom Models**: Support for fine-tuned domain-specific models
+6. **Multi-agent Coordination**: Teams of specialized agents