@mmmbuto/nexuscli 0.8.9 → 0.9.0

package/lib/server/docs/ARCHITECTURE.md

@@ -1,441 +0,0 @@
-# NexusCLI Architecture Overview
-
-**Version**: 0.1.0
-**Created**: 2025-11-17
-**Status**: Design Phase
-
----
-
-## 🎯 Vision
-
-NexusCLI is a **distributed control plane** for orchestrating CLI operations across multiple remote nodes, inspired by the architecture of NexusChat's wrapper system but generalized for any CLI tool.
-
-**Philosophy**: Separate "what to do" (Control Plane) from "how to execute" (Node Agents).
-
----
-
-## 🏗️ System Architecture
-
-### High-Level Overview
-
-```
-┌─────────────────────────────────────────────────────────────────────┐
-│                          WEB INTERFACE                              │
-│                      (React + Vite Frontend)                        │
-│                                                                     │
-│  - Job submission dashboard                                         │
-│  - Real-time execution logs (SSE streaming)                         │
-│  - Node status monitoring                                           │
-│  - Historical job results                                           │
-└────────────────────────────┬────────────────────────────────────────┘
-                             │ REST API + SSE
-                             ↓
-┌─────────────────────────────────────────────────────────────────────┐
-│                      CONTROL PLANE (NexusCLI)                       │
-│                      Express.js Backend                             │
-│                                                                     │
-│  ┌─────────────┐  ┌──────────────┐  ┌───────────────┐             │
-│  │   Job       │  │    Node      │  │   Stream      │             │
-│  │   Manager   │  │   Registry   │  │   Manager     │             │
-│  └─────────────┘  └──────────────┘  └───────────────┘             │
-│                                                                     │
-│  ┌─────────────────────────────────────────────────────────┐      │
-│  │              Session Storage (RAM + Disk)                │      │
-│  │  - Active jobs (Map)                                     │      │
-│  │  - Completed jobs (JSON files)                           │      │
-│  │  - Node registry (in-memory + periodic sync)             │      │
-│  └─────────────────────────────────────────────────────────┘      │
-└────────────────────────────┬────────────────────────────────────────┘
-                             │ HTTP REST API
-                             ↓
-         ┌───────────────────┴───────────────────┐
-         │                                       │
-         ↓                                       ↓
-┌──────────────────┐                    ┌──────────────────┐
-│  NEXUS-WRAPPER   │                    │  NEXUS-WRAPPER   │
-│    (Node 1)      │                    │    (Node 2)      │
-│                  │                    │                  │
-│  - HTTP Server   │       ...          │  - HTTP Server   │
-│  - PTY Spawner   │                    │  - PTY Spawner   │
-│  - Output Parser │                    │  - Output Parser │
-│  - Job Storage   │                    │  - Job Storage   │
-└────────┬─────────┘                    └────────┬─────────┘
-         │                                       │
-         ↓                                       ↓
-  ┌──────────────┐                       ┌──────────────┐
-  │  CLI Tools   │                       │  CLI Tools   │
-  │ (bash, git,  │                       │ (python,     │
-  │  docker...)  │                       │  npm...)     │
-  └──────────────┘                       └──────────────┘
-```
-
----
-
-## 🧩 Components
-
-### 1. Control Plane (NexusCLI Backend)
-
-**Location**: `/var/www/cli.wellanet.dev/backend/`
-
-**Responsibilities**:
-- Accept job requests from frontend/API clients
-- Maintain registry of available nodes
-- Route jobs to appropriate nodes
-- Aggregate and stream execution events
-- Persist job history and results
-
-**Key Modules**:
-
-#### Job Manager
-```javascript
-// Handles job lifecycle
-class JobManager {
-  create(jobSpec)      // Create and queue job
-  assign(jobId, nodeId) // Assign job to node
-  execute(jobId)       // Trigger execution on wrapper
-  stream(jobId)        // SSE stream of job events
-  cancel(jobId)        // Kill running job
-  getResult(jobId)     // Retrieve completed job result
-}
-```
-
-#### Node Registry
-```javascript
-// Tracks registered nodes
-class NodeRegistry {
-  register(nodeInfo)   // Register new node
-  heartbeat(nodeId)    // Update node status
-  getAvailable()       // Get online nodes
-  selectNode(criteria) // Select best node for job
-  unregister(nodeId)   // Remove offline node
-}
-```
-
-#### Stream Manager (SSE)
-```javascript
-// Manages SSE streams to clients
-class StreamManager {
-  createStream(jobId)  // Open SSE connection
-  emit(jobId, event)   // Send event to stream
-  closeStream(jobId)   // Close connection
-}
-```
-
----
-
-### 2. Node Agent (nexus-wrapper)
-
-**Future Location**: Separate repository (e.g., `~/Dev/nexus-wrapper/`)
-
-**Responsibilities**:
-- Expose HTTP API for job execution
-- Spawn PTY processes for CLI tools
-- Parse stdout/stderr into structured events
-- Stream events back to Control Plane
-- Store local job results
-
-**Architecture** (based on NexusChat claude-wrapper):
-
-```javascript
-// Main server (Express.js)
-const app = express();
-
-// Wrapper class (spawns CLI processes)
-class CliWrapper {
-  constructor(binPath) {
-    this.binPath = binPath;
-    this.activeJobs = new Map();
-  }
-
-  async execute({ jobId, command, workingDir, timeout, onStatus }) {
-    // Spawn PTY process (like NexusChat)
-    const ptyProcess = pty.spawn('/bin/bash', ['-c', command], {
-      cwd: workingDir,
-      env: process.env,
-    });
-
-    // Parse output
-    const parser = new OutputParser();
-
-    ptyProcess.onData((data) => {
-      const events = parser.parse(data);
-      events.forEach(event => onStatus(event));
-    });
-
-    ptyProcess.onExit(({ exitCode }) => {
-      onStatus({ type: 'done', exitCode });
-    });
-  }
-}
-
-// Output Parser (adapted from NexusChat)
-class OutputParser {
-  parse(chunk) {
-    // Regex-based parsing
-    // Returns: [{ type, category, message, ... }]
-  }
-}
-
-// Session Manager (2-layer persistence)
-class SessionManager {
-  constructor(storageDir) {
-    this.jobs = new Map();      // RAM cache
-    this.storageDir = storageDir; // Disk storage
-  }
-
-  create(jobId, data) { /* ... */ }
-  get(jobId) { /* ... */ }
-  update(jobId, data) { /* ... */ }
-  delete(jobId) { /* ... */ }
-}
-```
-
----
-
-### 3. Frontend (React Dashboard)
-
-**Location**: `/var/www/cli.wellanet.dev/frontend/`
-
-**Components**:
-
-- **Node Status Panel**: Live status of all registered nodes
-- **Job Submission Form**: Create new jobs with tool/command selection
-- **Execution Log**: Real-time streaming output (SSE)
-- **Job History**: Browse past job results
-- **Batch Operations**: Execute jobs on multiple nodes
-
----
-
-## 🔄 Execution Flow
-
-### Single Job Execution
-
-```
-1. User submits job via frontend
-   POST /api/v1/jobs
-   {
-     "nodeId": "node-001",
-     "tool": "bash",
-     "command": "systemctl status nginx"
-   }
-
-2. Control Plane validates and queues job
-   - Check node is online
-   - Create job record
-   - Return jobId + stream endpoint
-
-3. Control Plane forwards job to wrapper
-   POST http://node-001:5000/api/v1/jobs
-   {
-     "jobId": "job-123",
-     "tool": "bash",
-     "command": "systemctl status nginx"
-   }
-
-4. Wrapper spawns PTY process
-   - Launch bash command in PTY
-   - Attach output parser
-   - Start SSE stream to Control Plane
-
-5. Wrapper streams events
-   - Parse stdout line-by-line
-   - Emit status events (executing, tool, output_chunk)
-   - Forward to Control Plane
-
-6. Control Plane aggregates and relays
-   - Receive events from wrapper
-   - Add metadata (timestamp, nodeId)
-   - Stream to frontend via SSE
-
-7. Frontend displays real-time output
-   - Render log entries
-   - Update status indicators
-   - Show completion notification
-
-8. Job completes
-   - Wrapper saves result to disk
-   - Control Plane updates job status
-   - Close SSE streams
-```
-
----
-
-## 📊 Data Flow
-
-### Request → Response Flow
-
-```
-┌──────────┐   1. POST /jobs    ┌──────────────┐   2. POST /jobs   ┌──────────┐
-│ Frontend │ ──────────────────▶│ Control Plane│ ────────────────▶ │ Wrapper  │
-└──────────┘                    └──────────────┘                   └──────────┘
-     │                               │                                   │
-     │ 3. GET /jobs/:id/stream       │                                   │
-     │    (SSE connection)            │                                   │
-     │◀──────────────────────────────┤                                   │
-     │                               │                                   │
-     │                               │ 4. Spawn PTY (node-pty)           │
-     │                               │ ◀─────────────────────────────────┤
-     │                               │                                   │
-     │                               │ 5. SSE events                     │
-     │                               │    (status, output_chunk, done)   │
-     │                               │ ◀─────────────────────────────────┤
-     │                               │                                   │
-     │ 6. Relay SSE events           │                                   │
-     │ ◀─────────────────────────────┤                                   │
-     │   data: {"type":"output_...   │                                   │
-     │   data: {"type":"done"...     │                                   │
-     │                               │                                   │
-     │ 7. GET /jobs/:id (final)      │                                   │
-     │ ──────────────────────────────▶                                   │
-     │ ◀─────────────────────────────┤                                   │
-     │   {"status":"completed",...}  │                                   │
-```
-
----
-
-## 🔐 Security Model
-
-### Authentication
-
-**Control Plane**:
-- JWT tokens for API clients (frontend, external services)
-- Role-based access control (RBAC)
-- Rate limiting per user
-
-**Node Wrapper**:
-- Bearer token authentication (issued during registration)
-- IP whitelist (optional - only accept Control Plane IP)
-- TLS/SSL for HTTP communication
-
-### Authorization
-
-**Job Execution**:
-- Users can only execute jobs on nodes they have access to
-- Tool restrictions per node (e.g., production nodes: read-only tools)
-- Command whitelisting/blacklisting
-
-### Isolation
-
-**Sandbox**:
-- Each job runs in isolated working directory
-- No access to wrapper process environment
-- Resource limits (CPU, memory, timeout)
-
----
-
-## 📈 Scalability Considerations
-
-### Horizontal Scaling
-
-**Control Plane**:
-- Stateless API server (can run multiple instances)
-- Shared Redis for session storage (alternative to disk)
-- Load balancer for API requests
-
-**Node Wrappers**:
-- Independent agents (no coordination needed)
-- Auto-scaling based on job queue length
-- Node pooling (reserve capacity for burst loads)
-
-### Performance Optimizations
-
-**Based on NexusChat production experience**:
-
-1. **PTY Buffering**: Use `node-pty` for efficient I/O
-2. **Event Batching**: Group multiple output_chunk events
-3. **Compression**: Gzip SSE streams for large outputs
-4. **Caching**: Cache node capabilities in Redis
-5. **Lazy Loading**: Load job results on-demand from disk
-
----
-
-## 🧪 Testing Strategy
-
-### Unit Tests
-
-- Job Manager: create, assign, cancel operations
-- Node Registry: registration, heartbeat, selection logic
-- Output Parser: regex patterns, state machine transitions
-
-### Integration Tests
-
-- End-to-end job execution flow
-- SSE streaming accuracy
-- Error handling and retries
-- Concurrent job execution
-
-### Load Tests
-
-- 100 concurrent jobs across 10 nodes
-- 1000 jobs/hour sustained throughput
-- Node failure recovery
-- Network partition tolerance
-
----
-
-## 🚀 Deployment
-
-### Control Plane (VPS1)
-
-```bash
-# Production deployment
-cd /var/www/cli.wellanet.dev/backend
-npm install --production
-npm run build
-
-# Systemd service
-sudo systemctl start nexuscli-backend.service
-sudo systemctl enable nexuscli-backend.service
-```
-
-### Node Wrapper (any server)
-
-```bash
-# Install wrapper
-cd /opt/nexus-wrapper
-npm install --production
-
-# Configure
-cat > config.json <<EOF
-{
-  "controlPlaneUrl": "https://cli.wellanet.dev",
-  "nodeId": "node-001",
-  "port": 5000,
-  "tools": ["bash", "git", "docker"]
-}
-EOF
-
-# Start service
-sudo systemctl start nexus-wrapper.service
-sudo systemctl enable nexus-wrapper.service
-```
-
----
-
-## 🔮 Future Enhancements
-
-### Phase 2: Advanced Features
-
-- **WebSocket support**: Alternative to SSE for bidirectional communication
-- **Job scheduling**: Cron-like scheduling for recurring jobs
-- **Job templates**: Predefined workflows (e.g., "Deploy to Production")
-- **Approval workflows**: Multi-stage approvals for sensitive operations
-
-### Phase 3: WellaNet Integration
-
-- **Compute marketplace**: Nodes bid for job execution
-- **Wellacoin tokens**: Pay-per-execution model
-- **Resource optimization**: AI-based node selection
-- **Decentralized registry**: Blockchain-based node tracking
-
----
-
-## 📚 Related Documents
-
-- [API & Wrapper Contract](./API_WRAPPER_CONTRACT.md) - Detailed API specification
-- [NexusChat CLAUDE.md](../../CLAUDE.md) - Development workflow
-- [NexusChat Wrapper Source](/var/www/chat.mmmbuto.com/servers/claude-code-server/) - Reference implementation
-
----
-
-_Generated by Claude Code (Sonnet 4.5) - 2025-11-17_