grim-reaper 1.0.29

data/docs/WEB_SERVICES_APIS.md

@@ -0,0 +1,573 @@
+////////////////////////////////////////////
+// curl -fsSL https://grim.so | sudo bash //
+//     ██████╗ ██████╗ ██╗███╗   ███╗     //
+//    ██╔════╝ ██╔══██╗██║████╗ ████║     //
+//    ██║  ███╗██████╔╝██║██╔████╔██║     //
+//    ██║   ██║██╔══██╗██║██║╚██╔╝██║     //
+//    ╚██████╔╝██║  ██║██║██║ ╚═╝ ██║     //
+//     ╚═════╝ ╚═╝  ╚═╝╚═╝╚═╝     ╚═╝     //
+//     Death Defying Data Protection      //
+////////////////////////////////////////////
+
+# 🌐 Web Services & APIs
+
+**The Digital Interface of Grim Reaper** - High-performance web services and REST APIs that provide seamless access to all Grim Reaper capabilities through modern web interfaces, real-time dashboards, and comprehensive API endpoints.
+
+## Overview
+
+The Web Services & APIs category provides modern web interfaces, REST APIs, and real-time monitoring capabilities. It includes FastAPI-based web services, interactive dashboards, API gateways, and WebSocket support for real-time communication and monitoring.
+
+## Architecture
+
+```
+    🌐 WEB SERVICES & APIs LAYER
+           |
+    ┌──────┼──────┐
+    │      │      │
+FastAPI   Dashboard API
+Services  Interface Gateway
+```
+
+## Core Components
+
+### 🚀 FastAPI Web Services (sh_grim/dashboard.sh + py_grim/dashboard integration)
+
+**Purpose:** High-performance web dashboard with real-time monitoring and management interface.
+
+#### Key Features
+- **Interactive Dashboard**: Real-time system monitoring and control
+- **REST API Integration**: Full API access to all system functions
+- **Real-time Updates**: Live data streaming and updates
+- **Responsive Design**: Mobile-friendly responsive interface
+- **User Management**: Multi-user access and role management
+- **Customizable Views**: Configurable dashboard layouts
+
+#### Commands
+```bash
+grim dashboard start            # Start web dashboard
+grim dashboard stop             # Stop dashboard
+grim dashboard restart          # Restart dashboard
+grim dashboard status           # Check dashboard status
+grim dashboard config           # Configure dashboard
+grim dashboard init             # Initialize dashboard
+grim dashboard setup            # Run setup wizard
+grim dashboard logs             # View dashboard logs
+grim dashboard help             # Display dashboard help
+```
+
+#### Dashboard Features
+- **System Overview**: Real-time system status and metrics
+- **Performance Monitoring**: Live performance data and graphs
+- **Backup Management**: Backup status and management interface
+- **Security Dashboard**: Security status and threat monitoring
+- **Configuration Management**: System configuration interface
+- **Log Viewer**: Real-time log viewing and analysis
+
+#### Configuration
+```yaml
+dashboard_configuration:
+  server:
+    host: "0.0.0.0"
+    port: 8080
+    workers: 4
+    timeout: 30
+    
+  security:
+    ssl_enabled: false
+    authentication: true
+    session_timeout: 3600
+    
+  features:
+    real_time_updates: true
+    websocket_enabled: true
+    api_access: true
+    
+  customization:
+    theme: "dark"
+    refresh_interval: 5
+    max_data_points: 1000
+```
+
+### 🌐 FastAPI Web Server (py_grim/grim_web/* via throne)
+
+**Purpose:** High-performance FastAPI web server with comprehensive REST API endpoints.
+
+#### Key Features
+- **FastAPI Framework**: Modern, fast web framework
+- **REST API**: Comprehensive REST API endpoints
+- **OpenAPI Documentation**: Auto-generated API documentation
+- **WebSocket Support**: Real-time bidirectional communication
+- **Middleware Support**: CORS, authentication, rate limiting
+- **Async Support**: High-performance async operations
+
+#### Commands
+```bash
+grim web start              # Start FastAPI web server
+grim web stop               # Stop all web services
+grim web restart            # Restart web server
+grim web gateway            # Start API gateway with load balancing
+grim web api                # Start API application
+grim web status             # Show web services status
+grim web help               # Display web help
+```
+
+#### API Endpoints
+- **Health Check**: `GET /health` - System health status
+- **Root Endpoint**: `GET /` - API documentation and root
+- **TuskLang Integration**: `GET /tusktsk/*` - TuskLang integration endpoints
+- **REST API**: `GET /api/*` - Comprehensive REST API endpoints
+- **WebSocket**: `ws://localhost:8080/ws` - Real-time monitoring
+
+#### API Documentation
+```yaml
+api_endpoints:
+  health:
+    path: "/health"
+    method: "GET"
+    description: "System health check"
+    
+  metrics:
+    path: "/api/metrics"
+    method: "GET"
+    description: "System metrics"
+    
+  status:
+    path: "/api/status"
+    method: "GET"
+    description: "Service status"
+    
+  backup:
+    path: "/api/backup"
+    method: "POST"
+    description: "Create backup"
+    
+  restore:
+    path: "/api/restore"
+    method: "POST"
+    description: "Restore from backup"
+```
+
+### 🚪 API Gateway (py_grim/grim_gateway/gateway.py via throne)
+
+**Purpose:** Load-balanced API gateway with advanced routing and security features.
+
+#### Key Features
+- **Load Balancing**: Intelligent load balancing across services
+- **Rate Limiting**: API rate limiting and throttling
+- **Authentication**: Centralized authentication and authorization
+- **Request Routing**: Advanced request routing and filtering
+- **Monitoring**: Gateway performance monitoring
+- **Security**: API security and protection features
+
+#### Commands
+```bash
+grim gateway start                       # Start API gateway
+grim gateway stop                        # Stop gateway
+grim gateway status                      # Gateway status
+grim gateway config                      # Configure gateway
+grim gateway help                        # Display gateway help
+```
+
+#### Gateway Features
+- **Load Balancing**: Round-robin, least connections, weighted
+- **Health Checks**: Service health monitoring and failover
+- **SSL Termination**: SSL/TLS termination and management
+- **Request Filtering**: Request validation and filtering
+- **Response Caching**: Intelligent response caching
+- **Logging**: Comprehensive request/response logging
+
+### 🔌 WebSocket Support
+
+**Purpose:** Real-time bidirectional communication for live monitoring and updates.
+
+#### Key Features
+- **Real-time Updates**: Live data streaming and updates
+- **Bidirectional Communication**: Full-duplex communication
+- **Connection Management**: Automatic connection management
+- **Event Broadcasting**: Broadcast events to multiple clients
+- **Authentication**: WebSocket authentication and security
+- **Performance Optimization**: Optimized for high-performance
+
+#### WebSocket Endpoints
+- **Monitoring**: `ws://localhost:8080/ws/monitoring` - Real-time monitoring
+- **Alerts**: `ws://localhost:8080/ws/alerts` - Real-time alerts
+- **Logs**: `ws://localhost:8080/ws/logs` - Live log streaming
+- **Metrics**: `ws://localhost:8080/ws/metrics` - Live metrics
+
+#### WebSocket Events
+```json
+{
+  "type": "system_update",
+  "timestamp": "2024-01-15T10:30:00Z",
+  "data": {
+    "cpu_usage": 45.2,
+    "memory_usage": 67.8,
+    "disk_usage": 23.1
+  }
+}
+```
+
+## API Reference
+
+### Core API Endpoints
+
+#### System Management
+```http
+GET /api/system/status          # Get system status
+GET /api/system/health          # Get system health
+GET /api/system/info            # Get system information
+POST /api/system/restart        # Restart system services
+```
+
+#### Backup Management
+```http
+GET /api/backup/list            # List all backups
+POST /api/backup/create         # Create new backup
+GET /api/backup/{id}/status     # Get backup status
+POST /api/backup/{id}/restore   # Restore from backup
+DELETE /api/backup/{id}         # Delete backup
+```
+
+#### Monitoring
+```http
+GET /api/monitoring/metrics     # Get system metrics
+GET /api/monitoring/alerts      # Get active alerts
+POST /api/monitoring/start      # Start monitoring
+POST /api/monitoring/stop       # Stop monitoring
+```
+
+#### Security
+```http
+GET /api/security/scan          # Run security scan
+GET /api/security/status        # Get security status
+POST /api/security/fix          # Fix security issues
+GET /api/security/report        # Get security report
+```
+
+### Authentication & Authorization
+
+#### API Authentication
+```http
+POST /api/auth/login            # Login
+POST /api/auth/logout           # Logout
+GET /api/auth/status            # Check authentication status
+POST /api/auth/refresh          # Refresh token
+```
+
+#### API Keys
+```http
+GET /api/keys/list              # List API keys
+POST /api/keys/create           # Create new API key
+DELETE /api/keys/{id}           # Delete API key
+```
+
+### Response Formats
+
+#### Success Response
+```json
+{
+  "success": true,
+  "data": {
+    "id": "backup_123",
+    "status": "completed",
+    "size": "1.2GB",
+    "created_at": "2024-01-15T10:30:00Z"
+  },
+  "message": "Backup created successfully"
+}
+```
+
+#### Error Response
+```json
+{
+  "success": false,
+  "error": {
+    "code": "BACKUP_FAILED",
+    "message": "Backup operation failed",
+    "details": "Insufficient disk space"
+  },
+  "timestamp": "2024-01-15T10:30:00Z"
+}
+```
+
+## Integration Patterns
+
+### Complete Web Service Setup
+```bash
+# 1. Initialize web services
+grim web init
+
+# 2. Start FastAPI server
+grim web start
+
+# 3. Start API gateway
+grim web gateway
+
+# 4. Start dashboard
+grim dashboard start
+
+# 5. Verify services
+grim web status
+```
+
+### API Integration Example
+```bash
+# Health check
+curl http://localhost:8000/health
+
+# Create backup via API
+curl -X POST http://localhost:8000/api/backup/create \
+  -H "Content-Type: application/json" \
+  -d '{"name": "api_backup", "path": "/data"}'
+
+# Get system metrics
+curl http://localhost:8000/api/metrics
+
+# WebSocket connection
+wscat -c ws://localhost:8080/ws/monitoring
+```
+
+### Dashboard Integration
+```bash
+# Access dashboard
+open http://localhost:8080
+
+# API documentation
+open http://localhost:8000/docs
+
+# Health status
+curl http://localhost:8000/health
+
+# System status
+curl http://localhost:8000/api/status
+```
+
+## Configuration
+
+### Web Services Configuration
+```yaml
+web_services_configuration:
+  fastapi:
+    host: "0.0.0.0"
+    port: 8000
+    workers: 4
+    timeout: 30
+    reload: false
+    
+  dashboard:
+    host: "0.0.0.0"
+    port: 8080
+    refresh_interval: 5
+    theme: "dark"
+    
+  gateway:
+    host: "0.0.0.0"
+    port: 9000
+    load_balancing: true
+    rate_limiting: true
+    
+  security:
+    ssl_enabled: false
+    authentication: true
+    cors_enabled: true
+    rate_limit: 1000
+```
+
+### API Configuration
+```yaml
+api_configuration:
+  endpoints:
+    health_check: "/health"
+    metrics: "/api/metrics"
+    status: "/api/status"
+    
+  authentication:
+    enabled: true
+    method: "jwt"
+    token_expiry: 3600
+    
+  rate_limiting:
+    enabled: true
+    requests_per_minute: 1000
+    burst_limit: 100
+    
+  logging:
+    level: "INFO"
+    format: "json"
+    file: "/var/log/grim/api.log"
+```
+
+### Dashboard Configuration
+```yaml
+dashboard_configuration:
+  interface:
+    theme: "dark"
+    language: "en"
+    timezone: "UTC"
+    
+  features:
+    real_time_updates: true
+    websocket_enabled: true
+    api_access: true
+    
+  customization:
+    refresh_interval: 5
+    max_data_points: 1000
+    chart_types: ["line", "bar", "pie"]
+    
+  security:
+    session_timeout: 3600
+    max_sessions: 10
+    ip_whitelist: []
+```
+
+## Best Practices
+
+### API Design
+1. **RESTful Design**: Follow REST principles
+2. **Consistent Naming**: Use consistent naming conventions
+3. **Error Handling**: Implement proper error handling
+4. **Versioning**: Use API versioning
+5. **Documentation**: Maintain comprehensive documentation
+
+### Security
+1. **Authentication**: Implement proper authentication
+2. **Authorization**: Use role-based access control
+3. **Rate Limiting**: Implement rate limiting
+4. **Input Validation**: Validate all inputs
+5. **HTTPS**: Use HTTPS in production
+
+### Performance
+1. **Caching**: Implement appropriate caching
+2. **Load Balancing**: Use load balancing
+3. **Monitoring**: Monitor API performance
+4. **Optimization**: Optimize database queries
+5. **Compression**: Use response compression
+
+## Troubleshooting
+
+### Common Issues
+
+#### Web Service Failures
+```bash
+# Check web service status
+grim web status
+
+# View web service logs
+grim log tail web.log
+
+# Restart web services
+grim web restart
+
+# Check port availability
+grim health-check network
+```
+
+#### Dashboard Issues
+```bash
+# Check dashboard status
+grim dashboard status
+
+# Restart dashboard
+grim dashboard restart
+
+# View dashboard logs
+grim dashboard logs
+
+# Check dashboard configuration
+grim dashboard config
+```
+
+#### API Gateway Issues
+```bash
+# Check gateway status
+grim gateway status
+
+# Restart gateway
+grim gateway restart
+
+# Check gateway configuration
+grim gateway config
+
+# View gateway logs
+grim log tail gateway.log
+```
+
+#### WebSocket Issues
+```bash
+# Test WebSocket connection
+wscat -c ws://localhost:8080/ws/monitoring
+
+# Check WebSocket status
+grim web status --websocket
+
+# View WebSocket logs
+grim log tail websocket.log
+```
+
+## Performance Metrics
+
+### Key Performance Indicators
+- **Response Time**: <100ms for API requests
+- **Throughput**: >1000 requests/second
+- **Uptime**: >99.9%
+- **Error Rate**: <0.1%
+- **Concurrent Connections**: >1000 WebSocket connections
+
+### Monitoring Dashboard
+Access web service metrics at:
+- **API Dashboard**: http://localhost:8080/api-metrics
+- **Performance Dashboard**: http://localhost:8080/performance
+- **Health Dashboard**: http://localhost:8080/health
+- **API Documentation**: http://localhost:8000/docs
+
+## Development
+
+### API Development
+```bash
+# Start development server
+grim web start --dev
+
+# Run API tests
+grim test-framework web
+
+# Generate API documentation
+grim docs generate api-docs
+
+# Validate API schema
+grim web validate-schema
+```
+
+### Dashboard Development
+```bash
+# Start dashboard in development mode
+grim dashboard start --dev
+
+# Customize dashboard
+grim dashboard customize
+
+# Add custom widgets
+grim dashboard add-widget
+
+# Test dashboard functionality
+grim dashboard test
+```
+
+## Future Enhancements
+
+### Planned Features
+- **GraphQL Support**: GraphQL API implementation
+- **Microservices**: Microservices architecture
+- **Container Support**: Docker and Kubernetes support
+- **Cloud Integration**: Multi-cloud API management
+- **Advanced Analytics**: Advanced analytics and reporting
+
+### Roadmap
+- **Q1 2024**: GraphQL API implementation
+- **Q2 2024**: Microservices architecture
+- **Q3 2024**: Container support
+- **Q4 2024**: Cloud integration
+
+---
+
+**The Web Services & APIs provide modern, high-performance interfaces for accessing all Grim Reaper capabilities through web dashboards and REST APIs.** 

data/lib/grim_reaper/core.rb

@@ -0,0 +1,130 @@
+# frozen_string_literal: true
+
+require "json"
+require "yaml"
+require "open3"
+
+module GrimReaper
+  # Main Grim Reaper class that orchestrates all modules
+  class Core
+    attr_reader :config, :modules, :grim_root
+
+    def initialize(config = {})
+      @config = config
+      @grim_root = find_grim_root
+      @modules = {}
+      load_modules
+    end
+
+    # Find Grim Reaper root directory
+    def find_grim_root
+      # Try to find from current directory
+      current_dir = Dir.pwd
+      10.times do
+        if File.exist?(File.join(current_dir, "throne", "grim_throne.sh")) ||
+           File.exist?(File.join(current_dir, "throne", "rb_grim_throne.sh")) ||
+           File.exist?(File.join(current_dir, "tsk_flask", "grim_admin_server.py"))
+          return current_dir
+        end
+
+        parent_dir = File.dirname(current_dir)
+        break if parent_dir == current_dir
+        current_dir = parent_dir
+      end
+
+      # Try common installation paths
+      possible_paths = [
+        ENV["HOME"] + "/reaper",
+        ENV["HOME"] + "/.reaper",
+        "/root/reaper",
+        "/root/.reaper",
+        "/usr/local/reaper",
+        "/usr/share/reaper",
+        Dir.pwd
+      ]
+
+      possible_paths.each do |path|
+        if Dir.exist?(path) && (
+          File.exist?(File.join(path, "throne", "grim_throne.sh")) ||
+          File.exist?(File.join(path, "throne", "rb_grim_throne.sh")) ||
+          File.exist?(File.join(path, "tsk_flask", "grim_admin_server.py"))
+        )
+          return path
+        end
+      end
+
+      raise Error, "Could not find Grim Reaper root directory"
+    end
+
+    # Load all available Grim Reaper modules
+    def load_modules
+      @modules[:shell] = ShellModule.new(@config, @grim_root)
+      @modules[:python] = PythonModule.new(@config, @grim_root)
+      @modules[:go] = GoModule.new(@config, @grim_root)
+      @modules[:security] = SecurityModule.new(@config, @grim_root)
+    rescue LoadError => e
+      warn "Warning: Could not load module: #{e.message}"
+    end
+
+    # Execute a command across all modules
+    def execute(command, *args)
+      results = {}
+      @modules.each do |name, module_instance|
+        begin
+          results[name] = module_instance.execute(command, *args)
+        rescue => e
+          results[name] = { error: e.message, module: name.to_s }
+        end
+      end
+      results
+    end
+
+    # Get status from all modules
+    def status
+      status_data = {}
+      @modules.each do |name, module_instance|
+        begin
+          status_data[name] = module_instance.status
+        rescue => e
+          status_data[name] = { status: "error", error: e.message, module: name.to_s }
+        end
+      end
+      status_data
+    end
+
+    # Execute a Ruby-specific command
+    def execute_ruby_command(command, *args)
+      ruby_throne_path = File.join(@grim_root, "throne", "rb_grim_throne.sh")
+      
+      unless File.exist?(ruby_throne_path)
+        raise Error, "Ruby throne script not found at #{ruby_throne_path}"
+      end
+
+      unless File.executable?(ruby_throne_path)
+        File.chmod(0o755, ruby_throne_path)
+      end
+
+      cmd = [ruby_throne_path, command, *args].compact
+      stdout, stderr, status = Open3.capture3(*cmd, chdir: @grim_root)
+
+      {
+        command: command,
+        args: args,
+        stdout: stdout,
+        stderr: stderr,
+        exit_code: status.exitstatus,
+        success: status.success?
+      }
+    end
+
+    # Get backup directory
+    def backup_dir
+      ENV["HOME"] + "/.graveyard"
+    end
+
+    # Get installation directory
+    def install_dir
+      ENV["HOME"] + "/reaper"
+    end
+  end
+end