flow-frame-core 0.1.0

package/README.md

@@ -0,0 +1,64 @@
+# Flow-Frame
+
+**Flow-Frame** is an advanced automation platform designed to orchestrate complex workflows involving AI, media processing, browser interactions, and email management.
+
+It acts as a central nervous system for automation, connecting LLMs (OpenAI) with real-world interfaces (Gmail, Chrome, File System) and heavy computation tasks (Image/Audio processing).
+
+## 🚀 Quick Start
+
+1.  **Install Dependencies**:
+    ```bash
+    npm install
+    ```
+2.  **Setup Environment**:
+    Create a `.env` file with your API keys.
+3.  **Start Server**:
+    ```bash
+    npm start
+    ```
+
+## 🧪 Testing
+
+This project uses **Jest** for unit and integration testing.
+
+### Running Tests
+To run the automated test suite:
+```bash
+npm test
+```
+
+### Writing Tests
+- Place unit tests alongside source files (e.g., `src/utils.test.ts`).
+- Tests should use `import` syntax (ESM).
+- Existing manual testing scripts (e.g. `*.manual-test.ts`) are excluded from the automated runner.
+
+## 📚 Documentation
+
+Detailed documentation is available in the `docs/` folder:
+
+-   **[Setup & Installation](docs/setup.md)**: Configuration including `src/config.json` and GCP.
+-   **[Architecture](docs/architecture.md)**: System design, services, and resiliency.
+-   **[API Reference](docs/api.md)**: Endpoints for workflows, config files, media.
+-   **[Browser Automation](docs/browser_automation.md)**: `robotjs` integration and extension details.
+-   **[Email Automation](docs/email_automation.md)**: Gmail polling and filtering.
+-   **[Workflows](docs/workflows.md)**: How to build and run workflows.
+
+## 🌟 Key Features
+
+-   **Agentic UI Planning**: Deeply integrated AI that maps web pages and uses `robotjs` to simulate human clicks and typing.
+-   **Dynamic Workflows**: Can materialize file structures from JSON and execute them (`build-workflow.js`).
+-   **Production Resilience**: Global error handlers prevent server crashes (`server.js`).
+-   **Media Pipeline**: Native support for processing Audio, Images, and PDFs.
+
+## 📂 Project Structure
+
+-   `src/server.js`: Main entry point with error handling.
+-   `src/operations.js`: Low-level OS automation (mouse/keyboard).
+-   `src/configEndpoints.js`: API for managing file-based configs.
+-   `src/services/`: Business logic and AI services.
+-   `src/extension/`: Chrome extension for browser integration.
+-   `src/gmail-poller/`: Email automation tools.
+
+## License
+
+Private / Proprietary.

package/dist/Dockerfile

@@ -0,0 +1,86 @@
+###############################
+# Base image with system libs #
+###############################
+FROM node:20-slim AS base
+
+ENV DEBIAN_FRONTEND=noninteractive \
+    PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true \
+    PYTHONUNBUFFERED=1
+
+ARG WITH_ROBOTJS=0
+# System packages for native deps (canvas, sharp, tesseract, etc.)
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    python3 \
+    python3-pip \
+    python3-venv \
+    git \
+    pkg-config \
+    libpixman-1-dev libcairo2-dev libpango1.0-dev libjpeg62-turbo-dev libgif-dev librsvg2-dev \
+    libvips42 \
+    tesseract-ocr \
+    libgl1 libglib2.0-0 libsm6 libxrender1 libxext6 \
+    ffmpeg \
+    ${WITH_ROBOTJS:+libxtst-dev libx11-dev} \
+    && ln -s /usr/bin/python3 /usr/bin/python \
+    && rm -rf /var/lib/apt/lists/*
+
+    # Torch channel can be: cpu | cu121 | cu124 | rocm6.2 (etc.)
+ARG TORCH_CHANNEL=cu126
+
+RUN python3 -m pip install --upgrade pip --break-system-packages && \
+    if [ "$TORCH_CHANNEL" = "cpu" ]; then \
+      python3 -m pip install --no-cache-dir --break-system-packages \
+        torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu ; \
+    else \
+      python3 -m pip install --no-cache-dir --break-system-packages \
+        torch torchvision torchaudio --index-url https://download.pytorch.org/whl/${TORCH_CHANNEL} ; \
+    fi && \
+    python3 -m pip install --no-cache-dir --break-system-packages ultralytics
+
+# Copy and install trainer requirements
+COPY trainer/requirements.txt /app/trainer/requirements.txt
+RUN python3 -m pip install --no-cache-dir --break-system-packages -r /app/trainer/requirements.txt
+
+WORKDIR /app
+
+###############################
+# Dev dependencies layer      #
+###############################
+FROM base AS deps-dev
+COPY package*.json ./
+RUN npm install
+
+###############################
+# Production deps layer       #
+###############################
+FROM base AS deps-prod
+COPY package*.json ./
+RUN npm install --omit=dev && npm cache clean --force
+
+###############################
+# Development image           #
+###############################
+FROM base AS dev
+ENV NODE_ENV=development PORT=8080
+COPY --from=deps-dev /app/node_modules ./node_modules
+COPY . .
+EXPOSE 8080
+# Use nodemon for autoreload (mounted volume will overlay source code)
+CMD ["npm", "run", "dev"]
+
+###############################
+# Production build image      #
+###############################
+FROM base AS prod
+ENV NODE_ENV=production PORT=8080 NODE_OPTIONS="--max-old-space-size=4096"
+COPY --from=deps-prod /app/node_modules ./node_modules
+COPY . .
+RUN chmod +x start.sh
+EXPOSE 8080
+HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
+    CMD node -e 'fetch(`http://localhost:${process.env.PORT||8080}/health`).then(r=>{if(!r.ok)process.exit(1)}).catch(()=>process.exit(1))' 2>/dev/null || exit 1
+CMD ["./start.sh"]
+
+# Default stage (can be overridden):
+FROM prod AS default

package/dist/GPU_DEPLOYMENT_README.md

@@ -0,0 +1,324 @@
+# GPU-Enabled VM Deployment Documentation
+
+## Overview
+
+This documentation covers the deployment of a GPU-accelerated virtual machine on Google Cloud Platform with NVIDIA A100 GPU support for machine learning workloads, specifically optimized for YOLO training and computer vision tasks.
+
+## Architecture
+
+### Components
+- **VM Instance**: `a2-highgpu-1g` (A100 GPU, 12 vCPUs, 85GB RAM)
+- **OS**: Ubuntu 22.04 LTS
+- **GPU**: NVIDIA A100-SXM4-40GB
+- **Container Runtime**: Docker with NVIDIA Container Toolkit
+- **Application**: midjourney-worker container with GPU acceleration
+
+### Network Configuration
+- **Zone**: us-central1-f
+- **Project**: flowframe-auto
+- **Container Registry**: us-central1-docker.pkg.dev/flowframe-auto/worker-images/midjourney-worker:latest
+
+## Prerequisites
+
+### Google Cloud Setup
+1. **Project**: Ensure `flowframe-auto` project exists
+2. **Billing**: GPU-enabled VMs require billing to be enabled
+3. **APIs**: Enable Compute Engine API and Container Registry API
+4. **Service Account**: Service account with appropriate permissions
+5. **Authentication**: `gcloud auth login` and `gcloud config set project flowframe-auto`
+
+### Local Environment
+- **gcloud CLI**: Latest version installed
+- **Docker**: For local testing (optional)
+- **Node.js**: For running npm scripts
+- **Git**: For repository access
+
+## Quick Start
+
+### Deploy GPU VM
+```bash
+npm run deploy-gpu-vm
+```
+
+### Check Status
+```bash
+npm run gpu-vm-status
+```
+
+### SSH Access
+```bash
+npm run gpu-vm-ssh
+```
+
+## Available Commands
+
+### Deployment Commands
+```bash
+# Deploy the GPU VM
+npm run deploy-gpu-vm
+
+# Build and deploy (combined)
+npm run build-deploy
+
+# Check VM status
+npm run gpu-vm-status
+```
+
+### Management Commands
+```bash
+# SSH into the VM
+npm run gpu-vm-ssh
+
+# View startup logs
+npm run gpu-vm-logs
+
+# Stop the VM
+npm run gpu-vm-stop
+
+# Start the VM
+npm run gpu-vm-start
+
+# Delete the VM
+npm run gpu-vm-delete
+```
+
+## Detailed Deployment Process
+
+### 1. VM Creation
+The deployment script creates an Ubuntu 22.04 VM with:
+- A100 GPU attached
+- 400GB boot disk
+- NVIDIA GPU Cloud image
+- Startup script for GPU driver installation
+
+### 2. GPU Driver Installation
+The startup script automatically:
+- Updates the system
+- Installs NVIDIA drivers (version 550)
+- Configures Docker with NVIDIA Container Toolkit
+- Sets up GPU runtime for containers
+
+### 3. Container Deployment
+Once the VM is ready:
+- Docker daemon is configured for GPU access
+- midjourney-worker container is pulled and started
+- GPU acceleration is enabled via `--gpus all` flag
+
+### 4. Service Configuration
+The container runs with:
+- Port 8080 exposed
+- Automatic restart on failure
+- GPU memory access
+- CUDA runtime support
+
+## Performance Specifications
+
+### Hardware
+- **GPU**: NVIDIA A100-SXM4-40GB
+- **GPU Memory**: 40GB HBM2e
+- **CUDA Cores**: 6,912
+- **Tensor Cores**: 432
+- **Memory Bandwidth**: 1,555 GB/s
+- **CPU**: 12 vCPUs (AMD EPYC)
+- **RAM**: 85GB
+
+### Expected Performance
+- **YOLO Training**: ~10-50x faster than CPU
+- **Image Processing**: ~20-100x faster than CPU
+- **Memory Usage**: 0-40GB depending on model size
+- **Power Consumption**: 45-400W depending on load
+
+## Monitoring and Troubleshooting
+
+### Check GPU Status
+```bash
+# SSH into VM and run
+nvidia-smi
+```
+
+### View Container Logs
+```bash
+# From local machine
+npm run gpu-vm-ssh
+# Then inside VM
+sudo docker logs midjourney-worker-gpu
+```
+
+### Common Issues
+
+#### 1. Authentication Errors
+```
+Error: denied: Unauthenticated request
+```
+**Solution**: Ensure service account has Artifact Registry permissions
+
+#### 2. GPU Not Detected
+```
+nvidia-smi: command not found
+```
+**Solution**: Wait for startup script completion (5-10 minutes)
+
+#### 3. Container Won't Start
+```
+docker: Error response from daemon: could not select device driver
+```
+**Solution**: Restart Docker daemon after NVIDIA driver installation
+
+#### 4. Out of Memory
+```
+CUDA out of memory
+```
+**Solution**: Reduce batch size or use mixed precision training
+
+### Startup Script Logs
+```bash
+# View startup script progress
+npm run gpu-vm-logs
+```
+
+## Cost Optimization
+
+### VM Pricing (us-central1-f)
+- **A100 GPU**: $2.93/hour
+- **vCPUs**: $0.034/hour × 12 = $0.408/hour
+- **RAM**: $0.0048/hour × 85GB = $0.408/hour
+- **Disk**: $0.040/hour × 400GB = $0.016/hour
+- **Total**: ~$3.75/hour
+
+### Usage Tips
+- Stop VM when not in use: `npm run gpu-vm-stop`
+- Use preemptible instances for cost savings (if acceptable)
+- Monitor usage with `nvidia-smi` to optimize workloads
+
+## Security Considerations
+
+### Network Security
+- VM is accessible via SSH with gcloud authentication
+- Container registry access requires service account authentication
+- No public IP by default (access via gcloud proxy)
+
+### Service Account Permissions
+Required permissions:
+- `compute.instances.*`
+- `artifactregistry.repositories.downloadArtifacts`
+- `storage.objects.get` (for data access)
+
+## Advanced Configuration
+
+### Custom Startup Script
+Modify `gpu-startup.sh` for additional setup:
+```bash
+# Add custom packages
+apt install -y python3-pip
+pip install torch torchvision torchaudio
+
+# Mount additional disks
+# Configure monitoring
+# Set up logging
+```
+
+### Environment Variables
+```bash
+# Inside container
+export CUDA_VISIBLE_DEVICES=0
+export TORCH_USE_CUDA_DSA=1
+```
+
+### GPU Memory Management
+```python
+# In Python code
+torch.cuda.set_per_process_memory_fraction(0.8)  # Use 80% of GPU memory
+torch.cuda.empty_cache()  # Clear cache when needed
+```
+
+## Integration with Existing Workflows
+
+### YOLO Training Integration
+The GPU VM is optimized for YOLO model training:
+- Automatic GPU detection
+- Mixed precision support
+- Batch size optimization
+- Multi-GPU support (if scaled)
+
+### API Access
+Once deployed, access the service at:
+```
+http://[VM_EXTERNAL_IP]:8080
+```
+
+### Data Pipeline
+- Upload training data to Google Cloud Storage
+- Mount GCS buckets in container
+- Process data with GPU acceleration
+
+## Support and Maintenance
+
+### Regular Maintenance
+```bash
+# Update NVIDIA drivers
+sudo ubuntu-drivers autoinstall
+
+# Update Docker
+sudo apt update && sudo apt upgrade docker-ce
+
+# Restart services
+sudo systemctl restart docker
+```
+
+### Backup Strategy
+- Container images stored in Artifact Registry
+- Training data in Cloud Storage
+- VM disks can be snapshotted
+
+### Monitoring
+- Use `nvidia-smi` for GPU monitoring
+- Docker stats for container performance
+- Cloud Monitoring for VM metrics
+
+## Troubleshooting Checklist
+
+1. **VM won't start**: Check quota limits and billing
+2. **GPU not available**: Wait for startup script completion
+3. **Container fails**: Check authentication and image availability
+4. **Performance issues**: Monitor GPU utilization and memory usage
+5. **Network issues**: Verify firewall rules and service account permissions
+
+## Future Enhancements
+
+### Planned Features
+- Auto-scaling based on workload
+- Multi-GPU support
+- Cost optimization with spot instances
+- Integration with Vertex AI
+- Automated backup and recovery
+
+### Performance Optimizations
+- TensorRT integration
+- Mixed precision training
+- GPU memory optimization
+- Distributed training support
+
+---
+
+## Quick Reference
+
+### Deploy Everything
+```bash
+npm run deploy-gpu-vm
+```
+
+### Check Everything is Working
+```bash
+npm run gpu-vm-status
+npm run gpu-vm-ssh
+nvidia-smi
+sudo docker ps
+```
+
+### Clean Up
+```bash
+npm run gpu-vm-stop  # When done
+npm run gpu-vm-delete  # When finished
+```
+
+For additional support, check the startup script logs or contact the development team.

package/dist/OPS_AGENT_README.md

@@ -0,0 +1,174 @@
+# Google Cloud Ops Agent Integration
+
+## Overview
+The Google Cloud Ops Agent has been integrated into your startup script (`start.sh`) to provide comprehensive monitoring and logging for your VM.
+
+## What It Monitors
+
+### System Metrics
+- **CPU utilization** (per core and overall)
+- **Memory usage** (RAM, swap)
+- **Disk I/O** (read/write operations, latency)
+- **Network traffic** (bytes in/out, packets)
+- **GPU metrics** (if GPU present - utilization, memory, temperature)
+
+### System Logs
+- **Syslog** (`/var/log/syslog`)
+- **Authentication logs** (`/var/log/auth.log`)
+- **Kernel messages** (`/var/log/kern.log`)
+- **Application logs** (configurable)
+
+### Process Metrics
+- **Process count and status**
+- **Resource usage per process**
+- **Custom application metrics** (can be added)
+
+## Integration Details
+
+### Automatic Installation Process
+The Ops Agent installation in `start.sh` follows this robust logic:
+
+1. **Check Status**: Verify if Ops Agent is already installed and running
+2. **Start if Stopped**: If installed but not running, attempt to restart it
+3. **Install if Missing**: If not installed, download and install the latest version
+4. **Verify Installation**: Confirm the agent is working after installation
+5. **Continue Always**: Your application starts regardless of Ops Agent status
+
+### Benefits
+- ✅ **Always Available**: Installs automatically every time your VM starts
+- ✅ **Self-Healing**: Restarts the agent if it's stopped
+- ✅ **Non-Blocking**: Your app continues even if monitoring fails
+- ✅ **Latest Version**: Always installs the most recent Ops Agent
+- ✅ **Cross-Platform**: Works on Ubuntu/Debian and CentOS/RHEL
+
+### Error Handling
+- Installation failures don't stop your application startup
+- Graceful fallback if Ops Agent is unavailable
+- Detailed logging of installation process
+- Automatic retry mechanisms
+
+## Verification
+
+### Check Ops Agent Status
+Run the verification script:
+```bash
+./check-ops-agent.sh
+```
+
+This will show:
+- Service status
+- Configuration files
+- Recent log entries
+- What metrics are being collected
+
+### Cloud Monitoring Console
+View metrics in Google Cloud Console:
+1. Go to **Cloud Monitoring** > **Metrics Explorer**
+2. Filter by your VM instance
+3. Look for metrics like:
+   - `compute.googleapis.com/instance/cpu/utilization`
+   - `compute.googleapis.com/instance/memory/balloon/ram_used`
+   - `compute.googleapis.com/instance/disk/read_bytes_count`
+
+## Manual Installation (If Needed)
+
+If you need to manually install or reinstall the Ops Agent:
+
+```bash
+# Run the manual installer (recommended)
+./install-ops-agent.sh
+
+# Or use the direct Google method
+curl -sSO https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh
+sudo bash add-google-cloud-ops-agent-repo.sh --also-install
+```
+
+The manual installer script (`install-ops-agent.sh`) provides:
+- Interactive prompts and status checks
+- Automatic OS detection
+- Service restart capabilities
+- Comprehensive verification
+- Better error handling than the direct method
+
+## Custom Configuration
+
+### Adding Application Logs
+Edit `/etc/google-cloud-ops-agent/config.yaml` to add custom log collection:
+```yaml
+logging:
+  receivers:
+    my-app:
+      type: files
+      include_paths:
+        - /var/log/app/*.log
+      record_log_file_path: true
+  service:
+    pipelines:
+      my-app-logs:
+        receivers: [my-app]
+```
+
+### Custom Metrics
+Add Prometheus-style metrics collection:
+```yaml
+metrics:
+  receivers:
+    prometheus:
+      type: prometheus
+      config:
+        scrape_configs:
+          - job_name: 'my-app'
+            static_configs:
+              - targets: ['localhost:8080']
+  service:
+    pipelines:
+      prometheus:
+        receivers: [prometheus]
+```
+
+## Troubleshooting
+
+### Ops Agent Not Starting
+```bash
+# Check service status
+sudo systemctl status google-cloud-ops-agent
+
+# View logs
+sudo journalctl -u google-cloud-ops-agent -f
+
+# Restart service
+sudo systemctl restart google-cloud-ops-agent
+```
+
+### Metrics Not Appearing
+- Wait 5-10 minutes after installation
+- Check VM has proper IAM permissions for Monitoring
+- Verify Ops Agent service is running
+- Check `/var/log/google-cloud-ops-agent/` for errors
+
+### High Resource Usage
+The Ops Agent is lightweight but you can adjust collection intervals in the config file if needed.
+
+## Files Overview
+
+- **`start.sh`**: Main startup script with automatic Ops Agent installation
+- **`check-ops-agent.sh`**: Comprehensive verification script
+- **`install-ops-agent.sh`**: Manual installation/reinstallation script
+- **`/etc/google-cloud-ops-agent/config.yaml`**: Ops Agent configuration file
+- **`/var/log/google-cloud-ops-agent/`**: Ops Agent log files
+
+## Benefits
+
+✅ **Automatic monitoring** - No manual setup required
+✅ **Comprehensive coverage** - System, GPU, and application metrics
+✅ **Cloud integration** - Direct integration with Cloud Monitoring
+✅ **Alerting ready** - Set up alerts on metrics thresholds
+✅ **Historical data** - Metrics retained for analysis
+✅ **Multi-environment** - Works across dev, staging, and production
+
+## Next Steps
+
+1. **Set up alerts** in Cloud Monitoring for critical metrics
+2. **Create dashboards** for your key performance indicators
+3. **Configure log-based metrics** for application-specific monitoring
+4. **Set up uptime checks** for your application endpoints