gitarsenal-cli 1.5.2 → 1.5.4

package/README.md

@@ -1,74 +1,119 @@
 # GitArsenal CLI
 
-A tool for creating and managing GPU-accelerated development environments using Modal.
+**Run any GitHub repository instantly with pre-configured GPU environments.**
 
-## Features
+GitArsenal CLI makes it incredibly easy to run any GitHub repository without worrying about setup, dependencies, or environment configuration. Just point it at a repository and start coding with GPU acceleration.
 
-- Create Modal containers with GPU support
-- Clone repositories and run setup commands
-- Persistent storage with Modal volumes
-- SSH access to containers
-- API key management for various services
+## Why GitArsenal CLI?
 
-## API Key Management
+- **Zero Setup**: No need to install dependencies, configure environments, or manage GPU drivers
+- **GPU Ready**: Every environment comes with GPU acceleration (A10G, A100, H100)
+- **Persistent Storage**: Your work and data persist between sessions
+- **SSH Access**: Connect directly to your running environment
+- **API Key Management**: Securely store and auto-inject API keys for services like OpenAI, Weights & Biases, and Hugging Face
 
-The CLI now supports secure storage of API keys for various services. Keys are stored in `~/.gitarsenal/keys/` with proper permissions (only readable by the current user).
+## Quick Start
 
-### Supported Services
+### Run any GitHub repository
 
-- `openai` - OpenAI API keys for debugging and assistance
-- `wandb` - Weights & Biases API keys for experiment tracking
-- `huggingface` - Hugging Face tokens for model access
+```bash
+# Basic usage - clone and run any repo
+gitarsenal container --repo-url https://github.com/username/awesome-project.git
+
+# With GPU acceleration
+gitarsenal container --gpu A10G --repo-url https://github.com/username/awesome-project.git
 
-### Managing API Keys
+# With custom setup commands
+gitarsenal container --gpu A100 --repo-url https://github.com/username/awesome-project.git --setup-commands "pip install -r requirements.txt" "python setup.py install"
+```
 
-#### Adding a new API key
+### Examples
 
 ```bash
-# Add a key interactively (will prompt for the key)
-python test_modalSandboxScript.py keys add --service openai
+# Run a machine learning project
+gitarsenal container --gpu A100 --repo-url https://github.com/username/transformer-project.git --setup-commands "pip install torch transformers" "wandb login"
 
-# Add a key directly (not recommended for security)
-python test_modalSandboxScript.py keys add --service wandb --key YOUR_API_KEY
+# Run a web development project
+gitarsenal container --repo-url https://github.com/username/react-app.git --setup-commands "npm install" "npm start"
+
+# Run a data science project with persistent storage
+gitarsenal container --gpu A10G --repo-url https://github.com/username/data-analysis.git --volume-name my-data --setup-commands "pip install pandas numpy matplotlib"
 ```
 
-#### Listing saved API keys
+## API Key Management
+
+Store your API keys once and use them across all projects:
 
 ```bash
-python test_modalSandboxScript.py keys list
-```
+# Add API keys for seamless integration
+gitarsenal keys add --service openai
+gitarsenal keys add --service wandb
+gitarsenal keys add --service huggingface
 
-#### Viewing a specific API key (masked)
+# View your saved keys
+gitarsenal keys list
 
-```bash
-python test_modalSandboxScript.py keys view --service huggingface
+# Remove a key
+gitarsenal keys delete --service openai
 ```
 
-#### Deleting an API key
+### Supported Services
+
+- **OpenAI** - For debugging and AI assistance
+- **Weights & Biases** - For experiment tracking
+- **Hugging Face** - For model access and downloads
+
+## Features
+
+### Automatic Environment Setup
+The CLI automatically:
+- Clones your repository
+- Installs dependencies based on your setup commands
+- Configures GPU acceleration
+- Sets up persistent storage
+- Injects your saved API keys
+
+### Persistent Storage
+Keep your work safe with persistent volumes:
 
 ```bash
-python test_modalSandboxScript.py keys delete --service openai
+# Create a persistent environment
+gitarsenal container --repo-url https://github.com/username/project.git --volume-name my-work
+
+# Your data, models, and work will persist between sessions
 ```
 
-## Creating a Modal Container
+### SSH Access
+Connect directly to your running environment:
 
 ```bash
-# Basic container creation
-python test_modalSandboxScript.py container --gpu A10G --repo-url https://github.com/username/repo.git
-
-# With setup commands
-python test_modalSandboxScript.py container --gpu A100 --repo-url https://github.com/username/repo.git --setup-commands "pip install -r requirements.txt" "python setup.py install"
+# Get SSH connection details
+gitarsenal container --repo-url https://github.com/username/project.git --ssh
 
-# With volume for persistent storage
-python test_modalSandboxScript.py container --gpu A10G --repo-url https://github.com/username/repo.git --volume-name my-persistent-volume
+# Connect via SSH to your environment
+ssh user@your-environment-ip
 ```
 
-## Automatic API Key Usage
+## Workflow
+
+1. **Choose a repository** - Any GitHub repo you want to work with
+2. **Run the command** - Specify GPU, setup commands, and storage
+3. **Start coding** - Your environment is ready with all dependencies installed
+4. **Save your work** - Use persistent volumes to keep your progress
+
+## Perfect For
+
+- **Machine Learning Projects** - GPU-accelerated training with pre-configured environments
+- **Data Science** - Jupyter notebooks with all dependencies ready
+- **Web Development** - Full-stack projects with development servers
+- **Research** - Reproducible environments for academic work
+- **Hackathons** - Quick setup for rapid prototyping
 
-When using commands that require API keys (like `wandb login` or `huggingface-cli login`), the system will:
+## Getting Started
 
-1. Check if a saved API key exists for the service
-2. If found, use the saved key automatically
-3. If not found, prompt for the key and offer to save it for future use
+1. Install the CLI (see installation instructions)
+2. Add your API keys: `gitarsenal keys add --service openai`
+3. Run any repository: `gitarsenal container --repo-url https://github.com/username/project.git`
+4. Start coding!
 
-This makes it easy to work with multiple projects that require the same API keys without having to re-enter them each time. 
+No more "works on my machine" - every environment is identical and ready to go. 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "gitarsenal-cli",
-  "version": "1.5.2",
+  "version": "1.5.4",
   "description": "CLI tool for creating Modal sandboxes with GitHub repositories",
   "main": "index.js",
   "bin": {

package/python/credentials_manager.py

@@ -119,6 +119,24 @@ class CredentialsManager:
         def validate_openai_key(key):
             # Basic validation - OpenAI keys usually start with "sk-" and are 51 chars
             return key.startswith("sk-") and len(key) > 40
+        
+        # First check environment variable
+        env_key = os.environ.get("OPENAI_API_KEY")
+        if env_key and validate_openai_key(env_key):
+            return env_key
+            
+        # Then try to fetch from server using fetch_modal_tokens if available
+        try:
+            from fetch_modal_tokens import get_tokens
+            _, _, api_key = get_tokens()
+            if api_key and validate_openai_key(api_key):
+                # Set in environment for future use
+                os.environ["OPENAI_API_KEY"] = api_key
+                return api_key
+        except ImportError:
+            pass
+        except Exception as e:
+            print(f"⚠️ Error fetching API key from server: {e}")
             
         prompt = "To debug failed commands, an OpenAI API key is needed.\nYou can get your API key from: https://platform.openai.com/api-keys"
         return self.get_credential("openai_api_key", prompt, is_password=True, validate_func=validate_openai_key)

package/python/fetch_modal_tokens.py

@@ -41,7 +41,7 @@ def fetch_default_tokens_from_gitarsenal():
                 openai_api_key = data.get("openaiApiKey")
                 
                 if token_id and token_secret:
-                    print("✅ Successfully fetched default tokens from gitarsenal.dev")
+                    # print("✅ Successfully fetched default tokens from gitarsenal.dev")
                     return token_id, token_secret, openai_api_key
                 else:
                     print("❌ Modal tokens not found in gitarsenal.dev response")
@@ -153,6 +153,13 @@ def get_tokens():
         print("💡 Please check your network connection and API endpoints")
         return None, None, None
     
+    # Debug print the full token values
+    # print("\n🔍 DEBUG: FULL TOKEN VALUES:")
+    # print(f"🔍 DEBUG: MODAL_TOKEN_ID: {token_id}")
+    # print(f"🔍 DEBUG: MODAL_TOKEN_SECRET: {token_secret}")
+    # print(f"🔍 DEBUG: OPENAI_API_KEY: {openai_api_key}")
+    # print("🔍 DEBUG: END OF TOKEN VALUES\n")
+    
     # Set the tokens in environment variables
     os.environ["MODAL_TOKEN_ID"] = token_id
     os.environ["MODAL_TOKEN_SECRET"] = token_secret

package/python/fix_modal_token.py

@@ -51,8 +51,6 @@ try:
     print(f"✅ Using tokens from proxy server or defaults")
 except (ImportError, ValueError) as e:
     # If the module is not available or tokens are invalid, use hardcoded tokens
-    TOKEN_ID = "ak-sLhYqCjkvixiYcb9LAuCHp"
-    TOKEN_SECRET = "as-fPzD0Zm0dl6IFAEkhaH9pq"  # Real token secret from fr8mafia profile
     print(f"⚠️ Using default tokens")
 
 print("🔧 Fixing Modal token (basic implementation)...")

package/python/fix_modal_token_advanced.py

@@ -27,9 +27,7 @@ try:
     # print(f"✅ Using tokens from proxy server or defaults")
 except ImportError:
     # If the module is not available, use hardcoded tokens
-    # print(f"⚠️ Using default tokens")
-    TOKEN_ID = "ak-sLhYqCjkvixiYcb9LAuCHp"
-    TOKEN_SECRET = "as-fPzD0Zm0dl6IFAEkhaH9pq"
+    print(f"⚠️ Using default tokens")
 
 # print("🔧 Advanced Modal Token Fixer")
 

package/python/modal_auth_patch.py

@@ -26,8 +26,6 @@ try:
     print(f"✅ Using tokens from proxy server or defaults")
 except ImportError:
     # If the module is not available, use hardcoded tokens
-    # TOKEN_ID = "ak-sLhYqCjkvixiYcb9LAuCHp"
-    # TOKEN_SECRET = "as-fPzD0Zm0dl6IFAEkhaH9pq"  # Real token secret from fr8mafia profile
     print(f"⚠️ Using default tokens")
 
 # Set environment variables

package/python/modal_token_solution.py

@@ -29,8 +29,6 @@ try:
     print(f"✅ Using tokens from proxy server or defaults")
 except ImportError:
     # If the module is not available, use hardcoded tokens
-    # TOKEN_ID = "ak-sLhYqCjkvixiYcb9LAuCHp"
-    # TOKEN_SECRET = "as-fPzD0Zm0dl6IFAEkhaH9pq"  # Real token secret from fr8mafia profile
     print(f"⚠️ Using default tokens")
 
 print("🔧 Modal Token Solution - Comprehensive Fix")

package/python/test_modalSandboxScript.py

@@ -36,15 +36,21 @@ if args.proxy_api_key:
 # First, try to fetch tokens from the proxy server
 try:
     # Import the fetch_modal_tokens module
-    # print("🔄 Fetching tokens from proxy server...")
+    print("🔄 Fetching tokens from proxy server...")
     from fetch_modal_tokens import get_tokens
     token_id, token_secret, openai_api_key = get_tokens()
     
+    # Debug print the tokens
+    # print("🔍 DEBUG: Modal Tokens Fetched:")
+    # print(f"🔍 DEBUG: Token ID: {token_id}")
+    # print(f"🔍 DEBUG: Token Secret: {token_secret}")
+    # print(f"🔍 DEBUG: OpenAI API Key: {openai_api_key}")
+    
     # Check if we got valid tokens
     if token_id is None or token_secret is None:
         raise ValueError("Could not get valid tokens")
     
-    # print(f"✅ Tokens fetched successfully")
+    print(f"✅ Tokens fetched successfully")
     
     # Explicitly set the environment variables again to be sure
     os.environ["MODAL_TOKEN_ID"] = token_id
@@ -336,6 +342,11 @@ def call_openai_for_debug(command, error_output, api_key=None, current_dir=None,
     print(f"🔍 DEBUG: Error output length: {len(error_output) if error_output else 0}")
     print(f"🔍 DEBUG: Current directory: {current_dir}")
     print(f"🔍 DEBUG: Sandbox available: {sandbox is not None}")
+    print(f"🔍 DEBUG: API key provided: {'Yes' if api_key else 'No'}")
+    if api_key:
+        print(f"🔍 DEBUG: API key value: {api_key}")
+    else:
+        print(f"🔍 DEBUG: API key from environment: {os.environ.get('OPENAI_API_KEY')}")
     
     # Define _to_str function locally to avoid NameError
     def _to_str(maybe_bytes):
@@ -370,6 +381,24 @@ def call_openai_for_debug(command, error_output, api_key=None, current_dir=None,
         # First try environment variable
         api_key = os.environ.get("OPENAI_API_KEY")
         print(f"🔍 DEBUG: API key from environment: {'Found' if api_key else 'Not found'}")
+        if api_key:
+            print(f"🔍 DEBUG: Environment API key value: {api_key}")
+        
+        # If not in environment, try to fetch from server using fetch_modal_tokens
+        if not api_key:
+            try:
+                print("🔍 DEBUG: Trying to fetch API key from server...")
+                from fetch_modal_tokens import get_tokens
+                _, _, api_key = get_tokens()
+                if api_key:
+                    print("✅ Successfully fetched OpenAI API key from server")
+                    print(f"🔍 DEBUG: Fetched OpenAI API key value: {api_key}")
+                    # Set in environment for this session
+                    os.environ["OPENAI_API_KEY"] = api_key
+                else:
+                    print("⚠️ Could not fetch OpenAI API key from server")
+            except Exception as e:
+                print(f"⚠️ Error fetching API key from server: {e}")
         
         # Store the API key in a persistent file if found
         if api_key:
@@ -391,6 +420,7 @@ def call_openai_for_debug(command, error_output, api_key=None, current_dir=None,
                         api_key = f.read().strip()
                     if api_key:
                         print("✅ Loaded OpenAI API key from saved file")
+                        print(f"🔍 DEBUG: API key from file: {api_key}")
                         print(f"🔍 DEBUG: API key length: {len(api_key)}")
                         # Also set in environment for this session
                         os.environ["OPENAI_API_KEY"] = api_key
@@ -407,8 +437,14 @@ def call_openai_for_debug(command, error_output, api_key=None, current_dir=None,
             try:
                 from credentials_manager import CredentialsManager
                 credentials_manager = CredentialsManager()
-                api_key = os.environ.get("OPENAI_API_KEY")
-                print(f"🔍 DEBUG: API key from credentials manager: {'Found' if api_key else 'Not found'}")
+                api_key = credentials_manager.get_openai_api_key()
+                if api_key:
+                    print(f"🔍 DEBUG: API key from credentials manager: Found")
+                    print(f"🔍 DEBUG: Credentials manager API key value: {api_key}")
+                    # Set in environment for this session
+                    os.environ["OPENAI_API_KEY"] = api_key
+                else:
+                    print(f"🔍 DEBUG: API key from credentials manager: Not found")
             except ImportError as e:
                 print(f"🔍 DEBUG: Credentials manager not available: {e}")
                 # Fall back to direct input if credentials_manager is not available
@@ -431,6 +467,7 @@ def call_openai_for_debug(command, error_output, api_key=None, current_dir=None,
                     print("❌ No API key provided. Skipping debugging.")
                     return None
                 print("✅ API key received successfully!")
+                print(f"🔍 DEBUG: User-provided API key: {api_key}")
                 # Save the API key to environment for future use in this session
                 os.environ["OPENAI_API_KEY"] = api_key
             except KeyboardInterrupt:
@@ -1766,7 +1803,9 @@ cd "{current_dir}"
                 print(f"🔍 DEBUG: Sandbox available: {sandbox is not None}")
                 print(f"🔍 DEBUG: Debug output preview: {debug_output[:200]}...")
                 
-                fix_command = call_openai_for_debug(cmd_to_execute, debug_output, current_dir=current_dir, sandbox=sandbox)
+                # Get the API key from environment or use the one that was fetched earlier
+                api_key = os.environ.get("OPENAI_API_KEY")
+                fix_command = call_openai_for_debug(cmd_to_execute, debug_output, api_key=api_key, current_dir=current_dir, sandbox=sandbox)
                 
                 print(f"🔍 DEBUG: call_openai_for_debug returned: {fix_command}")
                 
@@ -2438,7 +2477,7 @@ ssh_app = modal.App("ssh-container-app")
     memory=8192,
     serialized=True,
 )
-def ssh_container_function(ssh_password, repo_url=None, repo_name=None, setup_commands=None):
+def ssh_container_function(ssh_password, repo_url=None, repo_name=None, setup_commands=None, openai_api_key=None):
     import subprocess
     import time
     import os
@@ -2452,6 +2491,13 @@ def ssh_container_function(ssh_password, repo_url=None, repo_name=None, setup_co
     # Setup environment
     os.environ['PS1'] = r'\[\e[1;32m\]modal:\[\e[1;34m\]\w\[\e[0m\]$ '
     
+    # Set OpenAI API key if provided
+    if openai_api_key:
+        os.environ['OPENAI_API_KEY'] = openai_api_key
+        print(f"✅ Set OpenAI API key in container environment (length: {len(openai_api_key)})")
+    else:
+        print("⚠️ No OpenAI API key provided to container")
+    
     # Clone repository if provided
     if repo_url:
         repo_name_from_url = repo_name or repo_url.split('/')[-1].replace('.git', '')
@@ -2523,7 +2569,9 @@ def ssh_container_function(ssh_password, repo_url=None, repo_name=None, setup_co
                     print(f"🔍 DEBUG: Error output length: {len(error_output)}")
                     print(f"🔍 DEBUG: Current directory: {current_dir}")
                     
-                    fix_command = call_openai_for_debug(cmd, error_output, current_dir=current_dir)
+                    # Get the API key from environment or use the one that was fetched earlier
+                    api_key = os.environ.get("OPENAI_API_KEY")
+                    fix_command = call_openai_for_debug(cmd, error_output, api_key=api_key, current_dir=current_dir)
                     
                     print(f"🔍 DEBUG: call_openai_for_debug returned: {fix_command}")
                     
@@ -2641,12 +2689,16 @@ def create_modal_ssh_container(gpu_type, repo_url=None, repo_name=None, setup_co
         print("🔍 DEBUG: Checking environment variables")
         modal_token_id = os.environ.get("MODAL_TOKEN_ID")
         modal_token = os.environ.get("MODAL_TOKEN")
+        openai_api_key = os.environ.get("OPENAI_API_KEY")
         print(f"🔍 token exists: {'Yes' if modal_token_id else 'No'}")
         print(f"🔍 token exists: {'Yes' if modal_token else 'No'}")
+        print(f"🔍 openai_api_key exists: {'Yes' if openai_api_key else 'No'}")
         if modal_token_id:
             print(f"🔍 token length: {len(modal_token_id)}")
         if modal_token:
             print(f"🔍 token length: {len(modal_token)}")
+        if openai_api_key:
+            print(f"🔍 openai_api_key length: {len(openai_api_key)}")
         
         # Try to access Modal token to check authentication
         try:
@@ -2860,7 +2912,7 @@ def create_modal_ssh_container(gpu_type, repo_url=None, repo_name=None, setup_co
         serialized=True,
         volumes=volumes_config if volumes_config else None,
     )
-    def ssh_container_function():
+    def ssh_container_function(ssh_password=None, repo_url=None, repo_name=None, setup_commands=None, openai_api_key=None):
         """Start SSH container with password authentication and optional setup."""
         import subprocess
         import time
@@ -2869,6 +2921,13 @@ def create_modal_ssh_container(gpu_type, repo_url=None, repo_name=None, setup_co
         # Set root password
         subprocess.run(["bash", "-c", f"echo 'root:{ssh_password}' | chpasswd"], check=True)
         
+        # Set OpenAI API key if provided
+        if openai_api_key:
+            os.environ['OPENAI_API_KEY'] = openai_api_key
+            print(f"✅ Set OpenAI API key in container environment (length: {len(openai_api_key)})")
+        else:
+            print("⚠️ No OpenAI API key provided to container")
+        
         # Start SSH service
         subprocess.run(["service", "ssh", "start"], check=True)
         
@@ -2934,7 +2993,9 @@ def create_modal_ssh_container(gpu_type, repo_url=None, repo_name=None, setup_co
                         print(f"🔍 DEBUG: Error output length: {len(error_output)}")
                         print(f"🔍 DEBUG: Current directory: {current_dir}")
                         
-                        fix_command = call_openai_for_debug(cmd, error_output, current_dir=current_dir)
+                        # Get the API key from environment or use the one that was fetched earlier
+                        api_key = os.environ.get("OPENAI_API_KEY")
+                        fix_command = call_openai_for_debug(cmd, error_output, api_key=api_key, current_dir=current_dir)
                         
                         print(f"🔍 DEBUG: call_openai_for_debug returned: {fix_command}")
                         
@@ -3030,7 +3091,9 @@ def create_modal_ssh_container(gpu_type, repo_url=None, repo_name=None, setup_co
         # Start the container in a new thread to avoid blocking
         with modal.enable_output():
             with app.run():
-                ssh_container_function.remote()
+                # Get the API key from environment
+                api_key = os.environ.get("OPENAI_API_KEY")
+                ssh_container_function.remote(ssh_password, repo_url, repo_name, setup_commands, api_key)
                 
         # Clean up Modal token after container is successfully created
         cleanup_modal_token()
@@ -3722,7 +3785,7 @@ def create_ssh_container_function(gpu_type="a10g", timeout_minutes=60, volume=No
         serialized=True,
         volumes=volumes if volumes else None,
     )
-    def ssh_container(ssh_password, repo_url=None, repo_name=None, setup_commands=None):
+    def ssh_container(ssh_password, repo_url=None, repo_name=None, setup_commands=None, openai_api_key=None):
         import subprocess
         import time
         import os
@@ -3730,6 +3793,13 @@ def create_ssh_container_function(gpu_type="a10g", timeout_minutes=60, volume=No
         # Set root password
         subprocess.run(["bash", "-c", f"echo 'root:{ssh_password}' | chpasswd"], check=True)
         
+        # Set OpenAI API key if provided
+        if openai_api_key:
+            os.environ['OPENAI_API_KEY'] = openai_api_key
+            print(f"✅ Set OpenAI API key in container environment (length: {len(openai_api_key)})")
+        else:
+            print("⚠️ No OpenAI API key provided to container")
+        
         # Start SSH service
         subprocess.run(["service", "ssh", "start"], check=True)
         

package/python/test_modal_auth.py

@@ -19,8 +19,6 @@ try:
     print(f"✅ Using tokens from proxy server or defaults")
 except ImportError:
     # If the module is not available, use hardcoded tokens
-    # TOKEN_ID = "ak-sLhYqCjkvixiYcb9LAuCHp"
-    # TOKEN_SECRET = "as-fPzD0Zm0dl6IFAEkhaH9pq"  # Real token secret from fr8mafia profile
     print(f"⚠️ Using hardcoded tokens")
 
 # Set tokens directly in environment