gitarsenal-cli 1.1.1 → 1.1.2

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "gitarsenal-cli",
-  "version": "1.1.1",
+  "version": "1.1.2",
   "description": "CLI tool for creating Modal sandboxes with GitHub repositories",
   "main": "index.js",
   "bin": {

package/python/README.md

@@ -13,50 +13,51 @@ cd gitarsenal-cli/python
 pip install -r requirements.txt
 ```
 
-## Credentials Setup
+## First-Time Setup
 
-GitArsenal CLI requires several API keys and tokens to function properly:
+Before using GitArsenal CLI, you need to set up your credentials and authenticate with Modal:
 
-- **OpenAI API Key**: Used for debugging failed commands
-- **Modal Token**: Used to create cloud environments
-- **Hugging Face Token**: Used for accessing Hugging Face models
-- **Weights & Biases API Key**: Used for experiment tracking
-
-You can set up your credentials using the credentials manager:
+### 1. Install Modal
 
 ```bash
-python manage_credentials.py setup
+pip install modal
 ```
 
-This will guide you through setting up all required credentials, which will be stored securely in your home directory.
+### 2. Create a Modal Account and Get a Token
 
-### Managing Individual Credentials
+If you don't have a Modal account:
+1. Go to https://modal.com and create an account
+2. Run the following command to authenticate:
+   ```bash
+   modal token new
+   ```
+3. Follow the instructions to complete authentication
 
-You can also manage individual credentials:
+### 3. Set Up GitArsenal Credentials
 
 ```bash
-# Set a specific credential
-python manage_credentials.py set openai_api_key
+# Set up all required credentials
+./gitarsenal.py credentials setup
+```
 
-# View a credential (masked for security)
-python manage_credentials.py get modal_token
+This will guide you through setting up:
+- **Modal Token**: Required for creating cloud environments
+- **OpenAI API Key**: Used for debugging failed commands (optional)
+- **Hugging Face Token**: Used for accessing Hugging Face models (optional)
+- **Weights & Biases API Key**: Used for experiment tracking (optional)
 
-# Clear a specific credential
-python manage_credentials.py clear huggingface_token
+## Usage
 
-# Clear all credentials
-python manage_credentials.py clear
+### Creating a Modal Sandbox
 
-# List all saved credentials (without showing values)
-python manage_credentials.py list
+```bash
+./gitarsenal.py sandbox --gpu A10G --repo-url "https://github.com/username/repo.git"
 ```
 
-## Usage
-
-### Creating a Modal Sandbox
+### Creating an SSH Container
 
 ```bash
-python test_modalSandboxScript.py --gpu A10G --repo-url "https://github.com/username/repo.git"
+./gitarsenal.py ssh --gpu A10G --repo-url "https://github.com/username/repo.git"
 ```
 
 ### Options
@@ -66,6 +67,31 @@ python test_modalSandboxScript.py --gpu A10G --repo-url "https://github.com/user
 - `--repo-name`: Repository name override
 - `--setup-commands`: Setup commands to run
 - `--volume-name`: Name of the Modal volume for persistent storage
+- `--timeout`: Container timeout in minutes (SSH mode only, default: 60)
+
+## Managing Credentials
+
+You can manage your credentials using the following commands:
+
+```bash
+# Set up all credentials
+./gitarsenal.py credentials setup
+
+# Set a specific credential
+./gitarsenal.py credentials set modal_token
+
+# View a credential (masked for security)
+./gitarsenal.py credentials get modal_token
+
+# Clear a specific credential
+./gitarsenal.py credentials clear huggingface_token
+
+# Clear all credentials
+./gitarsenal.py credentials clear
+
+# List all saved credentials (without showing values)
+./gitarsenal.py credentials list
+```
 
 ## Security
 
@@ -73,23 +99,40 @@ Your credentials are stored securely in `~/.gitarsenal/credentials.json` with re
 
 ## Troubleshooting
 
-If you encounter authentication issues:
+### Modal Authentication Issues
 
-1. Check that your credentials are set up correctly:
+If you see errors like "Token missing" or "Could not authenticate client":
+
+1. Ensure Modal is installed:
    ```bash
-   python manage_credentials.py list
+   pip install modal
    ```
 
-2. If needed, clear and reset your credentials:
+2. Get a new Modal token:
    ```bash
-   python manage_credentials.py clear
-   python manage_credentials.py setup
+   modal token new
    ```
 
-3. Ensure Modal CLI is installed and authenticated:
+3. Save the token in GitArsenal:
    ```bash
-   pip install modal
-   modal token new
+   ./gitarsenal.py credentials set modal_token
+   ```
+
+### API Timeout Issues
+
+If the GitArsenal API times out when analyzing repositories, the tool will automatically use fallback setup commands based on the detected programming language and technologies.
+
+### Other Issues
+
+1. Check that your credentials are set up correctly:
+   ```bash
+   ./gitarsenal.py credentials list
+   ```
+
+2. If needed, clear and reset your credentials:
+   ```bash
+   ./gitarsenal.py credentials clear
+   ./gitarsenal.py credentials setup
    ```
 
 ## License

package/python/gitarsenal.py

@@ -9,6 +9,55 @@ import argparse
 import sys
 import os
 import subprocess
+import json
+from pathlib import Path
+
+def check_modal_auth():
+    """Check if Modal is authenticated and guide user if not"""
+    try:
+        # Try to run a simple Modal command to check authentication
+        result = subprocess.run(["modal", "app", "list"], 
+                              capture_output=True, text=True, timeout=10)
+        
+        # If the command was successful, Modal is authenticated
+        if result.returncode == 0:
+            return True
+            
+        # Check for specific authentication errors
+        if "Token missing" in result.stderr or "Could not authenticate" in result.stderr:
+            print("\n" + "="*80)
+            print("🔑 MODAL AUTHENTICATION REQUIRED")
+            print("="*80)
+            print("GitArsenal requires Modal authentication to create cloud environments.")
+            print("\nTo authenticate with Modal, you need to:")
+            print("1. Create a Modal account at https://modal.com if you don't have one")
+            print("2. Run the following command to get a token:")
+            print("   modal token new")
+            print("3. Then set up your credentials in GitArsenal:")
+            print("   ./gitarsenal.py credentials set modal_token")
+            print("\nAfter completing these steps, try your command again.")
+            print("="*80)
+            return False
+            
+        # Other errors
+        print(f"⚠️ Modal command returned error: {result.stderr}")
+        return False
+        
+    except FileNotFoundError:
+        print("\n" + "="*80)
+        print("❌ MODAL CLI NOT INSTALLED")
+        print("="*80)
+        print("GitArsenal requires the Modal CLI to be installed.")
+        print("\nTo install Modal CLI, run:")
+        print("   pip install modal")
+        print("\nAfter installation, set up your credentials:")
+        print("1. Run 'modal token new' to authenticate")
+        print("2. Run './gitarsenal.py credentials set modal_token'")
+        print("="*80)
+        return False
+    except Exception as e:
+        print(f"⚠️ Error checking Modal authentication: {e}")
+        return False
 
 def main():
     parser = argparse.ArgumentParser(description="GitArsenal CLI - GPU-accelerated cloud environments")
@@ -82,8 +131,41 @@ def main():
             
         return run_script("manage_credentials.py", cred_args)
     
+    # For sandbox and SSH commands, check Modal authentication first
+    if args.command in ["sandbox", "ssh"]:
+        # Check if Modal is authenticated
+        if not check_modal_auth():
+            print("\n⚠️ Please authenticate with Modal before proceeding.")
+            return 1
+        
+        # Try to load credentials and set Modal token if available
+        try:
+            from credentials_manager import CredentialsManager
+            credentials_manager = CredentialsManager()
+            credentials = credentials_manager.load_credentials()
+            
+            if "modal_token" in credentials:
+                # Set the Modal token in the environment
+                os.environ["MODAL_TOKEN_ID"] = credentials["modal_token"]
+                print("✅ Using Modal token from credentials")
+                
+                # Try to authenticate with the token
+                try:
+                    token_result = subprocess.run(
+                        ["modal", "token", "set", credentials["modal_token"]], 
+                        capture_output=True, text=True
+                    )
+                    if token_result.returncode == 0:
+                        print("✅ Successfully authenticated with Modal")
+                except Exception as e:
+                    print(f"⚠️ Error setting Modal token: {e}")
+        except ImportError:
+            print("⚠️ Could not load credentials manager")
+        except Exception as e:
+            print(f"⚠️ Error loading credentials: {e}")
+    
     # Handle sandbox command
-    elif args.command == "sandbox":
+    if args.command == "sandbox":
         sandbox_args = []
         
         if args.gpu:

package/python/requirements.txt

@@ -0,0 +1,4 @@
+modal>=0.56.4
+requests>=2.31.0
+pathlib>=1.0.1
+python-dotenv>=1.0.0 

package/python/test_modalSandboxScript.py

@@ -478,36 +478,73 @@ def create_modal_sandbox(gpu_type, repo_url=None, repo_name=None, setup_commands
     
     # Check if Modal is authenticated
     try:
-        # Try to list apps to check if authentication is working
-        import subprocess
-        result = subprocess.run(["modal", "app", "list"], capture_output=True, text=True)
-        if result.returncode != 0 and "not authenticated" in result.stderr:
-            print("🔑 Modal authentication required")
-            
-            # Try to use credentials manager first
+        # Try to import modal first to check if it's installed
+        import modal
+        
+        # Try to access Modal token to check authentication
+        try:
+            # This will raise an exception if not authenticated
+            modal.config.get_current_workspace_name()
+            print("✅ Modal authentication verified")
+        except modal.exception.AuthError:
+            print("\n" + "="*80)
+            print("🔑 MODAL AUTHENTICATION REQUIRED")
+            print("="*80)
+            print("GitArsenal requires Modal authentication to create cloud environments.")
+            
+            # Try to get token from credentials manager
+            modal_token = None
             if credentials_manager:
-                modal_token = credentials_manager.get_modal_token()
-                if modal_token:
-                    # Set the token in the environment
-                    os.environ["MODAL_TOKEN_ID"] = modal_token
-                    print("✅ Modal token set from credentials manager")
-                    
-                    # Try to authenticate with the token
-                    try:
-                        token_result = subprocess.run(["modal", "token", "set", modal_token], 
-                                                    capture_output=True, text=True)
-                        if token_result.returncode == 0:
-                            print("✅ Successfully authenticated with Modal")
-                        else:
-                            print(f"⚠️ Failed to authenticate with Modal: {token_result.stderr}")
-                            print("Please run 'modal token new' manually and try again")
+                try:
+                    modal_token = credentials_manager.get_modal_token()
+                    if modal_token:
+                        # Set the token in the environment
+                        os.environ["MODAL_TOKEN_ID"] = modal_token
+                        print("✅ Modal token set from credentials manager")
+                        
+                        # Try to authenticate with the token
+                        try:
+                            import subprocess
+                            token_result = subprocess.run(
+                                ["modal", "token", "set", modal_token], 
+                                capture_output=True, text=True
+                            )
+                            if token_result.returncode == 0:
+                                print("✅ Successfully authenticated with Modal")
+                            else:
+                                print(f"⚠️ Failed to authenticate with Modal: {token_result.stderr}")
+                                print("\nPlease authenticate manually:")
+                                print("1. Run 'modal token new' to get a new token")
+                                print("2. Then restart this command")
+                                return None
+                        except Exception as e:
+                            print(f"⚠️ Error setting Modal token: {e}")
                             return None
-                    except Exception as e:
-                        print(f"⚠️ Error setting Modal token: {e}")
-                        return None
-            else:
-                print("⚠️ Modal is not authenticated. Please run 'modal token new' first")
+                except Exception as e:
+                    print(f"⚠️ Error getting Modal token: {e}")
+            
+            if not modal_token:
+                print("\nTo authenticate with Modal, you need to:")
+                print("1. Create a Modal account at https://modal.com if you don't have one")
+                print("2. Run the following command to get a token:")
+                print("   modal token new")
+                print("3. Then set up your credentials in GitArsenal:")
+                print("   ./gitarsenal.py credentials set modal_token")
+                print("\nAfter completing these steps, try your command again.")
+                print("="*80)
                 return None
+    except ImportError:
+        print("\n" + "="*80)
+        print("❌ MODAL PACKAGE NOT INSTALLED")
+        print("="*80)
+        print("GitArsenal requires the Modal package to be installed.")
+        print("\nTo install Modal, run:")
+        print("   pip install modal")
+        print("\nAfter installation, authenticate with Modal:")
+        print("1. Run 'modal token new'")
+        print("2. Then run './gitarsenal.py credentials set modal_token'")
+        print("="*80)
+        return None
     except Exception as e:
         print(f"⚠️ Error checking Modal authentication: {e}")
         print("Continuing anyway, but Modal operations may fail")
@@ -2393,53 +2430,177 @@ def fetch_setup_commands_from_api(repo_url):
         
         # Make the API request
         print(f"🌐 Making POST request to: {api_url}")
-        response = requests.post(api_url, json=payload, timeout=60)
-        
-        print(f"📥 API Response status code: {response.status_code}")
-        
-        if response.status_code == 200:
-            try:
-                data = response.json()
-                print(f"📄 API Response data received")
-                
-                # Extract setup commands from the response
-                if "setupInstructions" in data and "commands" in data["setupInstructions"]:
-                    commands = data["setupInstructions"]["commands"]
-                    print(f"✅ Successfully fetched {len(commands)} setup commands from API")
-                    
-                    # Print the commands for reference
-                    for i, cmd in enumerate(commands, 1):
-                        print(f"  {i}. {cmd}")
+        try:
+            response = requests.post(api_url, json=payload, timeout=60)
+            
+            print(f"📥 API Response status code: {response.status_code}")
+            
+            if response.status_code == 200:
+                try:
+                    data = response.json()
+                    print(f"📄 API Response data received")
                     
-                    return commands
-                else:
-                    print("⚠️ API response did not contain setupInstructions.commands field")
-                    print("📋 Available fields in response:")
-                    for key in data.keys():
-                        print(f"  - {key}")
-                    return []
-            except json.JSONDecodeError as e:
-                print(f"❌ Failed to parse API response as JSON: {e}")
-                print(f"Raw response: {response.text[:500]}...")
-                return []
-        else:
-            print(f"❌ API request failed with status code: {response.status_code}")
-            print(f"Error response: {response.text[:500]}...")
-            return []
-    except requests.exceptions.ConnectionError:
-        print(f"❌ Connection error: Could not connect to {api_url}")
-        print("⚠️ Make sure the API server is running at localhost:3000")
-        return []
+                    # Extract setup commands from the response
+                    if "setupInstructions" in data and "commands" in data["setupInstructions"]:
+                        commands = data["setupInstructions"]["commands"]
+                        print(f"✅ Successfully fetched {len(commands)} setup commands from API")
+                        
+                        # Print the commands for reference
+                        for i, cmd in enumerate(commands, 1):
+                            print(f"  {i}. {cmd}")
+                        
+                        return commands
+                    else:
+                        print("⚠️ API response did not contain setupInstructions.commands field")
+                        print("📋 Available fields in response:")
+                        for key in data.keys():
+                            print(f"  - {key}")
+                        # Return fallback commands
+                        return generate_fallback_commands(gitingest_data)
+                except json.JSONDecodeError as e:
+                    print(f"❌ Failed to parse API response as JSON: {e}")
+                    print(f"Raw response: {response.text[:500]}...")
+                    # Return fallback commands
+                    return generate_fallback_commands(gitingest_data)
+            elif response.status_code == 504:
+                print(f"❌ API request timed out (504 Gateway Timeout)")
+                print("⚠️ The server took too long to respond. Using fallback commands instead.")
+                # Return fallback commands
+                return generate_fallback_commands(gitingest_data)
+            else:
+                print(f"❌ API request failed with status code: {response.status_code}")
+                print(f"Error response: {response.text[:500]}...")
+                # Return fallback commands
+                return generate_fallback_commands(gitingest_data)
+        except requests.exceptions.Timeout:
+            print("❌ API request timed out after 60 seconds")
+            print("⚠️ Using fallback commands instead")
+            # Return fallback commands
+            return generate_fallback_commands(gitingest_data)
+        except requests.exceptions.ConnectionError:
+            print(f"❌ Connection error: Could not connect to {api_url}")
+            print("⚠️ Using fallback commands instead")
+            # Return fallback commands
+            return generate_fallback_commands(gitingest_data)
     except Exception as e:
         print(f"❌ Error fetching setup commands from API: {e}")
         import traceback
         traceback.print_exc()
-        return []
+        # Return fallback commands
+        return generate_fallback_commands(None)
     finally:
         # Clean up the temporary directory
         print(f"🧹 Cleaning up temporary directory...")
         shutil.rmtree(temp_dir, ignore_errors=True)
 
+def generate_fallback_commands(gitingest_data):
+    """Generate fallback setup commands based on repository analysis"""
+    print("\n" + "="*80)
+    print("📋 GENERATING FALLBACK SETUP COMMANDS")
+    print("="*80)
+    print("Using basic repository analysis to generate setup commands")
+    
+    # Default commands that work for most repositories
+    default_commands = [
+        "apt-get update -y",
+        "apt-get install -y git curl wget",
+        "pip install --upgrade pip setuptools wheel"
+    ]
+    
+    # If we don't have any analysis data, return default commands
+    if not gitingest_data:
+        print("⚠️ No repository analysis data available. Using default commands.")
+        return default_commands
+    
+    # Extract language and technologies information
+    detected_language = gitingest_data.get("system_info", {}).get("detected_language", "Unknown")
+    detected_technologies = gitingest_data.get("system_info", {}).get("detected_technologies", [])
+    primary_package_manager = gitingest_data.get("system_info", {}).get("primary_package_manager", "Unknown")
+    
+    # Add language-specific commands
+    language_commands = []
+    
+    print(f"📋 Detected primary language: {detected_language}")
+    print(f"📋 Detected technologies: {', '.join(detected_technologies) if detected_technologies else 'None'}")
+    print(f"📋 Detected package manager: {primary_package_manager}")
+    
+    # Python-specific commands
+    if detected_language == "Python" or primary_package_manager == "pip":
+        print("📦 Adding Python-specific setup commands")
+        
+        # Check for requirements.txt
+        requirements_check = [
+            "if [ -f requirements.txt ]; then",
+            "  echo 'Installing from requirements.txt'",
+            "  pip install -r requirements.txt",
+            "elif [ -f setup.py ]; then",
+            "  echo 'Installing from setup.py'",
+            "  pip install -e .",
+            "fi"
+        ]
+        language_commands.extend(requirements_check)
+        
+        # Add common Python packages
+        language_commands.append("pip install pytest numpy pandas matplotlib")
+    
+    # JavaScript/Node.js specific commands
+    elif detected_language in ["JavaScript", "TypeScript"] or primary_package_manager in ["npm", "yarn", "pnpm"]:
+        print("📦 Adding JavaScript/Node.js-specific setup commands")
+        
+        # Install Node.js if not available
+        language_commands.append("apt-get install -y nodejs npm")
+        
+        # Check for package.json
+        package_json_check = [
+            "if [ -f package.json ]; then",
+            "  echo 'Installing from package.json'",
+            "  npm install",
+            "fi"
+        ]
+        language_commands.extend(package_json_check)
+    
+    # Java specific commands
+    elif detected_language == "Java" or primary_package_manager in ["maven", "gradle"]:
+        print("📦 Adding Java-specific setup commands")
+        
+        language_commands.append("apt-get install -y openjdk-11-jdk maven gradle")
+        
+        # Check for Maven or Gradle
+        build_check = [
+            "if [ -f pom.xml ]; then",
+            "  echo 'Building with Maven'",
+            "  mvn clean install -DskipTests",
+            "elif [ -f build.gradle ]; then",
+            "  echo 'Building with Gradle'",
+            "  gradle build --no-daemon",
+            "fi"
+        ]
+        language_commands.extend(build_check)
+    
+    # Go specific commands
+    elif detected_language == "Go" or primary_package_manager == "go":
+        print("📦 Adding Go-specific setup commands")
+        
+        language_commands.append("apt-get install -y golang-go")
+        language_commands.append("go mod tidy")
+    
+    # Rust specific commands
+    elif detected_language == "Rust" or primary_package_manager == "cargo":
+        print("📦 Adding Rust-specific setup commands")
+        
+        language_commands.append("curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y")
+        language_commands.append("source $HOME/.cargo/env")
+        language_commands.append("cargo build")
+    
+    # Combine all commands
+    all_commands = default_commands + language_commands
+    
+    print("\n📋 Generated fallback setup commands:")
+    for i, cmd in enumerate(all_commands, 1):
+        print(f"  {i}. {cmd}")
+    
+    return all_commands
+
 def generate_basic_repo_analysis_from_url(repo_url):
     """Generate basic repository analysis data from a repository URL."""
     import tempfile