crunr 0.1.0__tar.gz

crunr-0.1.0/PKG-INFO

@@ -0,0 +1,153 @@
+Metadata-Version: 2.4
+Name: crunr
+Version: 0.1.0
+Summary: Run any compute job on AWS with a single command
+License-Expression: MIT
+Keywords: aws,ec2,cloud,gpu,machine-learning,spot
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: System :: Distributed Computing
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: boto3>=1.34
+Requires-Dist: botocore>=1.34
+Requires-Dist: rich>=13.7
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4; extra == "dev"
+Requires-Dist: pytest-mock>=3.12; extra == "dev"
+Requires-Dist: moto[ec2,sts]>=5.0; extra == "dev"
+Requires-Dist: ruff>=0.3; extra == "dev"
+Requires-Dist: mypy>=1.8; extra == "dev"
+Requires-Dist: boto3-stubs[ec2,sts]>=1.34; extra == "dev"
+
+# crunr
+
+Run any compute job on AWS EC2 with a single command — no DevOps required.
+
+```
+crunr run train.py --gpu
+```
+
+crunr provisions an instance, uploads your code, streams live output, downloads results, and terminates the instance automatically. Zero idle cost.
+
+## Install
+
+```bash
+pip install crunr
+```
+
+Requires Python 3.10+ and an AWS account.
+
+## Quick start
+
+```bash
+# 1. Configure AWS credentials (one-time)
+crunr auth
+
+# 2. Run a script on the cheapest CPU instance
+crunr run script.py
+
+# 3. Run on a GPU instance
+crunr run train.py --gpu
+
+# 4. Specify minimum VRAM
+crunr run train.py --gpu --memory 24
+
+# 5. Pass environment variables
+crunr run train.py --env EPOCHS=50 --env LR=0.001
+```
+
+## How it works
+
+1. **Provision** — selects the cheapest matching spot instance, falls back to on-demand
+2. **Sync** — uploads your local directory to the instance (rsync or scp+tar)
+3. **Execute** — runs your command with live log streaming
+4. **Collect** — downloads any `outputs/` directory back to your machine
+5. **Terminate** — instance is always destroyed, even on Ctrl+C or crash
+
+## Commands
+
+| Command | Description |
+|---|---|
+| `crunr auth` | Configure AWS credentials |
+| `crunr run <script>` | Run a job on EC2 |
+| `crunr jobs` | Show job history |
+| `crunr ps` | List running instances |
+| `crunr clean` | Terminate all orphaned instances |
+
+## `crunr run` options
+
+```
+--gpu               Request a GPU instance (cheapest available)
+--memory GB         Minimum GPU VRAM or RAM in GB
+--instance TYPE     Exact EC2 instance type (e.g. g5.xlarge)
+--disk GB           Root EBS volume size (default: 8 GB CPU, 100 GB GPU)
+--env KEY=VALUE     Environment variable passed to the job (repeatable)
+--dir PATH          Local directory to sync (default: current directory)
+--on-demand         Use on-demand pricing instead of spot
+--profile NAME      AWS credential profile
+--region REGION     Override AWS region
+```
+
+## Saving outputs
+
+Your script can write files to an `outputs/` directory. crunr automatically downloads it after the job finishes:
+
+```python
+import os
+os.makedirs("outputs", exist_ok=True)
+with open("outputs/result.txt", "w") as f:
+    f.write("done")
+```
+
+## AWS IAM permissions
+
+The IAM user needs these permissions:
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": [
+        "ec2:RunInstances", "ec2:TerminateInstances", "ec2:DescribeInstances",
+        "ec2:DescribeImages", "ec2:DescribeSpotPriceHistory",
+        "ec2:CreateKeyPair", "ec2:DeleteKeyPair", "ec2:DescribeKeyPairs",
+        "ec2:CreateSecurityGroup", "ec2:DescribeSecurityGroups",
+        "ec2:AuthorizeSecurityGroupIngress",
+        "ec2:CreateTags",
+        "sts:GetCallerIdentity"
+      ],
+      "Resource": "*"
+    },
+    {
+      "Effect": "Allow",
+      "Action": "iam:CreateServiceLinkedRole",
+      "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/*"
+    }
+  ]
+}
+```
+
+## Cost
+
+You only pay for the time the instance runs. Spot instances are typically **60–90% cheaper** than on-demand.
+
+Example costs (spot, us-east-1):
+- `t3.micro` CPU job — ~$0.003/hr
+- `g4dn.xlarge` GPU (T4) — ~$0.16/hr
+- `g5.xlarge` GPU (A10G) — ~$0.34/hr
+
+Data transfer into EC2 is free. Transfer out costs ~$0.09/GB.
+
+## License
+
+MIT

crunr-0.1.0/README.md

@@ -0,0 +1,124 @@
+# crunr
+
+Run any compute job on AWS EC2 with a single command — no DevOps required.
+
+```
+crunr run train.py --gpu
+```
+
+crunr provisions an instance, uploads your code, streams live output, downloads results, and terminates the instance automatically. Zero idle cost.
+
+## Install
+
+```bash
+pip install crunr
+```
+
+Requires Python 3.10+ and an AWS account.
+
+## Quick start
+
+```bash
+# 1. Configure AWS credentials (one-time)
+crunr auth
+
+# 2. Run a script on the cheapest CPU instance
+crunr run script.py
+
+# 3. Run on a GPU instance
+crunr run train.py --gpu
+
+# 4. Specify minimum VRAM
+crunr run train.py --gpu --memory 24
+
+# 5. Pass environment variables
+crunr run train.py --env EPOCHS=50 --env LR=0.001
+```
+
+## How it works
+
+1. **Provision** — selects the cheapest matching spot instance, falls back to on-demand
+2. **Sync** — uploads your local directory to the instance (rsync or scp+tar)
+3. **Execute** — runs your command with live log streaming
+4. **Collect** — downloads any `outputs/` directory back to your machine
+5. **Terminate** — instance is always destroyed, even on Ctrl+C or crash
+
+## Commands
+
+| Command | Description |
+|---|---|
+| `crunr auth` | Configure AWS credentials |
+| `crunr run <script>` | Run a job on EC2 |
+| `crunr jobs` | Show job history |
+| `crunr ps` | List running instances |
+| `crunr clean` | Terminate all orphaned instances |
+
+## `crunr run` options
+
+```
+--gpu               Request a GPU instance (cheapest available)
+--memory GB         Minimum GPU VRAM or RAM in GB
+--instance TYPE     Exact EC2 instance type (e.g. g5.xlarge)
+--disk GB           Root EBS volume size (default: 8 GB CPU, 100 GB GPU)
+--env KEY=VALUE     Environment variable passed to the job (repeatable)
+--dir PATH          Local directory to sync (default: current directory)
+--on-demand         Use on-demand pricing instead of spot
+--profile NAME      AWS credential profile
+--region REGION     Override AWS region
+```
+
+## Saving outputs
+
+Your script can write files to an `outputs/` directory. crunr automatically downloads it after the job finishes:
+
+```python
+import os
+os.makedirs("outputs", exist_ok=True)
+with open("outputs/result.txt", "w") as f:
+    f.write("done")
+```
+
+## AWS IAM permissions
+
+The IAM user needs these permissions:
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": [
+        "ec2:RunInstances", "ec2:TerminateInstances", "ec2:DescribeInstances",
+        "ec2:DescribeImages", "ec2:DescribeSpotPriceHistory",
+        "ec2:CreateKeyPair", "ec2:DeleteKeyPair", "ec2:DescribeKeyPairs",
+        "ec2:CreateSecurityGroup", "ec2:DescribeSecurityGroups",
+        "ec2:AuthorizeSecurityGroupIngress",
+        "ec2:CreateTags",
+        "sts:GetCallerIdentity"
+      ],
+      "Resource": "*"
+    },
+    {
+      "Effect": "Allow",
+      "Action": "iam:CreateServiceLinkedRole",
+      "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/*"
+    }
+  ]
+}
+```
+
+## Cost
+
+You only pay for the time the instance runs. Spot instances are typically **60–90% cheaper** than on-demand.
+
+Example costs (spot, us-east-1):
+- `t3.micro` CPU job — ~$0.003/hr
+- `g4dn.xlarge` GPU (T4) — ~$0.16/hr
+- `g5.xlarge` GPU (A10G) — ~$0.34/hr
+
+Data transfer into EC2 is free. Transfer out costs ~$0.09/GB.
+
+## License
+
+MIT

crunr-0.1.0/crunr/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.1.0"

crunr-0.1.0/crunr/auth.py

@@ -0,0 +1,230 @@
+"""AWS credential management: interactive wizard, list, verify, set-default."""
+
+from __future__ import annotations
+
+import configparser
+import os
+import stat
+import sys
+from pathlib import Path
+from typing import Optional
+
+import boto3
+import botocore.exceptions
+
+from . import console
+from .config import REGIONS
+
+# ---------------------------------------------------------------------------
+# File paths
+# ---------------------------------------------------------------------------
+
+AWS_DIR = Path.home() / ".aws"
+CREDS_FILE = AWS_DIR / "credentials"
+CONFIG_FILE = AWS_DIR / "config"
+
+
+# ---------------------------------------------------------------------------
+# Public API
+# ---------------------------------------------------------------------------
+
+def cmd_auth(profile_name: str = "default") -> None:
+    """Interactive wizard to add/update an AWS profile."""
+    console.header(
+        "crunr auth",
+        f"Configure AWS credentials → profile: [bold]{profile_name}[/bold]",
+    )
+
+    console.step("Enter your AWS credentials")
+    console.info("  Find these at: AWS Console → IAM → Users → Security credentials")
+
+    key_id = console.prompt("AWS Access Key ID")
+    if not key_id:
+        console.error("Access Key ID cannot be empty.")
+        sys.exit(1)
+
+    secret = console.prompt_secret("AWS Secret Access Key")
+    if not secret:
+        console.error("Secret Access Key cannot be empty.")
+        sys.exit(1)
+
+    region = console.choose("Select a region", REGIONS)
+
+    _ensure_aws_dir()
+    _write_credentials(profile_name, key_id, secret)
+    _write_config(profile_name, region)
+
+    console.ok(f"Credentials written to {CREDS_FILE}")
+
+    with console.spinner("Verifying credentials with AWS STS…"):
+        ok, identity, err = _verify(profile_name)
+
+    if ok:
+        console.ok(f"Authenticated as: [bold]{identity}[/bold]")
+        if profile_name != "default":
+            console.info(f"  Use with: crunr run script.py --profile {profile_name}")
+    else:
+        console.error(f"Credential verification failed: {err}")
+        console.warn("Credentials were saved but may be invalid. Double-check them with:")
+        console.info(f"  crunr auth --verify {profile_name}")
+        sys.exit(1)
+
+
+def cmd_list_profiles() -> None:
+    """Print a table of all configured profiles."""
+    profiles = _read_all_profiles()
+    default = _read_default_profile_name()
+    for p in profiles:
+        p["is_default"] = (p["name"] == default)
+    console.profiles_table(profiles)
+
+
+def cmd_verify(profile_name: str) -> None:
+    """Test that a profile's credentials still work."""
+    with console.spinner(f"Verifying profile [bold]{profile_name}[/bold]…"):
+        ok, identity, err = _verify(profile_name)
+    if ok:
+        console.ok(f"Profile [bold]{profile_name}[/bold] is valid — authenticated as [bold]{identity}[/bold]")
+    else:
+        console.error(f"Profile [bold]{profile_name}[/bold] failed: {err}")
+        sys.exit(1)
+
+
+def cmd_set_default(profile_name: str) -> None:
+    """Copy an existing profile's settings into the [default] slot."""
+    profiles = {p["name"]: p for p in _read_all_profiles()}
+    if profile_name not in profiles:
+        console.error(f"Profile '{profile_name}' not found. Run 'crunr auth --list' to see available profiles.")
+        sys.exit(1)
+
+    p = profiles[profile_name]
+    _ensure_aws_dir()
+
+    # Read key and secret from the existing profile entry
+    creds = _read_configparser(CREDS_FILE)
+    section = profile_name if profile_name != "default" else "default"
+    if not creds.has_section(section):
+        console.error(f"No credentials found for profile '{profile_name}'.")
+        sys.exit(1)
+
+    key_id = creds.get(section, "aws_access_key_id", fallback="")
+    secret = creds.get(section, "aws_secret_access_key", fallback="")
+    _write_credentials("default", key_id, secret)
+    _write_config("default", p["region"])
+
+    console.ok(f"Default profile now points to [bold]{profile_name}[/bold] ({p['region']})")
+
+
+def get_boto3_session(profile: Optional[str] = None, region: Optional[str] = None) -> boto3.Session:
+    """Return a boto3 Session using the given profile (or default)."""
+    try:
+        session = boto3.Session(profile_name=profile, region_name=region)
+        # Force credential resolution now so errors surface early
+        creds = session.get_credentials()
+        if creds is None:
+            raise RuntimeError("No AWS credentials found.")
+        return session
+    except botocore.exceptions.ProfileNotFound:
+        console.error(
+            f"AWS profile '{profile}' not found.\n"
+            "  Run 'crunr auth' to configure credentials, or 'crunr auth --list' to see profiles."
+        )
+        sys.exit(1)
+    except botocore.exceptions.NoCredentialsError:
+        console.error(
+            "No AWS credentials found.\n"
+            "  Run 'crunr auth' to set up your AWS credentials."
+        )
+        sys.exit(1)
+
+
+# ---------------------------------------------------------------------------
+# Internal helpers
+# ---------------------------------------------------------------------------
+
+def _ensure_aws_dir() -> None:
+    AWS_DIR.mkdir(mode=0o700, parents=True, exist_ok=True)
+
+
+def _read_configparser(path: Path) -> configparser.ConfigParser:
+    cp = configparser.ConfigParser()
+    if path.exists():
+        cp.read(path)
+    return cp
+
+
+def _write_credentials(profile_name: str, key_id: str, secret: str) -> None:
+    cp = _read_configparser(CREDS_FILE)
+    section = profile_name  # credentials file uses bare profile names
+    if not cp.has_section(section):
+        cp.add_section(section)
+    cp.set(section, "aws_access_key_id", key_id)
+    cp.set(section, "aws_secret_access_key", secret)
+
+    with CREDS_FILE.open("w") as f:
+        cp.write(f)
+
+    _set_private(CREDS_FILE)
+
+
+def _write_config(profile_name: str, region: str) -> None:
+    cp = _read_configparser(CONFIG_FILE)
+    # config file uses "profile <name>" for non-default profiles
+    section = "default" if profile_name == "default" else f"profile {profile_name}"
+    if not cp.has_section(section):
+        cp.add_section(section)
+    cp.set(section, "region", region)
+    cp.set(section, "output", "json")
+
+    with CONFIG_FILE.open("w") as f:
+        cp.write(f)
+
+    _set_private(CONFIG_FILE)
+
+
+def _set_private(path: Path) -> None:
+    try:
+        path.chmod(stat.S_IRUSR | stat.S_IWUSR)  # 0o600
+    except OSError:
+        pass  # Windows — best effort
+
+
+def _verify(profile_name: str) -> tuple[bool, str, str]:
+    try:
+        session = boto3.Session(profile_name=profile_name)
+        sts = session.client("sts")
+        identity = sts.get_caller_identity()
+        arn = identity.get("Arn", "unknown")
+        return True, arn, ""
+    except botocore.exceptions.ProfileNotFound as e:
+        return False, "", str(e)
+    except botocore.exceptions.ClientError as e:
+        return False, "", str(e)
+    except botocore.exceptions.NoCredentialsError as e:
+        return False, "", str(e)
+    except Exception as e:  # noqa: BLE001
+        return False, "", str(e)
+
+
+def _read_all_profiles() -> list[dict]:  # type: ignore[type-arg]
+    cp_creds = _read_configparser(CREDS_FILE)
+    cp_conf = _read_configparser(CONFIG_FILE)
+
+    profiles = []
+    for section in cp_creds.sections():
+        key_id = cp_creds.get(section, "aws_access_key_id", fallback="")
+        # Mask key for display
+        masked = key_id[:4] + "****" + key_id[-4:] if len(key_id) >= 8 else "****"
+
+        # Find region from config file
+        conf_section = "default" if section == "default" else f"profile {section}"
+        region = cp_conf.get(conf_section, "region", fallback="us-east-1")
+
+        profiles.append({"name": section, "region": region, "key_id": masked})
+
+    return profiles
+
+
+def _read_default_profile_name() -> str:
+    """Returns the name of the current default profile (always 'default')."""
+    return "default"