openmodal 0.3.0__tar.gz

openmodal-0.3.0/.github/workflows/docs.yml

@@ -0,0 +1,18 @@
+name: Deploy docs
+on:
+  push:
+    branches: [main]
+
+permissions:
+  contents: write
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+      - run: pip install mkdocs-material
+      - run: mkdocs gh-deploy --force

openmodal-0.3.0/.gitignore

@@ -0,0 +1,8 @@
+__pycache__/
+*.pyc
+dist/
+*.egg-info/
+_reference/
+.env
+jobs/
+site/

openmodal-0.3.0/PKG-INFO

@@ -0,0 +1,12 @@
+Metadata-Version: 2.4
+Name: openmodal
+Version: 0.3.0
+Summary: Run Modal on your own cloud. Open-source, cloud-agnostic serverless GPU compute.
+Requires-Python: ==3.12.*
+Requires-Dist: click>=8.3
+Requires-Dist: google-auth>=2.49
+Requires-Dist: kubernetes>=35.0
+Requires-Dist: requests>=2.33
+Provides-Extra: harbor
+Requires-Dist: harbor; extra == 'harbor'
+Requires-Dist: tenacity>=9.0; extra == 'harbor'

openmodal-0.3.0/README.md

@@ -0,0 +1,39 @@
+# OpenModal
+
+A cloud-agnostic runtime that implements [Modal](https://modal.com)'s Python interface.
+
+I built this because I wanted to run Modal on my own GCP account. Modal's API is clean and I didn't want to learn a different one. So OpenModal lets you write the same code and run it on your own infrastructure.
+
+```python
+import openmodal
+
+app = openmodal.App("my-experiment")
+
+@app.function(gpu="H100")
+def train(config):
+    ...
+
+results = train.map(configs)
+```
+
+## What works
+
+- `f.local()`, `f.remote()`, `f.map()`
+- GPU serving with auto scale-to-zero
+- Custom images, secrets, retries, volumes
+- GKE with spot GPUs (H100, A100, L4)
+- CLI: `openmodal run`, `deploy`, `stop`, `ps`
+
+## Get started
+
+```bash
+pip install openmodal
+gcloud auth login
+openmodal run examples/hello_world.py
+```
+
+[Setup guide](docs/setup.md) · [Examples](docs/examples/) · [Modal docs](https://modal.com/docs/guide) (same API, just swap the import)
+
+## License
+
+Apache-2.0

openmodal-0.3.0/docs/examples/harbor.md

@@ -0,0 +1,78 @@
+# Running SWE-bench with Harbor
+
+Run SWE-bench evaluations using [Harbor](https://harborframework.com) with OpenModal as the compute backend.
+
+## Install
+
+```bash
+pip install "openmodal[harbor]"
+```
+
+## Run
+
+```bash
+harbor run \
+  --agent mini-swe-agent \
+  --model openai/gpt-5.4 \
+  --environment-import-path openmodal.integrations.harbor_env:ModalEnvironment \
+  --dataset swe-bench/swe-bench-verified \
+  --n-tasks 1
+```
+
+This creates a sandbox on GKE, runs the agent against a real SWE-bench task, verifies the patch, and reports results.
+
+## What happens
+
+1. Harbor downloads a SWE-bench task (e.g., a Django bug)
+2. OpenModal creates a Kubernetes pod with the task's Docker image
+3. The agent runs inside the pod — reads the bug, edits code, runs tests
+4. Harbor uploads test files, runs verification, reports pass/fail
+5. Pod is cleaned up
+
+## Options
+
+**Different agents:**
+```bash
+# Claude Code
+harbor run --agent claude-code --model anthropic/claude-sonnet-4-5-20250929 \
+  --environment-import-path openmodal.integrations.harbor_env:ModalEnvironment \
+  --dataset swe-bench/swe-bench-verified --n-tasks 5
+
+# OpenHands
+harbor run --agent openhands --model openai/gpt-5.4 \
+  --environment-import-path openmodal.integrations.harbor_env:ModalEnvironment \
+  --dataset swe-bench/swe-bench-verified --n-tasks 5
+
+# SWE-agent
+harbor run --agent swe-agent --model openai/gpt-5.4 \
+  --environment-import-path openmodal.integrations.harbor_env:ModalEnvironment \
+  --dataset swe-bench/swe-bench-verified --n-tasks 5
+```
+
+**Multiple attempts:**
+```bash
+harbor run --agent mini-swe-agent --model openai/gpt-5.4 \
+  --environment-import-path openmodal.integrations.harbor_env:ModalEnvironment \
+  --dataset swe-bench/swe-bench-verified --n-tasks 10 --n-attempts 3
+```
+
+**View results:**
+```bash
+harbor view jobs
+```
+
+## How it compares to Modal
+
+The only difference from running Harbor with Modal is the `--environment-import-path` flag. With Modal:
+
+```bash
+harbor run --agent mini-swe-agent --model openai/gpt-5.4 --env modal --dataset swe-bench/swe-bench-verified
+```
+
+With OpenModal:
+
+```bash
+harbor run --agent mini-swe-agent --model openai/gpt-5.4 --environment-import-path openmodal.integrations.harbor_env:ModalEnvironment --dataset swe-bench/swe-bench-verified
+```
+
+Same agents, same datasets, same results — just runs on your own GCP infrastructure.

openmodal-0.3.0/docs/examples/hello_world.md

@@ -0,0 +1,65 @@
+# Hello, world!
+
+This example demonstrates the core features of OpenModal:
+
+- Run functions locally with `f.local()`
+- Run functions remotely on GCP with `f.remote()`
+- Run functions in parallel with `f.map()`
+
+## The code
+
+```python
+import sys
+import openmodal
+
+app = openmodal.App("example-hello-world")
+
+@app.function()
+def f(i):
+    if i % 2 == 0:
+        print("hello", i)
+    else:
+        print("world", i, file=sys.stderr)
+    return i * i
+
+@app.local_entrypoint()
+def main():
+    # Run locally
+    print(f.local(1000))
+
+    # Run remotely on GCP
+    print(f.remote(1000))
+
+    # Run in parallel on GCP
+    total = 0
+    for ret in f.map(range(200)):
+        total += ret
+    print(total)
+```
+
+## Run it
+
+```bash
+openmodal run examples/hello_world.py
+```
+
+```
+✓ Initialized.
+✓ Created objects.
+✓ Container created. (2 vCPU, 2 GB RAM • 34.135.113.28 • 14s)
+✓ Container ready. (60s total)
+hello 1000
+1000000
+1000000
+2646700
+✓ Containers cleaned up.
+✓ App completed.
+```
+
+## What happened?
+
+1. `f.local(1000)` ran on your machine — printed `hello 1000`, returned `1000000`
+2. `f.remote(1000)` created a GCE container, sent the function call to it, and returned the result
+3. `f.map(range(200))` sent 200 calls to the remote container in parallel and streamed results back
+
+The container was automatically cleaned up when the script finished.

openmodal-0.3.0/docs/examples/index.md

@@ -0,0 +1,28 @@
+# Examples
+
+OpenModal is API-compatible with Modal. If you've seen a Modal example, you can run it
+on OpenModal by replacing `import modal` with `import openmodal`.
+
+For hundreds of additional examples, see the [Modal examples gallery](https://modal.com/docs/examples) —
+the same code works with OpenModal on your own GCP infrastructure.
+
+## Getting started
+
+- [Hello, world!](hello_world.md) — `f.local()`, `f.remote()`, `f.map()`
+- [Web scraper](web_scraper.md) — custom images, async, parallel execution, CLI args
+
+## GPU serving
+
+- [vLLM serving](vllm_serving.md) — deploy a quantized LLM on H100 with auto scale-to-zero
+
+## Sandboxes
+
+- [Sandboxes](sandbox.md) — isolated containers for SWE agents, parallel execution
+
+## Training
+
+- [SFT finetuning](sft_finetune.md) — LoRA finetuning with Unsloth on a single H100
+
+## Benchmarks
+
+- [SWE-bench with Harbor](harbor.md) — run SWE-bench evaluations on your own GCP

openmodal-0.3.0/docs/examples/sandbox.md

@@ -0,0 +1,87 @@
+# Sandboxes
+
+Sandboxes are isolated containers you can exec commands into — like SSH into a fresh machine. They're used by SWE agents to run code, edit files, and run tests in a clean environment.
+
+## Basic usage
+
+```python
+import openmodal
+
+app = openmodal.App("my-agent")
+image = openmodal.Image.debian_slim().apt_install("git").pip_install("requests")
+
+sandbox = openmodal.Sandbox.create(image=image, app=app, timeout=300)
+
+result = sandbox.exec("echo hello")
+print(result.output)      # "hello"
+print(result.returncode)  # 0
+
+sandbox.exec("git clone https://github.com/pallets/click.git /workspace")
+sandbox.exec("cd /workspace && python3 -m pytest tests/")
+
+sandbox.terminate()
+```
+
+## How it works
+
+Each sandbox is a Kubernetes pod on GKE:
+
+1. `Sandbox.create()` → creates a pod with your image, keeps it alive with `sleep`
+2. `sandbox.exec(command)` → runs bash commands inside the pod via Kubernetes exec API
+3. Files persist between execs — the pod stays alive until you terminate it
+4. `sandbox.terminate()` → deletes the pod
+
+## Parallel sandboxes
+
+Multiple sandboxes run simultaneously on the same cluster. Each is fully isolated.
+
+```python
+import concurrent.futures
+
+def run_agent(agent_id):
+    sandbox = openmodal.Sandbox.create(image=image, app=app)
+    sandbox.exec(f"echo 'agent {agent_id}' > /tmp/id.txt")
+    result = sandbox.exec("cat /tmp/id.txt")
+    sandbox.terminate()
+    return result.output
+
+with concurrent.futures.ThreadPoolExecutor(max_workers=4) as pool:
+    results = list(pool.map(run_agent, range(4)))
+```
+
+## Performance
+
+With cached images on warm nodes:
+
+| Operation | Latency |
+|---|---|
+| `Sandbox.create()` | ~5s |
+| `sandbox.exec()` | ~0.2s |
+| `sandbox.terminate()` | instant |
+| 4 parallel sandboxes | ~5s (not 4x5s) |
+
+First run builds the image (~2-3 min via Cloud Build). After that, the image is cached and creation is fast.
+
+## Isolation
+
+Each sandbox is its own pod:
+- Separate filesystem — files in one sandbox don't appear in another
+- Separate processes — nothing shared between sandboxes
+- Separate network — each pod gets its own IP
+
+## Run the example
+
+```bash
+openmodal run examples/sandbox.py
+```
+
+```
+Launching 4 sandboxes in parallel...
+
+Agent 0 (sandbox-test-4bb1bb27): create=5.3s total=11.3s
+Agent 1 (sandbox-test-6f4a83f3): create=5.3s total=11.5s
+Agent 2 (sandbox-test-c5696129): create=5.2s total=11.7s
+Agent 3 (sandbox-test-c8ecb817): create=4.2s total=10.4s
+
+All 4 agents passed in 11.7s
+```

openmodal-0.3.0/docs/examples/sft_finetune.md

@@ -0,0 +1,88 @@
+# SFT finetuning with Unsloth
+
+Finetune an LLM with LoRA on a single GPU using [Unsloth](https://github.com/unslothai/unsloth)'s optimized training. Based on [Modal's Unsloth example](https://modal.com/docs/examples/unsloth-finetune).
+
+## Run
+
+```bash
+openmodal run examples/sft_finetune.py
+```
+
+With custom settings:
+
+```bash
+openmodal run examples/sft_finetune.py --model-name unsloth/Qwen3-4B --max-steps 1000 --lora-r 32
+```
+
+## What it does
+
+1. Spins up an H100 GPU on GKE
+2. Downloads Qwen3-4B (4-bit quantized) and the FineTome-100k dataset
+3. Applies LoRA adapters and trains with Unsloth's optimized kernels
+4. Saves checkpoints and final model to persistent GCS volumes
+5. Supports resuming from checkpoints if interrupted
+
+## The code
+
+```python
+import openmodal
+
+app = openmodal.App("sft-finetune")
+
+train_image = (
+    openmodal.Image.debian_slim()
+    .uv_pip_install(
+        "accelerate", "datasets", "peft",
+        "transformers", "trl",
+        "unsloth[cu128-torch270]",
+    )
+    .env({"HF_HOME": "/model_cache"})
+)
+
+model_cache = openmodal.Volume.from_name("sft-model-cache", create_if_missing=True)
+checkpoints = openmodal.Volume.from_name("sft-checkpoints", create_if_missing=True)
+
+@app.function(
+    image=train_image,
+    gpu="H100",
+    volumes={"/model_cache": model_cache, "/checkpoints": checkpoints},
+    timeout=6 * 60 * 60,
+    retries=3,
+)
+def finetune(config):
+    from unsloth import FastLanguageModel
+    from trl import SFTTrainer
+
+    model, tokenizer = FastLanguageModel.from_pretrained(
+        model_name=config.model_name,
+        load_in_4bit=True,
+    )
+    # ... LoRA setup, dataset loading, training ...
+    trainer.train()
+    model.save_pretrained("/checkpoints/final_model")
+
+@app.local_entrypoint()
+def main(model_name="unsloth/Qwen3-4B", max_steps=5):
+    finetune.remote(TrainingConfig(model_name=model_name, max_steps=max_steps))
+```
+
+## Features used
+
+| Feature | How it's used |
+|---|---|
+| `gpu="H100"` | Single H100 for training |
+| `Volume.from_name(create_if_missing=True)` | Persistent storage for model weights, datasets, checkpoints |
+| `retries=3` | Auto-retry on preemption (spot instances) |
+| `timeout=6*60*60` | 6 hour max training time |
+| `finetune.remote(config)` | Runs training on the cloud GPU |
+| CLI args (`--max-steps`, `--lora-r`) | Tweak hyperparameters from command line |
+
+## Compared to Modal
+
+The only difference from Modal's example is the import line:
+
+```python
+import openmodal  # instead of: import modal
+```
+
+Everything else — image definition, volumes, GPU selection, remote execution — is the same API.

openmodal-0.3.0/docs/examples/vllm_serving.md

@@ -0,0 +1,115 @@
+# GPU serving with vLLM
+
+Deploy a model on a GPU and get an OpenAI-compatible endpoint. Scales to zero when idle.
+
+## The code
+
+```python
+import openmodal
+
+MODEL_NAME = "Qwen/Qwen3.5-0.8B"
+
+vllm_image = (
+    openmodal.Image.from_registry("nvidia/cuda:12.8.0-devel-ubuntu22.04", add_python="3.12")
+    .entrypoint([])
+    .apt_install("git")
+    .uv_pip_install("vllm", "huggingface-hub==0.36.0",
+                    extra_options="--extra-index-url https://wheels.vllm.ai/nightly")
+    .pip_install("transformers @ git+https://github.com/huggingface/transformers.git@main")
+)
+
+app = openmodal.App("vllm-test")
+
+@app.function(
+    image=vllm_image,
+    gpu="H100",
+    scaledown_window=5 * 60,
+    timeout=10 * 60,
+)
+@openmodal.web_server(port=8000, startup_timeout=20 * 60)
+@openmodal.concurrent(max_inputs=8)
+def serve():
+    import subprocess
+    subprocess.Popen([
+        "vllm", "serve", MODEL_NAME,
+        "--host", "0.0.0.0", "--port", "8000",
+        "--served-model-name", MODEL_NAME,
+        "--max-model-len", "4096",
+        "--enforce-eager",
+    ])
+```
+
+## Deploy
+
+```bash
+openmodal deploy examples/vllm_serving.py
+```
+
+```
+openmodal deploy: vllm-test
+  building image...
+  image: us-central1-docker.pkg.dev/.../vllm-test:a9b8fa41ec13
+  creating container (H100)...
+  waiting for healthy (timeout: 1200s)...
+  serve => http://104.155.171.209:8000
+deploy complete.
+```
+
+## Query
+
+```bash
+curl http://104.155.171.209:8000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"model":"Qwen/Qwen3.5-0.8B","messages":[{"role":"user","content":"What is 2+2?"}],"max_tokens":16}'
+```
+
+Works with any OpenAI client:
+
+```python
+from openai import OpenAI
+client = OpenAI(base_url="http://104.155.171.209:8000/v1", api_key="unused")
+resp = client.chat.completions.create(
+    model="Qwen/Qwen3.5-0.8B",
+    messages=[{"role": "user", "content": "What is 2+2?"}],
+)
+print(resp.choices[0].message.content)
+```
+
+## Stop
+
+```bash
+openmodal stop vllm-test
+```
+
+## Under the hood
+
+When you run `openmodal deploy`, here's what happens:
+
+**Building the image**
+
+Your image definition (`debian_slim().pip_install(...)`) gets turned into a Dockerfile and built via Google Cloud Build. The built image is stored in Artifact Registry. If you deploy the same code again, the image is already cached and this step is skipped.
+
+**Starting the server**
+
+OpenModal sees `gpu="H100"` + `@web_server` and picks GKE (Kubernetes) as the backend. It creates three things:
+
+- A **Deployment** — tells Kubernetes "run one copy of this container with an H100 GPU"
+- A **Service** — gives it a public IP so you can send requests to it
+- A **CronJob** — checks every minute if anyone is using the server
+
+GKE doesn't have an H100 machine sitting around, so it provisions one (a spot instance, ~60% cheaper). This takes a few minutes. Once the machine is ready, your container starts, vLLM loads the model, and the health check passes.
+
+**Scaling down**
+
+The CronJob runs every minute and checks: are there any active TCP connections to port 8000? If there haven't been any for `scaledown_window` seconds (5 min in this example), it scales the Deployment to 0 — meaning the container is stopped.
+
+Once the container is gone, the H100 machine has nothing running on it. GKE's node autoscaler notices this and removes the machine after ~5 minutes. Now you're paying $0 for GPUs.
+
+**Costs**
+
+| State | What you pay |
+|---|---|
+| Serving requests | ~$1.20/hr (H100 spot) |
+| Idle, within scaledown window | Same |
+| Scaled to zero | ~$0.10/hr (cluster overhead) |
+| Cluster deleted | $0 |

openmodal-0.3.0/docs/examples/web_scraper.md

@@ -0,0 +1,117 @@
+# Web scraper
+
+This example shows how to build a distributed web scraper with OpenModal,
+progressing from a simple local script to parallel remote execution with
+custom container images.
+
+## Step 1: Scrape links locally
+
+Start with plain Python:
+
+```python
+import re
+import urllib.request
+
+def get_links(url):
+    response = urllib.request.urlopen(url)
+    html = response.read().decode("utf8")
+    links = []
+    for match in re.finditer('href="(.*?)"', html):
+        links.append(match.group(1))
+    return links
+
+if __name__ == "__main__":
+    print(get_links("http://example.com"))
+```
+
+```bash
+python webscraper.py
+# ['https://www.iana.org/domains/example']
+```
+
+## Step 2: Run it remotely
+
+Add OpenModal — the only changes are the import, the decorator, and the entrypoint:
+
+```python
+import re
+import urllib.request
+import openmodal
+
+app = openmodal.App(name="example-webscraper")
+
+@app.function()
+def get_links(url):
+    response = urllib.request.urlopen(url)
+    html = response.read().decode("utf8")
+    links = []
+    for match in re.finditer('href="(.*?)"', html):
+        links.append(match.group(1))
+    return links
+
+@app.local_entrypoint()
+def main(url: str = "http://example.com"):
+    links = get_links.remote(url)
+    print(links)
+```
+
+```bash
+openmodal run examples/webscraper.py --url http://example.com
+```
+
+```
+✓ Initialized.
+✓ Created objects.
+['https://iana.org/domains/example']
+✓ App completed.
+```
+
+The function ran on a GCE container, not on your machine.
+
+## Step 3: Add dependencies with a custom image
+
+Use `requests` and `beautifulsoup4` for better HTML parsing.
+Define a custom container image with the dependencies:
+
+```python
+import openmodal
+
+app = openmodal.App("example-webscraper-requests")
+
+scraper_image = openmodal.Image.debian_slim().pip_install("requests", "beautifulsoup4")
+
+@app.function(image=scraper_image)
+async def get_links(url: str) -> list[str]:
+    import asyncio
+    import requests
+    from bs4 import BeautifulSoup
+
+    resp = await asyncio.to_thread(requests.get, url, timeout=10)
+    soup = BeautifulSoup(resp.text, "html.parser")
+    return [a["href"] for a in soup.find_all("a", href=True)]
+
+@app.local_entrypoint()
+def main():
+    urls = ["http://example.com", "http://modal.com"]
+    for links in get_links.map(urls):
+        for link in links:
+            print(link)
+```
+
+```bash
+openmodal run examples/webscraper_requests.py
+```
+
+The first run builds the Docker image (takes ~2 minutes). Subsequent runs
+use the cached image and start much faster.
+
+## What this demonstrates
+
+| Feature | How it's used |
+|---|---|
+| `f.remote(url)` | Run a single function call on GCP |
+| `f.map(urls)` | Run multiple calls in parallel |
+| `Image.debian_slim()` | Base container image with Python |
+| `.pip_install(...)` | Add Python packages to the image |
+| `async def` | Async functions work transparently |
+| CLI args (`--url`) | Entrypoint parameters become CLI flags |

openmodal-0.3.0/docs/index.md

@@ -0,0 +1,39 @@
+# OpenModal
+
+A cloud-agnostic runtime that implements [Modal](https://modal.com)'s Python interface.
+
+I built this because I wanted to run Modal on my own GCP account. Modal's API is clean and I didn't want to learn a different one. So OpenModal lets you write the same code and run it on your own infrastructure.
+
+```python
+import openmodal
+
+app = openmodal.App("my-experiment")
+
+@app.function(gpu="H100")
+def train(config):
+    ...
+
+results = train.map(configs)
+```
+
+## What works
+
+- `f.local()`, `f.remote()`, `f.map()`
+- GPU serving with auto scale-to-zero
+- Custom images, secrets, retries, volumes
+- GKE with spot GPUs (H100, A100, L4)
+- CLI: `openmodal run`, `deploy`, `stop`, `ps`
+
+## Get started
+
+```bash
+pip install openmodal
+gcloud auth login
+openmodal run examples/hello_world.py
+```
+
+[Setup guide](setup.md) · [Examples](examples/index.md) · [Modal docs](https://modal.com/docs/guide) (same API, just swap the import)
+
+## License
+
+Apache-2.0