omnirun 0.2.0__tar.gz → 0.2.2__tar.gz

{omnirun-0.2.0 → omnirun-0.2.2}/.github/workflows/publish.yml

@@ -22,6 +22,9 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v7
 
+      - name: Check tag matches package version
+        run: ./scripts/check-version.sh "${GITHUB_REF_NAME}"
+
       - name: Install uv
         uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
         with:

{omnirun-0.2.0 → omnirun-0.2.2}/DESIGN.md

@@ -282,10 +282,13 @@ Worker access to private repos — **no git credentials ever leave the laptop**:
     bundle base64 inside the kernel's `run.py`** (next to the base64 bootstrap) — *not* a
     dataset. The old dataset approach 409'd every kernel that referenced a still-processing
     dataset (a systematic race) and needed a create/delete lifecycle; embedding removes
-    both. A size guard (`MAX_EMBED_B64`, 40 MiB, covering bundle + env together) rejects
-    oversized repos, so this suits code-sized repos (data is never shipped — jobs fetch
-    their own). The bootstrap clones/fetches from the bundle into the object store. In the
-    public-repo case no bundle is embedded, so the ceiling is a non-issue.
+    both. A size guard (`KAGGLE_MAX_SOURCE_BYTES`, ~1 MiB, measuring the whole `run.py`
+    kernel source — embedded bootstrap + bundle + env — against Kaggle's real push limit,
+    overridable per backend via `max_source_bytes`) rejects oversized pushes early with a
+    size-naming error instead of an opaque Kaggle-side failure, so this suits code-sized
+    repos (data is never shipped — jobs fetch their own). The bootstrap clones/fetches from
+    the bundle into the object store. In the public-repo case no bundle is embedded, so the
+    ceiling is a non-issue.
 
   This keeps the invariant intact: a public clone needs no credentials, and private repos
   still never touch origin from the worker. What changed is that public-repo workers now

{omnirun-0.2.0 → omnirun-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: omnirun
-Version: 0.2.0
+Version: 0.2.2
 Summary: Run jobs from your repo anywhere: Slurm over SSH, any SSH box, Kaggle, Colab, or marketplace GPUs
 Author-email: Dmitrii Mukhutdinov <flyingleafe@gmail.com>
 License: MIT
@@ -140,7 +140,9 @@ section name is yours (you can have several of the same type). All backends
 accept `enabled = false` to keep the config without probing it. Every backend
 also accepts `project_root` (point it at an existing checkout to reuse its
 `.git`/`.venv`) and `max_parallel` (queue-daemon cap on concurrent jobs, default
-`1`).
+`1`). `project_root` may be a single path (applied to every repo) or a table
+mapping repo slug -> path, with an optional `default` key as fallback, so one
+backend can serve several repos (each getting the right checkout).
 
 ```toml
 [policy]
@@ -159,6 +161,7 @@ poll_interval_s = 10.0       # scheduler tick: refresh running + place pending
 type = "local"
 # root = "$HOME/.omnirun"    # OMNIRUN_ROOT: where jobs live
 # project_root = "$HOME/proj"  # reuse an existing checkout's .git and .venv
+# project_root = { my-repo = "$HOME/proj" }  # or per-repo (by slug), else default path
 # max_parallel = 2           # queue daemon: run up to 2 jobs here at once
 
 # ---- any machine you can ssh into ----

{omnirun-0.2.0 → omnirun-0.2.2}/README.md

@@ -120,7 +120,9 @@ section name is yours (you can have several of the same type). All backends
 accept `enabled = false` to keep the config without probing it. Every backend
 also accepts `project_root` (point it at an existing checkout to reuse its
 `.git`/`.venv`) and `max_parallel` (queue-daemon cap on concurrent jobs, default
-`1`).
+`1`). `project_root` may be a single path (applied to every repo) or a table
+mapping repo slug -> path, with an optional `default` key as fallback, so one
+backend can serve several repos (each getting the right checkout).
 
 ```toml
 [policy]
@@ -139,6 +141,7 @@ poll_interval_s = 10.0       # scheduler tick: refresh running + place pending
 type = "local"
 # root = "$HOME/.omnirun"    # OMNIRUN_ROOT: where jobs live
 # project_root = "$HOME/proj"  # reuse an existing checkout's .git and .venv
+# project_root = { my-repo = "$HOME/proj" }  # or per-repo (by slug), else default path
 # max_parallel = 2           # queue daemon: run up to 2 jobs here at once
 
 # ---- any machine you can ssh into ----

{omnirun-0.2.0 → omnirun-0.2.2}/TESTING.md

@@ -116,15 +116,27 @@ bootstrap script — there is **no per-job dataset** anymore. This replaced a
 dataset-based design that hit a systematic **409** (Kaggle rejects a kernel
 referencing a still-processing dataset). Consequence: kaggle no longer creates or
 deletes any datasets; nothing worker-side to reap. Only caveat is bundle size — a
-code-only repo bundle is well under the embed cap; a repo with large committed
-blobs will be rejected client-side before push.
-
-**Newly added (unit-tested, NOT yet live-verified):** for a **public** repo the
-kernel now clones the repo directly over its own internet connection and no bundle
-is embedded (`repo.remote_clone_plan` gates on public + reachable). Kaggle also now
-injects a gitignored `<repo>/.env` (base64-embedded as `ENV_B64`, decoded to a 0600
-file and sourced) — parity with Colab. Both paths pass unit tests; a live Kaggle
-(and Colab) job against a public repo has not been re-run yet.
+code-only repo bundle is well under the source cap; a repo with large committed
+blobs is rejected client-side before push (see the measured cap below).
+
+**Public-repo direct clone — LIVE-VERIFIED (this session).** A CPU kernel was
+submitted against `github.com/flyingleafe/omnirun` (public) at the pushed master
+tip with `env=none`: the worker ran `git clone --bare` over its own connection,
+checked out the exact sha (`Preparing worktree (detached HEAD 6fcedf0)` →
+`CLONE OK cwd=…/.trees/6fcedf04a860`, `has_pyproject=True`), no bundle embedded,
+and `proof.txt` was pulled back. So a **public repo of any size** runs on Kaggle,
+unconstrained by the source cap. Kaggle also injects a gitignored `<repo>/.env`
+(base64 `ENV_B64` → 0600 file → sourced) — unit-tested, parity with Colab.
+
+**Kernel-source cap — MEASURED (this session).** Pushing `run.py` payloads of
+increasing size to the kernels API: `<=1 MiB` (1,048,570 B) **accepted**,
+`>=1.1 MiB` **rejected with HTTP 400** — so Kaggle's limit is **1 MiB**. The
+pre-submit guard (`KAGGLE_MAX_SOURCE_BYTES = 1 MiB`, override via
+`max_source_bytes`) now measures the full `run.py` (bootstrap + any bundle + any
+`.env`) against this and fails early naming size, instead of the old 40 MiB
+`MAX_EMBED_B64` guard that let 1–40 MiB pushes fail opaquely on Kaggle's side.
+(Aside: the current kaggle client exposes `kernels_delete(ref, no_confirm=True)`,
+used here to clean up the probe kernels — a future `gc`/`cancel` could use it.)
 
 - Provide: API token at `~/.config/kaggle/kaggle.json` (kaggle.com → Settings →
   Create New Token) or `KAGGLE_USERNAME`/`KAGGLE_KEY` env vars. The account
@@ -202,7 +214,8 @@ module.
 
 **Kaggle** (`src/omnirun/backends/kaggle.py`) — CONFIRMED LIVE (P100 job ran; output pulled)
 - [x] Bundle delivery (private/unpushed repo): **embedded base64 in `run.py`**, no dataset — sidesteps the 409 the old dataset design hit. `dataset_create_new`/`dataset_delete` are no longer used.
-- [ ] Public-repo **direct clone** (worker `git clone` over its own connection, no bundle) — unit-tested via `repo.remote_clone_plan`; not yet run live on Kaggle.
+- [x] Public-repo **direct clone** (worker `git clone` over its own connection, no bundle) — **LIVE-VERIFIED**: CPU job cloned `flyingleafe/omnirun` at master tip, ran in the worktree, `proof.txt` pulled.
+- [x] Kernel-source push cap — **MEASURED**: 1 MiB (≤1 MiB accepted, ≥1.1 MiB → HTTP 400); `KAGGLE_MAX_SOURCE_BYTES` guard set to match, overridable via `max_source_bytes`.
 - [ ] `.env` injection (`ENV_B64` embedded → 0600 file → sourced) — unit-tested; not yet run live on Kaggle.
 - [x] `kernels_status` response shape + status strings (`queued/running/complete/error/cancelAcknowledged`).
 - [x] `kernels_output` kwarg + downloaded log format (`<slug>.log`).

{omnirun-0.2.0 → omnirun-0.2.2}/flake.nix

@@ -71,6 +71,9 @@
 
           shellHook = ''
             ${pre-commit-check.shellHook}
+            # Guard against tagging a release without bumping the version.
+            # pre-commit owns only the pre-commit hook, so pre-push is ours.
+            ln -sf "$PWD/scripts/pre-push" "$(git rev-parse --git-path hooks)/pre-push"
             export LD_LIBRARY_PATH=${pkgs.lib.makeLibraryPath [ pkgs.stdenv.cc.cc.lib pkgs.zlib ]}:$LD_LIBRARY_PATH
             export UV_PYTHON=${python}/bin/python
             uv sync --quiet

{omnirun-0.2.0 → omnirun-0.2.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "omnirun"
-version = "0.2.0"
+version = "0.2.2"
 description = "Run jobs from your repo anywhere: Slurm over SSH, any SSH box, Kaggle, Colab, or marketplace GPUs"
 readme = "README.md"
 requires-python = ">=3.12"

omnirun-0.2.2/scripts/check-version.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# Verify the package version is internally consistent, and optionally that it
+# matches an expected version (e.g. a release tag).
+#
+#   scripts/check-version.sh            # pyproject.toml == src/omnirun/__init__.py
+#   scripts/check-version.sh v0.2.2     # ...and both equal 0.2.2
+#
+# Used by both the publish workflow and the pre-push hook so the rule lives in
+# one place.
+set -euo pipefail
+
+root="$(cd "$(dirname "$0")/.." && pwd)"
+
+extract() { grep -m1 -E "$2" "$root/$1" | sed -E 's/.*"([^"]+)".*/\1/'; }
+
+pyproject="$(extract pyproject.toml '^version *= *"')"
+init="$(extract src/omnirun/__init__.py '^__version__ *= *"')"
+
+if [ "$pyproject" != "$init" ]; then
+  echo "version mismatch: pyproject.toml=$pyproject src/omnirun/__init__.py=$init" >&2
+  exit 1
+fi
+
+if [ "$#" -ge 1 ]; then
+  expected="${1#v}"
+  if [ "$pyproject" != "$expected" ]; then
+    echo "version mismatch: expected $expected but package version is $pyproject" >&2
+    exit 1
+  fi
+fi
+
+echo "version OK: $pyproject"

omnirun-0.2.2/scripts/pre-push

@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+# Refuse to push a release tag whose commit does not declare a matching version.
+#
+# Installed as .git/hooks/pre-push by the devShell (see flake.nix). It inspects
+# the *pushed commit's* content (via `git show <sha>:path`), so it catches a
+# stale worktree too. CI (publish.yml) enforces the same rule server-side.
+set -euo pipefail
+
+version_at() { git show "$1:$2" | grep -m1 -E "$3" | sed -E 's/.*"([^"]+)".*/\1/'; }
+
+status=0
+while read -r _local_ref local_sha remote_ref _remote_sha; do
+  case "$remote_ref" in
+    refs/tags/v[0-9]*) ;;
+    *) continue ;;
+  esac
+  # Deleting a tag pushes the zero sha; nothing to check.
+  case "$local_sha" in
+    0000000000000000000000000000000000000000) continue ;;
+  esac
+
+  tag="${remote_ref#refs/tags/}"
+  pyproject="$(version_at "$local_sha" pyproject.toml '^version *= *"')"
+  init="$(version_at "$local_sha" src/omnirun/__init__.py '^__version__ *= *"')"
+
+  if [ "$tag" != "v$pyproject" ]; then
+    echo "pre-push: tag $tag != pyproject version $pyproject (commit ${local_sha:0:12})" >&2
+    status=1
+  fi
+  if [ "$pyproject" != "$init" ]; then
+    echo "pre-push: pyproject $pyproject != __init__ $init (commit ${local_sha:0:12})" >&2
+    status=1
+  fi
+done
+
+if [ "$status" -ne 0 ]; then
+  echo "pre-push: bump the version before tagging (see scripts/check-version.sh)" >&2
+fi
+exit "$status"

{omnirun-0.2.0 → omnirun-0.2.2}/src/omnirun/__init__.py

@@ -1,3 +1,3 @@
 """omnirun — run jobs from your repo anywhere."""
 
-__version__ = "0.2.0"
+__version__ = "0.2.2"

{omnirun-0.2.0 → omnirun-0.2.2}/src/omnirun/backends/colab.py

@@ -32,7 +32,7 @@ from datetime import datetime, timedelta, timezone
 from pathlib import Path
 from typing import Any
 
-from omnirun.backends import jobdir
+from omnirun.backends import jobdir, tarsafe
 from omnirun.backends.base import Backend, BackendError, register
 from omnirun.bootstrap import (
     HEARTBEAT_STALE_S,
@@ -360,7 +360,9 @@ class ColabBackend(Backend):
                 BootstrapParams(
                     omnirun_root=COLAB_ROOT,
                     project_root=jobdir.project_root_of(
-                        COLAB_ROOT, spec.repo.slug, self.config.project_root
+                        COLAB_ROOT,
+                        spec.repo.slug,
+                        self.config.project_root_for(spec.repo.slug),
                     ),
                     code=code,
                 ),
@@ -583,7 +585,7 @@ class ColabBackend(Backend):
             if not local_tar.exists():
                 raise BackendError("colab download produced no local file")
             with tarfile.open(local_tar) as tf:
-                tf.extractall(td, filter="data")
+                tarsafe.extract_all(tf, Path(td))
             outputs = Path(td) / "outputs"
             if outputs.is_dir():
                 shutil.copytree(outputs, dest, dirs_exist_ok=True)

{omnirun-0.2.0 → omnirun-0.2.2}/src/omnirun/backends/kaggle.py

@@ -34,7 +34,7 @@ from pathlib import Path
 from typing import Any
 
 from omnirun.backends.base import Backend, BackendError, register
-from omnirun.backends import jobdir
+from omnirun.backends import jobdir, tarsafe
 from omnirun.bootstrap import (
     BootstrapParams,
     CodeSource,
@@ -76,10 +76,15 @@ MAX_MEM_GB = 30
 MAX_DISK_GB = 55
 KAGGLE_ROOT = "/kaggle/tmp/omnirun"  # venv + tree scratch; /kaggle/working stays small
 RESULT_TAR = "omnirun-job.tar.gz"
-# Cap on the base64 git bundle embedded in the kernel source. Kaggle rejects
-# oversized kernel pushes; fail early with a clear message instead. Generous —
-# a code-only repo bundle is normally well under a megabyte.
-MAX_EMBED_B64 = 40 * 1024 * 1024
+# Cap on the kernel source (run.py) omnirun pushes: the embedded bootstrap +
+# any git bundle (private/unpushed repos) + any out-of-band .env all count
+# against it. Kaggle rejects an oversized push with an opaque HTTP 400, so we
+# guard at the real threshold and fail early naming size as the cause. Measured
+# live against the kernels API: <=1 MiB accepted, >=1.1 MiB rejected — i.e. the
+# limit is 1 MiB. A private/unpushed repo therefore only fits when its bundle is
+# small; a *public* repo is cloned by the worker directly (no bundle shipped) and
+# is unaffected by this cap. Override per backend with `max_source_bytes`.
+KAGGLE_MAX_SOURCE_BYTES = 1024 * 1024
 LOG_POLL_INTERVAL_S = 30.0  # poll etiquette: >=30s against the kaggle API
 
 
@@ -365,6 +370,29 @@ class KaggleBackend(Backend):
 
     # ---- submit ---------------------------------------------------------------
 
+    def _guard_source_size(self, harness: str, *, shipped_bundle: bool) -> None:
+        """Fail early, with a size-naming message, when the kernel source is over
+        Kaggle's real push limit — instead of letting the push fail opaquely."""
+        limit = int(self.config.extra("max_source_bytes", KAGGLE_MAX_SOURCE_BYTES))
+        size = len(harness.encode())
+        if size <= limit:
+            return
+        # the git bundle (full history + tree at the sha) usually dominates when
+        # a private/unpushed repo forces one; a public repo ships no bundle.
+        remedy = (
+            "push the repo to a public remote so the worker clones it directly "
+            "(no bundle shipped), slim the git history (shallow/squash) or exclude "
+            "heavy dirs from the tree, "
+            if shipped_bundle
+            else "slim the tree or "
+        )
+        raise BackendError(
+            f"kaggle kernel source is {size / (1024 * 1024):.1f} MB, over the "
+            f"~{limit / (1024 * 1024):.1f} MB Kaggle push limit (the push would "
+            f"fail opaquely) — {remedy}run this job on another backend, or raise "
+            "the backend's `max_source_bytes` if Kaggle accepts more"
+        )
+
     def submit(self, spec: JobSpec, offer: Offer) -> JobHandle:
         api = self._api()
         user = self._username(api)
@@ -401,31 +429,27 @@ class KaggleBackend(Backend):
             env_b64 = (
                 base64.b64encode(envf.read_text().encode()).decode() if envf else ""
             )
-            if len(bundle_b64) + len(env_b64) > MAX_EMBED_B64:
-                raise BackendError(
-                    f"embedded payload is too large for a kaggle kernel "
-                    f"(~{(len(bundle_b64) + len(env_b64)) // (1024 * 1024)}MB base64) "
-                    "— slim the git history (shallow/squash), push a public repo, or "
-                    "run this job on another backend"
-                )
 
             script = generate_bootstrap(
                 notebook_env_spec(spec),
                 BootstrapParams(
                     omnirun_root=KAGGLE_ROOT,
                     project_root=jobdir.project_root_of(
-                        KAGGLE_ROOT, spec.repo.slug, self.config.project_root
+                        KAGGLE_ROOT,
+                        spec.repo.slug,
+                        self.config.project_root_for(spec.repo.slug),
                     ),
                     code=code,
                 ),
             )
             boot_b64 = base64.b64encode(script.encode()).decode()
 
+            harness = _render_harness(job_id, boot_b64, bundle_b64, env_b64)
+            self._guard_source_size(harness, shipped_bundle=bool(bundle_b64))
+
             k_dir = stage / "kernel"
             k_dir.mkdir()
-            (k_dir / "run.py").write_text(
-                _render_harness(job_id, boot_b64, bundle_b64, env_b64)
-            )
+            (k_dir / "run.py").write_text(harness)
             meta: dict[str, Any] = {
                 "id": kernel_ref,
                 "title": slug,  # title must slugify to the id slug
@@ -610,7 +634,7 @@ class KaggleBackend(Backend):
             extract = Path(td) / "extract"
             extract.mkdir()
             with tarfile.open(tar) as tf:
-                tf.extractall(extract, filter="data")
+                tarsafe.extract_all(tf, extract)
             outputs = extract / "outputs"
             if outputs.is_dir():
                 shutil.copytree(outputs, dest, dirs_exist_ok=True)

{omnirun-0.2.0 → omnirun-0.2.2}/src/omnirun/backends/local.py

@@ -113,7 +113,10 @@ class LocalBackend(Backend):
     def submit(self, spec: JobSpec, offer: Offer) -> JobHandle:
         root = jobdir.remote_root(self.exec, self.config.root)
         project_root = jobdir.resolve_project_root(
-            self.exec, root, spec.repo.slug, self.config.project_root
+            self.exec,
+            root,
+            spec.repo.slug,
+            self.config.project_root_for(spec.repo.slug),
         )
         params = BootstrapParams(
             omnirun_root=root,

{omnirun-0.2.0 → omnirun-0.2.2}/src/omnirun/backends/marketplace.py

@@ -286,7 +286,7 @@ class MarketplaceBackend(Backend, ABC):
             self._wait_ssh(ex)
             root = jobdir.remote_root(ex, self.config.root)
             project_root = jobdir.resolve_project_root(
-                ex, root, spec.repo.slug, self.config.project_root
+                ex, root, spec.repo.slug, self.config.project_root_for(spec.repo.slug)
             )
             params = BootstrapParams(
                 omnirun_root=root,

{omnirun-0.2.0 → omnirun-0.2.2}/src/omnirun/backends/slurm.py

@@ -181,6 +181,8 @@ class SlurmBackend(Backend):
 
         reasons: list[str] = []
         notes: list[str] = []
+        if res.wants_gpu() and self.config.extra("has_gpus", True) is False:
+            reasons.append(f"backend {self.name} is CPU-only (has_gpus = false)")
         if res.wants_gpu() and res.gpu_type and self.config.gpu_map:
             if _gpu_map_lookup(self.config.gpu_map, res.gpu_type) is None:
                 reasons.append(
@@ -258,7 +260,7 @@ class SlurmBackend(Backend):
         root = self.config.root
         job_dir = jobdir.job_dir_of(root, spec.job_id)
         project_root = jobdir.project_root_of(
-            root, spec.repo.slug, self.config.project_root
+            root, spec.repo.slug, self.config.project_root_for(spec.repo.slug)
         )
         sbatch = render_sbatch(spec, self.config, job_dir, root)
         bootstrap = generate_bootstrap(
@@ -281,7 +283,7 @@ class SlurmBackend(Backend):
         ex = self.exec_
         root = jobdir.remote_root(ex, self.config.root)
         project_root = jobdir.resolve_project_root(
-            ex, root, spec.repo.slug, self.config.project_root
+            ex, root, spec.repo.slug, self.config.project_root_for(spec.repo.slug)
         )
         params = BootstrapParams(
             omnirun_root=root,

{omnirun-0.2.0 → omnirun-0.2.2}/src/omnirun/backends/ssh.py

@@ -217,7 +217,7 @@ class SshBackend(Backend):
         ex = self.exec_
         root = jobdir.remote_root(ex, self.config.root)
         project_root = jobdir.resolve_project_root(
-            ex, root, spec.repo.slug, self.config.project_root
+            ex, root, spec.repo.slug, self.config.project_root_for(spec.repo.slug)
         )
         params = BootstrapParams(
             omnirun_root=root,

omnirun-0.2.2/src/omnirun/backends/tarsafe.py

@@ -0,0 +1,38 @@
+"""Tolerant tar extraction for the notebook backends (kaggle, colab), which
+return a job's outputs as a tarball the client must unpack.
+
+Python >=3.12 defaults tar extraction to the PEP-706 ``data`` filter. That
+filter is right (it refuses path traversal, device nodes and links pointing
+outside the destination), but ``extractall`` aborts the *whole* archive the
+first time the filter rejects a member. The common offender is a symlink whose
+target is an absolute path — every Weights & Biases run leaves one at
+``wandb/*/logs/debug-core.log`` — so a single such link would strand every
+output of an otherwise-successful job.
+
+``extract_all`` applies the same ``data`` filter per member but skips (with a
+warning) any member the filter rejects, so the rest of the tree still lands.
+"""
+
+from __future__ import annotations
+
+import sys
+import tarfile
+from pathlib import Path
+
+
+def extract_all(tf: tarfile.TarFile, dest: Path) -> list[str]:
+    """Extract every member of ``tf`` under ``dest`` with the safe ``data``
+    filter, skipping (not aborting on) any member the filter rejects. Returns
+    the names of the skipped members (also warned to stderr)."""
+    skipped: list[str] = []
+    for member in tf.getmembers():
+        try:
+            tf.extract(member, dest, filter="data")
+        except tarfile.FilterError as e:
+            skipped.append(member.name)
+            print(
+                f"omnirun: warning: skipped unsafe archive entry "
+                f"{member.name!r} ({type(e).__name__})",
+                file=sys.stderr,
+            )
+    return skipped

{omnirun-0.2.0 → omnirun-0.2.2}/src/omnirun/backends/vast.py

@@ -37,23 +37,25 @@ OFFER_LIMIT = 5
 MIN_RELIABILITY = 0.95
 
 # normalized name -> vast gpu_name values (their catalog naming)
+# The REST /bundles/ endpoint matches gpu_name with spaces ("A100 SXM4");
+# underscore forms match nothing (the vast CLI translates them, the API doesn't).
 VAST_GPU_NAMES: dict[str, list[str]] = {
     "H200": ["H200"],
-    "H100": ["H100_SXM", "H100_PCIE", "H100_NVL"],
-    "A100": ["A100_SXM4", "A100_PCIE", "A100X"],
-    "A100-80": ["A100_SXM4", "A100_PCIE"],
-    "A6000": ["RTX_A6000"],
+    "H100": ["H100 SXM", "H100 PCIE", "H100 NVL"],
+    "A100": ["A100 SXM4", "A100 PCIE", "A100X"],
+    "A100-80": ["A100 SXM4", "A100 PCIE"],
+    "A6000": ["RTX A6000"],
     "L40": ["L40", "L40S"],
     "L4": ["L4"],
-    "V100": ["Tesla_V100"],
-    "4090": ["RTX_4090"],
-    "3090": ["RTX_3090"],
-    "5090": ["RTX_5090"],
+    "V100": ["Tesla V100"],
+    "4090": ["RTX 4090"],
+    "3090": ["RTX 3090"],
+    "5090": ["RTX 5090"],
 }
 
 
 def vast_gpu_names(gpu_type: str) -> list[str]:
-    return VAST_GPU_NAMES.get(gpu_type, [gpu_type.replace(" ", "_").replace("-", "_")])
+    return VAST_GPU_NAMES.get(gpu_type, [gpu_type.replace("_", " ").replace("-", " ")])
 
 
 def normalize_vast_gpu(name: str, gpu_ram_mb: float | None = None) -> str:
@@ -84,6 +86,10 @@ class VastBackend(MarketplaceBackend):
         }
         if res.gpu_type is not None:
             filt["gpu_name"] = {"in": vast_gpu_names(res.gpu_type)}
+            if res.gpu_type == "A100-80":
+                # cheapest-first + OFFER_LIMIT would otherwise fill the page
+                # with 40GB A100s that normalization then discards
+                filt["gpu_ram"] = {"gte": 70_000}
         elif res.min_vram_gb is not None:
             # vast reports gpu_ram in MB (verify live)
             filt["gpu_ram"] = {"gte": res.min_vram_gb * 1024}

{omnirun-0.2.0 → omnirun-0.2.2}/src/omnirun/bootstrap.py

@@ -188,6 +188,33 @@ def _indent(block: str, n: int) -> str:
     return "".join(pad + line + "\n" for line in block.splitlines())
 
 
+def _heredoc_delim(body: str) -> str:
+    """A heredoc terminator that provably does not occur as a line in `body`,
+    so embedding an arbitrary command in a heredoc can never terminate early."""
+    lines = body.splitlines()
+    base = "OMNIRUN_CMD_EOF"
+    delim = base
+    i = 0
+    while delim in lines:
+        i += 1
+        delim = f"{base}_{i}"
+    return delim
+
+
+def _command_block(command: str) -> str:
+    """Body of run_cmd(): the user command carried byte-exact via a single-quoted
+    heredoc and run with `eval`.
+
+    A single-quoted heredoc disables every expansion and preserves the body
+    literally — newlines, embedded heredocs, quotes, backslashes all survive — so
+    multi-line scripts reach the worker unmangled. `eval` runs it in the current
+    shell, so env activation, forwarded exports and pre-run lines all still apply.
+    Crucially the command is NOT indented: indenting would shift the user's own
+    heredoc terminators off column 0 and break them."""
+    delim = _heredoc_delim(command)
+    return f"eval \"$(cat <<'{delim}'\n{command}\n{delim}\n)\"\n"
+
+
 def _default_project_root(slug: str) -> str:
     return f"$OMNIRUN_ROOT/projects/{slug}"
 
@@ -307,8 +334,7 @@ status running
 STARTED_AT=$(now)
 set +e
 run_cmd() {{
-{_indent(spec.command, 2)}\
-}}
+{_command_block(spec.command)}}}
 # run in a subshell: a bare `exit N` in the user command must not kill
 # bootstrap.sh itself (result.json would never be written -> job shows LOST)
 ( run_cmd ) > >(tee "$JOB_DIR/logs/stdout.log") 2> >(tee "$JOB_DIR/logs/stderr.log" >&2)

{omnirun-0.2.0 → omnirun-0.2.2}/src/omnirun/config.py

@@ -48,8 +48,10 @@ class BackendConfig(BaseModel, extra="allow"):
     env_setup: list[str] = Field(default_factory=list)  # module loads, exports
     # Where a project's shared checkout + .venv live on the worker. Default
     # "$OMNIRUN_ROOT/projects/<slug>"; point at an existing clone to reuse it
-    # (its .git becomes the object store, its .venv the shared env).
-    project_root: str | None = None
+    # (its .git becomes the object store, its .venv the shared env). A str
+    # applies to every repo; a dict maps repo slug -> path (with an optional
+    # "default" key as fallback) so one backend can serve several repos.
+    project_root: str | dict[str, str] | None = None
 
     # static capability declaration (ssh backend; probe checks live via nvidia-smi)
     gpus: list[GpuDecl] = Field(default_factory=list)
@@ -74,6 +76,15 @@ class BackendConfig(BaseModel, extra="allow"):
     def extra(self, key: str, default: Any = None) -> Any:
         return (self.model_extra or {}).get(key, default)
 
+    def project_root_for(self, slug: str) -> str | None:
+        """Configured project root for a repo. A str applies to every repo; a
+        dict is keyed by slug, falling back to a "default" key, then to None
+        (the built-in "$OMNIRUN_ROOT/projects/<slug>" path)."""
+        pr = self.project_root
+        if isinstance(pr, dict):
+            return pr.get(slug) or pr.get("default")
+        return pr
+
 
 class DaemonConfig(BaseModel):
     host: str = "127.0.0.1"  # localhost socket; bind 0.0.0.0 + auth to go remote

{omnirun-0.2.0 → omnirun-0.2.2}/tests/test_bootstrap.py

@@ -133,6 +133,38 @@ def test_failing_command_propagates_exit_code(
     assert "about to fail" in (job_dir / "logs" / "stdout.log").read_text()
 
 
+def test_multiline_heredoc_command_runs_byte_exact(
+    sample_repo: Path, job_spec: JobSpec, tmp_path: Path
+) -> None:
+    """Regression (#3): a multi-line command with a heredoc must reach the worker
+    byte-identical. The old indent-per-line embedding shifted the heredoc
+    terminator off column 0 (breaking it) and prefixed every body line — this
+    proves the body is written verbatim, indentation and all."""
+    command = (
+        'cat <<EOF > "$OMNIRUN_OUTPUT/cfg.yaml"\n'
+        "key: value\n"
+        "  two-space-indent: kept\n"
+        "EOF\n"
+        'echo "MULTILINE OK"'
+    )
+    spec = job_spec.model_copy(
+        update={
+            "job_id": JobSpec.make_job_id("heredoc"),
+            "command": command,
+            "outputs": [],
+        }
+    )
+    root = tmp_path / "omnirun_root"
+    script = stage(sample_repo, spec, root)
+    proc = run_bootstrap(script)
+
+    job_dir = root / "jobs" / spec.job_id
+    assert proc.returncode == 0, (job_dir / "logs" / "bootstrap.log").read_text()
+    cfg = job_dir / "outputs" / "cfg.yaml"
+    assert cfg.read_text() == "key: value\n  two-space-indent: kept\n"
+    assert "MULTILINE OK" in (job_dir / "logs" / "stdout.log").read_text()
+
+
 def test_bare_exit_in_command_still_writes_result(
     sample_repo: Path, job_spec: JobSpec, tmp_path: Path
 ) -> None:

omnirun-0.2.2/tests/test_config.py

@@ -0,0 +1,37 @@
+"""BackendConfig helpers: project_root_for resolution (str / dict / None)."""
+
+from __future__ import annotations
+
+from omnirun.config import BackendConfig
+
+
+def make_config(**kw) -> BackendConfig:
+    return BackendConfig.model_validate({"type": "slurm", **kw})
+
+
+def test_project_root_for_none_is_default():
+    assert make_config().project_root_for("proj") is None
+
+
+def test_project_root_for_str_applies_to_every_repo():
+    c = make_config(project_root="$HOME/shared")
+    assert c.project_root_for("proj") == "$HOME/shared"
+    assert c.project_root_for("other") == "$HOME/shared"
+
+
+def test_project_root_for_dict_slug_hit():
+    c = make_config(project_root={"hns": "$HOME/projects/hns"})
+    assert c.project_root_for("hns") == "$HOME/projects/hns"
+
+
+def test_project_root_for_dict_no_match_falls_through_to_none():
+    c = make_config(project_root={"hns": "$HOME/projects/hns"})
+    assert c.project_root_for("auraflow") is None
+
+
+def test_project_root_for_dict_default_fallback():
+    c = make_config(
+        project_root={"hns": "$HOME/projects/hns", "default": "$HOME/managed"}
+    )
+    assert c.project_root_for("hns") == "$HOME/projects/hns"  # slug wins over default
+    assert c.project_root_for("auraflow") == "$HOME/managed"  # fallback

{omnirun-0.2.0 → omnirun-0.2.2}/tests/test_kaggle.py

@@ -496,6 +496,49 @@ def test_pull_outputs(fake_api, backend, tmp_path):
     assert files == [dest / "model.txt"]
 
 
+def test_pull_outputs_tolerates_absolute_symlink(fake_api, backend, tmp_path):
+    """Regression (#1): every W&B run leaves a symlink to an absolute path under
+    wandb/*/logs/. Python's data filter rejects it; pull must skip that one entry
+    and still recover the real outputs rather than aborting the whole archive."""
+    buf = io.BytesIO()
+    with tarfile.open(fileobj=buf, mode="w:gz") as tf:
+        data = b"metric,value\n1,2\n"
+        good = tarfile.TarInfo("outputs/results.csv")
+        good.size = len(data)
+        tf.addfile(good, io.BytesIO(data))
+        link = tarfile.TarInfo("outputs/wandb/run-x/logs/debug-core.log")
+        link.type = tarfile.SYMTYPE
+        link.linkname = "/kaggle/tmp/omnirun/abs/debug-core.log"  # absolute target
+        tf.addfile(link)
+    fake_api.output_files = {"omnirun-job.tar.gz": buf.getvalue()}
+    dest = tmp_path / "results"
+    files = backend.pull_outputs(make_handle(), dest)
+    assert (dest / "results.csv").read_bytes() == data
+    assert [p.name for p in files] == ["results.csv"]  # link skipped, csv kept
+
+
+def test_submit_guards_oversized_source(fake_api, monkeypatch):
+    """Regression (#2): an oversized kernel source must fail early with a
+    size-naming message, not sail past the guard and fail opaquely on Kaggle."""
+    monkeypatch.setattr(
+        kaggle_mod, "_create_bundle", lambda root, sha, dest: _big_bundle(dest)
+    )
+    backend = KaggleBackend(
+        "kaggle",
+        BackendConfig.model_validate({"type": "kaggle", "max_source_bytes": 4096}),
+    )
+    spec = make_spec(gpu_type="P100")
+    offer = backend.probe(spec.resources)[0]
+    with pytest.raises(BackendError, match=r"kaggle kernel source is .* over the"):
+        backend.submit(spec, offer)
+
+
+def _big_bundle(dest: Path) -> Path:
+    dest = Path(dest)
+    dest.write_bytes(b"x" * 8192)  # base64 ~11k, over the 4k test limit
+    return dest
+
+
 def test_cancel_without_api_support_points_at_website(fake_api, backend):
     with pytest.raises(BackendError, match="kaggle.com"):
         backend.cancel(make_handle())

{omnirun-0.2.0 → omnirun-0.2.2}/tests/test_marketplaces.py

@@ -474,7 +474,7 @@ def test_vast_probe_filter_and_parsing():
 
     body = json.loads(route.calls[0].request.content)
     assert body["type"] == "ondemand"
-    assert body["gpu_name"] == {"in": ["H100_SXM", "H100_PCIE", "H100_NVL"]}
+    assert body["gpu_name"] == {"in": ["H100 SXM", "H100 PCIE", "H100 NVL"]}
     assert body["num_gpus"] == {"eq": 1}
     assert body["rentable"] == {"eq": True}
     assert body["verified"] == {"eq": True}

{omnirun-0.2.0 → omnirun-0.2.2}/tests/test_slurm.py

@@ -295,6 +295,22 @@ def test_render_payload_sbatch_then_bootstrap():
     assert fake.files == {}
 
 
+def test_render_payload_project_root_dict_matches_slug():
+    # spec slug is "proj": a per-repo dict resolves to that repo's checkout...
+    fake = FakeExec()
+    b = make_backend(fake, root=ROOT, project_root={"proj": "/data/existing/proj"})
+    payload = b.render_payload(make_spec(), offer=None)
+    assert 'PROJECT_ROOT="/data/existing/proj"' in payload
+
+
+def test_render_payload_project_root_dict_no_match_uses_default():
+    # ...and an unrelated slug falls through to the built-in "$root/projects/<slug>".
+    fake = FakeExec()
+    b = make_backend(fake, root=ROOT, project_root={"other": "/data/existing/other"})
+    payload = b.render_payload(make_spec(), offer=None)
+    assert f'PROJECT_ROOT="{ROOT}/projects/proj"' in payload
+
+
 # --- status mapping -------------------------------------------------------------------
 
 
@@ -515,6 +531,16 @@ def test_probe_warns_when_type_requested_but_no_map():
     assert "gpu_map" in offer.notes
 
 
+def test_probe_unfit_for_gpu_job_when_cpu_only():
+    fake = FakeExec()
+    (offer,) = make_backend(fake, has_gpus=False).probe(ResourceSpec(gpus=1))
+    assert not offer.fits
+    assert "CPU-only" in offer.unfit_reasons[0]
+    # CPU jobs still fit
+    (offer,) = make_backend(fake, has_gpus=False).probe(ResourceSpec())
+    assert offer.fits
+
+
 def test_probe_wait_tier1_idle_nodes():
     fake = FakeExec()
     fake.add(r"sinfo -p gpu -t idle", stdout="node01 gpu:a100:4\nnode02 (null)\n")

omnirun-0.2.2/tests/test_tarsafe.py

@@ -0,0 +1,53 @@
+"""Tolerant tar extraction (shared by the kaggle + colab pull paths)."""
+
+from __future__ import annotations
+
+import io
+import os
+import tarfile
+from pathlib import Path
+
+from omnirun.backends import tarsafe
+
+
+def _tar_with_abs_symlink(dest_target: str) -> io.BytesIO:
+    buf = io.BytesIO()
+    with tarfile.open(fileobj=buf, mode="w:gz") as tf:
+        data = b"real output\n"
+        good = tarfile.TarInfo("outputs/results.txt")
+        good.size = len(data)
+        tf.addfile(good, io.BytesIO(data))
+        link = tarfile.TarInfo("outputs/wandb/run-x/logs/debug-core.log")
+        link.type = tarfile.SYMTYPE
+        link.linkname = dest_target
+        tf.addfile(link)
+    buf.seek(0)
+    return buf
+
+
+def test_extract_all_skips_absolute_symlink_and_keeps_rest(tmp_path: Path) -> None:
+    # every W&B run leaves such a link; the default data filter would abort here.
+    buf = _tar_with_abs_symlink("/etc/hostname")
+    with tarfile.open(fileobj=buf) as tf:
+        skipped = tarsafe.extract_all(tf, tmp_path)
+    assert skipped == ["outputs/wandb/run-x/logs/debug-core.log"]
+    assert (tmp_path / "outputs" / "results.txt").read_bytes() == b"real output\n"
+    assert not (tmp_path / "outputs" / "wandb" / "run-x" / "logs").exists()
+
+
+def test_extract_all_extracts_normal_tree_completely(tmp_path: Path) -> None:
+    src = tmp_path / "src"
+    (src / "sub").mkdir(parents=True)
+    (src / "a.txt").write_text("a")
+    (src / "sub" / "b.txt").write_text("b")
+    os.symlink("a.txt", src / "rel")  # a *relative* symlink is allowed by data filter
+    tar = tmp_path / "t.tar.gz"
+    with tarfile.open(tar, "w:gz") as tf:
+        tf.add(src, arcname="outputs")
+    out = tmp_path / "out"
+    with tarfile.open(tar) as tf:
+        skipped = tarsafe.extract_all(tf, out)
+    assert skipped == []
+    assert (out / "outputs" / "a.txt").read_text() == "a"
+    assert (out / "outputs" / "sub" / "b.txt").read_text() == "b"
+    assert (out / "outputs" / "rel").is_symlink()