alloc 0.0.1__tar.gz → 0.0.3__tar.gz

alloc-0.0.3/PKG-INFO

@@ -0,0 +1,190 @@
+Metadata-Version: 2.4
+Name: alloc
+Version: 0.0.3
+Summary: Engineer-first training calibration: estimate VRAM fit, profile short runs, and pick GPU configs under real budget constraints.
+Author-email: Alloc Labs <hello@alloclabs.com>
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://alloclabs.com
+Project-URL: Repository, https://github.com/alloc-labs/alloc
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: typer>=0.9.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: httpx>=0.24.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: pynvml>=11.5.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+
+# alloc
+
+**Find and fix training bottlenecks. Zero code changes.**
+
+[![PyPI](https://img.shields.io/pypi/v/alloc)](https://pypi.org/project/alloc/)
+[![Python](https://img.shields.io/pypi/pyversions/alloc)](https://pypi.org/project/alloc/)
+[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](LICENSE)
+
+```bash
+pip install alloc
+alloc run python train.py
+```
+
+```
+alloc v0.0.2 — Calibrate
+
+ Run Summary
+  Peak VRAM       31.2 GB / 40.0 GB (A100)
+  VRAM used       78.0%
+  Avg GPU util    72.3%
+  Avg power       287 W
+  Duration        24.1s (auto-stopped: metrics stable at 18.2s)
+  Step time       148.5 ms (p50) / 152.1 ms (p90)
+  Throughput      42.3 samples/sec
+
+  Artifact: alloc_artifact.json.gz
+```
+
+That's it. No decorators, no config files, no code changes. Alloc wraps your command, profiles GPU usage, and tells you what's wrong.
+
+---
+
+## What you get
+
+**`alloc diagnose`** reads your training script and tells you exactly what to change:
+
+```bash
+alloc diagnose train.py
+```
+```
+alloc diagnose — 3 findings in train.py
+
+ CRITICAL  DL005 — DataLoader running in main thread
+   train.py:47   num_workers=0 → num_workers=8
+   num_workers=0 loads data in the main thread, blocking GPU computation entirely.
+   Expected impact: ~30-50% faster training with parallel data loading
+
+ WARNING   PREC002 — Using fp16, consider bf16
+   train.py:56   dtype: float16 → dtype: bfloat16
+   H100 supports bf16 natively — eliminates loss scaling overhead.
+   Expected impact: ~5-10% speedup, eliminates GradScaler complexity
+
+ INFO      THRU001 — cudnn.benchmark not enabled
+   Add: torch.backends.cudnn.benchmark = True
+   Expected impact: ~5-10% speedup for fixed-size inputs
+
+Summary: 1 critical, 1 warning, 1 info
+Run with --diff to generate patches | --json for CI output
+```
+
+**`alloc ghost`** estimates VRAM before you launch:
+
+```bash
+alloc ghost train.py --dtype bf16
+```
+```
+ Ghost Scan — 7.0B params (bf16)
+
+  Model weights       13.04 GB
+  Gradients           13.04 GB
+  Optimizer (Adam)    78.23 GB
+  Activations (est.)   0.50 GB
+  Buffer (10%)        10.48 GB
+
+  Total VRAM         115.28 GB
+```
+
+**`alloc scan`** ranks GPU configs without a GPU:
+
+```bash
+alloc scan --model llama-3-70b --gpu H100-80GB --num-gpus 8
+```
+
+---
+
+## Works with everything
+
+Alloc wraps your launch command. No framework-specific setup required.
+
+```bash
+alloc run python train.py
+alloc run torchrun --nproc_per_node=4 train.py
+alloc run accelerate launch train.py
+alloc run srun python train.py           # Slurm
+alloc run ray job submit -- python train.py
+```
+
+Multi-GPU detection is automatic (discovers all GPUs in the process tree).
+
+---
+
+## Deeper signals (optional)
+
+Add a one-line callback for step-level timing:
+
+```python
+# HuggingFace
+from alloc import HuggingFaceCallback
+trainer = Trainer(..., callbacks=[HuggingFaceCallback()])
+
+# Lightning
+from alloc import LightningCallback
+trainer = Trainer(..., callbacks=[LightningCallback()])
+```
+
+This unlocks step time p50/p90, throughput, and dataloader bottleneck detection.
+
+---
+
+## All commands
+
+| Command | What it does |
+|---------|-------------|
+| `alloc run <cmd>` | Profile a training run (auto-stops when stable) |
+| `alloc diagnose <script>` | AST analysis with specific fix suggestions |
+| `alloc ghost <script>` | Estimate VRAM before launching |
+| `alloc scan --model <name>` | Rank GPU configs remotely (no GPU needed) |
+| `alloc catalog list` | Browse 13 GPUs with specs and pricing |
+| `alloc init` | Configure GPU fleet and budget (`.alloc.yaml`) |
+| `alloc login` | Authenticate for dashboard + auto-upload |
+
+Every command supports `--json` for CI/CD integration.
+
+---
+
+## Dashboard
+
+Log in to get team visibility, budget tracking, and optimization proposals:
+
+```bash
+alloc login --browser
+alloc run python train.py    # auto-uploads when logged in
+```
+
+Dashboard at [alloclabs.com](https://www.alloclabs.com)
+
+---
+
+## Design principles
+
+1. **Zero config** — `alloc run python train.py` works out of the box
+2. **Never crash training** — all Alloc failures are caught silently
+3. **No monkey-patching** — external monitoring only, deeper signals opt-in
+4. **Local-first** — works in air-gapped environments, no internet required
+
+---
+
+## Links
+
+- [Website](https://www.alloclabs.com)
+- [Documentation](https://www.alloclabs.com/docs)
+- [PyPI](https://pypi.org/project/alloc/)

alloc-0.0.3/README.md

@@ -0,0 +1,162 @@
+# alloc
+
+**Find and fix training bottlenecks. Zero code changes.**
+
+[![PyPI](https://img.shields.io/pypi/v/alloc)](https://pypi.org/project/alloc/)
+[![Python](https://img.shields.io/pypi/pyversions/alloc)](https://pypi.org/project/alloc/)
+[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](LICENSE)
+
+```bash
+pip install alloc
+alloc run python train.py
+```
+
+```
+alloc v0.0.2 — Calibrate
+
+ Run Summary
+  Peak VRAM       31.2 GB / 40.0 GB (A100)
+  VRAM used       78.0%
+  Avg GPU util    72.3%
+  Avg power       287 W
+  Duration        24.1s (auto-stopped: metrics stable at 18.2s)
+  Step time       148.5 ms (p50) / 152.1 ms (p90)
+  Throughput      42.3 samples/sec
+
+  Artifact: alloc_artifact.json.gz
+```
+
+That's it. No decorators, no config files, no code changes. Alloc wraps your command, profiles GPU usage, and tells you what's wrong.
+
+---
+
+## What you get
+
+**`alloc diagnose`** reads your training script and tells you exactly what to change:
+
+```bash
+alloc diagnose train.py
+```
+```
+alloc diagnose — 3 findings in train.py
+
+ CRITICAL  DL005 — DataLoader running in main thread
+   train.py:47   num_workers=0 → num_workers=8
+   num_workers=0 loads data in the main thread, blocking GPU computation entirely.
+   Expected impact: ~30-50% faster training with parallel data loading
+
+ WARNING   PREC002 — Using fp16, consider bf16
+   train.py:56   dtype: float16 → dtype: bfloat16
+   H100 supports bf16 natively — eliminates loss scaling overhead.
+   Expected impact: ~5-10% speedup, eliminates GradScaler complexity
+
+ INFO      THRU001 — cudnn.benchmark not enabled
+   Add: torch.backends.cudnn.benchmark = True
+   Expected impact: ~5-10% speedup for fixed-size inputs
+
+Summary: 1 critical, 1 warning, 1 info
+Run with --diff to generate patches | --json for CI output
+```
+
+**`alloc ghost`** estimates VRAM before you launch:
+
+```bash
+alloc ghost train.py --dtype bf16
+```
+```
+ Ghost Scan — 7.0B params (bf16)
+
+  Model weights       13.04 GB
+  Gradients           13.04 GB
+  Optimizer (Adam)    78.23 GB
+  Activations (est.)   0.50 GB
+  Buffer (10%)        10.48 GB
+
+  Total VRAM         115.28 GB
+```
+
+**`alloc scan`** ranks GPU configs without a GPU:
+
+```bash
+alloc scan --model llama-3-70b --gpu H100-80GB --num-gpus 8
+```
+
+---
+
+## Works with everything
+
+Alloc wraps your launch command. No framework-specific setup required.
+
+```bash
+alloc run python train.py
+alloc run torchrun --nproc_per_node=4 train.py
+alloc run accelerate launch train.py
+alloc run srun python train.py           # Slurm
+alloc run ray job submit -- python train.py
+```
+
+Multi-GPU detection is automatic (discovers all GPUs in the process tree).
+
+---
+
+## Deeper signals (optional)
+
+Add a one-line callback for step-level timing:
+
+```python
+# HuggingFace
+from alloc import HuggingFaceCallback
+trainer = Trainer(..., callbacks=[HuggingFaceCallback()])
+
+# Lightning
+from alloc import LightningCallback
+trainer = Trainer(..., callbacks=[LightningCallback()])
+```
+
+This unlocks step time p50/p90, throughput, and dataloader bottleneck detection.
+
+---
+
+## All commands
+
+| Command | What it does |
+|---------|-------------|
+| `alloc run <cmd>` | Profile a training run (auto-stops when stable) |
+| `alloc diagnose <script>` | AST analysis with specific fix suggestions |
+| `alloc ghost <script>` | Estimate VRAM before launching |
+| `alloc scan --model <name>` | Rank GPU configs remotely (no GPU needed) |
+| `alloc catalog list` | Browse 13 GPUs with specs and pricing |
+| `alloc init` | Configure GPU fleet and budget (`.alloc.yaml`) |
+| `alloc login` | Authenticate for dashboard + auto-upload |
+
+Every command supports `--json` for CI/CD integration.
+
+---
+
+## Dashboard
+
+Log in to get team visibility, budget tracking, and optimization proposals:
+
+```bash
+alloc login --browser
+alloc run python train.py    # auto-uploads when logged in
+```
+
+Dashboard at [alloclabs.com](https://www.alloclabs.com)
+
+---
+
+## Design principles
+
+1. **Zero config** — `alloc run python train.py` works out of the box
+2. **Never crash training** — all Alloc failures are caught silently
+3. **No monkey-patching** — external monitoring only, deeper signals opt-in
+4. **Local-first** — works in air-gapped environments, no internet required
+
+---
+
+## Links
+
+- [Website](https://www.alloclabs.com)
+- [Documentation](https://www.alloclabs.com/docs)
+- [PyPI](https://pypi.org/project/alloc/)

{alloc-0.0.1 → alloc-0.0.3}/pyproject.toml

@@ -4,18 +4,17 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "alloc"
-version = "0.0.1"
+version = "0.0.3"
 description = "Engineer-first training calibration: estimate VRAM fit, profile short runs, and pick GPU configs under real budget constraints."
 readme = "README.md"
 license = "Apache-2.0"
-requires-python = ">=3.8"
+requires-python = ">=3.9"
 authors = [{name = "Alloc Labs", email = "hello@alloclabs.com"}]
 classifiers = [
     "Development Status :: 3 - Alpha",
     "Intended Audience :: Developers",
     "Topic :: Scientific/Engineering :: Artificial Intelligence",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
@@ -27,10 +26,10 @@ dependencies = [
     "httpx>=0.24.0",
     "pydantic>=2.0.0",
     "pyyaml>=6.0",
+    "pynvml>=11.5.0",
 ]
 
 [project.optional-dependencies]
-gpu = ["pynvml>=11.5.0"]
 dev = ["pytest>=7.0.0", "pytest-cov>=4.0.0"]
 
 [project.scripts]

{alloc-0.0.1 → alloc-0.0.3}/src/alloc/__init__.py

@@ -2,7 +2,7 @@
 
 from __future__ import annotations
 
-__version__ = "0.0.1"
+__version__ = "0.0.3"
 
 from alloc.ghost import ghost, GhostReport
 from alloc.callbacks import AllocCallback as HuggingFaceCallback

alloc-0.0.3/src/alloc/artifact_loader.py

@@ -0,0 +1,179 @@
+"""Artifact loader — parse alloc_artifact.json.gz for runtime-enhanced diagnosis.
+
+Loads the artifact created by `alloc run`, extracting GPU metrics, timing data,
+and per-rank distributed information for use by Phase 2 diagnosis rules.
+
+Never crashes. Returns None on any failure.
+"""
+
+from __future__ import annotations
+
+import glob
+import gzip
+import json
+import os
+from dataclasses import dataclass, field
+from typing import Dict, List, Optional
+
+
+@dataclass
+class ArtifactData:
+    """Parsed runtime artifact — structured for rule consumption."""
+
+    # Hardware (from probe)
+    gpu_name: Optional[str] = None
+    gpu_count: int = 1
+    per_gpu_vram_total_mb: Optional[float] = None
+    per_gpu_vram_used_mb: Optional[List[float]] = None
+    gpu_utilization_pct: Optional[List[float]] = None
+    power_draw_w: Optional[List[float]] = None
+    sm_version: Optional[str] = None
+    interconnect: Optional[str] = None
+
+    # Timing (from callbacks — may be None)
+    step_times_ms: Optional[List[float]] = None
+    step_time_p50_ms: Optional[float] = None
+    step_time_p90_ms: Optional[float] = None
+    throughput_samples_per_sec: Optional[float] = None
+    dataloader_wait_pct: Optional[float] = None
+
+    # Per-rank data (from distributed callbacks)
+    per_rank_peak_vram_mb: Optional[List[float]] = None
+    per_rank_step_times_ms: Optional[List[List[float]]] = None
+
+    # Run metadata
+    exit_code: Optional[int] = None
+    duration_s: Optional[float] = None
+    command: Optional[str] = None
+    git_sha: Optional[str] = None
+    is_oom: bool = False
+
+    # Aggregate metrics (computed from samples)
+    avg_gpu_util: Optional[float] = None
+    peak_vram_mb: Optional[float] = None
+
+
+def load_artifact(path: str) -> Optional[ArtifactData]:
+    """Load and parse alloc_artifact.json.gz. Returns None on any failure."""
+    try:
+        if path.endswith(".gz"):
+            with gzip.open(path, "rt", encoding="utf-8") as f:
+                raw = json.load(f)
+        else:
+            with open(path, "r", encoding="utf-8") as f:
+                raw = json.load(f)
+    except Exception:
+        return None
+
+    return _parse_artifact(raw)
+
+
+def find_artifact(directory: str = ".") -> Optional[str]:
+    """Find most recent alloc_artifact*.json.gz in directory. Returns path or None."""
+    patterns = [
+        os.path.join(directory, "alloc_artifact*.json.gz"),
+        os.path.join(directory, "alloc_artifact*.json"),
+    ]
+    candidates = []  # type: List[str]
+    for pattern in patterns:
+        candidates.extend(glob.glob(pattern))
+
+    if not candidates:
+        return None
+
+    # Sort by modification time, return newest
+    return max(candidates, key=os.path.getmtime)
+
+
+def _parse_artifact(raw: dict) -> ArtifactData:
+    """Parse raw artifact JSON into ArtifactData."""
+    data = ArtifactData()
+
+    probe = raw.get("probe") or {}
+    hardware = raw.get("hardware") or {}
+    context = raw.get("context") or {}
+
+    # Hardware
+    data.gpu_name = hardware.get("gpu_name") or probe.get("gpu_name")
+    gpu_count = hardware.get("num_gpus_detected")
+    data.gpu_count = gpu_count if gpu_count is not None and gpu_count > 0 else 1
+    data.per_gpu_vram_total_mb = _float_or_none(
+        hardware.get("gpu_total_vram_mb") or probe.get("gpu_total_vram_mb")
+    )
+    data.sm_version = hardware.get("sm_version")
+    data.interconnect = probe.get("interconnect_type")
+
+    # Peak VRAM — from probe samples or direct field
+    peak = _float_or_none(probe.get("peak_vram_mb"))
+    data.peak_vram_mb = peak
+
+    # Per-GPU VRAM: use per_rank_peak_vram_mb if available, else single peak
+    per_rank = probe.get("per_rank_peak_vram_mb")
+    if isinstance(per_rank, list) and per_rank:
+        data.per_gpu_vram_used_mb = [float(v) for v in per_rank]
+    elif peak is not None:
+        data.per_gpu_vram_used_mb = [peak]
+
+    data.per_rank_peak_vram_mb = data.per_gpu_vram_used_mb
+
+    # GPU utilization from samples
+    samples = probe.get("samples") or []
+    if samples:
+        utils = [s.get("gpu_util_pct") for s in samples if s.get("gpu_util_pct") is not None]
+        if utils:
+            data.gpu_utilization_pct = [float(u) for u in utils]
+            data.avg_gpu_util = sum(data.gpu_utilization_pct) / len(data.gpu_utilization_pct)
+
+        powers = [s.get("power_w") for s in samples if s.get("power_w") is not None]
+        if powers:
+            data.power_draw_w = [float(p) for p in powers]
+
+    # Avg GPU util from probe aggregate
+    if data.avg_gpu_util is None:
+        data.avg_gpu_util = _float_or_none(probe.get("avg_gpu_util"))
+
+    # Timing (from callback sidecar data merged into probe)
+    data.step_time_p50_ms = _float_or_none(probe.get("step_time_ms_p50"))
+    data.step_time_p90_ms = _float_or_none(probe.get("step_time_ms_p90"))
+    data.throughput_samples_per_sec = _float_or_none(probe.get("samples_per_sec"))
+    data.dataloader_wait_pct = _float_or_none(probe.get("dataloader_wait_pct"))
+
+    # Run metadata
+    data.exit_code = probe.get("exit_code")
+    data.duration_s = _float_or_none(probe.get("duration_seconds"))
+    data.command = probe.get("command")
+
+    # Git SHA from context
+    git_ctx = context.get("git") or {}
+    data.git_sha = git_ctx.get("commit_sha")
+
+    # OOM detection: exit_code != 0 AND VRAM utilization > 95%
+    data.is_oom = _detect_oom(data)
+
+    return data
+
+
+def _detect_oom(data: ArtifactData) -> bool:
+    """Detect probable OOM from exit code and VRAM utilization."""
+    if data.exit_code is None or data.exit_code == 0:
+        return False
+
+    if data.per_gpu_vram_used_mb and data.per_gpu_vram_total_mb:
+        total = data.per_gpu_vram_total_mb
+        if total > 0:
+            max_used = max(data.per_gpu_vram_used_mb)
+            utilization = max_used / total
+            if utilization > 0.95:
+                return True
+
+    return False
+
+
+def _float_or_none(val) -> Optional[float]:
+    """Convert a value to float, returning None on failure."""
+    if val is None:
+        return None
+    try:
+        return float(val)
+    except (ValueError, TypeError):
+        return None