opencode-skills-collection 3.1.12 → 3.1.13

package/bundled-skills/huggingface-spaces/references/known-errors.md

@@ -0,0 +1,232 @@
+# Known errors
+
+Check if this is a known issue before trying your own fix. Entries are keyed by the substring that actually appears in `runtime.errorMessage`, the build log, or a Python traceback — grep this file for the error you saw.
+
+If you hit something not listed here and figure out a fix, please ask your human to PR it back so future runs benefit.
+
+---
+
+## Build / config errors
+
+These come from the Space build pipeline before the app starts. Read with `hf spaces logs <id> --build --tail 500` — find the **first** error, not the last.
+
+### `CONFIG_ERROR: torch version in requirements.txt is not compatible with ZeroGPU`
+
+**Cause**: `requirements.txt` pins `torch==X.Y.Z` to a version outside the supported set (`2.8.0`, `2.9.1`, `2.10.0`, `2.11.0`).
+**Fix**: Unpin torch (preferred — the runtime preinstalls the latest supported version), or pin to one of the supported values.
+
+### `Cannot install … because these package versions have conflicting dependencies` / `ResolutionImpossible`
+
+**Cause**: A dep conflicts with the Gradio SDK pinned by `sdk_version:` in README. Most commonly `pydantic`, `uvicorn`, `huggingface_hub`, or `jinja2` pinned to old values that the SDK no longer accepts.
+**Fix**: Unpin the offender. For `gradio[mcp]` specifically, `uvicorn>=0.31.1` and `pydantic>=2.11.10` are required.
+
+### Build hangs in dependency resolution > 10 min
+
+**Cause**: pip backtracking through a deep version space.
+**Fix**: Pin the conflicting transitive dep. The `--build` logs will show which one. Bump `startup_duration_timeout: 1h` in README frontmatter if heavy downloads are expected.
+
+### `ModuleNotFoundError: No module named 'pkg_resources'`
+
+**Cause**: setuptools 81 dropped `pkg_resources`; an old package's `setup.py` imports it.
+**Fix**: Bump or unpin the offender. Typical culprits: `deepspeed==0.15.x` (training-only — usually safe to drop from inference Spaces), `openai-whisper==20231117`.
+
+### `400 Bad Request` from `/api/validate-yaml` during `create_repo` / `upload_file`
+
+**Cause**: README frontmatter failed server validation. Most common: `short_description` over the (undocumented) character cap — target ≤ 60.
+**Fix**: Shorten `short_description`. Long descriptions go in the README body. Also double-check `colorFrom`/`colorTo` are one of `red|yellow|green|blue|indigo|purple|pink|gray`.
+
+### `403 Forbidden` from `create_repo` with `space_hardware="zero-a10g"`
+
+**Cause**: The user isn't on PRO / Team / Enterprise so the API rejects ZeroGPU at creation.
+**Fix**: Retry without `space_hardware=`. Keep `hardware:` out of README frontmatter (silently ignored anyway). The Space is created on CPU; point the user at PRO upgrade or [community grant](grants.md).
+
+### `403 Forbidden` from `create_commit(..., create_pr=True)`
+
+**Cause**: Upstream Space has Discussions disabled.
+**Fix**: Ask the maintainer to enable Discussions, or push directly if you have write access.
+
+---
+
+## Startup / RUNTIME_ERROR
+
+These come from `hf spaces logs <id> --tail 500`.
+
+### `RuntimeError: CUDA has been initialized before importing the spaces package`
+
+**Cause**: Something triggered CUDA init in the main process before `import spaces`. Usually wrong import order; sometimes a third-party lib eagerly initializing CUDA at import time (e.g. `numba.cuda`).
+**Fix**: Reorder so `import spaces` is first. For numba-using stacks (NeMo, RAPIDS bits):
+```python
+import os
+os.environ.setdefault("NUMBA_DISABLE_CUDA", "1")
+import spaces
+```
+
+### `RuntimeError: No @spaces.GPU function detected during startup`
+
+**Cause**: The function bound to `.click(fn=...)` / `.submit(...)` isn't decorated. Decorating an inner helper doesn't count — the startup scan only walks Gradio's registered handlers.
+**Fix**: Decorate the function Gradio binds. If that conflicts with another decorator, wrap explicitly:
+```python
+@spaces.GPU(duration=60)
+def gpu_inner(...): ...
+def gradio_handler(...): return gpu_inner(...)
+```
+(Or just decorate `gradio_handler` directly.)
+
+### `ImportError: cannot import name 'HfFolder' from 'huggingface_hub'`
+
+**Cause**: Old gradio (`4.44` and similar) imports `HfFolder` from `huggingface_hub`, which was removed in recent hub releases.
+**Fix**: Two options.
+- Pin `huggingface-hub==0.25.0` in `requirements.txt` (keeps old gradio happy).
+- Bump `sdk_version` in README to `5.x` or `6.x` (also fixes a lot of other API breaks).
+If a Gradio custom component locks the major (`gradio-image-prompter`, `gradio_litmodel3d`, …), install it with `--no-deps` so its `gradio<5.0` requirement doesn't bind.
+
+### `ImportError: cannot import name 'is_traceable_wrapper_subclass' from 'torch.utils._python_dispatch'`
+
+**Cause**: A dep with `torchaudio<2.1` / `torch<2` in its `setup.py` (e.g. `demucs`, `audiocraft`) downgraded torch silently. The build succeeded, the app booted, and `import spaces` then died on a missing torch symbol.
+**Fix**: Install the offender from `app.py` with `--no-deps` *before* `import spaces`:
+```python
+import subprocess, sys
+subprocess.run([sys.executable, "-m", "pip", "install", "--no-deps",
+                "git+https://github.com/facebookresearch/demucs"], check=True)
+import spaces
+```
+List its actual runtime deps (`dora-search einops julius lameenc openunmix pyyaml tqdm` for demucs) yourself in `requirements.txt`.
+
+### `_pickle.UnpicklingError: Weights only load failed`
+
+**Cause**: `torch.load` weights-only default flipped to `True` in torch 2.6. Old checkpoints pickling numpy/object globals fail.
+**Fix**: For trusted upstream checkpoints, monkey-patch before the package import:
+```python
+import torch
+_orig = torch.load
+torch.load = lambda *a, **k: _orig(*a, **{**k, "weights_only": k.get("weights_only", False)})
+```
+
+### Stuck at `ZeroGPU init – 10.0%` then 60 s timeout
+
+**Cause**: A library called `cuInit` in the parent process, poisoning the fork (most often `numba.cuda` via NeMo). The actual `@spaces.GPU` body never starts.
+**Fix**: `os.environ.setdefault("NUMBA_DISABLE_CUDA", "1")` as the first line of `app.py`, before `import spaces`.
+
+### `RUNTIME_ERROR` right after long `APP_STARTING`, logs sparse
+
+**Cause**: Boot exceeded `startup_duration_timeout` (default 30 min). Big-model loads commonly trigger this.
+**Fix**: Bump `startup_duration_timeout: 1h` in README frontmatter. For Gradio 6 specifically, also set `GRADIO_SSR_MODE=false` via `hf spaces variables add <id> --env GRADIO_SSR_MODE=false` to dodge SSR health-check timeouts during slow boot.
+
+### `RUNNING` but the public URL returns 404
+
+**Cause**: The Space is private. Anonymous Client / browser hits return 404.
+**Fix**: Authenticate. `gradio_client.Client(space, token=os.environ["HF_TOKEN"])`. The kwarg is `token=`, not `hf_token=`.
+
+### `workload was not healthy after 30 min`
+
+**Cause**: Infra-side scheduling or a build that genuinely can't finish in time.
+**Fix**: Usually not actionable in code. Bump `startup_duration_timeout` if heavy downloads are expected; otherwise wait or report.
+
+### Exit code 128 / containerd / scheduling failure
+
+**Cause**: HF infra glitch.
+**Fix**: `hf spaces restart <id> --factory-reboot`. If it persists, retry later or report. Not fixable in code.
+
+---
+
+## Inference-time errors
+
+These appear in `hf spaces logs <id> --follow` while a request is running.
+
+### `ZeroGPU illegal duration`
+
+**Cause**: `@spaces.GPU(duration=N)` is larger than the visitor's tier per-call cap.
+**Fix**: Lower `N`. Tier caps live in the [ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu).
+
+### `ZeroGPU quota exceeded (X requested vs Y left)`
+
+**Cause**: The visitor's remaining quota < `requested duration`. The comparison is `requested vs remaining`, not `actual vs remaining` — a 10-second task left at the default 60 s blocks the user as soon as their remaining drops below 60 s.
+**Fix**: Lower `duration` to the realistic worst case. For input-dependent runtime, use a callable estimator.
+
+### `RuntimeError: NVML_SUCCESS == r INTERNAL ASSERT FAILED at .../CUDACachingAllocator.cpp`
+
+**Cause**: Allocator fragmentation under transient memory spikes (high-res pixel-space ops, large attention activations, SR models, video DiTs). Not a clean OOM.
+**Fix**: Set expandable segments at the **very top** of `app.py`, before any torch import:
+```python
+import os
+os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
+import spaces
+import torch
+```
+Usually a single-line fix that replaces lowering resolution or moving to `xlarge`.
+
+### Call hangs forever on first `@spaces.GPU` entry
+
+**Cause**: The decorated function returned a CUDA tensor. Unpickling it in the main process triggers `torch.cuda._lazy_init()`, which ZeroGPU blocks.
+**Fix**: Convert to CPU before returning: `return tensor.cpu()` or `.cpu().numpy()`. For `gr.State`, scrub before yielding.
+
+### `PicklingError` at call entry
+
+**Cause**: An argument crossing the fork boundary contains an unpicklable object — file handle, lock, lambda, closure, or `gr.SelectData`.
+**Fix**: Extract the picklable fields in a thin un-decorated wrapper, pass plain values to the `@spaces.GPU` function. For `gr.SelectData` specifically, pull out `evt.index[0]`, `evt.index[1]` etc. outside the decorator.
+
+### `RecursionError` inside `gr.SelectData.__getattr__`
+
+**Cause**: Same as above — `gr.SelectData` doesn't survive pickle.
+**Fix**: Same — extract its fields before crossing the boundary.
+
+### `CUDA error … flash_fwd_launch_template.h: no kernel image is available for execution on the device` (or `:188: invalid argument`)
+
+**Cause**: A Flash Attention 3 kernel was loaded — directly via `kernels-community/{flash-attn3,vllm-flash-attn3,sgl-flash-attn3}`, or indirectly via an old `xformers` wheel that auto-dispatched to FA3. FA3 has no Blackwell sm_120 build.
+**Fix**: Use `attn_implementation="sdpa"`, or `"flash_attention_2"` with the FA2 wheel from `multimodalart/zerogpu-blackwell-wheels`. For xformers, the prebuilt wheel from the same dataset auto-dispatches to FA2 — no monkey-patch.
+
+### `NotImplementedError: sgl_flash_attn3 is only supported on sm80 and above with CUDA >= 12.3`
+
+**Cause**: `kernels-community/sgl-flash-attn3` rejects sm_120 at runtime despite the error wording. Same root cause as the FA3 entry above.
+**Fix**: Same — SDPA or FA2.
+
+### `ImportError: cannot import name 'flash_attn_varlen_func' from 'flash_attn'` / model insists on `attn_implementation="flash_attention_2"`
+
+**Cause**: The model imports flash_attn at module top with no escape hatch, and the runtime doesn't ship it.
+**Fix**: Install the prebuilt `flash_attn-2.8.3-cp310-cp310-linux_x86_64.whl` from `multimodalart/zerogpu-blackwell-wheels`. Requires `python_version: "3.10"` in README (wheel is cp310 only). Real `flash_attn_2_cuda` satisfies xformers' `flash_attn_gpu` probe too.
+
+For transformers `AutoModel`-style configs that aren't a hard import, swap `attn_implementation="flash_attention_2"` → `"sdpa"`. Torch-native, zero deps.
+
+### `selective_scan_cuda.so undefined symbol` / `_torchaudio.abi3.so undefined symbol`
+
+**Cause**: A direct-URL prebuilt CUDA wheel pinned to an old torch ABI (`cu12torch2.4cxx11abiFALSE`, etc.) — won't load on the current runtime.
+**Fix**: Drop the URL-pinned wheel from `requirements.txt`. Use a torch-current wheel from `multimodalart/zerogpu-blackwell-wheels`, kernels-community, or upstream's release page.
+
+### `TypeError: 'dict' object is not hashable` inside `jinja2/utils.py:get`
+
+**Cause**: Old gradio (4.44) + modern starlette / jinja2 cache clash.
+**Fix**: Either pin `jinja2<3.2` + `starlette<0.40` (keeps old gradio), or bump `sdk_version` past 5.0.
+
+---
+
+## Smoke-test / client errors
+
+### `Client.__init__() got an unexpected keyword argument 'hf_token'`
+
+**Cause**: Older `gradio_client` API used `hf_token=`; current uses `token=`.
+**Fix**: `Client(space, token=os.environ["HF_TOKEN"])`.
+
+### `httpx.ReadTimeout` on `client.predict(...)`
+
+**Cause**: Default timeout too small for the GPU duration.
+**Fix**: `Client(..., httpx_kwargs={"timeout": 600})`. Set this to at least your `@spaces.GPU(duration=N)` plus 60 s.
+
+### `404` on what looks like a valid endpoint name
+
+**Cause**: Streaming endpoint (function uses `yield`), or a custom `gr.Server` `@app.get/post` route that doesn't appear in `view_api()`.
+**Fix**: For streaming, use `client.submit(...).result()` (or iterate the job). For custom routes, use `httpx.post(base_url + "/route", ...)` directly — bypass `gradio_client`.
+
+### Result file looks empty / dimensions wrong
+
+**Cause**: HTTP 200 ≠ correct output. The model snapped to a different size, or the wrong endpoint was hit.
+**Fix**: Sniff the returned file's magic bytes (`glTF`, `\x89PNG`, `RIFF…WEBP`, `RIFF…WAVE`, `[4:8]==b"ftyp"`) and check returned dimensions match what was requested.
+
+---
+
+## Submitting new entries
+
+If you hit an error not in this file and figure out the fix, please ask your human to PR it back to this skill. Format:
+
+- A 1-line heading with the exact error substring an agent would grep for.
+- **Cause**: one sentence on what triggered it.
+- **Fix**: concrete commands or code. If the fix needs more than 5 lines of narrative, point to another reference file (e.g. [`debugging.md`](debugging.md), [`zerogpu.md`](zerogpu.md)) for depth.

package/bundled-skills/huggingface-spaces/references/requirements.md

@@ -0,0 +1,153 @@
+# requirements.txt for Spaces
+
+Rules for what to pin, what to leave alone, where to source CUDA wheels, and which torch-side-cars drift silently.
+
+## What's preinstalled (do not list)
+
+The Gradio SDK base image already installs these on every hardware tier — listing them in `requirements.txt` causes resolution failures or, worse, lets pip silently drift the runtime out of compatibility:
+
+| Package | Pinning rules |
+|---|---|
+| `gradio` | Don't list. Locked by `sdk_version:` in README frontmatter; pinning here is ignored or breaks. |
+| `spaces` | Don't list. Platform-pinned; a user pin always loses. |
+| `huggingface_hub` | Don't list by default. Pin only as a workaround for old `gradio<5` that imports the removed `HfFolder` symbol (see [`known-errors.md`](known-errors.md)). |
+| `torch` | **Pinnable, but only within `{2.8.0, 2.9.1, 2.10.0, 2.11.0}`.** Anything outside causes `CONFIG_ERROR: torch version in requirements.txt is not compatible`. Default is to leave unpinned (runtime preinstalls 2.11), but pinning is appropriate when (a) a specific version is known-good for your model, (b) you're matching a CUDA-extension wheel's `torch2.X` tag, or (c) a dep would otherwise drag torch outside the supported set. When you pin torch, also pin `torchvision` / `torchaudio` to the matching minor — see the "Torch-family side-car drift" section below. |
+
+## What to list
+
+Everything you actually `import`, including the often-forgotten:
+
+- `torchvision`, `torchaudio` — **not** preinstalled. Leave unpinned; pip resolves against the installed `torch` major.minor.
+- `accelerate` — needed whenever you use `device_map=`. Listing it also silences `low_cpu_mem_usage=False` warnings.
+- `sentencepiece` — required by most LLM tokenizers; rarely transitive.
+- `einops` — required by `flash_attn.layers.rotary` and many model repos.
+- Domain libs: `diffusers`, `transformers`, `safetensors`, `pillow`, `numpy`, etc.
+
+If a research repo ships a Python package directory (`models/`, `pipeline/`, …), just upload the directory with the rest of the Space — the whole repo root is importable as `/home/user/app`. **Do not** try to reference local paths from `requirements.txt`.
+
+## Pinning torch
+
+ZeroGPU accepts only `2.8.0`, `2.9.1`, `2.10.0`, `2.11.0`. Default is unpinned (runtime preinstalls the latest). Pinning is fine — and sometimes warranted — within that set:
+
+- A specific torch is known-good for your model (numerics, attention kernel availability, etc.).
+- A direct-URL CUDA wheel encodes a `torch2.X` tag (see "Prebuilt CUDA wheels" below) — pin torch to match.
+- A dep's `setup.py` would otherwise downgrade torch outside the supported set.
+
+`2.8.0` is the safest fallback for old requirements that refuse modern torch. `2.10.0` / `2.11.0` is the sweet spot for new code. When you pin torch, also pin `torchvision` / `torchaudio` to the matching minor — see the side-car drift section.
+
+When a dep would silently downgrade torch (e.g. some forks of `demucs`, `audiocraft` pin `torchaudio<2.1`), install the offender from `app.py` with `--no-deps` rather than pinning torch around it:
+
+```python
+import subprocess, sys
+subprocess.run([sys.executable, "-m", "pip", "install", "--no-deps",
+                "git+https://github.com/facebookresearch/demucs"], check=True)
+import spaces  # safe now — torch wasn't touched
+```
+
+List the offender's real runtime deps yourself in `requirements.txt`.
+
+## Torch-family side-car drift
+
+`torchvision`, `torchaudio`, `torchcodec` are built against a specific `torch` major.minor. Listing them unpinned **usually** works, but two known drift patterns:
+
+- `torchaudio==2.11.0` (and later) **dropped its `Requires-Dist: torch==X.Y.Z` line**. With torch pinned to 2.10, pip silently resolves torchaudio to 2.11.0 and the import fails on ABI mismatch.
+- `torchcodec` declares no torch dependency in PyPI metadata at all.
+
+Verification after `pip install` or `uv lock --upgrade`:
+
+```bash
+curl -s https://pypi.org/pypi/<pkg>/<version>/json \
+  | python3 -c "import json,sys,re; rd=json.load(sys.stdin)['info'].get('requires_dist') or []; \
+                print('\n'.join(x for x in rd if re.match(r'^torch(?![a-z])', x)) or '(no torch constraint)')"
+```
+
+When PyPI is silent, fall back to the project's README compatibility table (torchcodec's lives at https://github.com/pytorch/torchcodec).
+
+## Prebuilt CUDA wheels — the Blackwell wheels dataset
+
+For CUDA-extension packages without an upstream wheel matching the ZeroGPU torch / cuda / cxx11-abi cell, use the canonical prebuilt wheels at:
+
+> https://huggingface.co/datasets/multimodalart/zerogpu-blackwell-wheels
+
+Wheels live at `wheels/<cell>/<package>-<ver>-<tag>.whl`. Current cells:
+
+- `pt212-cu130-cp310` — built against torch 2.12 / CUDA 13.0 / Python 3.10. **Works on the live ZeroGPU runtime (torch 2.11)** for all packages below.
+- `pt212-cu130-cp312`, `pt212-cu130-cp313` — same matrix at other Python versions.
+- `pt28-cu128-cp310` — older fallback for Spaces stuck on torch 2.8 / CUDA 12.8.
+
+Reference by direct URL in `requirements.txt`:
+
+```
+https://huggingface.co/datasets/multimodalart/zerogpu-blackwell-wheels/resolve/main/wheels/pt212-cu130-cp310/<wheel>
+```
+
+### Per-package status
+
+Verified empirically against the live runtime + the real Spaces that previously shipped runtime patches:
+
+| Package | Wheel | Replaces | Caveats |
+|---|---|---|---|
+| `xformers` | `xformers-0.0.34+3da0fc92.d20260528-cp39-abi3-linux_x86_64.whl` | MEA→SDPA monkey-patch shim; Cutlass-force shim | `Requires: torch>=2.10`. Auto-dispatch picks FA2 (`fa2F@2.5.7-pt`) on sm_120. Classic Cutlass / FA3 still reject sm_120 but auto-dispatch never selects them now. |
+| `flash_attn` | `flash_attn-2.8.3-cp310-cp310-linux_x86_64.whl` | Committed `flash_attn/` stub package; `sys.modules["flash_attn"] = ...` injection | **cp310 only** — requires `python_version: "3.10"` in README. Needs `einops` for `flash_attn.layers.rotary`. Real `flash_attn_2_cuda` satisfies xformers' `hasattr(flash_attn.flash_attn_interface, "flash_attn_gpu")` probe. |
+| `pytorch3d` | `pytorch3d-0.7.9-cp310-cp310-linux_x86_64.whl` | Runtime `pip install git+...pytorch3d.git` inside `@spaces.GPU` | Needs `numpy`, `iopath`, `fvcore` listed. No torch pin in metadata; loads cleanly on torch 2.11. |
+| `nvdiffrast` | `nvdiffrast-0.4.0-cp310-cp310-linux_x86_64.whl` | Runtime build with `TORCH_CUDA_ARCH_LIST=12.0` | Needs `numpy`. `RasterizeGLContext` in 0.4.0 is a deprecation alias for `RasterizeCudaContext` — no headless-GL footgun. |
+| `diff_gaussian_rasterization` | `diff_gaussian_rasterization-0.0.0-cp310-cp310-linux_x86_64.whl` | Runtime build from `graphdeco-inria/diff-gaussian-rasterization.git` | **Upstream Inria API only** (returns 2-tuple `(color, radii)`). Does NOT match the ashawkey fork (4-tuple including alpha+depth) used by `ashawkey/LGM`, `dylanebert/LGM-mini`, etc. Forks need their own wheel. |
+| `torchmcubes` | `torchmcubes-0.1.0-cp310-cp310-linux_x86_64.whl` | Runtime `pip install git+...torchmcubes.git` | **sm_120 only** (no fatbin for older archs). Works on ZeroGPU / Blackwell; not portable to a dedicated T4 / L4 / A10G Space. |
+
+### Pattern
+
+```
+# requirements.txt
+numpy
+einops
+https://huggingface.co/datasets/multimodalart/zerogpu-blackwell-wheels/resolve/main/wheels/pt212-cu130-cp310/flash_attn-2.8.3-cp310-cp310-linux_x86_64.whl
+https://huggingface.co/datasets/multimodalart/zerogpu-blackwell-wheels/resolve/main/wheels/pt212-cu130-cp310/xformers-0.0.34+3da0fc92.d20260528-cp39-abi3-linux_x86_64.whl
+```
+
+```yaml
+# README frontmatter — pin Python to match wheel cell
+python_version: "3.10"
+```
+
+**Do not** install these from `@spaces.GPU` startup. The previous "subprocess.check_call pip install at first GPU acquire" pattern is now strictly worse than the wheel URL — slower cold start, eats `duration` budget, breaks reproducibility, and the build sometimes exceeds the `@spaces.GPU(duration=1500)` cap.
+
+### When you need a wheel that's not in the dataset
+
+Three options, in preference order:
+
+1. **kernels-community** — https://huggingface.co/kernels-community handles ABI matching for you. Often the simplest path; no version pinning needed.
+2. **Upstream wheel matrix** — e.g. flash-attention's releases page ships a fairly complete `cu12 / torch / Python` matrix at https://github.com/Dao-AILab/flash-attention/releases. Pin `torch==X.Y.Z` in `requirements.txt` to match the wheel's `torch2.X` tag.
+3. **Build it yourself and host on HF Hub.** Last resort — see [`debugging.md`](debugging.md) for the in-`@spaces.GPU` source-build pattern as a stopgap while a wheel is being built.
+
+## Reading a CUDA wheel filename
+
+```
+flash_attn-2.8.3+cu130torch2.12cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
+```
+
+| Tag | Meaning |
+|---|---|
+| `cu130` | CUDA major version (13.0) |
+| `torch2.12` | torch major.minor the wheel was compiled against |
+| `cxx11abiFALSE` | C++ stdlib ABI choice (`TRUE` or `FALSE`) |
+| `cp310-cp310` | CPython version (3.10) |
+
+ABI / symbol mismatches at any of these → `ImportError` on first import. Pin `torch` to match `torch2.X`. Set `python_version:` to match `cp3XX`.
+
+## Don't pin `xformers`
+
+Leave bare in `requirements.txt` (or use the prebuilt URL above). Pip picks the wheel matching your installed torch.
+
+## Don't pin `spaces`
+
+Even if a `uv export` produces it, exclude with `--no-emit-package spaces`. The platform always pins its own version.
+
+## Specifically about Python version
+
+Pinning `python_version:` is effectively required:
+
+- ZeroGPU officially supports **3.10.13** and **3.12.12**.
+- The runtime default is 3.10.
+- Pinning to a `cp3XX` wheel matrix (e.g. `cp310` flash_attn wheel) forces matching Python.
+
+Both `"3.12"` and `"3.12.12"` forms are accepted in YAML.

package/bundled-skills/huggingface-spaces/references/zerogpu.md

@@ -0,0 +1,276 @@
+# ZeroGPU
+
+Read this whenever the Space targets ZeroGPU (`zero-a10g` flavor). The SKILL.md's 3-rule summary is a starting point; this file covers the model in enough detail to debug and design.
+
+For numerical limits (per-tier daily quota minutes, runs-per-day caps, current backing GPU, supported Python / torch versions): https://huggingface.co/docs/hub/spaces-zerogpu. Those values change over time and are deliberately kept out of this skill.
+
+## The mental model
+
+A ZeroGPU Space runs as **two processes**:
+
+- **Main web process** — long-lived. Imports `app.py`, launches Gradio. Holds no VRAM and, after the startup "pack" step, no model weights in RAM either.
+- **GPU worker** — short-lived. Forked per `@spaces.GPU` request (or reused if warm). Eventually killed by the ZeroGPU scheduler when another Space needs the slot. Your code never kills its own worker.
+
+`import spaces` monkey-patches `torch.cuda.*` in the main process so that `.to("cuda")` and `torch.cuda.is_available()` work at module scope **without** a real GPU attached. Module-level `model.to("cuda")` is intercepted: the tensor data physically stays in main-process RAM at this point, with a CUDA-presenting "fake" tensor registered alongside. At a startup "pack" step, the backend writes those original CPU tensors to disk via `O_DIRECT` and frees the RAM. After pack, main holds no weights anywhere.
+
+When a `@spaces.GPU` call lands, the scheduler routes it to a worker:
+
+- **Cold worker** — forked from the main process; torch is unpatched; real CUDA is initialized; weights are streamed disk → pinned host → VRAM via a double-buffered pipeline. This is the cold-start cost.
+- **Warm worker** — alive worker bound to the same slot; init is skipped; weights stay on VRAM from the previous call.
+
+A warm worker eventually dies when another Space needs the slot. Occasional cold starts on a low-traffic Space are normal.
+
+## The three rules
+
+### 1. `import spaces` before any CUDA-touching import
+
+```python
+import spaces      # FIRST
+import torch       # then this
+```
+
+If something initializes CUDA before `import spaces`, the patch can't apply and you get `RuntimeError: CUDA has been initialized before importing the spaces package`. For libraries that eagerly init CUDA on import (e.g. `numba.cuda`, NeMo via numba), set the disable env *before* the import:
+
+```python
+import os
+os.environ.setdefault("NUMBA_DISABLE_CUDA", "1")
+import spaces
+```
+
+### 2. Load models at module scope, `.to("cuda")` eagerly
+
+```python
+pipe = DiffusionPipeline.from_pretrained("...", torch_dtype=torch.bfloat16).to("cuda")
+```
+
+Do **not** lazy-load inside `@spaces.GPU`. The hijack is designed for module-level placement; deferring it puts tens of seconds of checkpoint I/O + dtype cast + GPU move inside every cold request.
+
+Use the **string `"cuda"`** — never an integer device id. ZeroGPU re-allocates device ids per request, so `.to(0)`, `device_map={"": 0}`, `torch.cuda.set_device(0)` silently break.
+
+For plain `from_pretrained` loads, use `.to("cuda")`, **not** `device_map="cuda"` (which routes through `accelerate.set_module_tensor_to_device` and calls `torch._C._cuda_init()` at load time, bypassing the hijack). The exception is loaders that are ZeroGPU-aware — notably the `bitsandbytes` quantization path; `from_pretrained(..., quantization_config=BitsAndBytesConfig(...))` works with `device_map="cuda"`.
+
+**Preloading multiple variants** (e.g. base + refiner, image + video model) is fine as long as their combined VRAM fits. Load all of them sequentially at module scope into a dict, then key per request. Don't unload/reload between requests — that puts the load cost back on the user.
+
+### 3. Decorate the function Gradio binds
+
+ZeroGPU's startup scan walks Gradio's registered event handlers for `@spaces.GPU`-marked functions. If you decorate `inner_helper` but `click(fn=outer)` is what's wired up, you get `RuntimeError: No @spaces.GPU function detected during startup`. Always decorate the function passed to the event handler.
+
+```python
+@spaces.GPU(duration=60)
+def generate(prompt):
+    return pipe(prompt).images[0]
+
+btn.click(fn=generate, inputs=prompt_box, outputs=image_out)
+```
+
+## Sizing duration
+
+`@spaces.GPU(duration=N)` means "reserve N seconds of GPU time." Two failure modes:
+
+- **`ZeroGPU illegal duration`** — `N` exceeds the visitor's tier cap. Lowering `duration` is the only fix.
+- **`ZeroGPU quota exceeded`** — the visitor's remaining quota is less than `requested`. Compared as `requested vs remaining`, not `actual vs remaining` — so a 10-second task left at the default 60 s blocks the user as soon as their remaining drops below 60 s.
+
+Smaller `duration` also ranks **higher** in the queue. Both reasons push toward declaring the realistic worst case, not a comfortable margin.
+
+**Pick the value — don't guess.** A too-high duration deploys cleanly then errors on the first call; too-low silently truncates. Methodology:
+
+1. Ship with a placeholder (e.g. 180 s).
+2. Instrument with `time.perf_counter()` and return the seconds in the response.
+3. Run 2–3 representative calls via `gradio_client`.
+4. Set `duration = round(measured_max × 1.4)`.
+
+For input-dependent runtime, pass a **callable**:
+
+```python
+def _estimate(prompt, steps, *args, **kwargs):
+    # Swallow extras with *args, **kwargs — Gradio passes progress= positionally
+    # and a strict signature will raise "takes 5 positional arguments but 6 were given"
+    return min(240, 60 + int(steps * 3.5))
+
+@spaces.GPU(duration=_estimate)
+def generate(prompt, steps, ..., progress=gr.Progress(track_tqdm=True)):
+    ...
+```
+
+## Sizing memory: `large` vs `xlarge`
+
+`size="large"` (default) is half the backing card (48 GB on Blackwell). `size="xlarge"` is the full card (96 GB) and costs **2× quota** per second — plus higher queue waits. Use `large` unless the workload genuinely OOMs.
+
+Rough VRAM sizing:
+
+| Mode | Memory rule | 7B | 27B | 70B |
+|------|------------|------|------|------|
+| bf16 | `params × 2` GB | 14 GB ✓ large | 54 GB → xlarge | 140 GB → quant + xlarge |
+| int8 | `params × 1` GB | 7 GB ✓ large | 27 GB ✓ large | 70 GB → xlarge |
+| 4-bit (NF4 / int4) | `params × ~0.55` GB | 4 GB ✓ large | 15 GB ✓ large | 40 GB ✓ large |
+
+Numbers are for weights only; activations and KV cache add on top (significant for long context).
+
+## Quantization
+
+ZeroGPU supports two quantization stacks: **`bitsandbytes`** (drop-in for transformers, well-trodden) and **`torchao`** (torch-native, newer, smaller install). Pick by what your model's `from_pretrained` actually wires up; if both work, default to `bitsandbytes` for transformers LLMs and `torchao` for diffusers.
+
+### bitsandbytes (NF4 / int8)
+
+```python
+import spaces, torch
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+
+bnb = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_use_double_quant=True,
+    bnb_4bit_compute_dtype=torch.bfloat16,
+)
+tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
+model = AutoModelForCausalLM.from_pretrained(
+    MODEL_ID,
+    quantization_config=bnb,
+    device_map="cuda",            # OK here — bnb's loader is ZeroGPU-aware
+    dtype=torch.bfloat16,
+).eval()
+```
+
+This is the one case where `device_map="cuda"` is **safe** on ZeroGPU at module scope (bitsandbytes' loader path intercepts cleanly). For non-bnb loads, stick to `.to("cuda")`.
+
+`load_in_8bit=True` swaps the 4-bit block for int8 — same hijack-safe loader. Bigger but higher quality, no `compute_dtype` knob.
+
+### torchao
+
+```python
+import spaces, torch
+from diffusers import DiffusionPipeline
+from torchao.quantization import quantize_, Int8WeightOnlyConfig
+
+pipe = DiffusionPipeline.from_pretrained(MODEL_ID, torch_dtype=torch.bfloat16).to("cuda")
+quantize_(pipe.transformer, Int8WeightOnlyConfig())   # mutates in place
+```
+
+`torchao` is more flexible (fine-grained per-module quantization, `Int4WeightOnlyConfig`, `Float8WeightOnlyConfig`, etc.) and works with diffusers' `from_pretrained(..., quantization_config=TorchAoConfig(...))` integration too. No CUDA build dependency — installs as a wheel.
+
+### Attention backends
+
+Default to `attn_implementation="sdpa"` — torch-native, works everywhere. Reach for an FA backend only when the upstream repo uses it by default or strongly recommends it. If it breaks on Blackwell, fall back to SDPA.
+
+**Flash Attention 2** works via the prebuilt wheel at `multimodalart/zerogpu-blackwell-wheels` ([`requirements.md`](requirements.md)). The wheel's real `flash_attn_2_cuda` also satisfies xformers' import-time probes.
+
+**Flash Attention 3** is not currently usable on Blackwell sm_120. `kernels-community/flash-attn3`, `vllm-flash-attn3`, and `sgl-flash-attn3` all fail with `no kernel image is available` or `NotImplementedError`. Use SDPA or FA2 instead.
+
+**xformers** is available via the prebuilt wheel — auto-dispatch picks FA2 on sm_120 with no monkey-patch.
+
+## Concurrency
+
+Handlers run **concurrently by default**. Three rules:
+
+1. **No mutable global state.** Handlers writing to a module-level dict / list race each other.
+2. **No fixed output paths.** Two concurrent calls writing to `output.png` clobber each other (and leak data across users). Use `tempfile.NamedTemporaryFile(suffix=...)`.
+3. **Read-only globals are safe** — models, tokenizers, configs loaded once and only read inside handlers.
+
+## Process isolation and pickle
+
+`@spaces.GPU` runs in a separate fork. Arguments and return values cross via pickle:
+
+- **Only picklable objects** in/out. File handles, locks, lambdas, closures over unpicklable state → `PicklingError`.
+- **Never return CUDA tensors.** Unpickling in the main process triggers `torch.cuda._lazy_init()`, which ZeroGPU blocks → the call hangs. Convert to CPU first: `return tensor.cpu()` or `.cpu().numpy()`.
+- CPU tensors, numpy arrays, PIL Images, plain Python objects work fine.
+- `gr.SelectData` is a special case — its `__getattr__` recurses under pickle. Extract the fields you need (`evt.index[0]`, etc.) in a thin un-decorated wrapper, pass plain values to the `@spaces.GPU` function.
+
+### `gr.State` across the fork
+
+`gr.State` is pickled on every yield. The handler receives a **copy**:
+
+- In-place mutations inside the fork are invisible to other handlers until you explicitly `yield` the mutated value back.
+- Yielding `gr.update()` for a state slot **skips** the update — other handlers continue to see pre-yield value.
+- For large state, minimize how often you yield it — ideally once at the end.
+- CUDA tensors inside state must be CPU-d before yielding (same `_lazy_init` issue).
+
+## Generators and streaming
+
+`@spaces.GPU` supports generator functions — first-class for progressive UI updates:
+
+```python
+@spaces.GPU(duration=120)
+def generate(prompt):
+    yield gr.update(value=None, label="Starting…")
+    for step in range(num_steps):
+        latent = step_fn(...)
+        yield gr.update(value=preview(latent), label=f"Step {step+1}/{num_steps}")
+    yield gr.update(value=final_image, label="Done")
+```
+
+`gr.Progress(track_tqdm=True)` and `yield` compete with each other — pick one.
+
+For streaming previews **inside** a diffusers `callback_on_step_end`, use a thread + queue inside the decorator (forks share threads):
+
+```python
+@spaces.GPU(duration=180)
+def generate(prompt, num_steps):
+    q = queue.Queue()
+    DONE = object()
+    def cb(pipe, step, t, kw):
+        q.put((step, taef1_preview(kw["latents"])))
+        return kw
+    def run():
+        out = pipeline(prompt=prompt, num_inference_steps=num_steps,
+                       callback_on_step_end=cb,
+                       callback_on_step_end_tensor_inputs=["latents"])
+        q.put((DONE, out))
+    threading.Thread(target=run, daemon=True).start()
+    while True:
+        idx, payload = q.get()
+        if idx is DONE: break
+        yield gr.update(value=payload, label=f"Step {idx+1}/{num_steps}")
+```
+
+**Do not** use `ProcessPoolExecutor` / `multiprocessing.Pool` inside `@spaces.GPU` — the daemonic fork can't spawn children (`AssertionError: daemonic processes are not allowed to have children`). Threads only.
+
+## Compilation
+
+`torch.compile` is **not supported** on ZeroGPU. Use PyTorch ahead-of-time inductor (AoTI), supported from torch 2.8+. Full guide: https://huggingface.co/blog/zerogpu-aoti. The `spaces` package exposes `aoti_capture`, `aoti_compile`, `aoti_apply`, `aoti_blocks_load` for the workflow.
+
+## Local development
+
+**Do NOT** wrap `import spaces` in `try/except` with a no-op fallback. Off-ZeroGPU, the `spaces` package is *already* a true no-op — the heavyweight behavior is gated on `SPACES_ZERO_GPU=1`, set only on ZeroGPU. `@spaces.GPU` returns the undecorated function unchanged elsewhere. The Gradio base image installs `spaces` on every hardware tier, so a duplicate onto T4 / A10G / CPU works without code changes too.
+
+That said: **iterate ON the Space, not locally.** The Space environment (Python, torch, CUDA, drivers, env vars) differs from yours; passing local tests doesn't prove the Space works. Push early — even with the app not fully polished — and use the rung ladder ([`debugging.md`](debugging.md)) against the live URL.
+
+## Allocator config for memory pressure
+
+If your workload hits transient allocation spikes (high-res pixel-space ops, large attention activations, SR models, video DiTs) and you see:
+
+```
+RuntimeError: NVML_SUCCESS == r INTERNAL ASSERT FAILED at .../CUDACachingAllocator.cpp
+```
+
+set expandable segments at the **very top** of `app.py`, before any torch import:
+
+```python
+import os
+os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
+import spaces
+import torch
+```
+
+Often single-line fix for what looks like an OOM. See [`known-errors.md`](known-errors.md).
+
+## Example caching
+
+`gr.Examples` defaults on ZeroGPU:
+
+- `cache_examples=True`
+- `cache_mode="lazy"` (eager would pre-run examples at startup, but no GPU is attached at startup)
+
+Don't override to `cache_mode="eager"` on ZeroGPU — it will fail or burn the creator's daily quota. The cache is keyed by example **file path**, not content hash: regenerating an asset in place serves the stale cached output. Bump a `cache_version` constant if you replace example files.
+
+## Real-time sessions
+
+For real-time apps (webcam, audio streaming), the per-call fork model is too costly. ZeroGPU supports reusable "real-time sessions" — one GPU allocation amortized across many small requests. Reference Spaces:
+
+- https://huggingface.co/spaces/diffusers/unofficial-SDXL-Turbo-i2i-t2i
+- https://huggingface.co/spaces/huggingface-projects/rf-detr-realtime-webcam
+
+## When things go wrong
+
+For specific error strings (CUDA init order, illegal duration, allocator asserts, PicklingError, returning CUDA tensors, …): [`known-errors.md`](known-errors.md). It covers the ZeroGPU-specific patterns alongside everything else and is the single error lookup for the skill.
+
+When the log endpoint can't explain a failure (device-side asserts, OOM under specific shapes, race conditions in pickle), dev mode + SSH is the last-resort tool — see [`debugging.md`](debugging.md).