tribunal-kit 9.2.3 → 9.2.5

package/.agent/skills/huggingface-spaces/references/known-errors.md

@@ -0,0 +1,232 @@
+# Known errors
+
+Check if this is a known issue before trying your own fix. Entries are keyed by the substring that actually appears in `runtime.errorMessage`, the build log, or a Python traceback — grep this file for the error you saw.
+
+If you hit something not listed here and figure out a fix, please ask your human to PR it back so future runs benefit.
+
+---
+
+## Build / config errors
+
+These come from the Space build pipeline before the app starts. Read with `hf spaces logs <id> --build --tail 500` — find the **first** error, not the last.
+
+### `CONFIG_ERROR: torch version in requirements.txt is not compatible with ZeroGPU`
+
+**Cause**: `requirements.txt` pins `torch==X.Y.Z` to a version outside the supported set (`2.8.0`, `2.9.1`, `2.10.0`, `2.11.0`).
+**Fix**: Unpin torch (preferred — the runtime preinstalls the latest supported version), or pin to one of the supported values.
+
+### `Cannot install … because these package versions have conflicting dependencies` / `ResolutionImpossible`
+
+**Cause**: A dep conflicts with the Gradio SDK pinned by `sdk_version:` in README. Most commonly `pydantic`, `uvicorn`, `huggingface_hub`, or `jinja2` pinned to old values that the SDK no longer accepts.
+**Fix**: Unpin the offender. For `gradio[mcp]` specifically, `uvicorn>=0.31.1` and `pydantic>=2.11.10` are required.
+
+### Build hangs in dependency resolution > 10 min
+
+**Cause**: pip backtracking through a deep version space.
+**Fix**: Pin the conflicting transitive dep. The `--build` logs will show which one. Bump `startup_duration_timeout: 1h` in README frontmatter if heavy downloads are expected.
+
+### `ModuleNotFoundError: No module named 'pkg_resources'`
+
+**Cause**: setuptools 81 dropped `pkg_resources`; an old package's `setup.py` imports it.
+**Fix**: Bump or unpin the offender. Typical culprits: `deepspeed==0.15.x` (training-only — usually safe to drop from inference Spaces), `openai-whisper==20231117`.
+
+### `400 Bad Request` from `/api/validate-yaml` during `create_repo` / `upload_file`
+
+**Cause**: README frontmatter failed server validation. Most common: `short_description` over the (undocumented) character cap — target ≤ 60.
+**Fix**: Shorten `short_description`. Long descriptions go in the README body. Also double-check `colorFrom`/`colorTo` are one of `red|yellow|green|blue|indigo|purple|pink|gray`.
+
+### `403 Forbidden` from `create_repo` for a Gradio / Docker Space
+
+**Cause**: Gradio and Docker Spaces run on compute and need a paid plan (PRO / Team / Enterprise). The only free exception is ZeroGPU — 2 Spaces for personal accounts in good standing (verified email, account older than 30 days).
+**Fix**: On a free account, pass `space_hardware="zero-a10g"` rather than omitting it — `cpu-basic` is gated too, so dropping the flavor makes this worse, not better. If ZeroGPU is also rejected, the account is over its 2-Space cap or not in good standing: delete an unused ZeroGPU Space, upgrade to PRO, or ship a Static Space and apply for a [community grant](grants.md). Keep `hardware:` out of README frontmatter (silently ignored anyway).
+
+### `403 Forbidden` from `create_commit(..., create_pr=True)`
+
+**Cause**: Upstream Space has Discussions disabled.
+**Fix**: Ask the maintainer to enable Discussions, or push directly if you have write access.
+
+---
+
+## Startup / RUNTIME_ERROR
+
+These come from `hf spaces logs <id> --tail 500`.
+
+### `RuntimeError: CUDA has been initialized before importing the spaces package`
+
+**Cause**: Something triggered CUDA init in the main process before `import spaces`. Usually wrong import order; sometimes a third-party lib eagerly initializing CUDA at import time (e.g. `numba.cuda`).
+**Fix**: Reorder so `import spaces` is first. For numba-using stacks (NeMo, RAPIDS bits):
+```python
+import os
+os.environ.setdefault("NUMBA_DISABLE_CUDA", "1")
+import spaces
+```
+
+### `RuntimeError: No @spaces.GPU function detected during startup`
+
+**Cause**: The function bound to `.click(fn=...)` / `.submit(...)` isn't decorated. Decorating an inner helper doesn't count — the startup scan only walks Gradio's registered handlers.
+**Fix**: Decorate the function Gradio binds. If that conflicts with another decorator, wrap explicitly:
+```python
+@spaces.GPU(duration=60)
+def gpu_inner(...): ...
+def gradio_handler(...): return gpu_inner(...)
+```
+(Or just decorate `gradio_handler` directly.)
+
+### `ImportError: cannot import name 'HfFolder' from 'huggingface_hub'`
+
+**Cause**: Old gradio (`4.44` and similar) imports `HfFolder` from `huggingface_hub`, which was removed in recent hub releases.
+**Fix**: Two options.
+- Pin `huggingface-hub==0.25.0` in `requirements.txt` (keeps old gradio happy).
+- Bump `sdk_version` in README to `5.x` or `6.x` (also fixes a lot of other API breaks).
+If a Gradio custom component locks the major (`gradio-image-prompter`, `gradio_litmodel3d`, …), install it with `--no-deps` so its `gradio<5.0` requirement doesn't bind.
+
+### `ImportError: cannot import name 'is_traceable_wrapper_subclass' from 'torch.utils._python_dispatch'`
+
+**Cause**: A dep with `torchaudio<2.1` / `torch<2` in its `setup.py` (e.g. `demucs`, `audiocraft`) downgraded torch silently. The build succeeded, the app booted, and `import spaces` then died on a missing torch symbol.
+**Fix**: Install the offender from `app.py` with `--no-deps` *before* `import spaces`:
+```python
+import subprocess, sys
+subprocess.run([sys.executable, "-m", "pip", "install", "--no-deps",
+                "git+https://github.com/facebookresearch/demucs"], check=True)
+import spaces
+```
+List its actual runtime deps (`dora-search einops julius lameenc openunmix pyyaml tqdm` for demucs) yourself in `requirements.txt`.
+
+### `_pickle.UnpicklingError: Weights only load failed`
+
+**Cause**: `torch.load` weights-only default flipped to `True` in torch 2.6. Old checkpoints pickling numpy/object globals fail.
+**Fix**: For trusted upstream checkpoints, monkey-patch before the package import:
+```python
+import torch
+_orig = torch.load
+torch.load = lambda *a, **k: _orig(*a, **{**k, "weights_only": k.get("weights_only", False)})
+```
+
+### Stuck at `ZeroGPU init – 10.0%` then 60 s timeout
+
+**Cause**: A library called `cuInit` in the parent process, poisoning the fork (most often `numba.cuda` via NeMo). The actual `@spaces.GPU` body never starts.
+**Fix**: `os.environ.setdefault("NUMBA_DISABLE_CUDA", "1")` as the first line of `app.py`, before `import spaces`.
+
+### `RUNTIME_ERROR` right after long `APP_STARTING`, logs sparse
+
+**Cause**: Boot exceeded `startup_duration_timeout` (default 30 min). Big-model loads commonly trigger this.
+**Fix**: Bump `startup_duration_timeout: 1h` in README frontmatter. For Gradio 6 specifically, also set `GRADIO_SSR_MODE=false` via `hf spaces variables add <id> --env GRADIO_SSR_MODE=false` to dodge SSR health-check timeouts during slow boot.
+
+### `RUNNING` but the public URL returns 404
+
+**Cause**: The Space is private. Anonymous Client / browser hits return 404.
+**Fix**: Authenticate. `gradio_client.Client(space, token=os.environ["HF_TOKEN"])`. The kwarg is `token=`, not `hf_token=`.
+
+### `workload was not healthy after 30 min`
+
+**Cause**: Infra-side scheduling or a build that genuinely can't finish in time.
+**Fix**: Usually not actionable in code. Bump `startup_duration_timeout` if heavy downloads are expected; otherwise wait or report.
+
+### Exit code 128 / containerd / scheduling failure
+
+**Cause**: HF infra glitch.
+**Fix**: `hf spaces restart <id> --factory-reboot`. If it persists, retry later or report. Not fixable in code.
+
+---
+
+## Inference-time errors
+
+These appear in `hf spaces logs <id> --follow` while a request is running.
+
+### `ZeroGPU illegal duration`
+
+**Cause**: `@spaces.GPU(duration=N)` is larger than the visitor's tier per-call cap.
+**Fix**: Lower `N`. Tier caps live in the [ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu).
+
+### `ZeroGPU quota exceeded (X requested vs Y left)`
+
+**Cause**: The visitor's remaining quota < `requested duration`. The comparison is `requested vs remaining`, not `actual vs remaining` — a 10-second task left at the default 60 s blocks the user as soon as their remaining drops below 60 s.
+**Fix**: Lower `duration` to the realistic worst case. For input-dependent runtime, use a callable estimator.
+
+### `RuntimeError: NVML_SUCCESS == r INTERNAL ASSERT FAILED at .../CUDACachingAllocator.cpp`
+
+**Cause**: Allocator fragmentation under transient memory spikes (high-res pixel-space ops, large attention activations, SR models, video DiTs). Not a clean OOM.
+**Fix**: Set expandable segments at the **very top** of `app.py`, before any torch import:
+```python
+import os
+os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
+import spaces
+import torch
+```
+Usually a single-line fix that replaces lowering resolution or moving to `xlarge`.
+
+### Call hangs forever on first `@spaces.GPU` entry
+
+**Cause**: The decorated function returned a CUDA tensor. Unpickling it in the main process triggers `torch.cuda._lazy_init()`, which ZeroGPU blocks.
+**Fix**: Convert to CPU before returning: `return tensor.cpu()` or `.cpu().numpy()`. For `gr.State`, scrub before yielding.
+
+### `PicklingError` at call entry
+
+**Cause**: An argument crossing the fork boundary contains an unpicklable object — file handle, lock, lambda, closure, or `gr.SelectData`.
+**Fix**: Extract the picklable fields in a thin un-decorated wrapper, pass plain values to the `@spaces.GPU` function. For `gr.SelectData` specifically, pull out `evt.index[0]`, `evt.index[1]` etc. outside the decorator.
+
+### `RecursionError` inside `gr.SelectData.__getattr__`
+
+**Cause**: Same as above — `gr.SelectData` doesn't survive pickle.
+**Fix**: Same — extract its fields before crossing the boundary.
+
+### `CUDA error … flash_fwd_launch_template.h: no kernel image is available for execution on the device` (or `:188: invalid argument`)
+
+**Cause**: A Flash Attention 3 kernel was loaded — directly via `kernels-community/{flash-attn3,vllm-flash-attn3,sgl-flash-attn3}`, or indirectly via an old `xformers` wheel that auto-dispatched to FA3. FA3 has no Blackwell sm_120 build.
+**Fix**: Use `attn_implementation="sdpa"`, or `"flash_attention_2"` with the FA2 wheel from `multimodalart/zerogpu-blackwell-wheels`. For xformers, the prebuilt wheel from the same dataset auto-dispatches to FA2 — no monkey-patch.
+
+### `NotImplementedError: sgl_flash_attn3 is only supported on sm80 and above with CUDA >= 12.3`
+
+**Cause**: `kernels-community/sgl-flash-attn3` rejects sm_120 at runtime despite the error wording. Same root cause as the FA3 entry above.
+**Fix**: Same — SDPA or FA2.
+
+### `ImportError: cannot import name 'flash_attn_varlen_func' from 'flash_attn'` / model insists on `attn_implementation="flash_attention_2"`
+
+**Cause**: The model imports flash_attn at module top with no escape hatch, and the runtime doesn't ship it.
+**Fix**: Install the prebuilt `flash_attn-2.8.3-cp310-cp310-linux_x86_64.whl` from `multimodalart/zerogpu-blackwell-wheels`. Requires `python_version: "3.10"` in README (wheel is cp310 only). Real `flash_attn_2_cuda` satisfies xformers' `flash_attn_gpu` probe too.
+
+For transformers `AutoModel`-style configs that aren't a hard import, swap `attn_implementation="flash_attention_2"` → `"sdpa"`. Torch-native, zero deps.
+
+### `selective_scan_cuda.so undefined symbol` / `_torchaudio.abi3.so undefined symbol`
+
+**Cause**: A direct-URL prebuilt CUDA wheel pinned to an old torch ABI (`cu12torch2.4cxx11abiFALSE`, etc.) — won't load on the current runtime.
+**Fix**: Drop the URL-pinned wheel from `requirements.txt`. Use a torch-current wheel from `multimodalart/zerogpu-blackwell-wheels`, kernels-community, or upstream's release page.
+
+### `TypeError: 'dict' object is not hashable` inside `jinja2/utils.py:get`
+
+**Cause**: Old gradio (4.44) + modern starlette / jinja2 cache clash.
+**Fix**: Either pin `jinja2<3.2` + `starlette<0.40` (keeps old gradio), or bump `sdk_version` past 5.0.
+
+---
+
+## Smoke-test / client errors
+
+### `Client.__init__() got an unexpected keyword argument 'hf_token'`
+
+**Cause**: Older `gradio_client` API used `hf_token=`; current uses `token=`.
+**Fix**: `Client(space, token=os.environ["HF_TOKEN"])`.
+
+### `httpx.ReadTimeout` on `client.predict(...)`
+
+**Cause**: Default timeout too small for the GPU duration.
+**Fix**: `Client(..., httpx_kwargs={"timeout": 600})`. Set this to at least your `@spaces.GPU(duration=N)` plus 60 s.
+
+### `404` on what looks like a valid endpoint name
+
+**Cause**: Streaming endpoint (function uses `yield`), or a custom `gr.Server` `@app.get/post` route that doesn't appear in `view_api()`.
+**Fix**: For streaming, use `client.submit(...).result()` (or iterate the job). For custom routes, use `httpx.post(base_url + "/route", ...)` directly — bypass `gradio_client`.
+
+### Result file looks empty / dimensions wrong
+
+**Cause**: HTTP 200 ≠ correct output. The model snapped to a different size, or the wrong endpoint was hit.
+**Fix**: Sniff the returned file's magic bytes (`glTF`, `\x89PNG`, `RIFF…WEBP`, `RIFF…WAVE`, `[4:8]==b"ftyp"`) and check returned dimensions match what was requested.
+
+---
+
+## Submitting new entries
+
+If you hit an error not in this file and figure out the fix, please ask your human to PR it back to this skill. Format:
+
+- A 1-line heading with the exact error substring an agent would grep for.
+- **Cause**: one sentence on what triggered it.
+- **Fix**: concrete commands or code. If the fix needs more than 5 lines of narrative, point to another reference file (e.g. [`debugging.md`](debugging.md), [`zerogpu.md`](zerogpu.md)) for depth.

package/.agent/skills/huggingface-spaces/references/requirements.md

@@ -0,0 +1,169 @@
+# requirements.txt for Spaces
+
+Rules for what to pin, what to leave alone, where to source CUDA wheels, and which torch-side-cars drift silently.
+
+## What's preinstalled (do not list)
+
+The Gradio SDK base image already installs these on every hardware tier — listing them in `requirements.txt` causes resolution failures or, worse, lets pip silently drift the runtime out of compatibility:
+
+| Package | Pinning rules |
+|---|---|
+| `gradio` | Don't list. Locked by `sdk_version:` in README frontmatter; pinning here is ignored or breaks. |
+| `spaces` | Don't list. Platform-pinned; a user pin always loses. |
+| `huggingface_hub` | Don't list by default. Pin only as a workaround for old `gradio<5` that imports the removed `HfFolder` symbol (see [`known-errors.md`](known-errors.md)). |
+| `torch` | **Pinnable, but only within `{2.8.0, 2.9.1, 2.10.0, 2.11.0}`.** Anything outside causes `CONFIG_ERROR: torch version in requirements.txt is not compatible`. Default is to leave unpinned (runtime preinstalls 2.11), but pinning is appropriate when (a) a specific version is known-good for your model, (b) you're matching a CUDA-extension wheel's `torch2.X` tag, or (c) a dep would otherwise drag torch outside the supported set. When you pin torch, also pin `torchvision` / `torchaudio` to the matching minor — see the "Torch-family side-car drift" section below. |
+
+## What to list
+
+Everything you actually `import`, including the often-forgotten:
+
+- `torchvision`, `torchaudio` — **not** preinstalled. Leave unpinned; pip resolves against the installed `torch` major.minor.
+- `accelerate` — needed whenever you use `device_map=`. Listing it also silences `low_cpu_mem_usage=False` warnings.
+- `sentencepiece` — required by most LLM tokenizers; rarely transitive.
+- `einops` — required by `flash_attn.layers.rotary` and many model repos.
+- Domain libs: `diffusers`, `transformers`, `safetensors`, `pillow`, `numpy`, etc.
+
+If a research repo ships a Python package directory (`models/`, `pipeline/`, …), just upload the directory with the rest of the Space — the whole repo root is importable as `/home/user/app`. **Do not** try to reference local paths from `requirements.txt`.
+
+## Pinning torch
+
+ZeroGPU accepts only `2.8.0`, `2.9.1`, `2.10.0`, `2.11.0`. Default is unpinned (runtime preinstalls the latest). Pinning is fine — and sometimes warranted — within that set:
+
+- A specific torch is known-good for your model (numerics, attention kernel availability, etc.).
+- A direct-URL CUDA wheel encodes a `torch2.X` tag (see "Prebuilt CUDA wheels" below) — pin torch to match.
+- A dep's `setup.py` would otherwise downgrade torch outside the supported set.
+
+`2.8.0` is the safest fallback for old requirements that refuse modern torch. `2.10.0` / `2.11.0` is the sweet spot for new code. When you pin torch, also pin `torchvision` / `torchaudio` to the matching minor — see the side-car drift section.
+
+When a dep would silently downgrade torch (e.g. some forks of `demucs`, `audiocraft` pin `torchaudio<2.1`), install the offender from `app.py` with `--no-deps` rather than pinning torch around it:
+
+```python
+import subprocess, sys
+subprocess.run([sys.executable, "-m", "pip", "install", "--no-deps",
+                "git+https://github.com/facebookresearch/demucs"], check=True)
+import spaces  # safe now — torch wasn't touched
+```
+
+List the offender's real runtime deps yourself in `requirements.txt`.
+
+## Torch-family side-car drift
+
+`torchvision`, `torchaudio`, `torchcodec` are built against a specific `torch` major.minor. Listing them unpinned **usually** works, but two known drift patterns:
+
+- `torchaudio==2.11.0` (and later) **dropped its `Requires-Dist: torch==X.Y.Z` line**. With torch pinned to 2.10, pip silently resolves torchaudio to 2.11.0 and the import fails on ABI mismatch.
+- `torchcodec` declares no torch dependency in PyPI metadata at all.
+
+Verification after `pip install` or `uv lock --upgrade`:
+
+```bash
+curl -s https://pypi.org/pypi/<pkg>/<version>/json \
+  | python3 -c "import json,sys,re; rd=json.load(sys.stdin)['info'].get('requires_dist') or []; \
+                print('\n'.join(x for x in rd if re.match(r'^torch(?![a-z])', x)) or '(no torch constraint)')"
+```
+
+When PyPI is silent, fall back to the project's README compatibility table (torchcodec's lives at https://github.com/pytorch/torchcodec).
+
+## Prebuilt CUDA wheels — the Blackwell wheels dataset
+
+This is the **first** thing to reach for when a CUDA-extension package has no upstream wheel matching the ZeroGPU torch / CUDA / cxx11-abi cell. Prefer it over any runtime workaround (`pip install git+…` inside `@spaces.GPU`, committed stub packages, `sys.modules` injection, monkey-patch shims), all of which are slower, fragile, and eat `duration` budget. Canonical prebuilt sm_120 wheels:
+
+> https://huggingface.co/datasets/multimodalart/zerogpu-blackwell-wheels
+
+Wheels live at `wheels/<cell>/<wheel>`. A **cell** encodes torch × CUDA × Python as `pt<torch>-cu<cuda>-cp<pyver>`. Every cell ships the same seven packages:
+
+`flash_attn` (two versions: `2.8.3` and `2.7.4.post1`), `xformers`, `pytorch3d`, `nvdiffrast`, `diff_gaussian_rasterization`, `torchmcubes`.
+
+Current cells (12):
+
+| torch | CUDA | Python cells available |
+|---|---|---|
+| 2.8.0 (`pt28`) | 12.8 | cp310, cp311, cp312 |
+| 2.9.1 (`pt291`) | 12.8 | cp310 |
+| 2.10.0 (`pt210`) | 12.8 / 13.0 | cp310 (cu128), cp312 (cu130) |
+| 2.11.0 (`pt211`) | 13.0 | cp312, cp313 |
+| 2.12.0 (`pt212`) | 13.0 | cp310, cp311, cp312, cp313 |
+
+### Picking a cell
+
+1. **Match `cp<pyver>` to your `python_version:`.** The wheels are cp-ABI-specific — a `cp310` wheel needs Python 3.10, `cp312` needs 3.12, etc. (`flash_attn` now ships cp310 **through** cp313, so this is a free choice, not a forced pin to 3.10 as in older versions of this doc.)
+2. **The wheels are torch-minor-tolerant.** The `flash_attn` / `xformers` filenames encode no torch version, so a `pt212-cu130-cp310` wheel runs fine on the live torch-2.11 cp310 runtime. `flash_attn` ships cp310 through cp313, so the Python choice is free. Pick the highest-torch cell for your Python version unless you've pinned an older torch — then match it (torch 2.8 → a `pt28-cu128-cp3XX` cell).
+3. **Copy the *exact* filename from the cell you pick.** The `xformers` build hash differs across cells (`0.0.34+3da0fc92…` on the cu130 / torch≥2.10 cells, `0.0.34+41531cee…` on the cu128 / torch 2.8–2.9 cells). Don't hardcode one filename across cells — list the cell (`hf download --repo-type dataset multimodalart/zerogpu-blackwell-wheels --include "wheels/<cell>/*"` or the Hub file browser) and copy what's there.
+
+Reference by direct URL in `requirements.txt`:
+
+```
+https://huggingface.co/datasets/multimodalart/zerogpu-blackwell-wheels/resolve/main/wheels/<cell>/<wheel>
+```
+
+### Per-package status
+
+Each package's version is constant across cells; only the `cp`/torch/CUDA tags in the filename change. The "instead of" column is the runtime workaround to avoid — the wheel is the clean path.
+
+| Package | Instead of | Version + caveats |
+|---|---|---|
+| `flash_attn` | committing a `flash_attn/` stub package; `sys.modules["flash_attn"] = …` injection | **2.8.3** (default) or **2.7.4.post1** (repos that pin `flash-attn<2.8`). Ships **cp310–cp313**. Needs `einops` for `flash_attn.layers.rotary`. Built `FLASH_ATTN_CUDA_ARCHS=120` (sm_120 only). This is FlashAttention-**2** — FA3/FA4 do **not** run on sm_120 (no TMEM); see [`zerogpu.md`](zerogpu.md) → Attention backends. Its real `flash_attn_2_cuda` also satisfies xformers' `flash_attn_gpu` probe. |
+| `xformers` | an MEA→SDPA monkey-patch shim; a Cutlass-force shim | **0.0.34** (`Requires: torch>=2.10`). **Build hash differs per cell** — copy the exact filename. Auto-dispatch picks FA2 (`fa2F`) on sm_120; classic Cutlass / FA3 reject sm_120 but auto-dispatch never selects them. |
+| `pytorch3d` | a runtime `pip install git+…pytorch3d.git` inside `@spaces.GPU` | **0.7.9**. Needs `numpy`, `iopath`, `fvcore` listed. No torch pin in metadata; loads cleanly on torch 2.11. |
+| `nvdiffrast` | a runtime build with `TORCH_CUDA_ARCH_LIST=12.0` | **0.4.0**. Needs `numpy`. `RasterizeGLContext` is a deprecation alias for `RasterizeCudaContext` — no headless-GL footgun. |
+| `diff_gaussian_rasterization` | a runtime build from `graphdeco-inria/diff-gaussian-rasterization.git` | **Upstream Inria API only** (returns 2-tuple `(color, radii)`). Does NOT match the ashawkey fork (4-tuple incl. alpha+depth) used by `ashawkey/LGM`, `dylanebert/LGM-mini`, etc. Forks need their own wheel. |
+| `torchmcubes` | a runtime `pip install git+…torchmcubes.git` | **0.1.0**. **sm_120 only** (no fatbin for older archs). Works on ZeroGPU / Blackwell; not portable to a dedicated T4 / L4 / A10G Space. |
+
+### Pattern
+
+Resolve the exact filenames from your chosen cell, then:
+
+```
+# requirements.txt  (cell = pt212-cu130-cp310 → needs python_version "3.10")
+numpy
+einops
+https://huggingface.co/datasets/multimodalart/zerogpu-blackwell-wheels/resolve/main/wheels/pt212-cu130-cp310/flash_attn-2.8.3-cp310-cp310-linux_x86_64.whl
+https://huggingface.co/datasets/multimodalart/zerogpu-blackwell-wheels/resolve/main/wheels/pt212-cu130-cp310/xformers-0.0.34+3da0fc92.d20260528-cp39-abi3-linux_x86_64.whl
+```
+
+```yaml
+# README frontmatter — pin Python to match the wheel cell's cp tag
+python_version: "3.10"
+```
+
+**Do not** install these from `@spaces.GPU` startup. A `subprocess.check_call` pip-install at first GPU acquire is strictly worse than the wheel URL — slower cold start, eats `duration` budget, breaks reproducibility, and the build sometimes exceeds the `@spaces.GPU(duration=1500)` cap.
+
+### When you need a wheel that's not in the dataset
+
+Three options, in preference order:
+
+1. **kernels-community** — https://huggingface.co/kernels-community handles ABI matching for you. Often the simplest path; no version pinning needed.
+2. **Upstream wheel matrix** — e.g. flash-attention's releases page ships a fairly complete `cu12 / torch / Python` matrix at https://github.com/Dao-AILab/flash-attention/releases. Pin `torch==X.Y.Z` in `requirements.txt` to match the wheel's `torch2.X` tag.
+3. **Build it yourself and host on HF Hub.** Last resort — see [`debugging.md`](debugging.md) for the in-`@spaces.GPU` source-build pattern as a stopgap while a wheel is being built.
+
+## Reading a CUDA wheel filename
+
+```
+flash_attn-2.8.3+cu130torch2.12cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
+```
+
+| Tag | Meaning |
+|---|---|
+| `cu130` | CUDA major version (13.0) |
+| `torch2.12` | torch major.minor the wheel was compiled against |
+| `cxx11abiFALSE` | C++ stdlib ABI choice (`TRUE` or `FALSE`) |
+| `cp310-cp310` | CPython version (3.10) |
+
+ABI / symbol mismatches at any of these → `ImportError` on first import. Pin `torch` to match `torch2.X`. Set `python_version:` to match `cp3XX`.
+
+## Don't pin `xformers`
+
+Leave bare in `requirements.txt` (or use the prebuilt URL above). Pip picks the wheel matching your installed torch.
+
+## Don't pin `spaces`
+
+Even if a `uv export` produces it, exclude with `--no-emit-package spaces`. The platform always pins its own version.
+
+## Specifically about Python version
+
+Pinning `python_version:` is effectively required:
+
+- ZeroGPU officially supports **3.10.13** and **3.12.12**.
+- The runtime default is 3.10.
+- Pinning to a `cp3XX` wheel matrix (e.g. `cp310` flash_attn wheel) forces matching Python.
+
+Both `"3.12"` and `"3.12.12"` forms are accepted in YAML.