mlclaw 0.1.0 → 0.2.2

package/assets/hf-tooling/skills/huggingface-spaces/references/gradio.md

@@ -0,0 +1,200 @@
+# Gradio for Spaces
+
+Patterns and quirks specific to running Gradio inside a Space. Assumes you're already comfortable with stock Gradio components and `gr.Blocks` / `gr.Interface`.
+
+For deeper Gradio API guidance — components, layouts, event listeners, chatbots, the Gradio 5→6 migration — use the dedicated `huggingface-gradio` skill. Install it with `hf skills add huggingface-gradio` (add `--claude --global` to also install for Claude Code, user-level).
+
+For ZeroGPU-specific decorator + worker semantics, see [`zerogpu.md`](zerogpu.md).
+
+## Themes and layout
+
+- Default theme preference: `gr.themes.Citrus()`. Alternatives: `gr.themes.Soft()` or no theme. Pick once and don't over-style.
+- For apps that don't need full-width, constrain with CSS so they're readable on 4K displays:
+  ```python
+  CSS = """
+  #col-container { max-width: 1100px; margin: 0 auto; }
+  .dark .gradio-container { color: var(--body-text-color); }
+  """
+  with gr.Blocks(theme=gr.themes.Citrus(), css=CSS) as demo: ...
+  ```
+  Always include the `.dark .gradio-container` override with `gr.themes.Citrus()` — without it Citrus's dark mode renders dark text on a dark background (text inherits unset colors). The same fix is harmless (and worth keeping) under other themes.
+- Width-cap with `!important` if Gradio 6's own breakpoints fight you — target `main`, `.gradio-container`, and the inner fillable wrapper, otherwise the width caps but goes flush-left.
+
+## Minimal layout most demos converge to
+
+```python
+with gr.Row():
+    prompt = gr.Textbox(show_label=False, placeholder="…", container=False, scale=4)
+    run = gr.Button("Run", variant="primary", scale=1)
+output = gr.Image(...)
+with gr.Accordion("Advanced settings", open=False):
+    ...
+```
+
+`container=False` on the inline Textbox removes the default outer border for a tighter look.
+
+## `gr.Markdown` for the intro
+
+Be succinct. Title, one-line description, a links section. Don't over-explain how it works — that's what the model card is for.
+
+## `gr.Examples`
+
+**Add `gr.Examples` whenever it makes sense** (the app takes user input and representative inputs exist). It's the first thing a visitor clicks and it doubles as an instant smoke test. Prefer examples that mirror the **official examples from the original repo / model card** — the prompts, images, and settings the authors showcase — over inventing your own.
+
+Keep each example row to the **few variables a user actually changes** (the prompt, the input image). Give the handler **default values for everything else** (steps, guidance, seed, …) so an example row is `["a cat on a windowsill"]`, not a wall of knobs. Design the signature so the interesting inputs come first and the rest default:
+
+```python
+@spaces.GPU(duration=60)
+def generate(prompt, seed=42, steps=28, guidance=5.0):   # only `prompt` varies in examples
+    ...
+
+gr.Examples(
+    examples=[
+        ["a cat sitting on a windowsill"],
+        ["mountains at sunset, photorealistic"],
+    ],
+    inputs=[prompt],          # just the varied inputs; the rest use the handler defaults
+    outputs=output,
+    fn=generate,
+    cache_examples=True,
+    cache_mode="lazy",
+)
+```
+
+Why these flags:
+
+- `cache_examples=True` makes example clicks instant (no re-inference per visitor).
+- **`cache_mode="lazy"`** — caches on first user click for each example. **Required on ZeroGPU** (Gradio's default on ZeroGPU is already lazy via `GRADIO_CACHE_MODE=lazy`). Eager would pre-run every example at app startup, but ZeroGPU has no GPU attached at startup — it'd fail and burn the creator's daily quota.
+- `cache_examples=True` silently disables `run_on_click` / `run_examples_on_click`. If your app relies on click-only behavior, set `cache_examples=False`.
+
+The cache key is the **example row's file path**, not a content hash. Regenerating an asset in place serves the stale cached output forever. If you replace example files, bump a `cache_version` marker or wipe `.gradio/cached_examples/<id>/`.
+
+Hot-reload (rung 1) does **not** rebuild the cache. A `cache_version` bump needs a real commit + restart.
+
+## Streaming and generators
+
+`gr.Interface(fn=...)` and `.click(fn=...)` both accept generator functions. Each `yield` pushes a new value:
+
+```python
+@spaces.GPU(duration=120)
+def generate(prompt):
+    yield gr.update(value=None, label="Starting…")
+    for k in range(num_steps):
+        yield gr.update(value=preview(k), label=f"Step {k+1}/{num_steps}")
+    yield gr.update(value=final, label="Done")
+```
+
+Use `gr.update(label=...)` for status narration — feels like a status line without a separate component.
+
+**Caveat**: `gr.Progress(track_tqdm=True)` and `yield` partial outputs fight each other — pick one streaming mechanism.
+
+## Useful UX bits
+
+- `progress=gr.Progress(track_tqdm=True)` in the GPU function gives a free tqdm-driven progress bar.
+- Pair a "Randomize seed" checkbox with the seed input, and write the actually-used seed back so users can re-run deterministically:
+  ```python
+  randomize = gr.Checkbox(label="Randomize seed", value=True)
+  seed = gr.Number(label="Seed", value=0, precision=0)
+
+  @spaces.GPU
+  def gen(..., seed, randomize_seed):
+      if randomize_seed:
+          seed = random.randint(0, 2**31 - 1)
+      seed = int(seed)
+      yield ..., gr.update(value=seed)   # write back into the Seed input
+      ...
+
+  run.click(gen, inputs=[..., seed, randomize], outputs=[..., seed])
+  ```
+
+## Custom HTML components
+
+Stock Gradio components cover 95% of cases. When they don't, `gr.HTML(...)` lets you build contextual custom UI without leaving the Gradio Space (no need to switch to Docker).
+
+Guide: https://www.gradio.app/guides/custom-HTML-components
+Example with a 3D camera-angle picker that makes sense in context: https://huggingface.co/spaces/multimodalart/qwen-image-multiple-angles-3d-camera
+
+Don't reach for this if a stock component covers the need.
+
+## Custom frontends — `gr.Server`
+
+For fully custom frontends with their own HTML/JS, while keeping Gradio's queue + GPU scheduling:
+
+```python
+from gradio import Server
+from fastapi.responses import HTMLResponse
+
+app = Server(title="my-app")
+
+@spaces.GPU(duration=60)
+def _run_gpu(prompt): return inference(prompt)
+
+@app.api(name="generate", concurrency_limit=1, time_limit=180)
+def generate(prompt: str) -> str:
+    return _run_gpu(prompt)        # @app.api wraps, doesn't stack with @spaces.GPU
+
+@app.get("/", response_class=HTMLResponse)
+async def homepage():
+    return open("index.html").read()
+
+demo = app                          # HF runtime expects `demo`
+if __name__ == "__main__":
+    demo.launch(ssr_mode=False)
+```
+
+Required for the custom `/` route to actually serve:
+
+```bash
+hf spaces variables add <ns>/<name> --env GRADIO_SSR_MODE=false
+```
+
+`launch(ssr_mode=False)` is ignored on HF — must be the env var.
+
+Valid `@app.api` kwargs: `name`, `description`, `concurrency_limit`, `concurrency_id`, `queue`, `batch`, `max_batch_size`, `api_visibility`, `time_limit`, `stream_every`.
+
+**Don't stack `@spaces.GPU` and `@app.api`** on the same function — silently breaks request flow. Keep them on separate functions.
+
+**Two-ceiling coordination**: both `@spaces.GPU(duration=N)` and `@app.api(time_limit=M)` apply; the lower wins. Set `time_limit` to your max duration across modes — a too-low `time_limit` kills the request even if GPU duration would have allowed it.
+
+Hot-reload (rung 1 in [`debugging.md`](debugging.md)) does **not** work with `gr.Server` — always `hf upload` or commit.
+
+Reference Space: https://huggingface.co/spaces/huggingface-projects/rf-detr-realtime-webcam
+
+## Slow-startup Gradio (big-model Spaces)
+
+For Spaces that take 10–20 min to load weights:
+
+- Set `startup_duration_timeout: 1h` in README frontmatter (default 30 min).
+- Disable SSR: `hf spaces variables add <id> --env GRADIO_SSR_MODE=false`. Otherwise the SSR health check times out before the app finishes loading.
+
+## Expose the Space as an MCP server
+
+Launch with `demo.launch(mcp_server=True)` (Gradio 5+) so every API endpoint is also exposed as an MCP tool — free, and makes the Space usable by agents. For it to be useful:
+
+- **Every API-triggered function needs a docstring and type hints.** Each Gradio event handler is auto-exposed over the API; Gradio turns the signature into the MCP tool's input schema and the docstring into its description. A function without them still works but surfaces as an opaque, unusable tool.
+- Give the important endpoints stable names (`api_name="generate"` on the `.click(...)`, or `@app.api(name=...)`), so tool names don't churn.
+- MCP mode can pull in extra deps (`gradio[mcp]`); if `launch` complains, see the pin note in [`known-errors.md`](known-errors.md).
+
+```python
+@spaces.GPU(duration=60)
+def generate(prompt: str, seed: int = 42) -> str:
+    """Generate an image from a text prompt.
+
+    Args:
+        prompt: what to generate.
+        seed: RNG seed for reproducibility.
+    """
+    ...
+
+btn.click(generate, inputs=[prompt, seed], outputs=out, api_name="generate")
+demo.launch(mcp_server=True)
+```
+
+## Don't
+
+- `gr.Button(text="X")` — `text=` was removed; use `gr.Button("X")`.
+- `gr.Button(type="button")` — drop the kwarg.
+- `.click(..., _js=share_js)` — renamed to `js=`.
+- `.style(height=...)` — removed in gradio 4+.
+- `gr.ImageMask(brush_color=...)` — kwarg removed.
+- `demo.launch(mcp_server=True)` on gradio 4.x — only valid on 5+.

package/assets/hf-tooling/skills/huggingface-spaces/references/grants.md

@@ -0,0 +1,56 @@
+# Community GPU grants
+
+When a non-PRO user has a good use case for ZeroGPU (open research demo, hobbyist project, educational tool, institutional showcase) and doesn't want to subscribe, they can request a free community grant from Hugging Face.
+
+## The flow
+
+1. **Build the Space.** Create it as `--flavor cpu-basic` (the user is not PRO, so creating with `--flavor zero-a10g` will fail at `create_repo`). Code the app for ZeroGPU anyway — `import spaces`, `@spaces.GPU`, module-scope `.to("cuda")`. The Space will technically run on CPU until the grant is approved, but it'll be ready to switch over instantly.
+
+   In this mode you **can't iterate-with-real-inference** before the grant — the CPU Space won't actually run heavy compute. Get the app to BUILD cleanly and `RUNNING` (even if the runtime would OOM on real input), then submit.
+
+2. **Submit a Community Tab discussion** on the Space. Title:
+
+   ```
+   Apply for a GPU community grant: <Personal|Company|Academic> project
+   ```
+
+   Pick the closest fit. Body:
+
+   ```
+   Description of the app: one paragraph on what it does + who it's for.
+   Justification: one paragraph on why this should run on ZeroGPU
+   (open-source, research, educational, etc.). 
+   ```
+
+   If the user didn't give you a justification, a reasonable default is "Non-PRO wants to build a public ZeroGPU app — happy to provide more context if helpful."
+
+3. **Wait.** Open and publicly-facing applications by researchers, tinkerers, and institutions are typically approved. Approval can take days.
+
+4. **Once approved**, the Space is moved to ZeroGPU automatically — no code change needed. The user comes back and you can iterate / refine with real GPU access.
+
+## When to suggest this
+
+- User is not on PRO but their use case is a clear fit for ZeroGPU (a public ML demo, not a private tool).
+- The model fits in `large` (≤ 48 GB VRAM at the chosen precision).
+
+## When NOT to suggest this
+
+- Private / commercial / closed-source projects — push the user toward PRO instead.
+- The model genuinely needs dedicated paid hardware (huge LLM, vLLM/JAX/ONNX as main model with heavy init) — `canPay=True` users can use paid flavors directly.
+- The user is already PRO — they have ZeroGPU access; no grant needed.
+
+## Posting the request programmatically
+
+```python
+from huggingface_hub import HfApi
+
+api = HfApi(token="hf_...")
+api.create_discussion(
+    repo_id="<ns>/<space>",
+    repo_type="space",
+    title="Apply for a GPU community grant: Personal project",
+    description="<description and justification>",
+)
+```
+
+The Community Tab must be enabled on the Space (default — keep it on).

package/assets/hf-tooling/skills/huggingface-spaces/references/inference-providers.md

@@ -0,0 +1,85 @@
+# Inference Providers — when not to host the model
+
+Some Spaces don't need a GPU at all. If the model is available through HF Inference Providers (Cerebras, Fireworks, Together, Replicate, OpenRouter, etc.), the Space can be a thin Gradio shell that proxies to a hosted endpoint:
+
+- Zero VRAM, no `@spaces.GPU`, no model download.
+- Works for models too large to fit on ZeroGPU (120B+).
+- Hardware can be `cpu-basic` — no GPU at all.
+
+## When to use this pattern
+
+- **Stateless chat or text completion** with a big model.
+- **The user wants a public demo of a frontier-scale model** that obviously doesn't fit on a single 48 GB MIG.
+- **The user wants to ship something fast** without worrying about quantization / sharding.
+
+## When NOT to use this pattern
+
+- The model isn't available on any Inference Provider. Check with:
+  ```bash
+  curl "https://huggingface.co/api/models/<ns>/<repo>?expand[]=inferenceProviderMapping"
+  ```
+- The Space needs **custom decoding** (special sampling, tool use, retrieval, anything stateful or interactive across calls).
+- The Space needs **multimodal** beyond what the provider exposes.
+- The user explicitly wants to own the inference stack (model loading, decoding, performance tuning).
+
+For those, host the model yourself on ZeroGPU — see [`zerogpu.md`](zerogpu.md).
+
+## Two billing modes
+
+Choose based on who pays for inference.
+
+### Mode A — Space creator pays (simple)
+
+Set `HF_TOKEN` as a Space secret. The Space uses `InferenceClient` directly. Every visitor's call is billed to the Space creator's account.
+
+```python
+import os, gradio as gr
+from huggingface_hub import InferenceClient
+
+client = InferenceClient(api_key=os.environ["HF_TOKEN"], provider="fireworks-ai")
+
+def chat(msg, history):
+    return client.chat_completion(
+        model="<org>/<model>",
+        messages=[*history, {"role": "user", "content": msg}],
+        max_tokens=512,
+    ).choices[0].message.content
+
+gr.ChatInterface(chat).launch()
+```
+
+Use when you want users to "just click and try it" — no sign-in friction. Cost is on you.
+
+### Mode B — Visitor pays (recommended for public demos)
+
+`gr.LoginButton` + `gr.load("models/...")` with `accept_token=button`. Each visitor signs in with their HF account; inference is billed to **their** account.
+
+```python
+import gradio as gr
+
+with gr.Blocks(fill_height=True) as demo:
+    with gr.Sidebar():
+        button = gr.LoginButton("Sign in")
+    gr.load("models/<org>/<model>", accept_token=button, provider="fireworks-ai")
+demo.launch()
+```
+
+README frontmatter needs:
+
+```yaml
+hf_oauth: true
+hf_oauth_scopes:
+  - inference-api
+```
+
+This is the **recommended pattern for public demos** — sustainable cost-wise, and visitors get to use their own provider quotas (which most have paid for or get free).
+
+## Hardware
+
+`cpu-basic`. No GPU. Don't put `--flavor zero-a10g` — you'd waste a paid grant.
+
+## Anti-pattern: `@spaces.GPU` wrapping a provider call
+
+If you do use Inference Providers, do **not** wrap the call in `@spaces.GPU`. The decorator reserves a GPU slot on your Space for the full `duration=`, but the function does no GPU work — just an HTTP call out. You burn your own ZeroGPU quota for nothing.
+
+A provider-proxy Space wants `cpu-basic` hardware and zero `@spaces.GPU`.

package/assets/hf-tooling/skills/huggingface-spaces/references/known-errors.md

@@ -0,0 +1,232 @@
+# Known errors
+
+Check if this is a known issue before trying your own fix. Entries are keyed by the substring that actually appears in `runtime.errorMessage`, the build log, or a Python traceback — grep this file for the error you saw.
+
+If you hit something not listed here and figure out a fix, please ask your human to PR it back so future runs benefit.
+
+---
+
+## Build / config errors
+
+These come from the Space build pipeline before the app starts. Read with `hf spaces logs <id> --build --tail 500` — find the **first** error, not the last.
+
+### `CONFIG_ERROR: torch version in requirements.txt is not compatible with ZeroGPU`
+
+**Cause**: `requirements.txt` pins `torch==X.Y.Z` to a version outside the supported set (`2.8.0`, `2.9.1`, `2.10.0`, `2.11.0`).
+**Fix**: Unpin torch (preferred — the runtime preinstalls the latest supported version), or pin to one of the supported values.
+
+### `Cannot install … because these package versions have conflicting dependencies` / `ResolutionImpossible`
+
+**Cause**: A dep conflicts with the Gradio SDK pinned by `sdk_version:` in README. Most commonly `pydantic`, `uvicorn`, `huggingface_hub`, or `jinja2` pinned to old values that the SDK no longer accepts.
+**Fix**: Unpin the offender. For `gradio[mcp]` specifically, `uvicorn>=0.31.1` and `pydantic>=2.11.10` are required.
+
+### Build hangs in dependency resolution > 10 min
+
+**Cause**: pip backtracking through a deep version space.
+**Fix**: Pin the conflicting transitive dep. The `--build` logs will show which one. Bump `startup_duration_timeout: 1h` in README frontmatter if heavy downloads are expected.
+
+### `ModuleNotFoundError: No module named 'pkg_resources'`
+
+**Cause**: setuptools 81 dropped `pkg_resources`; an old package's `setup.py` imports it.
+**Fix**: Bump or unpin the offender. Typical culprits: `deepspeed==0.15.x` (training-only — usually safe to drop from inference Spaces), `openai-whisper==20231117`.
+
+### `400 Bad Request` from `/api/validate-yaml` during `create_repo` / `upload_file`
+
+**Cause**: README frontmatter failed server validation. Most common: `short_description` over the (undocumented) character cap — target ≤ 60.
+**Fix**: Shorten `short_description`. Long descriptions go in the README body. Also double-check `colorFrom`/`colorTo` are one of `red|yellow|green|blue|indigo|purple|pink|gray`.
+
+### `403 Forbidden` from `create_repo` with `space_hardware="zero-a10g"`
+
+**Cause**: The user isn't on PRO / Team / Enterprise so the API rejects ZeroGPU at creation.
+**Fix**: Retry without `space_hardware=`. Keep `hardware:` out of README frontmatter (silently ignored anyway). The Space is created on CPU; point the user at PRO upgrade or [community grant](grants.md).
+
+### `403 Forbidden` from `create_commit(..., create_pr=True)`
+
+**Cause**: Upstream Space has Discussions disabled.
+**Fix**: Ask the maintainer to enable Discussions, or push directly if you have write access.
+
+---
+
+## Startup / RUNTIME_ERROR
+
+These come from `hf spaces logs <id> --tail 500`.
+
+### `RuntimeError: CUDA has been initialized before importing the spaces package`
+
+**Cause**: Something triggered CUDA init in the main process before `import spaces`. Usually wrong import order; sometimes a third-party lib eagerly initializing CUDA at import time (e.g. `numba.cuda`).
+**Fix**: Reorder so `import spaces` is first. For numba-using stacks (NeMo, RAPIDS bits):
+```python
+import os
+os.environ.setdefault("NUMBA_DISABLE_CUDA", "1")
+import spaces
+```
+
+### `RuntimeError: No @spaces.GPU function detected during startup`
+
+**Cause**: The function bound to `.click(fn=...)` / `.submit(...)` isn't decorated. Decorating an inner helper doesn't count — the startup scan only walks Gradio's registered handlers.
+**Fix**: Decorate the function Gradio binds. If that conflicts with another decorator, wrap explicitly:
+```python
+@spaces.GPU(duration=60)
+def gpu_inner(...): ...
+def gradio_handler(...): return gpu_inner(...)
+```
+(Or just decorate `gradio_handler` directly.)
+
+### `ImportError: cannot import name 'HfFolder' from 'huggingface_hub'`
+
+**Cause**: Old gradio (`4.44` and similar) imports `HfFolder` from `huggingface_hub`, which was removed in recent hub releases.
+**Fix**: Two options.
+- Pin `huggingface-hub==0.25.0` in `requirements.txt` (keeps old gradio happy).
+- Bump `sdk_version` in README to `5.x` or `6.x` (also fixes a lot of other API breaks).
+If a Gradio custom component locks the major (`gradio-image-prompter`, `gradio_litmodel3d`, …), install it with `--no-deps` so its `gradio<5.0` requirement doesn't bind.
+
+### `ImportError: cannot import name 'is_traceable_wrapper_subclass' from 'torch.utils._python_dispatch'`
+
+**Cause**: A dep with `torchaudio<2.1` / `torch<2` in its `setup.py` (e.g. `demucs`, `audiocraft`) downgraded torch silently. The build succeeded, the app booted, and `import spaces` then died on a missing torch symbol.
+**Fix**: Install the offender from `app.py` with `--no-deps` *before* `import spaces`:
+```python
+import subprocess, sys
+subprocess.run([sys.executable, "-m", "pip", "install", "--no-deps",
+                "git+https://github.com/facebookresearch/demucs"], check=True)
+import spaces
+```
+List its actual runtime deps (`dora-search einops julius lameenc openunmix pyyaml tqdm` for demucs) yourself in `requirements.txt`.
+
+### `_pickle.UnpicklingError: Weights only load failed`
+
+**Cause**: `torch.load` weights-only default flipped to `True` in torch 2.6. Old checkpoints pickling numpy/object globals fail.
+**Fix**: For trusted upstream checkpoints, monkey-patch before the package import:
+```python
+import torch
+_orig = torch.load
+torch.load = lambda *a, **k: _orig(*a, **{**k, "weights_only": k.get("weights_only", False)})
+```
+
+### Stuck at `ZeroGPU init – 10.0%` then 60 s timeout
+
+**Cause**: A library called `cuInit` in the parent process, poisoning the fork (most often `numba.cuda` via NeMo). The actual `@spaces.GPU` body never starts.
+**Fix**: `os.environ.setdefault("NUMBA_DISABLE_CUDA", "1")` as the first line of `app.py`, before `import spaces`.
+
+### `RUNTIME_ERROR` right after long `APP_STARTING`, logs sparse
+
+**Cause**: Boot exceeded `startup_duration_timeout` (default 30 min). Big-model loads commonly trigger this.
+**Fix**: Bump `startup_duration_timeout: 1h` in README frontmatter. For Gradio 6 specifically, also set `GRADIO_SSR_MODE=false` via `hf spaces variables add <id> --env GRADIO_SSR_MODE=false` to dodge SSR health-check timeouts during slow boot.
+
+### `RUNNING` but the public URL returns 404
+
+**Cause**: The Space is private. Anonymous Client / browser hits return 404.
+**Fix**: Authenticate. `gradio_client.Client(space, token=os.environ["HF_TOKEN"])`. The kwarg is `token=`, not `hf_token=`.
+
+### `workload was not healthy after 30 min`
+
+**Cause**: Infra-side scheduling or a build that genuinely can't finish in time.
+**Fix**: Usually not actionable in code. Bump `startup_duration_timeout` if heavy downloads are expected; otherwise wait or report.
+
+### Exit code 128 / containerd / scheduling failure
+
+**Cause**: HF infra glitch.
+**Fix**: `hf spaces restart <id> --factory-reboot`. If it persists, retry later or report. Not fixable in code.
+
+---
+
+## Inference-time errors
+
+These appear in `hf spaces logs <id> --follow` while a request is running.
+
+### `ZeroGPU illegal duration`
+
+**Cause**: `@spaces.GPU(duration=N)` is larger than the visitor's tier per-call cap.
+**Fix**: Lower `N`. Tier caps live in the [ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu).
+
+### `ZeroGPU quota exceeded (X requested vs Y left)`
+
+**Cause**: The visitor's remaining quota < `requested duration`. The comparison is `requested vs remaining`, not `actual vs remaining` — a 10-second task left at the default 60 s blocks the user as soon as their remaining drops below 60 s.
+**Fix**: Lower `duration` to the realistic worst case. For input-dependent runtime, use a callable estimator.
+
+### `RuntimeError: NVML_SUCCESS == r INTERNAL ASSERT FAILED at .../CUDACachingAllocator.cpp`
+
+**Cause**: Allocator fragmentation under transient memory spikes (high-res pixel-space ops, large attention activations, SR models, video DiTs). Not a clean OOM.
+**Fix**: Set expandable segments at the **very top** of `app.py`, before any torch import:
+```python
+import os
+os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
+import spaces
+import torch
+```
+Usually a single-line fix that replaces lowering resolution or moving to `xlarge`.
+
+### Call hangs forever on first `@spaces.GPU` entry
+
+**Cause**: The decorated function returned a CUDA tensor. Unpickling it in the main process triggers `torch.cuda._lazy_init()`, which ZeroGPU blocks.
+**Fix**: Convert to CPU before returning: `return tensor.cpu()` or `.cpu().numpy()`. For `gr.State`, scrub before yielding.
+
+### `PicklingError` at call entry
+
+**Cause**: An argument crossing the fork boundary contains an unpicklable object — file handle, lock, lambda, closure, or `gr.SelectData`.
+**Fix**: Extract the picklable fields in a thin un-decorated wrapper, pass plain values to the `@spaces.GPU` function. For `gr.SelectData` specifically, pull out `evt.index[0]`, `evt.index[1]` etc. outside the decorator.
+
+### `RecursionError` inside `gr.SelectData.__getattr__`
+
+**Cause**: Same as above — `gr.SelectData` doesn't survive pickle.
+**Fix**: Same — extract its fields before crossing the boundary.
+
+### `CUDA error … flash_fwd_launch_template.h: no kernel image is available for execution on the device` (or `:188: invalid argument`)
+
+**Cause**: A Flash Attention 3 kernel was loaded — directly via `kernels-community/{flash-attn3,vllm-flash-attn3,sgl-flash-attn3}`, or indirectly via an old `xformers` wheel that auto-dispatched to FA3. FA3 has no Blackwell sm_120 build.
+**Fix**: Use `attn_implementation="sdpa"`, or `"flash_attention_2"` with the FA2 wheel from `multimodalart/zerogpu-blackwell-wheels`. For xformers, the prebuilt wheel from the same dataset auto-dispatches to FA2 — no monkey-patch.
+
+### `NotImplementedError: sgl_flash_attn3 is only supported on sm80 and above with CUDA >= 12.3`
+
+**Cause**: `kernels-community/sgl-flash-attn3` rejects sm_120 at runtime despite the error wording. Same root cause as the FA3 entry above.
+**Fix**: Same — SDPA or FA2.
+
+### `ImportError: cannot import name 'flash_attn_varlen_func' from 'flash_attn'` / model insists on `attn_implementation="flash_attention_2"`
+
+**Cause**: The model imports flash_attn at module top with no escape hatch, and the runtime doesn't ship it.
+**Fix**: Install the prebuilt `flash_attn-2.8.3-cp310-cp310-linux_x86_64.whl` from `multimodalart/zerogpu-blackwell-wheels`. Requires `python_version: "3.10"` in README (wheel is cp310 only). Real `flash_attn_2_cuda` satisfies xformers' `flash_attn_gpu` probe too.
+
+For transformers `AutoModel`-style configs that aren't a hard import, swap `attn_implementation="flash_attention_2"` → `"sdpa"`. Torch-native, zero deps.
+
+### `selective_scan_cuda.so undefined symbol` / `_torchaudio.abi3.so undefined symbol`
+
+**Cause**: A direct-URL prebuilt CUDA wheel pinned to an old torch ABI (`cu12torch2.4cxx11abiFALSE`, etc.) — won't load on the current runtime.
+**Fix**: Drop the URL-pinned wheel from `requirements.txt`. Use a torch-current wheel from `multimodalart/zerogpu-blackwell-wheels`, kernels-community, or upstream's release page.
+
+### `TypeError: 'dict' object is not hashable` inside `jinja2/utils.py:get`
+
+**Cause**: Old gradio (4.44) + modern starlette / jinja2 cache clash.
+**Fix**: Either pin `jinja2<3.2` + `starlette<0.40` (keeps old gradio), or bump `sdk_version` past 5.0.
+
+---
+
+## Smoke-test / client errors
+
+### `Client.__init__() got an unexpected keyword argument 'hf_token'`
+
+**Cause**: Older `gradio_client` API used `hf_token=`; current uses `token=`.
+**Fix**: `Client(space, token=os.environ["HF_TOKEN"])`.
+
+### `httpx.ReadTimeout` on `client.predict(...)`
+
+**Cause**: Default timeout too small for the GPU duration.
+**Fix**: `Client(..., httpx_kwargs={"timeout": 600})`. Set this to at least your `@spaces.GPU(duration=N)` plus 60 s.
+
+### `404` on what looks like a valid endpoint name
+
+**Cause**: Streaming endpoint (function uses `yield`), or a custom `gr.Server` `@app.get/post` route that doesn't appear in `view_api()`.
+**Fix**: For streaming, use `client.submit(...).result()` (or iterate the job). For custom routes, use `httpx.post(base_url + "/route", ...)` directly — bypass `gradio_client`.
+
+### Result file looks empty / dimensions wrong
+
+**Cause**: HTTP 200 ≠ correct output. The model snapped to a different size, or the wrong endpoint was hit.
+**Fix**: Sniff the returned file's magic bytes (`glTF`, `\x89PNG`, `RIFF…WEBP`, `RIFF…WAVE`, `[4:8]==b"ftyp"`) and check returned dimensions match what was requested.
+
+---
+
+## Submitting new entries
+
+If you hit an error not in this file and figure out the fix, please ask your human to PR it back to this skill. Format:
+
+- A 1-line heading with the exact error substring an agent would grep for.
+- **Cause**: one sentence on what triggered it.
+- **Fix**: concrete commands or code. If the fix needs more than 5 lines of narrative, point to another reference file (e.g. [`debugging.md`](debugging.md), [`zerogpu.md`](zerogpu.md)) for depth.