opencode-skills-collection 3.1.12 → 3.1.13

package/bundled-skills/huggingface-lora-space-builder/references/base-models/ltx.md

@@ -0,0 +1,205 @@
+# LTX reference
+
+The LTX family covers `LTX-Video` (0.9.x series), `LTX-2`, and `LTX-2.3`. All target text-to-video, image-to-video, and video-to-video.
+
+Diffusers support varies across versions:
+
+| Base model series         | Diffusers support                        | LoRA loading via diffusers? |
+|---------------------------|------------------------------------------|-----------------------------|
+| `Lightricks/LTX-Video` (0.9.x) | Yes — `LTXPipeline`, `LTXImageToVideoPipeline`, `LTXConditionPipeline` | Yes |
+| `Lightricks/LTX-2`        | Yes — `LTX2Pipeline` from `diffusers.pipelines.ltx2` | Yes (recent diffusers) |
+| `Lightricks/LTX-2.3`      | Partial — diffusers support is still limited overall, so the original `Lightricks/LTX-2.3` repo can still be used via the native path. Diffusers-converted variants exist at `dg845/LTX-2.3-Diffusers` and `dg845/LTX-2.3-Distilled-Diffusers`, supporting regular LoRAs via the standard pipelines, IC LoRAs via `LTX2InContextPipeline`, and HDR IC-LoRAs via `LTX2HDRPipeline` (PR #13572, merged 2026-05-15; needs `git+https://github.com/huggingface/diffusers`) | Yes on the diffusers variants — standard `load_lora_weights` + `set_adapters`. Native path still required for some configurations. |
+
+When in doubt about LTX-2.3, check the LoRA's model card for an example snippet. If it imports from `ltx_video` or a native module rather than `diffusers`, use the native path.
+
+## Pipelines (diffusers path)
+
+> **Before using this table, verify against the base model's own card on the Hub.** This table is best-effort and can lag a recent release (LTX moves fast). The diffusers snippet on the base model's Hub page is source of truth. See `SKILL.md` Phase 2 for the procedure.
+
+| Task                    | Pipeline                          | Pretrained ID                                        |
+|-------------------------|-----------------------------------|------------------------------------------------------|
+| Text-to-video           | `LTXPipeline`                     | `Lightricks/LTX-Video`                               |
+| Image-to-video          | `LTXImageToVideoPipeline`         | `Lightricks/LTX-Video`                               |
+| Video-to-video / keyframes | `LTXConditionPipeline`         | `Lightricks/LTX-Video-0.9.5` or later                |
+| Spatial upscale         | `LTXLatentUpsamplePipeline`       | `Lightricks/ltxv-spatial-upscaler-0.9.8`             |
+| LTX-2 T2V/I2V           | `LTX2Pipeline`                    | `Lightricks/LTX-2`                                   |
+| LTX-2.3 in-context (IC LoRAs) | `LTX2InContextPipeline`     | `dg845/LTX-2.3-Distilled-Diffusers` (preferred for demos; non-distilled `dg845/LTX-2.3-Diffusers` also works) |
+
+`LTXConditionPipeline` is the workhorse for V2V — it takes a `LTXVideoCondition` (or list of them) plus optional first-frame image, and produces a video conditioned on the input.
+
+## Required dependencies
+
+LTX pipelines need extras beyond the standard diffusers/transformers/peft set, because video output requires file-format support:
+
+- **`imageio`** and **`imageio-ffmpeg`** — required by `diffusers.utils.export_to_video`. Without them, video export fails at runtime even though model loading succeeds. Always include both.
+- **`sentencepiece`** — required by the T5 text encoder some LTX variants use. Include if you see tokenizer-related ImportErrors at startup.
+- **`av`** — `pyav`, useful when reading input videos in non-trivial formats. Include for V2V pipelines that take video input via `load_video`.
+
+For LTX-2 and LTX-2.3, the latest diffusers from git is often required since pipeline classes (`LTX2Pipeline`, conditioning APIs) land before pip releases:
+
+```
+git+https://github.com/huggingface/diffusers
+```
+
+For LTX-2.3 native path:
+
+```
+git+https://github.com/Lightricks/LTX-Video.git
+```
+
+If `from_pretrained` fails with class-not-found errors for a recent LTX variant, switch to git diffusers.
+
+## Default load (T2V)
+
+```python
+import torch
+from diffusers import LTXPipeline
+from diffusers.utils import export_to_video
+
+pipe = LTXPipeline.from_pretrained(
+    "Lightricks/LTX-Video",
+    torch_dtype=torch.bfloat16,
+)
+pipe.to("cuda")
+pipe.load_lora_weights("user/my-ltx-lora")
+```
+
+## Default load (V2V via LTXConditionPipeline)
+
+```python
+import torch
+from diffusers import LTXConditionPipeline
+from diffusers.pipelines.ltx.pipeline_ltx_condition import LTXVideoCondition
+from diffusers.utils import load_video, export_to_video
+
+pipe = LTXConditionPipeline.from_pretrained(
+    "Lightricks/LTX-Video-0.9.5",
+    torch_dtype=torch.bfloat16,
+)
+pipe.to("cuda")
+pipe.load_lora_weights("user/my-ltx-vlora")
+
+video = load_video("input.mp4")
+condition = LTXVideoCondition(video=video, frame_index=0)
+
+frames = pipe(
+    conditions=[condition],
+    prompt="...",
+    negative_prompt="worst quality, jittery, blurry",
+    width=768, height=512,
+    num_frames=121,
+    num_inference_steps=50,
+).frames[0]
+export_to_video(frames, "out.mp4", fps=24)
+```
+
+## LTX-2 (diffusers path)
+
+```python
+from diffusers.pipelines.ltx2 import LTX2Pipeline
+pipe = LTX2Pipeline.from_pretrained("Lightricks/LTX-2", torch_dtype=torch.bfloat16)
+```
+
+LTX-2 supports a two-stage pipeline (base + latent upsample) for production quality. For a demo, the single-stage pipeline is usually sufficient and faster.
+
+**Don't default two-stage on for IC-LoRAs — check the card.** The common LTX-2.3 two-stage recipe runs stage 2 as an x2 latent-upsample + refine **with the IC-LoRA disabled** (`disable_lora()`), re-rendering on the bare base — which re-degrades exactly the LoRA-conditioned detail (text, identity). Whether two-stage is appropriate is per-LoRA, per the card: e.g. the *In/Out-painting* card explicitly says "both tasks use a two-stage pipeline," whereas the *Ingredients* card recommends a single-pass 30-step recipe and never mentions two stages. Single-stage is the safer demo default unless the card calls for two-stage.
+
+## Inference defaults
+
+- **Resolution**: typically multiples of 32. Common sizes: 768×512, 1216×704, 704×480.
+- **Frame count**: typically `8k+1` (e.g. 121, 161, 257). Compute from `duration_seconds * fps` and round.
+- **fps**: 24 by default; some LoRAs are trained at different rates (16, 30) — check the model card.
+- **`num_inference_steps`**: 30–50 for non-distilled. Distilled checkpoints (look for "distilled" in the name) often run at 8–12.
+- **`negative_prompt`**: LTX is sensitive to negative prompts. A good default: `"worst quality, inconsistent motion, blurry, jittery, distorted"`.
+- **Distilled IC-LoRA — disable audio guidance too, not just `guidance_scale`.** `LTX2InContextPipeline` computes `do_classifier_free_guidance = guidance_scale > 1 OR audio_guidance_scale > 1`, and **`audio_guidance_scale` defaults to `7.0`**. So `guidance_scale=1.0` is not enough — CFG stays on via audio (and `stg_scale` defaults on), which doubles/mis-batches the forward against the in-context reference tokens (wrong recipe, sometimes a runtime error). For a distilled IC-LoRA pass all four: `pipe(..., guidance_scale=1.0, stg_scale=0.0, audio_guidance_scale=1.0, audio_stg_scale=0.0)`.
+
+## Frame-count helper
+
+```python
+def num_frames_for_duration(seconds, fps=24, base=8):
+    raw = seconds * fps
+    return ((int(raw) - 1) // base) * base + 1
+```
+
+## Native pipeline path (LTX-2.3 fallback)
+
+The native repo is a fallback when the diffusers path doesn't work, or for specific conditionings that diffusers may not support yet. Try `LTX2InContextPipeline` on the diffusers path first (see the IC-LoRAs section above).
+
+**The native repo depends on the model generation** — pick by the LoRA's base model (and what its card's snippet imports):
+
+| Base model | Native repo | Package(s) / import | Pipeline classes |
+|---|---|---|---|
+| `LTX-Video` (0.9.x) | `github.com/Lightricks/LTX-Video` | `ltx_video` | `from ltx_video.pipelines import LTXPipeline` |
+| `LTX-2`, `LTX-2.3` | `github.com/Lightricks/LTX-2` | `ltx-core` + `ltx-pipelines` (editable) | `ltx_pipelines.ic_lora.ICLoraPipeline`, `TI2VidOneStagePipeline`, … |
+
+For the 0.9.x series the pattern looks like:
+
+```bash
+# in requirements.txt
+git+https://github.com/Lightricks/LTX-Video.git
+```
+
+```python
+from ltx_video.pipelines import LTXPipeline as NativeLTXPipeline
+# ... model loading per the native repo's README
+```
+
+The native path doesn't use `load_lora_weights`. Instead, LoRAs are usually wired in at pipeline construction time, often via a list of LoRA configurations or by pointing at a fused checkpoint.
+
+When the LoRA's model card has a Python snippet using the native repo, copy its construction pattern verbatim. The native API changes more often than diffusers' does, so don't paraphrase.
+
+### LTX-2.x native path on ZeroGPU — gotchas
+
+For `LTX-2` / `LTX-2.3` (the `ltx-core` + `ltx-pipelines` repo), clone + `pip install -e packages/ltx-core packages/ltx-pipelines` at runtime in `app.py` and pin a commit. Three things bite on ZeroGPU:
+
+- **Native loader bypasses ZeroGPU virtualization → "No CUDA GPUs available" at startup.** The native safetensors loader does `safe_open(path, device="cuda")` and copies host→device inside safetensors' own C++ (`cudaMemcpy`), **bypassing `torch.Tensor.to`** — the call ZeroGPU patches to virtualize + pack module-scope weights. Nothing packs and module-scope placement raises *"No CUDA GPUs are available."* Fix: monkeypatch the loader to open on CPU then move with `.to`:
+  ```python
+  with safetensors.safe_open(shard, framework="pt", device="cpu") as f:
+      value = f.get_tensor(name).to(device=device)   # torch path → ZeroGPU-virtualisable
+  ```
+  Also: patch attention to **SDPA** (FA3 crashes on Blackwell ZeroGPU), and **never call `torch.cuda.*` / `get_device_capability()` at module scope** (it poisons virtualization).
+- **Native two-stage pins two 22B transformers → offload-disk overflow.** `ICLoraPipeline` builds two independent `ModelLedger`s (stage-1 with the LoRA, stage-2 without), each loading its **own** full transformer. The CLI loads them sequentially, but ZeroGPU pins all weights at module scope, so enabling stage 2 pins **both** (~143G) and overflows the offload disk (`OSError: [Errno 28] No space left on device`, ~96G cap). For a demo: pin stage 1, then have stage 2 **reuse stage 1's pinned modules** (`for n in [...]: setattr(s2, n, getattr(s1, n))`) so only one transformer is resident (~71G).
+- **`ICLoraPipeline` is distilled-only.** Per the LTX-2 `ltx-pipelines` guide, IC-LoRA inference runs in distilled-only mode (fixed 8-step sigmas, no `num_inference_steps`/`guidance_scale`/`negative_prompt`); the docs do not describe non-distilled IC-LoRA or IC-LoRA + CFG/STG (guidance lives only on full-model pipelines like `TI2VidOneStagePipeline`, and in two-stage is stage-1-only). So if a card recommends a *non-distilled* recipe (e.g. 30 steps + guidance + STG on a `…-dev` base), there is no stock IC-LoRA pipeline for it — use the supported distilled `ICLoraPipeline` and note the recipe difference, or fall back to the diffusers `LTX2InContextPipeline` (which exposes steps/guidance).
+
+## IC-LoRAs (in-context conditioning)
+
+LTX IC-LoRAs condition the model on a reference video alongside a first-frame image. The diffusers path uses `LTX2InContextPipeline` with standard `load_lora_weights` + `set_adapters`.
+
+Common flavors:
+
+- **Pose / depth / canny IC-LoRAs**: the reference video must be preprocessed into the control signal (pose skeletons, depth maps, edge maps) before being passed as conditioning. Passing a raw video with appearance information leaks color/style through.
+- **Outpaint IC-LoRAs**: the input video is padded with black margins to a target aspect ratio and passed as conditioning. The model fills the black regions.
+- **Frame interpolation / extension IC-LoRAs**: the input is a sparse set of keyframes; the model fills in between or extends.
+- **Audio-driven lip-sync IC-LoRAs**: take a reference video and a new audio track; the model re-syncs lip motion to match the new audio. UI needs both a video input and an audio input (e.g. for video translation, voiceover replacement, multilingual dubbing).
+
+Each of these implies different preprocessing in the demo and different UI shape (an aspect-ratio picker for outpaint, a pose preview for pose-control, a frame-pair input for keyframe extension). Read `references/adapting-to-the-lora.md` and shape the UI to the specific IC-LoRA.
+
+## Quantization gotcha (LTX-2.3, native path only)
+
+On the native path, some LTX-2.3 IC-LoRAs ship with FP8 quantization policies that fuse LoRA deltas into transformer weights at model-load time, using a Triton CUDA kernel. ZeroGPU has no real CUDA at module-load time (only the emulation layer) — this can crash the Space at startup. The diffusers path via `LTX2InContextPipeline` does not hit this: LoRAs are loaded through PEFT as separate adapter weights and applied at runtime rather than fused into the base transformer at load.
+
+Two workarounds when the user hits this on the native path:
+
+- **Skip quantization for the LoRA-fusion stage.** LTX-2.3 has a two-stage pipeline; apply quantization only to the second (non-LoRA) stage. Set `stage_1_quantization=None`.
+- **Pre-fuse the LoRA into a standalone checkpoint.** Download base + LoRA, fuse on a dev GPU, push the fused checkpoint to the Hub under the user's namespace, and have the Space load the fused checkpoint with no LoRA.
+
+The second option is preferable when the user plans to demo the same LoRA repeatedly — Space startup is much faster.
+
+## ZeroGPU duration guidance for LTX
+
+- Short T2V (3 seconds, 24fps, distilled): 60–90 seconds.
+- Standard T2V (5 seconds, 50 steps): 120–180 seconds.
+- I2V: similar to T2V at the same duration.
+- V2V with preprocessing (pose extraction, etc.): add 10–30 seconds for preprocessing overhead. Set duration to 180+.
+- LTX-2.3 two-stage: 240–360 seconds.
+
+Set `@spaces.GPU(duration=...)` to comfortably exceed expected generation time.
+
+## Things to watch
+
+- **Latest pipelines often need git diffusers.** Pin to `git+https://github.com/huggingface/diffusers` in `requirements.txt` for any LTX variant released in the last few weeks.
+- **`git+diffusers` is a moving target — verify LTX2 output quality.** Bare `@main` inherits whatever's there. The LTX2 text connector carried a token-reversal regression (PR #13564) that scrambled prompt tokens/registers — degrading prompt adherence and fine detail (e.g. garbled on-screen text), worst on short prompts — until **PR #13931 (merged 2026-06-19)** fixed it. If LTX2 output looks weak or garbled, confirm the installed diffusers commit is **at or past #13931**, and prefer pinning to a known-good commit (`git+https://github.com/huggingface/diffusers@<sha>`) over bare `@main` for reproducibility.
+- **Don't `torch.compile`.** LTX is fast enough on ZeroGPU without it; compile is incompatible anyway.
+- **`enable_vae_tiling()` for higher resolutions.** LTX VAE memory grows with resolution; enable tiling for outputs above 768px on either axis.
+- **Negative prompts matter more than for image models.** Don't ship with an empty negative_prompt unless the LoRA's model card says so.
+- **Frame-rate mismatch can produce glitches.** If the LoRA was trained at 24 fps and the demo passes 30, motion looks wrong. Use the LoRA's recommended fps.

package/bundled-skills/huggingface-lora-space-builder/references/base-models/qwen-image.md

@@ -0,0 +1,144 @@
+# Qwen-Image and Qwen-Image-Edit reference
+
+The Qwen-Image family is fully supported in `diffusers`. Both base and edit variants accept LoRAs via the standard `load_lora_weights` interface.
+
+## Pipelines
+
+> **Before using this table, verify against the base model's own card on the Hub.** This table is best-effort and can lag a recent release. The diffusers snippet on the base model's Hub page is source of truth for which pipeline class to import. See `SKILL.md` Phase 2 for the procedure.
+
+| Base model                            | Pipeline class                | Task                                |
+|---------------------------------------|-------------------------------|-------------------------------------|
+| `Qwen/Qwen-Image`                     | `QwenImagePipeline`           | Text-to-image                       |
+| `Qwen/Qwen-Image-Edit`                | `QwenImageEditPipeline`       | Image editing (instruction-driven)  |
+| `Qwen/Qwen-Image-Edit-2509`           | `QwenImageEditPlusPipeline`   | Image editing, multi-image input    |
+| `Qwen/Qwen-Image-Edit-2511`           | `QwenImageEditPlusPipeline`   | Image editing, latest variant       |
+
+The 2509 and 2511 variants use a *different* pipeline class than the original `QwenImageEditPipeline` — they take a list of input images and have different default parameters. Don't assume that variants in the same family share a pipeline class. Loading a 2511-trained LoRA onto `QwenImageEditPipeline` produces broken output; the failure is silent (no exception), so verifying against the base model card is the only way to catch it.
+
+The 2511 variant integrates several popular community LoRAs into the base, which can mean a LoRA trained against earlier Qwen-Image-Edit may behave subtly differently when loaded against 2511; if the LoRA's model card specifies which Edit variant it was trained on, match it.
+
+## Required dependencies
+
+Qwen-Image and Qwen-Image-Edit pipelines need extras beyond the standard diffusers/transformers/peft set, because the text encoder is `Qwen2_5_VLForConditionalGeneration` (Qwen 2.5-VL):
+
+- **`torchvision`** — required by `Qwen2VLVideoProcessor`, which the text encoder's processor pulls in transitively. Missing this is a startup-time `ImportError` ("Qwen2VLVideoProcessor requires the Torchvision library"). Always include in `requirements.txt` for any Qwen-Image Space.
+- **`sentencepiece`** — required by some Qwen tokenizer paths. Include if you see tokenizer-related ImportErrors at startup.
+
+The 2511 variant in particular often requires the latest `diffusers` from git, since `QwenImageEditPlusPipeline` and 2511-specific fixes land before pip releases:
+
+```
+git+https://github.com/huggingface/diffusers
+```
+
+If `from_pretrained("Qwen/Qwen-Image-Edit-2511", ...)` fails with a class-not-found or attribute error, switch the requirement to git.
+
+## Default load (T2I)
+
+```python
+import torch
+from diffusers import QwenImagePipeline
+
+pipe = QwenImagePipeline.from_pretrained(
+    "Qwen/Qwen-Image",
+    torch_dtype=torch.bfloat16,
+)
+pipe.to("cuda")
+pipe.load_lora_weights("user/my-qwen-lora")
+```
+
+`pytorch_lora_weights.safetensors` is the conventional filename. If the repo has a different name, pass `weight_name="..."`.
+
+For multiple adapters or when you want to control LoRA scale at inference time, use `set_adapters`:
+
+```python
+pipe.load_lora_weights("user/my-qwen-lora", adapter_name="mylora")
+pipe.set_adapters(["mylora"], adapter_weights=[0.9])
+```
+
+## Default load (Image Edit)
+
+For original `Qwen-Image-Edit`:
+
+```python
+import torch
+from diffusers import QwenImageEditPipeline
+
+pipe = QwenImageEditPipeline.from_pretrained(
+    "Qwen/Qwen-Image-Edit",
+    torch_dtype=torch.bfloat16,
+)
+pipe.to("cuda")
+pipe.load_lora_weights("user/my-qwen-edit-lora")
+```
+
+For `Qwen-Image-Edit-2509` and `Qwen-Image-Edit-2511`:
+
+```python
+import torch
+from diffusers import QwenImageEditPlusPipeline
+
+pipe = QwenImageEditPlusPipeline.from_pretrained(
+    "Qwen/Qwen-Image-Edit-2511",  # or 2509
+    torch_dtype=torch.bfloat16,
+)
+pipe.to("cuda")
+pipe.load_lora_weights("user/my-qwen-edit-lora")
+```
+
+`QwenImageEditPlusPipeline` accepts `image=<PIL>` or `image=[<PIL>, <PIL>, ...]` for multi-image edits. `QwenImageEditPipeline` accepts a single image. Default parameters differ slightly between the two — see "Inference defaults" below.
+
+## Inference defaults
+
+For non-distilled Qwen-Image:
+- `num_inference_steps`: 50 by default; the LoRA's model card may recommend lower.
+- `true_cfg_scale`: typical 4.0.
+- `width`/`height`: multiples of 16, ideally 1024 or 1328 along the long axis.
+
+For Qwen-Image-Edit (original):
+- `num_inference_steps`: 30–50 typical, often less for distilled variants.
+- `true_cfg_scale`: 4.0 typical.
+- Input image gets internally resized; passing a reasonable resolution (1024px on the long side) is fine.
+
+For Qwen-Image-Edit-2509 / 2511 (`QwenImageEditPlusPipeline`):
+- `num_inference_steps`: 40 typical for 2511; 50 for 2509.
+- `true_cfg_scale`: 4.0.
+- `guidance_scale`: 1.0 (the new pipeline uses `true_cfg_scale` as the active CFG; standard `guidance_scale` is kept at 1.0).
+- Input is a list of one or more PIL images.
+
+For Lightning / few-step LoRAs (e.g. `lightx2v/Qwen-Image-Lightning-*`):
+- `num_inference_steps`: 4 or 8 (read the LoRA's model card — they ship 4-step and 8-step variants).
+- `true_cfg_scale`: usually 1.0 (CFG disabled).
+- Often comes with a custom scheduler config — see the lightx2v README for the exact `FlowMatchEulerDiscreteScheduler` config to use.
+
+## Resolution buckets
+
+Qwen-Image uses 16-pixel-aligned resolutions. When the user picks an aspect ratio, compute `width` and `height` as multiples of 16. A helper:
+
+```python
+def round_to_bucket(w, h, multiple=16):
+    return (w // multiple) * multiple, (h // multiple) * multiple
+```
+
+For image-edit pipelines, resize the input image to the nearest bucket while preserving aspect; don't crop.
+
+## ZeroGPU duration guidance
+
+- Standard 50-step Qwen-Image T2I at 1024×1024: 60–90 seconds.
+- 4-step Lightning Qwen-Image: 15–25 seconds.
+- Qwen-Image-Edit 30 steps: 60–90 seconds.
+
+Set `@spaces.GPU(duration=...)` accordingly.
+
+## Common LoRA patterns on Qwen-Image
+
+- **Style LoRAs (T2I).** Standard load. Trigger word usually present. UI: prompt + aspect ratio.
+- **Subject / character LoRAs (T2I).** Standard load. Trigger word almost always present. UI: prompt with the trigger pre-prepended in code, possibly an example prompt highlighting the trigger.
+- **Lighting / aesthetic LoRAs (T2I).** Often paired with a recommended LoRA scale ≠ 1.0 — check the model card.
+- **Edit LoRAs (image-to-image, on Qwen-Image-Edit).** Specific instructions baked in. The LoRA might require a specific instruction phrasing — match the pattern from the model card. UI: input image + instruction textbox.
+- **Lightning-distilled LoRAs.** Lock step count and CFG to recommended values; hide the sliders.
+
+## Things to watch
+
+- **VAE memory.** For 1328×1328 outputs, consider `pipe.enable_vae_tiling()` and `pipe.enable_vae_slicing()` after loading. Keep them off for smaller resolutions to avoid quality loss.
+- **Don't compile the transformer on ZeroGPU.** `torch.compile` won't work; the speedup options on ZeroGPU are limited to reducing steps or using FP8-distilled variants.
+- **Negative prompts work** but defaults are often empty string. Don't expose a negative prompt in the UI unless the LoRA's behavior actually benefits from it.

package/bundled-skills/huggingface-lora-space-builder/references/creative-mode.md

@@ -0,0 +1,262 @@
+# Creative mode: custom HTML/JS UIs in Gradio
+
+When a LoRA's natural input shape doesn't fit any standard Gradio component or Hub custom component, you can drop down to plain HTML/CSS/JS inside a Gradio app. This file is about *how* — the Gradio primitives that make it work, and the discipline that keeps it from turning into a tangled mess.
+
+This is the third rung of the component ladder (see `tasks.md` → "Picking components"):
+
+1. **Stock Gradio components.** First choice. Almost always enough for T2I and a lot of I2I.
+2. **Hub custom components.** `gradio_image_annotation`, `gradio_imageslider`, `gradio_modal`, `gradio_rangeslider`, etc. No JS, just a `pip install` and an import.
+3. **Creative mode (this file).** Custom HTML/JS, when the user's input shape is something none of the above expresses well — point sets, trajectories, brush strokes, region selections with metadata, timeline scrubbing, 3D rotation gizmos, color picking on regions, drag-resize handles on media, keyframed inputs, anything where the user manipulates *a thing on top of media*.
+
+Skipping rung 2 is a common mistake. If a Hub custom component fits, use it — `gradio_image_annotation` already covers bbox drawing, label assignment, and basic editing without a single line of JS.
+
+But rung 2 has its own discipline (see "Hub custom components are fragile" below). The most common failure mode for these Spaces is **the custom component silently fails to render**: the Python side imports fine, the page loads, the API smoke-test in Phase 6 of `SKILL.md` passes — and yet the widget just isn't on the page. A user opening the Space sees the surrounding layout (buttons, accordions, outputs) but no upload zone, no annotator, nothing. If you don't actually look at the rendered page in a browser, you'll ship a broken Space and not know it.
+
+## Hub custom components are fragile
+
+Treat any `gradio_*` package from the Hub as load-bearing-and-untested-against-your-Gradio-version until you've seen it render. The most common failure modes:
+
+- **Version mismatch silent breakage.** A Hub component built against Gradio N may load (because its declared `gradio<N+2,>=N` range covers your `sdk_version`) but mount to an empty DOM node on a slightly newer Gradio. No traceback in the build logs. No Python error. The component simply doesn't appear, and the rest of the column flows up around the gap. This is what produces "Generate button at the top of an empty left column" Spaces.
+- **Stale releases.** Many Hub custom components were last published 1–2 years ago. The Gradio frontend has changed since. Check the package's release date against the Gradio version you're targeting; if there's a multi-major-version gap, expect breakage.
+- **Mismatched param shapes.** The component's Python signature accepts a parameter the Svelte side no longer reads. Your `disable_edit_boxes=True` does nothing, or worse, throws on the JS side and the whole component fails to mount.
+
+Discipline before committing to a Hub component:
+
+1. **Check the package's last release date** (PyPI page or `pip index versions <pkg>`). If it's older than the Gradio release in your `sdk_version`, treat it as suspect.
+2. **Smoke-test the component in isolation** — a five-line Gradio app with just that one component, locally, *before* integrating it into the full app. If it renders, fine. If it's missing or blank, you've caught the breakage cheaply.
+3. **When a Hub component fails to render, don't iterate on its kwargs.** Drop to either (a) split stock components (e.g. `gr.Image` for upload + a sibling widget for the box coords) or (b) rung 3 (custom HTML/JS via `gr.HTML`). Twiddling `disable_edit_boxes` / `use_default_label` / `sources` is not going to bring a Svelte component back from a JS-side mount failure.
+
+The same discipline applies to less obviously "custom" components if they were added recently — `gr.ImageSlider`, for example, can render unexpectedly when paired with a custom component on the same page.
+
+## When creative mode is the right call
+
+Reach for it when the user's *natural* input is structurally outside what stock components express:
+
+- **Spatial input on top of media.** Drawing arrows on a frame, painting strokes on an image, dropping points along a trajectory, selecting irregular regions, drawing curves.
+- **Multi-shape annotation.** Source-and-destination box pairs, multiple labeled regions, ordered sequences of points/boxes that are semantically distinct.
+- **Continuous-with-snapping controls.** 3D rotation gizmos, dial/wheel controls, timeline scrubbers — anything where a slider would technically work but feel wrong.
+- **Composite controls bound together.** A canvas plus a color picker plus a brush-size dial that all feed the same structured input, where binding three separate components and reasoning about their joint state is uglier than rolling one widget.
+- **Live preview that depends on multiple inputs.** Something the user wants to see *immediately* as they manipulate, where a server roundtrip per change is too slow.
+
+If the input is "a number," "a string," or "an image," you don't need this. Don't build a custom canvas because it would look cool — build it because the LoRA's input shape demands it.
+
+## The Gradio primitives
+
+Before reading the patterns below, it's worth re-checking the current Gradio docs for anything that landed recently:
+
+- `gr.HTML` — https://www.gradio.app/docs/gradio/html
+- Custom components — https://www.gradio.app/guides/custom-components-in-five-minutes
+- `Blocks.launch(head=, css=)` — https://www.gradio.app/docs/gradio/blocks#blocks-launch
+
+WebFetch these if you're unsure about a signature. The custom-HTML surface area evolves and lagging on it produces Spaces that "work" in stale ways.
+
+The primitives that creative mode is built from:
+
+### `gr.HTML` for arbitrary markup
+
+Drop any HTML into the page. The block becomes a regular Gradio component, but its content is whatever you write. You're responsible for everything inside it: layout, styling, interactivity.
+
+```python
+gr.HTML("""
+<div id="my-widget" style="...">
+  <canvas id="my-canvas" width="512" height="512"></canvas>
+  <div id="my-status"></div>
+</div>
+""")
+```
+
+### `demo.launch(head=..., css=...)`
+
+Inject `<script>` and `<style>` tags into the page `<head>`. This is how you load external JS libraries (Three.js, p5, fabric, anime.js, …) or define page-wide CSS that needs to be in `<head>` rather than inline.
+
+```python
+head = '<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>'
+css  = '.fillable {max-width: 1200px !important}'
+demo.launch(head=head, css=css)
+```
+
+Use the highest-quality CDN you can (cdnjs, jsdelivr, unpkg). Pin the version (`/three.js/r128/`, not `/three.js/latest/`). Loading from the LoRA author's personal server is a recipe for the Space breaking when their server moves.
+
+### `elem_id` and `elem_classes` for addressing
+
+Every Gradio component accepts `elem_id="..."` and `elem_classes=[...]`. JS uses these to find the rendered DOM nodes:
+
+```python
+prompt_box = gr.Textbox(elem_id="my-prompt", elem_classes=["hidden-input"])
+```
+
+```javascript
+const promptBox = document.getElementById("my-prompt");
+// Note: the Gradio component wraps an <input> or <textarea>;
+// you usually want the inner element:
+const inner = promptBox.querySelector("input, textarea");
+```
+
+### Two ways JS pushes state into Python
+
+This is the part that trips people up. Pick whichever fits your case and stick with it for the whole widget — mixing them in one app is confusing.
+
+**Approach A — Hidden Gradio input + DOM event dispatch.** Define a hidden `gr.Textbox` (or `gr.JSON`, `gr.Number`, etc.). From JS, set its inner `<input>`/`<textarea>` value and dispatch synthetic `input` and `change` events so Gradio's reactivity fires.
+
+```python
+state_json = gr.Textbox(value="{}", elem_id="state-json", visible=False)
+```
+
+```javascript
+function setGradioValue(elemId, value) {
+  const container = document.getElementById(elemId);
+  if (!container) return;
+  const el = container.querySelector("input, textarea");
+  if (!el) return;
+  const proto = el.tagName === "TEXTAREA"
+    ? HTMLTextAreaElement.prototype
+    : HTMLInputElement.prototype;
+  const setter = Object.getOwnPropertyDescriptor(proto, "value").set;
+  setter.call(el, value);
+  el.dispatchEvent(new Event("input",  { bubbles: true }));
+  el.dispatchEvent(new Event("change", { bubbles: true }));
+}
+
+setGradioValue("state-json", JSON.stringify(myState));
+```
+
+The native setter dance (`Object.getOwnPropertyDescriptor(...).set.call(el, value)`) is necessary because React intercepts plain `el.value = ...` assignments and they won't trigger a re-render. The DOM event dispatch is what makes Gradio see the change.
+
+This approach works in any Gradio version, with any component type. Downsides: lots of glue, fragile to DOM structure changes, easy to write race conditions.
+
+**Approach B — `gr.HTML` subclass with `html_template`/`js_on_load`.** Newer Gradio supports subclassing `gr.HTML` and providing custom props plus a `js_on_load` script. The script gets a `props` object with custom values and a `trigger()` function for emitting events back to Python. State binding is value-based (Gradio reads the `value` prop just like any other component).
+
+```python
+class PointPicker(gr.HTML):
+    def __init__(self, value=None, image_url=None, **kwargs):
+        super().__init__(
+            value=value or {"points": []},
+            html_template="<canvas id='pp-canvas' width='512' height='512'></canvas>",
+            js_on_load="""
+                const canvas = document.getElementById('pp-canvas');
+                canvas.addEventListener('click', (e) => {
+                    const rect = canvas.getBoundingClientRect();
+                    const x = (e.clientX - rect.left) / rect.width;
+                    const y = (e.clientY - rect.top)  / rect.height;
+                    props.value = {points: [...(props.value?.points || []), {x, y}]};
+                    trigger('change', props.value);
+                });
+            """,
+            image_url=image_url,
+            **kwargs,
+        )
+```
+
+Cleaner, value-based, no hidden inputs. Downsides: requires a Gradio version recent enough to support the subclass shape, and the JS lives inside a Python string, which IDEs don't lint.
+
+**Picking between A and B:** if you're starting fresh and on current Gradio, prefer B — the binding semantics are saner. Reach for A when you need to integrate with already-existing Gradio components (sliders that should mirror the canvas state, prompt boxes the canvas writes into) or when the UI shell is large enough that putting it inside a `gr.HTML` subclass is awkward.
+
+### Two ways Python pushes state into JS
+
+- **For B (subclass):** return `gr.update(prop=value)` from an event handler. The widget's polling loop sees the prop change and reacts.
+- **For A (hidden input):** write to a hidden Gradio Textbox from your event handler. JS polls the Textbox value (or hooks into Gradio's mutation observer) and reacts.
+
+Either way, expect a small polling interval (50–200ms is typical). Don't poll faster — it will dominate CPU on weaker machines.
+
+### `js=` on event handlers
+
+For tiny JS-only transforms (e.g. "scroll to result on click"), event handlers accept a `js=` arg that runs in the browser without a server roundtrip:
+
+```python
+btn.click(fn=infer, inputs=[...], outputs=[...], js="() => { window.scrollTo(0, 0); }")
+```
+
+Useful for UI polish (scroll, focus, show/hide loaders), not for state management.
+
+## The communication contract
+
+The single most important discipline in creative mode: **define the JSON shape that flows between JS and Python, and treat it as the API.**
+
+Write it down at the top of `app.py` as a comment, even one line. Pick names that survive translation between camelCase JS and snake_case Python (or commit to one and convert at the boundary):
+
+```python
+# State shape on the JS↔Python wire:
+#   {
+#     "src":    {"x1": float, "y1": float, "x2": float, "y2": float} | null,
+#     "dst":    {"x1": float, "y1": float, "x2": float, "y2": float} | null,
+#     "label":  string | null
+#   }
+```
+
+Once it's written down, both sides have a target to conform to and you'll catch shape drift early.
+
+Common shape archetypes — none of these are required, just illustrative:
+
+- **Point set:** `{"points": [{"x": 0.3, "y": 0.4}, ...]}`
+- **Stroke set:** `{"strokes": [{"color": "#ff0", "size": 12, "points": [...]}, ...]}`
+- **Regions:** `{"regions": [{"label": "subject", "bbox": {...}}, ...]}`
+- **Transform:** `{"azimuth": 45, "elevation": 0, "distance": 1.0}`
+- **Trajectory with time:** `{"keyframes": [{"t": 0.0, "x": ...}, {"t": 0.5, ...}]}`
+
+All coordinates normalized to `[0, 1]` is almost always the right call — it survives image resizing without rescaling math.
+
+## External libraries
+
+Use what helps. Stay light when you can.
+
+- **Pure canvas + DOM** is enough for: drawing rectangles/points/lines, brush strokes, drag-and-drop handles, image overlays.
+- **SVG** for: vector overlays, especially when the marks need to scale crisply or be styled with CSS.
+- **Three.js** for: 3D rotation gizmos, depth visualizations, anything where you need WebGL rendering of a small scene. Pulled in via `head=` script tag.
+- **fabric.js / paper.js / konva** for: complex shape-on-canvas interactions when raw canvas is getting ugly. Worth it once you've added more than ~200 lines of canvas glue.
+- **p5.js** for: rapid prototyping of generative-art-style canvases. Heavier than necessary for production.
+
+When in doubt, raw canvas. Pulling in fabric for a single rectangle is overkill; pulling in three.js because you wanted "fancier sliders" is a smell.
+
+## Pitfalls
+
+These bite repeatedly. Read before writing.
+
+- **Silent mount failure of Hub custom components.** Covered in detail above. Worth restating because it's the failure mode that the API smoke-test cannot catch: the component imports, the page loads, and the widget is just absent from the DOM. Always verify in a browser; never trust "the build went green" as proof a Hub custom component renders.
+
+- **Init race with Gradio mount.** The `js_on_load` (B) or a top-level `<script>` injected via `head` may run before Gradio has rendered the DOM nodes you're trying to hook into. Guard with a short `setTimeout` or wait for the element to exist:
+  ```javascript
+  function init() {
+    const el = document.getElementById("my-widget");
+    if (!el) { setTimeout(init, 50); return; }
+    // ... do work
+  }
+  init();
+  ```
+
+- **Double init.** If the user navigates between tabs or Gradio re-renders, your init can fire twice. Stash a flag: `if (window.__myWidgetInited) return; window.__myWidgetInited = true;`.
+
+- **Base64 image transfer cost.** Sending a full-resolution image as a base64 string through a hidden Textbox is fine for previews but punishingly slow for 4K images. Downscale on the JS side before stuffing into state, or pass the image through a real `gr.Image` component and only send the *interaction state* (boxes, points) through the hidden channel.
+
+- **File inputs don't roundtrip through hidden Textboxes.** A hidden `gr.File` won't accept arbitrary JS-set values. If the user uploads via a custom `<input type="file">`, you have two options: (a) read the file in JS, base64-encode, and write to a hidden Textbox; (b) wire the custom file input to programmatically click a real `gr.File` (`document.querySelector("#real-file input").click()` is brittle but works).
+
+- **Polling cadence.** 100ms is a reasonable default; 50ms feels snappy but burns CPU on weaker machines; 500ms feels laggy. Don't poll inside `requestAnimationFrame` for state sync — that's 60Hz and overkill.
+
+- **Mobile / touch.** If the widget lives in a Space people will open on their phones, handle `touchstart`/`touchmove`/`touchend` alongside mouse events. The two-event-system thing is annoying but unavoidable.
+
+- **ZeroGPU duration when payloads grow.** If the custom UI lets users submit large state (many points, big strokes, multiple boxes), the server-side processing time can grow with input size. Re-check `@spaces.GPU(duration=...)` against worst-case payloads.
+
+- **Custom run buttons.** If you build your own "Run" button in HTML and have it trigger a real Gradio button via `.click()`, make sure state-sync (write to hidden inputs) happens *before* the click, with enough delay for events to propagate (`setTimeout(() => realBtn.click(), 50)` is the usual fix).
+
+- **CSS isolation.** Spaces inject their own CSS. If your widget styles get clobbered, increase specificity (`#my-widget .foo` instead of `.foo`) or use `!important` sparingly. Don't fight it with inline styles for everything — that gets unreadable fast.
+
+## Smoke-test caveat — applies to rung 2 AND rung 3
+
+`gradio info` / `gradio predict` (Phase 6 of `SKILL.md`) only exercise the Python endpoint. They tell you nothing about what actually renders. For any Space that uses a Hub custom component (rung 2) or custom HTML/JS (rung 3), the Python side can be perfectly correct *and* the user can see a broken UI.
+
+The two distinct failure modes:
+
+- **Rung 2 silent mount failure.** Hub custom component imports cleanly, gets a slot in the Gradio config, and just doesn't appear in the rendered DOM. You see the components around it but the widget itself is gone — leaving e.g. a Generate button at the top of an otherwise empty column. No error anywhere. See "Hub custom components are fragile" above.
+- **Rung 3 broken JS.** Server-side green, but `js_on_load` errored, or a CDN script failed to load, or your event handlers never bound. The widget mounts but doesn't do anything when clicked.
+
+After the API smoke-test in Phase 6 passes, **open the Space URL in a browser and verify both that every component is visible and that one full interaction (upload → click Generate → see result) works** before sharing. This is not optional for these Spaces — it's the only check that catches the failure modes above.
+
+If the user is in an environment where you can't drive a browser yourself, ask them to open the Space URL and confirm the upload zone is visible and accepts an image before declaring the Space done. The cost of asking is one extra message; the cost of skipping is shipping a Space that looks empty.
+
+## Real-world examples
+
+These two Spaces use the patterns above for very different LoRAs. They're useful as concrete proof the patterns work, not as templates to copy:
+
+- **3D camera control via Three.js** (Approach B, `gr.HTML` subclass): https://huggingface.co/spaces/multimodalart/qwen-image-multiple-angles-3d-camera
+- **Bounding-box drag/resize via canvas** (Approach A, hidden inputs + custom Run button): https://huggingface.co/spaces/linoyts/FLUX.2-klein-Move
+
+A short look at each is worth it before designing a new widget — they show what "production polish" feels like in this register (snap-to-nearest animations, status overlays, cursor changes on hover, mobile-touch handling). But your widget will look different, because your LoRA wants different inputs.