comfyui-mcp 0.22.0 → 0.23.0

package/plugin/skills/triton-sageattention/SKILL.md

@@ -295,6 +295,21 @@ rather than trying to satisfy the dependency.
   still imports with CUDA. If broken, **roll back** (`pip install
   torch==<old>+cu<line> --index-url https://download.pytorch.org/whl/cu<line>`,
   or uninstall the bad wheel) and re-apply the sdpa fallback.
+- **Stale Triton cache after a torch/GPU/driver change.** Triton caches compiled
+  kernels in `~/.triton` (`%USERPROFILE%\.triton` on Windows). After upgrading torch,
+  swapping GPUs, a driver update, or a failed compile, that cache can go stale and
+  cause `torch.compile`/SageAttention runs to fail *even though the install is
+  correct* — recurring compile errors, `RuntimeError` in a Triton kernel, or a hang
+  on the first sample. **Fix: clear the cache and re-run** (Triton recompiles fresh):
+  ```
+  # Windows
+  rmdir /s /q "%USERPROFILE%\.triton"
+  # macOS / Linux
+  rm -rf ~/.triton
+  ```
+  Safe to delete — it's a pure cache. Do this BEFORE assuming the wheel is wrong
+  (it's a much cheaper fix than a reinstall/roll-back). If it recurs every run, the
+  install is genuinely mismatched (see the wheel-mismatch trap above).
 - **MSVC missing (Windows Triton).** `torch.compile`/Triton errors like "Microsoft
   Visual C++ ... required", `cl.exe not found`, or `PY_SSIZE_T_CLEAN`/DLL load
   failures usually mean no MSVC toolchain. Install **Visual Studio Build Tools (C++

package/plugin/skills/troubleshooting/SKILL.md

@@ -71,6 +71,62 @@ The GPU does not have enough VRAM to hold the model weights, intermediate tensor
 | Flux Schnell | ~48GB | ~24GB | ~12GB |
 | LTXV | ~20GB+ | ~10GB+ | ~6GB |
 
+## Launch Flags — VRAM / Cache / Attention / Precision
+
+ComfyUI's startup flags tune the speed↔VRAM tradeoff. Match them to the detected
+GPU (the panel orchestrator reports VRAM/GPU/torch/sage in its env block; pick the
+tier from there). Set them on the process that launches ComfyUI (or the
+`--panel-orchestrator` / `connect` command's ComfyUI, not the agent).
+
+### VRAM mode (pick ONE by card size)
+
+| Flag | Card | Behavior |
+|------|------|----------|
+| `--gpu-only` | 16GB+ | Everything (CLIP/VAE/UNet) stays on GPU — fastest, max VRAM |
+| `--highvram` | 12–16GB | Models stay resident in GPU after use, no CPU offload |
+| `--normalvram` | 8–12GB | Default balance — unload to CPU RAM when idle |
+| `--lowvram` | 6–8GB | Split the UNet, aggressive CPU offload — slower |
+| `--novram` | 4–6GB | Extreme split/offload — for OOM even on lowvram, or long videos |
+| `--cpu` | <4GB / no GPU | CPU only (very slow) |
+
+`--reserve-vram N` (GB) leaves headroom for the OS/other apps — bump it if you OOM
+intermittently mid-run (VAE decode / audio round-trips spike).
+
+### Cache (RAM vs re-run speed)
+
+| Flag | Effect |
+|------|--------|
+| `--cache-classic` | Default aggressive caching (fastest re-runs, most RAM) |
+| `--cache-lru N` | Keep the last N node results (bounded RAM) |
+| `--cache-ram N` | Cap cache to N GB of headroom |
+| `--cache-none` | No caching — minimal RAM, re-runs every node |
+
+### Attention (speed vs compatibility)
+
+| Flag | Notes |
+|------|-------|
+| `--use-sage-attention` | **Recommended** — fast + efficient (needs SageAttention + Triton; see `triton-sageattention`) |
+| `--use-flash-attention` | Very fast on supported GPUs |
+| `--use-pytorch-cross-attention` | PyTorch 2.x native — best compatibility |
+| `--use-split-cross-attention` | Lower VRAM, slower |
+| `--use-quad-cross-attention` | Sub-quadratic optimization |
+| (omit) | Auto-selects xFormers if available |
+
+### Precision (UNet)
+
+| Flag | Effect |
+|------|--------|
+| `--fp16-unet` | Half precision, ~50% VRAM |
+| `--bf16-unet` | BFloat16, good balance (newer GPUs) |
+| `--fp8_e4m3fn-unet` | 8-bit float, max savings (newest GPUs) |
+
+**Typical recipes:**
+- **RTX 4090/5090 (24–32GB):** `--gpu-only --use-sage-attention --cache-classic`
+- **12–16GB:** `--highvram --use-sage-attention` (or `--fp8_e4m3fn-unet` for big models)
+- **8GB:** `--normalvram --use-sage-attention --cache-lru 20`
+- **6GB:** `--lowvram --use-split-cross-attention --cache-none`
+- **OOM on long video:** `--novram --reserve-vram 2`
+
 ## Device Mismatch
 
 ### Error Pattern

package/plugin/skills/wan-multitalk/SKILL.md

@@ -0,0 +1,97 @@
+---
+name: wan-multitalk
+description: Build WAN MultiTalk audio-driven talking-avatar / lip-sync video workflows — MeiGen-AI MultiTalk on WAN 2.1 14B I2V via kijai WanVideoWrapper (portrait + audio → lip-synced video)
+globs:
+  - "**/*.json"
+---
+
+# WAN MultiTalk — Audio-Driven Talking Avatar
+
+## Overview
+
+MultiTalk (MeiGen-AI) drives a **still portrait's lip-sync and head motion from an
+audio track**. It runs on WAN 2.1 14B Image-to-Video via kijai's
+**ComfyUI-WanVideoWrapper**: Wav2Vec speech embeddings condition the WAN sampler so
+the mouth/expression follow the speech, while the lightx2v step-distill LoRA keeps
+it to a few sampling steps.
+
+Use it for talking heads, dubbing, and single-speaker avatar clips (~10s at 480p).
+It is **distinct from `wan-animate`** (pose/motion-driven character animation) — this
+is *audio → lip-sync*, not reference-video motion transfer.
+
+Pack: `wan-multitalk` (480p, ~10s). Higher-res/longer variants exist in the source
+bundle (720p, long-context) as VRAM/duration knobs on the same graph.
+
+## Pipeline (node graph)
+
+```
+LoadImage (portrait) ─┐
+LoadAudio ─ AudioSeparation ─ AudioCrop ─ DownloadAndLoadWav2VecModel ─ MultiTalkWav2VecEmbeds ─┐
+                                                                                                 ▼
+WanVideoModelLoader (WAN 2.1 14B I2V GGUF) ─ MultiTalkModelLoader ─ WanVideoLoraSelect (lightx2v)
+   + LoadWanVideoT5TextEncoder (umt5) + WanVideoTextEncode + WanVideoClipVisionEncode (clip_vision_h)
+   + WanVideoVAELoader ──────────────────────────────────────────────────────────────────────────┘
+                                                     ▼
+                       WanVideoImageToVideoMultiTalk ─ WanVideoSampler ─ WanVideoDecode ─ VHS_VideoCombine
+```
+
+Key nodes (all kijai WanVideoWrapper unless noted):
+- **DownloadAndLoadWav2VecModel** — auto-downloads the Wav2Vec speech model on first
+  run (no manifest entry needed).
+- **MultiTalkWav2VecEmbeds** — turns the (separated, cropped) speech into the
+  embeddings that steer the mouth/expression.
+- **MultiTalkModelLoader** + **WanVideoImageToVideoMultiTalk** — the MultiTalk head
+  on top of the WAN I2V model.
+- **AudioSeparation** — isolate the voice from music/noise before embedding (cleaner
+  lip-sync). **AudioCrop** — trim to the segment you want to animate.
+- **ImageResizeKJv2** (KJNodes), **VHS_VideoCombine** (VideoHelperSuite) — resize +
+  mux to mp4.
+
+## Models
+
+| File | Loader | Folder |
+|------|--------|--------|
+| `Wan2.1_14b_Image_to_Video_480p_GGUF_Q8.gguf` | WanVideoModelLoader | `diffusion_models/` |
+| `WanVideo_2_1_Multitalk_14B_fp32.safetensors` | MultiTalkModelLoader | `diffusion_models/` |
+| `umt5_xxl_fp8_e4m3fn_scaled.safetensors` | LoadWanVideoT5TextEncoder | `text_encoders/` |
+| `Wan2_1_VAE_bf16.safetensors` | WanVideoVAELoader | `vae/` |
+| `clip_vision_h.safetensors` | CLIPVisionLoader | `clip_vision/` |
+| `Wan21_I2V_14B_lightx2v_cfg_step_distill_lora_rank64_fixed.safetensors` | WanVideoLoraSelect | `loras/` |
+
+Sources: kijai `Kijai/WanVideo_comfy`, MeiGen-AI `MeiGen-AI/MeiGen-MultiTalk`, GGUF
+`city96/Wan2.1-I2V-14B-480P-gguf`. See `packs/wan-multitalk/manifest.yaml` (some URLs
+are best-effort — verify per mirror). Wav2Vec auto-downloads.
+
+## Inputs & key parameters
+
+- **Portrait** (LoadImage): front-facing, clear face, neutral-ish expression works
+  best. Resized by ImageResizeKJv2 to the target (480p).
+- **Audio** (LoadAudio): the speech track. AudioSeparation isolates the voice;
+  AudioCrop selects the segment (drives clip length).
+- **Steps**: low (the lightx2v distill LoRA is why — typically ~4–8). Raising steps
+  rarely helps and costs time.
+- **BlockSwap** (WanVideoBlockSwap): trade VRAM for speed — increase blocks swapped
+  to CPU on lower-VRAM cards.
+
+## VRAM tiers (from the source bundle's variants)
+
+| Target | Approx VRAM | Lever |
+|--------|-------------|-------|
+| 480p 10s | ~8–12 GB | base |
+| 480p low-VRAM | ~6–8.4 GB | more BlockSwap, GGUF quant, lower quality |
+| 720p 10s | ~11–16 GB | higher res |
+
+Pair with the VRAM launch-flags guidance (see `troubleshooting`): `--use-sage-attention`
++ appropriate `--*vram` mode; MultiTalk benefits from `--reserve-vram` headroom for
+the Wav2Vec + VAE round-trips.
+
+## Gotchas
+
+- **Audio must be voice-isolated** for good lip-sync — skipping AudioSeparation on a
+  music-heavy track makes the mouth chase the wrong signal.
+- **One speaker.** This graph is single-speaker; multi-speaker MultiTalk needs the
+  multi-embed variant (not in this pack).
+- **Wav2Vec first run** downloads a model — the first render is slower.
+- If lips look under-driven, check the MultiTalk embeds are actually wired into
+  `WanVideoImageToVideoMultiTalk` (not bypassed), and that the audio isn't silent
+  after AudioCrop.