omnius 1.0.591 → 1.0.592

package/docs/sana-and-video-generation-integration-plan.md

@@ -0,0 +1,712 @@
+# Sana (Image) + `/video` (Video Generation) Integration Plan
+
+> **Status**: Plan only. Builds a complete, anchored handoff for a downstream implementation agent.
+> **Goal**: (A) Promote NVIDIA Sana to the **primary image-generation model** for `/image` and the image-generation tool, (B) ship an entirely new **`/video` pipeline** modeled exactly on the existing `/image` and `/sound`-`/music` patterns, including its agent tool, Telegram public/private wiring, weight management menus, and TUI ASCII preview (video → thumbnail frame).
+>
+> **Reference inventory**: `/home/robit/Downloads/open_source_video_generation_models_agent_package.md`
+> **Sana upstream**: <https://github.com/NVlabs/Sana>
+
+---
+
+## 0. TL;DR for the implementation agent
+
+You are extending three parallel pipelines that already exist for `/image`, `/sound`, and `/music`. Replicate them for video. Do **not** invent new abstractions. Reuse:
+
+1. The `ImageGenerateTool` (preset list, fallback ladder, Python venv bootstrapper, sidecar JSON, progress events, prompt expander, ASCII preview).
+2. The `AudioGenerateTool` (kind discriminator — `sound`/`music`, per-kind defaults, project profile fallback).
+3. The TUI command surface in [packages/cli/src/tui/commands.ts](packages/cli/src/tui/commands.ts) + [packages/cli/src/tui/command-registry.ts](packages/cli/src/tui/command-registry.ts).
+4. The Telegram bridge wiring in [packages/cli/src/tui/telegram-bridge.ts](packages/cli/src/tui/telegram-bridge.ts) + [packages/cli/src/tui/telegram-creative-tools.ts](packages/cli/src/tui/telegram-creative-tools.ts).
+5. Media classification + Telegram routing in [packages/cli/src/tui/media-routing.ts](packages/cli/src/tui/media-routing.ts) (already supports `kind: "video"` → `sendVideo`).
+
+**Two deliverables**:
+
+| # | Deliverable | Files (new / edited) |
+| - | --- | --- |
+| 1 | Make Sana the primary image generator | edit [packages/execution/src/tools/image-generate.ts](packages/execution/src/tools/image-generate.ts) (presets, ladder, defaults, runner) |
+| 2 | New `/video` pipeline | new [packages/execution/src/tools/video-generate.ts](packages/execution/src/tools/video-generate.ts); edits to [packages/execution/src/index.ts](packages/execution/src/index.ts), [packages/cli/src/tui/commands.ts](packages/cli/src/tui/commands.ts), [packages/cli/src/tui/command-registry.ts](packages/cli/src/tui/command-registry.ts), [packages/cli/src/tui/omnius-directory.ts](packages/cli/src/tui/omnius-directory.ts), [packages/cli/src/tui/telegram-bridge.ts](packages/cli/src/tui/telegram-bridge.ts), [packages/cli/src/tui/telegram-creative-tools.ts](packages/cli/src/tui/telegram-creative-tools.ts) |
+
+---
+
+## 1. Architectural Anchors — How the existing `/image` pipeline is shaped
+
+These line refs are the contract you replicate for `/video` and the spots where Sana becomes the primary model.
+
+### 1.1 Tool surface — `ImageGenerateTool`
+
+File: [packages/execution/src/tools/image-generate.ts](packages/execution/src/tools/image-generate.ts) (2115 lines)
+
+| Anchor | Lines | What it is |
+| --- | --- | --- |
+| Tool class header | [1206-1215](packages/execution/src/tools/image-generate.ts#L1206-L1215) | `ImageGenerateTool implements Tool`, `name = "generate_image"`, description used by the LLM. |
+| Tool parameters schema | [1216-1279](packages/execution/src/tools/image-generate.ts#L1216-L1279) | JSON-schema for `prompt`, `model`, `backend`, `aspect_ratio`, `width`, `height`, `steps`, `guidance`, `seed`, `action`, `fallback`, `strict_model`, `expand_prompt`. |
+| Constructor + defaults wiring | [1281-1305](packages/execution/src/tools/image-generate.ts#L1281-L1305) | `defaults: ImageGenerateToolDefaults` carries model/backend/promptExpander from settings/Telegram. |
+| `execute(args)` entry | [1325-1390](packages/execution/src/tools/image-generate.ts#L1325-L1390) | Dispatch to list/setup/prewarm/generate; calls `imageGenerationFallbackCandidates`. |
+| Backend dispatch | [1459-1463](packages/execution/src/tools/image-generate.ts#L1459-L1463) | `ollama` / `sdcpp` / `diffusers` per candidate. |
+| Sidecar writer | [1502-1534](packages/execution/src/tools/image-generate.ts#L1502-L1534) | Writes `<image>.json` alongside the PNG with original/expanded prompt, model, backend, seed, dimensions. |
+| Python runner template | [507-641](packages/execution/src/tools/image-generate.ts#L507-L641) | `DIFFUSERS_RUNNER` heredoc; auto-pick FluxPipeline / SD3Pipeline / AutoPipelineForText2Image. |
+| Venv bootstrap | [1077-1130](packages/execution/src/tools/image-generate.ts#L1077-L1130) | `ensurePythonFor("diffusers" | "sdcpp")`. |
+| Cache layout | [1037-1052](packages/execution/src/tools/image-generate.ts#L1037-L1052) | HF/Torch/pip caches pinned to `.omnius/image-gen/{huggingface,torch,cache,pip-cache}`. |
+| Preset list | [147-486](packages/execution/src/tools/image-generate.ts#L147-L486) | `IMAGE_GENERATION_MODEL_PRESETS` (Z-Image Turbo, FLUX1 dev variants, SD3.5, Hunyuan, SDXL Turbo, SD-Turbo, LCM, Sana Sprint, sdcpp). |
+| Quality ladder | [488-501](packages/execution/src/tools/image-generate.ts#L488-L501) | `IMAGE_GENERATION_QUALITY_LADDER` — ordered IDs that drive `imageGenerationFallbackCandidates` (lines [832-868](packages/execution/src/tools/image-generate.ts#L832-L868)). |
+| Default constants | [59-60](packages/execution/src/tools/image-generate.ts#L59-L60) | `DEFAULT_DIFFUSERS_IMAGE_MODEL = "stabilityai/sdxl-turbo"` (**this becomes Sana**), `DEFAULT_OLLAMA_IMAGE_MODEL = "x/z-image-turbo"`. |
+
+### 1.2 TUI surface — `/image`
+
+File: [packages/cli/src/tui/commands.ts](packages/cli/src/tui/commands.ts) (15065 lines)
+
+| Anchor | Lines | What it is |
+| --- | --- | --- |
+| Dispatcher case | [1590-1592](packages/cli/src/tui/commands.ts#L1590-L1592) | `case "image": return handleImageCommand(...)`. |
+| `handleImageCommand` | [9169-9246](packages/cli/src/tui/commands.ts#L9169-L9246) | Parses args, runs menu/list/setup/prewarm/generate, renders ASCII preview. |
+| `parseImageCommand` | [8759-8779](packages/cli/src/tui/commands.ts#L8759-L8779) | Flag/positional parser used by `/image`. |
+| `showImageModelsMenu` | [9054-9145](packages/cli/src/tui/commands.ts#L9054-L9145) | Hardware-aware fit menu with Del-to-delete-weights. |
+| `renderImageModelList` | [8877-8911](packages/cli/src/tui/commands.ts#L8877-L8911) | `/image list` output. |
+| `prewarmImageModel` | [9147-9167](packages/cli/src/tui/commands.ts#L9147-L9167) | Runs `ImageGenerateTool#execute({action: "prewarm"})`. |
+| ASCII preview | [9226-9244](packages/cli/src/tui/commands.ts#L9226-L9244) | `buildImageAsciiPreview` + `extractSavedImagePath` from [image-ascii-preview.ts](packages/cli/src/tui/image-ascii-preview.ts). |
+| Progress formatter | [9248-9260](packages/cli/src/tui/commands.ts#L9248-L9260) | Renders the `omnius_progress` JSON stream. |
+
+### 1.3 Command registry surface
+
+File: [packages/cli/src/tui/command-registry.ts](packages/cli/src/tui/command-registry.ts)
+
+| Anchor | Lines | What it is |
+| --- | --- | --- |
+| Slash help entries | [136-152](packages/cli/src/tui/command-registry.ts#L136-L152) | `/image`, `/sound`, `/music` signatures + descriptions. |
+| Category overrides | [344-435](packages/cli/src/tui/command-registry.ts#L344-L435) | `image: "media"` — video must be added here too. |
+| User-only set | [479-544](packages/cli/src/tui/command-registry.ts#L479-L544) | Both `/image` and `/sound`-`/music` are user-only; video should join. |
+| Networked set | [547-576](packages/cli/src/tui/command-registry.ts#L547-L576) | `image` is networked because Ollama/HF pulls hit the network. Same applies to video. |
+| Telegram bot command export | [681-709](packages/cli/src/tui/command-registry.ts#L681-L709) | `buildTelegramBotCommands` — adding `/video` here automatically wires the Telegram BotFather command. |
+
+### 1.4 Settings persistence
+
+File: [packages/cli/src/tui/omnius-directory.ts](packages/cli/src/tui/omnius-directory.ts)
+
+| Anchor | Lines | What it is |
+| --- | --- | --- |
+| `OmniusSettings` interface | [171-235](packages/cli/src/tui/omnius-directory.ts#L171-L235) | Adds `imageModel/imageBackend/soundModel/soundBackend/musicModel/musicBackend` — extend with `videoModel`/`videoBackend`/`videoKind` (default sub-kind). |
+| Project/Global save+merge | [251-291](packages/cli/src/tui/omnius-directory.ts#L251-L291) | Already merges arbitrary keys — no changes needed beyond the interface. |
+
+### 1.5 Telegram bridge surface
+
+File: [packages/cli/src/tui/telegram-bridge.ts](packages/cli/src/tui/telegram-bridge.ts) (8798 lines)
+
+| Anchor | Lines | What it is |
+| --- | --- | --- |
+| Tool imports | [85-104](packages/cli/src/tui/telegram-bridge.ts#L85-L104) | Imports `ImageGenerateTool`, `AudioGenerateTool`, `TtsGenerateTool` — extend with `VideoGenerateTool`. |
+| Admin-DM tool list | [6119-6184](packages/cli/src/tui/telegram-bridge.ts#L6119-L6184) | `new ImageGenerateTool(...)`, `new AudioGenerateTool(...)` — add `new VideoGenerateTool(...)`. |
+| Quota tag map | [6246-6253](packages/cli/src/tui/telegram-bridge.ts#L6246-L6253) | `generate_image|generate_audio|generate_tts|create_audio_file` → `"generation"` quota. Extend regex to include `generate_video`. |
+| Public/group creative tool factory call | [6205-6213](packages/cli/src/tui/telegram-bridge.ts#L6205-L6213) | Invokes `buildTelegramCreativeTools` — the place that scopes creative tools to a per-chat workspace. |
+| Defaults helpers | [7346-7368](packages/cli/src/tui/telegram-bridge.ts#L7346-L7368) | `imageGenerationDefaultsForRepo` / `audioGenerationDefaultsForRepo` — mirror with `videoGenerationDefaultsForRepo`. |
+| Generated-image sidecar reader | [8095-8158](packages/cli/src/tui/telegram-bridge.ts#L8095-L8158) | Reads `<image>.json` after sendPhoto to enable reply-context awareness — re-use for `<video>.json` after sendVideo. |
+| Multipart upload | [8050-8093](packages/cli/src/tui/telegram-bridge.ts#L8050-L8093) | `sendMediaReference` — already routes `kind: "video"` to `sendVideo` via `mediaTelegramMethod` in [media-routing.ts](packages/cli/src/tui/media-routing.ts) (lines [129-144](packages/cli/src/tui/media-routing.ts#L129-L144)). |
+
+### 1.6 Telegram public/group creative workspace
+
+File: [packages/cli/src/tui/telegram-creative-tools.ts](packages/cli/src/tui/telegram-creative-tools.ts)
+
+| Anchor | Lines | What it is |
+| --- | --- | --- |
+| Tool factory | [139-157](packages/cli/src/tui/telegram-creative-tools.ts#L139-L157) | `buildTelegramCreativeTools` returns scoped instances of FileWrite/FileEdit/StructuredFile/Image/Audio/Tts + CreativeAudioFile. **Add `VideoGenerateTool` here.** |
+| Scoped tool wrapper | [159-293](packages/cli/src/tui/telegram-creative-tools.ts#L159-L293) | Path guarding, encrypted blob storage, attachment notice. The `generate_image|generate_audio|generate_tts` regex on line 174 must include `generate_video`. |
+| Manifest pickup of generated artifacts | [116-137](packages/cli/src/tui/telegram-creative-tools.ts#L116-L137) | `collectGeneratedArtifactPathsFromText` parses lines like `Image generated: <path>` and `Sound generated: ...` — extend marker regex to also catch `Video generated:`. |
+| Public artifact policy | [99-105](packages/cli/src/tui/telegram-creative-tools.ts#L99-L105) | Blocks executable extensions. `.mp4/.webm/.mov/.mkv` are not on the blocklist — videos are allowed. |
+
+### 1.7 Media routing (kind classification + Telegram method)
+
+File: [packages/cli/src/tui/media-routing.ts](packages/cli/src/tui/media-routing.ts) (already complete for video)
+
+| Anchor | Lines | Status |
+| --- | --- | --- |
+| `VIDEO_EXT` set | [27](packages/cli/src/tui/media-routing.ts#L27) | `.mp4 .mov .mkv .webm .avi .m4v` — already covers all model outputs. |
+| `classifyMedia` | [117-127](packages/cli/src/tui/media-routing.ts#L117-L127) | Returns `"video"` for the above extensions. |
+| `mediaTelegramMethod("video")` | [139-140](packages/cli/src/tui/media-routing.ts#L139-L140) | Routes to `sendVideo`. **No edits needed.** |
+
+### 1.8 Execution package exports
+
+File: [packages/execution/src/index.ts](packages/execution/src/index.ts)
+
+Currently exports `ImageGenerateTool`, `ImageGenerateToolDefaults`, `AudioGenerateTool`, `AudioGenerateToolDefaults`, `VideoUnderstandTool`. **Add** exports for `VideoGenerateTool`, `VideoGenerateToolDefaults`, `VideoGenerationBackend`, `VideoGenerationPreset`, `VIDEO_GENERATION_MODEL_PRESETS`, `inferVideoGenerationBackend`, `videoGenerationSetupPlan`, `videoGenerationDir`, `videoGenerationQualityLadder`, `DEFAULT_DIFFUSERS_VIDEO_MODEL`.
+
+---
+
+## 2. Deliverable 1 — Sana as primary image model
+
+### 2.1 What "primary" means here
+
+The `/image` quality ladder ([packages/execution/src/tools/image-generate.ts:488](packages/execution/src/tools/image-generate.ts#L488-L501)) defines fallback order. Today `SECONDARY_FLUX_DEV_MODEL` sits at slot 0. With Sana installed, FLUX still requires gated HF access and 12GB+ VRAM; Sana 1.6B runs on 8-12GB VRAM under Apache/NSCL with **no gating**, has a Diffusers pipeline class shipping in `diffusers>=0.32`, and was already a fallback rung as `Sana Sprint 0.6B`. We promote the **SANA-1.5 1.6B 1024px** variant to slot 0 and add **SANA-1.5 4.8B** as a higher-quality option above it.
+
+### 2.2 Sana model variants to register
+
+Verified against <https://github.com/NVlabs/Sana> and HF model cards.
+
+| ID | Pipeline | Steps | CFG | Resolution | dtype | Min VRAM | Notes |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| `Efficient-Large-Model/SANA1.5_1.6B_1024px_diffusers` | `SanaPipeline` | 20 | 4.5 | 1024×1024 | bfloat16 | 8 GB (CPU offload) / 12 GB recommended | **New primary.** Apache+NSCL/Gemma terms. Text enc.: `google/gemma-2-2b-it`. |
+| `Efficient-Large-Model/SANA1.5_4.8B_1024px_diffusers` | `SanaPipeline` | 20 | 4.5 | 1024×1024 | bfloat16 | 16 GB / 24 GB recommended | Higher-fidelity Sana 1.5. |
+| `Efficient-Large-Model/Sana_1600M_1024px_MultiLing` | `SanaPipeline` | 20 | 4.5 | 1024×1024 | bfloat16 | 8 GB / 12 GB | Multilingual variant (CN/emoji/EN). |
+| `Efficient-Large-Model/Sana_1600M_2Kpx_BF16_diffusers` | `SanaPipeline` | 20 | 4.5 | 2048×2048 | bfloat16 | 16 GB / 22 GB tiled | 2K quality. |
+| `Efficient-Large-Model/Sana_1600M_4Kpx_BF16_diffusers` | `SanaPipeline` | 20 | 4.5 | 4096×4096 | bfloat16 | 22 GB tiled | 4K — gated by tiling. |
+| `Efficient-Large-Model/Sana_Sprint_0.6B_1024px_diffusers` | `SanaSprintPipeline` (or `AutoPipelineForText2Image`) | 4 | 0 | 1024×1024 | float16 | 6 GB | **Existing preset** — keep as the smallest/fastest path. |
+
+### 2.3 Edits to `image-generate.ts`
+
+1. Update `DEFAULT_DIFFUSERS_IMAGE_MODEL` (line [59](packages/execution/src/tools/image-generate.ts#L59)):
+   ```ts
+   export const DEFAULT_DIFFUSERS_IMAGE_MODEL = "Efficient-Large-Model/SANA1.5_1.6B_1024px_diffusers";
+   ```
+2. **Insert new presets** between the existing Sana Sprint preset (line [453](packages/execution/src/tools/image-generate.ts#L453-L469)) and the SDXL Turbo block. Use the same `ImageGenerationPreset` shape — copy from Sana Sprint and adjust `id/label/install/category/sizeClass/quality/minVramGB/recommendedVramGB/steps/guidance/width/height/note`.
+3. **Update `IMAGE_GENERATION_QUALITY_LADDER`** (line [488](packages/execution/src/tools/image-generate.ts#L488-L501)) so Sana 1.5 4.8B → Sana 1.5 1.6B → existing FLUX trio → Z-Image → … Sana 1.5 1.6B should appear ahead of the FLUX entries because it has no gating risk; the rest of the ladder stays as a safety net.
+4. **Patch `_pipeline_class`** in `DIFFUSERS_RUNNER` heredoc (line [529-538](packages/execution/src/tools/image-generate.ts#L529-L538)): add an explicit branch for Sana, since `AutoPipelineForText2Image` already picks `SanaPipeline` in diffusers ≥ 0.32 but Sana 1.5 needs a `text_encoder` dtype cast to bf16 (per Sana HF docs). Append after the `flux` branch:
+   ```python
+   if "sana" in lowered:
+       from diffusers import SanaPipeline
+       return SanaPipeline
+   ```
+   And in the call site, after `pipe = pipeline_cls.from_pretrained(...)`:
+   ```python
+   if "sana" in args.model.lower() and hasattr(pipe, "text_encoder"):
+       try:
+           pipe.text_encoder.to(torch.bfloat16)
+       except Exception:
+           pass
+   ```
+5. **Bump `_large_model` heuristic** (line [540-542](packages/execution/src/tools/image-generate.ts#L540-L542)) to enable `enable_model_cpu_offload` for the 4.8B Sana and the 2K/4K variants:
+   ```python
+   return any(token in lowered for token in [
+       "flux.1", "flux.2", "stable-diffusion-3.5", "hunyuan", "z-image", "janus",
+       "sana1.5_4.8b", "sana_1600m_2kpx", "sana_1600m_4kpx",
+   ])
+   ```
+6. **Description string** update at line [1209-1215](packages/execution/src/tools/image-generate.ts#L1209-L1215): swap "SDXL Turbo default" for "**Sana 1.5 1.6B default**".
+7. **Add Sana to fallback alternates**: when a FLUX preset fails (gated/oom), Sana should be a target. Set `fallbackFor: ["lllyasviel/flux1-dev-bnb-nf4", "ChuckMcSneed/FLUX.1-dev"]` on the new Sana 1.5 1.6B preset entry.
+
+### 2.4 No new venv needed
+
+`SanaPipeline` ships in `diffusers` (already in `DIFFUSERS_PYTHON_PACKAGES` at line [130-140](packages/execution/src/tools/image-generate.ts#L130-L140)). Sana 1.5 also pulls `google/gemma-2-2b-it` — already covered by `transformers` + `sentencepiece` + `protobuf`. **No requirements changes.**
+
+### 2.5 First-run prewarm (Sana)
+
+Plan and run automatically via the existing `prewarmImageModel(ctx, "Efficient-Large-Model/SANA1.5_1.6B_1024px_diffusers", "diffusers")` ([commands.ts:9147](packages/cli/src/tui/commands.ts#L9147)) path. The runner already streams `omnius_progress` JSON, which the TUI/Telegram surfaces consume identically.
+
+---
+
+## 3. Deliverable 2 — `/video` pipeline
+
+### 3.1 Scope
+
+- TUI: `/video`, `/video <prompt>`, `/video --model <id> <prompt>`, `/video setup <runtime>`, `/video list`, `/video prewarm`.
+- Sub-kinds: `text-to-video` (T2V, default) and `image-to-video` (I2V, when an `--image` flag or attached image is present).
+- Agent tool: `generate_video` (Diffusers-first; Ollama is not a path because no current Ollama model emits video). Telegram public + admin contexts.
+- Telegram BotFather command `/video` auto-registered via `buildTelegramBotCommands` since the command registry covers it.
+- Outputs: MP4 saved under `.omnius/videos/vid-<timestamp>-<rand>.mp4` with thumbnail PNG at `<video>.png` and sidecar JSON at `<video>.json`.
+- Preview: TUI renders ASCII preview of the thumbnail; Telegram auto-attaches video via existing `sendVideo` routing.
+
+### 3.2 New types — `VideoGenerationBackend`, `VideoGenerationKind`, `VideoGenerationPreset`
+
+To live in [packages/execution/src/tools/video-generate.ts](packages/execution/src/tools/video-generate.ts) (new file).
+
+```ts
+export type VideoGenerationKind = "t2v" | "i2v";
+export type VideoGenerationBackend = "auto" | "diffusers" | "comfyui";
+
+export interface VideoGenerationPreset {
+  id: string;
+  label: string;
+  kinds: VideoGenerationKind[];     // models can support t2v, i2v, or both
+  backend: Exclude<VideoGenerationBackend, "auto">;
+  pipelineClass:                    // signals which Diffusers class to use
+    | "WanPipeline"
+    | "MochiPipeline"
+    | "CogVideoXPipeline"
+    | "CogVideoXImageToVideoPipeline"
+    | "LTXPipeline"
+    | "LTXConditionPipeline"
+    | "HunyuanVideoPipeline"
+    | "AutoPipelineForText2Video";
+  install: string;                  // canonical pip/cli line shown in setup
+  category: string;
+  sizeClass: string;
+  quality: string;
+  output: string;                   // e.g. "5s 720p MP4 at 24fps"
+  minVramGB: number;
+  recommendedVramGB: number;
+  deployment: string;
+  steps: number;
+  guidance?: number;
+  numFrames: number;
+  fps: number;
+  width: number;
+  height: number;
+  dtype: "bfloat16" | "float16";
+  needsCpuOffload: boolean;         // forces enable_model_cpu_offload + tiling
+  needsVae?: { repoSubfolder: string }; // Wan2.2 separate AutoencoderKLWan
+  fallbackFor?: string[];
+  note: string;
+}
+
+export const DEFAULT_DIFFUSERS_VIDEO_MODEL = "Wan-AI/Wan2.2-TI2V-5B-Diffusers";
+```
+
+### 3.3 Video preset registry (Phase 1 + Phase 2 from the reference doc)
+
+These are the verified Diffusers-runnable models. Each entry uses the shape above.
+
+| ID | Pipeline | Kinds | dtype | Frames | FPS | W×H | Steps | CFG | Min/Rec VRAM | License |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| **`Wan-AI/Wan2.2-TI2V-5B-Diffusers`** | `WanPipeline` (+`AutoencoderKLWan`) | t2v, i2v | bfloat16 | 121 | 24 | 1280×704 | 50 | 5.0 | 24 / 80 | Apache 2.0 |
+| `genmo/mochi-1-preview` | `MochiPipeline` | t2v | bfloat16 (variant `bf16`) | 84 | 30 | 848×480 | 64 | 4.5 | 22 (offload) / 42 | Apache 2.0 |
+| `zai-org/CogVideoX-5b` | `CogVideoXPipeline` | t2v | bfloat16 | 49 | 8 | 720×480 | 50 | 6.0 | 5 (offload) / 15 | Apache 2.0 |
+| `zai-org/CogVideoX-2b` | `CogVideoXPipeline` | t2v | bfloat16 | 49 | 8 | 720×480 | 50 | 6.0 | 4 / 8 | Apache 2.0 |
+| `Lightricks/LTX-Video` | `LTXPipeline` | t2v | bfloat16 | 121 | 24 | 832×480 | 30 | n/a | 12 / 20 | LTX Open-Weights (non-commercial) |
+| `Lightricks/LTX-Video-0.9.8-dev` | `LTXConditionPipeline` | i2v | bfloat16 | 96 | 24 | 832×480 | 30 | n/a | 16 / 24 | LTX Open-Weights |
+| `tencent/HunyuanVideo` | `HunyuanVideoPipeline` | t2v | bfloat16 | 129 | 24 | 1280×720 | 50 | 6.0 | 60 / 80 | Tencent Hunyuan Community |
+| `Wan-AI/Wan2.2-I2V-A14B-Diffusers` | `WanPipeline` | i2v | bfloat16 | 121 | 24 | 1280×720 | 50 | 5.0 | 40 / 80 | Apache 2.0 |
+| `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `WanPipeline` | t2v | bfloat16 | 121 | 24 | 1280×720 | 50 | 5.0 | 40 / 80 | Apache 2.0 |
+
+**Default**: `Wan-AI/Wan2.2-TI2V-5B-Diffusers` — practical 24GB VRAM target, supports both T2V and I2V, becomes the agent default.
+
+**Quality ladder** (consumer-first, gated/biggest last):
+```text
+1. Wan2.2 TI2V 5B            (primary; both t2v+i2v)
+2. LTX-Video 0.9.8 (T2V)     (fast iteration)
+3. CogVideoX-5B              (5 GB VRAM with offload)
+4. Mochi 1 Preview           (permissive)
+5. CogVideoX-2B              (smoke-test fallback)
+6. Wan2.2 I2V A14B           (premium I2V)
+7. Wan2.2 T2V A14B           (premium T2V)
+8. HunyuanVideo              (cinematic, datacenter)
+```
+
+### 3.4 Python runner template (`DIFFUSERS_VIDEO_RUNNER`)
+
+Mirror `DIFFUSERS_RUNNER` from `image-generate.ts`. Key differences:
+
+- Selects pipeline class from `model` via the same lowered-string heuristics (`wan`, `mochi`, `cogvideox`, `ltx`, `hunyuanvideo`). For Wan, additionally instantiate `AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)` and pass `vae=vae`.
+- For I2V, accept `--image <path>` and feed via `image=load_image(args.image)`.
+- Use `from diffusers.utils import export_to_video` to write MP4 at the model's preferred fps.
+- After export, run `ffmpeg -hide_banner -loglevel error -ss 00:00:00 -i <out.mp4> -frames:v 1 -q:v 2 <out.mp4>.png` (or use OpenCV) to produce a thumbnail; emit thumbnail path in the final JSON line.
+- Emit the same `{"omnius_progress": true, "stage": ..., "message": ..., "percent": ...}` lines on stderr so the TUI/telegram progress formatter keeps working.
+- Output final JSON line: `{"ok": true, "path": "<mp4>", "thumbnail": "<mp4>.png", "frames": N, "width": W, "height": H, "fps": F, "duration_seconds": D, "model": "<id>", "device": "cuda"}`.
+
+```python
+def _video_pipeline(model):
+    lowered = model.lower()
+    if "wan" in lowered:
+        from diffusers import WanPipeline, AutoencoderKLWan
+        return ("wan", WanPipeline, AutoencoderKLWan)
+    if "mochi" in lowered:
+        from diffusers import MochiPipeline
+        return ("mochi", MochiPipeline, None)
+    if "cogvideox" in lowered:
+        from diffusers import CogVideoXPipeline, CogVideoXImageToVideoPipeline
+        return ("cogvideox", CogVideoXPipeline, CogVideoXImageToVideoPipeline)
+    if "ltx" in lowered:
+        from diffusers import LTXPipeline, LTXConditionPipeline
+        return ("ltx", LTXPipeline, LTXConditionPipeline)
+    if "hunyuanvideo" in lowered:
+        from diffusers import HunyuanVideoPipeline
+        return ("hunyuan", HunyuanVideoPipeline, None)
+    from diffusers import AutoPipelineForText2Video
+    return ("auto", AutoPipelineForText2Video, None)
+```
+
+The runner script lives at `.omnius/video-gen/diffusers_text2video.py` and is written by `ensureRunner(repoRoot, "video-diffusers")` (new variant of [image-generate.ts:1132-1141](packages/execution/src/tools/image-generate.ts#L1132-L1141)).
+
+### 3.5 Python deps
+
+```ts
+const DIFFUSERS_VIDEO_PACKAGES = [
+  "torch",
+  "torchvision",
+  "diffusers",
+  "transformers",
+  "accelerate",
+  "safetensors",
+  "pillow",
+  "sentencepiece",
+  "protobuf",
+  "imageio",
+  "imageio-ffmpeg",
+  "ftfy",     // common Hunyuan/CogVideoX requirement
+  "einops",   // LTX
+  "av",       // Wan VAE
+];
+```
+
+Cache layout mirrors image-gen at `.omnius/video-gen/{huggingface,torch,cache,pip-cache,.venv}`.
+
+### 3.6 `VideoGenerateTool` class skeleton
+
+Direct mirror of `ImageGenerateTool`. Required surface:
+
+```ts
+export class VideoGenerateTool implements Tool {
+  name = "generate_video";
+  description =
+    "Generate a short video from a text prompt or text+image using a local Diffusers video model. " +
+    "Default model: Wan-AI/Wan2.2-TI2V-5B-Diffusers (24GB-class GPU, both text-to-video and image-to-video). " +
+    "Pass mode='t2v' (default) or mode='i2v' with image=<path|url>. Optional duration_seconds, fps, aspect_ratio, " +
+    "negative_prompt, seed. Saves an MP4 under .omnius/videos and returns the file path. Outputs a thumbnail PNG " +
+    "next to the MP4 so chat surfaces can render a preview.";
+  parameters = { /* prompt, image, mode, model, backend, duration_seconds, fps,
+                    num_frames, width, height, aspect_ratio, steps, guidance,
+                    negative_prompt, seed, action, fallback, strict_model, expand_prompt */ };
+  setProgressCallback(...) { ... }
+  setPromptExpander(...) { ... }    // optional — reuses the same expander contract
+  async execute(args): Promise<ToolResult> { ... }
+}
+```
+
+Key parameter notes:
+
+- `mode`: `"t2v" | "i2v"`. If `image` is passed and `mode` is unset, infer `"i2v"`.
+- `duration_seconds` + `fps` derive `num_frames` if not provided.
+- `aspect_ratio`: same `W:H` parser as image — resolves around the preset's longest side, snapped to a multiple of 16 (LTX requires multiples of 32; for those models snap to 32 in the runner-side validation).
+- `negative_prompt`: forwarded to all models that accept it; ignored otherwise.
+- `strict_model`/`fallback`: same semantics as `ImageGenerateTool`.
+
+Reuse helpers from image-gen by lifting them into a tiny shared module (`packages/execution/src/tools/internal/generative-runtime.ts` if you want cleanliness, otherwise duplicate locally to keep the change surface tight):
+
+- `runProcess`, `cleanProgressText`, `parsePercent`, `parseStructuredProgress`, `parseRunnerJson`
+- `ensurePythonFor` (parameterize by `kind: "video-diffusers"` + dir + import-check string)
+- `imageGenerationPythonEnv` → generalize as `pythonGenerationEnv(rootDir)`
+
+> **Recommendation**: duplicate locally for the first cut to avoid touching the image-generate file beyond Sana edits. A follow-up PR can extract `internal/generative-runtime.ts`.
+
+### 3.7 Sidecar JSON for video
+
+Exact same shape as image sidecar ([image-generate.ts:1502-1534](packages/execution/src/tools/image-generate.ts#L1502-L1534)), extended fields:
+
+```json
+{
+  "version": 1,
+  "kind": "video-generation",
+  "video_path": "/repo/.omnius/videos/vid-…mp4",
+  "thumbnail_path": "/repo/.omnius/videos/vid-…mp4.png",
+  "original_prompt": "...",
+  "expanded_prompt": "...",
+  "prompt_was_expanded": true,
+  "mode": "t2v",
+  "image_input": null,
+  "model": "Wan-AI/Wan2.2-TI2V-5B-Diffusers",
+  "backend": "diffusers",
+  "width": 1280,
+  "height": 704,
+  "num_frames": 121,
+  "fps": 24,
+  "duration_seconds": 5.04,
+  "seed": null,
+  "aspect_ratio": "16:9",
+  "created_at": "2026-…Z"
+}
+```
+
+`recordOutboundGeneratedImagePrompt` in [telegram-bridge.ts:8102-8129](packages/cli/src/tui/telegram-bridge.ts#L8102-L8129) needs a parallel `recordOutboundGeneratedVideoPrompt` (or generalize the helper to take `kind`). The existing `sendMediaReference` path already accepts a `sourcePromptPath` option (see [telegram-bridge.ts:8087-8088](packages/cli/src/tui/telegram-bridge.ts#L8087-L8088)) — extend the conditional from `media.kind === "image"` to `media.kind === "image" || media.kind === "video"`.
+
+### 3.8 TUI command — `/video`
+
+Edit [packages/cli/src/tui/commands.ts](packages/cli/src/tui/commands.ts):
+
+1. Add dispatcher case beside `/image` ([line 1590-1600](packages/cli/src/tui/commands.ts#L1590-L1600)):
+   ```ts
+   case "video": {
+     return handleVideoCommand(ctx, arg, hasLocal);
+   }
+   ```
+2. Implement `handleVideoCommand`, `showVideoModelsMenu`, `renderVideoModelList`, `prewarmVideoModel`, `rateVideoPresetForHardware`, `videoModelDiskStats`, `deleteVideoModelWeights`, `formatVideoGenerationProgress`. Mirror the image helpers at [commands.ts:9054-9260](packages/cli/src/tui/commands.ts#L9054-L9260) — keep names parallel for grep-ability.
+3. Reuse `parseImageCommand` ([line 8759-8779](packages/cli/src/tui/commands.ts#L8759-L8779)) as-is — it is already generic over flags. Add `--image <path>` recognition through the existing flag pass-through.
+4. For ASCII preview, after generation completes, the runner emits a `thumbnail` field in its final JSON. Pipe that into `buildImageAsciiPreview` and render with a "Generated video" label and `Tap to play: <path>` line. Use `extractSavedImagePath`-style helper `extractSavedVideoPath(text, repoRoot)` matching `Video generated: <path>`.
+
+### 3.9 Command registry — `/video`
+
+Edit [packages/cli/src/tui/command-registry.ts](packages/cli/src/tui/command-registry.ts):
+
+1. Slash help block (after the `/music` entries at [line 147-151](packages/cli/src/tui/command-registry.ts#L147-L151)):
+   ```ts
+   ["/video", "Open video-generation model/setup menu"],
+   ["/video <prompt>", "Generate a video from a prompt and show an ASCII thumbnail"],
+   ["/video --model <model> <prompt>", "Generate with an explicit video model"],
+   ["/video --image <path|url> <prompt>", "Image-to-video animation"],
+   ["/video setup <diffusers|comfyui>", "Show setup commands for a video-generation backend"],
+   ["/video list", "List video models by category, quality, size, and hardware fit"],
+   ```
+2. Add `video: "media"` to `CATEGORY_OVERRIDES` ([line 344-435](packages/cli/src/tui/command-registry.ts#L344-L435)).
+3. Add `"video"` to the `USER_ONLY` and `NETWORKED` sets ([line 479-600](packages/cli/src/tui/command-registry.ts#L479-L600)).
+
+### 3.10 Settings interface
+
+Edit [packages/cli/src/tui/omnius-directory.ts](packages/cli/src/tui/omnius-directory.ts) at [line 228-229](packages/cli/src/tui/omnius-directory.ts#L228-L229):
+```ts
+/** Preferred video-generation model for /video and video-generation tool defaults */
+videoModel?: string;
+/** Preferred video-generation backend for /video */
+videoBackend?: "auto" | "diffusers" | "comfyui";
+/** Default sub-kind for /video when image input is absent */
+videoKind?: "t2v" | "i2v";
+```
+
+### 3.11 Telegram bridge wiring
+
+Edit [packages/cli/src/tui/telegram-bridge.ts](packages/cli/src/tui/telegram-bridge.ts):
+
+1. Imports at [line 85-104](packages/cli/src/tui/telegram-bridge.ts#L85-L104) — add `VideoGenerateTool` + `VideoGenerateToolDefaults`.
+2. Admin tool list at [line 6165-6168](packages/cli/src/tui/telegram-bridge.ts#L6165-L6168) — insert `new VideoGenerateTool(repoRoot, videoDefaults)`.
+3. Compute `videoDefaults` in the same place as `imageDefaults`/`audioDefaults` (search for `imageGenerationDefaultsForRepo`). Add:
+   ```ts
+   private videoGenerationDefaultsForRepo(repoRoot: string): VideoGenerateToolDefaults {
+     const settings = resolveSettings(repoRoot);
+     return {
+       model: typeof settings.videoModel === "string" && settings.videoModel.trim()
+         ? settings.videoModel
+         : undefined,
+       backend: settings.videoBackend,
+       defaultKind: settings.videoKind,
+     };
+   }
+   ```
+4. Quota regex at [line 6249](packages/cli/src/tui/telegram-bridge.ts#L6249):
+   ```ts
+   if (/^(generate_image|generate_audio|generate_video|generate_tts|create_audio_file)$/.test(toolName)) return "generation";
+   ```
+5. **Quota limits**: video is heavy. Add a dedicated `TelegramPublicQuotaKind` `"video-generation"` with a tighter window (e.g. 2/hour) in `TELEGRAM_PUBLIC_TOOL_QUOTAS` (search the same file for its declaration) and route `generate_video` to it explicitly.
+6. Public-context creative tool factory call at [line 6205-6212](packages/cli/src/tui/telegram-bridge.ts#L6205-L6212) — pass `videoDefaults` into the new factory signature.
+7. Outbound sidecar pickup at [line 8087-8093](packages/cli/src/tui/telegram-bridge.ts#L8087-L8093) — extend the condition so a successful `sendVideo` also calls `recordOutboundGeneratedMediaPrompt(chatId, messageId, videoPath, caption, "video")`. Read the `<video>.json` sidecar and store it on the chat history entry just like images today.
+8. Telegram BotFather command list — `/video` will register automatically via `listCommandRegistry` consumed by `buildTelegramBotCommands` ([command-registry.ts:681-709](packages/cli/src/tui/command-registry.ts#L681-L709)).
+9. System-prompt strings at [line 5555-5556](packages/cli/src/tui/telegram-bridge.ts#L5555-L5556): add a parallel sentence telling the model "For video generation requests, decide whether `generate_video` is appropriate (image-to-video if the user attached an image)."
+
+### 3.12 Telegram public/group creative tools
+
+Edit [packages/cli/src/tui/telegram-creative-tools.ts](packages/cli/src/tui/telegram-creative-tools.ts):
+
+1. Imports at [line 22-30](packages/cli/src/tui/telegram-creative-tools.ts#L22-L30) — add `VideoGenerateTool` + `VideoGenerateToolDefaults`.
+2. Factory signature at [line 139-145](packages/cli/src/tui/telegram-creative-tools.ts#L139-L145):
+   ```ts
+   export function buildTelegramCreativeTools(
+     repoRoot: string,
+     chatId: TelegramChatId | undefined,
+     backendUrl?: string,
+     imageDefaults: ImageGenerateToolDefaults = {},
+     audioDefaults: AudioGenerateToolDefaults = {},
+     videoDefaults: VideoGenerateToolDefaults = {},
+   ): Tool[] {
+   ```
+3. Factory body at [line 148-157](packages/cli/src/tui/telegram-creative-tools.ts#L148-L157) — append `scopedTool(new VideoGenerateTool(root, videoDefaults), root, "generate")`.
+4. Scoped wrapper regex at [line 174](packages/cli/src/tui/telegram-creative-tools.ts#L174):
+   ```ts
+   if (base.name === "generate_image" || base.name === "generate_audio" || base.name === "generate_video" || base.name === "generate_tts") {
+   ```
+5. `collectGeneratedArtifactPathsFromText` marker regex at [line 128](packages/cli/src/tui/telegram-creative-tools.ts#L128):
+   ```ts
+   line.match(/(?:Image generated|Sound generated|Music generated|Video generated|TTS generated|Created [A-Z]+ file|Created|Overwrote|Saved to):\s*([^\n\r(]+)/i);
+   ```
+6. **No blocklist edits needed** — `.mp4/.webm/.mov/.mkv/.m4v` are not in `PUBLIC_EXECUTABLE_ARTIFACT_EXTENSIONS` at [line 64-74](packages/cli/src/tui/telegram-creative-tools.ts#L64-L74), so videos already pass policy.
+
+### 3.13 Execution package exports
+
+Edit [packages/execution/src/index.ts](packages/execution/src/index.ts) — alongside existing `ImageGenerateTool` / `AudioGenerateTool` exports, re-export the entire `VideoGenerateTool` surface (class, defaults type, presets, backend type, ladder helper, setup plan, dir helper, default constant).
+
+### 3.14 First-run prewarm strategy
+
+Wan2.2 TI2V 5B is ~10 GB on disk and pulls T5 encoder + AutoencoderKLWan in addition to the transformer. The first `/video` call (or first `/video setup diffusers`) will:
+
+1. Create `.omnius/video-gen/.venv`.
+2. `pip install` `DIFFUSERS_VIDEO_PACKAGES`.
+3. Pull `Wan-AI/Wan2.2-TI2V-5B-Diffusers` via `WanPipeline.from_pretrained(...)` and `AutoencoderKLWan.from_pretrained(..., subfolder="vae")`.
+4. Run a `--prewarm` pass that loads the pipeline + VAE and exits. The runner must emit the same `omnius_progress` JSON for percent/stage so the TUI bar and Telegram updates work unchanged.
+
+Document this in the existing `--help`-style command output for `/video setup diffusers`.
+
+### 3.15 Hardware fit heuristic for video
+
+Add `rateVideoPresetForHardware(preset, specs)` in `commands.ts`. Use the same VRAM-based scoring as `rateImagePresetForHardware` but include a "frames * resolution / dtype" penalty so 4K Wan and 720p Hunyuan land in red on a 16GB GPU. Reuse `imageFitIcon`. Keep three labels: `excellent / comfortable / offload-only / heavy-cloud`.
+
+### 3.16 Tool description for the LLM (`generate_video` system surfacing)
+
+Critical for autonomous use. Make the tool description list:
+
+- Default model + that it runs both T2V and I2V.
+- That **I2V** is triggered when an `image` is provided (path or URL).
+- That `duration_seconds` ≤ 6 is recommended on consumer GPUs.
+- That the tool **takes long enough that the agent should set expectations** (≥ 2 minutes on 24GB even for the small Wan); see Telegram quota in §3.11.
+- That a thumbnail is auto-generated and attached for previews.
+- That the sidecar JSON enables reply-context — the agent can reference "what prompt made this video?".
+
+This matches the existing `ImageGenerateTool` description style ([image-generate.ts:1208-1215](packages/execution/src/tools/image-generate.ts#L1208-L1215)).
+
+### 3.17 Public Telegram safety review
+
+- **Public users cannot exfiltrate the repo**: `VideoGenerateTool` writes into the per-chat workspace root passed to `scopedTool` (the wrapper enforces `--output` is rewritten under `rootAbs` exactly like image+audio). Pattern: see [telegram-creative-tools.ts:171-235](packages/cli/src/tui/telegram-creative-tools.ts#L171-L235).
+- **Public users cannot bomb GPU**: `generate_video` lands under `"generation"` (or new `"video-generation"`) quota.
+- **Public users cannot pull arbitrary models**: agent gets `videoDefaults.model = settings.videoModel` and the description discourages exploring other models; admin can still override.
+- **Public users cannot use I2V to leak files**: scoped wrapper already materializes `image` arg through `materializeTelegramCreativeArtifactForSend` for any local-path inputs ([telegram-creative-tools.ts:187-203](packages/cli/src/tui/telegram-creative-tools.ts#L187-L203)). Add the same TTS-style key list for video — recognized image-input keys: `image`, `image_path`, `init_image`, `source_image`, `reference_image`.
+
+### 3.18 Thumbnail strategy
+
+Emit thumbnail from the runner with one of:
+
+- `ffmpeg -hide_banner -loglevel error -y -i <out.mp4> -frames:v 1 -q:v 2 <out.mp4>.png`
+- Or `imageio.get_reader(out_path).get_data(0).save(thumb_path)` if ffmpeg is missing.
+
+Set TUI ASCII preview to `<video>.png` since `image-ascii-preview.ts` already handles PNGs.
+
+### 3.19 ComfyUI path (optional, deferred to follow-up)
+
+The reference doc names ComfyUI as a strong runtime for Wan/Hunyuan. The plan defers ComfyUI to Phase 2 — implement the backend type now (`"comfyui"`) but `videoGenerationSetupPlan("comfyui", ...)` returns a "not yet supported in this build; use diffusers" message until a ComfyUI worker is wired. This keeps the menu layout future-proof without blocking shipping.
+
+### 3.20 LTX license note (HIGH attention)
+
+`Lightricks/LTX-Video` ships under the LTX Open-Weights License — **non-commercial**. The preset entry's `note` MUST surface this, and the LLM tool description should advise that the agent default (Wan2.2 TI2V 5B, Apache 2.0) is the safer baseline. We do not promote LTX to default; FLUX-style gating logic ([image-generate.ts:1031-1033](packages/execution/src/tools/image-generate.ts#L1031-L1033)) inspires the LTX error path: detect `LTX-Video-Open-Weights-License` files in the HF response and surface a license-accept reminder.
+
+### 3.21 HunyuanVideo gating note
+
+Tencent HunyuanVideo requires HF auth (license click-through). Reuse the existing gated-repo detection at [image-generate.ts:1031-1033](packages/execution/src/tools/image-generate.ts#L1031-L1033) — generalize it into `formatVideoFailure`.
+
+---
+
+## 4. Cross-cutting checklist for the downstream agent
+
+### 4.1 Build / type pipeline
+
+- `tsc --build` from the monorepo root must stay green. The new exports in `packages/execution/src/index.ts` need the new file to compile cleanly.
+- Update `packages/execution/tsconfig.tsbuildinfo` consumers automatically; no manual edits.
+- Add unit tests next to the image tests (look for `packages/cli/tests/image-ascii-preview.test.ts`-style pattern) — at minimum, register a `video-generate.preset.test.ts` that asserts every preset entry parses and the ladder is non-empty.
+
+### 4.2 Smoke tests (post-merge)
+
+Run the three reference doc smoke prompts (§8 in the package) via:
+
+```
+omnius /video setup diffusers
+omnius /video "A locked-off cinematic shot of a matte black cube on wet asphalt at night..."
+omnius /video --image .omnius/images/<sample>.png "Animate this image with subtle camera push-in..."
+```
+
+Then exercise via Telegram in (1) admin DM, (2) public group with bot added (use `aiwg-status` style ad-hoc check). Confirm:
+
+- ASCII thumbnail renders in TUI.
+- `sendVideo` delivers the file with caption containing the prompt.
+- Reply-to-video carries `originalPrompt` into the next turn (because of the sidecar pickup at §3.11.7).
+- Public quota blocks the third call within the window.
+
+### 4.3 Telegram BotFather refresh
+
+After shipping, run `aiwg ralph-status`-equivalent: the bot's command set updates via `setMyCommands` on startup. Confirm `/video` appears in the in-app menu by sending `/start` to the bot.
+
+### 4.4 Docs to drop alongside the PR
+
+- Update [README.md](README.md) media section (search for `/image` mention) to reference `/video`.
+- Add a one-line bullet to [docs/voice-flow-architecture.md](docs/voice-flow-architecture.md) or create a peer doc `docs/video-flow-architecture.md` summarizing the runner shape.
+
+### 4.5 Risk register
+
+| Risk | Mitigation |
+| --- | --- |
+| Disk: Wan 5B = ~10 GB; Hunyuan ≈ 60 GB | Document in `/video setup` plan output; never trigger Hunyuan pull on a fresh install — keep it behind explicit selection. |
+| Diffusers pipeline class drift between versions | Lock `diffusers>=0.32` in `DIFFUSERS_VIDEO_PACKAGES`; surface clear "update diffusers" failure note in `formatVideoFailure`. |
+| ffmpeg missing for thumbnail | Fall back to `imageio` in the runner; bridge gracefully if both fail (skip preview, send video without ASCII). |
+| Public spam (video is expensive) | Dedicated `video-generation` quota (≤ 2/hour/user); Telegram message acknowledges "this may take 2-5 minutes" before kickoff. |
+| User runs `/video` with no GPU | `rateVideoPresetForHardware` returns "too heavy" / red icon; `prewarmVideoModel` refuses if `specs.gpuVramGB === 0` and points to cloud. |
+| Aspect ratio not divisible by required factor | Runner snaps to multiple-of-32 for LTX, multiple-of-16 elsewhere; emit a `_progress("setup", "snapping ...")` line so the user sees the adjustment. |
+
+---
+
+## 5. Suggested implementation order
+
+1. **Sana primary swap** in `image-generate.ts` (one file, no new tool surface). Build, smoke `/image "test"`.
+2. **Skeleton video tool** in `packages/execution/src/tools/video-generate.ts` — types, preset registry, runner heredoc, no Tool class yet. `tsc` must compile.
+3. **`VideoGenerateTool` class** with `execute({ action: "list" })` only. Wire export. Build.
+4. **CogVideoX-2B path** — the smallest model — to validate the full Diffusers stack: prewarm + generate + thumbnail + sidecar. This is the equivalent of "Phase 1 smoke test" from the reference doc.
+5. **Wan2.2 TI2V 5B path** — verify T2V and I2V on a 24 GB-class GPU (or note CPU-offload expectation).
+6. **TUI surface** — dispatcher case, `handleVideoCommand`, menu, list, prewarm, ASCII preview hook.
+7. **Command registry** entries.
+8. **Settings interface** entries.
+9. **Telegram admin DM** wiring — add to `adminTools`, defaults helper, quota regex.
+10. **Telegram public/group** wiring — `buildTelegramCreativeTools` signature + factory body + marker regex.
+11. **Outbound sidecar pickup** for `sendVideo`.
+12. **License/gating warnings** for LTX + Hunyuan.
+13. **Tests** — preset shape, ladder, parseImageCommand reused for video flags.
+14. **Docs** — README + plan-link.
+
+Each step is independently shippable; do not bundle 1+2+3 into one commit.
+
+---
+
+## 6. File-by-file changelist
+
+| File | Change | Owner |
+| --- | --- | --- |
+| [packages/execution/src/tools/image-generate.ts](packages/execution/src/tools/image-generate.ts) | Promote Sana 1.5 1.6B to default; add 4.8B/2K/4K presets; patch `_pipeline_class`; update description; update ladder. | Sana swap |
+| [packages/execution/src/tools/video-generate.ts](packages/execution/src/tools/video-generate.ts) **(new)** | Full `VideoGenerateTool` class + presets + runner template + helpers. | Video |
+| [packages/execution/src/index.ts](packages/execution/src/index.ts) | Re-export Video* surface. | Video |
+| [packages/cli/src/tui/commands.ts](packages/cli/src/tui/commands.ts) | `case "video"`, `handleVideoCommand`, `showVideoModelsMenu`, `renderVideoModelList`, `prewarmVideoModel`, `formatVideoGenerationProgress`, `rateVideoPresetForHardware`, `videoModelDiskStats`, `deleteVideoModelWeights`, `extractSavedVideoPath`. | Video |
+| [packages/cli/src/tui/command-registry.ts](packages/cli/src/tui/command-registry.ts) | `/video` signatures; `CATEGORY_OVERRIDES.video = "media"`; add `video` to `USER_ONLY` + `NETWORKED`. | Video |
+| [packages/cli/src/tui/omnius-directory.ts](packages/cli/src/tui/omnius-directory.ts) | `videoModel`/`videoBackend`/`videoKind` on `OmniusSettings`. | Video |
+| [packages/cli/src/tui/telegram-bridge.ts](packages/cli/src/tui/telegram-bridge.ts) | Import; admin tool list; `videoGenerationDefaultsForRepo`; quota regex; outbound sidecar pickup; system prompt phrasing; new quota class. | Video |
+| [packages/cli/src/tui/telegram-creative-tools.ts](packages/cli/src/tui/telegram-creative-tools.ts) | Import; signature; factory body; scoped-wrapper regex; marker regex. | Video |
+| [packages/cli/tests/](packages/cli/tests/) | New `video-generate.preset.test.ts`; extend image preset test if shared registry helper introduced. | Video |
+| [README.md](README.md) | One-line `/video` doc near `/image`. | Video |
+| [docs/sana-and-video-generation-integration-plan.md](docs/sana-and-video-generation-integration-plan.md) **(this doc)** | Reference. | — |
+
+---
+
+## 7. Open decisions for the user
+
+1. **Default video model**: Wan2.2 TI2V 5B (recommended — 24GB-class, handles both T2V and I2V, Apache 2.0). Alternate: CogVideoX-5B if 16GB target is preferred for the broader install base.
+2. **Whether to also expose ComfyUI runtime now**: this plan defers it to a follow-up. Wire the type but not the backend.
+3. **Public Telegram quota for video**: suggested 2/hour/user. Configurable.
+4. **Whether the new tool should also generate **audio-with-video** (Wan2.2 S2V / LTX-2.3)**: out of scope for v1; document as a Phase 4 follow-up.
+
+---
+
+## 8. Reference excerpts (verified)
+
+- Wan2.2 TI2V 5B Diffusers: `WanPipeline` + `AutoencoderKLWan` subfolder VAE; 50 steps, CFG 5.0, 121 frames @ 24fps, 1280×704; min 24GB; Apache 2.0. (HF `Wan-AI/Wan2.2-TI2V-5B-Diffusers`)
+- CogVideoX-5B: `CogVideoXPipeline`; 50 steps, CFG 6.0, 49 frames @ 8fps, 720×480; min ~5GB with `enable_model_cpu_offload`. (HF `zai-org/CogVideoX-5b`)
+- Mochi 1 Preview: `MochiPipeline`; variant `bf16`; 64 steps, CFG 4.5 schedule, 84 frames @ 30fps, 848×480; min 22GB with offload; Apache 2.0. (HF `genmo/mochi-1-preview`)
+- LTX-Video: `LTXPipeline` / `LTXConditionPipeline`; 30 steps; frames divisible by 8+1; LTX Open-Weights non-commercial. (HF `Lightricks/LTX-Video`)
+- Sana 1.5 1.6B: `SanaPipeline`; 20 steps, CFG 4.5, 1024×1024, bfloat16; text encoder `google/gemma-2-2b-it`; NSCL v2-custom + Gemma terms; not gated. (HF `Efficient-Large-Model/SANA1.5_1.6B_1024px_diffusers`, <https://github.com/NVlabs/Sana>)
+
+---
+
+## 9. Quick map for the downstream agent (line-by-line cheat sheet)
+
+```text
+NEW    packages/execution/src/tools/video-generate.ts                 (full file, ~1500 LOC mirroring image-generate.ts)
+EDIT   packages/execution/src/tools/image-generate.ts
+         L59  default = "Efficient-Large-Model/SANA1.5_1.6B_1024px_diffusers"
+         L147 prepend 4 Sana 1.5 preset entries (1.6B / 4.8B / 1600M MultiLing / 2Kpx)
+         L488 ladder: place SANA1.5_4.8B then SANA1.5_1.6B above FLUX entries
+         L529 _pipeline_class: add Sana branch
+         L540 _large_model: enable offload for 4.8B / 2Kpx / 4Kpx
+         L1209 description: swap SDXL-Turbo for Sana 1.5 1.6B
+EDIT   packages/execution/src/index.ts                                (add Video* re-exports near L165-197)
+EDIT   packages/cli/src/tui/commands.ts
+         L1590 add `case "video"`
+         L8742 introduce ParsedVideoCommand parity OR reuse ParsedImageCommand
+         L9054 add showVideoModelsMenu mirror
+         L9147 add prewarmVideoModel mirror
+         L9169 add handleVideoCommand mirror
+EDIT   packages/cli/src/tui/command-registry.ts
+         L147 insert /video signatures
+         L344 add video: "media"
+         L506 add "video" to USER_ONLY
+         L555 add "video" to NETWORKED
+EDIT   packages/cli/src/tui/omnius-directory.ts
+         L228 append videoModel/videoBackend/videoKind
+EDIT   packages/cli/src/tui/telegram-bridge.ts
+         L85   add VideoGenerateTool import
+         L6165 add new VideoGenerateTool(repoRoot, videoDefaults)
+         L6205 pass videoDefaults to buildTelegramCreativeTools
+         L6249 add generate_video to quota regex (or new "video-generation" kind)
+         L7346 add videoGenerationDefaultsForRepo helper
+         L8087 broaden generated-media sidecar pickup beyond images
+         L5555 system-prompt note about generate_video
+EDIT   packages/cli/src/tui/telegram-creative-tools.ts
+         L24   add VideoGenerateTool / VideoGenerateToolDefaults imports
+         L128  add "Video generated" to marker regex
+         L139  add videoDefaults to factory signature
+         L152  add scopedTool(new VideoGenerateTool(...))
+         L174  add generate_video to the generation-routing regex
+```
+
+End of plan.