omnius 1.0.51 → 1.0.53

package/README.md

@@ -281,7 +281,7 @@ The agent uses tools autonomously in a loop — reading errors, fixing code, and
 - **Moondream vision** — see and interact with the desktop via Moondream VLM (caption, query, detect, point-and-click)
 - **Image generation with TUI previews** — `/image <prompt>` and the `generate_image` tool create PNGs under `.omnius/images/`, support explicit `--model` selection, default to NVIDIA Sana 1.5 1.6B and try a ranked quality fallback ladder (Sana 1.5 4.8B → Sana 1.5 1.6B → FLUX.1 dev → SD3.5 Large → smaller smoke-test models) when setup or generation fails, and render generated, pasted, screenshot, and camera-capture images as auto-sized ASCII previews via the bundled `image-to-ascii` renderer
 - **Sound and music generation** — `/sound` and `/music` generate WAV files under `.omnius/audio/`, auto-create backend venvs under `.omnius/audio-gen/`, and fall back from high-quality Stable Audio / AudioLDM / MusicGen tiers to smaller practical models when a larger setup or model download fails. Stable Audio uses Diffusers `StableAudioPipeline` instead of the build-prone `stable-audio-tools` package
-- **Video generation with thumbnail previews** — `/video <prompt>` (text-to-video) or `/video --image <path> <prompt>` (image-to-video) and the `generate_video` tool emit MP4s under `.omnius/videos/`, default to Wan2.2 TI2V 5B (Diffusers), fall back through LTX-Video → CogVideoX 5B → CogVideoX 2B on OOM/gating/download failure, write a first-frame PNG thumbnail next to the MP4 for TUI ASCII preview, and persist a `<video>.json` sidecar (original/expanded prompt, mode, frames, fps) so Telegram replies to a generated video carry the source prompt forward
+- **Video generation with thumbnail previews** — `/video <prompt>` (text-to-video) or `/video --image <path> <prompt>` (image-to-video) and the `generate_video` tool emit MP4s under `.omnius/videos/`, default to Sana-Video 480p (NVlabs, 2B Linear DiT, ICLR 2026 Oral), fall back through Wan2.2 TI2V 5B → LTX-Video → CogVideoX 5B → CogVideoX 2B on OOM/gating/download failure, write a first-frame PNG thumbnail next to the MP4 for TUI ASCII preview, and persist a `<video>.json` sidecar (original/expanded prompt, mode, frames, fps) so Telegram replies to a generated video carry the source prompt forward
 - **Desktop automation** — vision-guided clicking: describe a UI element in natural language, the agent finds and clicks it
 - **Auto-install desktop deps** — screenshot, mouse, OCR, and image tools auto-install missing system packages (scrot, xdotool, tesseract, imagemagick) on first use
 - **Hardware-rated model lists** — first-run setup, `/models`, `/score`, and `/image list` score model fit against detected RAM/VRAM/GPU so text and image model choices are visible before you switch or generate
@@ -362,7 +362,7 @@ The daemon auto-installs Python dependencies (OpenCLIP, torchaudio + soundfile,
 - **Fortemi-React bridge** — `/fortemi start/status/stop` connects to [fortemi-react](https://github.com/robit-man/fortemi-react) (browser-first PGlite+pgvector knowledge system) via JWT auth. Proxy tools: `fortemi_capture`, `fortemi_search`, `fortemi_list`, `fortemi_get` auto-register when bridge is connected
 - **Content ingestion** — `/ingest <file>` imports audio (transcribe via Whisper), PDF (pdftotext), or text files into structured memory with 800-char/100-overlap chunking (matches fortemi pattern)
 - **Image generation** — `generate_image` supports Ollama image models, Diffusers models, and stable-diffusion.cpp checkpoints/GGUF. NVIDIA Sana 1.5 1.6B is the practical default auto-install path under `.omnius/image-gen/.venv` (Apache 2.0 / NSCL, no HF gating); Sana 1.5 4.8B sits above it for 24 GB-class GPUs, with FLUX.1 dev and Stable Diffusion 3.5 Large remaining available as gated high-realism baselines. `/image list` groups models by type, size, quality expectations, and hardware fit. Generation falls through the ranked model ladder unless `strict_model=true` or `fallback=false` is set
-- **Video generation** — `generate_video` runs Diffusers-based video pipelines (`WanPipeline`, `CogVideoXPipeline`/`CogVideoXImageToVideoPipeline`, `MochiPipeline`, `LTXPipeline`/`LTXConditionPipeline`, `HunyuanVideoPipeline`) under `.omnius/video-gen/.venv`. Wan2.2 TI2V 5B is the default; the tool autodetects `mode=t2v|i2v`, snaps width/height/frames to each model's required quantum, emits a `<video>.mp4` plus first-frame PNG thumbnail and `<video>.json` sidecar, and falls back through smaller-VRAM models on OOM or gating failure. The Telegram public/group quota for video generation is 2 clips per hour per user (vs. 10 for image/audio)
+- **Video generation** — `generate_video` runs Diffusers-based video pipelines (`WanPipeline`, `CogVideoXPipeline`/`CogVideoXImageToVideoPipeline`, `MochiPipeline`, `LTXPipeline`/`LTXConditionPipeline`, `HunyuanVideoPipeline`) under `.omnius/video-gen/.venv`. Sana-Video 480p (NVlabs) is the default; the tool autodetects `mode=t2v|i2v`, snaps width/height/frames to each model's required quantum, emits a `<video>.mp4` plus first-frame PNG thumbnail and `<video>.json` sidecar, and falls back through Wan2.2 TI2V 5B → smaller-VRAM models on OOM or gating failure. ComfyUI backend available via `--backend comfyui` for all models with registered workflows. Set `with_audio=true` to auto-generate and mux a matched soundtrack. The Telegram public/group quota for video generation is 2 clips per hour per user (vs. 10 for image/audio)
 - **Node visualization** — [omnius.nexus](https://github.com/robit-man/omnius.nexus) Three.js dashboard: 5-color emotional state mapping (neutral/focused/stressed/dreaming/excited), dynamic node size by memory depth + IPFS storage, activity-modulated connections, identity synchrony golden threads between mutually-pinned agents
 - **TTS sanitizer** — strips markdown syntax (`##`, `**`, `` ` ``), emoji (prevents "white heavy checkmark"), box-drawing chars, and ANSI codes before feeding to ALL TTS engines
 - **LuxTTS gapless playback** — look-ahead pre-synthesis pipeline: next chunk synthesizes while current plays, eliminating inter-sentence gaps. Jetson ARM support with NVIDIA's prebuilt PyTorch wheel