@kolbo/kolbo-code-linux-arm64-musl 2.1.6 → 2.1.7

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@kolbo/kolbo-code-linux-arm64-musl",
-  "version": "2.1.6",
+  "version": "2.1.7",
   "os": [
     "linux"
   ],

package/skills/kolbo/SKILL.md

@@ -66,11 +66,11 @@ You have direct access to the Kolbo AI creative platform via MCP tools (auto-con
 | `get_moodboard` | Fetch a moodboard's master_prompt, style_guide, and images. |
 | `list_presets` | Browse generation presets (image/video/music templates with bundled style direction). |
 
-### Chat
+### Chat & Vision
 
 | Tool | Description |
 |------|-------------|
-| `chat_send_message` | Send a message to Kolbo AI chat. Supports web search and deep think modes. |
+| `chat_send_message` | Send a message to Kolbo AI chat. Pass `media_urls` (array of public URLs) to analyze images, videos, or audio — Smart Select auto-routes to Gemini vision when media is detected. Omit `model` for automatic routing. Supports web search and deep think modes. |
 | `chat_list_conversations` | List your SDK chat conversations. |
 | `chat_get_messages` | Fetch messages in a conversation (with media URLs). |
 
@@ -141,13 +141,15 @@ When making multiple generation calls:
 
 ## Transcription & Audio/Video Analysis
 
-Use `transcribe_audio` whenever the user provides an audio or video file and wants:
+Use `transcribe_audio` ONLY when the user explicitly asks for:
 - A text transcript
 - Subtitles (SRT format)
 - Word-by-word timed subtitles (for karaoke, motion graphics, Remotion captions, video editing)
-- Content analysis or summary of spoken content
+- Summary of what was **spoken/said** in the video
 - Dialogue extraction from video
 
+**Do NOT use `transcribe_audio` to "analyze" a video visually.** For visual analysis (what's on screen, what's shown, what prompts appear, etc.) use `upload_media` → `chat_send_message` with `media_urls`.
+
 ### Workflow
 1. Call `transcribe_audio` with the `source` (URL or absolute local file path)
 2. The tool returns:
@@ -184,25 +186,30 @@ Transcription supports files up to 30 minutes. For longer content, split the fil
 
 ### Visual Video/Audio/Image Analysis
 
-**DEFAULT RULE: When the user shares a video or image file without a specific instruction, always do visual analysis — never ask, never default to transcription.**
-
-`transcribe_audio` is ONLY for when the user explicitly says "transcribe", "subtitles", "SRT", or "what's being said". Everything else — "what do you see?", "describe this", "analyze this", "what's in this video?", "what prompts are shown?", or just pasting a file path with no instruction — is visual analysis via Gemini.
+**The agent has built-in vision — use the right tool for the media type:**
 
-**NEVER use ffmpeg, `ollama-vision`, or extract frames manually. NEVER ask the user whether to transcribe or analyze — just execute visual analysis.**
+| Media type | How to analyze |
+|------------|----------------|
+| **Image** (jpg, png, webp, etc.) | Read it directly with the `Read` tool — the agent sees images natively. No upload needed. |
+| **Video / Audio** | `upload_media` → `chat_send_message` with `media_urls` (Gemini handles video/audio) |
+| **Transcription** | `transcribe_audio` — ONLY when user explicitly says "transcribe", "subtitles", "SRT", or "what's being said" |
 
-**Workflow for visual analysis (do this every time):**
-1. `upload_media({ source: "/absolute/local/path/to/file.mp4" })` → get CDN URL (skip if already a public URL)
-2. `chat_send_message({ message: "<your question>", model: "gemini-2.5-pro", media_urls: ["<cdn-url>"] })`
+**NEVER use ffmpeg or frame extraction for analysis. NEVER ask the user — just pick the right path above.**
 
-**Routing table — commit to an action, do not ask:**
+**Video/Audio analysis workflow — Step 1 is NOT optional:**
+1. `upload_media({ source: "/absolute/local/path/to/file.mp4" })` → returns `{ url, thumbnail_url, ... }`
+   - **Use `url`** — the actual CDN URL. Ignore `thumbnail_url` (preview JPG only).
+2. `chat_send_message({ message: "<your question>", media_urls: [result.url] })`
+   - **`media_urls` is mandatory** — the model only sees the video if you pass the CDN URL here.
+   - Always an **array**: `media_urls: ["https://cdn.kolbo.ai/..."]`
+   - **Omit `model`** — Smart Select auto-routes to Gemini when media is detected
+   - **Sessions do NOT remember media between messages.** On retry: reuse the same CDN `url` (no re-upload) but always pass `media_urls` again.
+   - **Batch / many videos**: pass `model: "gemini-3.1-flash-lite-preview"` explicitly for cheaper bulk runs
 
-| Trigger | Action |
-|---------|--------|
-| User says "transcribe" / "subtitles" / "SRT" / "what's being said" | `transcribe_audio` only |
-| User says "analyze" / "describe" / "what do you see" / "what's in this" / "what's happening" | Visual analysis — `upload_media` → `chat_send_message` + Gemini |
-| User shares a file path or video URL with no instruction | Visual analysis — `upload_media` → `chat_send_message` + Gemini |
-| User shares a video and asks about on-screen text / prompts / UI | Visual analysis — `upload_media` → `chat_send_message` + Gemini |
-| User wants both transcript AND visual description | Both — run `transcribe_audio` AND `chat_send_message` + Gemini |
+**❌ Never do this:**
+- Pass a local file path in `media_urls` — it won't work, only CDN URLs work
+- Use the `.txt` URL from a transcription result as the video URL — that's text, not video
+- Skip `upload_media` and try to construct a URL yourself
 
 When in doubt, do visual analysis. Do not stop to ask.
 
@@ -512,6 +519,8 @@ Natural-language triggers that should prompt this skill + a tool call:
 - "Transcribe this podcast episode" → `transcribe_audio`
 - "What's being said in this video?" → `transcribe_audio` → analyze the text
 - "Generate word-by-word subtitles for this audio" → `transcribe_audio` → share `word_by_word_srt_url`
+- "Analyze this video" / "What do you see?" / "What's in this?" (with video file) → `upload_media` → `chat_send_message` with `media_urls` (omit model — auto-routes to Gemini)
+- "What prompts are shown in this video?" → `upload_media` → `chat_send_message` with `media_urls` (omit model — auto-routes to Gemini)
 - "Keep the same character across all these images" → `create_visual_dna` → `generate_image` with `visual_dna_ids`
 - "Upload this file to my media library" → `upload_media`
 - "What video models are available?" → `list_models` (video)

package/skills/photo-studio/SKILL.md

@@ -1,10 +1,10 @@
 ---
 name: photo-studio
 description: >
-  Local AI photo generation and editing using FLUX.2 Klein 4B, Z-Image Turbo, and Ollama Gemma4.
+  Local AI photo generation and editing using FLUX.2 Klein 4B and Z-Image Turbo.
   Use when the user wants to generate or edit images locally (no API cost, no rate limits).
   Models at I:/AI-Models/. Script at G:/Projects/Kolbo.AI/github/training-loras/scripts/photo-studio.py
-  Keywords: generate image, edit image, flux klein, z-image turbo, local diffusion, photo studio, gemma4
+  Keywords: generate image, edit image, flux klein, z-image turbo, local diffusion, photo studio
 ---
 
 # Photo Studio — Local AI Image Generation & Editing
@@ -18,11 +18,11 @@ description: >
 | FLUX.2 Klein 4B | `I:/AI-Models/flux2-klein-4b/` |
 | Z-Image Turbo | `I:/AI-Models/z-image-turbo/` |
 | Z-Image Adapter | `I:/AI-Models/z-image-turbo-adapter/zimage_turbo_training_adapter_v2.safetensors` |
-| LLM / Vision | Ollama Gemma4 (runs locally at `http://localhost:11434`) |
+| Vision / LLM | Agent's built-in vision (`Read` tool) for images. For video analysis load the `video-production` skill. |
 
 ## How to Run
 
-Always use the ai-toolkit venv (has Flux2KleinPipeline + ZImagePipeline + ollama package):
+Always use the ai-toolkit venv (has Flux2KleinPipeline + ZImagePipeline):
 
 ```bash
 "G:/Projects/Kolbo.AI/github/ai-toolkit/venv/Scripts/python.exe" \
@@ -43,8 +43,6 @@ Always use the ai-toolkit venv (has Flux2KleinPipeline + ZImagePipeline + ollama
 | `--steps N` | 20 | Inference steps |
 | `--cfg N` | 3.5 | Guidance scale |
 | `--seed N` | random | Deterministic seed |
-| `--analyze` | off | Analyze `--image` with Gemma4, use as base description |
-| `--enhance` | off | Enhance `--prompt` with Gemma4 before generating |
 | `--adapter` | off | Load Z-Image Turbo adapter (zimage only) |
 
 ## Common Recipes
@@ -64,18 +62,29 @@ python photo-studio.py \
   --model flux --width 1152 --height 2048
 ```
 
-### Analyze image and generate variation
-```bash
+### Analyze image then generate variation
+```
+# Step 1: Read the image — the agent sees it natively (built-in vision)
+Read("/abs/path/to/char.jpg")
+→ Agent describes the person: clothing, pose, features, style
+
+# Step 2: Use the description as the prompt
 python photo-studio.py \
-  --image char.jpg --analyze \
-  --prompt "standing upright, full body" \
+  --prompt "<description from agent vision> standing upright, full body" \
   --model flux
 ```
 
-### Auto-enhance a short prompt then generate
-```bash
+### Enhance a short prompt then generate
+```
+# Step 1: Enhance the prompt with Kolbo MCP
+chat_send_message({
+  message: "Expand this into a detailed image generation prompt for a photorealistic portrait: 'street fashion guy'",
+})
+→ { content: "A young man in his mid-20s wearing..." }
+
+# Step 2: Generate with the enhanced prompt
 python photo-studio.py \
-  --prompt "street fashion guy" --enhance \
+  --prompt "<enhanced prompt from Kolbo>" \
   --model zimage --width 1152 --height 2048
 ```
 
@@ -101,11 +110,10 @@ python photo-studio.py \
 - Default: `--steps 8 --cfg 0.0 --width 1152 --height 2048` (~30s per image)
 - Add `--adapter` to load the v2 training adapter
 
-### Ollama Gemma4 (vision + LLM)
-- Used for `--analyze` (describe input image) and `--enhance` (expand prompt)
-- Runs locally, no API key, no rate limits
-- Model: `gemma4` (9.6GB, multimodal)
-- Ollama auto-starts on Windows boot
+### Vision & Prompt Enhancement
+- For image analysis: use the agent's built-in vision — `Read` the image file directly, no MCP needed
+- For prompt enhancement: `chat_send_message` asking Kolbo to expand a short prompt (text-only, no vision)
+- Do NOT use `--analyze` or `--enhance` flags (those call a local model that is no longer used)
 
 ## When to use which model
 
@@ -117,6 +125,6 @@ python photo-studio.py \
 | Quick text-to-image | `flux` or `zimage` |
 | Portrait + face reference | `flux --image face.jpg` |
 
-## Prompt Tips (Gemma4 is the default prompter)
+## Prompt Tips
 
-When the user gives a short/vague prompt, always use `--enhance` to let Gemma4 expand it. For image editing, pair `--analyze --enhance` to get the best context from the source image.
+When the user gives a short/vague prompt, use `chat_send_message` to let Kolbo AI expand it before passing to the script. For image editing, first analyze the source image with the agent's built-in vision (`Read` the image), then use the description as the base prompt.

package/skills/video-production/SKILL.md

@@ -1,13 +1,16 @@
 ---
 name: video-production
 description: >
-  Full-stack video production assistant. Analyzes talking head footage,
+  Full-stack video production assistant. Analyzes video content visually (Gemini),
   generates transcriptions/SRT subtitles, plans and creates motion graphics (Remotion),
   generates B-roll images/videos, produces timeline XMLs for Premiere/DaVinci.
-  Use for: video analysis, transcription, subtitles, motion graphics, B-roll, shorts,
-  timeline XML, clip cutting, silence removal, After Effects, Premiere Pro, DaVinci Resolve.
+  Downloads YouTube videos with yt-dlp.
+  Use for: video analysis, visual analysis, describe video, what's in this video,
+  transcription, subtitles, motion graphics, B-roll, shorts, timeline XML, clip cutting,
+  silence removal, After Effects, Premiere Pro, DaVinci Resolve, YouTube download.
   Keywords: video edit, ffmpeg, remotion, after effects, premiere, davinci, shorts, subtitles,
-  motion graphics, clip, render, transcribe, xml, timeline, b-roll, talking head, analyze
+  motion graphics, clip, render, transcribe, xml, timeline, b-roll, talking head, analyze,
+  yt-dlp, youtube, download, gemini, vision
 allowed-tools:
   - Read
   - Write
@@ -24,29 +27,122 @@ allowed-tools:
 
 # Video Production — Strategy Map
 
+## ⚠️ DEFAULT RULE: Video Analysis = Visual Analysis (NOT Transcription)
+
+**The agent has built-in vision for images. For videos, always use Gemini via Kolbo MCP.**
+
+| Media type | Action |
+|------------|--------|
+| **Image** (jpg, png, etc.) | Agent reads it directly — no upload needed |
+| **Video** — "analyze", "describe", "what's in this?", "what prompts?", file path with no instruction | `upload_media` → `chat_send_message` + Gemini |
+| **Transcription** — "transcribe", "subtitles", "SRT", "what's being said", "captions" | `transcribe_audio` only |
+| Both visual + transcript | Run both |
+
+**Never use ffmpeg to extract frames for analysis. Never use local Ollama/vision models. Commit to the right action — do not ask the user. Wait for `chat_send_message` to return before proceeding — it polls until done (up to 2 min). Do NOT fall back to ffmpeg or any other approach if it takes time.**
+
+---
+
+## Kolbo MCP Tools (Active When `kolbo auth login` Is Done)
+
+These are available as MCP tools — use them directly without any Python/API key setup:
+
+| Tool | Use |
+|------|-----|
+| `upload_media` | Upload local file to Kolbo CDN → get stable public URL |
+| `chat_send_message` | Send message + `media_urls` array to Gemini for visual analysis |
+| `transcribe_audio` | Transcribe audio/video to text + SRT (ElevenLabs Scribe) |
+| `generate_image` | Generate B-roll images |
+| `generate_video` | Generate B-roll videos |
+| `generate_video_from_image` | Animate a still into video |
+| `generate_music` | Generate background music |
+| `generate_speech` | TTS for voiceover |
+| `generate_sound` | Sound effects |
+| `list_models` | Browse available models by type |
+| `check_credits` | Check remaining Kolbo credit balance |
+
+### Visual Analysis Workflow — MANDATORY for all video analysis
+
+**Step 1 is NOT optional. You cannot skip `upload_media` or construct the URL yourself.**
+
+```
+Step 1: upload_media({ source: "/absolute/path/to/video.mp4" })
+  → Returns: { url, thumbnail_url, ... }
+  → Save the "url" field — this is the CDN URL you will pass to Gemini
+  → NEVER use thumbnail_url (it's a JPG preview, not the video)
+
+Step 2: chat_send_message({
+  message: "Describe this video in detail. What is shown?",
+  media_urls: ["<url from step 1>"]   ← must be an array, must be the "url" field
+})
+→ returns: { content: "..." }
+```
+
+**❌ Common mistakes that break video analysis:**
+- Skipping `upload_media` and passing a local file path to `chat_send_message` — local paths don't work
+- Using the transcription `.txt` URL as the `media_urls` value — Gemini needs the actual video CDN URL
+- Using `thumbnail_url` instead of `url` from the `upload_media` response
+- Calling `transcribe_audio` first then passing its output URL as the video — transcription gives text, not video
+
+**Omit `model`** — Smart Select detects video/audio and auto-routes to Gemini.
+**Sessions do NOT remember media between messages.** On retry: reuse the same CDN `url` from step 1 (no re-upload needed) but always pass `media_urls` again.
+
+**Batch analysis (many videos)**: Pass `model: "gemini-3.1-flash-lite-preview"` explicitly for cheaper bulk runs.
+
+For YouTube videos — download first with yt-dlp (see below), then follow steps 1–2 above.
+
+---
+
 ## Pipeline
 
 ```
-Input video → Transcribe → Analyze → Plan segments
-  → Generate: Remotion compositions | B-roll | SRT subtitles
-  → Output: Premiere XML / DaVinci EDL / individual MP4s / SRT
+Input: local video / YouTube URL / uploaded file
+
+→ [DEFAULT] Visual Analysis: upload_media → chat_send_message (Gemini)
+→ [EXPLICIT REQUEST] Transcription: transcribe_audio → SRT / text
+→ [EDITING] FFmpeg: cut, silence removal, 9:16 conversion
+→ [MOTION GRAPHICS] Remotion: compositions, captions, B-roll
+→ Output: Premiere XML / DaVinci EDL / MP4s / SRT
 ```
 
 ## APIs & Capabilities
 
 | Service | Use |
 |---------|-----|
-| ElevenLabs Scribe | Primary transcription — word-level SRT, multilingual |
-| Claude | Content analysis, edit planning |
-| Google Gemini | Video understanding, visual analysis |
-| fal.ai (MCP) | Image & video B-roll generation |
-| Runway | Image-to-video, video-to-video |
-| FLUX / BFL | High quality still image generation |
-| ElevenLabs | TTS, voice cloning, SFX |
-| Suno | Background music generation |
+| Kolbo MCP (`upload_media` + `chat_send_message`) | **Primary** — visual video/image analysis via Gemini |
+| Kolbo MCP (`transcribe_audio`) | **Primary** — transcription, word-level SRT, multilingual |
+| yt-dlp | Download YouTube/social media videos |
+| FFmpeg | Local video editing, cutting, silence removal, format conversion |
 | Remotion Lambda | Cloud render motion graphics |
+| fal.ai (MCP) | Image & video B-roll generation |
+| ElevenLabs | TTS, voice cloning, SFX (via Kolbo MCP `generate_speech`) |
+| Suno | Background music (via Kolbo MCP `generate_music`) |
+
+> Kolbo MCP tools need no API keys — auth is handled by `kolbo auth login`.
+> FFmpeg/yt-dlp need to be installed locally on the machine.
 
-> Load API keys from the project's `.env` file or environment variables.
+## YouTube / Social Media Download (yt-dlp)
+
+Download video from YouTube, TikTok, Instagram, Twitter, etc.:
+
+```bash
+# Best quality MP4
+yt-dlp -f "bestvideo[height<=1080][ext=mp4]+bestaudio/best" \
+  --merge-output-format mp4 \
+  -o "%(id)s.%(ext)s" <url>
+
+# With subtitles
+yt-dlp -f "bestvideo[height<=1080][ext=mp4]+bestaudio/best" \
+  --write-auto-sub --sub-lang en --convert-subs srt \
+  --merge-output-format mp4 \
+  -o "%(id)s.%(ext)s" <url>
+
+# Audio only (for transcription)
+yt-dlp -f "bestaudio" --extract-audio --audio-format mp3 -o "%(id)s.%(ext)s" <url>
+```
+
+After download → upload to Kolbo CDN with `upload_media` → analyze visually with `chat_send_message`.
+
+---
 
 ## Key Rules