@kolbo/kolbo-code-linux-arm64-musl 2.1.5 → 2.1.7

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@kolbo/kolbo-code-linux-arm64-musl",
-  "version": "2.1.5",
+  "version": "2.1.7",
   "os": [
     "linux"
   ],

package/skills/kolbo/SKILL.md

@@ -66,11 +66,11 @@ You have direct access to the Kolbo AI creative platform via MCP tools (auto-con
 | `get_moodboard` | Fetch a moodboard's master_prompt, style_guide, and images. |
 | `list_presets` | Browse generation presets (image/video/music templates with bundled style direction). |
 
-### Chat
+### Chat & Vision
 
 | Tool | Description |
 |------|-------------|
-| `chat_send_message` | Send a message to Kolbo AI chat. Supports web search and deep think modes. |
+| `chat_send_message` | Send a message to Kolbo AI chat. Pass `media_urls` (array of public URLs) to analyze images, videos, or audio — Smart Select auto-routes to Gemini vision when media is detected. Omit `model` for automatic routing. Supports web search and deep think modes. |
 | `chat_list_conversations` | List your SDK chat conversations. |
 | `chat_get_messages` | Fetch messages in a conversation (with media URLs). |
 
@@ -141,13 +141,15 @@ When making multiple generation calls:
 
 ## Transcription & Audio/Video Analysis
 
-Use `transcribe_audio` whenever the user provides an audio or video file and wants:
+Use `transcribe_audio` ONLY when the user explicitly asks for:
 - A text transcript
 - Subtitles (SRT format)
 - Word-by-word timed subtitles (for karaoke, motion graphics, Remotion captions, video editing)
-- Content analysis or summary of spoken content
+- Summary of what was **spoken/said** in the video
 - Dialogue extraction from video
 
+**Do NOT use `transcribe_audio` to "analyze" a video visually.** For visual analysis (what's on screen, what's shown, what prompts appear, etc.) use `upload_media` → `chat_send_message` with `media_urls`.
+
 ### Workflow
 1. Call `transcribe_audio` with the `source` (URL or absolute local file path)
 2. The tool returns:
@@ -182,24 +184,34 @@ The regular `srt_url` groups words into readable subtitle lines (default 12 word
 ### Long Content
 Transcription supports files up to 30 minutes. For longer content, split the file first or provide segments.
 
-### Visual Video/Audio Analysis (what's happening, not just what's said)
-`transcribe_audio` only extracts **speech**. If the user wants to understand **what's visually happening** in a video (scenes, actions, objects, on-screen text) or needs a multimodal AI to reason about the content, use `chat_send_message` with a video-capable model instead.
+### Visual Video/Audio/Image Analysis
+
+**The agent has built-in vision — use the right tool for the media type:**
+
+| Media type | How to analyze |
+|------------|----------------|
+| **Image** (jpg, png, webp, etc.) | Read it directly with the `Read` tool — the agent sees images natively. No upload needed. |
+| **Video / Audio** | `upload_media` → `chat_send_message` with `media_urls` (Gemini handles video/audio) |
+| **Transcription** | `transcribe_audio` — ONLY when user explicitly says "transcribe", "subtitles", "SRT", or "what's being said" |
 
-**Video-capable models**: `gemini-2.5-pro`, `gemini-2.5-flash` — these can watch video and analyze visual content.
+**NEVER use ffmpeg or frame extraction for analysis. NEVER ask the user — just pick the right path above.**
 
-**Workflow for visual analysis:**
-1. Upload the video with `upload_media` to get a stable CDN URL
-2. Call `chat_send_message` with the video URL in the message and a video-capable model (e.g. `gemini-2.5-pro`)
-3. Ask your analysis question: "Describe what happens in this video", "What products are shown?", "Summarize the key scenes"
+**Video/Audio analysis workflow — Step 1 is NOT optional:**
+1. `upload_media({ source: "/absolute/local/path/to/file.mp4" })` → returns `{ url, thumbnail_url, ... }`
+   - **Use `url`** — the actual CDN URL. Ignore `thumbnail_url` (preview JPG only).
+2. `chat_send_message({ message: "<your question>", media_urls: [result.url] })`
+   - **`media_urls` is mandatory** — the model only sees the video if you pass the CDN URL here.
+   - Always an **array**: `media_urls: ["https://cdn.kolbo.ai/..."]`
+   - **Omit `model`** — Smart Select auto-routes to Gemini when media is detected
+   - **Sessions do NOT remember media between messages.** On retry: reuse the same CDN `url` (no re-upload) but always pass `media_urls` again.
+   - **Batch / many videos**: pass `model: "gemini-3.1-flash-lite-preview"` explicitly for cheaper bulk runs
 
-**When to use which:**
+**❌ Never do this:**
+- Pass a local file path in `media_urls` — it won't work, only CDN URLs work
+- Use the `.txt` URL from a transcription result as the video URL — that's text, not video
+- Skip `upload_media` and try to construct a URL yourself
 
-| User intent | Tool |
-|-------------|------|
-| "Transcribe this" / "What's being said?" | `transcribe_audio` |
-| "Generate subtitles" / "Word-by-word timing" | `transcribe_audio` |
-| "What's happening in this video?" / "Describe the scenes" | `chat_send_message` + Gemini |
-| "Analyze this video and transcribe it" | Both — `transcribe_audio` for text + `chat_send_message` for visual |
+When in doubt, do visual analysis. Do not stop to ask.
 
 ---
 
@@ -429,6 +441,20 @@ When the user shares an image and asks about it:
 
 ---
 
+## Sharing HTML Artifacts
+
+When you generate an HTML, SVG, or Mermaid artifact in the chat, a **Share** button appears in the artifact preview toolbar (next to Desktop / Mobile). Clicking it:
+
+1. Uploads the artifact to Kolbo's hosting platform
+2. Copies a permanent public URL to the clipboard (e.g. `https://api.kolbo.ai/api/shared-artifact-raw/<token>`)
+3. Shows a toast confirming the link was copied
+
+Anyone with the URL can view the rendered page — no login required.
+
+**Requirements:** You must be logged in (`kolbo auth login`). The share button returns an error toast if you are not authenticated.
+
+---
+
 ## Kolbo Code Documentation
 
 Full public documentation for Kolbo Code (the CLI you are running inside) lives at **[docs.kolbo.ai/docs/kolbo-code](https://docs.kolbo.ai/docs/kolbo-code)**. If the user asks about installation, authentication, voice input, supported languages, commands, or how to uninstall, point them to the matching page below rather than guessing:
@@ -493,6 +519,8 @@ Natural-language triggers that should prompt this skill + a tool call:
 - "Transcribe this podcast episode" → `transcribe_audio`
 - "What's being said in this video?" → `transcribe_audio` → analyze the text
 - "Generate word-by-word subtitles for this audio" → `transcribe_audio` → share `word_by_word_srt_url`
+- "Analyze this video" / "What do you see?" / "What's in this?" (with video file) → `upload_media` → `chat_send_message` with `media_urls` (omit model — auto-routes to Gemini)
+- "What prompts are shown in this video?" → `upload_media` → `chat_send_message` with `media_urls` (omit model — auto-routes to Gemini)
 - "Keep the same character across all these images" → `create_visual_dna` → `generate_image` with `visual_dna_ids`
 - "Upload this file to my media library" → `upload_media`
 - "What video models are available?" → `list_models` (video)

package/skills/photo-studio/SKILL.md

@@ -1,10 +1,10 @@
 ---
 name: photo-studio
 description: >
-  Local AI photo generation and editing using FLUX.2 Klein 4B, Z-Image Turbo, and Ollama Gemma4.
+  Local AI photo generation and editing using FLUX.2 Klein 4B and Z-Image Turbo.
   Use when the user wants to generate or edit images locally (no API cost, no rate limits).
   Models at I:/AI-Models/. Script at G:/Projects/Kolbo.AI/github/training-loras/scripts/photo-studio.py
-  Keywords: generate image, edit image, flux klein, z-image turbo, local diffusion, photo studio, gemma4
+  Keywords: generate image, edit image, flux klein, z-image turbo, local diffusion, photo studio
 ---
 
 # Photo Studio — Local AI Image Generation & Editing
@@ -18,11 +18,11 @@ description: >
 | FLUX.2 Klein 4B | `I:/AI-Models/flux2-klein-4b/` |
 | Z-Image Turbo | `I:/AI-Models/z-image-turbo/` |
 | Z-Image Adapter | `I:/AI-Models/z-image-turbo-adapter/zimage_turbo_training_adapter_v2.safetensors` |
-| LLM / Vision | Ollama Gemma4 (runs locally at `http://localhost:11434`) |
+| Vision / LLM | Agent's built-in vision (`Read` tool) for images. For video analysis load the `video-production` skill. |
 
 ## How to Run
 
-Always use the ai-toolkit venv (has Flux2KleinPipeline + ZImagePipeline + ollama package):
+Always use the ai-toolkit venv (has Flux2KleinPipeline + ZImagePipeline):
 
 ```bash
 "G:/Projects/Kolbo.AI/github/ai-toolkit/venv/Scripts/python.exe" \
@@ -43,8 +43,6 @@ Always use the ai-toolkit venv (has Flux2KleinPipeline + ZImagePipeline + ollama
 | `--steps N` | 20 | Inference steps |
 | `--cfg N` | 3.5 | Guidance scale |
 | `--seed N` | random | Deterministic seed |
-| `--analyze` | off | Analyze `--image` with Gemma4, use as base description |
-| `--enhance` | off | Enhance `--prompt` with Gemma4 before generating |
 | `--adapter` | off | Load Z-Image Turbo adapter (zimage only) |
 
 ## Common Recipes
@@ -64,18 +62,29 @@ python photo-studio.py \
   --model flux --width 1152 --height 2048
 ```
 
-### Analyze image and generate variation
-```bash
+### Analyze image then generate variation
+```
+# Step 1: Read the image — the agent sees it natively (built-in vision)
+Read("/abs/path/to/char.jpg")
+→ Agent describes the person: clothing, pose, features, style
+
+# Step 2: Use the description as the prompt
 python photo-studio.py \
-  --image char.jpg --analyze \
-  --prompt "standing upright, full body" \
+  --prompt "<description from agent vision> standing upright, full body" \
   --model flux
 ```
 
-### Auto-enhance a short prompt then generate
-```bash
+### Enhance a short prompt then generate
+```
+# Step 1: Enhance the prompt with Kolbo MCP
+chat_send_message({
+  message: "Expand this into a detailed image generation prompt for a photorealistic portrait: 'street fashion guy'",
+})
+→ { content: "A young man in his mid-20s wearing..." }
+
+# Step 2: Generate with the enhanced prompt
 python photo-studio.py \
-  --prompt "street fashion guy" --enhance \
+  --prompt "<enhanced prompt from Kolbo>" \
   --model zimage --width 1152 --height 2048
 ```
 
@@ -101,11 +110,10 @@ python photo-studio.py \
 - Default: `--steps 8 --cfg 0.0 --width 1152 --height 2048` (~30s per image)
 - Add `--adapter` to load the v2 training adapter
 
-### Ollama Gemma4 (vision + LLM)
-- Used for `--analyze` (describe input image) and `--enhance` (expand prompt)
-- Runs locally, no API key, no rate limits
-- Model: `gemma4` (9.6GB, multimodal)
-- Ollama auto-starts on Windows boot
+### Vision & Prompt Enhancement
+- For image analysis: use the agent's built-in vision — `Read` the image file directly, no MCP needed
+- For prompt enhancement: `chat_send_message` asking Kolbo to expand a short prompt (text-only, no vision)
+- Do NOT use `--analyze` or `--enhance` flags (those call a local model that is no longer used)
 
 ## When to use which model
 
@@ -117,6 +125,6 @@ python photo-studio.py \
 | Quick text-to-image | `flux` or `zimage` |
 | Portrait + face reference | `flux --image face.jpg` |
 
-## Prompt Tips (Gemma4 is the default prompter)
+## Prompt Tips
 
-When the user gives a short/vague prompt, always use `--enhance` to let Gemma4 expand it. For image editing, pair `--analyze --enhance` to get the best context from the source image.
+When the user gives a short/vague prompt, use `chat_send_message` to let Kolbo AI expand it before passing to the script. For image editing, first analyze the source image with the agent's built-in vision (`Read` the image), then use the description as the base prompt.

package/skills/video-production/SKILL.md

@@ -1,13 +1,16 @@
 ---
 name: video-production
 description: >
-  Full-stack video production assistant. Analyzes talking head footage,
+  Full-stack video production assistant. Analyzes video content visually (Gemini),
   generates transcriptions/SRT subtitles, plans and creates motion graphics (Remotion),
   generates B-roll images/videos, produces timeline XMLs for Premiere/DaVinci.
-  Use for: video analysis, transcription, subtitles, motion graphics, B-roll, shorts,
-  timeline XML, clip cutting, silence removal, After Effects, Premiere Pro, DaVinci Resolve.
+  Downloads YouTube videos with yt-dlp.
+  Use for: video analysis, visual analysis, describe video, what's in this video,
+  transcription, subtitles, motion graphics, B-roll, shorts, timeline XML, clip cutting,
+  silence removal, After Effects, Premiere Pro, DaVinci Resolve, YouTube download.
   Keywords: video edit, ffmpeg, remotion, after effects, premiere, davinci, shorts, subtitles,
-  motion graphics, clip, render, transcribe, xml, timeline, b-roll, talking head, analyze
+  motion graphics, clip, render, transcribe, xml, timeline, b-roll, talking head, analyze,
+  yt-dlp, youtube, download, gemini, vision
 allowed-tools:
   - Read
   - Write
@@ -24,29 +27,122 @@ allowed-tools:
 
 # Video Production — Strategy Map
 
+## ⚠️ DEFAULT RULE: Video Analysis = Visual Analysis (NOT Transcription)
+
+**The agent has built-in vision for images. For videos, always use Gemini via Kolbo MCP.**
+
+| Media type | Action |
+|------------|--------|
+| **Image** (jpg, png, etc.) | Agent reads it directly — no upload needed |
+| **Video** — "analyze", "describe", "what's in this?", "what prompts?", file path with no instruction | `upload_media` → `chat_send_message` + Gemini |
+| **Transcription** — "transcribe", "subtitles", "SRT", "what's being said", "captions" | `transcribe_audio` only |
+| Both visual + transcript | Run both |
+
+**Never use ffmpeg to extract frames for analysis. Never use local Ollama/vision models. Commit to the right action — do not ask the user. Wait for `chat_send_message` to return before proceeding — it polls until done (up to 2 min). Do NOT fall back to ffmpeg or any other approach if it takes time.**
+
+---
+
+## Kolbo MCP Tools (Active When `kolbo auth login` Is Done)
+
+These are available as MCP tools — use them directly without any Python/API key setup:
+
+| Tool | Use |
+|------|-----|
+| `upload_media` | Upload local file to Kolbo CDN → get stable public URL |
+| `chat_send_message` | Send message + `media_urls` array to Gemini for visual analysis |
+| `transcribe_audio` | Transcribe audio/video to text + SRT (ElevenLabs Scribe) |
+| `generate_image` | Generate B-roll images |
+| `generate_video` | Generate B-roll videos |
+| `generate_video_from_image` | Animate a still into video |
+| `generate_music` | Generate background music |
+| `generate_speech` | TTS for voiceover |
+| `generate_sound` | Sound effects |
+| `list_models` | Browse available models by type |
+| `check_credits` | Check remaining Kolbo credit balance |
+
+### Visual Analysis Workflow — MANDATORY for all video analysis
+
+**Step 1 is NOT optional. You cannot skip `upload_media` or construct the URL yourself.**
+
+```
+Step 1: upload_media({ source: "/absolute/path/to/video.mp4" })
+  → Returns: { url, thumbnail_url, ... }
+  → Save the "url" field — this is the CDN URL you will pass to Gemini
+  → NEVER use thumbnail_url (it's a JPG preview, not the video)
+
+Step 2: chat_send_message({
+  message: "Describe this video in detail. What is shown?",
+  media_urls: ["<url from step 1>"]   ← must be an array, must be the "url" field
+})
+→ returns: { content: "..." }
+```
+
+**❌ Common mistakes that break video analysis:**
+- Skipping `upload_media` and passing a local file path to `chat_send_message` — local paths don't work
+- Using the transcription `.txt` URL as the `media_urls` value — Gemini needs the actual video CDN URL
+- Using `thumbnail_url` instead of `url` from the `upload_media` response
+- Calling `transcribe_audio` first then passing its output URL as the video — transcription gives text, not video
+
+**Omit `model`** — Smart Select detects video/audio and auto-routes to Gemini.
+**Sessions do NOT remember media between messages.** On retry: reuse the same CDN `url` from step 1 (no re-upload needed) but always pass `media_urls` again.
+
+**Batch analysis (many videos)**: Pass `model: "gemini-3.1-flash-lite-preview"` explicitly for cheaper bulk runs.
+
+For YouTube videos — download first with yt-dlp (see below), then follow steps 1–2 above.
+
+---
+
 ## Pipeline
 
 ```
-Input video → Transcribe → Analyze → Plan segments
-  → Generate: Remotion compositions | B-roll | SRT subtitles
-  → Output: Premiere XML / DaVinci EDL / individual MP4s / SRT
+Input: local video / YouTube URL / uploaded file
+
+→ [DEFAULT] Visual Analysis: upload_media → chat_send_message (Gemini)
+→ [EXPLICIT REQUEST] Transcription: transcribe_audio → SRT / text
+→ [EDITING] FFmpeg: cut, silence removal, 9:16 conversion
+→ [MOTION GRAPHICS] Remotion: compositions, captions, B-roll
+→ Output: Premiere XML / DaVinci EDL / MP4s / SRT
 ```
 
 ## APIs & Capabilities
 
 | Service | Use |
 |---------|-----|
-| ElevenLabs Scribe | Primary transcription — word-level SRT, multilingual |
-| Claude | Content analysis, edit planning |
-| Google Gemini | Video understanding, visual analysis |
-| fal.ai (MCP) | Image & video B-roll generation |
-| Runway | Image-to-video, video-to-video |
-| FLUX / BFL | High quality still image generation |
-| ElevenLabs | TTS, voice cloning, SFX |
-| Suno | Background music generation |
+| Kolbo MCP (`upload_media` + `chat_send_message`) | **Primary** — visual video/image analysis via Gemini |
+| Kolbo MCP (`transcribe_audio`) | **Primary** — transcription, word-level SRT, multilingual |
+| yt-dlp | Download YouTube/social media videos |
+| FFmpeg | Local video editing, cutting, silence removal, format conversion |
 | Remotion Lambda | Cloud render motion graphics |
+| fal.ai (MCP) | Image & video B-roll generation |
+| ElevenLabs | TTS, voice cloning, SFX (via Kolbo MCP `generate_speech`) |
+| Suno | Background music (via Kolbo MCP `generate_music`) |
+
+> Kolbo MCP tools need no API keys — auth is handled by `kolbo auth login`.
+> FFmpeg/yt-dlp need to be installed locally on the machine.
 
-> Load API keys from the project's `.env` file or environment variables.
+## YouTube / Social Media Download (yt-dlp)
+
+Download video from YouTube, TikTok, Instagram, Twitter, etc.:
+
+```bash
+# Best quality MP4
+yt-dlp -f "bestvideo[height<=1080][ext=mp4]+bestaudio/best" \
+  --merge-output-format mp4 \
+  -o "%(id)s.%(ext)s" <url>
+
+# With subtitles
+yt-dlp -f "bestvideo[height<=1080][ext=mp4]+bestaudio/best" \
+  --write-auto-sub --sub-lang en --convert-subs srt \
+  --merge-output-format mp4 \
+  -o "%(id)s.%(ext)s" <url>
+
+# Audio only (for transcription)
+yt-dlp -f "bestaudio" --extract-audio --audio-format mp3 -o "%(id)s.%(ext)s" <url>
+```
+
+After download → upload to Kolbo CDN with `upload_media` → analyze visually with `chat_send_message`.
+
+---
 
 ## Key Rules
 

package/skills/ollama-vision/SKILL.md

@@ -1,105 +0,0 @@
----
-name: ollama-vision
-description: >
-  Batch image analysis using local Ollama + gemma4 (multimodal).
-  Use when the user needs to analyze, caption, classify, or extract text from images locally — 
-  free, offline, no rate limits, no API key needed.
-  Keywords: image analysis, batch images, captions, OCR, vision, gemma4, ollama, local AI
----
-
-# Ollama Vision — Batch Image Analysis with gemma4
-
-## Setup (already done on this machine)
-
-- Ollama installed and running (auto-starts on Windows boot)
-- Model: `gemma4` (9.6 GB, multimodal)
-- Python package: `ollama` v0.6.1 installed (pip, Python 3.10)
-- REST API available at `http://localhost:11434`
-
-## Core Pattern
-
-```python
-import ollama
-
-response = ollama.chat(model='gemma4', messages=[{
-    'role': 'user',
-    'content': 'Your prompt here',
-    'images': ['path/to/image.jpg']  # omit for text-only
-}])
-print(response['message']['content'])
-```
-
-## Batch Image Captioning Script
-
-```python
-import ollama
-from pathlib import Path
-import csv
-
-def caption_images(folder: str, prompt: str = "Write a short caption for this image.", output_csv: str = "captions.csv"):
-    images_dir = Path(folder)
-    extensions = {'.jpg', '.jpeg', '.png', '.webp', '.gif', '.bmp'}
-    image_files = [f for f in images_dir.iterdir() if f.suffix.lower() in extensions]
-    
-    results = []
-    for i, img_path in enumerate(image_files, 1):
-        print(f"[{i}/{len(image_files)}] Processing {img_path.name}...")
-        try:
-            response = ollama.chat(model='gemma4', messages=[{
-                'role': 'user',
-                'content': prompt,
-                'images': [str(img_path)]
-            }])
-            caption = response['message']['content'].strip()
-            results.append({'file': img_path.name, 'caption': caption})
-            print(f"  → {caption[:80]}...")
-        except Exception as e:
-            print(f"  ERROR: {e}")
-            results.append({'file': img_path.name, 'caption': f'ERROR: {e}'})
-    
-    with open(output_csv, 'w', newline='', encoding='utf-8') as f:
-        writer = csv.DictWriter(f, fieldnames=['file', 'caption'])
-        writer.writeheader()
-        writer.writerows(results)
-    
-    print(f"\nDone! Saved {len(results)} captions to {output_csv}")
-
-# Usage
-caption_images("./images", prompt="Describe this image in one sentence.")
-```
-
-## Common Prompts
-
-| Task | Prompt |
-|------|--------|
-| Caption | `"Write a short, descriptive caption for this image."` |
-| Alt text | `"Write alt text for this image for accessibility."` |
-| Classification | `"What category does this image belong to? Reply with one word."` |
-| OCR | `"Extract all text visible in this image."` |
-| Product description | `"Write a product description for the item shown in this image."` |
-| Social media | `"Write a catchy Instagram caption for this image."` |
-
-## REST API Alternative (no Python package needed)
-
-```python
-import requests, base64
-
-def analyze_image(image_path: str, prompt: str) -> str:
-    with open(image_path, "rb") as f:
-        img_b64 = base64.b64encode(f.read()).decode()
-    
-    response = requests.post("http://localhost:11434/api/generate", json={
-        "model": "gemma4",
-        "prompt": prompt,
-        "images": [img_b64],
-        "stream": False
-    })
-    return response.json()["response"]
-```
-
-## Tips
-
-- gemma4 handles JPG, PNG, WEBP, GIF, BMP
-- For large batches, add `time.sleep(0.5)` between requests to avoid overloading
-- Results are best when prompts are specific ("describe the main subject" vs "describe this")
-- Ollama must be running — check with `ollama list` in terminal