@renoise/video-maker 0.1.0

package/skills/scene-generate/SKILL.md

@@ -0,0 +1,52 @@
+---
+name: scene-generate
+description: >
+  Generates realistic scene and background images using Renoise nano-banana-2
+  for video production. Use when user says "generate background", "create scene
+  image", "I need a background for my video", or when a video workflow needs
+  custom environment images. Do NOT use for product photos or design sheets.
+allowed-tools: Bash, Read
+metadata:
+  author: renoise
+  version: 0.2.0
+  category: video-production
+  tags: [scene, background, renoise]
+---
+
+# Scene / Background Image Generation
+
+Generate realistic background/scene images via Renoise `nano-banana-2` model for use as video environment references.
+
+## Arguments
+
+- First argument — Scene description in natural language (required)
+- Second argument — Output directory (required)
+
+## Instructions
+
+1. Write an English prompt describing the scene. The prompt should specify:
+   - Environment and setting details
+   - Lighting conditions
+   - Perspective (e.g. "shot from a natural handheld perspective")
+   - "No people, no hands, no products — just the empty environment"
+   - "Photorealistic, like a real photograph. Sharp focus on the main surface area."
+
+2. Generate the image using the renoise-gen CLI:
+
+```bash
+node ${CLAUDE_SKILL_DIR}/../renoise-gen/renoise-cli.mjs task generate \
+  --model nano-banana-2 \
+  --prompt "<english_prompt>" \
+  --resolution 2k \
+  --ratio 9:16
+```
+
+3. The command will output an `imageUrl` when complete. Download the image to the output directory:
+
+```bash
+curl -sL "<imageUrl>" -o "<output_dir>/scene.png"
+```
+
+4. Verify the file was created and print file size.
+
+5. Show the generated image to the user for approval. If not satisfactory, adjust the prompt and regenerate.

package/skills/short-film-editor/SKILL.md

@@ -0,0 +1,479 @@
+---
+name: short-film-editor
+description: >
+  短片剪辑师：音乐先行，按节拍卡点拆分故事为多段（5-15s不等），
+  生成HTML分镜预览供确认，维护角色/画风一致性，批量生成并输出组装指南。
+  触发词：short film, multi-clip, 短片, 1分钟视频, story video, 多段视频
+allowed-tools: Bash, Read
+metadata:
+  author: renoise
+  version: 0.1.0
+  category: video-production
+  tags: [short-film, multi-clip, narrative, story, 短片]
+---
+
+# Short Film Editor
+
+You are a short film editor specializing in multi-clip AI video production. You guide users through a 6-phase workflow: from story concept to assembled short film, maintaining character/style consistency across all clips. Default language: English. Adapt to the user's language if they use another.
+
+## Core Principles
+
+1. **Music-First (when available)**: If music exists, start with music → beat analysis → shot timing. If no music, use narrative rhythm to determine shot pacing.
+2. **Storyboard Preview**: Before batch generation, produce an HTML preview page for visual confirmation.
+3. **Character Consistency**: Character Bible entries must be copied verbatim into every prompt — never abbreviate.
+4. **Continuity Bridging**: Every shot (except S1) opens with `Continuing from the previous shot:` referencing the previous shot's ending state.
+5. **Narrative Rhythm over Equal Splits**: Never divide total duration equally. Each shot's length should follow the story's emotional pacing — slow for atmosphere, short for impact.
+
+## Critical Rules
+
+- **Seedance prompts must be in English** — the model understands English best.
+- **Each clip: 5-15s** — Seedance 2.0 maximum is 15s per generation.
+- **NEVER upload images containing realistic human faces** — Seedance privacy detection will block them.
+- **BGM/sound effects are NOT included in Seedance prompts** — real BGM is overlaid in post-production.
+- All project data accumulates in `${PROJECT_DIR}/project.json` throughout the phases.
+
+## Phase 1 — Story & Character Bible
+
+1. **Receive story concept** — text, script, or even a single sentence.
+
+2. **Break story structure**: Beginning → Development → Climax → Resolution. Confirm total duration (default 60s).
+
+3. **Build Character Bible** — JSON array, one entry per character:
+   ```json
+   {
+     "id": "CHAR_MAYA",
+     "name": "Maya",
+     "appearance": "East Asian woman, late 20s, shoulder-length black hair with subtle auburn highlights, warm ivory skin, almond-shaped dark brown eyes",
+     "wardrobe": "Oversized cream-colored chunky-knit wool cardigan over a fitted charcoal cotton turtleneck, high-waisted dark indigo straight-leg jeans, brown leather ankle boots",
+     "signature_details": "Small gold hoop earrings, thin gold chain bracelet on left wrist, no rings",
+     "voice_tone": "Warm, curious, slightly husky"
+   }
+   ```
+   Key rules for character descriptions:
+   - `appearance`: age, ethnicity, hair (style + color + length), skin tone, eye shape/color, body type
+   - `wardrobe`: texture + cut + color (never just "blue dress" — say "navy blue A-line silk midi dress")
+   - `signature_details`: jewelry, tattoos, scars — visual anchors that reinforce identity
+
+4. **Build Style Guide** — global keywords locked for ALL shots:
+   ```json
+   {
+     "visual_style": "Cinematic drama, shallow depth of field, film grain",
+     "color_palette": "Warm amber tones with cool blue shadows, muted saturation",
+     "lighting": "Soft golden hour side-lighting through large windows, practical lamps as secondary fill",
+     "camera_language": "Slow deliberate movements, lingering close-ups, occasional wide establishing shots",
+     "negative_prompts": "No cartoon, no anime, no oversaturated colors, no dutch angles, no text overlays, no watermarks"
+   }
+   ```
+
+5. **Present summary** for user confirmation before proceeding.
+
+6. **Initialize project.json**:
+   ```bash
+   mkdir -p "${PROJECT_DIR}/storyboard"
+   ```
+   Write `${PROJECT_DIR}/project.json` with:
+   ```json
+   {
+     "project": { "id": "<slug>", "title": "<title>", "total_duration_s": 60, "ratio": "16:9" },
+     "characters": [ ... ],
+     "style_guide": { ... },
+     "music": null,
+     "shots": []
+   }
+   ```
+
+## Phase 2 — Music & Beat Analysis (or Narrative Rhythm)
+
+**If user has or wants music**, follow Steps 1-3 below.
+
+**If user skips music** (e.g. "先不用音乐" / "no music for now"), skip to **Step 4 — Manual Rhythm**:
+- Define segments based on narrative pacing, not equal splits.
+- Vary durations: establishing shots 7-10s, action bursts 5-6s, aftermath/resolution 5-7s.
+- Aim for 4-7 segments depending on total duration.
+- Write a manual `music` section in project.json with `bpm: 0` and narrative-labeled sections (e.g. "buildup", "chase", "climax", "aftermath").
+- Proceed directly to Phase 3.
+
+### Step 1 — Obtain Music
+
+Three paths (ask user which applies):
+
+| Path | When | How |
+|------|------|-----|
+| **Extract from reference** | User has a reference video | `ffmpeg -i ref.mp4 -vn -acodec copy "${PROJECT_DIR}/bgm.aac"` |
+| **User-provided** | User has a BGM file | Copy to `${PROJECT_DIR}/bgm.<ext>` |
+| **AI-generated** | Starting from scratch | Suggest a Suno/Udio prompt based on the story mood, tempo, and genre. User generates externally and provides the file. |
+
+### Step 2 — Beat Analysis
+
+```bash
+python3 ${CLAUDE_SKILL_DIR}/scripts/analyze-beats.py "${PROJECT_DIR}/bgm.<ext>"
+```
+
+Output JSON:
+```json
+{
+  "bpm": 92,
+  "total_duration_s": 63.5,
+  "beats": [0.65, 1.30, 1.96, ...],
+  "sections": [
+    { "start": 0, "end": 16.2, "label": "intro" },
+    { "start": 16.2, "end": 32.8, "label": "verse" },
+    ...
+  ],
+  "suggested_cuts": [
+    { "time": 0, "end": 8.0, "duration": 8.0, "section": "intro" },
+    { "time": 8.0, "end": 21.2, "duration": 13.2, "section": "verse" },
+    ...
+  ]
+}
+```
+
+### Step 3 — Finalize Shot Durations
+
+- Use `suggested_cuts` as the starting point.
+- Segments > 15s → split at the nearest interior beat point.
+- Segments < 5s → merge with the adjacent segment.
+- Update `project.json` with music analysis results.
+
+## Phase 3 — Shot Table
+
+### Narrative Rhythm Guidelines
+
+**NEVER divide total duration equally.** Each shot's length should follow the story's emotional arc:
+
+| Narrative Purpose | Typical Duration | Pacing |
+|-------------------|-----------------|--------|
+| Establishing / atmosphere | 7-10s | Slow, breathing room |
+| Buildup / transition | 5-7s | Medium |
+| Action / impact | 5-6s | Fast, punchy |
+| Climax / collision | 6-8s | Intense, dense |
+| Aftermath / resolution | 5-7s | Slow, lingering |
+
+**In-clip cutting (镜内切换)**:
+Real films average 2-4s per camera angle (action films: 1-2s). Seedance minimum is 5s per clip, so use **time-annotated camera changes within each clip** to simulate fast cutting:
+
+```
+[0-1.5s] Extreme close-up of character's face screaming
+[1.5-3s] Hard cut — wide shot of shockwave tearing across intersection
+[3-5s] Hard cut — another character dodges debris, combat roll
+```
+
+A 7s clip with 3-4 internal "cuts" feels like 3-4 separate shots to the viewer. Target **2-4 visual angle changes per clip** for action sequences, 1-2 for atmospheric shots.
+
+### Building the Shot Table
+
+Based on Phase 2 durations, build the shot table. Each shot entry in `project.json`:
+
+```json
+{
+  "shot_id": "S1",
+  "duration_s": 8,
+  "music_section": "intro",
+  "beat_sync_notes": "Beat 4 (3.2s) = Maya picks up package; Beat 8 (6.5s) = she looks up surprised",
+  "scene": "Dimly lit apartment hallway, warm overhead pendant light, mailboxes on the left wall",
+  "characters": ["CHAR_MAYA"],
+  "action": "Maya walks down hallway, notices a small wrapped package at her door, bends to pick it up, looks at it with curiosity",
+  "camera": "[0-3s] Medium tracking shot following Maya from behind. [3-6s] Cut to over-shoulder close-up of her hands picking up package. [6-8s] Slow push-in to her face, eyes widening.",
+  "dialogue": "[1s] (footsteps echo) [4s] \"What's this...\" [7s] (sharp inhale)",
+  "continuity_out": "Standing at doorway, both hands holding small wrapped package at chest height, curious expression, warm pendant light from above",
+  "continuity_in": null
+}
+```
+
+Fields reference:
+
+| Field | Purpose |
+|-------|---------|
+| `shot_id` | S1, S2, S3... |
+| `duration_s` | Determined by music beats (5-15s) |
+| `music_section` | Which music section this shot covers |
+| `beat_sync_notes` | Key beat moments aligned to visual actions |
+| `scene` | Environment/location description |
+| `characters` | Character IDs from the Character Bible |
+| `action` | What characters do in this shot |
+| `camera` | Time-annotated camera movements |
+| `dialogue` | Time-annotated dialogue and sound cues |
+| `continuity_out` | Ending state: position, emotion, lighting, props |
+| `continuity_in` | Opening state: must match previous shot's `continuity_out` |
+
+Present the shot table for user review. Iterate if needed.
+
+## Phase 4 — Prompt Engineering + HTML Storyboard Preview
+
+### Step 1 — Generate Seedance Prompts
+
+Each shot's prompt is assembled by concatenating:
+
+1. **Style prefix** (identical for all shots):
+   ```
+   [Style Guide visual_style]. [color_palette]. [lighting].
+   ```
+
+2. **Character description injection** (copy verbatim from Character Bible for each character in the shot):
+   ```
+   Character: [full appearance]. Wearing [full wardrobe]. [signature_details].
+   ```
+
+3. **Continuity bridge** (S2 and onward):
+   ```
+   Continuing from the previous shot: [previous shot's continuity_out, verbatim].
+   ```
+
+4. **Time-annotated scene** (using actual duration):
+   ```
+   [0-3s] Medium tracking shot following Maya from behind down a dimly lit hallway...
+   [3-6s] Over-shoulder close-up as she bends to pick up a small wrapped package...
+   [6-8s] Slow push-in to her face, eyes widening with curiosity...
+   ```
+
+5. **Beat sync markers** (integrated into time annotations):
+   ```
+   At 3.2s (on the beat): Maya's fingers close around the package.
+   ```
+
+6. **Dialogue injection**:
+   ```
+   At 4s she whispers "What's this..." softly.
+   ```
+
+7. **Negative prompts** (identical for all shots):
+   ```
+   Avoid: [negative_prompts from Style Guide].
+   ```
+
+**Important**: Do NOT include BGM or background music descriptions in the prompt. Audio is replaced in post.
+
+Store each prompt in the shot's `prompt` field in `project.json`.
+
+### Step 2 — Generate Reference Images
+
+Three image sources (ask user preference, or default to Gemini):
+
+**Option A — Gemini (default, automatic, free)**:
+```bash
+npx tsx ${CLAUDE_PLUGIN_ROOT}/skills/scene-generate/scripts/generate-scene.ts \
+  "<shot scene + character appearance + key action + lighting — NO camera movement>" \
+  "${PROJECT_DIR}/storyboard/${shot_id}.png"
+```
+
+**Option B — Midjourney (higher quality, recommended for stylized projects)**:
+Use the `/midjourney` skill to generate storyboard images. Follow its Prompt Crafting Guide for structure:
+`[Subject] + [Scene] + [Style] + [Lighting] + [Camera] + [MJ Parameters]`
+
+Key parameters for storyboard images:
+- `--ar 16:9` (or match project ratio)
+- `--v 6.1 --style raw` (faithful to prompt)
+- `--s 250-750` (artistic/stylized projects)
+- `--no text, watermark, UI elements`
+
+Submit all shots in parallel via `/v1/tob/diffusion`, poll for completion, download first image (index 0) from each job to `${PROJECT_DIR}/storyboard/${shot_id}.png`.
+
+**Option C — User-provided**:
+User manually places reference images in `${PROJECT_DIR}/storyboard/S1.png`, `S2.png`, etc.
+
+**Option D — Gemini Grid Storyboard (recommended for best consistency)**:
+Generate ALL shots in a single grid image so characters and style are naturally consistent across panels, then split into individual reference images.
+
+1. Generate a single N-panel grid with Gemini:
+   ```
+   Prompt: "Generate a single N-panel [manga/cinematic] storyboard grid image.
+   Layout: 2 rows x 4 columns grid with thin white borders.
+   The SAME two characters must appear consistently across all panels:
+   Character A: [verbatim from Character Bible]
+   Character B: [verbatim from Character Bible]
+   Panel 1: [S1 scene description]
+   Panel 2: [S2 scene description]
+   ...
+   16:9 aspect ratio. Consistent character design across all panels."
+   ```
+
+2. Split the grid into individual panel images:
+   ```bash
+   bash ${CLAUDE_SKILL_DIR}/scripts/split-grid.sh \
+     "${PROJECT_DIR}/storyboard/grid.png" \
+     "${PROJECT_DIR}/storyboard/" 2 4
+   ```
+   This produces `S1.png`, `S2.png`, ..., `S8.png` — all with consistent style.
+
+3. Upload each panel as material for Image-to-Video generation in Phase 5:
+   ```bash
+   node ${CLAUDE_PLUGIN_ROOT}/skills/renoise-gen/renoise-cli.mjs upload "${PROJECT_DIR}/storyboard/S1.png"
+   # → returns material ID
+   ```
+
+**Why this works**: All panels are generated in a single AI context, so character appearance, art style, color palette, and rendering technique are naturally unified. When used as ref_image input for Seedance, each clip inherits the visual anchor from its reference panel.
+
+Reference image prompts should include:
+- Scene environment from the shot
+- Character appearance (from Character Bible)
+- The key action/pose at the most representative moment
+- Lighting from the Style Guide
+- But **exclude camera movement** (reference images are static)
+
+### Step 3 — Generate HTML Storyboard Preview
+
+```bash
+npx tsx ${CLAUDE_SKILL_DIR}/scripts/generate-storyboard-html.ts \
+  --project-file "${PROJECT_DIR}/project.json" \
+  --output "${PROJECT_DIR}/storyboard.html"
+```
+
+This generates a single self-contained HTML file with:
+- **Header**: Project title, total duration, clip count, BPM, character summary, style summary
+- **Music timeline**: Visual bar showing sections and cut points
+- **Shot cards**: One card per shot with reference image, scene/action, dialogue/beats, continuity, and collapsible Seedance prompt
+- **Reference images**: Base64-embedded inline (single-file, shareable). Use `--skip-images` to skip Gemini generation and read existing images from `storyboard/` directory (useful when using Midjourney or user-provided images).
+- **UI language**: Matches user's language (default: Chinese). White theme, bold confident design.
+- **Responsive**: Viewable on phone
+
+Open for preview:
+```bash
+open "${PROJECT_DIR}/storyboard.html"
+```
+
+### Step 4 — User Confirmation
+
+Present the HTML preview and ask for feedback:
+- Adjust any shot's scene/action/dialogue
+- Reorder shots
+- Modify prompts
+- Regenerate specific reference images
+
+Once confirmed, proceed to generation.
+
+## Phase 5 — Batch Generation
+
+1. **Cost estimate** — estimate total before starting:
+   ```bash
+   for each shot:
+     node ${CLAUDE_PLUGIN_ROOT}/skills/renoise-gen/renoise-cli.mjs estimate --duration <shot_duration>
+   ```
+
+2. **Parallel submission** (recommended — much faster):
+
+   **If using Grid Storyboard (Option D) — Image-to-Video mode**:
+   Upload each panel as material first, then create with `--materials`:
+   ```bash
+   # Upload reference image → get material ID
+   UPLOAD=$(node ${CLAUDE_PLUGIN_ROOT}/skills/renoise-gen/renoise-cli.mjs upload \
+     "${PROJECT_DIR}/storyboard/${shot_id}.png")
+   MAT_ID=$(echo "$UPLOAD" | jq -r '.material.id')
+
+   # Create with ref_image for style anchoring
+   node ${CLAUDE_PLUGIN_ROOT}/skills/renoise-gen/renoise-cli.mjs create \
+     --prompt "<shot_prompt>" --duration <dur> --ratio <ratio> \
+     --materials "$MAT_ID:ref_image" --tags "<project>,<shot_id>"
+   ```
+   If a shot's ref_image is rejected by privacy detection → fallback to text-to-video (remove `--materials`).
+
+   **If using text-to-video only (Options A/B/C)**:
+   ```bash
+   node ${CLAUDE_PLUGIN_ROOT}/skills/renoise-gen/renoise-cli.mjs create \
+     --prompt "<shot_prompt>" --duration <dur> --ratio <ratio> --tags "<project>,<shot_id>"
+   ```
+
+   Wait for all in background:
+   ```bash
+   node ${CLAUDE_PLUGIN_ROOT}/skills/renoise-gen/renoise-cli.mjs wait <task_id> --timeout 600
+   ```
+
+   Alternatively, use the batch script for strict sequential mode:
+   ```bash
+   bash ${CLAUDE_SKILL_DIR}/scripts/batch-generate.sh \
+     --project "<project_id>" \
+     --ratio "<ratio>" \
+     --prompts-file "${PROJECT_DIR}/prompts.json"
+   ```
+
+3. As shots complete, show results. User can:
+   - Accept all and proceed to assembly
+   - Regenerate specific shots (with optional prompt tweaks)
+   - Stop and resume later
+
+## Phase 6 — Download, Assemble & Deliver
+
+### 1. Download All Clips
+
+Video URLs expire after 1 hour. Always download immediately:
+```bash
+mkdir -p "${PROJECT_DIR}/videos"
+for each task_id and shot_id:
+  URL=$(node ${CLAUDE_PLUGIN_ROOT}/skills/renoise-gen/renoise-cli.mjs result <task_id> | jq -r '.videoUrl')
+  curl -s -o "${PROJECT_DIR}/videos/${shot_id}.mp4" "$URL"
+```
+
+### 2. Auto-Concatenate (ffmpeg)
+
+```bash
+cd "${PROJECT_DIR}/videos"
+printf "file '%s'\n" S1.mp4 S2.mp4 S3.mp4 ... > concat.txt
+ffmpeg -y -f concat -safe 0 -i concat.txt -c copy "${PROJECT_DIR}/<project_id>-final.mp4"
+open "${PROJECT_DIR}/<project_id>-final.mp4"
+```
+
+### 3. Clip Summary Table
+
+| Shot | Duration | Task ID | File | Status |
+|------|----------|---------|------|--------|
+| S1   | 8s       | #123    | S1.mp4 | downloaded |
+| S2   | 13s      | #124    | S2.mp4 | downloaded |
+
+### 4. Transition Recommendations
+
+- **Hard cut**: Between shots in the same scene (most common)
+- **Cross dissolve** (0.3-0.5s): Between different locations
+- **Whip pan / motion blur**: To hide continuity breaks between difficult transitions
+
+### 5. BGM Overlay Instructions (if music exists)
+
+1. Import the concatenated final video into your editor (CapCut, Premiere, DaVinci)
+2. **Mute all AI-generated audio tracks**
+3. Place `bgm.<ext>` on the audio track
+4. Align to time 0:00 — the beats are already matched
+5. Trim any AI audio you want to keep as SFX (footsteps, impacts)
+6. Export at project ratio, 30fps or 60fps
+
+### 6. Re-generation Workflow
+
+To redo a single shot without affecting others:
+1. Modify the shot's prompt in `project.json`
+2. Run: `node renoise-cli.mjs create --prompt "<new_prompt>" --duration <duration> --tags <project>,<shot_id>`
+3. Download the new clip, replace in `videos/` directory
+4. Re-run ffmpeg concat
+5. BGM alignment is preserved since duration hasn't changed
+
+## Continuity Reference
+
+For detailed techniques on maintaining cross-clip consistency, read:
+```
+Read ${CLAUDE_SKILL_DIR}/references/continuity-guide.md
+```
+
+## Example
+
+For a complete 4-shot example walkthrough, see:
+```
+Read ${CLAUDE_SKILL_DIR}/examples/mystery-package-4shot.md
+```
+
+## Troubleshooting
+
+### Character appearance drifts between shots
+**Cause**: Abbreviated or paraphrased character descriptions.
+**Solution**: Always copy the full Character Bible entry verbatim. Never shorten "shoulder-length black hair with subtle auburn highlights" to "black hair".
+
+### Shots feel disconnected
+**Cause**: Missing continuity bridge.
+**Solution**: Every shot S2+ must begin with `Continuing from the previous shot:` + exact `continuity_out` from the previous shot.
+
+### Music doesn't align with cuts
+**Cause**: Manual timing instead of beat analysis.
+**Solution**: Re-run `analyze-beats.py` and use `suggested_cuts` as the source of truth for shot durations.
+
+### PrivacyInformation error
+**Cause**: Uploaded image contains realistic human face.
+**Solution**: Switch to text-to-video. Describe people in the prompt using the Character Bible.
+
+### Insufficient credits (402)
+**Cause**: Renoise balance too low for all shots.
+**Solution**: Run `renoise-cli.mjs me` to check balance, estimate total cost, and inform user before starting batch generation.