@farazirfan/costar-server-executor 1.7.37 → 1.7.39

package/skills/longform-video-generation/references/google_api_guide.md

@@ -0,0 +1,288 @@
+# Google Gemini API - Veo 3.1 Video Generation Guide
+
+## Official Documentation (November 2025)
+
+This guide is based on the official Google Gemini API documentation for Veo 3.1 video generation.
+
+## Key Models
+
+### Video Generation
+- **veo-3.1-generate-preview** - Latest Veo model with audio, 8-second clips
+- **veo-3.1-fast-generate-preview** - Faster generation, same quality
+
+### Image Generation (for Reference Images)
+- **gemini-2.5-flash-image** (Nano Banana) - Fast, 1K resolution
+- **gemini-3-pro-image-preview** (Nano Banana Pro) - Professional, up to 4K resolution
+
+## Image Generation for Reference Frames
+
+### Nano Banana Pro (Recommended for Production)
+
+```python
+from google import genai
+from google.genai import types
+
+client = genai.Client(api_key="your-api-key")
+
+# Generate high-quality reference image
+response = client.models.generate_content(
+    model="gemini-3-pro-image-preview",
+    contents="Cinematic still frame of tactical agent in dark alley, dramatic lighting",
+    config=types.GenerateContentConfig(
+        response_modalities=['IMAGE'],
+        image_config=types.ImageConfig(
+            aspect_ratio="16:9",
+            image_size="2K"  # or "4K" for maximum quality
+        )
+    )
+)
+
+# Save the image
+for part in response.parts:
+    if image := part.as_image():
+        image.save("reference_frame.png")
+```
+
+### Resolution Options
+
+| Resolution | Dimensions (16:9) | Tokens | Cost | Use Case |
+|------------|------------------|--------|------|----------|
+| 1K | 1376×768 | ~800 | $0.024 | Testing |
+| 2K | 2752×1536 | ~1200 | $0.036 | **Production** |
+| 4K | 5504×3072 | ~2000 | $0.06 | Maximum quality |
+
+## Smooth Video Transitions - SOLUTION TO JERKY CUTS
+
+### Problem: Jerky Transitions Between Scenes
+When concatenating videos with `-c copy`, there can be visible jumps or jerks at scene boundaries.
+
+### Solution 1: Crossfade Transitions (Recommended)
+
+Add a 0.5-second crossfade between scenes for smooth transitions:
+
+```python
+def concatenate_with_crossfade(video_paths: List[str], output_path: str, fade_duration: float = 0.5):
+    """Concatenate videos with crossfade transitions for smooth playback"""
+    
+    if len(video_paths) == 1:
+        subprocess.run(['cp', video_paths[0], output_path])
+        return output_path
+    
+    # Build filter complex for crossfades
+    filter_parts = []
+    
+    for i, video_path in enumerate(video_paths):
+        if i == 0:
+            # First video: fade out at end
+            filter_parts.append(f"[{i}:v]fade=t=out:st=7.5:d={fade_duration}:alpha=1[v{i}]")
+        elif i == len(video_paths) - 1:
+            # Last video: fade in at start only
+            filter_parts.append(f"[{i}:v]fade=t=in:st=0:d={fade_duration}:alpha=1[v{i}]")
+        else:
+            # Middle videos: fade in and out
+            filter_parts.append(
+                f"[{i}:v]fade=t=in:st=0:d={fade_duration}:alpha=1,"
+                f"fade=t=out:st=7.5:d={fade_duration}:alpha=1[v{i}]"
+            )
+    
+    # Overlay videos with crossfades
+    overlay_chain = "[v0]"
+    for i in range(1, len(video_paths)):
+        overlay_chain += f"[v{i}]overlay"
+        if i < len(video_paths) - 1:
+            overlay_chain += f"[tmp{i}];[tmp{i}]"
+    overlay_chain += "[vout]"
+    
+    filter_parts.append(overlay_chain)
+    
+    # Audio concatenation
+    audio_inputs = "".join(f"[{i}:a]" for i in range(len(video_paths)))
+    filter_parts.append(f"{audio_inputs}concat=n={len(video_paths)}:v=0:a=1[aout]")
+    
+    filter_complex = ";".join(filter_parts)
+    
+    # Build ffmpeg command
+    cmd = ['ffmpeg']
+    for video_path in video_paths:
+        cmd.extend(['-i', video_path])
+    
+    cmd.extend([
+        '-filter_complex', filter_complex,
+        '-map', '[vout]',
+        '-map', '[aout]',
+        '-c:v', 'libx264',
+        '-c:a', 'aac',
+        '-y',
+        output_path
+    ])
+    
+    subprocess.run(cmd, check=True, capture_output=True)
+    return output_path
+```
+
+### Solution 2: Re-encode with Matching Parameters
+
+Re-encode all clips to ensure consistent frame timing:
+
+```python
+def concatenate_smooth(video_paths: List[str], output_path: str):
+    """Re-encode for smooth concatenation"""
+    
+    # Create concat file
+    list_file = 'concat_list.txt'
+    with open(list_file, 'w') as f:
+        for video_path in video_paths:
+            f.write(f"file '{os.path.abspath(video_path)}'\n")
+    
+    # Re-encode with matching parameters
+    subprocess.run([
+        'ffmpeg',
+        '-f', 'concat',
+        '-safe', '0',
+        '-i', list_file,
+        '-c:v', 'libx264',  # Re-encode video
+        '-preset', 'fast',
+        '-crf', '18',  # High quality
+        '-c:a', 'aac',  # Re-encode audio
+        '-b:a', '192k',
+        '-y',
+        output_path
+    ], check=True, capture_output=True)
+    
+    os.remove(list_file)
+    return output_path
+```
+
+### Solution 3: Frame Blending at Boundaries
+
+Use xfade filter for professional transitions:
+
+```bash
+ffmpeg -i scene1.mp4 -i scene2.mp4 -i scene3.mp4 \
+  -filter_complex "\
+    [0:v][1:v]xfade=transition=fade:duration=0.5:offset=7.5[v01]; \
+    [v01][2:v]xfade=transition=fade:duration=0.5:offset=15.5[vout]; \
+    [0:a][1:a][2:a]concat=n=3:v=0:a=1[aout]" \
+  -map "[vout]" -map "[aout]" output.mp4
+```
+
+## Video Generation
+
+### Text-to-Video
+
+```python
+operation = client.models.generate_videos(
+    model="veo-3.1-generate-preview",
+    prompt="A cinematic shot of a tactical agent running through an alley",
+    config=types.GenerateVideosConfig(
+        duration_seconds=8,  # 4, 6, or 8
+        aspect_ratio="16:9",  # or "9:16"
+        resolution="720p"  # or "1080p" (only for 8s duration)
+    )
+)
+
+# Poll until complete
+while not operation.done:
+    time.sleep(10)
+    operation = client.operations.get(operation)
+
+# Download video
+video = operation.response.generated_videos[0]
+client.files.download(file=video.video)
+video.video.save("output.mp4")
+```
+
+### Image-to-Video (CORRECT METHOD)
+
+```python
+# Load reference image using types.Image.from_file()
+reference_img = types.Image.from_file(location="reference_frame.png")
+
+operation = client.models.generate_videos(
+    model="veo-3.1-generate-preview",
+    prompt="The agent continues running, dodging obstacles",
+    image=reference_img,  # Use as starting frame
+    config=types.GenerateVideosConfig(
+        duration_seconds=8,
+        aspect_ratio="16:9",
+        resolution="720p"
+    )
+)
+```
+
+## Best Practices for Smooth Videos
+
+### 1. Use Frame Chaining
+```python
+# Extract last frame from previous scene
+last_frame = extract_last_frame("scene_01.mp4")
+
+# Use as starting frame for next scene
+next_scene = generate_video(
+    prompt="Next action...",
+    image=types.Image.from_file(location=last_frame)
+)
+```
+
+### 2. Maintain Consistent Styling
+```python
+character = "A tactical agent in black gear"
+style = "Cinematic action movie, dramatic lighting"
+
+# Use same character/style in ALL scenes
+for scene_action in actions:
+    prompt = f"{character}, {style}. {scene_action}"
+```
+
+### 3. Add Crossfade Transitions
+Use the crossfade concatenation method above for professional results.
+
+### 4. Optimize Prompts for Continuity
+```python
+# Good: Mentions continuity
+"The agent continues running from the alley into the street..."
+
+# Bad: No continuity reference
+"The agent is in the street..."
+```
+
+## Cost Estimates (November 2025)
+
+### Nano Banana Pro Images
+- 1K: ~$0.024 per image
+- 2K: ~$0.036 per image
+- 4K: ~$0.06 per image
+
+### Veo 3.1 Videos
+- 8-second clip: ~$0.15 per video
+- Audio included automatically
+
+### Example: 24-Second Enhanced Video with Smooth Transitions
+- 1x Nano Banana Pro (2K): $0.036
+- 3x Veo 3.1 (8s each): $0.45
+- **Total: ~$0.49**
+
+## Common Issues & Solutions
+
+### Issue: Jerky Transitions ✅ SOLVED
+**Solution**: Use crossfade transitions (see above) or xfade filter
+
+### Issue: Character Inconsistency
+**Solution**: Use identical character description in all prompts
+
+### Issue: Low Quality Output
+**Solution**: Use Nano Banana Pro for reference images (2K or 4K)
+
+### Issue: Audio Pops at Transitions
+**Solution**: Use audio crossfade in filter_complex
+
+## Reference Links
+
+- Official Veo Documentation: https://ai.google.dev/gemini-api/docs/video
+- Image Generation: https://ai.google.dev/gemini-api/docs/image-generation
+- Pricing: https://ai.google.dev/gemini-api/docs/pricing
+- FFmpeg Crossfade: https://ffmpeg.org/ffmpeg-filters.html#xfade
+
+---
+
+Last Updated: November 2025 - Added smooth transition solutions