videowright 0.1.0

package/skill/references/audio/music/music.md

@@ -0,0 +1,171 @@
+# Background Music
+
+## When this is loaded
+
+You were routed here from [audio.md](../../audio.md) because the user wants to add background music to their video. This reference covers sourcing, approving, and managing music assets.
+
+## Overview
+
+Each music asset is a slug-named folder under `audio/originals/music/`. Music assets are immutable once used in a rendered track. The agent sources them (BYO, ElevenLabs, or Openverse), gets user approval, then references them in the audio plan as cues with volume curves and ducking.
+
+## Sourcing decision
+
+Ask the user how they want to source background music. Always ask when starting a new sourcing job, even if existing music in the project used a specific source -- don't assume continuity with prior assets.
+
+> How would you like to source background music?
+>
+> 1. **Bring your own** -- drop an audio file (mp3/wav) into the project and I will set it up.
+> 2. **Generate with ElevenLabs** -- describe the music you want and I will generate it via the ElevenLabs Music Generation API. **Requires a paid plan and API key.**
+> 3. **Search Openverse** -- search for freely-licensed music online. No API key required, free to use.
+
+Based on the answer, load the appropriate provider reference:
+
+- **BYO** -- load [providers/manual.md](providers/manual.md)
+- **ElevenLabs** -- load [providers/elevenlabs.md](providers/elevenlabs.md)
+- **Openverse** -- load [providers/openverse.md](providers/openverse.md)
+
+## Folder structure
+
+```
+audio/originals/music/
+  uplift_piano/
+    audio.mp3          # source audio (immutable after approval)
+    music.ts           # typed metadata with rich notes
+    generate.sh        # only for ElevenLabs-sourced music
+  tense_synth/
+    audio.mp3
+    music.ts
+    generate.sh
+```
+
+## Slug naming
+
+Music folders use short, semantic, snake_case names that describe the track's character:
+
+- `uplift_piano`
+- `tense_synth`
+- `calm_ambient`
+- `energetic_percussion`
+- `corporate_upbeat`
+
+The slug should convey the mood/instrument at a glance. Avoid generic names like `music_1` or `track_a`.
+
+## `music.ts` metadata
+
+Each music asset has a typed metadata file. The `notes` field is the key differentiator from SFX -- it should contain rich musical detail:
+
+```ts
+import type { MusicAsset } from "videowright";
+
+export const music: MusicAsset = {
+  name: "Uplift piano",
+  description: "Solo piano, gentle build to optimistic resolution",
+  length_s: 64.2,
+  source: "elevenlabs",
+  notes: `
+    BPM: 95
+    Key: C major
+    Mood: hopeful, restrained
+    Structure: ambient intro 0-8s, theme enters 8s, build 22-34s, beat drops at 34.2s, gentle outro 50s+
+    Loops: cleanly at the bar boundary (every ~10.1s after 8s)
+  `,
+};
+
+export default music;
+```
+
+Fields:
+
+| Field | Required | Description |
+|---|---|---|
+| `name` | Yes | Human-readable name for display and reference. |
+| `description` | Yes | One-line summary of the track's character. |
+| `length_s` | Yes | Duration in seconds, measured via ffprobe. Round to 1 decimal place. |
+| `source` | Yes | `"elevenlabs"`, `"user"`, or `"openverse"`. Records provenance. |
+| `notes` | Yes (strongly encouraged) | Free-text musical detail. See "What to capture in notes" below. |
+
+If the `MusicAsset` type is not yet exported from videowright, use an inline type annotation or `any` -- the shape is what matters.
+
+### What to capture in notes
+
+The `notes` field is free text. There are no structured fields for BPM, key, or mood -- instead, mix any useful detail freely. Common things worth noting:
+
+- **BPM** -- critical for beat-aligned visuals
+- **Key** -- useful if combining with other musical elements
+- **Mood** -- helps the agent choose volume curves and placement
+- **Structure** -- timestamps of sections (intro, build, drop, outro). Essential for slicing and ducking.
+- **Loop points** -- where the track loops cleanly, if applicable
+- **Beat positions** -- where notable rhythmic events land (useful as sync anchors)
+- **Instrument palette** -- what instruments are present
+- **Frequency range** -- if the track is mostly low-end, high-end, or full-spectrum
+
+Not all fields apply to every track. Write what is useful. Even brief notes ("ambient, no strong beat, loops after 8s") are better than nothing.
+
+## Approval UX
+
+After a music asset is sourced (file in place, `music.ts` written), present it for approval:
+
+1. **Print a clickable absolute `file://` link** to the audio file:
+
+   ```
+   Listen to the music track:
+   file:///absolute/path/to/audio/originals/music/uplift_piano/audio.mp3
+   ```
+
+2. **Prompt with exactly two options:**
+
+   > 1. **Approve** -- lock this asset for use in the audio plan.
+   > 2. **Discard and request changes** -- delete this asset and try again.
+
+3. **On Approve:**
+   - The asset is locked. Its folder and contents must not be modified or deleted.
+   - Proceed to integrate it into the audio plan (see "Integration into audio plan" below).
+
+4. **On Discard:**
+   - Delete the entire asset folder:
+     ```bash
+     rm -rf audio/originals/music/<slug>/
+     ```
+   - Ask: "What should change about this music?"
+   - Take feedback and re-source (loop back to the provider flow).
+
+### Pre-use deletion only
+
+A music folder can only be deleted if it has **never been used** in a rendered track. Once a track references it (via a plan snapshot), deletion would corrupt the snapshot. If the user asks to delete a used asset, refuse and explain which track(s) reference it.
+
+## Integration into audio plan
+
+After approval, the music is ready to be referenced in `audio/audio_plan.md` as a cue. See [../cue_template.md](../cue_template.md) for the cue format.
+
+Typical music cue:
+
+```
+### Cue 1 -- Background music
+Source: audio/originals/music/uplift_piano/
+Slice: 0-45.0
+Place at: 0.0
+Volume: 100% from 0s, ramps to 15% over 2.8-3.0s (VO starts at 3.0s), 15% from 3.0-22.0s, ramps to 50% over 22.0-22.3s (animation beat), ramps to 15% over 23.5-23.7s, 15% from 23.7-40.0s, ramps to 80% over 40.0-40.2s (outro), fade out 43.0-45.0s
+Fades: fade in 0-0.5s, fade out 43.0-45.0s
+Notes: Music bed throughout. Ducked under all VO segments. Rises during visual-only transitions and outro.
+```
+
+Key points:
+- `Source:` points to the music folder.
+- `Volume:` expresses the full ducking curve explicitly. The agent writes the actual levels at each point in time -- no opaque "duck under VO" directives. See [../styles.md](../styles.md) for level guidance.
+- Music typically spans most or all of the video. Use `Slice:` to trim if the track is longer than needed.
+- Ducking ramps should be ~0.2s (see [../styles.md](../styles.md)).
+- If the track has a strong beat, align visual transitions to beat boundaries where possible (compute from BPM in notes).
+
+## One track or many?
+
+For most product/demo videos (30s-3min), **one music track** is sufficient. The ducking curve provides enough variation. Multiple tracks increase complexity -- only use them if the video has distinct tonal sections requiring genuinely different music.
+
+## Edge cases
+
+| Situation | Behavior |
+|---|---|
+| Music is shorter than the video | Use `aloop` in ffmpeg to loop it. Note loop points in `music.ts` notes. See [../ffmpeg_cookbook.md](../ffmpeg_cookbook.md). |
+| Music is much longer than the video | Use `Slice:` in the cue to take only the needed portion. |
+| User wants music but no VO | Music plays at full level throughout. No ducking needed. Volume curve is simple: fade in, hold at 100%, fade out. |
+| User wants to swap music after a track is built | Source a new music asset in a new slug folder, approve it, update the audio plan cues to reference the new slug, rebuild the track. |
+| User provides a copyrighted track | Agent does not police copyright. Note in `music.ts` notes if the user mentions licensing (e.g., "Licensed via Epidemic Sound"). |

package/skill/references/audio/music/providers/elevenlabs.md

@@ -0,0 +1,170 @@
+# ElevenLabs Music Generation
+
+## When this is loaded
+
+The user chose ElevenLabs to generate background music. This reference covers the Music Generation API endpoint, prompt-writing tips, and the `generate.sh` template.
+
+## Prerequisites
+
+- `ELEVENLABS_API_KEY` set in `.env` at the project root. The key must have **Music Generation** permission enabled.
+- If the user does not have an API key, guide them through setup: see [../../voiceover/providers/elevenlabs.md](../../voiceover/providers/elevenlabs.md) (Step 2: Get the API key). The same key works for TTS, STT, SFX, and Music.
+
+## Cost notice
+
+> **Cost notice:** ElevenLabs charges credits for music generation. Longer tracks cost more credits. A 60-second track typically costs 2,000-5,000 credits depending on complexity. Check your remaining quota at https://elevenlabs.io/app/subscription before generating.
+
+## API endpoint
+
+**Endpoint:** `POST https://api.elevenlabs.io/v1/music-generation`
+
+**Request body:**
+
+```json
+{
+  "text": "<prompt describing the desired music>",
+  "duration_seconds": 60,
+  "prompt_influence": 0.5
+}
+```
+
+| Field | Required | Description |
+|---|---|---|
+| `text` | Yes | Natural-language description of the music. See prompt-writing tips below. |
+| `duration_seconds` | No | Target duration in seconds. If omitted, the API chooses (usually 30s). Recommended: specify to match your video length. |
+| `prompt_influence` | No | 0.0-1.0. Higher values follow the prompt more literally; lower values give more creative freedom. 0.5 is a good starting point for music. |
+
+**Response:** The API returns raw audio bytes (mp3) directly in the response body.
+
+## `generate.sh` template
+
+Write this script to `audio/originals/music/<slug>/generate.sh` for reproducibility:
+
+```bash
+#!/bin/bash
+# Generated Music: <name>
+# Prompt: <the exact prompt used>
+# Duration: <duration_seconds>s
+# Prompt influence: <prompt_influence>
+
+set -euo pipefail
+
+# Load API key from .env
+if [ -f .env ]; then
+  export $(grep -v '^#' .env | xargs)
+fi
+
+if [ -z "${ELEVENLABS_API_KEY:-}" ]; then
+  echo "Error: ELEVENLABS_API_KEY not set. Add it to .env" >&2
+  exit 1
+fi
+
+SLUG="<slug>"
+OUTPUT_DIR="audio/originals/music/${SLUG}"
+mkdir -p "${OUTPUT_DIR}"
+
+curl -X POST "https://api.elevenlabs.io/v1/music-generation" \
+  -H "xi-api-key: ${ELEVENLABS_API_KEY}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "text": "<prompt>",
+    "duration_seconds": <duration>,
+    "prompt_influence": <influence>
+  }' \
+  --output "${OUTPUT_DIR}/audio.mp3"
+
+echo "Music saved to ${OUTPUT_DIR}/audio.mp3"
+```
+
+Make the script executable: `chmod +x generate.sh`.
+
+**Important:** Run `generate.sh` from the **video folder** (the directory containing `timeline.ts`) so that relative paths resolve correctly.
+
+## Prompt-writing tips
+
+Music generation prompts benefit from structure. Include these dimensions:
+
+### Specify genre and instruments
+- Good: "Ambient electronic with soft piano chords and warm synth pads"
+- Bad: "Background music"
+
+### Describe mood and energy arc
+- Good: "Starts minimal and hopeful, builds gradually, reaches an optimistic peak at 30s, then settles into a gentle outro"
+- Bad: "Happy music"
+
+### Mention tempo and rhythm
+- Good: "Moderate tempo around 90-100 BPM, light percussion, no heavy drums"
+- Bad: "Not too fast"
+
+### Describe the use case
+- Good: "Background music for a product demo video. Needs to sit behind voiceover without competing. Should feel professional and modern."
+- Bad: "Music for a video"
+
+### Useful descriptors for product/demo videos
+
+| Dimension | Good choices |
+|---|---|
+| Genre | ambient, corporate, indie electronic, lo-fi, minimal, cinematic |
+| Instruments | piano, synth pads, soft guitar, muted percussion, strings, bells |
+| Mood | professional, optimistic, calm, confident, innovative, warm |
+| Energy | low-energy, mid-energy, building, steady, dynamic |
+| Avoid | aggressive drums, heavy bass drops, distorted guitars, vocals |
+
+### Duration guidance
+
+Match the music duration to your video:
+- Request slightly longer than needed (add 5-10s buffer). You can slice in the audio plan.
+- For videos with intro + content + outro: request the full duration so you get a natural arc.
+- For looping backgrounds: request at least one full musical phrase (usually 16-32s at standard tempos) so the loop point sounds clean.
+
+## Post-generation workflow
+
+After the curl command succeeds:
+
+1. **Verify the file exists and is non-empty:**
+   ```bash
+   ls -la audio/originals/music/<slug>/audio.mp3
+   ```
+
+2. **Measure duration via ffprobe:**
+   ```bash
+   ffprobe -v error -show_entries format=duration \
+     -of default=noprint_wrappers=1:nokey=1 \
+     audio/originals/music/<slug>/audio.mp3
+   ```
+
+3. **Ask the user to listen and describe the track.** Before writing metadata, ask the user to play the track and report:
+   - BPM (can often be inferred from the prompt's requested tempo; otherwise ask the user to estimate)
+   - Key (if discernible)
+   - Structure (where sections change, where the energy peaks)
+   - Any notable moments (beat drops, transitions, loop points)
+
+4. **Write `music.ts`** with rich metadata (see [../music.md](../music.md) for the shape). Set `source: "elevenlabs"`. Include all observed musical details in the `notes` field.
+
+5. **Trigger the approval UX** (see [../music.md](../music.md) -- Approval UX section).
+
+## Iteration on discard
+
+If the user discards and requests changes:
+
+1. Delete the folder: `rm -rf audio/originals/music/<slug>/`
+2. Ask what should change about the music.
+3. Adjust the prompt based on feedback. Common adjustments:
+   - "Too energetic" -- add "calm", "minimal", "ambient", reduce tempo
+   - "Too boring" -- add "building", "dynamic", increase tempo, add percussion
+   - "Wrong mood" -- change mood descriptors entirely
+   - "Too short" -- increase `duration_seconds`
+   - "Instruments are wrong" -- specify desired instruments, explicitly exclude unwanted ones ("no drums", "no vocals")
+   - "Needs a stronger ending" -- describe the outro in the prompt ("resolves to a clear final chord")
+4. Re-run with the updated prompt. Write a new `generate.sh` reflecting the new parameters.
+
+## Troubleshooting
+
+| Issue | Resolution |
+|---|---|
+| 401 Unauthorized | Check `ELEVENLABS_API_KEY` in `.env`. Ensure the key has Music Generation permission. |
+| 422 Unprocessable Entity | Prompt may be problematic. Keep prompts 20-500 characters. Avoid special characters. |
+| Empty or 0-byte response | API may have failed silently. Retry. If persistent, try a shorter duration or different prompt. |
+| Generated music does not match prompt | Increase `prompt_influence` (try 0.6-0.8). Be more specific about instruments and mood. |
+| Music has vocals/singing | Add "instrumental only, no vocals, no singing" to the prompt. |
+| Rate limited (429) | Wait and retry. Check quota at https://elevenlabs.io/app/subscription. |
+| Track is shorter than requested | The API may cap duration. Try requesting in smaller segments or accept the shorter track. |

package/skill/references/audio/music/providers/manual.md

@@ -0,0 +1,140 @@
+# Manual Music (Bring Your Own)
+
+## When this is loaded
+
+The user has their own music audio file and wants to add it to the project. This is the BYO path from [../music.md](../music.md).
+
+## Overview
+
+The manual flow takes a user-provided music file, places it in the correct folder, gathers rich metadata, and presents it for approval. No API calls needed.
+
+## Step 1: Set up the folder
+
+Choose a slug name for this music track. The slug should describe the track's character in short snake_case:
+
+- `uplift_piano`
+- `calm_ambient`
+- `energetic_corporate`
+- `dark_synth_intro`
+
+Create the directory:
+
+```bash
+mkdir -p audio/originals/music/<slug>/
+```
+
+## Step 2: Get the audio file
+
+Ask the user for the file. Two options:
+
+1. **File path.** The user provides a path to an existing file. Copy it into the folder:
+   ```bash
+   cp /path/to/their/file.mp3 audio/originals/music/<slug>/audio.mp3
+   ```
+
+2. **Drop-in.** Ask the user to place the file directly:
+   > Place your music file at `audio/originals/music/<slug>/audio.mp3` (or `audio.wav`).
+
+### Supported formats
+
+- MP3 and WAV are standard. Both work with ffmpeg.
+- Other formats (M4A, OGG, FLAC, AIFF) are also accepted by ffmpeg.
+- No specific sample rate or bitrate requirements.
+
+## Step 3: Gather metadata
+
+Music metadata should be richer than SFX. Ask the user:
+
+> I need details about this music track. Please share as much as you know:
+>
+> 1. **Name** -- a short descriptive name (e.g., "Uplift piano")
+> 2. **Description** -- one sentence summary (e.g., "Solo piano, gentle build to optimistic resolution")
+> 3. **Notes** -- any musical detail you know. Helpful things to include:
+>    - BPM (tempo)
+>    - Key (e.g., C major, A minor)
+>    - Mood (e.g., hopeful, tense, calm)
+>    - Structure (e.g., "intro 0-8s, main theme 8-30s, outro 30-40s")
+>    - Loop points (where it loops cleanly, if applicable)
+>    - Instrument palette
+>    - Licensing info (if relevant)
+>
+> Even partial info is useful -- I will ask follow-up questions if needed.
+
+### Agent-observed details
+
+After the user provides their info, the agent should also note any additional observations in `notes`:
+
+- If BPM was not provided, ask the user to estimate or measure it after listening
+- Ask the user about any structural transitions (energy changes, instrument entrances)
+- Ask whether the track has a clean ending or fades out
+- Ask whether it loops well
+
+Combine user-provided info and agent observations into the `notes` field.
+
+## Step 4: Measure duration
+
+Use ffprobe to get the exact length:
+
+```bash
+ffprobe -v error -show_entries format=duration \
+  -of default=noprint_wrappers=1:nokey=1 \
+  audio/originals/music/<slug>/audio.mp3
+```
+
+Round to 1 decimal place for `length_s`.
+
+## Step 5: Write `music.ts`
+
+Create `audio/originals/music/<slug>/music.ts`:
+
+```ts
+import type { MusicAsset } from "videowright";
+
+export const music: MusicAsset = {
+  name: "<name from user>",
+  description: "<description from user>",
+  length_s: <measured duration>,
+  source: "user",
+  notes: `
+    <combined notes from user + agent observations>
+  `,
+};
+
+export default music;
+```
+
+Set `source: "user"` to record that this was a BYO asset.
+
+There is no `generate.sh` for BYO assets.
+
+## Step 6: Trigger approval UX
+
+Follow the approval flow from [../music.md](../music.md):
+
+1. Print the clickable `file://` link to the audio file.
+2. Prompt: Approve or Discard and request changes.
+3. On Approve: asset is locked, ready for use in the audio plan.
+4. On Discard: delete the folder, ask what to change, loop back.
+
+For BYO assets, "discard and request changes" means the user needs to provide a different file. Ask them to supply a different track.
+
+## Expected folder state after approval
+
+```
+audio/originals/music/<slug>/
+  audio.mp3    # user-provided audio (immutable after approval)
+  music.ts     # typed metadata with rich notes
+```
+
+No `generate.sh` for BYO assets.
+
+## Edge cases
+
+| Situation | Behavior |
+|---|---|
+| Track has vocals | Warn that vocals may conflict with voiceover. Suggest an instrumental version if available, or note that heavy ducking will be needed. |
+| Track is very short (< 10s) | May need looping. Note loop-ability in `notes`. See [../../ffmpeg_cookbook.md](../../ffmpeg_cookbook.md) for `aloop`. |
+| Track is very long (> 5min) | Fine -- the audio plan will use `Slice:` to take only the needed portion. |
+| User does not know BPM/key | That is fine. BPM is optional -- ask the user to estimate if needed. Key is optional. Write what is known. |
+| User mentions licensing/attribution | Record in `notes` (e.g., "Licensed via Artlist, attribution required in credits"). The agent does not enforce licensing. |
+| User wants to replace after approval | Cannot edit in place. Source a new version in a new slug folder and update audio plan cues. |