@officexapp/vidfarm-devcli 0.21.10

package/.agents/skills/vidfarm-director/references/primitives.md

@@ -0,0 +1,307 @@
+## Primitive: image_remove_background
+
+Remove the background from any image URL. Result is a transparent PNG/WebP stored at a durable Vidfarm URL you can reuse in compositions or downstream primitives.
+
+- `POST /api/v1/primitives/images/remove-background`
+- Body: `{ "source_image_url": "https://...", "output_format": "webp" | "png" | "jpeg" }` (`output_format` defaults to `webp`)
+- Response: standard primitive job; poll for completion, then read `image.file_url` / `primary_file_url`
+- Billing: flat pass-through fee via RapidAPI (`rapidapi_remove_background` cost center)
+- Reach for this only for an **arbitrary photo with a messy background**. If the image already sits on a **flat solid background** (anything you generated on a controlled backdrop), use the far cheaper `image_remove_background_greenscreen` below instead.
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/images/remove-background" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{"source_image_url": "https://cdn.example.com/photo.jpg"}'
+```
+
+## Primitive: remove-greenscreen (image **or** video chroma-key)
+
+**Local chroma-key** background removal for an IMAGE **or** a VIDEO — no third-party API, so it costs a fraction of `image_remove_background`. Use it whenever the source sits on a **flat, solid, evenly-lit background**: a green screen, or any single fill color. An IMAGE returns a transparent PNG/WebP; a VIDEO returns a transparent **WebM (VP9 alpha)** that stays transparent in the `/editor` preview AND the final render (it composites over whatever layers sit behind it).
+
+- **Unified (auto-detect):** `POST /api/v1/primitives/remove-greenscreen` — detects image vs video from the source and dispatches. Preferred entry point.
+- Image-only: `POST /api/v1/primitives/images/remove-background-greenscreen` (flat alias: `POST /api/v1/primitives/remove-background-greenscreen`). Video-only: `POST /api/v1/primitives/videos/remove-greenscreen`.
+- Body: `{ "tracer": "...", "payload": { "source_url": "https://..." (image OR video), "media_type"?: "image" | "video" (skip auto-detect), "preset"?: "green" | "digital-green" | "blue" | "white" | "black" | "magenta", "key_color"?: "#00FF00", "tolerance"?: 0.3, "softness"?: 0.1, "despill"?: true, "output_format"?: "png" | "webp" (image only) }, "webhook_url"?: "..." }`
+  - `preset` — pick a common background color by name; fills `key_color` + tuned defaults. `key_color` (hex / `rgb(...)` / named) overrides the preset for a custom color/range.
+  - `tolerance` (0–1) — how close a pixel must be to key it out. Raise it if fringe survives, lower it if the subject is eaten. `softness` (0–1) feathers the edge; `despill` (default `true`) trims the residual key-color rim (green/blue only).
+- Response: standard primitive job; poll, then read `primary_file_url` (a `.png/.webp` for image, `.webm` for video). After keying, `set_layer_media` the URL back onto the source layer to swap the transparent cut-out in place.
+- Billing: the **real measured AWS compute cost × 1.2** (`greenscreen_chroma_key` cost center) — no flat fee. Far cheaper than the RapidAPI matting route.
+
+## Primitive: media_overlay ("create media overlay" — Vox-style)
+
+**The go-to way to mint a transparent overlay graphic to float over a composition.** So-called "Vox-style" animations are really just AI-generated illustrations with transparent backgrounds animated in HTML over the timeline — this primitive is that two-step process fused into one job: it (1) generates an AI image of your subject on a **forced flat key-color background** (using the caller's own BYOK image keys, with the greenscreen instruction injected into the prompt automatically) and (2) chroma-keys that background out, returning a ready-to-composite transparent PNG/WebP. **Prefer this over generate-then-manually-remove**, and reach for it *frequently* whenever a scene wants a floating illustration, prop, icon, sticker, or cut-out character.
+
+- `POST /api/v1/primitives/images/create-overlay` (flat alias: `POST /api/v1/primitives/create-media-overlay`)
+- Body: `{ "tracer": "...", "payload": { "prompt": "a cartoon rocket ship, flat vector illustration", "provider"?: "...", "model"?: "...", "prompt_attachments"?: ["https://..."], "aspect_ratio"?: "1:1", "image_size"?: "1K"|"2K"|"4K", "key_color"?: "#00FF00", "tolerance"?: 0.3, "softness"?: 0.1, "despill"?: true, "output_format"?: "png"|"webp" }, "webhook_url"?: "..." }`
+  - Just describe the **subject** in `prompt` — do NOT describe the background; the primitive appends the flat key-color background requirement for you.
+  - `key_color` lets you key against something other than green if your subject is itself green-heavy (e.g. key against `#FF00FF` magenta for a green frog).
+  - The chroma-key params (`tolerance`/`softness`/`despill`) behave exactly as in `image_remove_background_greenscreen`.
+- Response: `primary_file_url` is the finished transparent overlay; `greenscreen_source_url` is the raw pre-key frame (kept so you can re-key at a different tolerance without paying for another generation).
+- Billing: small platform wallet fee (`media_overlay` cost center) **plus** the BYOK image-generation cost on the caller's own provider key.
+- Placement: once the job settles, `add_layer` (or the editor's `generate_layer`/`add_layer`) the `primary_file_url` as an image layer over the video and animate it (Ken Burns, keyframes, entrance transitions) like any other overlay.
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/images/create-overlay" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{"tracer": "vox-overlay", "payload": {"prompt": "a friendly cartoon lightbulb mascot, flat vector illustration", "aspect_ratio": "1:1"}}'
+```
+
+## Primitive: videos/download
+
+Download a supported social/media post URL into a durable Vidfarm-hosted VISUAL asset. This route is intentionally broader than its old name suggests: it returns either a normal MP4 for video posts, or a slideshow payload for photo/carousel posts.
+
+- `POST /api/v1/primitives/videos/download`
+- Body: `{ "tracer": "...", "payload": { "source_url": "https://...", "quality"?: "best" | "hd" | "full_hd", "save_manifest"?: true }, "webhook_url"?: "..." }`
+- Response: standard primitive job. Poll `GET /api/v1/primitives/jobs/:jobId` until `status: "succeeded"`.
+  - Video post: read `primary_file_url` / `video.file_url` / `videoUrl` for the durable MP4.
+  - Photo/carousel post: read `mediaKind: "slideshow"`, ordered `slideImageUrls[]`, optional `slideAudioUrl`, and `primary_file_url` (first slide image).
+- Billing: RapidAPI pass-through wallet charge (`rapidapi_video_download`) plus the small mirror/extract pass (`video_download_lambda` when MP4 download is involved).
+- Free-plan / no-spend rule: do **not** use this paid route when the user wants a free path. In the web app, tell them to use the browser to find a downloader instead. Fallback wording is explicit: suggest Googling `"youtube video downloader"` or `"tiktok/twitter/instagram/etc audio/video downloader"`.
+- devcli wrapper: `vidfarm download-post <url> [--quality best|hd|full_hd]`
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/videos/download" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{"tracer": "demo-download-post", "payload": {"source_url": "https://www.tiktok.com/@example/video/123"}}'
+```
+
+## Primitive: audio/download
+
+Download just the original audio/music/voice track from a supported social/media post URL into a durable Vidfarm-hosted audio file. It reuses the same RapidAPI resolver as `videos/download`, but returns audio instead of a video/slideshow.
+
+- `POST /api/v1/primitives/audio/download`
+- Body: `{ "tracer": "...", "payload": { "source_url": "https://...", "save_manifest"?: true }, "webhook_url"?: "..." }`
+- Response: standard primitive job. Poll `GET /api/v1/primitives/jobs/:jobId` until `status: "succeeded"`, then read `primary_file_url` / `audio.file_url` / `audioUrl`.
+- Behavior: prefers the provider's direct audio track when present; otherwise resolves the source video and extracts audio server-side.
+- Billing: same RapidAPI wallet class as `videos/download`.
+- Free-plan / no-spend rule: same as the visual download route. Use the browser and, if needed, suggest Googling `"youtube audio downloader"` or `"tiktok/twitter/instagram/etc audio/video downloader"` instead of spending wallet credits.
+- devcli wrapper: `vidfarm download-post-audio <url>`
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/audio/download" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{"tracer": "demo-download-audio", "payload": {"source_url": "https://www.youtube.com/watch?v=example"}}'
+```
+
+## Primitive: video_remove_captions
+
+Remove burned-in captions/subtitles/on-screen text from any video URL (GhostCut-powered — the same pipeline auto-decompose uses for its caption-free mirror, exposed as a standalone reusable primitive). Result is a durable caption-free MP4 stored at a Vidfarm URL.
+
+- `POST /api/v1/primitives/videos/remove-captions` (alias: `POST /api/v1/primitives/remove-video-captions`)
+- Body: `{ "tracer": "...", "payload": { "source_video_url": "https://..." }, "webhook_url"?: "..." }`
+- Response: standard primitive job (`202 { job_id }`). The job probes the source, submits to GhostCut, polls to completion server-side (typically a few minutes), and mirrors the result. Poll `GET /api/v1/primitives/jobs/:jobId` until `status: "succeeded"`, then read `primary_file_url` / `video.file_url` (`captionsRemovedVideoUrl` in the output).
+- Billing: `~$0.10 per 30 seconds` of source video (per-chunk, 1-chunk minimum; `ghostcut_subtitle_removal` cost center), plus the small probe cost.
+- For the CURRENT fork's decompose flow, don't re-run this primitive — the caption-free mirror already exists via `GET /api/v1/compositions/:forkId/remove-video-captions`.
+- **Short clips only — hard cap ~15 minutes** (`GHOSTCUT_MAX_DURATION_SEC`, default 900s). Never point this at a long-form source: for "clips without captions" from a long video, run a raws hunt with `avoid_text: true` (scene selection — see "Raws") and, if needed, apply this primitive to the individual FINISHED raws.
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/videos/remove-captions" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{"tracer": "demo-remove-captions", "payload": {"source_video_url": "https://cdn.example.com/clip.mp4"}}'
+```
+
+## Primitive: media_dedupe
+
+Apply subtle camouflage transforms (zoom, tilt, rotate, saturation, playback speed, contrast, brightness, hue rotate, blur, horizontal flip, tint overlay) to any source **image or video** URL. Useful when reposting existing media and you want small perceptual differences to reduce duplicate-detection.
+
+- `POST /api/v1/primitives/media/dedupe`
+- Body: `{ "tracer": "...", "payload": { ...fields... }, "webhook_url"?: "..." }`
+  - Note: webhook delivery is not yet active — `webhook_url` is accepted and persisted on the job but never fired. Poll the job endpoints (`GET /api/v1/primitives/jobs/:jobId`) for completion.
+- Response: standard primitive job. Poll to completion, then read `primary_file_url` (also `video.file_url` for MP4 or `image.file_url` for stills)
+- Billing: metered as a HyperFrames render (via `hyperframes_lambda` on prod cloud render; free on a local serve box)
+
+Payload fields:
+
+- `source_media_url` (required, URL) — the image or video to transform
+- `media_type` (`"image" | "video"`, optional) — auto-detected from URL extension if omitted (`.mp4/.mov/.webm/.m4v` → video, else image)
+- `effects` (optional object). All fields optional; defaults camouflage lightly:
+  - `zoom` (default `1.04`) — scale factor
+  - `tilt` (default `3`) — degrees of 3D X-axis tilt (perspective 1000px)
+  - `rotate` (default `3`) — degrees of 2D rotation
+  - `saturation` (default `1.05`)
+  - `speed` (default `1.05`, video only) — playback rate, also compresses output duration accordingly
+  - `horizontal_flip` (default `false`)
+  - `contrast` (default `1.05`)
+  - `brightness` (default `1.05`)
+  - `hue_rotate` (default `0`) — degrees
+  - `blur` (default `0`) — pixels
+- `tint_color` (default `"#FF8C00"`), `tint_opacity` (default `0.08`) — subtle color overlay
+- `width` (default `1080`), `height` (default `1920`) — output canvas
+- `duration_ms` — output duration for video; if omitted, `fallback_duration_ms` (default `5000`) is used
+- `object_fit` (`"cover" | "contain" | "fill" | "none" | "scale-down"`, default `"cover"`)
+- `background_color` (default `"#000000"`) — visible when `object_fit` leaves letterboxing
+- `muted` (default `false`), `volume` (default `1`) — audio pass-through on video
+- `output_format` (`"png" | "jpeg" | "webp"`, default `"png"`) — image mode only; video mode always outputs MP4
+
+Video example (camouflage a reused clip):
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/media/dedupe" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{
+    "tracer": "dedupe-2026-07-01-abc",
+    "payload": {
+      "source_media_url": "https://cdn.example.com/reel.mp4",
+      "media_type": "video",
+      "duration_ms": 8500,
+      "effects": {
+        "zoom": 1.05,
+        "tilt": 2,
+        "rotate": -2,
+        "speed": 1.03,
+        "hue_rotate": 4,
+        "horizontal_flip": true
+      },
+      "tint_color": "#00A3FF",
+      "tint_opacity": 0.06
+    }
+  }'
+```
+
+Image example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/media/dedupe" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{
+    "tracer": "dedupe-still-01",
+    "payload": {
+      "source_media_url": "https://cdn.example.com/photo.jpg",
+      "media_type": "image",
+      "effects": { "zoom": 1.06, "rotate": -1, "hue_rotate": 8 },
+      "output_format": "webp"
+    }
+  }'
+```
+
+## Primitive: music (text → music)
+
+Generate music (instrumental, songs with lyrics, background beds, jingles, scores) via ElevenLabs. **Default this on freely — music is a core primitive.** `use_wallet_credits` defaults **true**: it runs on vidfarm's platform ElevenLabs key and bills the customer's wallet. Recommend keeping it on; set it false only to save wallet credits or to use the customer's OWN saved ElevenLabs key.
+
+- `POST /api/v1/primitives/music/generate` (alias: `POST /api/v1/primitives/music`)
+- Body: `{ "tracer": "...", "payload": { "prompt": "...", "music_length_ms"?: 30000, "composition_plan"?: {...}, "model"?: "...", "use_wallet_credits"?: true }, "webhook_url"?: "..." }`
+  - `prompt` — describe the vibe/genre/instrumentation. `music_length_ms` ≤ 300000 (5 min), default 30000.
+  - `composition_plan` (optional) — ElevenLabs section-by-section plan for granular lyric/style control (see the loadable `music` skill pack).
+- Result: durable mp3 (`primary_file_url` / `audio.file_url`). Mount as its own `<audio>` layer ~0.1–0.2 under narration.
+- devcli: `vidfarm music "<prompt>" --length 30` (`--own-key` to use your ElevenLabs key).
+
+## Primitive: tts (text → speech)
+
+Generate spoken narration audio from text. **`use_wallet_credits` defaults true** → high-quality ElevenLabs narration on vidfarm's platform key, billed to the wallet — the recommended default. Set it false to use the customer's OWN ElevenLabs key or a BYOK OpenAI/Gemini/OpenRouter key (BYOK is never wallet-billed). The **voice style is promptable** via `instructions` (tone/pacing/accent/emotion/persona). Result is a durable Vidfarm audio URL (mp3/wav) ready to drop into a composition as an audio layer.
+
+- `POST /api/v1/primitives/audio/speech` (alias: `POST /api/v1/primitives/tts`)
+- Body: `{ "tracer": "...", "payload": { "text": "...", "voice"?: "...", "instructions"?: "...", "use_wallet_credits"?: true, "provider"?: "openai" | "gemini" | "openrouter", "model"?: "...", "output_format"?: "mp3" | "wav" }, "webhook_url"?: "..." }`
+  - `text` (required, ≤8000 chars) — the words to speak, verbatim. Aliases: `input`, `script`.
+  - `voice` (optional) — on the ElevenLabs path, an ElevenLabs **voice_id** (or friendly name george/sarah/…); on the BYOK path a provider preset (OpenAI `alloy`/…; Gemini `Kore`/…). **Default a sensible voice and tell the user they can pick from many** — list them with `GET /api/v1/primitives/audio/voices` (or `vidfarm voices`).
+  - `instructions` (optional) — the voice-STYLE prompt. Aliases: `style`, `style_instructions`, `voice_style`.
+  - `output_format` defaults to `mp3` (Gemini/ElevenLabs may answer in wav — read `audio.content_type`).
+- Response: standard primitive job. Poll to completion, then read `primary_file_url` / `audio.file_url`.
+- Billing: wallet-billed on the platform ElevenLabs key (default); free/BYOK when `use_wallet_credits: false` and the caller's own key serves it.
+- devcli: `vidfarm tts "…" --style "…"` runs LOCAL-FIRST on your own env key (no job); add `--cloud` for the ElevenLabs platform job (`--own-key` = your key). `vidfarm voices` lists voices.
+
+## Primitive: audio/voices (list ElevenLabs voices)
+
+`GET /api/v1/primitives/audio/voices` returns the ElevenLabs voice catalog — `{ scope, default_voice_id, voice_library_url, voices: [{ voice_id, name, category, labels, description, preview_url }] }`. Default scope is vidfarm's platform account; add `?use_wallet_credits=false` to list the voices on the customer's OWN saved ElevenLabs key. Pick a `voice_id` and pass it as `voice` to `/audio/speech`. devcli: `vidfarm voices` (`--own-key` for the user's account).
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/audio/speech" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{
+    "tracer": "demo-tts",
+    "payload": {
+      "text": "Welcome back. Today we are testing the three most viral hooks.",
+      "voice": "coral",
+      "instructions": "energetic UGC creator, conversational, slightly breathless"
+    }
+  }'
+```
+
+## Primitive: stt (video/audio → transcript)
+
+Transcribe speech from **any video or audio URL** on the caller's saved AI provider keys (video sources are demuxed to audio automatically via the platform ffmpeg seam). One job returns **BOTH formats**:
+
+1. **Simple subtitle version** — the plain transcript (`output.text`, `transcript.txt`) plus timed SRT cues (`output.subtitles.srt`, `transcript.srt`) with no speaker labels — drop-in captions.
+2. **Advanced multi-speaker version** — speaker-attributed, timestamped segments (`output.segments`, each `{ speaker, start_sec, end_sec, text }`, with `output.speakers` listing the cast) for multi-narration sources like podcasts, interviews, and skits — stored durably as `transcript.json`.
+
+- `POST /api/v1/primitives/audio/transcribe` (alias: `POST /api/v1/primitives/stt`)
+- Body: `{ "tracer": "...", "payload": { "source_url": "https://...", "diarize"?: true, "word_timestamps"?: false, "use_wallet_credits"?: true, "language"?: "en", "prompt"?: "...", "provider"?: "gemini" | "openai" | "openrouter", "model"?: "...", "media_type"?: "auto" | "video" | "audio" }, "webhook_url"?: "..." }`
+  - `source_url` (required) — video OR audio URL. Aliases: `source_video_url`, `source_audio_url`, `video_url`, `audio_url`, `url`. For a fork's composition, pick the right source via `GET /api/v1/compositions/:forkId/remove-video-captions` first.
+  - `use_wallet_credits` (default `true`) — ElevenLabs Scribe on the platform key (native diarization + real word timestamps), billed to the wallet. Set false to use the customer's OWN ElevenLabs key or a BYOK gemini/openai/openrouter key.
+  - `diarize` (default `true`) — attribute segments to speakers. On the BYOK path speaker labels need a **Gemini key**; OpenAI/OpenRouter degrade to a single-speaker transcript. ElevenLabs Scribe (the default) diarizes natively.
+  - `prompt` (optional) — domain/vocabulary hint; `language` (optional) — expected language code.
+- Response: standard primitive job. Poll to completion; `output` carries both formats inline (very long transcripts set `text_truncated: true` and keep the full copy in the `transcript.json` artifact), and the artifacts `transcript.json` / `transcript.srt` / `transcript.txt` are durable URLs.
+- Limits: ~40 minutes of speech per job (20MB extracted audio) — trim longer sources with `/videos/trim` or `/audio/trim` and transcribe in parts.
+- Billing: wallet-billed on the platform ElevenLabs key (default); free/BYOK when `use_wallet_credits: false`. The tiny audio-demux compute step always bills (`primitive_media_lambda`), like every media primitive.
+- For the CURRENT fork, the decompose-time transcript may already exist via `video-context.json` — prefer it before paying for a new transcription; use this primitive for other URLs, multi-speaker attribution, or SRT output.
+- devcli: `vidfarm stt <file|url>` runs LOCAL-FIRST (local ffmpeg + your own key); add `--cloud` to run this route instead.
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/audio/transcribe" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{
+    "tracer": "demo-stt",
+    "payload": {
+      "source_url": "https://cdn.example.com/podcast-clip.mp4",
+      "diarize": true,
+      "language": "en"
+    }
+  }'
+```
+
+## Brainstorm primitives
+
+The `brainstorm/*` primitives are the strategy toolkit. They are reusable, billable AI reasoning steps — the same family the AI Copilot exposes as chip suggestions. Treat **product placement** as a first-class member of this family, right alongside angles and hooks:
+
+- `POST /api/v1/primitives/brainstorm/coldstart` — `{ payload: { user_message } }` → foundational questionnaire for a customer starting from zero.
+- `POST /api/v1/primitives/brainstorm/awareness_stages` — `{ payload: { offer_description } }` → which Eugene-Schwartz awareness stages to target first.
+- `POST /api/v1/primitives/brainstorm/angles` — `{ payload: { offer_description, problem_awareness, solution_awareness } }` → persuasive angles.
+- `POST /api/v1/primitives/brainstorm/hooks` — `{ payload: { offer_description } }` → many TikTok-native hooks.
+- `POST /api/v1/primitives/brainstorm/product_placement` — `{ payload: { source_video_url, offer_description } }` → **watches the video** and returns concrete, timestamped opportunities to natively place the product.
+
+### Primitive: brainstorm_product_placement
+
+Analyze a source video and identify where a product can be organically placed so it feels native rather than bolted on. This is a **multimodal** primitive: it feeds the actual video to a vision-capable model, so prefer a saved **Gemini** key (native video understanding). OpenRouter multimodal models also work; plain OpenAI chat keys cannot watch video.
+
+- `POST /api/v1/primitives/brainstorm/product_placement`
+- Body: `{ "tracer": "...", "payload": { "source_video_url": "...", "offer_description": "...", "count"?: 8, "provider"?: "gemini" }, "webhook_url"?: "..." }`
+  - `source_video_url` (required, URL) — the exact durable/public video to analyze. Aliases accepted: `video_url`, `source_url`, `url`. For a fork's composition, call `GET /api/v1/compositions/:forkId/remove-video-captions` first and pass the correct URL — the **original video** (`original_source_url`) or the **decomposed video** (`mirrored_url`).
+  - `offer_description` (required) — the product/offer to place. Aliases: `offer`, `description`, `details`.
+  - `count` (optional, 3–30, default 8) — number of opportunities; only send when the user asks for a specific number.
+- Response: standard primitive job. Poll `GET /api/v1/primitives/jobs/:jobId` to completion, then read `result.json` (also surfaced on the job output as `opportunities[]`), where each entry has `timestamp`, `scene`, `placement_type`, `placement_idea`, and `why_it_works`.
+
+You can also satisfy a quick, conversational product-placement question by reasoning over an attached video directly instead of calling this primitive — use the primitive when you want a durable, structured, billable artifact the customer can save and reuse.
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/brainstorm/product_placement" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{
+    "tracer": "product-placement-01",
+    "payload": {
+      "source_video_url": "https://cdn.example.com/source.mp4",
+      "offer_description": "A $29/mo AI meal-planning app for busy parents."
+    }
+  }'
+```

package/.agents/skills/vidfarm-director/references/rest-api.md

@@ -0,0 +1,84 @@
+# Vidfarm REST API — On-Demand Map
+
+Use this file when integrating through raw HTTP, looking up an endpoint, or auditing the complete public API. This is a navigation layer: the detailed request bodies, response shapes, state transitions, cost notes, and route-specific guardrails live in the linked domain references so the default skill context stays small.
+
+## Contents
+
+- [Client choice](#client-choice)
+- [Base URL and authentication](#base-url-and-authentication)
+- [Complete specification map](#complete-specification-map)
+- [Raw HTTP pattern](#raw-http-pattern)
+- [Operational rules](#operational-rules)
+- [Keeping the reference complete](#keeping-the-reference-complete)
+
+## Client choice
+
+The skill is documentation for an AI agent. It is not a client library and does not execute requests.
+
+- **Raw REST:** every cloud operation is available through HTTP and can be called from `curl`, `fetch`, Python, CI, a server, or any other HTTP client.
+- **Recommended desktop convenience:** `npx @officexapp/vidfarm-devcli ...` (or the installed `vidfarm` binary) wraps those same routes. Prefer it for interactive desktop work because it handles the `vidfarm-api-key` header, uploads, polling, pagination, local paths, composition pull/publish state, and readable errors.
+- **Use raw REST** when building a durable integration, working in an existing application stack, calling a route the devcli has not wrapped yet, or when the user explicitly requests HTTP.
+
+Do not imply that installing this skill installs the CLI. Do not imply that the CLI unlocks API capabilities unavailable through REST.
+
+## Base URL and authentication
+
+- Production base URL: `https://vidfarm.cc`
+- API-key authentication: `vidfarm-api-key: <key>`
+- Do not use `Authorization: Bearer` unless a specific non-API browser/login flow explicitly documents it.
+- Send `content-type: application/json` for JSON bodies and `accept: application/json` when JSON is expected.
+- Treat fork IDs, share tokens, upload URLs, and returned media URLs according to the route-specific guidance in the domain references.
+
+## Complete specification map
+
+For a focused task, load only the relevant file. If the user requests the **entire REST API specification**, load this file plus every file in the following table. Together they are the bundled director-facing REST specification.
+
+| API area | Load | Coverage |
+|---|---|---|
+| Accounts, login, provider keys, discovery, inspirations, templates, forks, versions, render, approval, sharing, scheduling, billing | `core-workflows.md` | Customer and composition lifecycle |
+| Composition HTML/JSON, timeline layers, editor actions, clone/fork, auto-decompose, video context, captions, transitions, speech regeneration | `editor-workflows.md` | Editing and decomposition |
+| Public raws, clip hunts, exact-range clipping, My Files, temporary files, folders, annotations, semantic search, recurring-character assets | `assets-and-sourcing.md` | Media and library APIs |
+| Direct automation, pagination, scripting, devcli-to-route parity, local/cloud boundaries, skill-pack distribution | `automation-and-local-dev.md` | Headless and desktop integration |
+| Image, video, TTS, STT, music, captions, overlays, inpaint, background/caption removal, product placement, async primitive jobs | `primitives.md` | Generation and transformation primitives |
+| Onboarding document persistence | `onboarding.md` | Durable strategy/context files |
+
+The route and payload examples in those files are the public, customer-facing contract. Architecture-only internals are intentionally excluded. When exact behavior is ambiguous, inspect the live response or route implementation rather than inventing a field.
+
+## Raw HTTP pattern
+
+```bash
+curl --fail-with-body \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "accept: application/json" \
+  "https://vidfarm.cc/discover/feed?limit=20"
+```
+
+For JSON mutations:
+
+```bash
+curl --fail-with-body \
+  -X POST \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -H "accept: application/json" \
+  --data '{"template_id":"template_example"}' \
+  "https://vidfarm.cc/api/v1/compositions"
+```
+
+Use environment variables or a secret manager for keys. Never paste secrets into a skill, composition, source file, command transcript intended for sharing, or generated report.
+
+## Operational rules
+
+- Follow `next_cursor`/cursor fields until exhausted when the user asks for all records; do not call the first page the full dataset.
+- Treat submission and generation routes as non-idempotent unless the route explicitly states otherwise. Check existing state before retrying.
+- Poll asynchronous jobs at the documented status endpoint and stop on terminal success or failure. Do not fabricate completion from an accepted response.
+- Upload local media through the documented presign/upload/finalize sequence. The devcli is recommended on desktop because it performs this sequence automatically.
+- Parse and mutate composition HTML as a DOM. Never edit it by string concatenation.
+- Render through `POST /api/v1/compositions/:forkId/render`; do not call the underlying renderer directly.
+- Surface wallet/provider costs before expensive operations and preserve returned job IDs for audit and retries.
+
+## Keeping the reference complete
+
+The canonical skill sources live in `.agents/skills/vidfarm-director/`. `SKILL.director.md` is a generated copy-friendly rollup; this REST map is deliberately excluded from that immediate rollup so ordinary skill loads remain lean. It is distributed in the multi-file skill pack and fetched on demand through the public skill-pack routes.
+
+When routes change, update the appropriate domain reference and run `npm run check:skills`. That check verifies every documented `/api/` and `/login/` method/path against the registered application routes. Add new public API areas to the table above so exhaustive API requests remain discoverable.

package/.agents/skills/vidfarm-media/SKILL.md

@@ -0,0 +1,158 @@
+---
+name: vidfarm-media
+description: Media resolution for video workflows — AI music generation, narration TTS, voice-matched speech regeneration ("same speaker, new words"), transcription, word-level timings, BGM, SFX, and image/clip sourcing through vidfarm. Use when a video needs audio (music, voiceover/narration, sound effects), rewording of existing narration in the original speaker's voice, caption word timings, or media assets. ElevenLabs powers music/TTS/STT via the platform key (use_wallet_credits, wallet-billed) or the customer's own ElevenLabs key. Ships the shared audio engine scripts/audio.mjs (audio_request.json → audio_meta.json) used by faceless-explainer, product-launch-video, and general-video; documents the vidfarm music/tts/stt primitives, the keyless local Kokoro + whisper.cpp engines, My Files / raws library / generation sourcing, and the free stock-media catalog search (`vidfarm media` / GET /primitives/media/search) across pixabay, openverse, and iconify.
+---
+
+# vidfarm-media
+
+One engine and one rule. The engine: `scripts/audio.mjs` turns a neutral `audio_request.json` into narration files, word timings, a music bed, and SFX — written into the project and described by one meta JSON. The rule: **the user never signs up for a third-party media account.** Audio and images are the user's own files (My Files / raws library), generated on the user's provider keys, produced by keyless local models, OR generated through vidfarm's own ElevenLabs integration — music/TTS/STT on the platform key billed to the wallet (`use_wallet_credits`, the default) or on the customer's own ElevenLabs key. If a source would need the user to sign up somewhere else, it is not a source.
+
+## Preflight (run once, before the brief)
+
+Check which engines this run will use and say so in one line — then continue. There is nothing to sign in to and no reason to stop.
+
+- Saved keys: `vidfarm provider-keys`, or env `OPENAI_API_KEY` / `GEMINI_API_KEY` / `OPENROUTER_API_KEY`.
+- Key present → BYOK cloud speech: `vidfarm tts` (promptable voice via `--style`) and `vidfarm stt` (real word timestamps on an OpenAI whisper-1 key).
+- No key → keyless local engines: Kokoro-82M TTS (`npx hyperframes tts`) and whisper.cpp timings (`npx hyperframes transcribe`). They download model weights on first use and never need an account.
+- Don't write keys into a per-repo `.env`. Details: `references/tts.md`.
+
+## The audio engine
+
+```bash
+node <SKILL_DIR>/scripts/audio.mjs --request ./audio_request.json --hyperframes . --out ./audio_meta.json [--only tts,bgm,sfx]
+```
+
+Adapters in faceless-explainer and product-launch-video resolve this engine at `../../vidfarm-media/scripts/audio.mjs` (override with `$HF_MEDIA_ENGINE`); general-video calls it directly. Full schema lives in the script's header comment; the shape:
+
+```jsonc
+// audio_request.json — one line per narrated segment; `id` is the caller's join key
+{
+  "provider": "auto",                  // auto = keyless local Kokoro; openai/gemini/openrouter = vidfarm BYOK
+  "voice": "af_heart",                 // Kokoro id, or a cloud preset (alloy…, Kore…) — see references/tts.md
+  "speed": 1.0,                        // Kokoro only
+  "lines": [
+    { "id": "01", "text": "Opening line.", "sfx": ["whoosh"] },
+    { "id": "02", "sfx": ["click"] }   // sfx-only lines are how adapters fetch SFX later
+  ],
+  "bgm": { "mode": "retrieve", "query": "calm cinematic underscore" } // "none" disables; no generate mode exists
+}
+```
+
+```jsonc
+// audio_meta.json — id-keyed; every path RELATIVE to --hyperframes
+{
+  "voices": [{ "id": "01", "path": "assets/voice/01.wav", "duration_s": 4.21,
+               "words": [{ "id": "w0", "text": "Opening", "start": 0.02, "end": 0.31 }] }],
+  "bgm": { "path": "assets/bgm/track.mp3", "query": "…", "duration_s": 22.4 },   // or null + a note
+  "sfx": [{ "id": "01", "file": "assets/sfx/whoosh.mp3", "offset_s": 0, "duration_s": 0.9, "volume": 0.35 }],
+  "notes": ["bgm: skipped — …"]        // every graceful skip is explained here
+}
+```
+
+Contract points:
+
+- `--only` **merges**: sections not listed are preserved from the existing `--out` meta (generate `--only tts,bgm`, then `--only sfx` later — nothing is lost).
+- Word timings are real (local whisper.cpp) when possible, else estimated by proportional character weighting. Timings never fail the run.
+- BGM is **not** loop-extended by the engine — the workflow assemblers loop-extend short beds to the video length at mount time. `bgm_pending` is never set; `scripts/wait-bgm.mjs` exists as an exit-0 shim.
+- Mount the results per `hyperframes-core`: each `voices[].path` and (`bgm.path`, `sfx[]`) as `<audio>` tracks; `voices[].words` drive captions.
+- **Multi-track mixing is native — keep voice, music, and SFX on SEPARATE `<audio>` layers, each at its own `data-volume`** (do NOT pre-mix them into one file). The runtime and both render paths mix all audio layers together with per-track volume honored (a real ffmpeg `amix` at render). Standard balance: **narration/voice ~1.0, BGM ~0.1–0.2** under the voice, SFX per-item (the engine already emits `sfx[].volume`, e.g. `0.35`). This is the whole reason each stem is a distinct track — the user (or a later edit) can re-level any track independently.
+- **De-combining an original on recreate.** When you're recreating a video whose original had music + narration baked into ONE track, don't try to reproduce that single bed — you cannot stem-separate baked audio. Instead build the mix fresh: a new narration track (`tts` / `regenerate-speech`) at ~1.0 **plus** a real music file (owned / user-provided / found via `vidfarm directory search "<query> music"` / generated with `vidfarm music "<vibe>"`) at ~0.1–0.2, as two separate `<audio>` layers, and drop the original combined track. Never fake a music layer or duplicate the voice track to stand in for music.
+
+## Music, TTS & STT (ElevenLabs + BYOK)
+
+**Music and speech are core primitives — use them freely.** ElevenLabs powers high-quality
+music, narration, and transcription. The `use_wallet_credits` flag (DEFAULT **true** on the
+music / TTS / STT primitives) runs on vidfarm's platform ElevenLabs key and bills the customer's
+wallet. **Keep it true** for the best voices/music out of the box; only set it **false** to save
+wallet credits or to use the customer's OWN saved ElevenLabs key (Settings → AI keys → ElevenLabs
+is an audio-only key that does NOT count as a qualified text/image/video key). With
+`use_wallet_credits: false` and no ElevenLabs key, TTS/STT fall back to a BYOK openai/gemini/
+openrouter key; music always needs ElevenLabs (own key or platform).
+
+| Need | Command / route | Notes |
+| --- | --- | --- |
+| **Music** (bed, beat, jingle, song, score) | `vidfarm music "upbeat lo-fi beat" --length 30` · `POST /api/v1/primitives/music/generate` | ElevenLabs. `use_wallet_credits` default true (platform key + wallet); `--own-key` = your ElevenLabs key. `music_length_ms` ≤ 300000 (5 min). Place as its own `<audio>` layer ~0.1–0.2 under narration. |
+| **Narration** (default) | `vidfarm tts "…" --cloud` · `POST /api/v1/primitives/audio/speech` | Default = ElevenLabs on the platform key (wallet-billed). Pick a voice with `--voice <voice_id>` (browse below). `--own-key` for your ElevenLabs/BYOK key. Local-first `vidfarm tts` (no `--cloud`) still runs on your env openai/gemini key. |
+| **List voices** | `vidfarm voices` · `GET /api/v1/primitives/audio/voices` | ElevenLabs voice catalog (voice_id, name, labels, preview_url). Default = platform account; `--own-key` / `?use_wallet_credits=false` = the customer's ElevenLabs account. **Default a sensible voice AND tell the user they can pick from many.** |
+| Narration, zero keys | `npx hyperframes tts "…" -v af_heart --json` | Kokoro-82M, local, WAV + duration in JSON |
+| **Transcript + SRT** | `vidfarm stt <file\|url> --cloud` · `POST /api/v1/primitives/audio/transcribe` | Default = ElevenLabs Scribe (native diarization + real word timestamps), wallet-billed. `--own-key`/BYOK: gemini labels speakers, openai/whisper-1 gives real word timings. |
+| Reword existing narration in the (approximate) original voice | `POST /api/v1/primitives/audio/regenerate-speech` | Listens, profiles the speaker (needs a Gemini key), rewords, regenerates with the closest preset voice + matched style. Approximation, never a clone. Details: `references/tts.md` |
+| Word timings, zero keys | `npx hyperframes transcribe <audio> --json` | whisper.cpp; writes word-level `transcript.json` |
+| Captions straight into a composition | `vidfarm captions generate <dir>` | whisper-1 real timings when an OpenAI key exists; estimates otherwise |
+
+Voices, styles, formats, the ElevenLabs SDK details, and the exact word-timestamp paths:
+`references/tts.md`, plus the loadable `music` and `text-to-speech` skill packs for the raw
+ElevenLabs API.
+
+## BGM and SFX resolution order
+
+1. **User file / URL** — `bgm.file` / `bgm.url` in the request; copied or downloaded into `assets/bgm/`.
+2. **File directory** — `vidfarm directory search "<query> music"` or the legacy `vidfarm files --search` (needs `$VIDFARM_API_KEY`); the user's own library, searchable by meaning when notes are annotated.
+3. **Generate it** — `vidfarm music "<query>" --length <sec>` (ElevenLabs; `use_wallet_credits` default) writes an mp3 you can pass back as `bgm.file`. Great when the user has no fitting track. The engine script itself has no generate mode — run `vidfarm music` yourself and feed the result in.
+4. **Local dir** — `$MUSIC_DIR` / `$SFX_DIR` (or request `bgm.dir` / `sfx_dir`), picked by filename keyword match against the query.
+5. **Skip, with a note** — a silent bed or a missing whoosh is fine. The meta's `notes[]` says exactly what was skipped and why.
+
+## The file directory (My Files + raws, unified)
+
+Everything the user owns is one navigable virtual directory (source of truth: `src/services/file-directory.ts`). Three canonical roots, each with nested subfolders; paths are canonical, copyable, and the display/navigation layer — a file's real handle is still its backend id.
+
+| Root | What it holds | Subfolder convention | Lifecycle |
+| --- | --- | --- | --- |
+| `/files` | durable personal files (My Files attachments) | free-form | permanent; vector-searchable via `notes` |
+| `/temp` | scratch space | auto-organized into date folders `YYYY-MM-DD` | auto-deleted after 30 days |
+| `/raws` | reusable clip / source-footage library | by source, `/raws/<source-slug>/` | permanent |
+
+Example canonical path: `/raws/product-demos/clip.mp4`.
+
+**devcli** (needs `$VIDFARM_API_KEY`):
+
+```bash
+vidfarm directory ls [path]                    # browse a folder; path defaults to "/" (lists the three roots)
+vidfarm directory search "<query>" [--path <scope>] [--mode auto|semantic|substring|path] [--limit N]
+```
+
+**REST** (cookie/api-key authed, on the vidfarm server):
+
+- `GET /api/v1/user/me/directory?path=/raws/demos` → `{ path, root, folders[], files[], next_offset }` (path `/` lists the three roots).
+- `POST /api/v1/user/me/directory/search` `{ query, path?, mode?, limit? }` → `{ results[], query, mode, scope, semantic }`. `mode`: `auto` (default = semantic vector + substring + absolute-path, combined), `semantic`, `substring`, `path`.
+- `PATCH /raws/:clipId` `{ folder_path }` → move a raw between folders (`""` = raws root).
+
+Every list/search item is the canonical **DirectoryItem**:
+
+```jsonc
+{ "path": "/raws/demos/clip.mp4", "root": "raws", "folderPath": "/raws/demos",
+  "name": "clip.mp4", "kind": "file",          // or "folder"
+  "id": "clip_…", "contentType": "video/mp4",  // id/contentType present for files
+  "viewUrl": "https://…", "thumbUrl": "https://…",
+  "sizeBytes": 0, "durationSec": 0, "meta": {} }
+```
+
+Legacy commands still work and map onto the same directory: `vidfarm files --search` / `vidfarm put-file` / `vidfarm annotate-file <id> --notes "…"` (annotate so semantic search finds an asset later) live under `/files`; `vidfarm raws search` / `vidfarm raws scan <video>` populate and query `/raws`.
+
+## Sourcing images and footage
+
+Same ownership rule, resolved through the file directory:
+
+- **Search what the user has** — `vidfarm directory search "<meaning>"` (scope with `--path /raws` or `--path /files`); or the legacy `vidfarm files --search` / `vidfarm raws search`.
+- **Populate raws** — `vidfarm raws scan <video>` hunts reusable raws out of the user's long-form footage into `/raws/<source>/`.
+- **Generate on the user's keys** — `vidfarm generate image|video --prompt "…"` (BYOK primitives; `--place <dir>` drops the result straight into a composition).
+- **Search the free stock catalog** — `vidfarm media search "<meaning>" --type <image|vector|icon|video|bgm|sfx>` (shorthand: `vidfarm media icon "home"`). Returns royalty-free, commercial-safe results, each tagged with its license — honor `attribution` when `attribution_required` is true. Web/API equivalent: `GET /api/v1/primitives/media/search?type=&q=&limit=`.
+- Never scrape or hotlink arbitrary third-party pages; the catalog above is the sanctioned licensed source, and a capture workflow's own screenshots + the user's brand assets are always fair game.
+
+### Free media catalog — sources & keys
+
+Still no customer signup: the catalog runs on **keyless** sources and **platform-held** keys, never a per-user account.
+
+| Source | Serves | Key |
+| --- | --- | --- |
+| **iconify** | `icon` (SVG) | keyless — always on |
+| **openverse** | `image` (CC), `bgm`, `sfx` (CC/CC0 audio) | keyless — always on |
+| **pixabay** | `image`, `vector`, `video` (photos/illustrations/stock video) | needs a **platform** `PIXABAY_API_KEY` |
+
+**Recommend keys when a search comes up short.** If `image`/`vector`/`video`
+returns nothing (or `providers_used` omits `pixabay`), the platform's Pixabay key
+isn't set — tell the operator they can unlock stock photos/vectors/video with a
+**free** key from <https://pixabay.com/api/docs/> (set `PIXABAY_API_KEY` server-side;
+`OPENVERSE_API_TOKEN` is optional and only raises rate limits). Icons, CC images,
+and bgm/sfx keep working without any key. This is an operator/platform key — the
+end user still never signs up for anything.