@mevdragon/vidfarm-devcli 0.10.0 → 0.11.0

package/README.md

@@ -50,7 +50,7 @@ Everything runs locally (`RECORDS_DRIVER=local`, `STORAGE_DRIVER=local`), includ
 
 Flags: `--port` (default 3000), `--dir` (default `./.vidfarm-local`), `--key` (bootstrap/browser key, or `VIDFARM_API_KEY`), `--fork <id>`, `--no-open`. `vidfarm <template_id>` is an alias for `serve <template_id>`.
 
-Beyond the local editor, the devcli also wraps the **entire director REST flow** as 1:1 commands — `discover`, `inspiration-add`, `fork`, `decompose`, `snapshot`, `render`, `approve`, `schedule`, `posts`, `upload`/`download`, `clips` (see below), plus a raw `vidfarm api <METHOD> <path>` passthrough. Each maps to exactly one REST route and prints the prod frontend URL to open. Run `vidfarm --help` for the full surface, or see `SKILL.director.md` § "`vidfarm-devcli` — full command surface".
+Beyond the local editor, the devcli also wraps the **entire director REST flow** as 1:1 commands — `discover`, `inspiration-add`, `fork`, `decompose`, `snapshot`, `render`, `approve`, `schedule`, `posts`, `upload`/`download`, `clips` (see below), `tts`/`stt` (see below), plus a raw `vidfarm api <METHOD> <path>` passthrough. Each maps to exactly one REST route and prints the prod frontend URL to open. Run `vidfarm --help` for the full surface, or see `SKILL.director.md` § "`vidfarm-devcli` — full command surface".
 
 ## Clip hunting: `vidfarm clips`
 
@@ -70,6 +70,24 @@ vidfarm clips scan --cloud --url "https://youtube.com/watch?v=…" --duration 30
 
 Duration is a **soft range** (10→5–20s, 30→20–40s), `--range` means only those windows are decoded (the cost saver on long videos), and `--no-text` selects text-free scenes — caption *removal* (GhostCut) stays a per-finished-clip primitive, never a long-form pass. Run `vidfarm clips --help` for everything (`list`, `match`, `preset`, `sources`, `export`).
 
+## Speech: `vidfarm tts` / `vidfarm stt`
+
+Text-to-speech and speech-to-text, **local-first** like `clips`: both default to your own `OPENAI_API_KEY` / `GEMINI_API_KEY` / `OPENROUTER_API_KEY` (no cloud job, no wallet), with `--cloud` as the explicit backup that runs the platform primitive on your saved BYOK keys instead. (Speech can't ride the local `claude`/`codex` agent — no audio I/O — so the local path always needs a provider key.)
+
+```bash
+# text → narration audio; the VOICE STYLE is promptable
+vidfarm tts "Welcome back. Today we test the three most viral hooks." \
+  --style "energetic UGC creator, conversational" --voice coral --out narration.mp3
+
+# video or audio → transcript, BOTH formats in one pass:
+#   simple subtitles (transcript.txt + timed transcript.srt)
+#   + multi-speaker segments (transcript.json: {speaker, start_sec, end_sec, text})
+vidfarm stt ./podcast.mp4 --out transcript          # local ffmpeg demux + your key
+vidfarm stt "https://cdn.example.com/clip.mp4" --cloud   # backup: POST /api/v1/primitives/audio/transcribe
+```
+
+Speaker labels (diarization, on by default) need a Gemini key — OpenAI/OpenRouter keys degrade to a plain single-speaker transcript. The matching REST primitives are `POST /api/v1/primitives/audio/speech` and `/audio/transcribe` (aliases `/tts`, `/stt`).
+
 ### Crash telemetry & opt-out
 
 The devcli can report **unexpected crashes** (bugs — not bad args, HTTP 4xx, or network blips) to Sentry so they can be fixed. It is **off unless a dedicated CLI Sentry DSN is configured** (`VIDFARM_DEVCLI_SENTRY_DSN`), reports crash-only, and **scrubs home-dir paths, API keys, and secrets** before anything leaves your machine (PII off). To opt out entirely:

package/SKILL.director.md

@@ -502,6 +502,8 @@ Send a stable `tracer` on export so retries are traceable and filterable in job
 | `vidfarm fork <template_id>` | `POST /api/v1/compositions` | fork a template |
 | `vidfarm pull <forkId> [--dir <p>]` | `GET .../compositions/:forkId/{composition.html,json,video-context.json,cast.json}` | sync a fork to disk + print gaps/scene keys |
 | `vidfarm generate <image\|video> --prompt "…"` | `POST /api/v1/primitives/{images,videos}/generate` (polls job) | generate AI media → finished URL |
+| `vidfarm tts "…" [--style "…"] [--voice <v>] [--out <file>]` | (LOCAL-FIRST: your own OPENAI/GEMINI/OPENROUTER_API_KEY → audio file on disk; `--cloud` = `POST /api/v1/primitives/audio/speech` + poll) | text → narration audio with a promptable voice style |
+| `vidfarm stt <file\|url> [--out <base>] [--no-diarize]` (alias: `transcribe`) | (LOCAL-FIRST: local ffmpeg demux + your own key; `--cloud` = `POST /api/v1/primitives/audio/transcribe` + poll) | video/audio → transcript in BOTH formats: simple subtitles (txt + SRT) and multi-speaker segments (json) |
 | `vidfarm place <dir> --src <url\|file> [--at\|--replace]` | (edits local composition.html; local files → serve disk store or temp upload) | drop media (URL **or local file**) into a gap / over a scene |
 | `vidfarm decompose <forkId>` | `POST .../compositions/:forkId/auto-decompose` | split source into scenes |
 | `vidfarm remove-video-captions <forkId>` (alias: `ghostcut`) | `GET .../compositions/:forkId/remove-video-captions` | subtitle-removal status |
@@ -730,6 +732,70 @@ curl -X POST "$VIDFARM_BASE/api/v1/primitives/media/dedupe" \
   }'
 ```
 
+## Primitive: tts (text → speech)
+
+Generate spoken narration audio from text on the caller's saved AI provider keys (OpenAI, Gemini, or OpenRouter — BYOK, never wallet-billed). The **voice style is promptable**: `instructions` is a free-form direction for tone, pacing, accent, emotion, and persona, on top of the provider's `voice` preset. Result is a durable Vidfarm audio URL (mp3/wav) ready to drop into a composition as an audio layer.
+
+- `POST /api/v1/primitives/audio/speech` (alias: `POST /api/v1/primitives/tts`)
+- Body: `{ "tracer": "...", "payload": { "text": "...", "voice"?: "...", "instructions"?: "...", "provider"?: "openai" | "gemini" | "openrouter", "model"?: "...", "output_format"?: "mp3" | "wav" }, "webhook_url"?: "..." }`
+  - `text` (required, ≤8000 chars) — the words to speak, verbatim. Aliases: `input`, `script`.
+  - `voice` (optional) — provider voice preset (OpenAI: `alloy`, `ash`, `coral`, …; Gemini: `Kore`, `Puck`, `Charon`, …).
+  - `instructions` (optional) — the voice-STYLE prompt, e.g. `"calm, warm bedtime narrator with a slight British accent"` or `"excited sports announcer, fast paced"`. Aliases: `style`, `style_instructions`, `voice_style`.
+  - `output_format` defaults to `mp3` (note: Gemini answers in wav regardless — read `audio.content_type`).
+- Response: standard primitive job. Poll to completion, then read `primary_file_url` / `audio.file_url`.
+- Billing: BYOK — the speech call runs on the customer's own provider key and is never wallet-billed.
+- devcli: `vidfarm tts "…" --style "…"` runs LOCAL-FIRST on your own env key (no job, no cloud); add `--cloud` to run this route instead.
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/audio/speech" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{
+    "tracer": "demo-tts",
+    "payload": {
+      "text": "Welcome back. Today we are testing the three most viral hooks.",
+      "voice": "coral",
+      "instructions": "energetic UGC creator, conversational, slightly breathless"
+    }
+  }'
+```
+
+## Primitive: stt (video/audio → transcript)
+
+Transcribe speech from **any video or audio URL** on the caller's saved AI provider keys (video sources are demuxed to audio automatically via the platform ffmpeg seam). One job returns **BOTH formats**:
+
+1. **Simple subtitle version** — the plain transcript (`output.text`, `transcript.txt`) plus timed SRT cues (`output.subtitles.srt`, `transcript.srt`) with no speaker labels — drop-in captions.
+2. **Advanced multi-speaker version** — speaker-attributed, timestamped segments (`output.segments`, each `{ speaker, start_sec, end_sec, text }`, with `output.speakers` listing the cast) for multi-narration sources like podcasts, interviews, and skits — stored durably as `transcript.json`.
+
+- `POST /api/v1/primitives/audio/transcribe` (alias: `POST /api/v1/primitives/stt`)
+- Body: `{ "tracer": "...", "payload": { "source_url": "https://...", "diarize"?: true, "language"?: "en", "prompt"?: "...", "provider"?: "gemini" | "openai" | "openrouter", "model"?: "...", "media_type"?: "auto" | "video" | "audio" }, "webhook_url"?: "..." }`
+  - `source_url` (required) — video OR audio URL. Aliases: `source_video_url`, `source_audio_url`, `video_url`, `audio_url`, `url`. For a fork's composition, pick the right source via `GET /api/v1/compositions/:forkId/remove-video-captions` first.
+  - `diarize` (default `true`) — attribute segments to speakers. **Speaker labels need a Gemini key** (audio-native model); OpenAI/OpenRouter keys degrade to a plain single-speaker transcript (`diarization: "none"`).
+  - `prompt` (optional) — domain/vocabulary hint; `language` (optional) — expected language code.
+- Response: standard primitive job. Poll to completion; `output` carries both formats inline (very long transcripts set `text_truncated: true` and keep the full copy in the `transcript.json` artifact), and the artifacts `transcript.json` / `transcript.srt` / `transcript.txt` are durable URLs.
+- Limits: ~40 minutes of speech per job (20MB extracted audio) — trim longer sources with `/videos/trim` or `/audio/trim` and transcribe in parts.
+- Billing: the AI transcription is BYOK (never wallet-billed); only the tiny audio-demux compute step bills (`primitive_media_lambda`), like every media primitive.
+- For the CURRENT fork, the decompose-time transcript may already exist via `video-context.json` — prefer it before paying for a new transcription; use this primitive for other URLs, multi-speaker attribution, or SRT output.
+- devcli: `vidfarm stt <file|url>` runs LOCAL-FIRST (local ffmpeg + your own key); add `--cloud` to run this route instead.
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/audio/transcribe" \
+  -H "vidfarm-api-key: $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{
+    "tracer": "demo-stt",
+    "payload": {
+      "source_url": "https://cdn.example.com/podcast-clip.mp4",
+      "diarize": true,
+      "language": "en"
+    }
+  }'
+```
+
 ## Brainstorm primitives
 
 The `brainstorm/*` primitives are the strategy toolkit. They are reusable, billable AI reasoning steps — the same family the AI Copilot exposes as chip suggestions. Treat **product placement** as a first-class member of this family, right alongside angles and hooks:

package/SKILL.platform.md

@@ -264,8 +264,9 @@ Primitives are hosted operations that are NOT templates — they're reusable bui
 - `download_media_video` — social/media URL → durable MP4
 - `hyperframes_render` — composition HTML → MP4 (the render primitive publishing uses)
 - `brainstorm` — LLM brainstorm route
+- `tts` / `stt` — speech, BYOK on the caller's saved provider keys. `POST /api/v1/primitives/audio/speech` (`primitive:tts`) turns text into narration audio with a promptable voice style (`instructions`); `POST /api/v1/primitives/audio/transcribe` (`primitive:stt`) transcribes any video or audio URL (video demuxes through the primitive-media ffmpeg seam first, 64kbps mono, ~40min/20MB cap) and returns **two formats**: a simple subtitle transcript (plain text + SRT cues) and an advanced multi-speaker version (speaker-attributed timed segments; diarization is Gemini-only — JSON-mode audio transcription in `src/services/speech.ts`). The raw provider speech calls live in `src/services/speech.ts`, shared by `ProviderService` (leased BYOK keys, usage metering) and the devcli `vidfarm tts`/`vidfarm stt` commands, which run LOCAL-FIRST on the user's own env key (local ffmpeg demux, no job, no wallet) with `--cloud` as the explicit backup route.
 
-Directors call primitives when they want a one-off asset before feeding it into a template. Templates call primitives internally as workflow steps. All primitives are metered against the caller's wallet.
+Directors call primitives when they want a one-off asset before feeding it into a template. Templates call primitives internally as workflow steps. All primitives are metered against the caller's wallet (BYOK AI calls excepted — e.g. tts/stt model spend rides the caller's own provider key; only compute like the stt demux step wallet-bills).
 
 ## Billing
 

package/dist/src/app.js

@@ -3089,6 +3089,8 @@ function buildPrimitiveEditorDocsRoutes() {
         { method: "POST", path: `${PRIMITIVES_PREFIX}/audio/probe`, summary: "Primitive audio probe job. Body must be { tracer, payload: { source_audio_url }, webhook_url? }. Returns durable JSON metadata." },
         { method: "POST", path: `${PRIMITIVES_PREFIX}/audio/concat`, summary: "Primitive audio concat job. Body must be { tracer, payload: { source_audio_urls, output_format?, audio_bitrate_kbps? }, webhook_url? }." },
         { method: "POST", path: `${PRIMITIVES_PREFIX}/audio/normalize`, summary: "Primitive audio normalization job. Body must be { tracer, payload: { source_audio_url, output_format?, audio_bitrate_kbps?, sample_rate_hz? }, webhook_url? }." },
+        { method: "POST", path: `${PRIMITIVES_PREFIX}/audio/speech`, summary: "Primitive text-to-speech (TTS) job using saved OpenAI, Gemini, or OpenRouter provider keys. Body must be { tracer, payload: { text, voice?, instructions?, provider?, model?, output_format? }, webhook_url? }. instructions is a free-form voice-style prompt (tone, pacing, accent, emotion, persona — e.g. \"calm, warm bedtime narrator with a slight British accent\"). Returns a durable platform audio URL (mp3 or wav). Alias: POST /api/v1/primitives/tts." },
+        { method: "POST", path: `${PRIMITIVES_PREFIX}/audio/transcribe`, summary: "Primitive speech-to-text (STT) job using saved provider keys. Accepts VIDEO or AUDIO sources; video audio is demuxed automatically. Body must be { tracer, payload: { source_url (aliases: source_video_url, source_audio_url, url), diarize?, language?, prompt?, provider?, model? }, webhook_url? }. Returns TWO formats in one job: a simple subtitle transcript (plain text + timed SRT cues, no speakers) and an advanced multi-speaker version (speaker-attributed timed segments for multi-narration), plus transcript.json/.srt/.txt artifacts. diarize defaults to true; speaker attribution needs a Gemini key (openai/openrouter degrade to a plain single-speaker transcript). Sources over ~40 minutes of speech must be trimmed first. Alias: POST /api/v1/primitives/stt." },
         { method: "POST", path: `${PRIMITIVES_PREFIX}/brainstorm/coldstart`, summary: "Primitive cold-start strategy questionnaire. Body should usually be { tracer, payload: { user_message }, webhook_url? }. user_message can be rough, like \"I don't know where to start\" or a messy description. count is optional only when the user explicitly wants more or fewer questions. offer_description remains accepted as a backward-compatible alias." },
         { method: "POST", path: `${PRIMITIVES_PREFIX}/brainstorm/awareness_stages`, summary: "Primitive awareness-stage recommendation. Body must be { tracer, payload: { offer_description }, webhook_url? }. Use this when the customer is unsure what type of ads to produce." },
         { method: "POST", path: `${PRIMITIVES_PREFIX}/brainstorm/angles`, summary: "Primitive ad-angle brainstorm. Body must be { tracer, payload: { offer_description, problem_awareness, solution_awareness }, webhook_url? }. Allowed values: problem_awareness is problem_unaware|problem_aware, solution_awareness is solution_unaware|solution_aware. count is optional only when the user explicitly wants more or fewer angles. Returns diverse persuasive TikTok angles with explanations." },
@@ -3682,7 +3684,7 @@ function normalizeOperationRequestTracer(tracer) {
 }
 function createTemplateHttpTool(input) {
     return tool({
-        description: "Call exactly one Vidfarm REST API route and inspect the response before deciding the next call. Use template routes for metadata, skill, config, operations, jobs, logs, and cancel routes. Use primitive routes under /api/v1/primitives for reusable image generation, image editing, inpainting, still rendering, social/media video download, AI video generation, video slide rendering, burned-in caption removal, video trim/extract/probe/mute/concat operations, audio trim/probe/concat/normalize operations, brainstorm strategy routes, and primitive job inspection; these primitive routes are allowed directly from editor chat even when they are not specific to the current template. Primitive POST bodies must use { tracer, payload: { ...primitive fields... }, webhook_url? }; do not put prompt, source_image_url, instruction, slides, html, or other primitive fields at the top level. Never say primitive image generation, image edit, inpaint, HTML render, video download, AI video generation, caption removal, trim, probe, extract-audio, concat, mute, video render, or brainstorm routes are unavailable here when the route exists. If the user asks to generate a new image similar to an attachment, call POST /api/v1/primitives/images/generate with body { tracer, payload: { prompt, prompt_attachments: [exact attachment URL] } }. If the user asks to revise the attached image itself, call POST /api/v1/primitives/images/edit with body { tracer, payload: { source_image_url, instruction } }. If the user asks to download a social/media post into a durable MP4, call POST /api/v1/primitives/videos/download with body { tracer, payload: { source_url|video_url|url, quality?: \"best\"|\"hd\"|\"full_hd\" } }. If the user asks to generate a new AI video, call POST /api/v1/primitives/videos/generate with body { tracer, payload: { prompt, provider?: \"openai\"|\"gemini\"|\"openrouter\", duration?: seconds, input_references?: [direct image asset URL], frame_images?: [{ image_url, frame_type }] } }; never send webpage or social post URLs in input_references. duration is seconds, not milliseconds, and duration_ms belongs only to /videos/render-slides slides. Default video models are openai/sora-2, gemini/veo-3.0-generate-001, and openrouter/bytedance/seedance-2.0. Use POST /api/v1/primitives/videos/render-slides only to assemble existing media URLs into an MP4 slideshow. Use POST /api/v1/primitives/media/dedupe to apply subtle dedupe transforms (zoom/tilt/rotate/saturation/speed/horizontal_flip/contrast/brightness/hue_rotate/blur/tint) to any image or video URL, with body { tracer, payload: { source_media_url, media_type?, effects?, tint_color?, tint_opacity?, width?, height?, duration_ms?, object_fit?, background_color?, output_format? }, webhook_url? }. Use POST /api/v1/primitives/videos/trim for clipping, /videos/extract-audio for demuxing, /videos/probe or /audio/probe for metadata-only JSON outputs, /videos/extract-frame for stills, /videos/mute to remove audio, /videos/replace-audio to swap tracks, /videos/concat for sequential muted MP4 joins, /audio/trim for clipped audio, /audio/concat for stitched audio, and /audio/normalize for leveled/re-encoded audio. If the user asks to remove burned-in captions, subtitles, or on-screen text from a video, call POST /api/v1/primitives/videos/remove-captions with body { tracer, payload: { source_video_url } } — an async job that returns a durable caption-free MP4 and bills about $0.10 per 30 seconds of source video. Caption removal is capped at ~15 minutes of source and must NEVER be used on long-form video: when the user wants short clips out of a long video (a podcast, stream VOD, webinar, or any YouTube/TikTok/IG/X URL), call POST /clips/scan instead with body { source_url | temp_file_id | s3_key, prompt, ranges?: [\"MM:SS-MM:SS\"], target_duration_sec?, aspect?: \"9:16\"|\"16:9\"|\"4:3\"|\"1:1\", avoid_text?, tracer? } — target_duration_sec is a SOFT range (30 → 20-40s), ranges restrict the hunt to those source windows (big cost saver on long videos), aspect crops every clip, and avoid_text prefers scenes WITHOUT on-screen text via scene SELECTION (never caption removal on the source). The hunt is async: it returns scan_id immediately; poll GET /clips/scan/:scanId, then browse results via GET /clips/feed?source=<source_video_id> or POST /clips/search. Hunts bill AWS compute only — the AI tagging runs on the user's own saved provider keys. To remove captions from a finished HUNTED clip, pass that clip's short mp4 to /videos/remove-captions afterwards. If the user explicitly asks for hooks, angles, awareness-stage advice, or a cold-start strategy questionnaire, call the matching brainstorm route immediately instead of brainstorming in chat from memory. Use POST /api/v1/primitives/brainstorm/coldstart when the customer is starting from zero, /brainstorm/awareness_stages when they do not know what ad types to make, /brainstorm/angles when they want persuasive angles, and /brainstorm/hooks when they want many openings. Product placement is a first-class brainstorm primitive like angles and hooks: when the user wants to identify product-placement opportunities in a video or repurpose a video into a native ad for their product, call POST /api/v1/primitives/brainstorm/product_placement with body { tracer, payload: { source_video_url, offer_description }, webhook_url? } — it watches the video and returns timestamped native placement opportunities and prefers a Gemini key. You may also analyze an attached video directly to answer product-placement questions conversationally instead of the primitive when the user just wants a quick take. For brainstorm routes, do not send count by default; only include count when the user explicitly asks for a specific number of hooks, angles, questions, or opportunities. When brainstorm jobs finish and the response includes a JSON artifact, tell the user to open and review that JSON in the chat UI because it contains the structured output. When the user asks for content ideas or says they do not know where to start, prefer the sequence coldstart -> awareness_stages -> angles -> hooks, then use the output to choose templates. For template operation POST bodies, use top-level template_preview_media=\"auto\" by default, \"include\" when the user wants the template to use catalog preview media as references where supported, and \"exclude\" when the user asks not to use preview media as references. Templates decide which generation steps honor that policy. If a POST returns a queued job, do not poll it in this same response; continue with any remaining requested POST calls, then summarize the queued job ids and tracers. If you know a job_id but not the template or primitive, use GET /api/v1/user/me/jobs/:jobId or its /logs and /artifacts subroutes instead of paging through job lists. Use GET /api/v1/compositions/:forkId/remove-video-captions to look up both the original source video URL and the subtitle-removed mirrored URL for the current fork before feeding a video URL into a primitive route; this is a read-only, non-billing status check — never confuse it with POST /remove-video-captions-poll (which advances the fork's decompose-time subtitle-removal job and can bill). Use POST /api/v1/approved/posts with title, caption, pinned_comment, media URLs, and tracer to approve a finished output into a preview/share page. Use POST /api/v1/approved/posts/:postId/schedules with destination_type, destination_id, scheduled_at, timezone, and optional settings/additional_notes to schedule an approved post. When a rendered video was built from slideshow frame images, include both versions in one media array: final MP4 first with kind=\"video\" and role=\"primary\", then every finished captioned slide frame image in order with kind=\"image\" and role=\"slide\". Prefer create-slideshow result.renderVideoInput.slides[].imageUrl or result.slides[].frameImageUrl for carousel media; do not use backgroundImageUrl, manifests, thumbnails, or a partial files array as the full slide set. If the user supplies exact media file URLs for approval, use those URLs as the backup source of truth, preserve their order, infer kind when needed, and assign primary/video and slide/image roles correctly. If the user asks for N distinct generated outputs, make N sequential POST calls rather than one combined POST and do not stop after the first queued job.",
+        description: "Call exactly one Vidfarm REST API route and inspect the response before deciding the next call. Use template routes for metadata, skill, config, operations, jobs, logs, and cancel routes. Use primitive routes under /api/v1/primitives for reusable image generation, image editing, inpainting, still rendering, social/media video download, AI video generation, video slide rendering, burned-in caption removal, video trim/extract/probe/mute/concat operations, audio trim/probe/concat/normalize operations, text-to-speech narration (audio/speech) and speech-to-text transcription (audio/transcribe) routes, brainstorm strategy routes, and primitive job inspection; these primitive routes are allowed directly from editor chat even when they are not specific to the current template. Primitive POST bodies must use { tracer, payload: { ...primitive fields... }, webhook_url? }; do not put prompt, source_image_url, instruction, slides, html, or other primitive fields at the top level. Never say primitive image generation, image edit, inpaint, HTML render, video download, AI video generation, caption removal, trim, probe, extract-audio, concat, mute, video render, text-to-speech, speech transcription, or brainstorm routes are unavailable here when the route exists. If the user asks to generate a new image similar to an attachment, call POST /api/v1/primitives/images/generate with body { tracer, payload: { prompt, prompt_attachments: [exact attachment URL] } }. If the user asks to revise the attached image itself, call POST /api/v1/primitives/images/edit with body { tracer, payload: { source_image_url, instruction } }. If the user asks to download a social/media post into a durable MP4, call POST /api/v1/primitives/videos/download with body { tracer, payload: { source_url|video_url|url, quality?: \"best\"|\"hd\"|\"full_hd\" } }. If the user asks to generate a new AI video, call POST /api/v1/primitives/videos/generate with body { tracer, payload: { prompt, provider?: \"openai\"|\"gemini\"|\"openrouter\", duration?: seconds, input_references?: [direct image asset URL], frame_images?: [{ image_url, frame_type }] } }; never send webpage or social post URLs in input_references. duration is seconds, not milliseconds, and duration_ms belongs only to /videos/render-slides slides. Default video models are openai/sora-2, gemini/veo-3.0-generate-001, and openrouter/bytedance/seedance-2.0. Use POST /api/v1/primitives/videos/render-slides only to assemble existing media URLs into an MP4 slideshow. Use POST /api/v1/primitives/media/dedupe to apply subtle dedupe transforms (zoom/tilt/rotate/saturation/speed/horizontal_flip/contrast/brightness/hue_rotate/blur/tint) to any image or video URL, with body { tracer, payload: { source_media_url, media_type?, effects?, tint_color?, tint_opacity?, width?, height?, duration_ms?, object_fit?, background_color?, output_format? }, webhook_url? }. Use POST /api/v1/primitives/videos/trim for clipping, /videos/extract-audio for demuxing, /videos/probe or /audio/probe for metadata-only JSON outputs, /videos/extract-frame for stills, /videos/mute to remove audio, /videos/replace-audio to swap tracks, /videos/concat for sequential muted MP4 joins, /audio/trim for clipped audio, /audio/concat for stitched audio, and /audio/normalize for leveled/re-encoded audio. If the user asks for AI voiceover, narration, or dub audio from text (a script, hook, or caption read aloud), call POST /api/v1/primitives/audio/speech with body { tracer, payload: { text, voice?, instructions? } } — instructions is a free-form voice-style prompt like \"excited sports announcer, fast paced\" and the job returns a durable audio URL you can place with add_layer. If the user asks to transcribe a video or audio, get subtitles/captions from speech, or identify who says what, call POST /api/v1/primitives/audio/transcribe with body { tracer, payload: { source_url, diarize?, language? } } — it accepts video or audio URLs (video audio is demuxed automatically) and one job returns BOTH a simple subtitle transcript (plain text + timed SRT cues) and an advanced multi-speaker version (speaker-attributed timed segments for multi-narration); speaker attribution needs a Gemini key. If the user asks to remove burned-in captions, subtitles, or on-screen text from a video, call POST /api/v1/primitives/videos/remove-captions with body { tracer, payload: { source_video_url } } — an async job that returns a durable caption-free MP4 and bills about $0.10 per 30 seconds of source video. Caption removal is capped at ~15 minutes of source and must NEVER be used on long-form video: when the user wants short clips out of a long video (a podcast, stream VOD, webinar, or any YouTube/TikTok/IG/X URL), call POST /clips/scan instead with body { source_url | temp_file_id | s3_key, prompt, ranges?: [\"MM:SS-MM:SS\"], target_duration_sec?, aspect?: \"9:16\"|\"16:9\"|\"4:3\"|\"1:1\", avoid_text?, tracer? } — target_duration_sec is a SOFT range (30 → 20-40s), ranges restrict the hunt to those source windows (big cost saver on long videos), aspect crops every clip, and avoid_text prefers scenes WITHOUT on-screen text via scene SELECTION (never caption removal on the source). The hunt is async: it returns scan_id immediately; poll GET /clips/scan/:scanId, then browse results via GET /clips/feed?source=<source_video_id> or POST /clips/search. Hunts bill AWS compute only — the AI tagging runs on the user's own saved provider keys. To remove captions from a finished HUNTED clip, pass that clip's short mp4 to /videos/remove-captions afterwards. If the user explicitly asks for hooks, angles, awareness-stage advice, or a cold-start strategy questionnaire, call the matching brainstorm route immediately instead of brainstorming in chat from memory. Use POST /api/v1/primitives/brainstorm/coldstart when the customer is starting from zero, /brainstorm/awareness_stages when they do not know what ad types to make, /brainstorm/angles when they want persuasive angles, and /brainstorm/hooks when they want many openings. Product placement is a first-class brainstorm primitive like angles and hooks: when the user wants to identify product-placement opportunities in a video or repurpose a video into a native ad for their product, call POST /api/v1/primitives/brainstorm/product_placement with body { tracer, payload: { source_video_url, offer_description }, webhook_url? } — it watches the video and returns timestamped native placement opportunities and prefers a Gemini key. You may also analyze an attached video directly to answer product-placement questions conversationally instead of the primitive when the user just wants a quick take. For brainstorm routes, do not send count by default; only include count when the user explicitly asks for a specific number of hooks, angles, questions, or opportunities. When brainstorm jobs finish and the response includes a JSON artifact, tell the user to open and review that JSON in the chat UI because it contains the structured output. When the user asks for content ideas or says they do not know where to start, prefer the sequence coldstart -> awareness_stages -> angles -> hooks, then use the output to choose templates. For template operation POST bodies, use top-level template_preview_media=\"auto\" by default, \"include\" when the user wants the template to use catalog preview media as references where supported, and \"exclude\" when the user asks not to use preview media as references. Templates decide which generation steps honor that policy. If a POST returns a queued job, do not poll it in this same response; continue with any remaining requested POST calls, then summarize the queued job ids and tracers. If you know a job_id but not the template or primitive, use GET /api/v1/user/me/jobs/:jobId or its /logs and /artifacts subroutes instead of paging through job lists. Use GET /api/v1/compositions/:forkId/remove-video-captions to look up both the original source video URL and the subtitle-removed mirrored URL for the current fork before feeding a video URL into a primitive route; this is a read-only, non-billing status check — never confuse it with POST /remove-video-captions-poll (which advances the fork's decompose-time subtitle-removal job and can bill). Use POST /api/v1/approved/posts with title, caption, pinned_comment, media URLs, and tracer to approve a finished output into a preview/share page. Use POST /api/v1/approved/posts/:postId/schedules with destination_type, destination_id, scheduled_at, timezone, and optional settings/additional_notes to schedule an approved post. When a rendered video was built from slideshow frame images, include both versions in one media array: final MP4 first with kind=\"video\" and role=\"primary\", then every finished captioned slide frame image in order with kind=\"image\" and role=\"slide\". Prefer create-slideshow result.renderVideoInput.slides[].imageUrl or result.slides[].frameImageUrl for carousel media; do not use backgroundImageUrl, manifests, thumbnails, or a partial files array as the full slide set. If the user supplies exact media file URLs for approval, use those URLs as the backup source of truth, preserve their order, infer kind when needed, and assign primary/video and slide/image roles correctly. If the user asks for N distinct generated outputs, make N sequential POST calls rather than one combined POST and do not stop after the first queued job.",
         inputSchema: z.object({
             method: z.enum(["GET", "POST"]).describe("HTTP method to use."),
             path: z.string().min(1).describe("Absolute path or same-origin URL for an allowed Vidfarm REST route."),
@@ -13773,7 +13775,9 @@ const AI_PRIMITIVE_IDS = new Set([
     "primitive:image_generate",
     "primitive:image_edit",
     "primitive:image_inpaint",
-    "primitive:video_generate"
+    "primitive:video_generate",
+    "primitive:tts",
+    "primitive:stt"
 ]);
 async function preflightAiProviderKeys(input) {
     if (!operationMayUseAiProvider(input)) {
@@ -13978,6 +13982,14 @@ async function createPrimitiveJob(c, input) {
         }
         throw error;
     }
+    if (!config.VIDFARM_JOB_STATE_MACHINE_ARN) {
+        // `vidfarm serve` boxes have no Step Functions worker — run the job
+        // in-process (detached, client keeps polling) instead of letting it sit
+        // queued forever, mirroring the local render path.
+        void runPrimitiveJobInProcess(job.id).catch((error) => {
+            console.error("local primitive job failed", job.id, error instanceof Error ? error.message : error);
+        });
+    }
     return c.json({
         job_id: job.id,
         tracer: job.tracer,
@@ -14282,6 +14294,12 @@ app.post(`${PRIMITIVES_PREFIX}/audio/trim`, async (c) => createPrimitiveJob(c, {
 app.post(`${PRIMITIVES_PREFIX}/audio/probe`, async (c) => createPrimitiveJob(c, { primitiveId: "primitive:audio_probe", operationName: "run" }));
 app.post(`${PRIMITIVES_PREFIX}/audio/concat`, async (c) => createPrimitiveJob(c, { primitiveId: "primitive:audio_concat", operationName: "run" }));
 app.post(`${PRIMITIVES_PREFIX}/audio/normalize`, async (c) => createPrimitiveJob(c, { primitiveId: "primitive:audio_normalize", operationName: "run" }));
+// Speech primitives (TTS/STT). Canonical paths follow the audio/* convention;
+// the flat /tts and /stt aliases keep them reachable under their plain names.
+app.post(`${PRIMITIVES_PREFIX}/audio/speech`, async (c) => createPrimitiveJob(c, { primitiveId: "primitive:tts", operationName: "run" }));
+app.post(`${PRIMITIVES_PREFIX}/tts`, async (c) => createPrimitiveJob(c, { primitiveId: "primitive:tts", operationName: "run" }));
+app.post(`${PRIMITIVES_PREFIX}/audio/transcribe`, async (c) => createPrimitiveJob(c, { primitiveId: "primitive:stt", operationName: "run" }));
+app.post(`${PRIMITIVES_PREFIX}/stt`, async (c) => createPrimitiveJob(c, { primitiveId: "primitive:stt", operationName: "run" }));
 app.post(`${PRIMITIVES_PREFIX}/brainstorm/hooks`, async (c) => createPrimitiveJob(c, { primitiveId: "primitive:brainstorm_hooks", operationName: "run" }));
 app.post(`${PRIMITIVES_PREFIX}/brainstorm/awareness_stages`, async (c) => createPrimitiveJob(c, { primitiveId: "primitive:brainstorm_awareness_stages", operationName: "run" }));
 app.post(`${PRIMITIVES_PREFIX}/brainstorm/angles`, async (c) => createPrimitiveJob(c, { primitiveId: "primitive:brainstorm_angles", operationName: "run" }));