reelkit-cli 0.1.3 → 0.2.0

package/README.md

@@ -59,6 +59,7 @@ reelkit render
 | `reelkit assets voices` | List narration voices |
 | `reelkit assets voiceover` | Record narration from `plan.json` |
 | `reelkit assets gen image` | Generate a scene's illustration |
+| `reelkit assets gen clip` | Generate a scene's video clip, or a green-screen one keyed to a transparent video |
 | `reelkit plan check` | Validate `plan.json` |
 | `reelkit check` | Check the composition without rendering |
 | `reelkit preview` | Two test frames per scene |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "reelkit-cli",
-  "version": "0.1.3",
+  "version": "0.2.0",
   "description": "CLI and Claude skill for making short-form video with a shared asset library.",
   "license": "MIT",
   "author": "Daniel Livshin",

package/skill/SKILL.md

@@ -29,7 +29,7 @@ Add `--footage` for a video the motion design should be laid over. User files st
 Read `reference/styles.md` and agree the video's look with the user: one of its looks, or their own. If they already described what they want, match it and confirm in one sentence. Ask anything still open in one message, each question with a default. If any on-screen text will be Hebrew, read `reference/hebrew-rtl.md` now: it changes how words may enter and how lines are written.
 
 ### 3. Plan
-Read `reference/scriptwriting.md` and `reference/scene-treatments.md`. Write the chosen look into the first scene's `notes`. Run `reelkit assets voices` and choose a voice that fits the idea, audience and language.
+Read `reference/scriptwriting.md` and `reference/scene-treatments.md` (and `reference/clips.md` if any scene might be a video clip). Write the chosen look into the first scene's `notes`. Run `reelkit assets voices` and choose a voice that fits the idea, audience and language.
 
 Write `plan.json`:
 
@@ -47,6 +47,7 @@ Write `plan.json`:
       "treatment": "motion-graphic",
       "onScreenText": ["up to three short items"],
       "imagePrompt": null,
+      "clipPrompt": null,
       "shareable": false,
       "imageTags": [],
       "userAssetIds": [],
@@ -57,8 +58,9 @@ Write `plan.json`:
 ```
 
 - 3 to 8 scenes. Scene ids are short, unique, lowercase with dashes.
-- `treatment` is `motion-graphic`, `illustration` or `footage-overlay`. With footage, `mode` is `"footage"` and every scene is `footage-overlay`; otherwise `mode` is `"motion"` and no scene is.
+- `treatment` is `motion-graphic`, `illustration`, `clip` or `footage-overlay`. With footage, `mode` is `"footage"` and every scene is `footage-overlay`; otherwise `mode` is `"motion"` and no scene is.
 - `imagePrompt` is set only for `illustration` scenes, with 3 to 6 `imageTags`. Set `shareable` to true only when the prompt is fully generic: no brand, product, person or detail specific to this user.
+- `clipPrompt` is set only for `clip` scenes (a generated or reused video clip is the scene's picture; the rest of the scene uses `imageTags` and `shareable` as an illustration does). Clips are scarce: most videos have none or one or two.
 - `userAssetIds` lists the ids of the user's files shown in that scene.
 - `pace` is `slow`, `normal` or `fast`.
 
@@ -69,11 +71,13 @@ Run `reelkit plan check`; it prints the estimated length to tell the user. Fix e
 ### 4. Voice
 `reelkit assets voiceover --all`. It records each scene and prints the real length of the video. If the user wants a different voice, change `voiceId` in `plan.json` and run it again with `--redo`.
 
-### 5. Images
+### 5. Images and clips
 Read `reference/asset-reuse.md`. For each illustration scene, search first:
 `reelkit assets search "<what the scene needs>" --kind image`
 Each result starts with a match percentage: how likely it is good enough to reuse. Pull the best result (`reelkit assets pull <id> --scene <sceneId>`) when its match is 60% or more and, reading its description, it fits the scene. Otherwise generate: `reelkit assets gen image --scene <sceneId>`. Never give two scenes the same image.
 
+For each clip scene, read `reference/clips.md`, then search `reelkit assets search "<what the clip shows>" --kind clip` and pull a 60% match (`reelkit assets pull <id> --scene <sceneId>`), or generate: `reelkit assets gen clip --scene <sceneId>` (add `--green` for a green-screen subject). It waits for the clip, which takes minutes; check what is left with `reelkit whoami`.
+
 ### 6. Composition
 Read `reference/kit.md`, `reference/remotion-composition.md`, `reference/motion-design.md` and `reference/captions.md`.
 

package/skill/reference/clips.md

@@ -0,0 +1,58 @@
+---
+name: clips
+description: Use when a scene might be a generated or reused video clip - B-roll that shows what the narration says, or a green-screen subject placed over your own graphics - including how to write the prompt, what a clip costs and how to layer it.
+---
+
+# Video clips
+
+A clip is a few seconds of generated video. It is the most expensive thing Reelkit makes: it takes minutes and each one uses part of a small monthly allowance. Use it where nothing else carries the point.
+
+## Is a clip worth it?
+- Use a clip when the sentence is about something that moves and cannot be drawn well in code or as one still: a person doing something, an animal, weather, a crowd, water, a place seen from a moving camera.
+- Do not use a clip for numbers, lists, steps, UI or anything with text (motion graphics do that better and cost nothing), nor for a concept one illustration shows (a still costs far less).
+- Most videos need none, or one or two. Plan a clip scene with `"treatment": "clip"` and a `clipPrompt`; give it `imageTags` (3 to 6) like an illustration, and set `shareable` by the same rule as images.
+- Check what is left before planning more: `reelkit whoami` shows `Clips: used/limit`, and when it resets. Clips are limited per month. If the allowance is gone, say so to the user and use an illustration or a motion graphic instead; do not work around it.
+
+## Search first
+`reelkit assets search "<what the clip shows>" --kind clip`. Each result starts with a match percentage. Pull the best one (`reelkit assets pull <id> --scene <sceneId>`) when it is 60% or more and its description fits the sentence; it is free and instant. Otherwise generate: `reelkit assets gen clip --scene <sceneId>`. Never give two scenes the same clip.
+
+## Writing a clip prompt
+- One subject, one action. "A fisherman pulls a net from a grey sea" is a clip; "a fisherman, his family at home, and the market" is three.
+- Say the camera: "slow push-in", "handheld tracking shot", "static wide shot", "slow pan left".
+- Say the light and the mood: "low golden sun", "overcast, soft light", "neon night".
+- No text, letters, logos, signs or screens with content in the picture: the generator draws them wrong. Add the words yourself in the composition.
+- It is 5 seconds by default (`--seconds 10` doubles the cost to the allowance only when the action truly needs it). Describe what happens in 5 seconds, not a story.
+- The scene runs as long as its narration. A clip shorter than the scene holds its last frame, so end the action on a moment that can sit still.
+- Generation takes up to several minutes. The command waits and prints a line every 30 seconds. If it gives up after ten minutes it prints the id and a `reelkit assets gen clip --resume <id> --scene <sceneId>` line: run it; it does not charge again.
+
+## B-roll rules
+- The clip shows exactly what the sentence says, at the moment it says it. If the sentence is abstract, the clip is the wrong tool.
+- Never the same clip twice in a video, and no two clip scenes in a row that look alike.
+- Speech never runs more than 3 seconds without something changing on screen: a new clip, a new text beat, a cut to a graphic. A 5-second clip under a 9-second scene needs a text change or a second visual in the middle.
+- Text over a clip must stay legible. Use `<ClipLayer src={urls[s.clipKey]} dim={0.25} />` to darken it, keep headlines in the calm part of the picture, and look at the preview frames for contrast on both the early and the late frame.
+
+## Green screen
+A green-screen clip is a subject filmed on a flat pure green background. It is for a person or an object that you place over something of your own: a presenter pointing at the user's screenshot, a character beside your title, a product over a gradient.
+- Generate it with `reelkit assets gen clip --scene <sceneId> --green`. The original is saved as `assets/clip-<sceneId>.mp4` and the green is keyed out locally into `assets/clip-<sceneId>.webm`, a video with a transparent background. The manifest has both: `s.clipKey` and `s.clipKeyedKey`. Use the `.webm`.
+- Prompt it as a subject, not a scene: "A woman in a blue jacket, waist up, waves and then points to her left. Flat, evenly lit, pure green background (#00FF00)." Keep the whole subject inside the frame with room around it; no green anywhere on the subject (clothes, props, eyes); no shadows, floor or reflections on the background; no camera move, because keying a moving frame edge flickers.
+- If the keyed edges look green or ragged in the preview, the prompt was at fault: regenerate once with a flatter green and an evenly lit subject, using `--redo`. A clip left unkeyed (the command says so) is keyed again for free by running the same command again.
+- Layer it above the scene's background and below the captions:
+
+```tsx
+import { Img } from "remotion";
+import { Captions, KeyedClip, SceneFrame } from "reelkit/kit";
+
+<SceneFrame from={s.startFrame} durationInFrames={s.durationFrames}>
+  <Img src={urls[s.userAssetKeys[0]]} style={{ width: "100%", height: "100%", objectFit: "cover" }} />
+  {s.clipKeyedKey ? <KeyedClip src={urls[s.clipKeyedKey]} x={0.72} y={1} scale={0.7} /> : null}
+  <Captions words={s.words} />
+</SceneFrame>
+```
+
+- A green-screen scene still has a `clip` treatment and a `clipPrompt`, with the background and the type in the composition. A library clip with the same use comes keyed too: `reelkit assets pull <id> --scene <sceneId>` writes the `.webm` beside the `.mp4` when the clip is marked green screen.
+- It costs the same as any clip and is harder to get right: at most one or two green-screen clips per video.
+
+## Rules
+- Search before generating, and generate once per scene. Do not regenerate to chase small improvements.
+- `--share` only for a generic clip with nothing specific to this user; the user's own files stay private.
+- Never claim a clip shows something it does not: look at the frames.

package/skill/reference/kit.md

@@ -34,6 +34,14 @@ FootageLayer { src: string; muted?: boolean; dim?: number }
   Full-frame user footage. Place it once, outside the scenes, as the bottom layer. dim (0-1) darkens it for legibility.
   <FootageLayer src={urls[manifest.footageKey]} dim={0.25} />
 
+ClipLayer { src: string; muted?: boolean; dim?: number }
+  Full-frame generated or library video clip as a scene's picture (treatment "clip"). Place it inside the scene's SceneFrame as the bottom layer, so it starts with the scene. When the scene is longer than the clip it holds the last frame; it never loops. dim (0-1) darkens it for legibility.
+  <ClipLayer src={urls[s.clipKey]} dim={0.2} />
+
+KeyedClip { src: string; x?: number; y?: number; scale?: number; muted?: boolean }
+  A green-screen clip with the green keyed out (the transparent .webm, s.clipKeyedKey). Lay it over your own background or screenshot, below the captions. x and y place its bottom centre as fractions of the frame (default x 0.5, y 1: bottom centre); scale is its width as a fraction of the frame width (default 1). Holds its last frame when the scene is longer.
+  <KeyedClip src={urls[s.clipKeyedKey]} x={0.5} y={1} scale={0.8} />
+
 BgMesh { bg: string; hero: string; accent?: string }
   The bottom layer of the video: the base colour with two slow-drifting soft colour fields. Use it instead of a flat background.
   <BgMesh bg={palette.bg} hero={palette.hero} accent={palette.accent} />

package/skill/reference/remotion-composition.md

@@ -48,7 +48,7 @@ Each `scenes` entry:
 - Never hard-code a duration that the manifest already gives. Look scenes up by id when a scene needs bespoke content.
 
 ## Media
-- Reference media only as `urls[path]`, where `path` is the file's project path, using the paths in the manifest: `s.voiceoverKey`, `s.imageKey`, `s.userAssetKeys`, `manifest.footageKey`. Never hard-code a URL.
+- Reference media only as `urls[path]`, where `path` is the file's project path, using the paths in the manifest: `s.voiceoverKey`, `s.imageKey`, `s.clipKey`, `s.clipKeyedKey`, `s.userAssetKeys`, `manifest.footageKey`. Never hard-code a URL.
 - A scene with narration has `s.voiceoverKey`: give it exactly one `<Voiceover src={urls[s.voiceoverKey]} />` inside its SceneFrame, and `<Captions words={s.words} />` unless the notes say otherwise.
 - Footage mode (`manifest.footageKey` is set): one `<FootageLayer>` at the bottom, outside the scenes.
 - User images: `<Img src={urls[key]} />` from remotion.
@@ -56,8 +56,9 @@ Each `scenes` entry:
 ## Layer stack (every video, bottom to top)
 1. Background: `<BgMesh>` from the kit, or `<FootageLayer>` in footage mode. Never a flat solid colour.
 2. Images: `<KenBurnsImage>` inside its scene. Every still image moves; alternate `direction` between consecutive image scenes.
-3. Graphics and type, inside each scene's SceneFrame.
-4. `<Grade>`, then `<Grain>`, then `<Vignette>`, once, at the very top of the video outside the scenes. In footage mode skip Grade.
+3. Clips: `<ClipLayer>` inside its scene (a clip scene's picture); a green-screen `<KeyedClip>` above the scene's background and below its type and captions. See `reference/clips.md`.
+4. Graphics and type, inside each scene's SceneFrame.
+5. `<Grade>`, then `<Grain>`, then `<Vignette>`, once, at the very top of the video outside the scenes. In footage mode skip Grade.
 
 ## Shared overlays (optional)
 - The library has ready-made overlay clips (`reelkit assets search "<description>" --kind overlay`): film marks, HUD frames, corner marks. They are white on black and are laid over the video with `<ScreenOverlay>`.

package/skill/reference/scene-treatments.md

@@ -5,9 +5,10 @@ description: Use when choosing each scene's visual treatment and writing image p
 
 # Choosing scene treatments
 
-## The three treatments
+## The four treatments
 - **motion-graphic** - animated text, numbers, shapes and icons drawn in code. The default. Best for hooks, lists, statistics, steps and calls to action.
 - **illustration** - one generated picture behind the text. Use when a concrete image carries the point better than type (an object, a place, a mood). At most half the scenes; each one costs money.
+- **clip** - a few seconds of generated or reused video as the scene's picture (B-roll), or a green-screen subject over your own graphics. Needs a `clipPrompt`. Rare and costly: see `reference/clips.md` before using it.
 - **footage-overlay** - text and graphics over the user's own video. Required for every scene when footage was supplied, and never used otherwise.
 
 ## Image prompts (illustration scenes only)

package/src/cli.ts

@@ -1,6 +1,6 @@
 import { Command } from "commander";
 import { ApiFailure } from "./api/client";
-import { assetsGenImage, assetsPull, assetsSearch, assetsUpload, assetsVoiceover, assetsVoices } from "./commands/assets";
+import { assetsGenClip, assetsGenImage, assetsPull, assetsSearch, assetsUpload, assetsVoiceover, assetsVoices } from "./commands/assets";
 import { authLogin, authLogout, whoami } from "./commands/auth";
 import { check, preview, render } from "./commands/build";
 import { init } from "./commands/init";
@@ -73,13 +73,18 @@ assets.command("upload <file>").description("Add one of your own files to this p
   .action(run(assetsUpload));
 assets.command("search <query>").description("Search the shared library by meaning; each result shows how well it fits").option("--kind <kind>", "image, overlay, sfx, music, component or clip").option("--limit <n>", "how many results")
   .action(run(assetsSearch));
-assets.command("pull <id>").description("Download a library item into this project").option("--scene <sceneId>", "use it as this scene's image").option("--force", "replace a component file that already exists")
-  .action(run(assetsPull));
+assets.command("pull <id>").description("Download a library item into this project").option("--scene <sceneId>", "use it as this scene's image or clip").option("--force", "replace a component file that already exists")
+  .action(run((ctx, id, opts) => assetsPull(ctx, id, opts)));
 assets.command("voices").description("List the narration voices").action(run(assetsVoices));
 assets.command("voiceover").description("Record the narration from plan.json").option("--scene <sceneId>", "one scene").option("--all", "every scene").option("--redo", "record again even if it exists")
   .action(run(assetsVoiceover));
-assets.command("gen").description("Generate an asset").command("image [prompt]").description("Generate a scene's illustration; the prompt defaults to the plan's")
+const gen = assets.command("gen").description("Generate an asset");
+gen.command("image [prompt]").description("Generate a scene's illustration; the prompt defaults to the plan's")
   .requiredOption("--scene <sceneId>", "the scene it is for").option("--redo", "generate a new image even if the scene has one").action(run(assetsGenImage));
+gen.command("clip [prompt]").description("Generate a scene's video clip (takes minutes); the prompt defaults to the plan's clipPrompt")
+  .requiredOption("--scene <sceneId>", "the scene it is for").option("--green", "film it on green and key the green out into a transparent .webm").option("--seconds <n>", "5 or 10 (default 5)")
+  .option("--share", "also add it to the shared library (only a generic clip)").option("--redo", "generate a new clip even if the scene has one").option("--resume <id>", "keep waiting for a clip already started, without paying again")
+  .action(run((ctx, prompt, opts) => assetsGenClip(ctx, prompt, opts)));
 
 program.command("plan").description("Work with plan.json").command("check").description("Validate plan.json and list what to fix or improve").action(run(planCheck));
 

package/src/commands/assets.ts

@@ -1,11 +1,12 @@
 import { randomUUID } from "node:crypto";
 import { copyFileSync, mkdirSync, readFileSync } from "node:fs";
 import { basename, dirname, resolve } from "node:path";
-import { download, uploadTo } from "../api/client";
+import { ApiFailure, download, uploadTo } from "../api/client";
 import { client, type Ctx, type Result } from "../context";
 import { LibraryKindSchema, UploadKindSchema } from "../contract";
 import { PACE_SPEED, type AssetManifest, type AssetRecord, type ScenePlan } from "../pipeline/schema";
-import { buildManifest, voiceoverStale, type Voiceover } from "../project/manifest";
+import { keyGreen } from "../project/chromakey";
+import { buildManifest, voiceoverStale, type ClipRecord, type Voiceover } from "../project/manifest";
 import { loadPlan } from "./plan";
 import { FILE_NAME } from "../render/validate";
 import { probeFile } from "../project/probe";
@@ -90,19 +91,35 @@ export function unsafeName(name: string, kind: string): boolean {
   return kind === "component" && (!FILE_NAME.test(name) || name === "Video.tsx");
 }
 
-export async function assetsPull(ctx: Ctx, id: string, opts: { scene?: string; force?: boolean }): Promise<Result> {
+// What a test may replace: the ffmpeg call that keys green out, and the clock the clip wait is timed by.
+export type ClipDeps = { key?: (input: string, output: string) => Promise<void>; now?: () => number };
+
+// The keyed copy sits beside its clip: clip.mp4 becomes clip.webm.
+const keyedPathOf = (path: string) => path.replace(/\.[^./]+$/, "") + ".webm";
+
+function setSceneClip(project: Project, sceneId: string, record: ClipRecord) {
+  project.writeJson(FILES.clips, { ...project.readJsonOr<Record<string, ClipRecord>>(FILES.clips, {}), [sceneId]: record });
+}
+
+export async function assetsPull(ctx: Ctx, id: string, opts: { scene?: string; force?: boolean }, deps: ClipDeps = {}): Promise<Result> {
   const project = openProject(ctx.cwd);
   if (!/^[A-Za-z0-9][A-Za-z0-9._-]*$/.test(id)) return { ok: false, summary: `"${id}" is not a library id. Copy the id from \`reelkit assets search\`.` };
   if (opts.scene) {
     // Checked before anything is downloaded.
-    const illustrations = loadPlan(project).scenes.filter((s) => s.treatment === "illustration").map((s) => s.id);
-    if (!illustrations.includes(opts.scene)) {
-      return { ok: false, summary: `Scene ${opts.scene} is not an illustration scene in plan.json. ${illustrations.length ? `The illustration scenes are: ${illustrations.join(", ")}.` : "The plan has no illustration scenes."}` };
+    const scenes = loadPlan(project).scenes;
+    const illustrations = scenes.filter((s) => s.treatment === "illustration").map((s) => s.id);
+    const clipScenes = scenes.filter((s) => s.treatment === "clip").map((s) => s.id);
+    if (!illustrations.includes(opts.scene) && !clipScenes.includes(opts.scene)) {
+      return { ok: false, summary: `Scene ${opts.scene} is not an illustration scene in plan.json. ${illustrations.length ? `The illustration scenes are: ${illustrations.join(", ")}.` : "The plan has no illustration scenes."}${clipScenes.length ? ` The clip scenes are: ${clipScenes.join(", ")}.` : ""}` };
     }
   }
   const pulled = await client(ctx)("libraryPull", { id });
   if (unsafeName(pulled.filename, pulled.item.kind)) return { ok: false, summary: `The library returned an unsafe file name for ${id}, so nothing was written. Report this item.` };
-  if (opts.scene && pulled.item.kind !== "image") return { ok: false, summary: `${id} is ${pulled.item.kind}, not an image, so it cannot be a scene's image.` };
+  if (opts.scene) {
+    const treatment = loadPlan(project).scenes.find((x) => x.id === opts.scene)?.treatment;
+    if (treatment === "clip" && pulled.item.kind !== "clip") return { ok: false, summary: `${id} is ${pulled.item.kind}, not a clip, so it cannot be scene ${opts.scene}'s clip.` };
+    if (treatment !== "clip" && pulled.item.kind !== "image") return { ok: false, summary: `${id} is ${pulled.item.kind}, not an image, so it cannot be a scene's image.` };
+  }
   // A component is code, not media: it goes beside Video.tsx, which may only import files in src/.
   if (pulled.item.kind === "component") {
     const path = `src/${pulled.filename}`;
@@ -116,15 +133,24 @@ export async function assetsPull(ctx: Ctx, id: string, opts: { scene?: string; f
   const path = `assets/lib/${id}/${pulled.filename}`;
   await download(pulled.url, project.path(path));
   recordPull(project, id, { path, kind: pulled.item.kind, title: pulled.item.title, meta: pulled.item.meta });
+  // A clip filmed on green also comes keyed, so it can be laid over a background.
+  const green = pulled.item.kind === "clip" && pulled.item.meta.greenScreen === true;
+  const keyedPath = green ? keyedPathOf(path) : undefined;
+  if (keyedPath) await (deps.key ?? keyGreen)(project.path(path), project.path(keyedPath));
+  const sceneIsClip = pulled.item.kind === "clip";
   let note: string | undefined;
   if (opts.scene) {
-    setSceneImage(project, opts.scene, path);
+    if (sceneIsClip) {
+      const durationSec = typeof pulled.item.meta.durationSec === "number" ? pulled.item.meta.durationSec : 5;
+      setSceneClip(project, opts.scene, { key: path, greenScreen: green, durationSec, ...(keyedPath ? { keyedKey: keyedPath } : {}) });
+    } else setSceneImage(project, opts.scene, path);
     if (project.exists(FILES.plan)) note = tryManifest(project, loadPlan(project)).note;
   }
+  const refs = keyedPath ? ` The original is urls["${path}"]; the keyed copy with a transparent background is urls["${keyedPath}"].` : ` Reference it in the composition as urls["${path}"].`;
   return {
-    ok: true, data: { path, kind: pulled.item.kind },
+    ok: true, data: { path, kind: pulled.item.kind, ...(keyedPath ? { keyedPath } : {}) },
     summary: [
-      opts.scene ? `Pulled ${id} as the image for scene ${opts.scene}.` : `Pulled ${id} to ${path}. Reference it in the composition as urls["${path}"].`,
+      opts.scene ? `Pulled ${id} as the ${sceneIsClip ? "clip" : "image"} for scene ${opts.scene}.${keyedPath ? refs : ""}` : `Pulled ${id} to ${path}.${refs}`,
       ...(note ? [note] : []),
     ].join("\n"),
   };
@@ -194,3 +220,113 @@ export async function assetsGenImage(ctx: Ctx, prompt: string | undefined, opts:
     summary: [`Generated the image for scene ${scene.id} at ${path}${img.libraryId ? `, and shared it to the library as ${img.libraryId} (awaiting review)` : ""}.`, ...(note ? [note] : [])].join("\n"),
   };
 }
+
+
+const CLIP_POLL_MS = 5_000;
+const CLIP_PROGRESS_MS = 30_000;
+const CLIP_TIMEOUT_MS = 10 * 60_000;
+
+// A clip takes minutes, so it is a job: start it (which reserves the clip from the quota), then ask every few seconds until it ends.
+// The id is the only handle on a paid job, so every way out after the start says it and how to resume.
+export async function assetsGenClip(
+  ctx: Ctx, prompt: string | undefined,
+  opts: { scene: string; green?: boolean; seconds?: string; share?: boolean; redo?: boolean; resume?: string },
+  deps: ClipDeps = {},
+): Promise<Result> {
+  const project = openProject(ctx.cwd);
+  const plan = loadPlan(project);
+  const scene = plan.scenes.find((s) => s.id === opts.scene);
+  if (!scene || scene.treatment !== "clip" || !scene.clipPrompt) return { ok: false, summary: `Scene ${opts.scene} is not a clip scene, so it does not take a clip. Set its treatment to "clip" and give it a clipPrompt in plan.json.` };
+  const seconds = opts.seconds === undefined ? 5 : Number(opts.seconds);
+  if (seconds !== 5 && seconds !== 10) return { ok: false, summary: `--seconds must be 5 or 10, not "${opts.seconds}".` };
+  if (opts.resume && !/^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/.test(opts.resume)) return { ok: false, summary: `"${opts.resume}" is not a clip id. Copy it from the earlier message.` };
+  if (opts.share && (!scene.shareable || scene.userAssetIds.length > 0 || project.footage())) {
+    return { ok: false, summary: `Scene ${scene.id} is private (it is not marked shareable in plan.json, or it shows your own files), so --share is refused. Run again without --share.` };
+  }
+
+  const key = deps.key ?? keyGreen;
+  const now = deps.now ?? Date.now;
+  const keyedKey = `assets/clip-${scene.id}.webm`;
+  const records = project.readJsonOr<Record<string, ClipRecord>>(FILES.clips, {});
+  const have = records[scene.id];
+  // A scene that already has its clip is not charged for again, whatever prompt is given; only --redo asks for a new one.
+  if (have && project.exists(have.key) && !opts.redo && !opts.resume) {
+    if (opts.green && have.greenScreen && !have.keyedKey) {
+      // The clip was paid for but not keyed (ffmpeg failed or --green came later): keying is free, so do it now.
+      const keyed = await keyScene(project, scene.id, have, key);
+      if (!keyed.ok) return keyed;
+      return { ok: true, data: { path: have.key, keyedPath: keyedKey }, summary: `Keyed the green out of scene ${scene.id}'s clip into ${keyedKey}.` };
+    }
+    return { ok: true, data: { path: have.key, skipped: true }, summary: `Scene ${scene.id} already has a clip at ${have.key}. Use --redo to generate a new one.` };
+  }
+
+  const api = client(ctx);
+  const started = now();
+  let id = opts.resume;
+  if (!id) {
+    const job = await api("clipStart", {
+      prompt: prompt ?? scene.clipPrompt, aspect: plan.aspect, greenScreen: Boolean(opts.green), durationSec: seconds,
+      shareable: Boolean(opts.share), tags: scene.imageTags,
+    });
+    id = job.id;
+    ctx.log(`Started clip ${id} for scene ${scene.id}. It usually takes a few minutes.`);
+  } else ctx.log(`Waiting for clip ${id} for scene ${scene.id}.`);
+  const resume = `reelkit assets gen clip --resume ${id} --scene ${scene.id}`;
+
+  try {
+    let lastLog = started;
+    for (;;) {
+      const st = await api("clipStatus", { id });
+      if (st.status === "failed") {
+        return { ok: false, data: { id, status: "failed" }, summary: `The clip for scene ${scene.id} failed: ${st.message ?? "the video provider gave no reason."} You were not charged. Change the prompt and run the command again.` };
+      }
+      if (st.status === "done") {
+        const path = `assets/clip-${scene.id}.${st.ext}`;
+        await download(st.url!, project.path(path));
+        const green = Boolean(st.greenScreen ?? opts.green);
+        const record: ClipRecord = { key: path, greenScreen: green, durationSec: st.durationSec ?? seconds };
+        setSceneClip(project, scene.id, record);
+        let keyedPath: string | undefined;
+        if (green || opts.green) {
+          // The paid clip is already saved above; if keying fails, running the command again keys it without a new charge.
+          const keyed = await keyScene(project, scene.id, { ...record, greenScreen: true }, key);
+          if (!keyed.ok) return keyed;
+          keyedPath = keyedKey;
+        }
+        const { note } = tryManifest(project, plan);
+        return {
+          ok: true, data: { id, path, durationSec: record.durationSec, ...(keyedPath ? { keyedPath } : {}), libraryId: st.libraryId ?? null },
+          summary: [
+            `Generated the clip for scene ${scene.id} at ${path}${keyedPath ? `, keyed to ${keyedPath}` : ""}${st.libraryId ? `, and shared it to the library as ${st.libraryId} (awaiting review)` : ""}.`,
+            ...(note ? [note] : []),
+          ].join("\n"),
+        };
+      }
+      const elapsed = now() - started;
+      if (elapsed >= CLIP_TIMEOUT_MS) {
+        return { ok: false, data: { id, status: "pending" }, summary: `The clip for scene ${scene.id} (id ${id}) is not ready after ${CLIP_TIMEOUT_MS / 60_000} minutes. It is still being made and is not lost: run \`${resume}\` to keep waiting for it; that does not charge again.` };
+      }
+      if (now() - lastLog >= CLIP_PROGRESS_MS) {
+        lastLog = now();
+        ctx.log(`Still making the clip for scene ${scene.id} (${Math.round(elapsed / 1000)}s so far)...`);
+      }
+      await ctx.sleep(CLIP_POLL_MS);
+    }
+  } catch (e) {
+    // A clip that is not found or a login that is wrong has nothing to resume; anything else may be a hiccup on a clip that is paid for.
+    if (e instanceof ApiFailure && (e.code === "not_found" || e.code === "unauthenticated")) throw e;
+    return { ok: false, data: { id }, summary: `Lost contact while the clip for scene ${scene.id} (id ${id}) was being made: ${e instanceof Error ? e.message : String(e)} The clip is not lost: run \`${resume}\`; that does not charge again.` };
+  }
+}
+
+// Keys a scene's saved clip and records the keyed copy. A failure leaves the record as it was.
+async function keyScene(project: Project, sceneId: string, record: ClipRecord, key: NonNullable<ClipDeps["key"]>): Promise<Result> {
+  const keyedKey = `assets/clip-${sceneId}.webm`;
+  try {
+    await key(project.path(record.key), project.path(keyedKey));
+  } catch (e) {
+    return { ok: false, data: { path: record.key }, summary: `The clip is saved at ${record.key}, but the green could not be keyed out: ${e instanceof Error ? e.message : String(e)}` };
+  }
+  setSceneClip(project, sceneId, { ...record, greenScreen: true, keyedKey });
+  return { ok: true, summary: "" };
+}

package/src/commands/auth.ts

@@ -113,6 +113,6 @@ export async function whoami(ctx: Ctx): Promise<Result> {
   const q = me.quota;
   return {
     ok: true, data: me,
-    summary: `${me.handle}\nVoiceover: ${q.voiceoverChars.used}/${q.voiceoverChars.limit} characters\nImages: ${q.images.used}/${q.images.limit}\nResets: ${q.resetsAt.slice(0, 10)}\nContributions: ${me.contributions}`,
+    summary: `${me.handle}\nVoiceover: ${q.voiceoverChars.used}/${q.voiceoverChars.limit} characters\nImages: ${q.images.used}/${q.images.limit}\nClips: ${q.clips.used}/${q.clips.limit}\nResets: ${q.resetsAt.slice(0, 10)}\nContributions: ${me.contributions}`,
   };
 }

package/src/contract/index.ts

@@ -39,9 +39,20 @@
 //   a file URL serves one file, an upload URL accepts uploads for one object only.
 // - The public listing's `previewUrl` is a preview that needs no login. An item with a preview of its own gets that one; otherwise an `image`
 //   or an `sfx` gets a URL of its own file (a picture and a short sound need no login to look at or hear), and every other kind (music,
-//   overlay, clip, component), whose full file is long media or code, gets no `previewUrl`.
+//   overlay, clip, component), whose full file is long media or code, gets no `previewUrl`. The one exception is a clip of ten seconds
+//   or less (`meta.durationSec`): short generated clips are made to be shared, so such a clip is previewed by its own file.
 // - Every returned URL needs no headers and stays valid for at least five minutes.
 // - The generation routes (`voiceover`, `images`) are synchronous: they return the URL of the finished file.
+// - A video clip takes minutes, so it is a job with two calls. `clipStart` checks the request, reserves one clip from the caller's monthly clip
+//   quota and answers at once with an id and `pending`; the client then asks `clipStatus` for that id until it is `done` or `failed`.
+//   `url`, `ext` and `contentType` are present exactly when the status is `done`, and `message` (one plain line saying what went wrong)
+//   exactly when it is `failed`. A failed clip is not charged: the reservation is released. A clip id belongs to its caller: another user's id,
+//   or an unknown one, is `not_found`.
+// - When the quota is used up, `clipStart` is 429 `quota_exceeded` with the reset date, as for images. When the server has no video provider
+//   configured, `clipStart` is 400 `invalid_request` with the message "Clip generation is not available on this server yet."
+// - A green-screen clip (`greenScreen: true`) is a subject filmed on a flat pure green background, meant to be keyed out by the client. The server
+//   stores and returns the original, green and all; it does not remove the background. A shareable clip also becomes a library item of kind
+//   `clip` in review, whose `meta` may carry `greenScreen: true` and `durationSec`, and its id is returned as `libraryId`.
 // - `durationSec` is the decoded audio length; `words` are in seconds from the start of that audio.
 // - Search returns published items only, and leaves out items that are not matches at all; the default `limit` is 8.
 import { z } from "zod";
@@ -89,7 +100,8 @@ export const LibraryItemSchema = z.object({
   title: z.string(),
   description: z.string(),
   tags: z.array(z.string()),
-  // width, height, durationSec, aspect, props, source, licence: whatever applies to the kind.
+  // width, height, durationSec, aspect, props, source, licence: whatever applies to the kind. A clip may carry `greenScreen: true` (filmed on a
+  // flat green background, to be keyed out) and `durationSec`.
   meta: z.record(z.string(), z.unknown()),
   visibility: z.enum(["private", "review", "published"]),
 });
@@ -105,7 +117,7 @@ export type Voice = z.infer<typeof VoiceSchema>;
 const Meter = z.object({ used: z.number(), limit: z.number() });
 export const MeSchema = z.object({
   userId: z.string(), handle: z.string(),
-  quota: z.object({ voiceoverChars: Meter, images: Meter, resetsAt: z.string() }),
+  quota: z.object({ voiceoverChars: Meter, images: Meter, clips: Meter, resetsAt: z.string() }),
   // How many of the user's own items are published in the shared library. An item still in review, or sent back, is not counted.
   contributions: z.number(),
 });
@@ -148,6 +160,20 @@ export const routes = {
     z.object({ prompt: text(2000).min(1), aspect: AspectSchema, shareable: z.boolean(), tags: z.array(text(40)).max(12) }),
     // libraryId is set when the image was also added to the shared library.
     z.object({ url: z.string(), ext: z.enum(["png", "jpg", "jpeg", "webp"]), contentType: z.string(), libraryId: z.string().optional() })),
+  clipStart: route("POST", "/clips", true,
+    z.object({
+      prompt: text(2000).min(1), aspect: AspectSchema, greenScreen: z.boolean().default(false), durationSec: z.union([z.literal(5), z.literal(10)]).default(5),
+      shareable: z.boolean(), tags: z.array(text(40)).max(12),
+    }),
+    z.object({ id: z.string(), status: z.literal("pending") })),
+  clipStatus: route("POST", "/clips/status", true,
+    z.object({ id: ItemId }),
+    z.object({
+      id: z.string(), status: z.enum(["pending", "done", "failed"]), message: z.string().optional(),
+      url: z.string().optional(), ext: z.enum(["mp4"]).optional(), contentType: z.string().optional(), durationSec: z.number().positive().optional(),
+      greenScreen: z.boolean().optional(), libraryId: z.string().optional(),
+    }).refine((r) => (r.status === "done") === (r.url !== undefined && r.ext !== undefined && r.contentType !== undefined) && (r.status === "failed") === (r.message !== undefined),
+      "url, ext and contentType belong to done and message to failed")),
   publicLibrary: route("GET", "/public/library", false,
     z.object({ q: text(200).optional(), kind: LibraryKindSchema.optional(), page: z.coerce.number().int().min(1).max(10000).optional() }),
     // With `q` the items are the best matches by meaning, best first, each with `match` (0..1), in one page (`hasMore` false). Under heavy

package/src/pipeline/schema.ts

@@ -8,9 +8,12 @@ export type Aspect = z.infer<typeof AspectSchema>;
 export const SceneSchema = z.object({
   id: z.string().regex(/^[a-z0-9-]+$/).describe("short unique id, lowercase letters, digits, dashes"),
   narration: z.string().min(1).describe("the words spoken in this scene"),
-  treatment: z.enum(["motion-graphic", "illustration", "footage-overlay"]),
+  // clip: a generated or reused video clip is the scene's picture.
+  treatment: z.enum(["motion-graphic", "illustration", "footage-overlay", "clip"]),
   onScreenText: z.array(z.string()),
   imagePrompt: z.string().nullable().describe("required when treatment is illustration, otherwise null"),
+  // Optional only so plans stored before clips existed still load.
+  clipPrompt: z.string().nullable().optional().describe("required when treatment is clip, otherwise null: one subject, one action, a camera move, lighting, no text"),
   shareable: z.boolean().describe("true only when imagePrompt is fully generic: no brand, product, person or detail specific to this user. Such images are saved to a library shared by all users."),
   imageTags: z.array(z.string()).describe("3 to 6 short tags describing the illustration; empty when imagePrompt is null"),
   userAssetIds: z.array(z.string()),
@@ -57,6 +60,9 @@ export const ManifestSceneSchema = z.object({
   voiceoverKey: z.string().optional(),
   words: z.array(WordTimingSchema),
   imageKey: z.string().optional(),
+  // The scene's video clip, and its transparent copy when it was made on a green background.
+  clipKey: z.string().optional(),
+  clipKeyedKey: z.string().optional(),
   userAssetKeys: z.array(z.string()),
 });
 export type ManifestScene = z.infer<typeof ManifestSceneSchema>;
@@ -103,6 +109,8 @@ export function validatePlan(
   for (const s of plan.scenes) {
     if (s.treatment === "illustration" && !s.imagePrompt)
       errors.push(`Scene ${s.id}: imagePrompt is required for an illustration.`);
+    if (s.treatment === "clip" && !s.clipPrompt)
+      errors.push(`Scene ${s.id}: clipPrompt is required for a clip.`);
     for (const id of s.userAssetIds)
       if (!allowed.has(id)) errors.push(`Scene ${s.id}: unknown user asset ${id}.`);
   }

package/src/project/chromakey.ts

@@ -0,0 +1,17 @@
+import { execFile } from "node:child_process";
+import { promisify } from "node:util";
+
+const run = promisify(execFile);
+
+// Turns the flat pure green of a green-screen clip into transparency: VP9 with an alpha channel, which Remotion plays with `transparent`.
+// The key is on pure green with a little give for compression noise; the despill pulls the green glow off the subject's edges.
+// The original is left alone.
+export async function keyGreen(input: string, output: string): Promise<void> {
+  const filter = "chromakey=color=0x00FF00:similarity=0.2:blend=0.1,despill=type=green:mix=0.5:expand=0,format=yuva420p";
+  try {
+    await run("ffmpeg", ["-v", "error", "-y", "-i", input, "-vf", filter, "-c:v", "libvpx-vp9", "-pix_fmt", "yuva420p", "-b:v", "0", "-crf", "30", "-an", output]);
+  } catch (e) {
+    if ((e as NodeJS.ErrnoException)?.code === "ENOENT") throw new Error("ffmpeg was not found. Install ffmpeg (macOS: `brew install ffmpeg`) and run the command again.");
+    throw new Error("ffmpeg could not key the green out of the clip. The original clip is saved; run the command again to retry.");
+  }
+}

package/src/project/manifest.ts

@@ -5,6 +5,9 @@ import { FILES, type Project } from "./project";
 // What a scene was recorded from is stored with it, so a changed script or voice is noticed.
 export type Voiceover = { key: string; durationSec: number; words: WordTiming[]; text?: string; voiceId?: string; speed?: number };
 
+// A scene's video clip: the original, and the transparent copy when it was filmed on green and keyed out. Stored in assets/clips.json by scene id.
+export type ClipRecord = { key: string; greenScreen: boolean; durationSec: number; keyedKey?: string };
+
 // True when the stored recording no longer matches what the plan asks for. An entry from before this was tracked counts as different.
 export function voiceoverStale(vo: Voiceover, scene: ScenePlan["scenes"][number], plan: ScenePlan): boolean {
   return vo.text !== scene.narration || vo.voiceId !== plan.voiceId || vo.speed !== PACE_SPEED[plan.pace ?? "normal"];
@@ -16,6 +19,7 @@ export function buildManifest(project: Project, plan: ScenePlan): AssetManifest
   const voiceovers = project.readJsonOr<Record<string, Voiceover>>(FILES.voiceovers, {});
   if (plan.scenes.some((s) => !voiceovers[s.id])) return undefined;
   const images = project.readJsonOr<Record<string, string>>(FILES.images, {});
+  const clips = project.readJsonOr<Record<string, ClipRecord>>(FILES.clips, {});
   const footage = project.footage();
   const byId = new Map(project.assets().map((a) => [a.id, a]));
 
@@ -41,6 +45,7 @@ export function buildManifest(project: Project, plan: ScenePlan): AssetManifest
       voiceoverKey: voiceovers[scene.id].key,
       words: voiceovers[scene.id].words,
       ...(scene.treatment === "illustration" && images[scene.id] ? { imageKey: images[scene.id] } : {}),
+      ...(scene.treatment === "clip" && clips[scene.id] ? { clipKey: clips[scene.id].key, ...(clips[scene.id].keyedKey ? { clipKeyedKey: clips[scene.id].keyedKey } : {}) } : {}),
       userAssetKeys: scene.userAssetIds.map((id) => {
         const a = byId.get(id);
         if (!a) throw new Error(`Scene ${scene.id} references unknown asset ${id}. Add it with \`reelkit assets upload\`.`);
@@ -56,6 +61,7 @@ export function buildManifest(project: Project, plan: ScenePlan): AssetManifest
 export function missingAssets(project: Project, plan: ScenePlan): string[] {
   const voiceovers = project.readJsonOr<Record<string, Voiceover>>(FILES.voiceovers, {});
   const images = project.readJsonOr<Record<string, string>>(FILES.images, {});
+  const clips = project.readJsonOr<Record<string, ClipRecord>>(FILES.clips, {});
   const problems: string[] = [];
   for (const s of plan.scenes) {
     if (!voiceovers[s.id]) problems.push(`Scene ${s.id} has no voiceover.`);
@@ -66,6 +72,11 @@ export function missingAssets(project: Project, plan: ScenePlan): string[] {
       if (!images[s.id]) problems.push(`Scene ${s.id} has no image.`);
       else if (!project.exists(images[s.id])) problems.push(`Missing file: ${images[s.id]}`);
     }
+    if (s.treatment === "clip") {
+      const clip = clips[s.id];
+      if (!clip) problems.push(`Scene ${s.id} has no clip. Run \`reelkit assets gen clip --scene ${s.id}\`.`);
+      else for (const key of [clip.key, ...(clip.keyedKey ? [clip.keyedKey] : [])]) if (!project.exists(key)) problems.push(`Missing file: ${key}`);
+    }
   }
   return problems;
 }

package/src/project/project.ts

@@ -5,7 +5,7 @@ import { AspectSchema, type AssetRecord } from "../pipeline/schema";
 
 export const FILES = {
   config: "reelkit.json", plan: "plan.json", manifest: "manifest.json",
-  assetIndex: "assets/index.json", voiceovers: "assets/voiceovers.json", images: "assets/images.json", library: "assets/library.json",
+  assetIndex: "assets/index.json", voiceovers: "assets/voiceovers.json", images: "assets/images.json", clips: "assets/clips.json", library: "assets/library.json",
 } as const;
 
 // "path: message", or just the message for a problem at the root of the file.

package/src/remotion/kit/ClipLayer.tsx

@@ -0,0 +1,12 @@
+import React from "react";
+import { useMedia } from "./media";
+import { AbsoluteFill, OffthreadVideo } from "remotion";
+
+// A generated or library video clip as a scene's picture. Place it inside the scene's SceneFrame, as the bottom layer, so it starts with the scene.
+// When the scene runs longer than the clip, the clip holds its last frame (it does not loop: a looped person or animal jumps back and shows the seam).
+export const ClipLayer: React.FC<{ src: string; muted?: boolean; dim?: number }> = ({ src, muted = true, dim = 0 }) => (
+  <AbsoluteFill>
+    <OffthreadVideo src={useMedia(src)} muted={muted} style={{ width: "100%", height: "100%", objectFit: "cover" }} />
+    {dim > 0 ? <AbsoluteFill style={{ backgroundColor: `rgba(0,0,0,${dim})` }} /> : null}
+  </AbsoluteFill>
+);

package/src/remotion/kit/KeyedClip.tsx

@@ -0,0 +1,15 @@
+import React from "react";
+import { useMedia } from "./media";
+import { AbsoluteFill, OffthreadVideo } from "remotion";
+
+// A green-screen clip with its green keyed out (the transparent .webm that `--green` makes), laid over whatever is beneath it.
+// x and y are where the clip's bottom-centre sits, as fractions of the frame (x 0 is the left edge, y 1 the bottom edge); scale is the clip's
+// width as a fraction of the frame width. The clip has the frame's own shape, so it is as tall as the frame is, times scale.
+// Like ClipLayer it holds its last frame when the scene is longer.
+export const KeyedClip: React.FC<{ src: string; x?: number; y?: number; scale?: number; muted?: boolean }> = ({ src, x = 0.5, y = 1, scale = 1, muted = true }) => (
+  <AbsoluteFill style={{ pointerEvents: "none" }}>
+    <div style={{ position: "absolute", left: `${x * 100}%`, top: `${y * 100}%`, width: `${scale * 100}%`, transform: "translate(-50%, -100%)" }}>
+      <OffthreadVideo src={useMedia(src)} muted={muted} transparent style={{ width: "100%", display: "block" }} />
+    </div>
+  </AbsoluteFill>
+);

package/src/remotion/kit/docs.ts

@@ -32,6 +32,14 @@ FootageLayer { src: string; muted?: boolean; dim?: number }
   Full-frame user footage. Place it once, outside the scenes, as the bottom layer. dim (0-1) darkens it for legibility.
   <FootageLayer src={urls[manifest.footageKey]} dim={0.25} />
 
+ClipLayer { src: string; muted?: boolean; dim?: number }
+  Full-frame generated or library video clip as a scene's picture (treatment "clip"). Place it inside the scene's SceneFrame as the bottom layer, so it starts with the scene. When the scene is longer than the clip it holds the last frame; it never loops. dim (0-1) darkens it for legibility.
+  <ClipLayer src={urls[s.clipKey]} dim={0.2} />
+
+KeyedClip { src: string; x?: number; y?: number; scale?: number; muted?: boolean }
+  A green-screen clip with the green keyed out (the transparent .webm, s.clipKeyedKey). Lay it over your own background or screenshot, below the captions. x and y place its bottom centre as fractions of the frame (default x 0.5, y 1: bottom centre); scale is its width as a fraction of the frame width (default 1). Holds its last frame when the scene is longer.
+  <KeyedClip src={urls[s.clipKeyedKey]} x={0.5} y={1} scale={0.8} />
+
 BgMesh { bg: string; hero: string; accent?: string }
   The bottom layer of the video: the base colour with two slow-drifting soft colour fields. Use it instead of a flat background.
   <BgMesh bg={palette.bg} hero={palette.hero} accent={palette.accent} />

package/src/remotion/kit/index.ts

@@ -1,4 +1,5 @@
 export { Captions } from "./Captions";
+export { ClipLayer } from "./ClipLayer";
 export { Counter } from "./Counter";
 export { Entrance, WordReveal } from "./Entrance";
 export { FootageLayer } from "./FootageLayer";
@@ -6,6 +7,7 @@ export { brandColor, Icon } from "./Icon";
 export type { GlyphName, IconName } from "./Icon";
 export type { BrandName } from "./brand-icons";
 export { KenBurnsImage } from "./KenBurnsImage";
+export { KeyedClip } from "./KeyedClip";
 export { BgMesh, Grade, Grain, Vignette } from "./Layers";
 export { LowerThird } from "./LowerThird";
 export { SceneFrame } from "./SceneFrame";

package/src/testing/conformance.ts

@@ -15,8 +15,13 @@ export type Harness = {
   // What the website does when a signed-in user enters the code.
   approve(userCode: string): unknown;
   close(): void | Promise<void>;
+  // Optional, for the clip tests. A prompt that makes a clip job fail (the fake fails any prompt containing "clip-fail"); without it the
+  // failing-job case is skipped.
+  clipFailPrompt?: string;
+  // How long to wait between two asks for a clip's status, in milliseconds: a server whose fake provider needs a moment sets it. Default 20.
+  clipPollMs?: number;
 };
-export type StartOptions = { voiceoverChars?: number; images?: number };
+export type StartOptions = { voiceoverChars?: number; images?: number; clips?: number };
 
 const PNG = new Uint8Array(Buffer.from("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==", "base64"));
 const DATE = /\d{4}-\d{2}-\d{2}/;
@@ -75,6 +80,8 @@ export function runConformance(name: string, start: (opts: StartOptions) => Prom
         expect(me.handle.length).toBeGreaterThan(0);
         expect(me.quota.voiceoverChars.used).toBe(0);
         expect(me.quota.images.used).toBe(0);
+        expect(me.quota.clips.used).toBe(0);
+        expect(me.quota.clips.limit).toBeGreaterThan(0);
         expect(me.quota.voiceoverChars.limit).toBeGreaterThan(0);
         expect(me.contributions).toBe(0);
         const resets = new Date(me.quota.resetsAt);
@@ -222,6 +229,10 @@ export function runConformance(name: string, start: (opts: StartOptions) => Prom
         ["a voiceover text over 5000 characters", (api: Api) => api("voiceover", { text: "a".repeat(5001) })],
         ["a speed over 1.3", (api: Api) => api("voiceover", { text: "hi", speed: 1.4 })],
         ["an unknown aspect", (api: Api) => api("images", { prompt: "a", aspect: "4:3" as never, shareable: false, tags: [] })],
+        ["a clip with no prompt", (api: Api) => api("clipStart", { prompt: "", aspect: "9:16", shareable: false, tags: [] })],
+        ["a clip of 7 seconds", (api: Api) => api("clipStart", { prompt: "a", aspect: "9:16", durationSec: 7 as never, shareable: false, tags: [] })],
+        ["a clip prompt with a NUL", (api: Api) => api("clipStart", { prompt: "a\u0000b", aspect: "9:16", shareable: false, tags: [] })],
+        ["a clip status id with a slash", (api: Api) => api("clipStatus", { id: "../x" })],
       ])("refuses %s as invalid_request", async (_what, call) => {
         expect((await failure(call(await as("user-bad000001")))).code).toBe("invalid_request");
       });
@@ -269,6 +280,8 @@ export function runConformance(name: string, start: (opts: StartOptions) => Prom
       it("generation needs a login", async () => {
         expect((await failure(anon("voiceover", { text: "hi" }))).code).toBe("unauthenticated");
         expect((await failure(anon("images", { prompt: "a", aspect: "9:16", shareable: false, tags: [] }))).code).toBe("unauthenticated");
+        expect((await failure(anon("clipStart", { prompt: "a", aspect: "9:16", shareable: false, tags: [] }))).code).toBe("unauthenticated");
+        expect((await failure(anon("clipStatus", { id: "clip-x" }))).code).toBe("unauthenticated");
       });
 
       it("images: a private one is only a file; a shareable one also enters the library for review, owned by the caller", async () => {
@@ -296,6 +309,95 @@ export function runConformance(name: string, start: (opts: StartOptions) => Prom
       });
     });
 
+    describe("clips", () => {
+      // Asks until the job is no longer pending: at most 20 times, with the harness's pause between asks.
+      const settle = async (api: Api, id: string) => {
+        for (let i = 0; i < 20; i++) {
+          const st = await api("clipStatus", { id });
+          if (st.status !== "pending") return st;
+          await new Promise((r) => setTimeout(r, h.clipPollMs ?? 20));
+        }
+        throw new Error(`clip ${id} was still pending after 20 asks`);
+      };
+      const ask = { aspect: "9:16" as const, shareable: false, tags: [] as string[] };
+
+      it("start answers pending at once, the job ends done with a file that downloads, and the clip is charged to this user only", async () => {
+        const api = await as("user-clip00001"), other = await as("user-clip00002");
+        const started = await api("clipStart", { ...ask, prompt: "a dog running on a beach" });
+        expect(started.status).toBe("pending");
+        expect(started.id).toBeTruthy();
+        // The clip is reserved at the start, before it is finished.
+        expect((await api("me", {})).quota.clips.used).toBe(1);
+        const done = await settle(api, started.id);
+        expect(done).toMatchObject({ id: started.id, status: "done", ext: "mp4" });
+        expect(done.url).toBeTruthy();
+        expect(done.contentType).toMatch(/^video\//);
+        expect(done.message).toBeUndefined();
+        await download(done.url!, join(dest, "clip.mp4"));
+        expect(readFileSync(join(dest, "clip.mp4")).length).toBeGreaterThan(0);
+        expect((await api("me", {})).quota.clips.used).toBe(1);
+        expect((await other("me", {})).quota.clips.used).toBe(0);
+      });
+
+      it("a clip id belongs to its caller: another user, or an unknown id, is not_found", async () => {
+        const api = await as("user-clip00003"), other = await as("user-clip00004");
+        const started = await api("clipStart", { ...ask, prompt: "a quiet harbour at dawn" });
+        expect((await failure(other("clipStatus", { id: started.id }))).code).toBe("not_found");
+        expect((await failure(api("clipStatus", { id: "clip-nosuchjob" }))).code).toBe("not_found");
+      });
+
+      it("a green-screen clip is accepted and finishes like any other", async () => {
+        const api = await as("user-clip00005");
+        const started = await api("clipStart", { ...ask, prompt: "a person waving, flat green background", greenScreen: true, durationSec: 10 });
+        const done = await settle(api, started.id);
+        expect(done.status).toBe("done");
+        await download(done.url!, join(dest, "green.mp4"));
+        expect(readFileSync(join(dest, "green.mp4")).length).toBeGreaterThan(0);
+      });
+
+      it("a shareable clip enters the library in review, owned by the caller, and is not found by search", async () => {
+        const api = await as("user-clip00006"), other = await as("user-clip00007");
+        const started = await api("clipStart", { ...ask, prompt: "a clipshare lighthouse in a storm", shareable: true, tags: ["lighthouse"] });
+        const done = await settle(api, started.id);
+        expect(done.status).toBe("done");
+        expect(done.libraryId).toBeTruthy();
+        expect((await api("libraryPull", { id: done.libraryId! })).item).toMatchObject({ kind: "clip", visibility: "review" });
+        expect((await api("librarySearch", { q: "clipshare lighthouse" })).items.map((i) => i.id)).not.toContain(done.libraryId);
+        expect((await failure(other("libraryPull", { id: done.libraryId! }))).code).toBe("not_found");
+        const plain = await settle(api, (await api("clipStart", { ...ask, prompt: "a private clip" })).id);
+        expect(plain.libraryId).toBeUndefined();
+      });
+
+      it("a failed job says why in one line and is not charged", async () => {
+        if (!h.clipFailPrompt) return;
+        const api = await as("user-clip00008");
+        const started = await api("clipStart", { ...ask, prompt: h.clipFailPrompt });
+        const st = await settle(api, started.id);
+        expect(st.status).toBe("failed");
+        expect(st.message?.length).toBeGreaterThan(0);
+        expect(st.message).not.toContain("\n");
+        expect(st.url).toBeUndefined();
+        expect((await api("me", {})).quota.clips.used).toBe(0);
+      });
+
+      it("at the limit a new clip is quota_exceeded with the reset date, charged nothing; other users are unaffected", async () => {
+        const t = await start({ clips: 1 });
+        try {
+          const api = createClient({ baseUrl: t.baseUrl, token: await t.login("user-clipq0001") });
+          const other = createClient({ baseUrl: t.baseUrl, token: await t.login("user-clipq0002") });
+          await api("clipStart", { ...ask, prompt: "first" });
+          const e = await failure(api("clipStart", { ...ask, prompt: "second" }));
+          expect(e.code).toBe("quota_exceeded");
+          expect(e.message).toMatch(DATE);
+          const me = await api("me", {});
+          expect(me.quota.clips).toEqual({ used: 1, limit: 1 });
+          expect(e.message).toContain(me.quota.resetsAt.slice(0, 10));
+          expect((await other("me", {})).quota.clips.used).toBe(0);
+          await other("clipStart", { ...ask, prompt: "third" });
+        } finally { await t.close(); }
+      });
+    });
+
     describe("quota", () => {
       it("passes exactly at the limit; one over is quota_exceeded with the reset date; other users are unaffected", async () => {
         const t = await start({ voiceoverChars: 10, images: 1 });

package/src/testing/fake-api.ts

@@ -1,11 +1,17 @@
+import { execFileSync } from "node:child_process";
 import { randomBytes } from "node:crypto";
+import { mkdtempSync, readFileSync, rmSync } from "node:fs";
 import { createServer } from "node:http";
 import type { AddressInfo } from "node:net";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
 import { MAX_UPLOAD_BYTES, routes, type ErrorCode, type LibraryItem, type RouteName, type Voice } from "../contract";
 import { PIXEL_PNG, silentWav } from "./fixtures";
 
 type Blob = { filename: string; contentType: string; bytes: Uint8Array };
 // owner is the uploader's user id; a seeded item has none. An upload is not an item for anyone until it is committed.
+// A clip of ten seconds or less is previewed by its own file, like a picture or a sound effect.
+const shortClip = (item: LibraryItem) => item.kind === "clip" && typeof item.meta.durationSec === "number" && item.meta.durationSec <= 10;
 type Stored = { item: LibraryItem; file?: Blob; filename: string; contentType: string; owner?: string; committed: boolean };
 
 export const FAKE_VOICES: Voice[] = [
@@ -37,10 +43,29 @@ function parseBody(raw: Buffer): unknown {
   catch { throw new Fail(400, "invalid_request", "The request body is not valid JSON."); }
 }
 
+// A clip job: pending for its first status call, then done (or failed, when the prompt says so). `charged` is whether it still holds its reservation.
+type ClipJob = { owner: string; prompt: string; aspect: string; greenScreen: boolean; durationSec: number; shareable: boolean; tags: string[]; asks: number; charged: boolean; libraryId?: string };
+const CLIP_FAIL = "clip-fail";
+// A prompt with this never finishes, so a test can see a client give up waiting.
+const CLIP_STALL = "clip-stall";
+
+// A real one-second MP4, made with ffmpeg. A green-screen one is pure green with a small red box, so keying it out has something to show.
+function makeClip(greenScreen: boolean): Blob {
+  const dir = mkdtempSync(join(tmpdir(), "rk-fakeclip-"));
+  const out = join(dir, "clip.mp4");
+  try {
+    const source = greenScreen ? "color=c=0x00FF00:s=320x180:r=10:d=1,drawbox=x=120:y=50:w=80:h=80:color=0xFF0000:t=fill" : "testsrc=s=320x180:r=10:d=1";
+    execFileSync("ffmpeg", ["-v", "error", "-f", "lavfi", "-i", source, "-c:v", "libx264", "-pix_fmt", "yuv420p", "-movflags", "+faststart", out], { stdio: "pipe" });
+    return { filename: "clip.mp4", contentType: "video/mp4", bytes: new Uint8Array(readFileSync(out)) };
+  } catch (e) {
+    throw new Error(`The fake API could not make its test clip: ffmpeg is needed (${e instanceof Error ? e.message : String(e)}).`);
+  } finally { rmSync(dir, { recursive: true, force: true }); }
+}
+
 export type FakeApi = Awaited<ReturnType<typeof startFakeApi>>;
 
 // An in-memory stand-in for the Reelkit API. It implements every route in the contract.
-export async function startFakeApi(opts: { voiceoverCharLimit?: number; imageLimit?: number; autoApprove?: boolean } = {}) {
+export async function startFakeApi(opts: { voiceoverCharLimit?: number; imageLimit?: number; clipLimit?: number; noClipProvider?: boolean; autoApprove?: boolean } = {}) {
   // token -> the user it belongs to
   const tokens = new Map<string, string>();
   const devices = new Map<string, { userCode: string; approved: boolean; userId: string }>();
@@ -51,15 +76,19 @@ export async function startFakeApi(opts: { voiceoverCharLimit?: number; imageLim
   // real server needs to be, so a client that retries a PUT, or sends a second one, fails here instead of working by luck.
   const uploads = new Map<string, { id: string; contentType: string; bytes: number; used: boolean }>();
   // Totals over every user (for tests of the CLI), and the same counts per user (what a quota is measured against).
-  const usage = { chars: 0, images: 0, pulls: 0, uploads: 0 };
-  const meters = new Map<string, { chars: number; images: number; uploads: number }>();
+  const usage = { chars: 0, images: 0, clips: 0, pulls: 0, uploads: 0 };
+  const meters = new Map<string, { chars: number; images: number; clips: number; uploads: number }>();
   const meter = (userId: string | undefined) => {
     const id = userId ?? "u-test";
-    if (!meters.has(id)) meters.set(id, { chars: 0, images: 0, uploads: 0 });
+    if (!meters.has(id)) meters.set(id, { chars: 0, images: 0, clips: 0, uploads: 0 });
     return meters.get(id)!;
   };
   const resetDate = () => nextMonthStart().toISOString();
-  const limits = { chars: opts.voiceoverCharLimit ?? 10_000, images: opts.imageLimit ?? 30 };
+  const limits = { chars: opts.voiceoverCharLimit ?? 10_000, images: opts.imageLimit ?? 30, clips: opts.clipLimit ?? 5 };
+  const jobs = new Map<string, ClipJob>();
+  // Made on first use and kept for the life of this instance.
+  const clipFiles: Partial<Record<"plain" | "green", Blob>> = {};
+  const clipFile = (green: boolean) => (clipFiles[green ? "green" : "plain"] ??= makeClip(green));
   const ALNUM = "abcdefghijklmnopqrstuvwxyz0123456789";
   const nextId = (prefix: string) => `${prefix}-${Array.from(randomBytes(12), (b) => ALNUM[b % ALNUM.length]).join("")}`;
   const secret = () => randomBytes(16).toString("hex");
@@ -88,7 +117,7 @@ export async function startFakeApi(opts: { voiceoverCharLimit?: number; imageLim
       const m = meter(userId);
       return {
         userId: userId ?? "u-test", handle: handleOf(userId ?? "u-test"),
-        quota: { voiceoverChars: { used: m.chars, limit: limits.chars }, images: { used: m.images, limit: limits.images }, resetsAt: resetDate() },
+        quota: { voiceoverChars: { used: m.chars, limit: limits.chars }, images: { used: m.images, limit: limits.images }, clips: { used: m.clips, limit: limits.clips }, resetsAt: resetDate() },
         // What the user gave the shared library that was accepted: their own items that are published. One waiting for review does not count.
         contributions: [...store.values()].filter((x) => x.owner === userId && x.committed && x.item.visibility === "published").length,
       };
@@ -155,6 +184,37 @@ export async function startFakeApi(opts: { voiceoverCharLimit?: number; imageLim
       if (shareable) store.set(id, { item: { id, kind: "image", title: cut(prompt, 60), description: prompt, tags, meta: { aspect }, visibility: "review" }, file, filename: file.filename, contentType: file.contentType, owner: userId, committed: true });
       return { url: fileUrl(file), ext: "png", contentType: "image/png", ...(shareable ? { libraryId: id } : {}) };
     },
+    clipStart: (input, { userId }) => {
+      if (opts.noClipProvider) throw new Fail(400, "invalid_request", "Clip generation is not available on this server yet.");
+      const m = meter(userId);
+      if (m.clips + 1 > limits.clips) throw new Fail(429, "quota_exceeded", `Clip quota used up. It resets on ${resetDate().slice(0, 10)}.`);
+      m.clips++;
+      usage.clips++;
+      const id = nextId("clip");
+      jobs.set(id, { owner: userId ?? "u-test", prompt: input.prompt, aspect: input.aspect, greenScreen: input.greenScreen, durationSec: input.durationSec, shareable: input.shareable, tags: input.tags, asks: 0, charged: true });
+      return { id, status: "pending" };
+    },
+    clipStatus: ({ id }, { userId }) => {
+      const job = jobs.get(id);
+      if (!job || job.owner !== (userId ?? "u-test")) throw new Fail(404, "not_found", `No clip with id ${id}.`);
+      // The first ask finds it still working; from the second it has ended.
+      if (++job.asks === 1 || job.prompt.includes(CLIP_STALL)) return { id, status: "pending" };
+      if (job.prompt.includes(CLIP_FAIL)) {
+        // A failed clip is not charged: the reservation is given back, once.
+        if (job.charged) { job.charged = false; meter(userId).clips--; usage.clips--; }
+        return { id, status: "failed", message: "The video provider could not make this clip. Change the prompt and try again." };
+      }
+      const file = clipFile(job.greenScreen);
+      if (job.shareable && !job.libraryId) {
+        job.libraryId = nextId("clip");
+        const meta = { aspect: job.aspect, durationSec: job.durationSec, ...(job.greenScreen ? { greenScreen: true } : {}) };
+        store.set(job.libraryId, { item: { id: job.libraryId, kind: "clip", title: cut(job.prompt, 60), description: job.prompt, tags: job.tags, meta, visibility: "review" }, file, filename: file.filename, contentType: file.contentType, owner: userId, committed: true });
+      }
+      return {
+        id, status: "done", url: fileUrl(file), ext: "mp4", contentType: file.contentType, durationSec: job.durationSec, greenScreen: job.greenScreen,
+        ...(job.libraryId ? { libraryId: job.libraryId } : {}),
+      };
+    },
     publicLibrary: ({ q, kind, page }) => {
       // Published items only. Newest first; with a query, best match first and equal matches newest first.
       const want = q ? words(q) : undefined;
@@ -164,7 +224,7 @@ export async function startFakeApi(opts: { voiceoverCharLimit?: number; imageLim
         .filter((x) => x.s.item.visibility === "published" && (!kind || x.s.item.kind === kind) && (!want || x.score > 0))
         .sort((a, b) => b.score - a.score || b.order - a.order).map((x) => x.s);
       const p = page ?? 1, size = 24;
-      const items = all.slice((p - 1) * size, p * size).map((s) => (s.file && PREVIEW_OF_ORIGINAL.includes(s.item.kind) ? { ...s.item, previewUrl: fileUrl(s.file) } : s.item));
+      const items = all.slice((p - 1) * size, p * size).map((s) => (s.file && (PREVIEW_OF_ORIGINAL.includes(s.item.kind) || shortClip(s.item)) ? { ...s.item, previewUrl: fileUrl(s.file) } : s.item));
       return { items, page: p, hasMore: all.length > p * size };
     },
   };
@@ -219,6 +279,8 @@ export async function startFakeApi(opts: { voiceoverCharLimit?: number; imageLim
   return {
     baseUrl: `${origin}/api/v1`,
     usage,
+    // Changes how many clips a user may start from now on, as `clipLimit` does at the start.
+    setClipLimit(n: number) { limits.clips = n; },
     approve(userCode: string, userId = "u-test") { for (const d of devices.values()) if (d.userCode === userCode) { d.approved = true; d.userId = userId; } },
     // A ready token, for tests that are not about logging in.
     login(userId = "u-test") { const t = nextId("tok"); tokens.set(t, userId); return t; },