reelkit-cli 0.1.3 → 0.3.0

package/README.md

@@ -53,12 +53,13 @@ reelkit render
 | `reelkit whoami` | Account, quota, contributions |
 | `reelkit install` | Install the skill into your coding agents (`--agent <id>`, `--force`) |
 | `reelkit init [name]` | Set up a video project, in a new folder named after it |
-| `reelkit assets upload <file>` | Add one of your own files (private unless `--share`) |
+| `reelkit assets upload <file>` | Add one of your own files (private unless `--share`); `--green` or `--cutout` also makes a transparent copy of a video (`--cutout` sends it to the server) |
 | `reelkit assets search "<query>"` | Search the shared library by meaning; each result shows a match percentage |
 | `reelkit assets pull <id>` | Download a library item (a component lands in `src/`) |
 | `reelkit assets voices` | List narration voices |
 | `reelkit assets voiceover` | Record narration from `plan.json` |
 | `reelkit assets gen image` | Generate a scene's illustration |
+| `reelkit assets gen clip` | Generate a scene's video clip, or a green-screen one keyed to a transparent video, or one with the subject cut out on the server |
 | `reelkit plan check` | Validate `plan.json` |
 | `reelkit check` | Check the composition without rendering |
 | `reelkit preview` | Two test frames per scene |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "reelkit-cli",
-  "version": "0.1.3",
+  "version": "0.3.0",
   "description": "CLI and Claude skill for making short-form video with a shared asset library.",
   "license": "MIT",
   "author": "Daniel Livshin",

package/skill/SKILL.md

@@ -23,13 +23,13 @@ Find out what the video is about and collect what the user has. Ask for nothing
 
 For each file the user gives, look at it, then register it with a description of what it shows:
 `reelkit assets upload ./logo.png --describe "Acme logo, white wordmark on blue"`
-Add `--footage` for a video the motion design should be laid over. User files stay on this machine.
+Add `--footage` for a video the motion design should be laid over. User files stay on this machine. A video of someone or something on a plain background can be made transparent: `--green` keys a green background locally for free, and `--cutout` removes any background on the server (the video is sent to Reelkit, so ask first; see `reference/clips.md`).
 
 ### 2. Look
 Read `reference/styles.md` and agree the video's look with the user: one of its looks, or their own. If they already described what they want, match it and confirm in one sentence. Ask anything still open in one message, each question with a default. If any on-screen text will be Hebrew, read `reference/hebrew-rtl.md` now: it changes how words may enter and how lines are written.
 
 ### 3. Plan
-Read `reference/scriptwriting.md` and `reference/scene-treatments.md`. Write the chosen look into the first scene's `notes`. Run `reelkit assets voices` and choose a voice that fits the idea, audience and language.
+Read `reference/scriptwriting.md` and `reference/scene-treatments.md` (and `reference/clips.md` if any scene might be a video clip). Write the chosen look into the first scene's `notes`. Run `reelkit assets voices` and choose a voice that fits the idea, audience and language.
 
 Write `plan.json`:
 
@@ -47,6 +47,7 @@ Write `plan.json`:
       "treatment": "motion-graphic",
       "onScreenText": ["up to three short items"],
       "imagePrompt": null,
+      "clipPrompt": null,
       "shareable": false,
       "imageTags": [],
       "userAssetIds": [],
@@ -57,8 +58,9 @@ Write `plan.json`:
 ```
 
 - 3 to 8 scenes. Scene ids are short, unique, lowercase with dashes.
-- `treatment` is `motion-graphic`, `illustration` or `footage-overlay`. With footage, `mode` is `"footage"` and every scene is `footage-overlay`; otherwise `mode` is `"motion"` and no scene is.
+- `treatment` is `motion-graphic`, `illustration`, `clip` or `footage-overlay`. With footage, `mode` is `"footage"` and every scene is `footage-overlay`; otherwise `mode` is `"motion"` and no scene is.
 - `imagePrompt` is set only for `illustration` scenes, with 3 to 6 `imageTags`. Set `shareable` to true only when the prompt is fully generic: no brand, product, person or detail specific to this user.
+- `clipPrompt` is set only for `clip` scenes (a generated or reused video clip is the scene's picture; the rest of the scene uses `imageTags` and `shareable` as an illustration does). Clips are scarce: most videos have none or one or two.
 - `userAssetIds` lists the ids of the user's files shown in that scene.
 - `pace` is `slow`, `normal` or `fast`.
 
@@ -69,11 +71,13 @@ Run `reelkit plan check`; it prints the estimated length to tell the user. Fix e
 ### 4. Voice
 `reelkit assets voiceover --all`. It records each scene and prints the real length of the video. If the user wants a different voice, change `voiceId` in `plan.json` and run it again with `--redo`.
 
-### 5. Images
+### 5. Images and clips
 Read `reference/asset-reuse.md`. For each illustration scene, search first:
 `reelkit assets search "<what the scene needs>" --kind image`
 Each result starts with a match percentage: how likely it is good enough to reuse. Pull the best result (`reelkit assets pull <id> --scene <sceneId>`) when its match is 60% or more and, reading its description, it fits the scene. Otherwise generate: `reelkit assets gen image --scene <sceneId>`. Never give two scenes the same image.
 
+For each clip scene, read `reference/clips.md`, then search `reelkit assets search "<what the clip shows>" --kind clip` and pull a 60% match (`reelkit assets pull <id> --scene <sceneId>`), or generate: `reelkit assets gen clip --scene <sceneId>` (add `--green` for a green-screen subject). It waits for the clip, which takes minutes; check what is left with `reelkit whoami`.
+
 ### 6. Composition
 Read `reference/kit.md`, `reference/remotion-composition.md`, `reference/motion-design.md` and `reference/captions.md`.
 

package/skill/reference/clips.md

@@ -0,0 +1,98 @@
+---
+name: clips
+description: Use when a scene might be a generated or reused video clip - B-roll that shows what the narration says, or a green-screen subject placed over your own graphics - including how to write the prompt, what a clip costs and how to layer it.
+---
+
+# Video clips
+
+A clip is a few seconds of generated video. It is the most expensive thing Reelkit makes: it takes minutes and each one uses part of a small monthly allowance. Use it where nothing else carries the point.
+
+## Is a clip worth it?
+- Use a clip when the sentence is about something that moves and cannot be drawn well in code or as one still: a person doing something, an animal, weather, a crowd, water, a place seen from a moving camera.
+- Do not use a clip for numbers, lists, steps, UI or anything with text (motion graphics do that better and cost nothing), nor for a concept one illustration shows (a still costs far less).
+- Most videos need none, or one or two. Plan a clip scene with `"treatment": "clip"` and a `clipPrompt`; give it `imageTags` (3 to 6) like an illustration, and set `shareable` by the same rule as images.
+- Check what is left before planning more: `reelkit whoami` shows `Clips: used/limit`, and when it resets. Clips are limited per month. If the allowance is gone, say so to the user and use an illustration or a motion graphic instead; do not work around it.
+
+## Search first
+`reelkit assets search "<what the clip shows>" --kind clip`. Each result starts with a match percentage. Pull the best one (`reelkit assets pull <id> --scene <sceneId>`) when it is 60% or more and its description fits the sentence; it is free and instant. Otherwise generate: `reelkit assets gen clip --scene <sceneId>`. Never give two scenes the same clip.
+
+## Writing a clip prompt
+- One subject, one action. "A fisherman pulls a net from a grey sea" is a clip; "a fisherman, his family at home, and the market" is three.
+- Say the camera: "slow push-in", "handheld tracking shot", "static wide shot", "slow pan left".
+- Say the light and the mood: "low golden sun", "overcast, soft light", "neon night".
+- No text, letters, logos, signs or screens with content in the picture: the generator draws them wrong. Add the words yourself in the composition.
+- It is 5 seconds by default (`--seconds 10` doubles the cost to the allowance only when the action truly needs it). Describe what happens in 5 seconds, not a story.
+- The scene runs as long as its narration. A clip shorter than the scene holds its last frame, so end the action on a moment that can sit still.
+- Generation takes up to several minutes. The command waits and prints a line every 30 seconds. If it gives up after ten minutes it prints the id and a `reelkit assets gen clip --resume <id> --scene <sceneId>` line: run it; it does not charge again.
+
+## B-roll rules
+- The clip shows exactly what the sentence says, at the moment it says it. If the sentence is abstract, the clip is the wrong tool.
+- Never the same clip twice in a video, and no two clip scenes in a row that look alike.
+- Speech never runs more than 3 seconds without something changing on screen: a new clip, a new text beat, a cut to a graphic. A 5-second clip under a 9-second scene needs a text change or a second visual in the middle.
+- Text over a clip must stay legible. Use `<ClipLayer src={urls[s.clipKey]} dim={0.25} />` to darken it, keep headlines in the calm part of the picture, and look at the preview frames for contrast on both the early and the late frame.
+
+## Green screen
+A green-screen clip is a subject filmed on a flat pure green background. It is for a person or an object that you place over something of your own: a presenter pointing at the user's screenshot, a character beside your title, a product over a gradient.
+- Generate it with `reelkit assets gen clip --scene <sceneId> --green`. The original is saved as `assets/clip-<sceneId>.mp4` and the green is keyed out locally into `assets/clip-<sceneId>.webm`, a video with a transparent background. The manifest has both: `s.clipKey` and `s.clipKeyedKey`. Use the `.webm`.
+- Prompt it as a subject, not a scene: "A woman in a blue jacket, waist up, waves and then points to her left. Flat, evenly lit, pure green background (#00FF00)." Keep the whole subject inside the frame with room around it; no green anywhere on the subject (clothes, props, eyes); no shadows, floor or reflections on the background; no camera move, because keying a moving frame edge flickers.
+- If the keyed edges look green or ragged in the preview, the prompt was at fault: regenerate once with a flatter green and an evenly lit subject, using `--redo`. A clip left unkeyed (the command says so) is keyed again for free by running the same command again.
+- Layer it above the scene's background and below the captions:
+
+```tsx
+import { Img } from "remotion";
+import { Captions, KeyedClip, SceneFrame } from "reelkit/kit";
+
+<SceneFrame from={s.startFrame} durationInFrames={s.durationFrames}>
+  <Img src={urls[s.userAssetKeys[0]]} style={{ width: "100%", height: "100%", objectFit: "cover" }} />
+  {s.clipKeyedKey ? <KeyedClip src={urls[s.clipKeyedKey]} x={0.72} y={1} scale={0.7} /> : null}
+  <Captions words={s.words} />
+</SceneFrame>
+```
+
+- A green-screen scene still has a `clip` treatment and a `clipPrompt`, with the background and the type in the composition. A library clip with the same use comes keyed too: `reelkit assets pull <id> --scene <sceneId>` writes the `.webm` beside the `.mp4` when the clip is marked green screen.
+- It costs the same as any clip and is harder to get right: at most one or two green-screen clips per video.
+
+## Your own green-screen footage
+If the user filmed themselves or a product in front of a green background, register it with `reelkit assets upload ./me.mp4 --green --describe "what it shows"`. The original is kept and a copy with the green made transparent, sound included, is written beside it; the command prints the `urls[...]` path of that copy. The key colour is read from the corners of the frame, so the green does not have to be perfect, but it does have to fill the corners: a video whose corners are not green is refused. For a talking presenter, leave the copy's sound on (`muted={false}`) and do not also play the original.
+
+## No green screen: cut the subject out
+Use this when the footage or the clip is not on a green background but the subject has to be layered over something else.
+- If the background is green, use `--green` instead: it is free and runs on this machine. A cutout is the fallback.
+- `reelkit assets upload ./me.mp4 --cutout --describe "what it shows"` sends the video to the Reelkit server, where a GPU model separates the subject, and writes a transparent `.webm` (sound kept) beside the original; the command prints its `urls[...]` path. It takes about a minute. If it times out, the command prints the line that continues it (`--resume <id>`); that neither uploads nor charges again.
+- The video leaves the user's machine, so ask the user before cutting out their footage.
+- At most 20 seconds of video per cutout, counted in whole seconds against a monthly quota; `reelkit whoami` shows what is left.
+- Use the result exactly like a keyed clip: `KeyedClip` with the `.webm`, and the Layers section below applies.
+- Edges are best when the subject is well lit and clearly different from the background. Hair and fast motion can shimmer: check both preview frames.
+- A generated clip can be cut out in one step: `reelkit assets gen clip --scene <sceneId> --cutout` makes the clip as usual, then cuts it out and records it as `s.clipKeyedKey`. If only the cutout fails, run the same command again: it does the cutout without paying for a new clip.
+
+## Layers
+A keyed clip is what makes real depth possible: things can sit behind the subject as well as in front. Build every such scene bottom to top, in this order, and keep to it:
+
+1. **Background**: a mesh, a picture, the user's screenshot, a clip.
+2. **Behind the subject**: the big title, a chart, a logo. The subject will cover part of it, which is the point: a headline that a person stands in front of reads as being in the room.
+3. **The subject**: the keyed clip.
+4. **In front of the subject**: small labels, a lower third, a callout pointing at what the subject holds.
+5. **Captions**, always last.
+
+- Text behind the subject must stay readable with part of it hidden: make it large, keep it to one to three words, and place it so the subject covers the middle or one end, never the first letters of a word.
+- Give the subject a soft contact shadow or a slightly darker patch on the background where it stands, or it floats.
+- Light direction should agree: if the clip is lit from the left, put the brighter side of the background on the left.
+- Keep the subject's size steady across the scene. Move the layers around it, a slow push on the background and a smaller one on the subject, to get depth.
+- Check both preview frames for a green fringe on the edges and for the subject hiding something that must be read.
+
+```tsx
+import { AbsoluteFill } from "remotion";
+import { BgMesh, Captions, KeyedClip, SceneFrame, fonts } from "reelkit/kit";
+
+<SceneFrame from={s.startFrame} durationInFrames={s.durationFrames}>
+  <BgMesh bg="#101014" hero="#C6F135" />
+  <AbsoluteFill style={{ alignItems: "center", paddingTop: "18%", fontFamily: fonts.display, fontWeight: 900, fontSize: 260, color: "#ffffff" }}>LAUNCH</AbsoluteFill>
+  {s.clipKeyedKey ? <KeyedClip src={urls[s.clipKeyedKey]} x={0.5} y={1} scale={0.85} /> : null}
+  <Captions words={s.words} />
+</SceneFrame>
+```
+
+## Rules
+- Search before generating, and generate once per scene. Do not regenerate to chase small improvements.
+- `--share` only for a generic clip with nothing specific to this user; the user's own files stay private.
+- Never claim a clip shows something it does not: look at the frames.

package/skill/reference/kit.md

@@ -34,6 +34,14 @@ FootageLayer { src: string; muted?: boolean; dim?: number }
   Full-frame user footage. Place it once, outside the scenes, as the bottom layer. dim (0-1) darkens it for legibility.
   <FootageLayer src={urls[manifest.footageKey]} dim={0.25} />
 
+ClipLayer { src: string; muted?: boolean; dim?: number }
+  Full-frame generated or library video clip as a scene's picture (treatment "clip"). Place it inside the scene's SceneFrame as the bottom layer, so it starts with the scene. When the scene is longer than the clip it holds the last frame; it never loops. dim (0-1) darkens it for legibility.
+  <ClipLayer src={urls[s.clipKey]} dim={0.2} />
+
+KeyedClip { src: string; x?: number; y?: number; scale?: number; muted?: boolean }
+  A green-screen clip with the green keyed out (the transparent .webm, s.clipKeyedKey). Lay it over your own background or screenshot, below the captions. x and y place its bottom centre as fractions of the frame (default x 0.5, y 1: bottom centre); scale is its width as a fraction of the frame width (default 1). Holds its last frame when the scene is longer.
+  <KeyedClip src={urls[s.clipKeyedKey]} x={0.5} y={1} scale={0.8} />
+
 BgMesh { bg: string; hero: string; accent?: string }
   The bottom layer of the video: the base colour with two slow-drifting soft colour fields. Use it instead of a flat background.
   <BgMesh bg={palette.bg} hero={palette.hero} accent={palette.accent} />

package/skill/reference/remotion-composition.md

@@ -48,7 +48,7 @@ Each `scenes` entry:
 - Never hard-code a duration that the manifest already gives. Look scenes up by id when a scene needs bespoke content.
 
 ## Media
-- Reference media only as `urls[path]`, where `path` is the file's project path, using the paths in the manifest: `s.voiceoverKey`, `s.imageKey`, `s.userAssetKeys`, `manifest.footageKey`. Never hard-code a URL.
+- Reference media only as `urls[path]`, where `path` is the file's project path, using the paths in the manifest: `s.voiceoverKey`, `s.imageKey`, `s.clipKey`, `s.clipKeyedKey`, `s.userAssetKeys`, `manifest.footageKey`. Never hard-code a URL.
 - A scene with narration has `s.voiceoverKey`: give it exactly one `<Voiceover src={urls[s.voiceoverKey]} />` inside its SceneFrame, and `<Captions words={s.words} />` unless the notes say otherwise.
 - Footage mode (`manifest.footageKey` is set): one `<FootageLayer>` at the bottom, outside the scenes.
 - User images: `<Img src={urls[key]} />` from remotion.
@@ -56,8 +56,9 @@ Each `scenes` entry:
 ## Layer stack (every video, bottom to top)
 1. Background: `<BgMesh>` from the kit, or `<FootageLayer>` in footage mode. Never a flat solid colour.
 2. Images: `<KenBurnsImage>` inside its scene. Every still image moves; alternate `direction` between consecutive image scenes.
-3. Graphics and type, inside each scene's SceneFrame.
-4. `<Grade>`, then `<Grain>`, then `<Vignette>`, once, at the very top of the video outside the scenes. In footage mode skip Grade.
+3. Clips: `<ClipLayer>` inside its scene (a clip scene's picture); a green-screen `<KeyedClip>` above the scene's background and below its type and captions. See `reference/clips.md`.
+4. Graphics and type, inside each scene's SceneFrame.
+5. `<Grade>`, then `<Grain>`, then `<Vignette>`, once, at the very top of the video outside the scenes. In footage mode skip Grade.
 
 ## Shared overlays (optional)
 - The library has ready-made overlay clips (`reelkit assets search "<description>" --kind overlay`): film marks, HUD frames, corner marks. They are white on black and are laid over the video with `<ScreenOverlay>`.

package/skill/reference/scene-treatments.md

@@ -5,9 +5,10 @@ description: Use when choosing each scene's visual treatment and writing image p
 
 # Choosing scene treatments
 
-## The three treatments
+## The four treatments
 - **motion-graphic** - animated text, numbers, shapes and icons drawn in code. The default. Best for hooks, lists, statistics, steps and calls to action.
 - **illustration** - one generated picture behind the text. Use when a concrete image carries the point better than type (an object, a place, a mood). At most half the scenes; each one costs money.
+- **clip** - a few seconds of generated or reused video as the scene's picture (B-roll), or a green-screen subject over your own graphics. Needs a `clipPrompt`. Rare and costly: see `reference/clips.md` before using it.
 - **footage-overlay** - text and graphics over the user's own video. Required for every scene when footage was supplied, and never used otherwise.
 
 ## Image prompts (illustration scenes only)

package/src/cli.ts

@@ -1,6 +1,6 @@
 import { Command } from "commander";
 import { ApiFailure } from "./api/client";
-import { assetsGenImage, assetsPull, assetsSearch, assetsUpload, assetsVoiceover, assetsVoices } from "./commands/assets";
+import { assetsGenClip, assetsGenImage, assetsPull, assetsSearch, assetsUpload, assetsVoiceover, assetsVoices } from "./commands/assets";
 import { authLogin, authLogout, whoami } from "./commands/auth";
 import { check, preview, render } from "./commands/build";
 import { init } from "./commands/init";
@@ -67,19 +67,28 @@ program.command("whoami").description("Show your account, quota and contribution
 program.command("init [name]").description("Set up a video project (in a new folder named after it, or in this folder)").option("--aspect <aspect>", "9:16, 16:9 or 1:1").action(run(init));
 
 const assets = program.command("assets").description("Find, add and generate the files a video needs");
-assets.command("upload <file>").description("Add one of your own files to this project (private unless --share)")
-  .option("--describe <text>", "what the file shows").option("--footage", "use this video as the footage to overlay")
+assets.command("upload <file>").description("Add one of your own files to this project (private unless --share; with --cutout the video is sent to the Reelkit server to be processed)")
+  .option("--describe <text>", "what the file shows").option("--footage", "use this video as the footage to overlay").option("--green", "the video is a subject on a green background: also make a copy with the green transparent (free, on this machine)")
+  .option("--cutout", "remove any background: the video is sent to the Reelkit server to be processed and a copy with a transparent background comes back (up to 20 seconds, counts against a monthly quota)")
+  .option("--resume <id>", "with --cutout: keep waiting for a cutout already started, without uploading or paying again")
   .option("--share", "also send it to the shared library").option("--kind <kind>", "library kind when sharing").option("--tags <tags>", "comma-separated tags when sharing")
-  .action(run(assetsUpload));
+  // Called with the arguments named: commander adds its own command object last, which must not be taken for the function's test hooks.
+  .action(run((ctx, file: string, opts: Parameters<typeof assetsUpload>[2]) => assetsUpload(ctx, file, opts)));
 assets.command("search <query>").description("Search the shared library by meaning; each result shows how well it fits").option("--kind <kind>", "image, overlay, sfx, music, component or clip").option("--limit <n>", "how many results")
   .action(run(assetsSearch));
-assets.command("pull <id>").description("Download a library item into this project").option("--scene <sceneId>", "use it as this scene's image").option("--force", "replace a component file that already exists")
-  .action(run(assetsPull));
+assets.command("pull <id>").description("Download a library item into this project").option("--scene <sceneId>", "use it as this scene's image or clip").option("--force", "replace a component file that already exists")
+  .action(run((ctx, id, opts) => assetsPull(ctx, id, opts)));
 assets.command("voices").description("List the narration voices").action(run(assetsVoices));
 assets.command("voiceover").description("Record the narration from plan.json").option("--scene <sceneId>", "one scene").option("--all", "every scene").option("--redo", "record again even if it exists")
   .action(run(assetsVoiceover));
-assets.command("gen").description("Generate an asset").command("image [prompt]").description("Generate a scene's illustration; the prompt defaults to the plan's")
+const gen = assets.command("gen").description("Generate an asset");
+gen.command("image [prompt]").description("Generate a scene's illustration; the prompt defaults to the plan's")
   .requiredOption("--scene <sceneId>", "the scene it is for").option("--redo", "generate a new image even if the scene has one").action(run(assetsGenImage));
+gen.command("clip [prompt]").description("Generate a scene's video clip (takes minutes); the prompt defaults to the plan's clipPrompt")
+  .requiredOption("--scene <sceneId>", "the scene it is for").option("--green", "film it on green and key the green out into a transparent .webm")
+  .option("--cutout", "make the clip normally, then cut the subject out on the server (the video is sent to the Reelkit server to be processed; counts against a monthly quota)").option("--seconds <n>", "5 or 10 (default 5)")
+  .option("--share", "also add it to the shared library (only a generic clip)").option("--redo", "generate a new clip even if the scene has one").option("--resume <id>", "keep waiting for a clip already started, without paying again")
+  .action(run((ctx, prompt, opts) => assetsGenClip(ctx, prompt, opts)));
 
 program.command("plan").description("Work with plan.json").command("check").description("Validate plan.json and list what to fix or improve").action(run(planCheck));