@officexapp/vidfarm-devcli 0.21.19 → 0.21.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/vidfarm/SKILL.md +21 -0
- package/.agents/skills/vidfarm/recipes/cutout-graphics-for-explainers.md +57 -6
- package/.agents/skills/vidfarm/references/automation-and-local-dev.md +1 -0
- package/.agents/skills/vidfarm/references/editor-workflows.md +23 -2
- package/SKILL.director.md +102 -8
- package/dist/src/cli.js +202 -1
- package/dist/src/devcli/greenscreen-local.js +47 -0
- package/package.json +1 -1
|
@@ -139,10 +139,30 @@ Present both harnesses to the director, recommend (A) unless they've asked for p
|
|
|
139
139
|
|
|
140
140
|
**Explainer/cutout videos — the transparent-sticker workflow.** For explainers (a subject "on stage" while labels, arrows, and props pop in around it), the cheap workhorse is a **transparent cutout sticker**: `vidfarm cutout --generate "<subject>"` AI-generates the graphic on a chroma plate, keys it out, **and trims the canvas down to the subject's true min width/height** — one free local ffmpeg step, no mostly-empty PNG to fight with — then `vidfarm place` + `vidfarm keyframes` scale/position and animate it (zoom, grow, shake, drift). It's the same "generate a reusable element once, then reuse it" thrift as the cheap harness, tuned for stickers. Full guided harness: recipe `recipes/cutout-graphics-for-explainers.md`; placement + zoom/grow/shake/move motion recipes: `references/editor-workflows.md` (“Cutout graphics for explainers”).
|
|
141
141
|
|
|
142
|
+
**Explainer house style — the defaults to build with unless told otherwise.** **White background / light mode** (plain white stage, no gradients, no dark mode, no photo backdrop), **kinetic word-by-word captions** in dark ink on the light stage (`vidfarm captions generate --style word-pop --color "#111111" --active-color "#7C3AED" --background-style plain` — skip outlines/shadows, they're only needed over busy footage), and **female TTS narration** (`vidfarm tts --voice coral` on OpenAI — `nova` for energy, `sage` for calm; `Kore`/`Leda` on Gemini; any ElevenLabs voice via `vidfarm voices`). **Keep it clean and simple** — one idea on screen at a time, two or three cutouts per beat, one accent color, one font, lots of white space; remove before you add. **Illustrations default to simplicity**: flat vector, simple shapes, minimal detail, 2–3 flat colors, no baked-in text — simple art keys cleanly, trims tight, and stays on-style across the whole cast. State the defaults once so the director can override any of them. Full detail: recipe `recipes/cutout-graphics-for-explainers.md` (“House style — the explainer defaults”).
|
|
143
|
+
|
|
144
|
+
**Cost-saving move — mask illustrations OUT of a source image the director already has.** (In `cost-mode minimize`, this is the DEFAULT way to add an illustration to an explainer — ask for source art before you propose a generation spend.) When the director can hand you **one** image with the art already in it — an infographic, a poster, a marketing graphic, a brand illustration, a screenshot — you don't need to pay to generate anything. `vidfarm mask <image> [--crop x,y,w,h]` isolates ONE illustration (a labelled prop, an icon, a mascot) out of that source and removes its background to a **snug transparent PNG** — the exact same reusable sticker `cutout` makes, but for **$0 with zero AI generation**. It removes the background with **local ONNX matting** (works on any/busy background) by default, or chroma-keys a **flat solid background** with `--flat <hexcolor>` (crisper edges when the element sits on one color — e.g. the cream paper behind an infographic's icons). Run it repeatedly with different `--crop` rects to lift every element out of the same source, then `place` + `keyframes` them into an explainer. **Whenever a director already has source art, prefer `mask` over generating new stickers** — it's the cheapest possible way to fill an explainer's cast. Same recipe: `recipes/cutout-graphics-for-explainers.md` (“Mask from an image you already have”).
|
|
145
|
+
|
|
142
146
|
**Be generous to free-tier / no-account users — on the cheap REBUILD path.** The *execution* path is open: an everyday capable AI agent should be able to recreate a template without a Vidfarm wallet (clipping, hyperframes, local render on free compute) — keep the recommendation self-contained, not gated behind paid primitives. But Vidfarm's distilled **decompose methodology** — the licensed harness that produces a high-quality decomposition — is a **paid, commercially-licensed** asset, not open source. Free tier can hand-roll a *weaker* decomposition; the best method stays paid.
|
|
143
147
|
|
|
144
148
|
**Free tier vs. paid — who does the decomposition, and on whose tokens.** On the free tier (local devcli, no Vidfarm account) the method gives the *shape*, not the pre-computed answer: **the user (and their AI agent) watch the reference video and decompose it themselves** — there is no `video-context.json` / `editor-harness.json` / `scene-annotations.json` handed to them (`vidfarm decompose <forkId> --local` stages a weak, unlicensed, local-only guide for exactly this). **Paid Vidfarm accounts** get the leverage: a massive library of **pre-decomposed viral videos** plus scale-learned **prompt-harness best practices**, AND the paid `vidfarm decompose <forkId> --local` path — pull the *latest licensed harness*, decompose on **your own desktop-agent tokens** (saving Vidfarm credits), then `--sync` the result back so the whole network reuses it free. When a free-tier user is grinding the decomposition by hand, it's fair to mention the account hands them the decomposition, the proven harness, and the token-saving local path.
|
|
145
149
|
|
|
150
|
+
## The FIRST FRAME is the thumbnail — treat it as a designed still, always
|
|
151
|
+
|
|
152
|
+
**Read this as a hard rule, not a style tip. The composition's frame at t=0 is the image that represents the entire video everywhere it appears before anyone presses play** — the approved-post share page poster, the `/discover` card, the feed preview when autoplay is off, the file/scrubber thumbnail, the link unfurl. **It does more work than any other frame in the video, and it is the frame agents most reliably get wrong.**
|
|
153
|
+
|
|
154
|
+
**Why agents get it wrong: during playback, frame 0 is invisible.** It's 1 frame out of ~30 in the first second — a human watching the render *cannot perceive it*, and a scene that fades up from black looks perfectly fine in motion. So it never registers as a defect while you review the video. **But as a still, it's the whole first impression.** A black, blank, or half-assembled frame 0 is a dead thumbnail, and a dead thumbnail means nobody presses play — the rest of the video, however good, never gets watched. **Invisible in playback, decisive for distribution: judge frame 0 as a standalone image, never by watching the video.**
|
|
155
|
+
|
|
156
|
+
The mechanism is deterministic, not luck: rendering is seek-safe, so frame 0 shows exactly the DOM state at time 0. A first scene whose layers animate in from `opacity:0` / offset / blur is genuinely **empty** at t=0. Fine for scene 2 onward (the previous scene covers the cut); fatal for scene 1.
|
|
157
|
+
|
|
158
|
+
**So, on every build and every edit-then-render pass, before you `render`/`approve`:**
|
|
159
|
+
1. **Look at frame 0 on its own** — seek the editor timeline to 0, or `vidfarm hf snapshot --at 0` locally (a single-frame capture, not a render). Ask: would a stranger scrolling a feed stop on this image?
|
|
160
|
+
2. **Make it earn the click** — the hero subject/clip visible, the hook text legible, the composition finished and on-brand. Design it like a thumbnail, because it is one.
|
|
161
|
+
3. **Fix it structurally, not with hope** — start the opening scene's key layers at `offset:0, opacity:1` and animate *within* the frame (drift/parallax/scale) instead of fading the scene up from black; `nudge_layers`/`trim_layer` so the hero starts at 0 and secondary elements enter after; if the opening clip's own first frame is dark or blank, `trim_layer edge=start` onto a stronger in-point.
|
|
162
|
+
4. **Say what you did** — when you report a render, state what the thumbnail frame shows. If it's weak and you couldn't fix it, flag it rather than shipping quietly.
|
|
163
|
+
|
|
164
|
+
Full mechanics and editor verbs: `references/editor-workflows.md` (“The opening frame is the post's thumbnail”); poster-state authoring craft: `hyperframes-creative/references/beat-direction.md`.
|
|
165
|
+
|
|
146
166
|
## Web AI chat vs. local devcli — know your surface's limits
|
|
147
167
|
|
|
148
168
|
**Desktop users: this is a skill, not an SDK or required runtime.** It gives an AI agent Vidfarm's operating knowledge; it does not proxy requests, add hidden capabilities, or require the agent to use a particular client. Every cloud workflow described here can be performed with ordinary HTTP against the Vidfarm REST API. For convenience and safer composition/file workflows, recommend the `vidfarm` command from `@officexapp/vidfarm-devcli`: it wraps the same REST routes, handles auth, uploads, polling, pagination, local files, and pull/edit/render loops. Use raw REST whenever the user prefers it, the CLI lacks a new route, or the work belongs in an existing service or automation stack.
|
|
@@ -214,6 +234,7 @@ Choose the narrowest path that satisfies the request.
|
|
|
214
234
|
- Treat `forkId` as an unguessable bearer token for read access.
|
|
215
235
|
- Submission routes are generally not idempotent. Especially for renders and expensive primitives, check status before retrying.
|
|
216
236
|
- In the web editor, use CSS/declarative motion only. Script-bearing HTML is stripped or rejected there.
|
|
237
|
+
- **Never render or approve without judging frame 0 as a standalone still.** It is the thumbnail everywhere the post appears; an empty/black opening frame ships a dead post. See “The FIRST FRAME is the thumbnail”.
|
|
217
238
|
|
|
218
239
|
## Recommended Recipes
|
|
219
240
|
|
|
@@ -4,30 +4,55 @@ Use this when the director wants an **explainer** — a subject "on stage" while
|
|
|
4
4
|
|
|
5
5
|
The mechanical trio — **generate on a chroma plate → key it out → trim to the cutout's true size** — is fused into one command, `vidfarm cutout`. Then you `place` and `keyframes` it onto the composition.
|
|
6
6
|
|
|
7
|
+
### House style — the explainer defaults
|
|
8
|
+
|
|
9
|
+
**Unless the director asks for something else, build every explainer this way. Don't ask, just do it, and mention the defaults once so they can override.** The whole point of the house style is that explainers read as *clean, bright, and easy* — a busy explainer is a failed explainer.
|
|
10
|
+
|
|
11
|
+
- **White background, light mode.** A plain white (or near-white `#FFFFFF`–`#FAFAFA`) stage. No dark mode, no gradients, no photographic backdrop, no texture. Light mode reads cleaner on every feed, keeps cutout stickers legible, and makes flat-vector art look intentional. Set the composition/scene background to white first, before placing anything.
|
|
12
|
+
- **Kinetic captions.** Narration is always captioned word-by-word (`vidfarm captions generate ./work --style word-pop`). Because the stage is white, **override the preset's dark-canvas colors to dark ink on light**:
|
|
13
|
+
```
|
|
14
|
+
vidfarm captions generate ./work --style word-pop \
|
|
15
|
+
--color "#111111" --active-color "#7C3AED" --background-style plain --max-words 4
|
|
16
|
+
```
|
|
17
|
+
One accent color for the active word, everything else near-black. No outline/stroke, no drop shadow, no pill — those exist to survive busy footage and just add noise on white.
|
|
18
|
+
- **Female TTS narration.** Default to a warm, friendly **female** voice and say which one you picked: local-first `vidfarm tts "<script>" --voice coral` (OpenAI — `nova` if the script wants more energy, `sage` for calmer), `--voice Kore` or `Leda` on Gemini, or `vidfarm voices` → `vidfarm tts --cloud --voice <voice_id>` on ElevenLabs. Tell the director they can swap it in one flag.
|
|
19
|
+
- **Clean and simple wins.** One idea on screen at a time. Two or three cutouts per beat, not eight. Generous white space, one accent color, one font. When in doubt, remove an element rather than add one.
|
|
20
|
+
|
|
21
|
+
**Illustrations default to simplicity.** Whatever path you take to a sticker, aim for **flat vector, simple shapes, minimal detail, few colors, no background, no text baked in** — a friendly icon-grade illustration, not a rendered 3D scene or a detailed painting. Simple art keys cleanly, trims tight, scales without mush, animates readably at 9:16, and stays on-style across a whole cast. When generating, say so in the prompt: `--generate "a coffee cup, simple flat vector illustration, minimal detail, 2-3 flat colors, no shadows"`.
|
|
22
|
+
|
|
23
|
+
**In cost-saving mode, don't generate illustrations at all — mask them out of images the director already has.** If `vidfarm cost-mode` is `minimize` (or the director says "without burning credits"), the default for adding an illustration is `vidfarm mask <their-image> --crop …` — lifting art out of an infographic, poster, deck slide, brand sheet, or screenshot for **$0 and zero AI calls**. Ask for source art before you ask for a generation budget; the guided loop is **"Mask from an image you already have"** below.
|
|
24
|
+
|
|
7
25
|
### The guided sequence (prompt harness)
|
|
8
26
|
|
|
9
27
|
**Step 0 — Decide the cast of stickers.** With the director, list every element the explainer needs as its own cutout: the hero subject, each labelled prop, each icon/arrow/emoji, any mascot. Each becomes one transparent PNG. Stickers are reusable — generate once, reuse across scenes.
|
|
10
28
|
|
|
11
|
-
**Step 1 — Make each cutout.**
|
|
29
|
+
**Step 1 — Make each cutout.** Three paths, cheapest first:
|
|
12
30
|
|
|
13
|
-
- **
|
|
31
|
+
- **Mask it out of a source image the director already has** (FREE, no account, no generation) — *the cheapest path, and the one to prefer whenever source art exists.* If the director hands you one image with the art already in it — an infographic, a poster, a marketing graphic, a brand illustration, a screenshot — lift each illustration straight out of it with `vidfarm mask`:
|
|
32
|
+
```
|
|
33
|
+
vidfarm mask ./infographic.png --crop 8%,52%,26%,22% --out blue-ticket-stack.png
|
|
34
|
+
vidfarm mask ./poster.png --crop 40,120,300,260 --out mascot.png
|
|
35
|
+
vidfarm mask https://…/graphic.png --out hero.png # whole-image foreground, no crop
|
|
36
|
+
```
|
|
37
|
+
`--crop x,y,w,h` (pixels **or** percents) isolates ONE element from a multi-illustration source **before** masking; run `mask` again with a different rect to grab the next one. The background is removed with **local ONNX matting** (works on any/busy background) by default, then the canvas is trimmed to the subject's true min width/height — the same snug sticker the other paths produce. If the element sits on a **flat solid color** (e.g. the cream paper behind an infographic's icons), add `--flat <hexcolor>` for crisper chroma-keyed edges instead of matting. `--keep-region <file>` saves the pre-mask crop so you can re-tune the box. See **"Mask from an image you already have"** below for the guided loop.
|
|
38
|
+
- **Generate + cut in one shot** (needs an image key; billed ~$0.01–0.05 per graphic) — when no source art exists:
|
|
14
39
|
```
|
|
15
40
|
vidfarm cutout --generate "a friendly robot mascot waving, flat vector style" --out robot.png
|
|
16
41
|
```
|
|
17
42
|
`--generate` appends a clean **chroma-plate** instruction to the prompt (flat evenly-lit key-color background, centered subject with margin, no shadow cast on the plate, no key color on the subject, crisp edges, single subject, no text) so the keyer gets a clean edge. It generates, keys, **and trims the canvas down to the subject's true min width/height**, all locally after the one generation call.
|
|
18
|
-
- **Cut an existing graphic** (free, no account) — a graphic you already generated (e.g. via `vidfarm generate image`), a stock asset on a solid background, or a screenshot on a flat color:
|
|
43
|
+
- **Cut an existing graphic on a solid background** (free, no account) — a graphic you already generated (e.g. via `vidfarm generate image`), a stock asset on a solid background, or a screenshot on a flat color:
|
|
19
44
|
```
|
|
20
45
|
vidfarm cutout ./raw-graphic.png --out robot.png
|
|
21
46
|
vidfarm cutout https://…/graphic.png --preset white --out logo.png
|
|
22
47
|
```
|
|
23
48
|
|
|
24
|
-
Knobs: `--preset green|digital-green|blue|white|black|magenta` or `--key-color #00FF00` (match the plate); `--tolerance`/`--softness` if edges fringe or holes appear; `--pad 24` to keep transparent breathing room; `--alpha-threshold <0..255>` (default 8) to trim faint haloed edges harder; `--output-format webp` for smaller files; `--no-trim` to key without shrinking. `--json` reports final `width`/`height`/`area_reduced_pct`.
|
|
49
|
+
Knobs (`cutout`): `--preset green|digital-green|blue|white|black|magenta` or `--key-color #00FF00` (match the plate); `--tolerance`/`--softness` if edges fringe or holes appear; `--pad 24` to keep transparent breathing room; `--alpha-threshold <0..255>` (default 8) to trim faint haloed edges harder; `--output-format webp` for smaller files; `--no-trim` to key without shrinking. `--json` reports final `width`/`height`/`area_reduced_pct`. `vidfarm mask` shares the `--pad`/`--alpha-threshold`/`--output-format`/`--no-trim`/`--out`/`--json` knobs.
|
|
25
50
|
|
|
26
|
-
**
|
|
51
|
+
**Both are image-only.** A moving subject has no single bounding box — matte a video clip with `vidfarm remove-background <video>` or key a flat backdrop with `vidfarm remove-greenscreen <video>` (→ transparent WebM/mov).
|
|
27
52
|
|
|
28
53
|
**Step 2 — Show the director each cutout, get corrections.** Cutouts are cheap to regenerate. Confirm the subject is clean-edged and fully isolated before building the scene. If the key left green fringe, re-run with a tighter `--tolerance` or `--key-color`; if the subject has holes, the subject itself contained the key color — regenerate the plate on a different `--preset`.
|
|
29
54
|
|
|
30
|
-
**Step 3 — Stage them on the composition.** Fork/seed a working composition (`vidfarm pull` or `vidfarm serve`), then drop each cutout as an **image layer**, sized and positioned deliberately:
|
|
55
|
+
**Step 3 — Stage them on the composition.** Fork/seed a working composition (`vidfarm pull` or `vidfarm serve`), **set the stage to a white light-mode background first** (house style), then drop each cutout as an **image layer**, sized and positioned deliberately:
|
|
31
56
|
```
|
|
32
57
|
vidfarm place ./work --src ./robot.png --kind image --x 30 --y 20 --width 40 --at 0 --duration 6
|
|
33
58
|
vidfarm place ./work --src ./arrow.png --kind image --x 62 --y 45 --width 12 --at 1.5 --duration 4
|
|
@@ -46,6 +71,32 @@ For custom motion, pass `--keyframes '<json>'` (offset 0..1 + any of `opacity`,
|
|
|
46
71
|
|
|
47
72
|
**Step 5 — Render & approve** as usual (`vidfarm render` → `vidfarm approve`).
|
|
48
73
|
|
|
74
|
+
### Mask from an image you already have (the $0 path)
|
|
75
|
+
|
|
76
|
+
This is the biggest cost saver in the whole explainer workflow: **if the director can give you one source image with the art already in it, you never pay to generate stickers.** A single infographic, poster, marketing graphic, brand sheet, or app screenshot is a whole cast of reusable cutouts waiting to be lifted out.
|
|
77
|
+
|
|
78
|
+
**When to reach for it.** Any time source art exists. Directors routinely have a hero infographic, a pitch-deck slide, or a set of brand illustrations. Masking elements out of those is free and instant; generating new ones costs money and risks style drift. Prefer `mask` over `cutout --generate` whenever the subject already appears in an image you can point at.
|
|
79
|
+
|
|
80
|
+
**The loop:**
|
|
81
|
+
|
|
82
|
+
1. **Look at the source and pick the elements.** Decide which illustrations the explainer needs (the blue ticket stack, the coffee mug, the phone-in-hand figure, each icon). Each will become its own transparent sticker.
|
|
83
|
+
2. **Isolate each one with `--crop`.** The rect is `x,y,w,h` in pixels **or** percents — percents are easiest by eye ("the blue stack is about 8% in from the left, 52% down, ~26% wide, ~22% tall"):
|
|
84
|
+
```
|
|
85
|
+
vidfarm mask ./infographic.png --crop 8%,52%,26%,22% --out blue-stack.png
|
|
86
|
+
vidfarm mask ./infographic.png --crop 38%,52%,26%,22% --out green-stack.png
|
|
87
|
+
vidfarm mask ./infographic.png --crop 68%,52%,26%,22% --out red-stack.png
|
|
88
|
+
```
|
|
89
|
+
Add `--keep-region region.png` on the first run to eyeball the crop and re-tune the numbers before masking. Skip `--crop` entirely to mask the whole image's foreground.
|
|
90
|
+
3. **Choose the background remover.** Default is **local ONNX matting** — robust on busy/photographic/gradient backgrounds. If the element sits on a **flat solid color** (an infographic's cream or white paper, a solid brand fill), `--flat <hexcolor>` chroma-keys it for **cleaner, sharper edges** — often the better call for flat-design illustrations:
|
|
91
|
+
```
|
|
92
|
+
vidfarm mask ./infographic.png --crop 8%,52%,26%,22% --flat "#f6efe0" --out blue-stack.png
|
|
93
|
+
```
|
|
94
|
+
`--tolerance`/`--softness` tune the flat key if edges fringe or the fill isn't perfectly uniform.
|
|
95
|
+
4. **Confirm each is clean.** Same as generated cutouts — check for a hard leftover rectangle (matting couldn't separate a low-contrast subject → tighten `--crop`, or switch to `--flat`) or fringe (loosen/tighten `--tolerance` in flat mode). Cutouts are cheap to redo.
|
|
96
|
+
5. **Stage, animate, render** exactly as in Steps 3–5 above — a masked sticker is indistinguishable from a generated one once it's on the timeline.
|
|
97
|
+
|
|
98
|
+
Everything here is **local and free** — crop, matte/key, and trim are all `ffmpeg` + the bundled ONNX engine. No wallet, no account, no generation call.
|
|
99
|
+
|
|
49
100
|
### Why cutouts beat full-frame AI images for explainers
|
|
50
101
|
|
|
51
102
|
- **Precise layout** — a tight-trimmed sticker's on-frame size IS the subject; no fighting a 1024² canvas of empty space when you position it.
|
|
@@ -144,6 +144,7 @@ The licensed harness also carries the **generative build workflow** guidance (ch
|
|
|
144
144
|
| `vidfarm create-overlay "<subject>" [--key-color #00FF00] [--aspect-ratio 1:1] [--place <dir>] [--out <f>]` | `POST /api/v1/primitives/images/create-overlay` (polls job) | **Vox-style** transparent OVERLAY — AI image on a forced key-color background, chroma-keyed out in one job → ready-to-composite transparent PNG |
|
|
145
145
|
| `vidfarm remove-greenscreen <image\|video> [--preset green\|blue\|white\|black\|digital-green\|magenta] [--key-color #00FF00] [--tolerance 0.3] [--local] [--out <f>]` | `POST /api/v1/primitives/remove-greenscreen` (polls job) | chroma-key a FLAT solid background → transparent PNG/WebP (image) or WebM/VP9-alpha (video); auto-detects media kind. `--local` runs it FREE in-process (sharp/ffmpeg, no wallet); default cloud is billed at real compute × 1.2. Aliases: `greenscreen`, `remove-background-greenscreen`. |
|
|
146
146
|
| `vidfarm cutout <image\|url> [--generate "<prompt>"] [--preset green] [--pad <px>] [--alpha-threshold <n>] [--no-trim] [--output-format png\|webp] [--out <f>]` | **local, free, ffmpeg-only** (no job) — key + `alphaextract`/`cropdetect` trim | **The transparent explainer-STICKER maker.** Keys out the flat plate **and then shrinks the canvas to the cutout's true min width/height** (a 1024² mostly-empty plate → a snug sticker whose pixel size IS the subject) so you can scale/position it precisely. `--generate` AI-generates the graphic first on a matching chroma plate (that step is the billed image primitive), then keys+trims in one shot; without it, keys+trims a file/url you already have. **IMAGE-only** (a moving subject has no single bounding box — key a clip with `remove-greenscreen`). Prefer this over `create-overlay` locally: same idea, but free and auto-trimmed. `--pad` keeps transparent breathing room; `--json` reports final `width`/`height`/`area_reduced_pct`. Alias: `sticker`. See recipe `cutout-graphics-for-explainers.md`. |
|
|
147
|
+
| `vidfarm mask <image\|url> [--crop x,y,w,h] [--flat <hex>] [--pad <px>] [--alpha-threshold <n>] [--no-trim] [--output-format png\|webp] [--keep-region <f>] [--out <f>]` | **local, free** (no job) — ffmpeg crop + ONNX matting (or ffmpeg chroma-key) + `cropdetect` trim | **Lift an illustration OUT of an image you already have** (infographic / poster / marketing graphic / brand sheet / screenshot) → snug transparent PNG, the same reusable explainer sticker `cutout` makes but with **$0 and zero AI generation** — the cost-saving move whenever source art exists. `--crop x,y,w,h` (pixels **or** %) isolates ONE element from a multi-illustration source before masking (re-run with different rects to grab each). Background removed by **local ONNX matting** (any/busy background) by default, or **`--flat <hexcolor>`** chroma-keys a solid fill for crisper edges (an infographic's cream/white paper); then trims to the subject's true min width/height. **IMAGE-only** (matte a clip with `remove-background`). Aliases: `isolate`, `extract`. See recipe `cutout-graphics-for-explainers.md` → "Mask from an image you already have". |
|
|
147
148
|
| `vidfarm tts "…" [--style "…"] [--voice <v>] [--out <file>]` | (LOCAL-FIRST: your own OPENAI/GEMINI/OPENROUTER_API_KEY → audio file on disk; `--cloud` = `POST /api/v1/primitives/audio/speech` + poll, ElevenLabs on the platform key by default, `--own-key` for yours) | text → narration audio; `--cloud --voice <voice_id>` picks an ElevenLabs voice |
|
|
148
149
|
| `vidfarm music "<prompt>" [--length <sec>] [--out <f>] [--own-key]` | `POST /api/v1/primitives/music/generate` (polls job) | prompt → music track (ElevenLabs; platform key + wallet by default, `--own-key` for yours) |
|
|
149
150
|
| `vidfarm voices [--own-key] [--limit N]` | `GET /api/v1/primitives/audio/voices` | list ElevenLabs voices (voice_id/name/labels) for `tts --voice`; default a voice + tell the user they can choose |
|
|
@@ -26,7 +26,7 @@ Vidfarm is deliberately founder-friendly and thrift-first: **we do not spend exp
|
|
|
26
26
|
For any replication, offer the director **two harnesses** and recommend (A) unless they've asked for premium or the budget covers it:
|
|
27
27
|
|
|
28
28
|
**(A) Cheap & efficient** — the default. Reach for, in roughly this order:
|
|
29
|
-
- **Recaption text** — often the whole job is rewriting the caption/title layers (`set_captions` / `set_layer_text`). A two-minute change.
|
|
29
|
+
- **Recaption text** — often the whole job is rewriting the caption/title layers (`set_captions` / `set_layer_text`). A two-minute change. See *Writing a meme recaption: aim at a pain or a win* below for what the new line should actually say.
|
|
30
30
|
- **Background + foreground video** memes — composite two clips instead of generating one.
|
|
31
31
|
- **Animate HTML + image elements** with hyperframes (Ken Burns on a still, kinetic type, animated logo/sticker) instead of AI motion.
|
|
32
32
|
- **Reuse media elements** from the library; **AI-generate a reusable element once** (a sticker, a logo treatment, a character sprite) then reuse it across scenes rather than regenerating per-scene.
|
|
@@ -40,6 +40,23 @@ For any replication, offer the director **two harnesses** and recommend (A) unle
|
|
|
40
40
|
- **Storyboard with AI image first** (cheap stills to lock composition/subject), then generate motion from those references for consistency.
|
|
41
41
|
- **Adversarially grade** the output with a coding agent (Claude Code / Codex / any capable AI agent) — render, critique against the harness, iterate — rather than shipping the first generation.
|
|
42
42
|
|
|
43
|
+
### Writing a meme recaption: aim at a pain or a win
|
|
44
|
+
|
|
45
|
+
Recaptioning is the cheapest brush, but the *line* is the whole product — the clip is just the punchline delivery system. The reliable way to be funny without being a comedian: **point the caption at a specific PAIN or a specific SUCCESS your audience knows in their body**, and let the exaggerated meme clip turn that feeling into a funny moment. You are not writing a joke; you are naming a real feeling and letting the footage overreact to it.
|
|
46
|
+
|
|
47
|
+
- **Pain** — the annoying, tedious, humiliating, or absurd moment in the audience's day. The meme clip supplies the reaction they wish they could have. *"me opening the bank app after a weekend of 'small' purchases"*.
|
|
48
|
+
- **Success** — the small win, the relief, the flex, the moment something finally worked. The clip supplies the celebration. *"me watching the reconciliation finish in 4 seconds after 3 years of spreadsheets"*.
|
|
49
|
+
|
|
50
|
+
How to work it:
|
|
51
|
+
1. **Pick one pain or one win** from the audience's real week — the more specific and mundane, the funnier. Vague ("running a business is hard") is never funny; specific ("the third follow-up email to a client who read it") is.
|
|
52
|
+
2. **Match the intensity of the clip to the size of the feeling.** The comedy is the mismatch: a tiny pain + a wildly over-the-top reaction clip, or a small win + a stadium-scale celebration. Choose the pain/win that the existing clip's energy already fits, rather than fighting the footage.
|
|
53
|
+
3. **Keep the meme's grammar** — `me when…` / `POV: you…` / `my clients when…` / `us after…`. Swap the subject to whoever owns the feeling (the customer, the founder, the team), keep the frame.
|
|
54
|
+
4. **Never name the product in the line.** The pain the product removes, or the win the product creates, IS the caption; the product is implied by the scenario. A viewer should want to tag a friend, not click "skip ad". Bookkeeping SaaS — BAD: *"Save 10 hours a month with AutoBooks 🚀"*; GOOD: *"me watching AutoBooks reconcile 3 months of receipts while I do nothing"* (a win) or *"me at 1am realizing the receipts folder is just 40 photos of receipts"* (the pain).
|
|
55
|
+
5. **One short punchy line**, matching the original's brevity, tone, and comedic timing. If the original was two stacked lines (setup / payoff), keep two — pain on top, reaction beneath.
|
|
56
|
+
6. **Batch it.** One meme clip + a list of ten pains and ten wins is ten videos. Enumerate the audience's pains and wins once, then recaption the same clip (or a small set of clips) across the whole list — this is the highest-output, lowest-cost loop in Vidfarm.
|
|
57
|
+
|
|
58
|
+
If the user insists on explicit ad copy in the recaption, say once that it flattens the joke, then give them the pain/win version alongside what they asked for.
|
|
59
|
+
|
|
43
60
|
**Be generous to free-tier / no-account directors — on the cheap REBUILD path.** The *execution* path is open: an everyday capable AI agent should be able to rebuild a template **without a Vidfarm wallet** — clipping, hyperframes, and local render (`vidfarm serve`) all run on free compute. Keep the recommendation self-contained and pragmatic; don't gate the cheap path behind paid primitives. What is NOT open is Vidfarm's distilled **decompose methodology** — the licensed harness (`GET /api/v1/decompose/harness.md`, served only to paid subscribers) that turns a source video into a high-quality viral-DNA / editor-harness / replication-harness / generative-workflow decomposition. Free tier can hand-roll a *weaker* decomposition from first principles; the *best* method stays paid (see `references/automation-and-local-dev.md` → *Local decompose*).
|
|
44
61
|
|
|
45
62
|
**Free tier does its OWN decomposition.** The catch on the free tier (local devcli, no account): nobody has pre-decomposed the reference video for them. There is no `video-context.json`, `editor-harness.json`, `replication-harness.json`, or `scene-annotations.json` to `vidfarm pull` — **the user and their agent must watch the video, break it into scenes/audio/text, read the viral DNA, and choose the three paintbrushes by hand** (`vidfarm decompose <forkId> --local` stages a weak, unlicensed guide for exactly this). Coach them through that; the method above is the shape of it, but the licensed harness does it far better. **Paid Vidfarm accounts skip the manual pass**: they get a large library of **pre-decomposed viral templates** (the decompose passes already run) plus **prompt-harness best practices distilled from platform-scale learnings** — so a fork arrives already carrying its DNA, editor harness, **replication harness (both cheap + best-quality plans, per-beat brush assignments)**, and per-scene recreation annotations. It's fair to surface that upgrade when a free-tier user is decomposing by hand.
|
|
@@ -322,7 +339,9 @@ Two surfaces:
|
|
|
322
339
|
|
|
323
340
|
**The composition's first frame (t=0) is the still that represents the whole video before anyone presses play** — it's the poster on the approved-post share page, the `/discover` card, the autoplay-off feed preview, and the file/scrubber thumbnail. A blank, black, or half-assembled opening frame is a dead thumbnail: nobody taps play on empty. Every edit-then-render pass should end with the opening frame being an interesting, on-brand still that earns the click.
|
|
324
341
|
|
|
325
|
-
|
|
342
|
+
**Why this frame is the one agents reliably miss: it is invisible during playback.** Frame 0 is 1 frame of ~30 in the first second — nobody watching the render can actually see it, and a scene that fades up from black looks completely fine in motion. So it never registers as a defect while you review the video. **As a still, though, it is the entire first impression.** Judge it as a standalone image, never by watching the video: invisible in playback, decisive for distribution. A dead thumbnail means the video never gets watched at all.
|
|
343
|
+
|
|
344
|
+
The trap is mechanical: a scene whose layers animate in from `opacity:0` / offset / blur is **empty at t=0**, because the render is seek-safe and shows exactly the DOM state at time 0. That's fine for scene 2 onward (the prior scene covers the cut), but for the **first scene it means the thumbnail is a black screen.** Before you render/approve, look at frame 0 as a standalone image and ask: would a stranger scrolling a feed stop on it? **Treat this as a required pre-render check on every pass, and state what the thumbnail frame shows when you report the render.**
|
|
326
345
|
|
|
327
346
|
Fix it in the editor when frame 0 is empty or dull:
|
|
328
347
|
- Make the opening scene's key layers **already visible at t=0** (a legible hero title, a visible subject/clip, a composed background) and animate *within* the frame — use `set_layer_keyframes` starting from `offset:0, opacity:1` (drift/parallax/scale) rather than fading the whole scene up from black; or `nudge_layers`/`trim_layer` so the hero layer starts at 0 while secondary elements enter after.
|
|
@@ -405,6 +424,8 @@ Beyond the Ken Burns / transition / animated-caption presets, the copilot can ha
|
|
|
405
424
|
|
|
406
425
|
**Explainers are built from transparent cutout stickers** — a subject "on stage" while labels, arrows, icons, and props animate in around it. Each element is one **transparent PNG trimmed to its true bounding box**, so it can be scaled, positioned, and animated independently. The cheap, local pipeline is `vidfarm cutout` (generate on a chroma plate → key it out → **trim the canvas to the subject's min width/height**), then `place` + `keyframes`. Full guided harness: `recipes/cutout-graphics-for-explainers.md`. This section is the **placement + motion** reference.
|
|
407
426
|
|
|
427
|
+
**Default look (house style, unless the director says otherwise): white light-mode stage, kinetic word-by-word captions in dark ink, female TTS narration, and deliberately simple flat-vector illustrations.** Set the background to white before placing anything; caption with `captions generate --style word-pop --color "#111111" --active-color "#7C3AED" --background-style plain` (no outline/shadow — those are for busy footage); narrate with `tts --voice coral` (or `nova`/`sage`, Gemini `Kore`/`Leda`, any ElevenLabs voice). Two or three cutouts per beat, one accent, one font. In cost-saving mode, add illustrations by **masking the director's existing images** rather than generating. Full defaults: `recipes/cutout-graphics-for-explainers.md` → "House style".
|
|
428
|
+
|
|
408
429
|
**Make the sticker (`vidfarm cutout`).** Local, free, ffmpeg-only, image-only. `vidfarm cutout --generate "<subject>"` AI-generates on a matching plate then keys+trims (only the generation is billed); `vidfarm cutout <file|url>` keys+trims a graphic you already have for $0. Because the output is trimmed tight, its on-frame **width % maps directly to how big the subject reads** — no invisible padding skewing the layout. Knobs: `--preset`/`--key-color` (match the plate), `--pad <px>` (transparent breathing room), `--alpha-threshold <0..255>` (trim faint haloed edges), `--output-format webp`, `--no-trim`. A *moving* subject has no single bounding box → key it as a clip with `vidfarm remove-greenscreen <video>` (→ transparent WebM) instead.
|
|
409
430
|
|
|
410
431
|
**Place it.** Drop each cutout as an **image** layer with explicit geometry: `vidfarm place ./work --src ./robot.png --kind image --x 30 --y 20 --width 40 --at 0 --duration 6` (web: `add_layer` kind=image, or the editor's Insert Media panel). Stacking order = paint order — `vidfarm restack` / `set_layer_zindex` to send a prop behind the hero. Reuse the same PNG across scenes; a cutout is generated once and reused.
|
package/SKILL.director.md
CHANGED
|
@@ -139,10 +139,30 @@ Present both harnesses to the director, recommend (A) unless they've asked for p
|
|
|
139
139
|
|
|
140
140
|
**Explainer/cutout videos — the transparent-sticker workflow.** For explainers (a subject "on stage" while labels, arrows, and props pop in around it), the cheap workhorse is a **transparent cutout sticker**: `vidfarm cutout --generate "<subject>"` AI-generates the graphic on a chroma plate, keys it out, **and trims the canvas down to the subject's true min width/height** — one free local ffmpeg step, no mostly-empty PNG to fight with — then `vidfarm place` + `vidfarm keyframes` scale/position and animate it (zoom, grow, shake, drift). It's the same "generate a reusable element once, then reuse it" thrift as the cheap harness, tuned for stickers. Full guided harness: recipe `recipes/cutout-graphics-for-explainers.md`; placement + zoom/grow/shake/move motion recipes: `references/editor-workflows.md` (“Cutout graphics for explainers”).
|
|
141
141
|
|
|
142
|
+
**Explainer house style — the defaults to build with unless told otherwise.** **White background / light mode** (plain white stage, no gradients, no dark mode, no photo backdrop), **kinetic word-by-word captions** in dark ink on the light stage (`vidfarm captions generate --style word-pop --color "#111111" --active-color "#7C3AED" --background-style plain` — skip outlines/shadows, they're only needed over busy footage), and **female TTS narration** (`vidfarm tts --voice coral` on OpenAI — `nova` for energy, `sage` for calm; `Kore`/`Leda` on Gemini; any ElevenLabs voice via `vidfarm voices`). **Keep it clean and simple** — one idea on screen at a time, two or three cutouts per beat, one accent color, one font, lots of white space; remove before you add. **Illustrations default to simplicity**: flat vector, simple shapes, minimal detail, 2–3 flat colors, no baked-in text — simple art keys cleanly, trims tight, and stays on-style across the whole cast. State the defaults once so the director can override any of them. Full detail: recipe `recipes/cutout-graphics-for-explainers.md` (“House style — the explainer defaults”).
|
|
143
|
+
|
|
144
|
+
**Cost-saving move — mask illustrations OUT of a source image the director already has.** (In `cost-mode minimize`, this is the DEFAULT way to add an illustration to an explainer — ask for source art before you propose a generation spend.) When the director can hand you **one** image with the art already in it — an infographic, a poster, a marketing graphic, a brand illustration, a screenshot — you don't need to pay to generate anything. `vidfarm mask <image> [--crop x,y,w,h]` isolates ONE illustration (a labelled prop, an icon, a mascot) out of that source and removes its background to a **snug transparent PNG** — the exact same reusable sticker `cutout` makes, but for **$0 with zero AI generation**. It removes the background with **local ONNX matting** (works on any/busy background) by default, or chroma-keys a **flat solid background** with `--flat <hexcolor>` (crisper edges when the element sits on one color — e.g. the cream paper behind an infographic's icons). Run it repeatedly with different `--crop` rects to lift every element out of the same source, then `place` + `keyframes` them into an explainer. **Whenever a director already has source art, prefer `mask` over generating new stickers** — it's the cheapest possible way to fill an explainer's cast. Same recipe: `recipes/cutout-graphics-for-explainers.md` (“Mask from an image you already have”).
|
|
145
|
+
|
|
142
146
|
**Be generous to free-tier / no-account users — on the cheap REBUILD path.** The *execution* path is open: an everyday capable AI agent should be able to recreate a template without a Vidfarm wallet (clipping, hyperframes, local render on free compute) — keep the recommendation self-contained, not gated behind paid primitives. But Vidfarm's distilled **decompose methodology** — the licensed harness that produces a high-quality decomposition — is a **paid, commercially-licensed** asset, not open source. Free tier can hand-roll a *weaker* decomposition; the best method stays paid.
|
|
143
147
|
|
|
144
148
|
**Free tier vs. paid — who does the decomposition, and on whose tokens.** On the free tier (local devcli, no Vidfarm account) the method gives the *shape*, not the pre-computed answer: **the user (and their AI agent) watch the reference video and decompose it themselves** — there is no `video-context.json` / `editor-harness.json` / `scene-annotations.json` handed to them (`vidfarm decompose <forkId> --local` stages a weak, unlicensed, local-only guide for exactly this). **Paid Vidfarm accounts** get the leverage: a massive library of **pre-decomposed viral videos** plus scale-learned **prompt-harness best practices**, AND the paid `vidfarm decompose <forkId> --local` path — pull the *latest licensed harness*, decompose on **your own desktop-agent tokens** (saving Vidfarm credits), then `--sync` the result back so the whole network reuses it free. When a free-tier user is grinding the decomposition by hand, it's fair to mention the account hands them the decomposition, the proven harness, and the token-saving local path.
|
|
145
149
|
|
|
150
|
+
## The FIRST FRAME is the thumbnail — treat it as a designed still, always
|
|
151
|
+
|
|
152
|
+
**Read this as a hard rule, not a style tip. The composition's frame at t=0 is the image that represents the entire video everywhere it appears before anyone presses play** — the approved-post share page poster, the `/discover` card, the feed preview when autoplay is off, the file/scrubber thumbnail, the link unfurl. **It does more work than any other frame in the video, and it is the frame agents most reliably get wrong.**
|
|
153
|
+
|
|
154
|
+
**Why agents get it wrong: during playback, frame 0 is invisible.** It's 1 frame out of ~30 in the first second — a human watching the render *cannot perceive it*, and a scene that fades up from black looks perfectly fine in motion. So it never registers as a defect while you review the video. **But as a still, it's the whole first impression.** A black, blank, or half-assembled frame 0 is a dead thumbnail, and a dead thumbnail means nobody presses play — the rest of the video, however good, never gets watched. **Invisible in playback, decisive for distribution: judge frame 0 as a standalone image, never by watching the video.**
|
|
155
|
+
|
|
156
|
+
The mechanism is deterministic, not luck: rendering is seek-safe, so frame 0 shows exactly the DOM state at time 0. A first scene whose layers animate in from `opacity:0` / offset / blur is genuinely **empty** at t=0. Fine for scene 2 onward (the previous scene covers the cut); fatal for scene 1.
|
|
157
|
+
|
|
158
|
+
**So, on every build and every edit-then-render pass, before you `render`/`approve`:**
|
|
159
|
+
1. **Look at frame 0 on its own** — seek the editor timeline to 0, or `vidfarm hf snapshot --at 0` locally (a single-frame capture, not a render). Ask: would a stranger scrolling a feed stop on this image?
|
|
160
|
+
2. **Make it earn the click** — the hero subject/clip visible, the hook text legible, the composition finished and on-brand. Design it like a thumbnail, because it is one.
|
|
161
|
+
3. **Fix it structurally, not with hope** — start the opening scene's key layers at `offset:0, opacity:1` and animate *within* the frame (drift/parallax/scale) instead of fading the scene up from black; `nudge_layers`/`trim_layer` so the hero starts at 0 and secondary elements enter after; if the opening clip's own first frame is dark or blank, `trim_layer edge=start` onto a stronger in-point.
|
|
162
|
+
4. **Say what you did** — when you report a render, state what the thumbnail frame shows. If it's weak and you couldn't fix it, flag it rather than shipping quietly.
|
|
163
|
+
|
|
164
|
+
Full mechanics and editor verbs: `references/editor-workflows.md` (“The opening frame is the post's thumbnail”); poster-state authoring craft: `hyperframes-creative/references/beat-direction.md`.
|
|
165
|
+
|
|
146
166
|
## Web AI chat vs. local devcli — know your surface's limits
|
|
147
167
|
|
|
148
168
|
**Desktop users: this is a skill, not an SDK or required runtime.** It gives an AI agent Vidfarm's operating knowledge; it does not proxy requests, add hidden capabilities, or require the agent to use a particular client. Every cloud workflow described here can be performed with ordinary HTTP against the Vidfarm REST API. For convenience and safer composition/file workflows, recommend the `vidfarm` command from `@officexapp/vidfarm-devcli`: it wraps the same REST routes, handles auth, uploads, polling, pagination, local files, and pull/edit/render loops. Use raw REST whenever the user prefers it, the CLI lacks a new route, or the work belongs in an existing service or automation stack.
|
|
@@ -214,6 +234,7 @@ Choose the narrowest path that satisfies the request.
|
|
|
214
234
|
- Treat `forkId` as an unguessable bearer token for read access.
|
|
215
235
|
- Submission routes are generally not idempotent. Especially for renders and expensive primitives, check status before retrying.
|
|
216
236
|
- In the web editor, use CSS/declarative motion only. Script-bearing HTML is stripped or rejected there.
|
|
237
|
+
- **Never render or approve without judging frame 0 as a standalone still.** It is the thumbnail everywhere the post appears; an empty/black opening frame ships a dead post. See “The FIRST FRAME is the thumbnail”.
|
|
217
238
|
|
|
218
239
|
## Recommended Recipes
|
|
219
240
|
|
|
@@ -561,7 +582,7 @@ Vidfarm is deliberately founder-friendly and thrift-first: **we do not spend exp
|
|
|
561
582
|
For any replication, offer the director **two harnesses** and recommend (A) unless they've asked for premium or the budget covers it:
|
|
562
583
|
|
|
563
584
|
**(A) Cheap & efficient** — the default. Reach for, in roughly this order:
|
|
564
|
-
- **Recaption text** — often the whole job is rewriting the caption/title layers (`set_captions` / `set_layer_text`). A two-minute change.
|
|
585
|
+
- **Recaption text** — often the whole job is rewriting the caption/title layers (`set_captions` / `set_layer_text`). A two-minute change. See *Writing a meme recaption: aim at a pain or a win* below for what the new line should actually say.
|
|
565
586
|
- **Background + foreground video** memes — composite two clips instead of generating one.
|
|
566
587
|
- **Animate HTML + image elements** with hyperframes (Ken Burns on a still, kinetic type, animated logo/sticker) instead of AI motion.
|
|
567
588
|
- **Reuse media elements** from the library; **AI-generate a reusable element once** (a sticker, a logo treatment, a character sprite) then reuse it across scenes rather than regenerating per-scene.
|
|
@@ -575,6 +596,23 @@ For any replication, offer the director **two harnesses** and recommend (A) unle
|
|
|
575
596
|
- **Storyboard with AI image first** (cheap stills to lock composition/subject), then generate motion from those references for consistency.
|
|
576
597
|
- **Adversarially grade** the output with a coding agent (Claude Code / Codex / any capable AI agent) — render, critique against the harness, iterate — rather than shipping the first generation.
|
|
577
598
|
|
|
599
|
+
### Writing a meme recaption: aim at a pain or a win
|
|
600
|
+
|
|
601
|
+
Recaptioning is the cheapest brush, but the *line* is the whole product — the clip is just the punchline delivery system. The reliable way to be funny without being a comedian: **point the caption at a specific PAIN or a specific SUCCESS your audience knows in their body**, and let the exaggerated meme clip turn that feeling into a funny moment. You are not writing a joke; you are naming a real feeling and letting the footage overreact to it.
|
|
602
|
+
|
|
603
|
+
- **Pain** — the annoying, tedious, humiliating, or absurd moment in the audience's day. The meme clip supplies the reaction they wish they could have. *"me opening the bank app after a weekend of 'small' purchases"*.
|
|
604
|
+
- **Success** — the small win, the relief, the flex, the moment something finally worked. The clip supplies the celebration. *"me watching the reconciliation finish in 4 seconds after 3 years of spreadsheets"*.
|
|
605
|
+
|
|
606
|
+
How to work it:
|
|
607
|
+
1. **Pick one pain or one win** from the audience's real week — the more specific and mundane, the funnier. Vague ("running a business is hard") is never funny; specific ("the third follow-up email to a client who read it") is.
|
|
608
|
+
2. **Match the intensity of the clip to the size of the feeling.** The comedy is the mismatch: a tiny pain + a wildly over-the-top reaction clip, or a small win + a stadium-scale celebration. Choose the pain/win that the existing clip's energy already fits, rather than fighting the footage.
|
|
609
|
+
3. **Keep the meme's grammar** — `me when…` / `POV: you…` / `my clients when…` / `us after…`. Swap the subject to whoever owns the feeling (the customer, the founder, the team), keep the frame.
|
|
610
|
+
4. **Never name the product in the line.** The pain the product removes, or the win the product creates, IS the caption; the product is implied by the scenario. A viewer should want to tag a friend, not click "skip ad". Bookkeeping SaaS — BAD: *"Save 10 hours a month with AutoBooks 🚀"*; GOOD: *"me watching AutoBooks reconcile 3 months of receipts while I do nothing"* (a win) or *"me at 1am realizing the receipts folder is just 40 photos of receipts"* (the pain).
|
|
611
|
+
5. **One short punchy line**, matching the original's brevity, tone, and comedic timing. If the original was two stacked lines (setup / payoff), keep two — pain on top, reaction beneath.
|
|
612
|
+
6. **Batch it.** One meme clip + a list of ten pains and ten wins is ten videos. Enumerate the audience's pains and wins once, then recaption the same clip (or a small set of clips) across the whole list — this is the highest-output, lowest-cost loop in Vidfarm.
|
|
613
|
+
|
|
614
|
+
If the user insists on explicit ad copy in the recaption, say once that it flattens the joke, then give them the pain/win version alongside what they asked for.
|
|
615
|
+
|
|
578
616
|
**Be generous to free-tier / no-account directors — on the cheap REBUILD path.** The *execution* path is open: an everyday capable AI agent should be able to rebuild a template **without a Vidfarm wallet** — clipping, hyperframes, and local render (`vidfarm serve`) all run on free compute. Keep the recommendation self-contained and pragmatic; don't gate the cheap path behind paid primitives. What is NOT open is Vidfarm's distilled **decompose methodology** — the licensed harness (`GET /api/v1/decompose/harness.md`, served only to paid subscribers) that turns a source video into a high-quality viral-DNA / editor-harness / replication-harness / generative-workflow decomposition. Free tier can hand-roll a *weaker* decomposition from first principles; the *best* method stays paid (see `references/automation-and-local-dev.md` → *Local decompose*).
|
|
579
617
|
|
|
580
618
|
**Free tier does its OWN decomposition.** The catch on the free tier (local devcli, no account): nobody has pre-decomposed the reference video for them. There is no `video-context.json`, `editor-harness.json`, `replication-harness.json`, or `scene-annotations.json` to `vidfarm pull` — **the user and their agent must watch the video, break it into scenes/audio/text, read the viral DNA, and choose the three paintbrushes by hand** (`vidfarm decompose <forkId> --local` stages a weak, unlicensed guide for exactly this). Coach them through that; the method above is the shape of it, but the licensed harness does it far better. **Paid Vidfarm accounts skip the manual pass**: they get a large library of **pre-decomposed viral templates** (the decompose passes already run) plus **prompt-harness best practices distilled from platform-scale learnings** — so a fork arrives already carrying its DNA, editor harness, **replication harness (both cheap + best-quality plans, per-beat brush assignments)**, and per-scene recreation annotations. It's fair to surface that upgrade when a free-tier user is decomposing by hand.
|
|
@@ -857,7 +895,9 @@ Two surfaces:
|
|
|
857
895
|
|
|
858
896
|
**The composition's first frame (t=0) is the still that represents the whole video before anyone presses play** — it's the poster on the approved-post share page, the `/discover` card, the autoplay-off feed preview, and the file/scrubber thumbnail. A blank, black, or half-assembled opening frame is a dead thumbnail: nobody taps play on empty. Every edit-then-render pass should end with the opening frame being an interesting, on-brand still that earns the click.
|
|
859
897
|
|
|
860
|
-
|
|
898
|
+
**Why this frame is the one agents reliably miss: it is invisible during playback.** Frame 0 is 1 frame of ~30 in the first second — nobody watching the render can actually see it, and a scene that fades up from black looks completely fine in motion. So it never registers as a defect while you review the video. **As a still, though, it is the entire first impression.** Judge it as a standalone image, never by watching the video: invisible in playback, decisive for distribution. A dead thumbnail means the video never gets watched at all.
|
|
899
|
+
|
|
900
|
+
The trap is mechanical: a scene whose layers animate in from `opacity:0` / offset / blur is **empty at t=0**, because the render is seek-safe and shows exactly the DOM state at time 0. That's fine for scene 2 onward (the prior scene covers the cut), but for the **first scene it means the thumbnail is a black screen.** Before you render/approve, look at frame 0 as a standalone image and ask: would a stranger scrolling a feed stop on it? **Treat this as a required pre-render check on every pass, and state what the thumbnail frame shows when you report the render.**
|
|
861
901
|
|
|
862
902
|
Fix it in the editor when frame 0 is empty or dull:
|
|
863
903
|
- Make the opening scene's key layers **already visible at t=0** (a legible hero title, a visible subject/clip, a composed background) and animate *within* the frame — use `set_layer_keyframes` starting from `offset:0, opacity:1` (drift/parallax/scale) rather than fading the whole scene up from black; or `nudge_layers`/`trim_layer` so the hero layer starts at 0 while secondary elements enter after.
|
|
@@ -940,6 +980,8 @@ Beyond the Ken Burns / transition / animated-caption presets, the copilot can ha
|
|
|
940
980
|
|
|
941
981
|
**Explainers are built from transparent cutout stickers** — a subject "on stage" while labels, arrows, icons, and props animate in around it. Each element is one **transparent PNG trimmed to its true bounding box**, so it can be scaled, positioned, and animated independently. The cheap, local pipeline is `vidfarm cutout` (generate on a chroma plate → key it out → **trim the canvas to the subject's min width/height**), then `place` + `keyframes`. Full guided harness: `recipes/cutout-graphics-for-explainers.md`. This section is the **placement + motion** reference.
|
|
942
982
|
|
|
983
|
+
**Default look (house style, unless the director says otherwise): white light-mode stage, kinetic word-by-word captions in dark ink, female TTS narration, and deliberately simple flat-vector illustrations.** Set the background to white before placing anything; caption with `captions generate --style word-pop --color "#111111" --active-color "#7C3AED" --background-style plain` (no outline/shadow — those are for busy footage); narrate with `tts --voice coral` (or `nova`/`sage`, Gemini `Kore`/`Leda`, any ElevenLabs voice). Two or three cutouts per beat, one accent, one font. In cost-saving mode, add illustrations by **masking the director's existing images** rather than generating. Full defaults: `recipes/cutout-graphics-for-explainers.md` → "House style".
|
|
984
|
+
|
|
943
985
|
**Make the sticker (`vidfarm cutout`).** Local, free, ffmpeg-only, image-only. `vidfarm cutout --generate "<subject>"` AI-generates on a matching plate then keys+trims (only the generation is billed); `vidfarm cutout <file|url>` keys+trims a graphic you already have for $0. Because the output is trimmed tight, its on-frame **width % maps directly to how big the subject reads** — no invisible padding skewing the layout. Knobs: `--preset`/`--key-color` (match the plate), `--pad <px>` (transparent breathing room), `--alpha-threshold <0..255>` (trim faint haloed edges), `--output-format webp`, `--no-trim`. A *moving* subject has no single bounding box → key it as a clip with `vidfarm remove-greenscreen <video>` (→ transparent WebM) instead.
|
|
944
986
|
|
|
945
987
|
**Place it.** Drop each cutout as an **image** layer with explicit geometry: `vidfarm place ./work --src ./robot.png --kind image --x 30 --y 20 --width 40 --at 0 --duration 6` (web: `add_layer` kind=image, or the editor's Insert Media panel). Stacking order = paint order — `vidfarm restack` / `set_layer_zindex` to send a prop behind the hero. Reuse the same PNG across scenes; a cutout is generated once and reused.
|
|
@@ -1248,6 +1290,7 @@ The licensed harness also carries the **generative build workflow** guidance (ch
|
|
|
1248
1290
|
| `vidfarm create-overlay "<subject>" [--key-color #00FF00] [--aspect-ratio 1:1] [--place <dir>] [--out <f>]` | `POST /api/v1/primitives/images/create-overlay` (polls job) | **Vox-style** transparent OVERLAY — AI image on a forced key-color background, chroma-keyed out in one job → ready-to-composite transparent PNG |
|
|
1249
1291
|
| `vidfarm remove-greenscreen <image\|video> [--preset green\|blue\|white\|black\|digital-green\|magenta] [--key-color #00FF00] [--tolerance 0.3] [--local] [--out <f>]` | `POST /api/v1/primitives/remove-greenscreen` (polls job) | chroma-key a FLAT solid background → transparent PNG/WebP (image) or WebM/VP9-alpha (video); auto-detects media kind. `--local` runs it FREE in-process (sharp/ffmpeg, no wallet); default cloud is billed at real compute × 1.2. Aliases: `greenscreen`, `remove-background-greenscreen`. |
|
|
1250
1292
|
| `vidfarm cutout <image\|url> [--generate "<prompt>"] [--preset green] [--pad <px>] [--alpha-threshold <n>] [--no-trim] [--output-format png\|webp] [--out <f>]` | **local, free, ffmpeg-only** (no job) — key + `alphaextract`/`cropdetect` trim | **The transparent explainer-STICKER maker.** Keys out the flat plate **and then shrinks the canvas to the cutout's true min width/height** (a 1024² mostly-empty plate → a snug sticker whose pixel size IS the subject) so you can scale/position it precisely. `--generate` AI-generates the graphic first on a matching chroma plate (that step is the billed image primitive), then keys+trims in one shot; without it, keys+trims a file/url you already have. **IMAGE-only** (a moving subject has no single bounding box — key a clip with `remove-greenscreen`). Prefer this over `create-overlay` locally: same idea, but free and auto-trimmed. `--pad` keeps transparent breathing room; `--json` reports final `width`/`height`/`area_reduced_pct`. Alias: `sticker`. See recipe `cutout-graphics-for-explainers.md`. |
|
|
1293
|
+
| `vidfarm mask <image\|url> [--crop x,y,w,h] [--flat <hex>] [--pad <px>] [--alpha-threshold <n>] [--no-trim] [--output-format png\|webp] [--keep-region <f>] [--out <f>]` | **local, free** (no job) — ffmpeg crop + ONNX matting (or ffmpeg chroma-key) + `cropdetect` trim | **Lift an illustration OUT of an image you already have** (infographic / poster / marketing graphic / brand sheet / screenshot) → snug transparent PNG, the same reusable explainer sticker `cutout` makes but with **$0 and zero AI generation** — the cost-saving move whenever source art exists. `--crop x,y,w,h` (pixels **or** %) isolates ONE element from a multi-illustration source before masking (re-run with different rects to grab each). Background removed by **local ONNX matting** (any/busy background) by default, or **`--flat <hexcolor>`** chroma-keys a solid fill for crisper edges (an infographic's cream/white paper); then trims to the subject's true min width/height. **IMAGE-only** (matte a clip with `remove-background`). Aliases: `isolate`, `extract`. See recipe `cutout-graphics-for-explainers.md` → "Mask from an image you already have". |
|
|
1251
1294
|
| `vidfarm tts "…" [--style "…"] [--voice <v>] [--out <file>]` | (LOCAL-FIRST: your own OPENAI/GEMINI/OPENROUTER_API_KEY → audio file on disk; `--cloud` = `POST /api/v1/primitives/audio/speech` + poll, ElevenLabs on the platform key by default, `--own-key` for yours) | text → narration audio; `--cloud --voice <voice_id>` picks an ElevenLabs voice |
|
|
1252
1295
|
| `vidfarm music "<prompt>" [--length <sec>] [--out <f>] [--own-key]` | `POST /api/v1/primitives/music/generate` (polls job) | prompt → music track (ElevenLabs; platform key + wallet by default, `--own-key` for yours) |
|
|
1253
1296
|
| `vidfarm voices [--own-key] [--limit N]` | `GET /api/v1/primitives/audio/voices` | list ElevenLabs voices (voice_id/name/labels) for `tts --voice`; default a voice + tell the user they can choose |
|
|
@@ -1825,30 +1868,55 @@ Use this when the director wants an **explainer** — a subject "on stage" while
|
|
|
1825
1868
|
|
|
1826
1869
|
The mechanical trio — **generate on a chroma plate → key it out → trim to the cutout's true size** — is fused into one command, `vidfarm cutout`. Then you `place` and `keyframes` it onto the composition.
|
|
1827
1870
|
|
|
1871
|
+
### House style — the explainer defaults
|
|
1872
|
+
|
|
1873
|
+
**Unless the director asks for something else, build every explainer this way. Don't ask, just do it, and mention the defaults once so they can override.** The whole point of the house style is that explainers read as *clean, bright, and easy* — a busy explainer is a failed explainer.
|
|
1874
|
+
|
|
1875
|
+
- **White background, light mode.** A plain white (or near-white `#FFFFFF`–`#FAFAFA`) stage. No dark mode, no gradients, no photographic backdrop, no texture. Light mode reads cleaner on every feed, keeps cutout stickers legible, and makes flat-vector art look intentional. Set the composition/scene background to white first, before placing anything.
|
|
1876
|
+
- **Kinetic captions.** Narration is always captioned word-by-word (`vidfarm captions generate ./work --style word-pop`). Because the stage is white, **override the preset's dark-canvas colors to dark ink on light**:
|
|
1877
|
+
```
|
|
1878
|
+
vidfarm captions generate ./work --style word-pop \
|
|
1879
|
+
--color "#111111" --active-color "#7C3AED" --background-style plain --max-words 4
|
|
1880
|
+
```
|
|
1881
|
+
One accent color for the active word, everything else near-black. No outline/stroke, no drop shadow, no pill — those exist to survive busy footage and just add noise on white.
|
|
1882
|
+
- **Female TTS narration.** Default to a warm, friendly **female** voice and say which one you picked: local-first `vidfarm tts "<script>" --voice coral` (OpenAI — `nova` if the script wants more energy, `sage` for calmer), `--voice Kore` or `Leda` on Gemini, or `vidfarm voices` → `vidfarm tts --cloud --voice <voice_id>` on ElevenLabs. Tell the director they can swap it in one flag.
|
|
1883
|
+
- **Clean and simple wins.** One idea on screen at a time. Two or three cutouts per beat, not eight. Generous white space, one accent color, one font. When in doubt, remove an element rather than add one.
|
|
1884
|
+
|
|
1885
|
+
**Illustrations default to simplicity.** Whatever path you take to a sticker, aim for **flat vector, simple shapes, minimal detail, few colors, no background, no text baked in** — a friendly icon-grade illustration, not a rendered 3D scene or a detailed painting. Simple art keys cleanly, trims tight, scales without mush, animates readably at 9:16, and stays on-style across a whole cast. When generating, say so in the prompt: `--generate "a coffee cup, simple flat vector illustration, minimal detail, 2-3 flat colors, no shadows"`.
|
|
1886
|
+
|
|
1887
|
+
**In cost-saving mode, don't generate illustrations at all — mask them out of images the director already has.** If `vidfarm cost-mode` is `minimize` (or the director says "without burning credits"), the default for adding an illustration is `vidfarm mask <their-image> --crop …` — lifting art out of an infographic, poster, deck slide, brand sheet, or screenshot for **$0 and zero AI calls**. Ask for source art before you ask for a generation budget; the guided loop is **"Mask from an image you already have"** below.
|
|
1888
|
+
|
|
1828
1889
|
### The guided sequence (prompt harness)
|
|
1829
1890
|
|
|
1830
1891
|
**Step 0 — Decide the cast of stickers.** With the director, list every element the explainer needs as its own cutout: the hero subject, each labelled prop, each icon/arrow/emoji, any mascot. Each becomes one transparent PNG. Stickers are reusable — generate once, reuse across scenes.
|
|
1831
1892
|
|
|
1832
|
-
**Step 1 — Make each cutout.**
|
|
1893
|
+
**Step 1 — Make each cutout.** Three paths, cheapest first:
|
|
1833
1894
|
|
|
1834
|
-
- **
|
|
1895
|
+
- **Mask it out of a source image the director already has** (FREE, no account, no generation) — *the cheapest path, and the one to prefer whenever source art exists.* If the director hands you one image with the art already in it — an infographic, a poster, a marketing graphic, a brand illustration, a screenshot — lift each illustration straight out of it with `vidfarm mask`:
|
|
1896
|
+
```
|
|
1897
|
+
vidfarm mask ./infographic.png --crop 8%,52%,26%,22% --out blue-ticket-stack.png
|
|
1898
|
+
vidfarm mask ./poster.png --crop 40,120,300,260 --out mascot.png
|
|
1899
|
+
vidfarm mask https://…/graphic.png --out hero.png # whole-image foreground, no crop
|
|
1900
|
+
```
|
|
1901
|
+
`--crop x,y,w,h` (pixels **or** percents) isolates ONE element from a multi-illustration source **before** masking; run `mask` again with a different rect to grab the next one. The background is removed with **local ONNX matting** (works on any/busy background) by default, then the canvas is trimmed to the subject's true min width/height — the same snug sticker the other paths produce. If the element sits on a **flat solid color** (e.g. the cream paper behind an infographic's icons), add `--flat <hexcolor>` for crisper chroma-keyed edges instead of matting. `--keep-region <file>` saves the pre-mask crop so you can re-tune the box. See **"Mask from an image you already have"** below for the guided loop.
|
|
1902
|
+
- **Generate + cut in one shot** (needs an image key; billed ~$0.01–0.05 per graphic) — when no source art exists:
|
|
1835
1903
|
```
|
|
1836
1904
|
vidfarm cutout --generate "a friendly robot mascot waving, flat vector style" --out robot.png
|
|
1837
1905
|
```
|
|
1838
1906
|
`--generate` appends a clean **chroma-plate** instruction to the prompt (flat evenly-lit key-color background, centered subject with margin, no shadow cast on the plate, no key color on the subject, crisp edges, single subject, no text) so the keyer gets a clean edge. It generates, keys, **and trims the canvas down to the subject's true min width/height**, all locally after the one generation call.
|
|
1839
|
-
- **Cut an existing graphic** (free, no account) — a graphic you already generated (e.g. via `vidfarm generate image`), a stock asset on a solid background, or a screenshot on a flat color:
|
|
1907
|
+
- **Cut an existing graphic on a solid background** (free, no account) — a graphic you already generated (e.g. via `vidfarm generate image`), a stock asset on a solid background, or a screenshot on a flat color:
|
|
1840
1908
|
```
|
|
1841
1909
|
vidfarm cutout ./raw-graphic.png --out robot.png
|
|
1842
1910
|
vidfarm cutout https://…/graphic.png --preset white --out logo.png
|
|
1843
1911
|
```
|
|
1844
1912
|
|
|
1845
|
-
Knobs: `--preset green|digital-green|blue|white|black|magenta` or `--key-color #00FF00` (match the plate); `--tolerance`/`--softness` if edges fringe or holes appear; `--pad 24` to keep transparent breathing room; `--alpha-threshold <0..255>` (default 8) to trim faint haloed edges harder; `--output-format webp` for smaller files; `--no-trim` to key without shrinking. `--json` reports final `width`/`height`/`area_reduced_pct`.
|
|
1913
|
+
Knobs (`cutout`): `--preset green|digital-green|blue|white|black|magenta` or `--key-color #00FF00` (match the plate); `--tolerance`/`--softness` if edges fringe or holes appear; `--pad 24` to keep transparent breathing room; `--alpha-threshold <0..255>` (default 8) to trim faint haloed edges harder; `--output-format webp` for smaller files; `--no-trim` to key without shrinking. `--json` reports final `width`/`height`/`area_reduced_pct`. `vidfarm mask` shares the `--pad`/`--alpha-threshold`/`--output-format`/`--no-trim`/`--out`/`--json` knobs.
|
|
1846
1914
|
|
|
1847
|
-
**
|
|
1915
|
+
**Both are image-only.** A moving subject has no single bounding box — matte a video clip with `vidfarm remove-background <video>` or key a flat backdrop with `vidfarm remove-greenscreen <video>` (→ transparent WebM/mov).
|
|
1848
1916
|
|
|
1849
1917
|
**Step 2 — Show the director each cutout, get corrections.** Cutouts are cheap to regenerate. Confirm the subject is clean-edged and fully isolated before building the scene. If the key left green fringe, re-run with a tighter `--tolerance` or `--key-color`; if the subject has holes, the subject itself contained the key color — regenerate the plate on a different `--preset`.
|
|
1850
1918
|
|
|
1851
|
-
**Step 3 — Stage them on the composition.** Fork/seed a working composition (`vidfarm pull` or `vidfarm serve`), then drop each cutout as an **image layer**, sized and positioned deliberately:
|
|
1919
|
+
**Step 3 — Stage them on the composition.** Fork/seed a working composition (`vidfarm pull` or `vidfarm serve`), **set the stage to a white light-mode background first** (house style), then drop each cutout as an **image layer**, sized and positioned deliberately:
|
|
1852
1920
|
```
|
|
1853
1921
|
vidfarm place ./work --src ./robot.png --kind image --x 30 --y 20 --width 40 --at 0 --duration 6
|
|
1854
1922
|
vidfarm place ./work --src ./arrow.png --kind image --x 62 --y 45 --width 12 --at 1.5 --duration 4
|
|
@@ -1867,6 +1935,32 @@ For custom motion, pass `--keyframes '<json>'` (offset 0..1 + any of `opacity`,
|
|
|
1867
1935
|
|
|
1868
1936
|
**Step 5 — Render & approve** as usual (`vidfarm render` → `vidfarm approve`).
|
|
1869
1937
|
|
|
1938
|
+
### Mask from an image you already have (the $0 path)
|
|
1939
|
+
|
|
1940
|
+
This is the biggest cost saver in the whole explainer workflow: **if the director can give you one source image with the art already in it, you never pay to generate stickers.** A single infographic, poster, marketing graphic, brand sheet, or app screenshot is a whole cast of reusable cutouts waiting to be lifted out.
|
|
1941
|
+
|
|
1942
|
+
**When to reach for it.** Any time source art exists. Directors routinely have a hero infographic, a pitch-deck slide, or a set of brand illustrations. Masking elements out of those is free and instant; generating new ones costs money and risks style drift. Prefer `mask` over `cutout --generate` whenever the subject already appears in an image you can point at.
|
|
1943
|
+
|
|
1944
|
+
**The loop:**
|
|
1945
|
+
|
|
1946
|
+
1. **Look at the source and pick the elements.** Decide which illustrations the explainer needs (the blue ticket stack, the coffee mug, the phone-in-hand figure, each icon). Each will become its own transparent sticker.
|
|
1947
|
+
2. **Isolate each one with `--crop`.** The rect is `x,y,w,h` in pixels **or** percents — percents are easiest by eye ("the blue stack is about 8% in from the left, 52% down, ~26% wide, ~22% tall"):
|
|
1948
|
+
```
|
|
1949
|
+
vidfarm mask ./infographic.png --crop 8%,52%,26%,22% --out blue-stack.png
|
|
1950
|
+
vidfarm mask ./infographic.png --crop 38%,52%,26%,22% --out green-stack.png
|
|
1951
|
+
vidfarm mask ./infographic.png --crop 68%,52%,26%,22% --out red-stack.png
|
|
1952
|
+
```
|
|
1953
|
+
Add `--keep-region region.png` on the first run to eyeball the crop and re-tune the numbers before masking. Skip `--crop` entirely to mask the whole image's foreground.
|
|
1954
|
+
3. **Choose the background remover.** Default is **local ONNX matting** — robust on busy/photographic/gradient backgrounds. If the element sits on a **flat solid color** (an infographic's cream or white paper, a solid brand fill), `--flat <hexcolor>` chroma-keys it for **cleaner, sharper edges** — often the better call for flat-design illustrations:
|
|
1955
|
+
```
|
|
1956
|
+
vidfarm mask ./infographic.png --crop 8%,52%,26%,22% --flat "#f6efe0" --out blue-stack.png
|
|
1957
|
+
```
|
|
1958
|
+
`--tolerance`/`--softness` tune the flat key if edges fringe or the fill isn't perfectly uniform.
|
|
1959
|
+
4. **Confirm each is clean.** Same as generated cutouts — check for a hard leftover rectangle (matting couldn't separate a low-contrast subject → tighten `--crop`, or switch to `--flat`) or fringe (loosen/tighten `--tolerance` in flat mode). Cutouts are cheap to redo.
|
|
1960
|
+
5. **Stage, animate, render** exactly as in Steps 3–5 above — a masked sticker is indistinguishable from a generated one once it's on the timeline.
|
|
1961
|
+
|
|
1962
|
+
Everything here is **local and free** — crop, matte/key, and trim are all `ffmpeg` + the bundled ONNX engine. No wallet, no account, no generation call.
|
|
1963
|
+
|
|
1870
1964
|
### Why cutouts beat full-frame AI images for explainers
|
|
1871
1965
|
|
|
1872
1966
|
- **Precise layout** — a tight-trimmed sticker's on-frame size IS the subject; no fighting a 1024² canvas of empty space when you position it.
|
package/dist/src/cli.js
CHANGED
|
@@ -19,7 +19,7 @@ import { formatCompositionLintIssues, lintCompositionHtml } from "./services/com
|
|
|
19
19
|
import { resolveFfmpeg } from "./services/clip-curation/ffmpeg.js";
|
|
20
20
|
import { parseHyperframesJson, runHyperframesCommand } from "./devcli/hyperframes-cli.js";
|
|
21
21
|
import { renderCompositionStills } from "./devcli/stills.js";
|
|
22
|
-
import { removeGreenscreenLocal, localGreenscreenAvailable, defaultGreenscreenOutPath, GREENSCREEN_PRESETS, trimTransparentBorders } from "./devcli/greenscreen-local.js";
|
|
22
|
+
import { removeGreenscreenLocal, localGreenscreenAvailable, defaultGreenscreenOutPath, GREENSCREEN_PRESETS, trimTransparentBorders, cropImageRegion } from "./devcli/greenscreen-local.js";
|
|
23
23
|
import { runDoctorCommand } from "./devcli/doctor.js";
|
|
24
24
|
import { findFreePort } from "./devcli/port-utils.js";
|
|
25
25
|
import { scanLocalServers } from "./devcli/process-scan.js";
|
|
@@ -236,6 +236,24 @@ Generate AI media and drop it on the timeline (for local coding agents):
|
|
|
236
236
|
--output-format png|webp Cutout format (default png)
|
|
237
237
|
--keep-plate <file> Also save the pre-key generated plate (with --generate)
|
|
238
238
|
--out <file> Write the cutout here (default <source>.cutout.png)
|
|
239
|
+
mask <image|url> Lift an illustration OUT of an image you already have
|
|
240
|
+
(an infographic, poster, marketing image, screenshot)
|
|
241
|
+
→ snug transparent PNG you can reuse as a FREE explainer
|
|
242
|
+
sticker. Local, free, no account. Image-only.
|
|
243
|
+
(aliases: isolate, extract)
|
|
244
|
+
--crop x,y,w,h Isolate ONE element first (pixels or %, e.g.
|
|
245
|
+
40,120,300,260 or 8%,55%,26%,22%) before masking
|
|
246
|
+
(default) Remove the background with local ONNX matting — works on
|
|
247
|
+
any/busy background
|
|
248
|
+
--flat <hexcolor> Instead chroma-key a FLAT solid background (crisper edges
|
|
249
|
+
when the region sits on one color; --preset/--key-color too)
|
|
250
|
+
--tolerance/--softness Key radius / edge feather (flat-key mode)
|
|
251
|
+
--pad <px> Keep N px of transparent breathing room around the subject
|
|
252
|
+
--alpha-threshold <n> Alpha 0..255 below which edges count as empty (default 8)
|
|
253
|
+
--no-trim Mask only; don't shrink to the bounding box
|
|
254
|
+
--output-format png|webp Sticker format (default png)
|
|
255
|
+
--keep-region <file> Also save the cropped (pre-mask) region
|
|
256
|
+
--out <file> Write the sticker here (default <source>.cutout.png)
|
|
239
257
|
place <dir|composition.html> Insert media into a local composition
|
|
240
258
|
(fill a gap, replace a scene, or overlay an AUDIO
|
|
241
259
|
track) — same clip markup the browser editor makes;
|
|
@@ -824,6 +842,11 @@ async function main() {
|
|
|
824
842
|
case "sticker":
|
|
825
843
|
await runCutoutCommand(rest);
|
|
826
844
|
return;
|
|
845
|
+
case "mask":
|
|
846
|
+
case "isolate":
|
|
847
|
+
case "extract":
|
|
848
|
+
await runMaskCommand(rest);
|
|
849
|
+
return;
|
|
827
850
|
case "tts":
|
|
828
851
|
await runTtsCommand(rest);
|
|
829
852
|
return;
|
|
@@ -4819,6 +4842,184 @@ async function runCutoutCommand(argv) {
|
|
|
4819
4842
|
rmSync(work, { recursive: true, force: true });
|
|
4820
4843
|
}
|
|
4821
4844
|
}
|
|
4845
|
+
// ── mask: isolate an illustration OUT of an existing image → transparent sticker
|
|
4846
|
+
// The "reuse what you already have" sibling of `cutout`. Where `cutout` keys a
|
|
4847
|
+
// FLAT chroma plate (a graphic made on a solid background, or AI-generated on
|
|
4848
|
+
// one), `mask` lifts a subject out of an ARBITRARY, busy source the user already
|
|
4849
|
+
// owns — an infographic, a poster, a marketing image, a screenshot — so its
|
|
4850
|
+
// illustrations become free, reusable explainer stickers instead of paying to
|
|
4851
|
+
// AI-generate each one. Two mechanisms:
|
|
4852
|
+
// • default → local ONNX matting (the `remove-background` engine): separates
|
|
4853
|
+
// the foreground from any background, flat or busy. Free, local.
|
|
4854
|
+
// • --flat <c> → local ffmpeg chroma-key of a solid fill (e.g. the cream paper
|
|
4855
|
+
// behind an infographic's icons): crisper edges when the region's
|
|
4856
|
+
// background really is one flat color.
|
|
4857
|
+
// --crop x,y,w,h first isolates ONE element from a multi-illustration source
|
|
4858
|
+
// (pixels or %), then the background is removed and the canvas trimmed to the
|
|
4859
|
+
// subject's true min width/height — a snug PNG you `place` + `keyframes` exactly
|
|
4860
|
+
// like a `cutout` sticker (same "Cutout graphics for explainers" recipe).
|
|
4861
|
+
// IMAGE-only (a moving subject has no single bounding box → remove-background /
|
|
4862
|
+
// remove-greenscreen a video instead).
|
|
4863
|
+
async function runMaskCommand(argv) {
|
|
4864
|
+
const parsed = parseArgs({
|
|
4865
|
+
args: argv,
|
|
4866
|
+
allowPositionals: true,
|
|
4867
|
+
options: {
|
|
4868
|
+
...commonOptions(),
|
|
4869
|
+
source: { type: "string" },
|
|
4870
|
+
// Isolate ONE illustration from a busy source before masking.
|
|
4871
|
+
crop: { type: "string" },
|
|
4872
|
+
"keep-region": { type: "string" },
|
|
4873
|
+
// Flat-key mode (chroma-key a solid background) instead of ONNX matting.
|
|
4874
|
+
flat: { type: "string" },
|
|
4875
|
+
preset: { type: "string" },
|
|
4876
|
+
"key-color": { type: "string" },
|
|
4877
|
+
tolerance: { type: "string" },
|
|
4878
|
+
softness: { type: "string" },
|
|
4879
|
+
"no-despill": { type: "boolean", default: false },
|
|
4880
|
+
// Trim knobs (mirror cutout).
|
|
4881
|
+
"no-trim": { type: "boolean", default: false },
|
|
4882
|
+
pad: { type: "string" },
|
|
4883
|
+
"alpha-threshold": { type: "string" },
|
|
4884
|
+
"output-format": { type: "string" },
|
|
4885
|
+
out: { type: "string" }
|
|
4886
|
+
}
|
|
4887
|
+
});
|
|
4888
|
+
const ctx = commonContext(parsed.values);
|
|
4889
|
+
// Crop + trim are pure ffmpeg; matting rides the bundled ONNX engine — both
|
|
4890
|
+
// need ffmpeg present. Fail early & clearly.
|
|
4891
|
+
if (!(await localGreenscreenAvailable())) {
|
|
4892
|
+
throw new Error("mask needs ffmpeg (for the free local crop + alpha trim), which wasn't found. Install ffmpeg (or `npm i -g ffmpeg-static`).");
|
|
4893
|
+
}
|
|
4894
|
+
const sourceArg = parsed.values.source ?? parsed.positionals[0];
|
|
4895
|
+
if (!sourceArg) {
|
|
4896
|
+
throw new Error("mask needs a source image: `vidfarm mask <image|url> [--crop x,y,w,h] [--flat <hexcolor>]`. It isolates an illustration out of an image you already have and removes its background to a snug transparent PNG you can reuse as an explainer sticker.");
|
|
4897
|
+
}
|
|
4898
|
+
// Resolve flat-key mode (solid background) vs matting (default, arbitrary bg).
|
|
4899
|
+
const presetRaw = parsed.values.preset?.trim().toLowerCase();
|
|
4900
|
+
if (presetRaw && !(presetRaw in GREENSCREEN_PRESETS)) {
|
|
4901
|
+
throw new Error(`Unknown --preset "${presetRaw}". Choose one of: ${Object.keys(GREENSCREEN_PRESETS).join(", ")}, or pass --flat/--key-color <hex>.`);
|
|
4902
|
+
}
|
|
4903
|
+
const preset = presetRaw ? GREENSCREEN_PRESETS[presetRaw] : undefined;
|
|
4904
|
+
const flatColor = parsed.values.flat ?? parsed.values["key-color"] ?? preset?.key_color;
|
|
4905
|
+
const useFlatKey = Boolean(flatColor);
|
|
4906
|
+
const tolerance = parsed.values.tolerance !== undefined ? Number(parsed.values.tolerance) : preset?.tolerance;
|
|
4907
|
+
const softness = parsed.values.softness !== undefined ? Number(parsed.values.softness) : preset?.softness;
|
|
4908
|
+
const despill = !parsed.values["no-despill"];
|
|
4909
|
+
const outputFormat = parsed.values["output-format"]?.trim().toLowerCase();
|
|
4910
|
+
const outExt = outputFormat === "webp" ? "webp" : "png";
|
|
4911
|
+
const work = mkdtempSync(path.join(tmpdir(), "vidfarm-mask-"));
|
|
4912
|
+
let stemHint = "mask";
|
|
4913
|
+
try {
|
|
4914
|
+
// ---- 1. Obtain the source image (local file or url; reject video) -------
|
|
4915
|
+
let sourcePath;
|
|
4916
|
+
const directPath = path.resolve(process.cwd(), sourceArg);
|
|
4917
|
+
const isLocalFile = !/^https?:\/\//i.test(sourceArg) && existsSync(directPath);
|
|
4918
|
+
if (isLocalFile) {
|
|
4919
|
+
if (/\.(mp4|mov|webm|m4v|mkv)(\?|#|$)/i.test(directPath)) {
|
|
4920
|
+
throw new Error("mask is image-only (a moving subject has no single bounding box). Matte a video with `vidfarm remove-background <video>` or key a flat backdrop with `vidfarm remove-greenscreen <video>`.");
|
|
4921
|
+
}
|
|
4922
|
+
sourcePath = directPath;
|
|
4923
|
+
stemHint = path.basename(directPath).replace(/\.[^.]+$/, "");
|
|
4924
|
+
}
|
|
4925
|
+
else {
|
|
4926
|
+
const sourceUrl = await resolveSingleMediaUrl(ctx, sourceArg);
|
|
4927
|
+
if (/\.(mp4|mov|webm|m4v|mkv)(\?|#|$)/i.test(sourceUrl)) {
|
|
4928
|
+
throw new Error("mask is image-only. Matte a video with `vidfarm remove-background <video>`.");
|
|
4929
|
+
}
|
|
4930
|
+
sourcePath = path.join(work, path.basename(new URL(sourceUrl).pathname) || "source.png");
|
|
4931
|
+
if (!ctx.json)
|
|
4932
|
+
console.log(`${DIM}Downloading source…${RESET}`);
|
|
4933
|
+
await downloadUrlToFile(sourceUrl, sourcePath);
|
|
4934
|
+
}
|
|
4935
|
+
// ---- 2. Optionally isolate ONE illustration from a busy source ----------
|
|
4936
|
+
let regionPath = sourcePath;
|
|
4937
|
+
let cropInfo = null;
|
|
4938
|
+
const cropArg = parsed.values.crop?.trim();
|
|
4939
|
+
if (cropArg) {
|
|
4940
|
+
const parts = cropArg.split(/[,\s]+/).filter(Boolean);
|
|
4941
|
+
if (parts.length !== 4) {
|
|
4942
|
+
throw new Error(`--crop wants four values "x,y,w,h" — pixels (40,120,300,260) or percents (8%,55%,26%,22%). Got "${cropArg}".`);
|
|
4943
|
+
}
|
|
4944
|
+
const cropped = path.join(work, `region.${outExt}`);
|
|
4945
|
+
cropInfo = await cropImageRegion({ sourcePath, outputPath: cropped, x: parts[0], y: parts[1], width: parts[2], height: parts[3] });
|
|
4946
|
+
regionPath = cropped;
|
|
4947
|
+
if (!ctx.json)
|
|
4948
|
+
console.log(`${DIM}Isolated a ${cropInfo.width}×${cropInfo.height} region from the ${cropInfo.sourceWidth}×${cropInfo.sourceHeight} source.${RESET}`);
|
|
4949
|
+
if (parsed.values["keep-region"]) {
|
|
4950
|
+
const keepPath = path.resolve(process.cwd(), String(parsed.values["keep-region"]));
|
|
4951
|
+
mkdirSync(path.dirname(keepPath), { recursive: true });
|
|
4952
|
+
copyFileSync(cropped, keepPath);
|
|
4953
|
+
if (!ctx.json)
|
|
4954
|
+
console.log(`${DIM}Saved the pre-mask region to ${keepPath}${RESET}`);
|
|
4955
|
+
}
|
|
4956
|
+
}
|
|
4957
|
+
// ---- 3. Remove the background → transparent PNG -------------------------
|
|
4958
|
+
const maskedPath = path.join(work, "masked.png");
|
|
4959
|
+
if (useFlatKey) {
|
|
4960
|
+
if (!ctx.json)
|
|
4961
|
+
console.log(`${DIM}Keying out the flat ${presetRaw ?? flatColor} background locally with ffmpeg (free)…${RESET}`);
|
|
4962
|
+
await removeGreenscreenLocal({ sourcePath: regionPath, mediaType: "image", outputPath: maskedPath, keyColor: flatColor, tolerance, softness, despill });
|
|
4963
|
+
}
|
|
4964
|
+
else {
|
|
4965
|
+
if (!ctx.json)
|
|
4966
|
+
console.log(`${DIM}Masking the illustration off its background with the local ONNX matting model (free, no account)…${RESET}`);
|
|
4967
|
+
const run = await runHyperframesCommand("remove-background", [regionPath, "-o", maskedPath, ...(ctx.json ? ["--json"] : [])], { stdio: ctx.json ? "capture" : "inherit" });
|
|
4968
|
+
if (run.code !== 0 || !existsSync(maskedPath)) {
|
|
4969
|
+
const summary = parseHyperframesJson(run.stdout) ?? parseHyperframesJson(run.stderr);
|
|
4970
|
+
const reason = (typeof summary?.error === "string" && summary.error) || run.stderr.trim().split("\n").pop() || "";
|
|
4971
|
+
throw new Error(`mask (matting) failed (exit ${run.code})${reason ? `: ${reason}` : ""}. If the region sits on a flat, solid background, retry with --flat <hexcolor> for a cleaner ffmpeg key.`);
|
|
4972
|
+
}
|
|
4973
|
+
}
|
|
4974
|
+
// ---- 4. Trim to the tight bounding box ----------------------------------
|
|
4975
|
+
const outPath = parsed.values.out
|
|
4976
|
+
? path.resolve(process.cwd(), String(parsed.values.out))
|
|
4977
|
+
: path.resolve(process.cwd(), `${stemHint}.cutout.${outExt}`);
|
|
4978
|
+
mkdirSync(path.dirname(outPath), { recursive: true });
|
|
4979
|
+
const noTrim = Boolean(parsed.values["no-trim"]);
|
|
4980
|
+
const padPx = parsed.values.pad !== undefined ? Number(parsed.values.pad) : 0;
|
|
4981
|
+
const alphaThreshold = parsed.values["alpha-threshold"] !== undefined ? Number(parsed.values["alpha-threshold"]) : undefined;
|
|
4982
|
+
let trim;
|
|
4983
|
+
if (noTrim) {
|
|
4984
|
+
trim = await trimTransparentBorders({ sourcePath: maskedPath, outputPath: outPath, padPx, alphaThreshold: 999 });
|
|
4985
|
+
}
|
|
4986
|
+
else {
|
|
4987
|
+
if (!ctx.json)
|
|
4988
|
+
console.log(`${DIM}Trimming transparent margins to the illustration's true size…${RESET}`);
|
|
4989
|
+
trim = await trimTransparentBorders({ sourcePath: maskedPath, outputPath: outPath, padPx, alphaThreshold });
|
|
4990
|
+
}
|
|
4991
|
+
const shrinkPct = trim.origWidth && trim.origHeight && trim.width && trim.height
|
|
4992
|
+
? Math.round((1 - (trim.width * trim.height) / (trim.origWidth * trim.origHeight)) * 100)
|
|
4993
|
+
: null;
|
|
4994
|
+
if (ctx.json) {
|
|
4995
|
+
printJson({
|
|
4996
|
+
ok: true,
|
|
4997
|
+
target: "local",
|
|
4998
|
+
mode: useFlatKey ? "flat-key" : "matting",
|
|
4999
|
+
out: trim.outputPath,
|
|
5000
|
+
format: trim.format,
|
|
5001
|
+
width: trim.width,
|
|
5002
|
+
height: trim.height,
|
|
5003
|
+
source_width: trim.origWidth,
|
|
5004
|
+
source_height: trim.origHeight,
|
|
5005
|
+
cropped: cropInfo ? { x: cropInfo.x, y: cropInfo.y, width: cropInfo.width, height: cropInfo.height } : null,
|
|
5006
|
+
trimmed: trim.trimmed,
|
|
5007
|
+
area_reduced_pct: shrinkPct,
|
|
5008
|
+
bytes: safeSize(trim.outputPath)
|
|
5009
|
+
});
|
|
5010
|
+
}
|
|
5011
|
+
else {
|
|
5012
|
+
console.log(`${GREEN}Masked illustration ready:${RESET} ${trim.outputPath} ${DIM}(${trim.width}×${trim.height}, ${formatBytes(safeSize(trim.outputPath))}${trim.trimmed && shrinkPct !== null ? `, −${shrinkPct}% area` : ""})${RESET}`);
|
|
5013
|
+
if (!trim.trimmed && !noTrim) {
|
|
5014
|
+
console.log(`${YELLOW}Note:${RESET} ${DIM}no transparent margin to trim — the subject reached the crop's edges, or the background didn't key. If matting left a hard rectangle behind, the background may be busy — try tightening --crop; if it's a flat fill, pass --flat <hexcolor>.${RESET}`);
|
|
5015
|
+
}
|
|
5016
|
+
console.log(`${DIM}Reuse it as a FREE explainer sticker: vidfarm place <dir> --src "${trim.outputPath}" --kind image [--ken-burns zoom-in]. Animate it (zoom/grow/shake/move) with vidfarm keyframes — see the skill's "Cutout graphics for explainers" recipe. Isolate more illustrations from the same source with different --crop rects.${RESET}`);
|
|
5017
|
+
}
|
|
5018
|
+
}
|
|
5019
|
+
finally {
|
|
5020
|
+
rmSync(work, { recursive: true, force: true });
|
|
5021
|
+
}
|
|
5022
|
+
}
|
|
4822
5023
|
// Poll a greenscreen job through the same backend it was submitted to (local
|
|
4823
5024
|
// in-process app or cloud). Mirrors pollPrimitiveJob's terminal conditions.
|
|
4824
5025
|
async function pollGreenscreenJob(ctx, target, jobId) {
|
|
@@ -284,6 +284,53 @@ export async function trimTransparentBorders(input) {
|
|
|
284
284
|
trimmed
|
|
285
285
|
};
|
|
286
286
|
}
|
|
287
|
+
/**
|
|
288
|
+
* Cut a rectangular sub-region out of a still — the "isolate ONE illustration
|
|
289
|
+
* from a busy source image" step used by `vidfarm mask` when the caller wants to
|
|
290
|
+
* grab a single element (a labelled prop, an icon, a mascot) out of a composite
|
|
291
|
+
* (an infographic, a poster, a screenshot) before removing its background. Pure
|
|
292
|
+
* ffmpeg (bundle-safe). Values accept pixels or a `%` of the source dimension,
|
|
293
|
+
* and are clamped into the frame so a slightly-oversized box can't error out.
|
|
294
|
+
*/
|
|
295
|
+
export async function cropImageRegion(input) {
|
|
296
|
+
if (!existsSync(input.sourcePath))
|
|
297
|
+
throw new Error(`No such source file: ${input.sourcePath}`);
|
|
298
|
+
const dims = await probeImageDimensions(input.sourcePath);
|
|
299
|
+
if (!dims)
|
|
300
|
+
throw new Error(`Couldn't read image dimensions for ${input.sourcePath}.`);
|
|
301
|
+
const resolveValue = (raw, span, label) => {
|
|
302
|
+
const t = String(raw).trim();
|
|
303
|
+
const pct = /^(-?\d+(?:\.\d+)?)%$/.exec(t);
|
|
304
|
+
const val = pct ? (Number(pct[1]) / 100) * span : Number(t);
|
|
305
|
+
if (!Number.isFinite(val))
|
|
306
|
+
throw new Error(`Invalid crop ${label} "${raw}" — use pixels (120) or a percent (15%).`);
|
|
307
|
+
return Math.round(val);
|
|
308
|
+
};
|
|
309
|
+
let x = resolveValue(input.x, dims.width, "x");
|
|
310
|
+
let y = resolveValue(input.y, dims.height, "y");
|
|
311
|
+
let w = resolveValue(input.width, dims.width, "width");
|
|
312
|
+
let h = resolveValue(input.height, dims.height, "height");
|
|
313
|
+
// Clamp the rect into the frame so an over-generous box still yields a crop.
|
|
314
|
+
x = Math.min(Math.max(0, x), Math.max(0, dims.width - 1));
|
|
315
|
+
y = Math.min(Math.max(0, y), Math.max(0, dims.height - 1));
|
|
316
|
+
w = Math.min(Math.max(1, w), dims.width - x);
|
|
317
|
+
h = Math.min(Math.max(1, h), dims.height - y);
|
|
318
|
+
const ffmpeg = await resolveFfmpeg();
|
|
319
|
+
const isWebp = /\.webp$/i.test(input.outputPath);
|
|
320
|
+
const args = [
|
|
321
|
+
"-hide_banner", "-y",
|
|
322
|
+
"-i", input.sourcePath,
|
|
323
|
+
"-frames:v", "1",
|
|
324
|
+
"-vf", `crop=${w}:${h}:${x}:${y},format=rgba`,
|
|
325
|
+
...(isWebp ? ["-c:v", "libwebp", "-lossless", "1", "-pix_fmt", "rgba"] : []),
|
|
326
|
+
input.outputPath
|
|
327
|
+
];
|
|
328
|
+
const run = await runFfmpeg(ffmpeg, args);
|
|
329
|
+
if (run.code !== 0 || !existsSync(input.outputPath)) {
|
|
330
|
+
throw new Error(`Crop failed (ffmpeg exit ${run.code})${tail(run.stderr)}.`);
|
|
331
|
+
}
|
|
332
|
+
return { outputPath: input.outputPath, x, y, width: w, height: h, sourceWidth: dims.width, sourceHeight: dims.height };
|
|
333
|
+
}
|
|
287
334
|
/** Default output path next to a source, matching remove-background's naming. */
|
|
288
335
|
export function defaultGreenscreenOutPath(sourcePath, mediaType, outputFormat) {
|
|
289
336
|
const stem = path.basename(sourcePath).replace(/\.[^.]+$/, "");
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@officexapp/vidfarm-devcli",
|
|
3
|
-
"version": "0.21.
|
|
3
|
+
"version": "0.21.21",
|
|
4
4
|
"description": "Local bridge for the Vidfarm Trackpad Editor. `vidfarm serve <template_id>` boots the FULL editor on localhost (disk-backed records/storage, free in-process render); edit composition.html on disk (Claude Code, Codex, etc.) and the browser live-morphs it.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|