@officexapp/vidfarm-devcli 0.21.26 → 0.21.28
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/editor-capabilities/SKILL.md +26 -0
- package/.agents/skills/vidfarm/SKILL.md +31 -9
- package/.agents/skills/vidfarm/recipes/local-edit-render-approve.md +3 -2
- package/.agents/skills/vidfarm/references/assets-and-sourcing.md +29 -3
- package/.agents/skills/vidfarm/references/automation-and-local-dev.md +59 -6
- package/.agents/skills/vidfarm/references/core-workflows.md +1 -1
- package/.agents/skills/vidfarm/references/editor-workflows.md +85 -7
- package/.agents/skills/vidfarm/references/onboarding.md +4 -4
- package/.agents/skills/vidfarm/references/primitives.md +33 -1
- package/.agents/skills/vidfarm-media/SKILL.md +3 -1
- package/.agents/skills/vidfarm-media/references/tts.md +1 -1
- package/SKILL.director.md +245 -33
- package/SKILL.md +11 -4
- package/clipper.md +3 -2
- package/dist/src/cli.js +478 -19
- package/dist/src/devcli/cost-mode.js +46 -18
- package/dist/src/devcli/qa-check.js +420 -0
- package/dist/src/devcli/sequence.js +619 -0
- package/dist/src/services/sequence-prompts.js +463 -0
- package/package.json +5 -1
- package/public/assets/file-directory-app.js +26 -26
- package/public/assets/homepage-client-app.js +12 -12
- package/public/serve-shells/editor.html +66 -3
- package/public/serve-shells/library-files.html +60 -3
- package/public/serve-shells/library-raws.html +60 -3
- package/public/serve-shells/tools-clipper.html +60 -3
- package/public/serve-shells/tools-image.html +1316 -617
- package/public/serve-shells/tools-video.html +880 -161
|
@@ -77,6 +77,32 @@ Audio is natively **multi-track**. The timeline mixes UNLIMITED simultaneous `<a
|
|
|
77
77
|
- **The headline move — split a combined original when recreating.** When the user recreates a template whose ORIGINAL had music + narration baked into ONE audio track, do NOT reproduce a single combined bed. Rebuild it as TWO independent tracks: a fresh narration track (`/api/v1/primitives/audio/speech`, or same-voice reword via `/api/v1/primitives/audio/regenerate-speech`) at ~1.0, and a separate real music track at ~0.1–0.2 — then mute or `remove_layer` the original combined source-audio layer so the old voice doesn't play under the new one. This hands the user independent voice/music volume and is the elegant workaround for AI TTS being unable to emit narration+music in one file.
|
|
78
78
|
- **Honesty (ties to the create-media rules):** you cannot un-mix / stem-separate the original's baked audio — the two tracks are BUILT from a fresh narration track PLUS a real music file (owned / user-provided / `browse_files` across `/files` and `/raws`), never a faked "music" layer and never the voice track duplicated. There is no music-generation primitive.
|
|
79
79
|
|
|
80
|
+
## The TEXT axis — social-native only, zero HTML slop (hard constraint)
|
|
81
|
+
|
|
82
|
+
You author into HTML, which makes it dangerously easy to build a **web page instead of a video**. This is the #1 way an AI-edited composition betrays itself. Apply on every text/graphic you place — and strip it when a decomposed fork or a pasted brand asset brings one in.
|
|
83
|
+
|
|
84
|
+
**The test:** *would this element exist if the video were shot on a phone and captioned in CapCut?* If its whole job is to look **clickable**, cut it. Nothing in a video is clickable.
|
|
85
|
+
|
|
86
|
+
**BANNED (never `add_layer` / `replace_composition_html` these):**
|
|
87
|
+
- **CTA buttons** — a filled/gradient capsule with action copy ("Sign Up for a Free Trial →", "Get Started", "Book a Call"), glow or drop shadow. A social CTA is *spoken* or a plain caption line.
|
|
88
|
+
- **Badge / chip / pill rows** — "✓ ID-Verified · ✓ No Credit Card Needed · ✓ 30-Min Trial". Say them as three *timed caption lines* on the footage instead.
|
|
89
|
+
- **Cards / panels** — a bordered, shadowed, or `backdrop-filter`-frosted rounded box holding a headline + subheading/URL. Text goes ON the footage, not in a floating panel.
|
|
90
|
+
- **Gradient text fills, neon border glows, elevation shadows, glassmorphism**, navbars, hero sections, feature grids, `<ul>` bullet lists, tables, "as seen in" strips.
|
|
91
|
+
- **Web-default type** — Inter/Roboto/system-ui/Arial/Helvetica at weight 400–600 and 16–24px.
|
|
92
|
+
|
|
93
|
+
**Greppable smell test:** typing `btn`, `badge`, `chip`, `tag`, `card`, `panel`, `container`, `row`/`col-`, `hero`, `cta`, `rounded-full`, `shadow-lg`, `backdrop-blur`, `bg-gradient-to-r`, `border border-…` → **stop and rewrite as timed text on footage.** A re-theme onto a SaaS/product subject is exactly where this creeps in — do NOT import the brand's website design language into the video.
|
|
94
|
+
|
|
95
|
+
**ALLOWED / encouraged:** arrows (drawn or animated), circle / scribble / underline highlight strokes, **italics**, ALL-CAPS, a single word in a pop color, mixed weights, inline emoji (sparingly), transparent cut-out stickers via `create-overlay`, and mock social UI (iMessage bubbles, a TikTok comment card, a countdown bar) — all native to the platform.
|
|
96
|
+
|
|
97
|
+
**Font + background regime (every caption/title, via `set_captions` / `set_layer_style` / `add_layer`):**
|
|
98
|
+
- **Font:** Montserrat (default) or TikTok Sans / Abel / Source Code Pro / Yesteryear — a family the composition actually imports, or it silently falls back to the slop sans. Weight **700–900**. `font_size` in px of the render canvas: **~36–64px** on a 1080-wide frame; never <28, never 0 (invisible). ~2 lines, ~5 words per line; `line_height` 0.95–1.15.
|
|
99
|
+
- **Position:** inside the **8%–85%** vertical safe zone (phone UI clips the edges) and clear of the right ~12% action rail — a centered box at `x:10 width:80` is safe. Lower-third ≈ `y:70`; a "POV:" top line ≈ `y:8`, never `y:0`.
|
|
100
|
+
- **Background — exactly one of four:** `background_style:"outline"` (stroke, the default look) · `"plain"` (bare + soft shadow) · an **active-word highlight pill** via `set_captions caption_style:"spotlight"|"karaoke"` (the *only* legitimate pill — it tracks the spoken word) · `"highlight-solid"`/`"highlight-translucent"` as a band that **hugs** the text (radius ≤~8px, no border, no shadow, no gradient, no blur, one text run — never a heading+subheading+URL stacked inside it). Anything else is a web card.
|
|
101
|
+
|
|
102
|
+
**There is no QA tool for you.** The devcli ships `vidfarm qa <dir>` — a free local blocklist pass over exactly the rules above — but it is **devcli-only with no REST twin**, so in the web editor you enforce this by reading your own output. When you hand a heavy job off to a local coding agent, tell them to run `vidfarm qa ./work` before rendering.
|
|
103
|
+
|
|
104
|
+
Deeper rationale and the devcli-side twins live in `vidfarm` → `references/editor-workflows.md` ("Social-native visual standard" / "TikTok-native caption standard").
|
|
105
|
+
|
|
80
106
|
## The `editor_action` verb catalog
|
|
81
107
|
|
|
82
108
|
Every mutation is one `editor_action` call with an `explanation`. Reference keys/tracks/durations from the latest `editor_context`; never invent `layer_key`s. Grouped by axis of work:
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: vidfarm
|
|
3
|
-
description: Use Vidfarm as a director. Browse/add inspiration videos, browse
|
|
3
|
+
description: Use Vidfarm as a director. Run a strategy **consultation** (the `brainstorm/*` chain — cold-start interview, awareness stages, persuasive angles, hooks, product placement). Browse/add inspiration videos, browse the free public raws catalog BY CATEGORY (curated shelves like scroll-stoppers/greenscreen/reaction — the cheapest way to source footage for one video, and a ready-made clip pool for bulk scripting N variants), fork a template into a composition, edit it in the Trackpad Editor (timeline-based like Premiere/DaVinci), auto-decompose source video into scenes, render to MP4, approve into a shareable post, and schedule it. Includes login, provider keys, discovery, versioning, uploads/downloads, and billing. Every step is available as raw REST; `vidfarm-devcli` wraps those routes and composes the file-backed scripting flows.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Vidfarm Director
|
|
@@ -81,13 +81,14 @@ Do not use this skill to author new templates from scratch, deploy platform infr
|
|
|
81
81
|
|
|
82
82
|
## Cost mode — ask before you spend, then remember it
|
|
83
83
|
|
|
84
|
-
Vidfarm work can burn real AI credits on the user's wallet / provider keys. **Save them money by default.** Before the first billed step of a session, ask the user which spend posture they want, and quote the per-video cost in one plain line each:
|
|
84
|
+
Vidfarm work can burn real AI credits on the user's wallet / provider keys. **Save them money by default.** Before the first billed step of a session, ask the user which spend posture they want, and quote the per-video cost in one plain line each. The first two are about *how much* AI you buy; the last two are two different *production methods* — rich-ai buys reusable greenscreen raws and animates them in HTML, pure-videogen generates every frame:
|
|
85
85
|
|
|
86
86
|
| Mode | Cost per video | What it means |
|
|
87
87
|
|---|---|---|
|
|
88
88
|
| **minimize** | **$0 — explicitly free** | free local compute + free stock media only; billed AI is refused without `--yes` |
|
|
89
89
|
| **hybrid** *(recommended)* | **~$0.01–$1** | free where it's free; pay AI only where it clearly wins |
|
|
90
|
-
| **rich-ai** | **$1+** |
|
|
90
|
+
| **rich-ai** | **$1+** | AI video gen mints **reusable greenscreen raws**, then hyperframes HTML/CSS motion remixes them with the rest of the library — the footage is saved and reused later |
|
|
91
|
+
| **pure-videogen** | **$5+** | every shot is generated: text script → image storyboard → frame-by-frame scene generation. No reuse, no HTML motion |
|
|
91
92
|
|
|
92
93
|
**Every dollar above is billed to the user's OWN AI provider keys (BYOK)** — the keys they save via `vidfarm add-provider-key <provider> <key>` or **Settings → Bring your own keys** (<https://vidfarm.cc/settings/developer>). Vidfarm isn't marking it up; the model providers charge it directly. (Vidfarm wallet credits are only spent when the user deliberately runs on the platform key instead of their own.) Say this out loud the first time you ask — it's why `minimize` really is $0: with no key spend, nothing can be charged.
|
|
93
94
|
|
|
@@ -98,14 +99,31 @@ Vidfarm work can burn real AI credits on the user's wallet / provider keys. **Sa
|
|
|
98
99
|
- **hybrid** *(recommend this)* — **~$0.01–$1 per video, on their BYOK key.** Free where it's free; pay for AI only where it clearly wins (a hero shot, a voice you can't fake locally). A mostly-hyperframes video with one generated image lands near the low end; a few AI images plus premium narration approaches the high end.
|
|
99
100
|
- **Narration starts free here too.** In BOTH `minimize` and `hybrid`, a bare `vidfarm tts "…"` runs the keyless local Kokoro-82M voice — no flag needed. Ask for a premium voice explicitly (`--style`, `--voice <provider preset>`, `--provider`, `--own-key`, or `--cloud`) when the piece needs one, and say so out loud so the user knows what they're paying for. `rich-ai` skips the free default entirely. If the local engine isn't installed on their machine, the run falls back to their provider key / cloud automatically and prints why.
|
|
100
101
|
- **Free-TTS draft, premium-TTS final — now the default shape, so SAY it out loud.** Narration gets rewritten several times before the script locks, and paying for premium ElevenLabs TTS on every draft wastes money re-voicing lines that will change. The CLI defaults to the free local voice in `hybrid`, so drafts are free automatically — but a robotic draft voice reads as "broken" to a user who wasn't told. **Tell them the first time you generate narration**: the draft voice is the free local one, and you'll re-voice the approved final script in premium TTS (`--cloud` ElevenLabs, or `--style`/`--voice` on their own key) right before the last render. If they'd rather hear the real voice the whole way, pass the premium flags from the start — and if the piece is voice-led (a testimonial, a brand spot where the voice IS the product), offer premium up front instead of drafting free.
|
|
101
|
-
- **rich-ai** — **$1+ per video, on their BYOK key
|
|
102
|
+
- **rich-ai** — **$1+ per video, on their BYOK key.** AI video generation is used, but **not to generate the finished video** — it's used to mint **reusable greenscreen raws**, which are keyed once and then animated/remixed in HTML. The shape of a rich-ai video:
|
|
103
|
+
1. **Generate the cast and props as greenscreen raws.** `vidfarm avatar "<who>" --say "<line>"` for a talking presenter, `vidfarm create-overlay "<subject>"` (or the free-keying `vidfarm cutout --generate "<prompt>"`) for illustrations and props, and `vidfarm generate video --prompt "<subject> … on a flat solid #00FF00 background"` for motion elements. Everything is generated on greenscreen *on purpose*, so it composites over any background later.
|
|
104
|
+
2. **Key it once, keep the plate.** The primitives key the plate in the same job (transparent WebM with audio for avatars, transparent PNG/WebP for overlays) and hand back `greenscreen_source_url` too — so you can re-key at a different tolerance without paying for a second generation. `--local` keys on the user's machine with free ffmpeg.
|
|
105
|
+
3. **Animate and remix in hyperframes, not in the generator.** Motion, camera moves, transitions, text, and staging are **HTML/CSS + hyperframes keyframes over the keyed raws**, combined with existing library raws and public raws. Generated seconds are expensive; HTML motion is free — so buy *assets* with AI and get *motion* for $0.
|
|
106
|
+
4. **SAVE every generated raw so the spend amortizes — this is the whole point of the mode.** A generated asset that only exists inside one composition was rented; one that's in the library was *bought*. Persist it the moment the job settles:
|
|
107
|
+
- **Foldered + annotated in the library** — `vidfarm put-file ./zara-keyed.webm --folder greenscreen/zara --notes "Zara, teal jacket, transparent WebM w/ audio, keyed from the green plate — reuse as the presenter in any 9:16 spot"`. The notes are vector-embedded, so `vidfarm files --search "presenter in a teal jacket"` finds it next month. Keep the un-keyed `greenscreen_source_url` plate too — re-keying it later is free, re-generating is not.
|
|
108
|
+
- **Footage-shaped raws** can go into the `/raws` library instead (`vidfarm clipper ./generated.mp4 --folder greenscreen-cast --name "shrug"`), where `vidfarm raws search "<meaning>"` / `raws list` / `raws export <ids…> --to ./dir` reach them.
|
|
109
|
+
- **Local by default; cloud when they want it.** Both stores live on the user's own machine under `~/.vidfarm` — free, offline, no account needed. If the user is on a **paid/Pro Vidfarm plan** and wants them everywhere (second machine, teammates, the web editor), mirror with **`vidfarm sync push /files`** and **`vidfarm sync push /raws`** (`sync pull` on the other side; `--dry-run` first). Offer the sync; don't assume it — a free-plan user keeps everything locally and loses nothing.
|
|
110
|
+
- **Say this out loud when you propose rich-ai**: the $1+ is a one-time purchase of a reusable cast/prop library, and the *next* video that reuses those raws can run in `hybrid` or even `minimize` for ~$0.
|
|
111
|
+
- Practical rule: before generating anything in rich-ai, **search what already exists** — `vidfarm raws search "<meaning>"` (their own library, free) then `vidfarm public-raws --category greenscreen --query "<meaning>"` (the free public shelf). Only generate the raws that genuinely don't exist yet.
|
|
112
|
+
- When a run is likely to go well past $1, say the number before you spend it.
|
|
113
|
+
- **pure-videogen** — **$5+ per video, on their BYOK key. The most expensive mode — never enter it by accident.** Here the *entire* video is generated footage: no raw reuse, no HTML motion, no compositing tricks. It buys cinematic continuity that keyed-raws-plus-HTML can't fake. Do it in this order — the planning is what keeps the cost from doubling:
|
|
114
|
+
1. **Script it in TEXT first.** Write the whole piece as a beat-by-beat script (shot list, on-screen action, dialogue/narration, duration per scene) and get the user to approve it *before* any generation call. Text iterations are free; regenerating a scene because the script changed is not.
|
|
115
|
+
2. **Storyboard it as IMAGES.** Generate a still keyframe per scene with `vidfarm generate image …` (cheap — cents, not dollars) and approve the look, cast, framing, and continuity on the stills. Lock characters with a reference/sprite card (`--ref`) so faces and wardrobe survive across scenes. Fix the storyboard until it's right; this is where art direction happens.
|
|
116
|
+
3. **Then generate the scenes frame-by-frame from those keyframes.** Drive each video generation off its approved storyboard still (start frame, and the previous scene's last frame as the next scene's start frame for continuity) — `frame_images` / `--ref` on the video primitives. Generate scene by scene and review each before generating the next; do not fan out the whole film in one shot.
|
|
117
|
+
4. **Assemble, don't re-render.** Cut the approved scenes onto the timeline and do captions, audio, and pacing in hyperframes/the editor as usual — never regenerate footage to fix a caption or a trim.
|
|
118
|
+
- Quote the real number before starting: N scenes × the provider's per-second video price, plus storyboard images. If the piece would survive as rich-ai (greenscreen cast on HTML backgrounds), **say so and offer that first** — it is typically 5× cheaper.
|
|
119
|
+
- **Don't hand-roll steps 1–4 — `vidfarm sequence` is that pipeline.** `vidfarm sequence init "<brief>" --dir ./x` scaffolds a plan (+ a `PLAN_TASK.md` you can direct yourself or hand to a coding agent), then `characters` → `boards` → `prompts` → `animate` → `assemble`, all resumable off one `sequence.json`. `vidfarm sequence run` walks it with a confirm gate before each spend; `--yes` runs it unattended. It writes the character-card and storyboard prompts in a block grammar that states reference priority (card = identity, board = staging) and forbids animating the board sheet itself — the two failure modes that otherwise eat a video credit each. Preview every prompt for $0 with `vidfarm sequence prompts --print board:part-01 | shot:part-01 | card:<slug>`. See `references/editor-workflows.md` → *`vidfarm sequence`*.
|
|
102
120
|
|
|
103
121
|
Then **ask whether to remember the choice**, and save it where it will actually be read next time:
|
|
104
122
|
|
|
105
|
-
- **Local coding agent (devcli):** run `vidfarm cost-mode <choice>` — it persists to `~/.vidfarm/cost-mode.json`, and every billed devcli command then respects it (in `minimize`, billed ops refuse to run without `--yes`; in `hybrid`/`rich-ai` they print each op's cost). Also offer to record it in the agent's own memory so a *future* session recalls it — but which memory differs by agent, so ask, don't assume: Claude Code → `CLAUDE.md` (or its memory dir); Codex / OpenCode / most others → `AGENTS.md`; or a plain note file the user names. A third-party memory tool the user already uses is fine too. Never write to a memory file the user didn't approve.
|
|
123
|
+
- **Local coding agent (devcli):** run `vidfarm cost-mode <choice>` — it persists to `~/.vidfarm/cost-mode.json`, and every billed devcli command then respects it (in `minimize`, billed ops refuse to run without `--yes`; in `hybrid`/`rich-ai`/`pure-videogen` they print each op's cost). Also offer to record it in the agent's own memory so a *future* session recalls it — but which memory differs by agent, so ask, don't assume: Claude Code → `CLAUDE.md` (or its memory dir); Codex / OpenCode / most others → `AGENTS.md`; or a plain note file the user names. A third-party memory tool the user already uses is fine too. Never write to a memory file the user didn't approve.
|
|
106
124
|
- **Web app UI (the /editor copilot, chat dock, /chat, /library):** there is no agent-memory file to write, so **do not** try to persist it — just ask each time, unless the user tells you their standing preference in this conversation (then honor it for the session).
|
|
107
125
|
|
|
108
|
-
If the user hasn't picked yet and you're about to spend, name the cheaper path and the cost, and ask. `vidfarm cost-mode` with no argument prints the current mode +
|
|
126
|
+
If the user hasn't picked yet and you're about to spend, name the cheaper path and the cost, and ask. `vidfarm cost-mode` with no argument prints the current mode + an explanation of each for you to relay.
|
|
109
127
|
|
|
110
128
|
## Default stance
|
|
111
129
|
|
|
@@ -147,6 +165,8 @@ Directors also accumulate a **reusable media asset library** — logos, stickers
|
|
|
147
165
|
|
|
148
166
|
Present both harnesses to the director, recommend (A) unless they've asked for premium or budget covers it, and explain the tradeoff in these terms. Full methodology: `references/editor-workflows.md` (“The three paintbrushes & two replication harnesses”); cost bands: `references/core-workflows.md` (Cost spectrum).
|
|
149
167
|
|
|
168
|
+
**"Create an avatar" = a TALKING-HEAD VIDEO with audio, keyed off a greenscreen.** When a director asks for an **avatar** — or a spokesperson, presenter, host, UGC creator, talking head — they never mean a still portrait and never a silent clip. Use **`vidfarm avatar "<who they are>" --say "<the line they speak>"`** (`POST /api/v1/primitives/videos/create-avatar`): it generates the talking head **with lip-synced spoken audio** on a **flat greenscreen plate at an exact known key color**, then keys that plate out in the same job, returning a **transparent presenter WebM** (audio intact) that composites over whatever background the scene needs — in the preview and the final render. Pass `--ref` (a headshot, or the character's `character_sprite_card.png`) to lock the likeness across videos, `--aspect-ratio`/`--duration` to fit the slot, and `--place ./dir --at <time>` to drop it straight onto a composition. **Never hand back the green plate as the finished avatar** — the green comes off before it goes on a timeline. `--local` keys the plate on your machine with free ffmpeg (generation still rides the user's own cloud video keys); `--keep-greenscreen` stops at the plate. Details: `references/primitives.md` → "Primitive: talking_avatar". Generation is AI video — the priciest thing here — so in `cost-mode minimize` offer a free talking-head clip (`vidfarm public-raws --category talking-head`) first.
|
|
169
|
+
|
|
150
170
|
**Explainer/cutout videos — the transparent-sticker workflow.** For explainers (a subject "on stage" while labels, arrows, and props pop in around it), the cheap workhorse is a **transparent cutout sticker**: `vidfarm cutout --generate "<subject>"` AI-generates the graphic on a chroma plate, keys it out, **and trims the canvas down to the subject's true min width/height** — one free local ffmpeg step, no mostly-empty PNG to fight with — then `vidfarm place` + `vidfarm keyframes` scale/position and animate it (zoom, grow, shake, drift). It's the same "generate a reusable element once, then reuse it" thrift as the cheap harness, tuned for stickers. Full guided harness: recipe `recipes/cutout-graphics-for-explainers.md`; placement + zoom/grow/shake/move motion recipes: `references/editor-workflows.md` (“Cutout graphics for explainers”).
|
|
151
171
|
|
|
152
172
|
**Explainer house style — the defaults to build with unless told otherwise.** **White background / light mode** (plain white stage, no gradients, no dark mode, no photo backdrop), **kinetic word-by-word captions** in dark ink on the light stage (`vidfarm captions generate --style word-pop --color "#111111" --active-color "#7C3AED" --background-style plain` — skip outlines/shadows, they're only needed over busy footage), and **female TTS narration** (`vidfarm tts --voice coral` on OpenAI — `nova` for energy, `sage` for calm; `Kore`/`Leda` on Gemini; any ElevenLabs voice via `vidfarm voices`). **Keep it clean and simple** — one idea on screen at a time, two or three cutouts per beat, one accent color, one font, lots of white space; remove before you add. **Illustrations default to simplicity**: flat vector, simple shapes, minimal detail, 2–3 flat colors, no baked-in text — simple art keys cleanly, trims tight, and stays on-style across the whole cast. State the defaults once so the director can override any of them. Full detail: recipe `recipes/cutout-graphics-for-explainers.md` (“House style — the explainer defaults”).
|
|
@@ -203,7 +223,9 @@ You may be running as the **in-web AI chat** (the /editor copilot, the chat dock
|
|
|
203
223
|
|
|
204
224
|
- **Determine the surface before claiming capabilities.** The web chat has only its declared tools and REST routes. It cannot execute arbitrary JavaScript/Python, open a shell, create a local repository script, or use the user's filesystem. Never tell a web-chat user that you ran code or wrote a script unless a dedicated declared tool actually did so. A desktop coding agent has a real shell and filesystem and MAY write/run scripts, perform arbitrary local computations over paginated API results, create reports/CSVs/JSON, edit composition files, and orchestrate long devcli workflows within the user's authorization.
|
|
205
225
|
- **The web AI chat can do all three paintbrushes** — clip raws, author HTML/hyperframe motion, and generate AI media — and it drives edits directly on the live timeline. Keep small-to-medium jobs here: text/caption swaps, a scene or two replaced, single generations, captions, approve/schedule. Just do them.
|
|
206
|
-
- **
|
|
226
|
+
- **No HTML slop — a video is not a web page.** Compositions are authored in HTML, so the #1 tell of an AI-edited video is landing-page furniture: gradient CTA "buttons" ("Sign Up for a Free Trial →"), rows of benefit chips/badges ("✓ No Credit Card Needed"), frosted/bordered cards holding a gradient headline + a URL, feature grids, bullet lists, web-default type (Inter/Roboto/Arial at weight 400-600). None of that exists in a real TikTok/Reel — **nothing in a video is clickable**. If you are typing `btn` / `badge` / `chip` / `card` / `rounded-full` / `shadow-lg` / `backdrop-blur` / `bg-gradient-to-r`, stop and rewrite it as timed text on the footage. Arrows, scribble/underline marks, italics, ALL-CAPS, single-word color pops, emoji, transparent cut-out stickers, and mock social UI (iMessage bubbles, comment cards) are all *fine* — they're native to the platform.
|
|
227
|
+
- **On devcli, QA every video you produce: `vidfarm qa ./work`.** Free, instant, local-only — it blocklists exactly the slop above plus font-regime/safe-zone drift, and prints a concrete fix per finding. **Feedback, not a gate**: it exits 0 even on findings, never runs automatically, and is a blocklist (unusual/stylized compositions pass untouched), so there's no reason not to run it before every publish. `--json` for scripted batches, `--strict` only if you want a CI failure. **Web-chat copilot: this command does not exist for you** (devcli-only, no REST twin) — apply the standard by hand, and when handing a heavy job to a local coding agent, tell them to run `vidfarm qa`.
|
|
228
|
+
- **Every production adheres to the TikTok-native caption standard.** On-screen text lives inside the readable safe zone (**~8%–85%** of a 9:16 frame — never pinned to the top/bottom edges the phone UI clips), uses the composition's bold font regime (**Montserrat** default / TikTok Sans, weight **700–900**, ~36–64px on a 1080-wide frame), and uses **exactly one of four valid backgrounds**: outline/stroke (`background_style:"outline"`, the default), plain + shadow (`"plain"`), an active-word highlight pill (`set_captions` `spotlight`/`karaoke` — the only legitimate pill), or a tight-hugging solid band (`"highlight-solid"`, radius ≤8px, no border/shadow/gradient/blur). Decomposed forks often inherit the source's edge-pinned caption in an off-regime font — fix it, don't inherit it. Local devcli renders auto-normalize position + font family only (never the slop), so author it correctly. Full rules in `references/editor-workflows.md` ("Social-native visual standard" + "TikTok-native caption standard").
|
|
207
229
|
- **Where the web chat struggles: complex, long, multi-step transformations.** A full multi-scene re-theme, an iterative render-critique-iterate loop, heavy scripted or batch work, or anything needing a real filesystem and many sequential tool calls will hit context limits, turn/timeout ceilings, and the web editor's constraints (CSS/declarative motion only — JS animation adapters are stripped on save). Don't grind a big transformation one layer at a time in a chat turn and stall.
|
|
208
230
|
- **Practical workaround — hand the heavy job to local devcli.** When a task is genuinely large or long-running, **proactively recommend the director run it locally with an AI coding agent** (Claude Code / OpenAI Codex / any capable agent): `vidfarm pull <forkId>` writes the composition + the `.harness/` grounding bundle to disk, the agent edits with the full devcli verb set and JS animation adapters, renders free with `vidfarm serve`, and `vidfarm publish` pushes it back. This is the **best-quality (B) harness's** natural home (adversarial grading with a coding agent). Frame it as "this is a big rebuild — you'll get a better, faster result running it locally with a coding agent; here's how," not as a dead end.
|
|
209
231
|
- **Offer a handoff, do not impersonate the desktop agent.** When web chat reaches that boundary, offer to save a Markdown handoff in My Files containing the objective, selected template/fork IDs, asset paths, grounding, constraints, completed work, and suggested devcli commands. Create it only after the user agrees. The desktop agent should read that document, pull the referenced fork, and then use its actual code/shell capabilities.
|
|
@@ -217,7 +239,7 @@ Read only the relevant reference file for the current task.
|
|
|
217
239
|
- Timeline editing, decompose, captions, motion, AI placement: `references/editor-workflows.md`
|
|
218
240
|
- Raws hunts, My Files, recurring characters, asset retrieval: `references/assets-and-sourcing.md`
|
|
219
241
|
- REST automation, `vidfarm` command surface, local serve loop, skill packs: `references/automation-and-local-dev.md`
|
|
220
|
-
- Getting-started interviews, strategy docs, onboarding flow: `references/onboarding.md`
|
|
242
|
+
- Getting-started interviews, **consultations**, strategy docs, onboarding flow: `references/onboarding.md`
|
|
221
243
|
- Primitive routes such as TTS, STT, music, overlays, background removal, product placement: `references/primitives.md`
|
|
222
244
|
- Complete REST API map and raw-HTTP conventions: `references/rest-api.md`. Load it only when the user asks for REST, an endpoint/schema, direct HTTP integration, or exhaustive API coverage. For the entire specification, follow its domain links and load every listed reference; do not preload them into ordinary director conversations.
|
|
223
245
|
|
|
@@ -239,7 +261,7 @@ HyperFrames authoring and rendering in this package are Vidfarm-native: local wo
|
|
|
239
261
|
|
|
240
262
|
Choose the narrowest path that satisfies the request.
|
|
241
263
|
|
|
242
|
-
1. If the user needs help figuring out what to make, read `references/onboarding.md` first.
|
|
264
|
+
1. If the user needs help figuring out what to make, **or asks for a "consultation"** (the `brainstorm/*` chain: cold-start interview → awareness stages → angles → hooks), read `references/onboarding.md` first.
|
|
243
265
|
2. If the user already knows the goal and needs a suitable template, read `references/core-workflows.md` and use the template discovery flow.
|
|
244
266
|
3. If the task is “change this video,” read `references/editor-workflows.md`.
|
|
245
267
|
4. If the task is “find footage” or “use our existing assets,” read `references/assets-and-sourcing.md`.
|
|
@@ -7,8 +7,9 @@ Use this when a coding agent is doing the work locally or the user wants a repro
|
|
|
7
7
|
3. Read `./work/.harness/agent-guide.md` and `./work/.harness/context.json` before editing.
|
|
8
8
|
4. Make deterministic edits to `composition.html` and optionally `composition.json`.
|
|
9
9
|
5. Validate with `vidfarm lint` or `vidfarm stills` when useful.
|
|
10
|
-
6.
|
|
11
|
-
7.
|
|
10
|
+
6. **QA before you render: `vidfarm qa ./work`.** Free, instant, devcli-only. It blocklists HTML slop (CTA buttons, benefit chip rows, frosted cards, gradient text, web-page classes/fonts) and checks the caption font regime + safe zone. Feedback only — exit 0 even on findings, never automatic — but it catches the #1 tell of an agent-made video, so run it on every production. Fix what's real, ignore what's a deliberate style call, then render.
|
|
11
|
+
7. Render with `vidfarm render <forkId> --dir ./work --wait`.
|
|
12
|
+
8. Approve the finished MP4 with `vidfarm approve --video <url|./final.mp4> --caption "..."`. This prints the shareable `share_url`.
|
|
12
13
|
|
|
13
14
|
**Approving a locally rendered file → cloud preview link.** Approve takes media by **URL**, not bytes, and an approved post is **permanent** — so the local MP4 must land in **durable My Files**, not the 30-day temp store (a temp video would 404 the share page after 30 days). The devcli presigns, PUTs the bytes **direct to S3**, finalizes, then approves with that durable URL — so `vidfarm approve --video ./final.mp4` handles files up to **200 MB**, bypasses the ~6 MB Lambda request-body limit, and the share link never breaks. By raw REST: `POST /api/v1/user/me/attachments/presign` → PUT to the returned S3 URL → `POST /api/v1/user/me/attachments` (finalize) → pass the returned `viewUrl` in the approve `media` array. Do not multipart-POST a big file to `.../attachments/upload` against the cloud host (Lambda-bound, ~6 MB cap). Add `vidfarm approve --temp` only when you want a disposable 30-day preview.
|
|
14
15
|
|
|
@@ -15,6 +15,30 @@ Both are async primitive jobs — poll `GET /api/v1/primitives/jobs/:jobId` unti
|
|
|
15
15
|
|
|
16
16
|
**Don't reach for this when the real goal is clips.** If the user wants short clips mined out of a long video, `POST /raws/scan` with `source_url` already downloads the source for you as part of the hunt (below) — a separate download call is wasted spend.
|
|
17
17
|
|
|
18
|
+
## Public raws catalog — browse BY CATEGORY (check here first)
|
|
19
|
+
|
|
20
|
+
Before you hunt footage and long before you generate it, check what the platform **already has**. The public raws catalog is a curated, pre-tagged, free-to-use pool of short clips, organised into **categories** (shelves) — and browsing it by category is one of the highest-leverage moves available to you, for two reasons:
|
|
21
|
+
|
|
22
|
+
1. **Making one video** — a category answers "what could go here?" faster than any search string. You don't have to describe the clip you want; you scan a shelf of clips that already fit the job.
|
|
23
|
+
2. **Bulk scripting** — a category IS a ready-made clip pool. Pull one shelf, and you have N interchangeable clips to fan a single composition across N variants (N hooks, N B-roll beds, N reaction cutaways) in one loop. This is the cheapest way to produce volume: no hunt, no downloads, no generation spend, no per-clip AI call.
|
|
24
|
+
|
|
25
|
+
**Always list the shelves before pulling clips** — it's one cheap call, it tells you what exists and how much of it, and it stops you inventing a query for footage that isn't there:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
vidfarm public-raws --categories # shelves + live counts + what each is FOR
|
|
29
|
+
vidfarm public-raws --category scroll-stoppers --limit 40
|
|
30
|
+
vidfarm public-raws --category greenscreen --query "person shrugging"
|
|
31
|
+
vidfarm public-raws --bookmark <raw_id> # save one into /raws/public/* for reuse
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
REST twins: `GET /api/v1/public-raws/categories` (shelves + counts) and `GET /api/v1/public-raws?category=<shelf>&q=…`. Web: `https://vidfarm.cc/discover/raws/categories`.
|
|
35
|
+
|
|
36
|
+
`--category` takes a **curated shelf** (`scroll-stoppers`, `greenscreen`, `reaction`, `talking-head`, `b-roll`, `lifestyle`, `product-shots`, `demos`, `text-graphics`) or an exact taxonomy key (`emotion:confused`, `content_type:reaction`). A clip can sit on more than one shelf — a greenscreen reaction is honestly both — so shelves overlap by design. Don't hardcode the shelf list: read it from `--categories`, which is the live source of truth.
|
|
37
|
+
|
|
38
|
+
What each shelf is for, in practice: **Scroll Stoppers** are half-second pattern-breaks to cut over your first frame so the thumb stops; **Greenscreen** clips key straight onto any background via the greenscreen primitive; **Reaction** clips are the cutaway that tells the viewer how to feel about what you just showed them; **B-Roll** buys you a cut whenever narration has to keep going.
|
|
39
|
+
|
|
40
|
+
A bookmarked public raw lands in the director's own `/raws/public/*` folder, after which it behaves like any other library raw (`set_layer_media` / `vidfarm set-media` to drop it on the timeline).
|
|
41
|
+
|
|
18
42
|
## Raws (long-form → short-form raws)
|
|
19
43
|
|
|
20
44
|
Mine a **long-form** video (podcast, stream VOD, webinar, any YouTube/TikTok/IG/X URL, or an upload) into a library of tagged, searchable **raws**. This is the `/raws` surface — browse it at `https://vidfarm.cc/library/raws` (the Library page's "Approved / Raws" tabs).
|
|
@@ -70,11 +94,11 @@ Local scans persist to a SQLite library under `~/.vidfarm` (free compute, subscr
|
|
|
70
94
|
|
|
71
95
|
## My Files (the user's asset library)
|
|
72
96
|
|
|
73
|
-
Each user has a persistent **My Files** filesystem — their own uploaded videos (mp4/mov/webm), images (png/jpg/jpeg/gif/webp/svg), audio (mp3/wav/m4a/aac), and documents (pdf/md/txt/csv) — organized into **virtual folders**. This is where a user keeps footage, brand assets, logos, music, briefs, scripts, and the durable strategy docs from Getting Started (
|
|
97
|
+
Each user has a persistent **My Files** filesystem — their own uploaded videos (mp4/mov/webm), images (png/jpg/jpeg/gif/webp/svg), audio (mp3/wav/m4a/aac), and documents (pdf/md/txt/csv) — organized into **virtual folders**. This is where a user keeps footage, brand assets, logos, music, briefs, scripts, and the durable strategy docs from Getting Started / a consultation (OFFER.md, awareness-levels.md, etc.). Both the editor AI copilot and an agent CLI can **read and write** it to find or save assets instead of asking the user to re-upload or paste URLs.
|
|
74
98
|
|
|
75
99
|
- **List** — `GET /api/v1/user/me/attachments` → `{ attachments: [{ id, fileName, contentType, sizeBytes, folderPath, viewUrl, createdAt }], folders: [string] }`. The `viewUrl` is a durable URL you can drop into a composition layer or pass into a primitive route. devcli: `vidfarm files [--folder <path>]`.
|
|
76
100
|
- **Read one** — resolve it from the list and stream/read its `viewUrl`. devcli: `vidfarm get-file <id> [dest]` (add `--print` to print text contents of md/txt/csv/json inline). For images/video/audio/pdf, reference the `viewUrl` as media — you can't read their bytes as text.
|
|
77
|
-
- **Write into My Files** — `POST /api/v1/user/me/attachments/upload` (multipart `file` + optional `folder_path`) persists a file into the library. devcli: `vidfarm put-file <localFile> --folder <path>`, or pipe/inline text without a temp file: `echo "…" | vidfarm put-file --stdin --as
|
|
101
|
+
- **Write into My Files** — `POST /api/v1/user/me/attachments/upload` (multipart `file` + optional `folder_path`) persists a file into the library. devcli: `vidfarm put-file <localFile> --folder <path>`, or pipe/inline text without a temp file: `echo "…" | vidfarm put-file --stdin --as OFFER.md --folder acme` / `vidfarm put-file --content "…" --as OFFER.md`. This is how an agent saves Getting Started / consultation context (OFFER.md — or OFFER_<NAME>.md for a multi-offer director — plus awareness-levels.md, persuasive-angles.md, ad-hooks.md) or drops a logo/product demo into a namescoped folder. In the web editor copilot the same capability is the **`browse_files` write action** (`action=write` with `file_name` + `content` + optional `folder_path`), which accepts text files (md/txt/csv/json/srt/vtt).
|
|
78
102
|
- **`vidfarm upload` is different** — it targets the **ephemeral** temporary-files store (`POST /api/v1/user/me/temporary-files/upload`) for a throwaway durable URL to drop straight into a composition or approved post. Use `put-file` when you want the asset to persist in the user's My Files library; use `upload` for one-shot media you just need a URL for. Namescope scratch uploads under a **`temp/` folder** (`vidfarm upload clip.mp4 --folder temp`) so they stay in one purgeable place. Better still, if you're on a `vidfarm serve` box, skip the upload entirely and reference the file off disk with `place --src ./clip.mp4` (see "Local file paths as media").
|
|
79
103
|
|
|
80
104
|
### Metadata notes + vector search (find assets by meaning)
|
|
@@ -114,6 +138,8 @@ Recurring characters that must look the same across videos have a **dedicated, b
|
|
|
114
138
|
|
|
115
139
|
**Consistency is then mechanical.** On every generation featuring the character, **pass the sprite card's `viewUrl` as the reference input** — `prompt_attachments` for image generation/edit, `input_references` for `videos/generate` and the editor's `generate_layer` — and lift wording from the manifest/`character_about.md` into the prompt.
|
|
116
140
|
|
|
141
|
+
**"Create an avatar" means a TALKING-HEAD VIDEO, not a portrait.** When a director asks for an avatar — or a spokesperson, presenter, host, UGC creator, talking head — generate a **video of that person speaking, with audio**, via `POST /api/v1/primitives/videos/create-avatar` (devcli: `vidfarm avatar "<who they are>" --say "<line>"`). By default it is shot on a **flat greenscreen plate at an exact key color** and the same job keys that plate out, handing back a **transparent presenter WebM** (audio preserved) you composite over whatever background the scene needs — the green never reaches the timeline. Pass `--ref` / `input_references` with the character's `character_sprite_card.png` (or the director's headshot) so the face stays on-model across videos, and `--say` with the exact line: a mute avatar is a broken deliverable. Full parameters, billing, and the `--local` free-keying path are in `references/primitives.md` → "Primitive: talking_avatar".
|
|
142
|
+
|
|
117
143
|
**Image shorthand — "character card" and "storyboard".** Two words the image generator understands as full layouts (spelled out automatically by the platform; you can also write them out yourself):
|
|
118
144
|
- **"character card"** (a.k.a. *character sheet / model sheet / sprite card*) → a **CHARACTER REFERENCE SHEET** on a neutral background: body shots (full / three-quarter / half / head-and-shoulders), a turnaround (front / side / back), an expression sheet of face close-ups (happy / sad / angry / blank-dazed), and a pose sheet (2–3 action poses), with small labels and the **same face / outfit / art style** across every panel. This is exactly the reusable identity reference above — generate it, then persist it as `character_sprite_card.png`.
|
|
119
145
|
- **"storyboard"** → a sheet of **sequential numbered panels** (a grid of frames), each a still of one shot with a short caption under it (action / camera move / voiceover), consistent characters + setting across panels. A storyboard is **also a style fit-check**, so render the panels in the **same visual style intended for the final video** (photoreal / 3D / anime / cinematic / etc.) — don't force a rough sketch unless no style was given.
|
|
@@ -137,4 +163,4 @@ The web copilot exposes the same three as `browse_files action=rename|move|copy`
|
|
|
137
163
|
|
|
138
164
|
**Assume My Files is multi-offer.** A user often runs more than one product, offer, brand, or region, and namescopes assets into folders accordingly — by product (`acme-skincare/`, `zensleep/`), by offer/campaign (`summer-sale/`), by region (`us/`, `eu/`), by asset type (`logos/`, `ugc-clips/`), or any arbitrary scheme. There is **no fixed layout** — read the `folders` tree first and reason about how this user organized things. Before pulling assets for a task, infer which folder(s) match the product/offer/region the current composition is about (match folder names to the composition title, video context, and what the user said), then scope reads to that folder so you never mix one brand's logo/product-shot/music into another brand's video. If the target offer is ambiguous, ask which product/offer/region (or which folder) this work is for rather than guessing across offers.
|
|
139
165
|
|
|
140
|
-
In the editor web copilot the same filesystem is exposed via the **`browse_files` tool** (`action=list` / `action=search` / `action=read` / `action=write` / `action=annotate` / `action=move` / `action=copy` / `action=rename`), so the copilot follows the identical reasoning: search or list to find the right offer's folder, then read an asset — or `write` a text doc (
|
|
166
|
+
In the editor web copilot the same filesystem is exposed via the **`browse_files` tool** (`action=list` / `action=search` / `action=read` / `action=write` / `action=annotate` / `action=move` / `action=copy` / `action=rename`), so the copilot follows the identical reasoning: search or list to find the right offer's folder, then read an asset — or `write` a text doc (OFFER.md, awareness-levels.md, …) or import a media URL (`source_url`) into that folder, annotating anything worth finding again. `action=rename`/`move`/`copy` reorganize the tree (see the three routes above) — use them to keep character folders and asset names tidy. `browse_files list` defaults to `path='/'` when `path` is omitted, so the `/raws` (hunted raws) and `/temp` (scratch) roots surface alongside the My Files folders instead of being hidden; a `/raws` listing also accepts a `content_type` filter (exact shot-kind — `talking_head`, `b_roll`, `product_shot`, `screen_recording`, …), and every listing paginates via `offset` / `limit`. The devcli equivalents are `vidfarm files [--search]` / `get-file` / `put-file [--notes]` / `annotate-file` / `directory rename|move|copy`.
|
|
@@ -88,6 +88,25 @@ vidfarm api POST "/api/v1/compositions/${FORK_ID}/render" --data '{"tracer":"scr
|
|
|
88
88
|
|
|
89
89
|
If you need many variants, keep the base fork fixed and fan out by cloning that fork or by reapplying the same edit function to multiple fork ids. Use the raw REST routes directly when you want maximum control; use `vidfarm-devcli` when you want auth, polling, and file helpers without writing the plumbing yourself.
|
|
90
90
|
|
|
91
|
+
**Where the N in "N variants" comes from: a public raws category.** The hard part of bulk scripting is usually not the loop, it's sourcing N distinct clips without N downloads or N generation calls. A curated public-raws shelf hands you that pool in one request — every clip pre-tagged, free, and already hosted, so the loop body is just a media swap:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
# 1. See what shelves exist (never hardcode the list — this is the live vocabulary).
|
|
95
|
+
vidfarm public-raws --categories --json
|
|
96
|
+
|
|
97
|
+
# 2. Pull one shelf as your clip pool.
|
|
98
|
+
vidfarm public-raws --category scroll-stoppers --limit 20 --json > pool.json
|
|
99
|
+
|
|
100
|
+
# 3. Fan the SAME fork edit across the pool — one variant per clip.
|
|
101
|
+
# (jq over pool.json → previewUrl; swap the hook layer; render each.)
|
|
102
|
+
for URL in $(jq -r '.raws[].previewUrl' pool.json); do
|
|
103
|
+
vidfarm set-media ./work --layer hook --src "$URL"
|
|
104
|
+
vidfarm render "$FORK_ID" --dir ./work --tracer "hooks-$(basename "$URL" .mp4)"
|
|
105
|
+
done
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
That is the cheapest volume play available: no hunt, no per-clip AI call, no wallet spend on sourcing. Pick the shelf that matches the beat you're varying — `scroll-stoppers` for hook A/B tests, `reaction` for cutaways, `b-roll` for narration beds, `greenscreen` for keyed foregrounds. See [Public raws catalog](#public-raws-catalog--browse-by-category-check-here-first).
|
|
109
|
+
|
|
91
110
|
## `vidfarm-devcli` — full command surface
|
|
92
111
|
|
|
93
112
|
`@officexapp/vidfarm-devcli` wraps the **entire director REST flow** as CLI commands. It is a thin shell over the REST API, not a second implementation: most named commands map 1:1 to one REST route, and the file-backed commands compose the documented routes for upload/download + render-polling. Auth via `--api-key <key>` or `VIDFARM_API_KEY`.
|
|
@@ -179,7 +198,7 @@ The licensed harness also carries the **generative build workflow** guidance (ch
|
|
|
179
198
|
| `vidfarm schedules <postId>` | `GET /api/v1/approved/posts/:postId/schedules` | browse scheduled posts |
|
|
180
199
|
| `vidfarm login <email>` / `vidfarm verify <email> <code>` | `POST /api/v1/user/request-otp` · `verify-otp` | get an API key |
|
|
181
200
|
| `vidfarm whoami` | `GET /api/v1/user/me` | who am I (also prints the active cost mode) |
|
|
182
|
-
| `vidfarm cost-mode [minimize\|hybrid\|rich-ai]` | (local `~/.vidfarm/cost-mode.json`) | show/set the money-saving preference every billed command respects; no arg = show + explain
|
|
201
|
+
| `vidfarm cost-mode [minimize\|hybrid\|rich-ai\|pure-videogen]` | (local `~/.vidfarm/cost-mode.json`) | show/set the money-saving preference every billed command respects; no arg = show + explain each; `--clear` forgets it |
|
|
183
202
|
| `vidfarm wallet [--job <id>\|--tracer <t>] [--limit <n>]` | `GET /api/v1/user/me/wallet` | cost log: balance + lifetime spend + recent charges. `--job <renderJobId>` prints **what that one video cost** (sums its charges); `--tracer <t>` sums a tracer. Cloud-only; readable on the free plan too (shows $0.00). Aliases: `spend`, `costs` |
|
|
184
203
|
| `vidfarm provider-keys` / `vidfarm add-provider-key <p> <secret>` | `GET`·`POST /api/v1/user/me/provider-keys` | manage AI keys |
|
|
185
204
|
| `vidfarm upload <file> [--folder <path>]` | presign → S3 PUT → finalize (`.../temporary-files/presign` + `.../temporary-files`) | upload → durable URL (ephemeral, 30-day TTL; prefer `--folder temp` for scratch). Goes direct to S3, so large files (up to **200 MB**) bypass the ~6 MB Lambda body limit |
|
|
@@ -200,6 +219,7 @@ The licensed harness also carries the **generative build workflow** guidance (ch
|
|
|
200
219
|
| `vidfarm raws preset list\|run\|save` / `raws export <ids…> --to <dir>` | (local library) | saved queries; copy raw MP4s out |
|
|
201
220
|
| `vidfarm lint <dir\|composition.html>` | (local static validation) | pre-publish composition check: timing, overlaps, preset names, media src |
|
|
202
221
|
| `vidfarm stills <dir> [--at 0,2.5,…]` | (local in-process render of PNG frames) | visually verify an edit without a full render |
|
|
222
|
+
| `vidfarm qa <dir\|composition.html> [--json] [--strict]` | (local static QA — **devcli-only**, no cloud/REST twin) | **social-native QA: HTML slop + font regime. Run it on EVERY video you produce.** Free, instant, feedback-only |
|
|
203
223
|
| `vidfarm doctor` | (local environment triage) | check ffmpeg/node/keys/agent CLI/poisoned env + list local serve/preview processes before debugging anything else; `--kill-orphans` reaps dead servers squatting ports (fixes the "Waiting for preview server…" hang) |
|
|
204
224
|
| `vidfarm skills list\|add <name>\|update` | `GET /skill-pack/index.json` · `/skill-pack/:name/*` | install/refresh skill packs (see "Skill packs — import on demand") |
|
|
205
225
|
| `vidfarm tts "…" --engine local` / `vidfarm stt <file> --engine whisper` | (keyless LOCAL engines: Kokoro-82M TTS, whisper.cpp STT) | narration + word-timestamp transcripts with zero keys and zero accounts |
|
|
@@ -213,26 +233,59 @@ The licensed harness also carries the **generative build workflow** guidance (ch
|
|
|
213
233
|
|
|
214
234
|
**Approving a locally rendered MP4 (the URL-first rule + the durability rule).** The approve route (`POST /api/v1/approved/posts`) and every media-taking route accept a `url`, never raw file bytes — so you never "upload to approve" in one shot. An approved post is a **permanent** share page, so the media must live in **durable My Files**, not the 30-day temp store (a temp-hosted video would 404 the share page after 30 days). The correct sequence for a big local file is: (1) `POST /api/v1/user/me/attachments/presign` with `{ file_name, content_type, size_bytes }` → (2) PUT the raw bytes to the returned presigned S3 URL → (3) `POST /api/v1/user/me/attachments` (finalize) → use the returned durable `viewUrl` as the approve media `url`. `vidfarm approve --video ./final.mp4` does all of this automatically (durable by default; up to **200 MB**). Never POST a large file as multipart to `.../attachments/upload` against the cloud host: that path proxies through Lambda and caps near 6 MB (it exists only as a fallback for local-storage `vidfarm serve` boxes). Only use the temp-store route (`.../temporary-files/*`, or `vidfarm approve --temp`) for a **throwaway** preview you don't mind losing in 30 days.
|
|
215
235
|
|
|
236
|
+
## `vidfarm qa` — the social-native QA pass (devcli-only, heavily recommended)
|
|
237
|
+
|
|
238
|
+
```bash
|
|
239
|
+
vidfarm qa ./work # human-readable findings + verdict
|
|
240
|
+
vidfarm qa ./work --json # machine-readable: rule / severity / where / fix
|
|
241
|
+
vidfarm qa ./work --strict # ALSO exit 1 on slop (only if you want a CI gate)
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
**Run this on every video you produce.** It is free, instant (pure DOM, no ffmpeg/Chrome/network), and it is the only automated check for the thing that most often ruins an agent-made video: **HTML slop**. Compositions are authored in HTML, so an agent's web-page instincts leak straight onto the frame as landing-page furniture that appears on every website and in **zero** real TikToks.
|
|
245
|
+
|
|
246
|
+
**It is feedback, not a gate.** Default exit code is **0** even when it finds slop; nothing in the render or publish path calls it; it never runs automatically. `--strict` exists only if you deliberately want a CI failure. A finding you disagree with is fine to ignore and say so — it is a lint, not a verdict on the work.
|
|
247
|
+
|
|
248
|
+
**It is a BLOCKLIST, not an allowlist.** It names specific known-bad web patterns. Everything it doesn't name is legal, so a weird, ugly, hand-made, or wildly stylized composition passes untouched. It will never push your videos toward one house style — if it fires on a genuine creative choice, that's a bug in the rule, not in your video.
|
|
249
|
+
|
|
250
|
+
What it flags:
|
|
251
|
+
|
|
252
|
+
| Rule | Severity | What trips it |
|
|
253
|
+
|---|---|---|
|
|
254
|
+
| `cta-button` | error | Action copy ("Sign Up for a Free Trial →") **inside** a filled/gradient rounded capsule. Bare CTA copy in a caption is fine — "BUY NOW" is real social copy |
|
|
255
|
+
| `badge-chip-row` | error | 2+ sibling small rounded filled tags — the "✓ No Credit Card Needed · ✓ 30-Min Trial" strip |
|
|
256
|
+
| `card-panel` / `glass-card` | error | A rounded box with a border/shadow/`backdrop-filter` wrapping 2+ elements. A tight legibility **band** (radius ≤8px, one text run, no border/shadow) stays legal |
|
|
257
|
+
| `gradient-text` | error | `background-clip:text` gradient headline fills |
|
|
258
|
+
| `clickable-element` | error/warn | `<button>`, `<form>`, `<input>`; `<a href>` warns |
|
|
259
|
+
| `web-framework-classes` | error/warn | Bootstrap/Tailwind class tokens (`btn`, `badge`, `card`, `hero`, `col-*`, `rounded-full`, `shadow-lg`, `backdrop-blur`, `bg-gradient-to-*`) or a linked CSS framework. A `<script>` CDN for GSAP/anime.js is fine |
|
|
260
|
+
| `page-structure` / `bullet-list` | error/warn | `<nav>`/`<header>`/`<footer>`/`<table>`; a `<ul>` with visible bullet markers |
|
|
261
|
+
| `font-regime` | error/warn | A text layer in a website body font (Inter/Roboto/Arial/system-ui → **error**) or any family outside the imported regime (Montserrat, TikTok Sans, Abel, Source Code Pro, Yesteryear → warn, it silently falls back at render) |
|
|
262
|
+
| `font-size` / `font-weight` | error/warn | `font-size:0` (invisible) is an error; sub-2.6%-of-canvas-width text and weight <600 warn |
|
|
263
|
+
| `caption-safe-zone` | warn | Text outside the 8%–85% band on a **portrait** canvas (landscape/square are exempt) |
|
|
264
|
+
|
|
265
|
+
Every finding carries a concrete `fix` line — the answer is always "say it as timed text on the footage", never just "delete it". Fold `--json` into scripted batch runs to QA N variants at once.
|
|
266
|
+
|
|
216
267
|
## Cost mode — the devcli's money-saving guardrail
|
|
217
268
|
|
|
218
|
-
The
|
|
269
|
+
The four modes, quoted as **cost per finished video**. The first two are spend postures; the last two are also *production methods* — `rich-ai` buys reusable greenscreen raws and animates them in HTML, `pure-videogen` generates every frame:
|
|
219
270
|
|
|
220
271
|
| Mode | Per video | Posture |
|
|
221
272
|
|---|---|---|
|
|
222
273
|
| `minimize` | **$0 — explicitly free** | free local engines + free stock media only; billed ops refuse without `--yes` |
|
|
223
274
|
| `hybrid` *(default recommendation)* | **~$0.01–$1** | free where free, AI where it clearly wins; each billed op prints its cost |
|
|
224
|
-
| `rich-ai` | **$1+** | AI
|
|
275
|
+
| `rich-ai` | **$1+** | AI video gen mints **reusable greenscreen raws** (saved to the library, reused later); motion is hyperframes HTML/CSS over the keyed raws |
|
|
276
|
+
| `pure-videogen` | **$5+** | script in text → storyboard in images → frame-by-frame scene generation. No reuse, no HTML motion |
|
|
225
277
|
|
|
226
278
|
**All of it bills to the user's own AI provider keys (BYOK)** — the keys saved with `vidfarm add-provider-key <provider> <key>` or at **Settings → Bring your own keys** (<https://vidfarm.cc/settings/developer>). The model providers charge those keys directly; Vidfarm wallet credits only come into play when the user deliberately runs on the platform key instead of their own. So `minimize` isn't "cheap", it's **zero**: nothing reaches a paid key at all.
|
|
227
279
|
|
|
228
|
-
`vidfarm cost-mode <minimize|hybrid|rich-ai>` records a single spend preference (in `~/.vidfarm/cost-mode.json`) that every **billed** command honors: `generate`, `music`, `decompose`, `inspiration-decompose`, `create`, `replicate`, `inpaint`, `create-overlay`, `cutout --generate` (only the generation step; a bare `cutout` on an existing graphic is free), and the cloud paths of `render --target cloud`, `tts --cloud`, `stt --cloud`, `remove-greenscreen` (non-`--local`). FREE local engines never gate (`render` local default, `tts --engine local`, `stt --engine whisper`, `remove-greenscreen --local`, `cutout` on a file/url, all the file-editing verbs). `vidfarm media search` (the free stock catalog — pixabay/openverse/iconify music, SFX, images, icons, stock video) is also free and never gates.
|
|
280
|
+
`vidfarm cost-mode <minimize|hybrid|rich-ai|pure-videogen>` records a single spend preference (in `~/.vidfarm/cost-mode.json`) that every **billed** command honors: `generate`, `music`, `decompose`, `inspiration-decompose`, `create`, `replicate`, `inpaint`, `create-overlay`, `cutout --generate` (only the generation step; a bare `cutout` on an existing graphic is free), and the cloud paths of `render --target cloud`, `tts --cloud`, `stt --cloud`, `remove-greenscreen` (non-`--local`). FREE local engines never gate (`render` local default, `tts --engine local`, `stt --engine whisper`, `remove-greenscreen --local`, `cutout` on a file/url, all the file-editing verbs). `vidfarm media search` (the free stock catalog — pixabay/openverse/iconify music, SFX, images, icons, stock video) is also free and never gates.
|
|
229
281
|
|
|
230
282
|
- **minimize ($0 videos)** — a billed op is **refused** unless you add `--yes`; the error names the free local alternative (which now includes the matching `vidfarm media search` for music/SFX/image/video). Use this to guarantee no surprise AI spend. Before paying to generate music, sound effects, or images, try `vidfarm media search "<meaning>" --type <bgm|sfx|image|vector|icon|video>` first — free royalty-free assets instead of a billed `music`/`generate` call. **Check the keyless sources first — Openverse (CC/CC0 music, SFX, images) and iconify (icons) need no account at all**, so they always work in `minimize`. Photos/vectors/stock-video need a **free Pixabay key** that **may already be saved** — check `vidfarm provider-keys` (or web **Settings → Bring your own keys** / <https://vidfarm.cc/settings/developer>) before assuming a short result means "no key." If absent, save one once: `vidfarm add-provider-key pixabay <key>` (free key from <https://pixabay.com/api/docs/>), the Settings surface, or hand it to the desktop AI agent to run that command.
|
|
231
283
|
- **minimize still gets CUSTOM images — via a free manual generator.** A refused `generate` is not the end of the road. Offer the user the manual loop (ask once, then make it the session default): **you write the prompt → they run it free in <https://meta.ai>, free-tier ChatGPT, or a free Hugging Face image Space (<https://huggingface.co/spaces>) → they hand the PNG back** via `vidfarm put-file ./sheet.png` or web **My Files**. Ask for **one sheet holding every graphic you need**, gridded on a **flat pure-green plate** (`#00FF00`), no text — one round trip instead of N, which saves the user's time and your tokens. Then split it locally for $0: `vidfarm mask ./sheet.png --crop x,y,w,h --flat "#00FF00" --out prop-a.png`, once per element (drop `--flat` and let local ONNX matting handle it if the tool ignored the green background). Full prompt template + loop: recipe `recipes/cutout-graphics-for-explainers.md` (“Free manual image-gen”).
|
|
232
284
|
- **hybrid (~$0.01–$1 per video)** *(default recommendation)* — billed ops run but print a one-line cost notice each, charged to the user's BYOK key.
|
|
233
|
-
- **rich-ai ($1+ per video)** — billed ops run without gating; cost is still printed. AI *video* generation is the line item that pushes a video well past $1 — quote it before running.
|
|
285
|
+
- **rich-ai ($1+ per video)** — billed ops run without gating; cost is still printed. AI *video* generation is the line item that pushes a video well past $1 — quote it before running. Spend it on **reusable greenscreen raws**, not on finished shots: `vidfarm avatar "<who>" --say "<line>"` for presenters, `vidfarm create-overlay "<subject>"` / `cutout --generate` for props and illustrations, or `generate video` prompted onto a flat key-color plate. The primitives key the plate in the same job and also hand back `greenscreen_source_url`, so re-keying at a different tolerance is free. Then **animate in hyperframes HTML/CSS over the keyed raws** — generated seconds cost money, motion doesn't — and **persist every asset**: `vidfarm put-file ./keyed.webm --folder greenscreen/<name> --notes "<what it is, when to reuse it>"` (notes are vector-embedded → `vidfarm files --search`), or `vidfarm clipper ./generated.mp4 --folder greenscreen-cast --name "<name>"` for footage-shaped raws in `/raws`. Both stores are **local by default** under `~/.vidfarm`; on a **paid/Pro plan** mirror them with `vidfarm sync push /files` and `vidfarm sync push /raws` (`sync pull` elsewhere, `--dry-run` first). Before generating, always search what already exists — `vidfarm raws search "<meaning>"` then `vidfarm public-raws --category greenscreen --query "<meaning>"`. That is what makes the mode amortize: the next video can reuse the same cast in `hybrid`/`minimize` for ~$0.
|
|
286
|
+
- **pure-videogen ($5+ per video)** — the most expensive mode; billed ops run without gating and a single run can be many dollars, so quote the number (N scenes × per-second video price + storyboard images) before starting. Everything is generated footage — no raw reuse, no HTML motion. Plan first so you don't pay twice: (1) write the whole piece as a **text script** (shot list, action, dialogue, per-scene duration) and get it approved; (2) **storyboard it as images** — one still per scene via `vidfarm generate image` (cents, not dollars), locking cast/wardrobe with `--ref` on a sprite card, and iterate on the stills until the look is right; (3) **generate scenes from those keyframes** — drive each `generate video` off its approved still (`--ref` / `frame_images`, chaining the previous scene's last frame into the next scene's start frame for continuity), one scene at a time with a review between; (4) **assemble on the timeline** — captions, audio, and pacing in hyperframes, never a regeneration to fix a trim. If the piece would survive as `rich-ai` (greenscreen cast over HTML backgrounds), offer that first — typically ~5× cheaper.
|
|
234
287
|
|
|
235
|
-
**Narration defaults to the FREE local voice in minimize AND hybrid.** A bare `vidfarm tts "…"` runs the keyless local Kokoro-82M engine in both of those modes — you no longer have to remember `--engine local`. A run **opts out** of that default by asking for a premium voice (`--style`, `--provider`, `--model`, `--own-key`, or a non-Kokoro `--voice` like `alloy`/`Kore`/an ElevenLabs id), by passing `--cloud`/`--engine byok`, or by being in `rich-ai`. If the local engine isn't installed on the machine (it needs `pip install kokoro-onnx soundfile` + a ~340MB model on first use), the run **falls back** to the user's provider key / cloud instead of failing — it prints the reason on stderr so you can tell the user why the voice changed.
|
|
288
|
+
**Narration defaults to the FREE local voice in minimize AND hybrid.** A bare `vidfarm tts "…"` runs the keyless local Kokoro-82M engine in both of those modes — you no longer have to remember `--engine local`. A run **opts out** of that default by asking for a premium voice (`--style`, `--provider`, `--model`, `--own-key`, or a non-Kokoro `--voice` like `alloy`/`Kore`/an ElevenLabs id), by passing `--cloud`/`--engine byok`, or by being in `rich-ai`/`pure-videogen`. If the local engine isn't installed on the machine (it needs `pip install kokoro-onnx soundfile` + a ~340MB model on first use), the run **falls back** to the user's provider key / cloud instead of failing — it prints the reason on stderr so you can tell the user why the voice changed.
|
|
236
289
|
|
|
237
290
|
Precedence: `--cost-mode <m>` flag → `VIDFARM_COST_MODE` env → the saved `cost-mode` → default (hybrid, flagged as "not set"). When nothing is saved and a billed op runs, the CLI prints a "no preference set — ask the user" nudge instead of silently spending, so the default posture really is *ask before you spend*.
|
|
238
291
|
|
|
@@ -73,7 +73,7 @@ To bring a new viral video into the catalog as a **private** template you own, i
|
|
|
73
73
|
- `DELETE /discover/templates/:entryId` — remove a private inspiration/template you own (accepts either the `inspiration_...` or minted `template_...` id).
|
|
74
74
|
- `POST /api/v1/inspirations/:inspirationId/decompose { user_prompt? }` — AI-decompose an inspiration's downloaded video into scenes (requires a saved provider key; same 120s source cap as auto-decompose).
|
|
75
75
|
|
|
76
|
-
devcli: `vidfarm inspiration-add <url|file.mp4>` (a local file path runs the presign→PUT→finalize flow for you), `vidfarm inspiration-rm <id>`, `vidfarm inspiration-decompose <id>`, `vidfarm discover [query] [--sort relevance|wow|automation|recent] [--cursor <next_cursor>]` to browse/search templates, `vidfarm public-raws [query] [--category <key>] [--type <raw_type>] [--bookmark <raw_id>]` to browse/save the public raws catalog, and `vidfarm videos [query] [--mine]` to browse/search source inspirations. On the web, the Discover page's **Add Template** modal accepts either a URL or a file upload.
|
|
76
|
+
devcli: `vidfarm inspiration-add <url|file.mp4>` (a local file path runs the presign→PUT→finalize flow for you), `vidfarm inspiration-rm <id>`, `vidfarm inspiration-decompose <id>`, `vidfarm discover [query] [--sort relevance|wow|automation|recent] [--cursor <next_cursor>]` to browse/search templates, `vidfarm public-raws [--categories | query] [--category <shelf|key>] [--type <raw_type>] [--bookmark <raw_id>]` to browse/save the public raws catalog — **start with `--categories`** to see the shelves and their counts, then pull one with `--category <shelf>`; a shelf doubles as a ready-made clip pool for bulk scripting (see [Public raws catalog](#public-raws-catalog--browse-by-category-check-here-first)), and `vidfarm videos [query] [--mine]` to browse/search source inspirations. On the web, the Discover page's **Add Template** modal accepts either a URL or a file upload.
|
|
77
77
|
|
|
78
78
|
## Fork a template
|
|
79
79
|
|
|
@@ -7,7 +7,7 @@ Almost every editor session is a director taking a template / fork / project and
|
|
|
7
7
|
|
|
8
8
|
Name the plan back in these terms ("I'll SWAP the captions and REPLACE the scenes"), then execute axis by axis. Be **proactive at the heavy end** — carry the whole transformation, don't wait to be micro-managed one layer at a time.
|
|
9
9
|
|
|
10
|
-
**Fuel a scene REPLACE with raw clips, not expensive AI video.** A heavy scenes-axis replace needs footage; sources in cost order: (1) the director's own library — search `/raws` and `/files` (`vidfarm raws search …`, `vidfarm files --search …` / `browse_files`); (2) **HUNT new raws** out of a long-form source (podcast/VOD/webinar or any YouTube/TikTok/IG/X URL) — `vidfarm raws scan <src> --prompt "<what the new scenes need>" --aspect <canvas> [--duration N --no-text --range …]` (local-first, free compute) or the async `POST /clips/scan` / `/raws/scan`; then reuse the picks (`set_layer_media` / `vidfarm set-media` to swap in place, `add_layer`/`vidfarm place` for net-new scenes); (3) `generate_layer` / `vidfarm generate` AI generation — the **expensive last resort**, only for scenes no real clip can cover. When a big scene re-work is asked for but no footage is given, **ask for a source to hunt (or point at the raws library) before AI-generating** — see [Raws](#raws-long-form--short-form-raws) and [Generate AI media …](#generate-ai-media-and-drop-it-on-the-timeline).
|
|
10
|
+
**Fuel a scene REPLACE with raw clips, not expensive AI video.** A heavy scenes-axis replace needs footage; sources in cost order: (1) the director's own library — search `/raws` and `/files` (`vidfarm raws search …`, `vidfarm files --search …` / `browse_files`); (1b) the **PUBLIC RAWS catalog, browsed by category** — free, pre-tagged, already on the platform, and no download or hunt required (`vidfarm public-raws --categories` to see the shelves, then `--category <shelf>`); (2) **HUNT new raws** out of a long-form source (podcast/VOD/webinar or any YouTube/TikTok/IG/X URL) — `vidfarm raws scan <src> --prompt "<what the new scenes need>" --aspect <canvas> [--duration N --no-text --range …]` (local-first, free compute) or the async `POST /clips/scan` / `/raws/scan`; then reuse the picks (`set_layer_media` / `vidfarm set-media` to swap in place, `add_layer`/`vidfarm place` for net-new scenes); (3) `generate_layer` / `vidfarm generate` AI generation — the **expensive last resort**, only for scenes no real clip can cover. When a big scene re-work is asked for but no footage is given, **ask for a source to hunt (or point at the raws library) before AI-generating** — see [Raws](#raws-long-form--short-form-raws) and [Generate AI media …](#generate-ai-media-and-drop-it-on-the-timeline).
|
|
11
11
|
|
|
12
12
|
**Audio is natively multi-track — overlay narration + music + SFX, each at its own volume.** A composition mixes UNLIMITED simultaneous `<audio>` layers; each sits on its own `data-track-index` and carries its own `data-volume` (0–2, default 1), and the runtime mixes them with per-track volume honored identically in the preview and the exported MP4 (a real ffmpeg `amix` of every audio layer at render). So you never need a pre-mixed file — lay **narration/voiceover at ~1.0 on one track, a music bed at ~0.1–0.2 on a separate track, and SFX on their own tracks**, each via `add_layer kind=audio` (web) / `vidfarm place --kind audio --volume …` (devcli), tuning levels later with the Inspector's Volume slider or `set_layer_media` (`volume`, `muted`). **The key move when recreating a template whose original baked music + narration into ONE audio track: rebuild it as TWO independent tracks** — a fresh narration track (`/audio/speech`, or same-voice reword via `/audio/regenerate-speech` / `vidfarm speech regenerate`) at ~1.0 and a separate real music track at ~0.1–0.2 — then mute/remove the original combined source-audio layer. This gives the director independent voice and music volume, and works around AI TTS being unable to emit narration+music in one file: you compose the mix on the timeline. Honesty: you can't un-mix / stem-separate the original's baked audio — the two tracks are a fresh narration track **plus** a real music file (owned / user-provided / `browse_files` / generated with `/api/v1/primitives/music/generate` (`vidfarm music`)), never a faked or duplicated voice layer.
|
|
13
13
|
|
|
@@ -17,7 +17,7 @@ Vidfarm is deliberately founder-friendly and thrift-first: **we do not spend exp
|
|
|
17
17
|
|
|
18
18
|
> **This decomposition is now a materialized artifact.** Every decomposed fork carries a **replication harness** (`replication-harness.json` / `editor_context.replication_harness` / `GET .../replication-harness.json`) that already contains BOTH plans as concrete per-beat brush assignments plus reusable-asset guidance and viral-DNA guardrails. When it's present, **execute its beats** instead of re-deriving them; the methodology below is how it was built and the fallback when it's absent (older forks / free tier). See the route section further down.
|
|
19
19
|
|
|
20
|
-
1. **Raw clips** — cut + remix existing long-form / short-form footage. The workhorse. Source in cost order: the director's own `/raws` + `/files` library → HUNT new raws out of a URL/VOD (`vidfarm raws scan`) → only then generate. Combining a **background video + a foreground video** (greenscreen / picture-in-picture) covers a huge share of "video meme" formats with zero generation.
|
|
20
|
+
1. **Raw clips** — cut + remix existing long-form / short-form footage. The workhorse. Source in cost order: the director's own `/raws` + `/files` library → the **public raws catalog by category** (`vidfarm public-raws --categories`, then `--category <shelf>`; free, pre-tagged, and the fastest way to fill a slot or fan one composition into N variants) → HUNT new raws out of a URL/VOD (`vidfarm raws scan`) → only then generate. Combining a **background video + a foreground video** (greenscreen / picture-in-picture) covers a huge share of "video meme" formats with zero generation.
|
|
21
21
|
2. **HTML/JS hyperframes** — video-from-HTML. Animate text, image elements, logos, stickers, charts and lower-thirds with CSS/declarative motion (web editor) or JS adapters (anime.js/GSAP/Lottie/Three, devcli only). Deterministic, cheap, re-themeable — the right brush for titles, kinetic captions, data beats, brand cards, and any "graphic" scene.
|
|
22
22
|
3. **Pure AI generation** — AI image / video / voice / music. The most expensive brush (AI **video** especially, $1–$10+); the last resort for a beat no clip or hyperframe can cover.
|
|
23
23
|
|
|
@@ -291,6 +291,46 @@ When `applies` is true, run the workflow **exactly in order, and honor the two c
|
|
|
291
291
|
|
|
292
292
|
When `applies` is **false** (talking-head, screen-recording, clip-remix, kinetic-text, meme-recaption, real b-roll), ignore this block and rebuild thrift-first from `harness.cheap.scenes[]` as above — do NOT reach for the generative pipeline. `status:"none"` → run `POST /auto-decompose` first.
|
|
293
293
|
|
|
294
|
+
#### `vidfarm sequence` — the generative workflow as one resumable command
|
|
295
|
+
|
|
296
|
+
The workflow above, run end-to-end from a desktop agent (or by hand) without gluing the steps together yourself. All state lives in a single `sequence.json`, so every step is resumable and rerunning a finished step is a no-op.
|
|
297
|
+
|
|
298
|
+
```bash
|
|
299
|
+
vidfarm sequence init "a lone windsurfer rides the wind across sunlit grass hills" \
|
|
300
|
+
--dir ./wind --parts 2 --panels 8 --duration 15 --aspect-ratio 16:9 \
|
|
301
|
+
--character Bree --character-description "athletic, platinum bob, cropped jacket, barefoot" \
|
|
302
|
+
--style "painterly anime illustration, crisp daylight, saturated blue sky"
|
|
303
|
+
|
|
304
|
+
vidfarm sequence run --dir ./wind # walks the whole pipeline, gate before each spend
|
|
305
|
+
vidfarm sequence run --dir ./wind --yes # fully autonomous, no prompts
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
`init` writes a **structurally complete but generically written** plan plus `PLAN_TASK.md`. The plan is what makes the output good, so the intended second step is to direct it — open `sequence.json`, or hand `PLAN_TASK.md` to a coding agent in that folder (`claude "read PLAN_TASK.md and do it"`), then `vidfarm sequence plan --dir ./wind --check`. Running without that step still works; it just produces generic direction.
|
|
309
|
+
|
|
310
|
+
| Step | Cost | What it does |
|
|
311
|
+
|---|---|---|
|
|
312
|
+
| `characters` | ~$0.05 each | One identity sheet per character — the identity authority for every later shot |
|
|
313
|
+
| `boards` | ~$0.05 each | One storyboard sheet per part, written in the block grammar below |
|
|
314
|
+
| `prompts` | free | Composes the video prompt per part, deterministically, from the plan |
|
|
315
|
+
| `animate` | **~$1+ each** | Generates the video per part, citing card + board as references |
|
|
316
|
+
| `assemble` | free | ffmpeg-concats the parts (`--music <file|url>`) into `renders/final.mp4` |
|
|
317
|
+
|
|
318
|
+
**The ordering is the point.** Boards cost cents and video costs dollars, so you read a board *before* you buy a shot. `sequence run` puts a confirm gate after `characters` and after `boards` for exactly that reason; `--yes` (or a non-TTY / `--json` run) skips them. Preview any prompt for free before spending:
|
|
319
|
+
|
|
320
|
+
```bash
|
|
321
|
+
vidfarm sequence prompts --dir ./wind --print board:part-01 # storyboard image prompt
|
|
322
|
+
vidfarm sequence prompts --dir ./wind --print shot:part-01 # video-model prompt
|
|
323
|
+
vidfarm sequence prompts --dir ./wind --print card:bree # character card prompt
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
Three rules are baked into the generated prompts and carry most of the quality:
|
|
327
|
+
|
|
328
|
+
1. **Reference priority is stated explicitly** — the character card controls identity; the storyboard controls staging, motion, geography and continuity. Without the split, video models copy faces off the board's rough panels.
|
|
329
|
+
2. **"Do not render the storyboard sheet itself"** — omit it and the model cheerfully animates the grid, borders and all.
|
|
330
|
+
3. **Panels are restated as `PANEL BEATS` in words**, so shot order survives even when the model reads the board loosely.
|
|
331
|
+
|
|
332
|
+
`parts[].panels[]` carries `lens`, `camera`, `action`, `state`, `style`, a `rhythm` (hold/build/burst/impact/…) and an `escalation` (L1 calm → L5 peak). Those feed the board's DIRECTOR STRIP — the rhythm/escalation tracks under the grid that make a board read as *timing* rather than as N unrelated stills. Set `parts[].extend_from: "part-01"` on part 2 and its video prompt opens with `extend` instead of re-establishing the world. `--board-style final` renders board panels in the final look (a style fit-check) instead of monochrome previz.
|
|
333
|
+
|
|
294
334
|
## Generate AI media and drop it on the timeline
|
|
295
335
|
|
|
296
336
|
Both surfaces can generate a NEW AI video/image (optionally conditioned on reference images) and place it into the timeline — filling a blank gap or replacing a scene — using the async generation primitives (`POST /api/v1/primitives/videos/generate`, `/images/generate`). Generation is a **queued job**: the POST returns `202 { job_id }`, and the finished media URL only appears after the job settles (`result.primary_file_url` / `video.file_url` / an artifact `public_url`). Poll `GET /api/v1/user/me/jobs/:jobId` until `status: "succeeded"`.
|
|
@@ -299,6 +339,8 @@ Both surfaces can generate a NEW AI video/image (optionally conditioned on refer
|
|
|
299
339
|
|
|
300
340
|
- **Editor web copilot**: use the `generate_layer` editor_action — it does the whole flow (submit job → placeholder clip → auto-swap on completion) and reads `pending_generations` from `editor_context`. Set `media_type`, `prompt`, `aspect_ratio` (match the canvas), and `intent` (`fill_gap` with `start`/`duration`, `replace_layer` with `replace_layer_key`, or `add`). For character consistency pass a cast `reference_url` — or the character's `character_sprite_card.png` viewUrl from `/files/characters/<slug>/` (see "Recurring characters are first-class") — in `input_references` (video) / `prompt_attachments` (image).
|
|
301
341
|
|
|
342
|
+
- **Avatars are a different route.** "Create an avatar" (spokesperson / presenter / host / UGC creator / talking head) is NOT plain `/videos/generate` or `generate_layer` — it's `POST /api/v1/primitives/videos/create-avatar` (devcli `vidfarm avatar "<who>" --say "<line>"`), which forces the spoken audio, the exact-key-color greenscreen plate, and the chroma key, and returns a transparent presenter WebM to layer over the scene. The web copilot calls it with `http_request`, then `add_layer`/`set_layer_media` the `primary_file_url` as a video layer. See `references/primitives.md` → "Primitive: talking_avatar".
|
|
343
|
+
|
|
302
344
|
- **Desktop agents (Claude Code / Codex)** — two devcli commands:
|
|
303
345
|
```bash
|
|
304
346
|
# 1) Sync the fork to disk and see where the blank space and scenes are
|
|
@@ -363,16 +405,52 @@ Three surfaces:
|
|
|
363
405
|
- **Editor UI (human)** — the Inspector's **Animate (Ken Burns)** dropdown on any selected image.
|
|
364
406
|
- **Desktop agents (devcli)** — `vidfarm place ./work --src <img> --kind image --ken-burns zoom-in [--ken-burns-intensity 0.3]`, or fused with generation: `vidfarm generate image --prompt "…" --place ./work --ken-burns pan-left`.
|
|
365
407
|
|
|
366
|
-
###
|
|
408
|
+
### Social-native visual standard — no HTML slop, ever
|
|
409
|
+
|
|
410
|
+
Compositions are authored in HTML, so the single most common way an AI-edited video betrays itself is by **looking like a web page**. A landing-page button, a row of feature chips, a frosted card with a gradient headline — these are everywhere on the web and **nowhere in a real social post**. A director scrolling TikTok/Reels/Shorts never sees them, so the moment one appears the video reads as machine-made and the retention dies. HTML and hyperframes are the right tool; **Bootstrap/Tailwind furniture is not.**
|
|
411
|
+
|
|
412
|
+
**The one test:** *would this element exist if the video were shot on a phone and captioned in CapCut?* If it only makes sense inside a browser — if its whole job is to look **clickable** — cut it. **Nothing in a video is clickable.**
|
|
413
|
+
|
|
414
|
+
**BANNED — never author, and strip on sight when a fork or a paste brings one in:**
|
|
415
|
+
|
|
416
|
+
- **CTA / "button" shapes.** Any filled capsule or rounded rect containing action copy — "Sign Up for a Free Trial →", "Learn More", "Get Started", "Book a Call" — especially gradient-filled with a glow or drop shadow. A social CTA is *spoken*, or a plain caption line, or an arrow pointing at the real UI. Not a button.
|
|
417
|
+
- **Badge / chip / pill rows.** The "✓ ID-Verified Tutors · ✓ No Credit Card Needed · ✓ 30-Min Trial" strip of small rounded tags. Nothing on TikTok is a `<span class="badge">`. Say the three benefits as three timed caption lines instead — one at a time, on the footage.
|
|
418
|
+
- **Cards, panels, containers.** A rounded-rect box (dark, bordered, shadowed, or `backdrop-filter` frosted) holding a heading plus a subheading/URL. Text belongs **on the footage**, not inside a floating panel with padding around it.
|
|
419
|
+
- **Gradient text fills, neon border glows, elevation shadows, glassmorphism, hover-implying strokes.**
|
|
420
|
+
- **Web page furniture of any kind:** navbars, hero sections, feature grids, two-column layouts, `<ul>` bullet lists with disc markers, tables, alerts, progress bars as decoration, logo-in-a-circle avatars, "as seen in" strips.
|
|
421
|
+
- **Corporate web type:** Inter, Roboto, system-ui, Arial, Helvetica, Georgia, Times — at web weights (400–600) and web sizes (16–24px). Instantly reads as "a screenshot of a website."
|
|
422
|
+
|
|
423
|
+
**Greppable smell test.** If you are typing `class="btn…"`, `badge`, `chip`, `tag`, `card`, `panel`, `container`, `row`/`col-`, `hero`, `cta`, `rounded-full`, `shadow-lg`, `backdrop-blur`, `bg-gradient-to-r`, `border border-…` — **stop.** You are building a web page, not a video. Rewrite as timed text on footage.
|
|
367
424
|
|
|
368
|
-
|
|
425
|
+
**ALLOWED and encouraged — these ARE social-native:**
|
|
426
|
+
|
|
427
|
+
- **Arrows** (drawn, animated, hand-style), circles/scribbles/underline strokes highlighting part of the frame, hand-drawn marks, checkmarks *as glyphs inside a caption line* (not as chips).
|
|
428
|
+
- **Italic / oblique** for emphasis, ALL-CAPS, a single word in a pop color, mixed weights across a stacked line.
|
|
429
|
+
- **Emoji inline in text** (sparingly), **sticker/cut-out overlays** on transparent PNG (`create-overlay`), mock social UI when the format calls for it (iMessage bubbles, a TikTok comment card, a fake DM, a countdown/progress bar) — these are native artifacts of the platform, not web furniture.
|
|
430
|
+
- **Full-bleed footage** with text sitting directly on it.
|
|
431
|
+
|
|
432
|
+
**On devcli there's a checker: `vidfarm qa <dir|composition.html>`.** Free, instant, local-only — a blocklist pass for everything above plus the font regime and safe zone, with a concrete fix per finding. **Run it on every video you produce.** It is feedback, not a gate (exit 0 even on findings, never runs automatically, `--strict` only if you want a CI failure) and a blocklist, not an allowlist (stylized/hand-made compositions pass untouched — it will not homogenize your videos). No cloud/REST twin: the web copilot enforces this standard by hand. Details in `references/automation-and-local-dev.md` ("`vidfarm qa`").
|
|
433
|
+
|
|
434
|
+
### TikTok-native caption standard (position + font + background) — always adhere
|
|
435
|
+
|
|
436
|
+
Short-form is watched on a phone, and the phone's UI eats the frame's edges. **Never pin on-screen text to the extreme top or bottom** — the top ~8% sits under the status bar / "Following · For You" tabs and the bottom ~15% under the username, caption text, music marquee, and action rail. Text there is literally clipped and reads as amateur. Three rules, applied to **every** caption/title/overlay you place or inherit:
|
|
369
437
|
|
|
370
438
|
- **Position → readable safe zone.** Keep the text box's vertical extent inside **~8%–85%** of canvas height (9:16). A lower-third subtitle lives around **y≈70%** (the `captions generate` default); a "me when…" / "POV:" meme line sits near the **top of the safe zone (y≈8%), not y:0**. Preserve the original top-vs-bottom anchoring — just pull it off the dead-zone edge, don't recenter everything. Also keep wide captions clear of the **right ~12%** action rail (a centered box at `x:10 width:80` is safe).
|
|
371
|
-
- **Font → the composition regime.** Use the bundled display fonts only — **Montserrat** (bold default, weight
|
|
439
|
+
- **Font → the composition regime.** Use the bundled display fonts only — **Montserrat** (bold default, weight **700–900**), **TikTok Sans**, Abel, Source Code Pro, Yesteryear. Don't request a font the composition doesn't import (it silently falls back to a web-default sans, which is exactly the slop look). Sizes are in PIXELS of a 1080-wide frame: **~36–64px** reads well, never below ~28px and **never 0** (invisible). Keep captions to ~2 lines / ~5 words per line; `line_height` 0.95–1.15 for stacked display lines.
|
|
440
|
+
- **Background → one of exactly four valid treatments.** Any text you place uses one of these and nothing else:
|
|
441
|
+
|
|
442
|
+
| # | Treatment | How to set it | When |
|
|
443
|
+
|---|---|---|---|
|
|
444
|
+
| 1 | **Bare text + heavy outline/stroke** (black stroke on white or a bright fill) | `background_style:"outline"` | The default TikTok look |
|
|
445
|
+
| 2 | **Bare text + soft drop shadow only**, no box | `background_style:"plain"` | Clean/cinematic over busy footage |
|
|
446
|
+
| 3 | **Highlight pill behind the ACTIVE word only** | `set_captions caption_style:"spotlight"` / `"karaoke"` (+ `caption_highlight_color`) | Hormozi/CapCut word-by-word. **The only legitimate "pill" in a video** — it tracks the spoken word, so it isn't a badge |
|
|
447
|
+
| 4 | **Solid band that tightly hugs the text lines** (CapCut "text box") | `background_style:"highlight-solid"` (or `"highlight-translucent"`) + a `background` color | Guaranteed legibility over noisy footage |
|
|
448
|
+
|
|
449
|
+
Treatment 4 is a **band, not a card**: it hugs the glyphs with minimal padding, corner radius ≤ ~8px, **no border, no drop shadow, no gradient, no blur**, and it wraps *one* text run — never a heading + subheading + URL stacked inside one rounded box. The moment it grows padding, a stroke, or a second element inside it, it has become a web card. Fix it.
|
|
372
450
|
|
|
373
|
-
**A common trap: decomposed templates mirror the source's caption placement**, so a forked meme can arrive with its caption pinned at `top:0` in a non-regime font
|
|
451
|
+
**A common trap: decomposed templates mirror the source's caption placement**, so a forked meme can arrive with its caption pinned at `top:0` in a non-regime font — and a re-theme prompt ("make this for my tutoring service") is exactly where an agent starts inventing landing-page CTAs and benefit chips because the *subject* is a SaaS product. **Fix to the standard, don't inherit it, and don't import the website's design language into the video.** When placing text yourself (`set_captions`, `set_layer_text`, `set_layer_style`, `add_layer`, devcli `place`/`captions`), set `y` / `font_family` / `font_weight` / `background_style` to the standard from the start.
|
|
374
452
|
|
|
375
|
-
> Local devcli renders enforce this automatically: `renderCompositionLocally` runs `normalizeTikTokCaptionLayout` (src/devcli/composition-edit.ts) on every production, clamping caption/text layers into the 8%–85% safe zone and coercing off-regime primary fonts to Montserrat. It
|
|
453
|
+
> Local devcli renders enforce part of this automatically: `renderCompositionLocally` runs `normalizeTikTokCaptionLayout` (src/devcli/composition-edit.ts) on every production, clamping caption/text layers into the 8%–85% safe zone and coercing off-regime primary fonts to Montserrat. It only fixes position and font family — it will happily render your Bootstrap card. Get it right in the composition so the editor preview, the local render, and any cloud render match.
|
|
376
454
|
|
|
377
455
|
### Animated captions — word-by-word caption styles (TikTok/CapCut)
|
|
378
456
|
|