@mevdragon/vidfarm-devcli 0.5.0 → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/SKILL.director.md +107 -15
- package/SKILL.platform.md +15 -11
- package/dist/src/cli.js +550 -7
- package/dist/src/devcli/composition-edit.js +287 -0
- package/package.json +2 -5
package/SKILL.director.md
CHANGED
|
@@ -72,7 +72,8 @@ Priority: Gemini → OpenAI → OpenRouter. If a director has no keys, smart dec
|
|
|
72
72
|
Templates are listed on the Vidfarm homepage and `/discover`. Each has a `template_...` id, a preview MP4, source video metadata, and viral DNA.
|
|
73
73
|
|
|
74
74
|
- Browser: browse `/discover`
|
|
75
|
-
- API: `GET /discover/feed` — returns `{ templates: [{ templateId, slugId, title, previewUrl, viralDna, durationSeconds, sourceType, ... }], next_cursor }`
|
|
75
|
+
- API: `GET /discover/feed` — returns `{ templates: [{ templateId, slugId, title, previewUrl, viralDna, durationSeconds, sourceType, promotions, keywords, summary, ... }], next_cursor }`
|
|
76
|
+
- Search: `GET /discover/feed?q=<offer>&limit=20` keyword-searches the catalog so you can answer "which templates suit my `<offer>`?" (e.g. `?q=weight%20loss%20app`). Ranking is a coarse keyword match over each template's title, viral DNA, and the decompose-derived `promotions` (offer categories the format can sell), `keywords`, and `summary` — read those fields to judge fit and rank the best 3-6 semantically. `GET /api/v1/videos?q=<offer>&limit=20[&mine=true]` runs the same search over source **inspirations**. `promotions`/`keywords`/`summary` are populated by the decompose pass, so a freshly ingested template that has not been decomposed yet won't match on them — decompose it (or fall back to broader/synonym terms) if a search comes up empty.
|
|
76
77
|
|
|
77
78
|
Each template exposes a public preview:
|
|
78
79
|
|
|
@@ -90,7 +91,7 @@ To bring a new viral video into the catalog as a **private** template you own, i
|
|
|
90
91
|
- `DELETE /discover/templates/:entryId` — remove a private inspiration/template you own (accepts either the `inspiration_...` or minted `template_...` id).
|
|
91
92
|
- `POST /api/v1/inspirations/:inspirationId/decompose { user_prompt? }` — AI-decompose an inspiration's downloaded video into scenes (requires a saved provider key; same 120s source cap as auto-decompose).
|
|
92
93
|
|
|
93
|
-
devcli: `vidfarm inspiration-add <url>`, `vidfarm inspiration-rm <id>`, `vidfarm inspiration-decompose <id>`, and `vidfarm
|
|
94
|
+
devcli: `vidfarm inspiration-add <url>`, `vidfarm inspiration-rm <id>`, `vidfarm inspiration-decompose <id>`, `vidfarm discover [query]` to browse/search templates, and `vidfarm videos [query] [--mine]` to browse/search source inspirations.
|
|
94
95
|
|
|
95
96
|
## Fork a template
|
|
96
97
|
|
|
@@ -122,7 +123,7 @@ The editor is a full timeline surface:
|
|
|
122
123
|
- **Timeline** — multi-track, drag/trim/split clips, group/ungroup layers
|
|
123
124
|
- **Inspector panel** — per-layer property editor (text, color, font, position, size, media source, playback start, volume)
|
|
124
125
|
- **Canvas frame overlay** — 8-handle resize + WYSIWYG positioning
|
|
125
|
-
- **Chat panel** — AI assistant that can programmatically edit the composition (
|
|
126
|
+
- **Chat panel** — AI assistant that can programmatically edit the composition (`add_layer`, `remove_layer`, `set_layer_timing/visual/style/media/text`, `group_layers`, `generate_layer`, `replace_composition_html`, ...). The assistant uses the caller's saved provider keys. **`generate_layer`** is the one-shot "generate AI media and place it": it submits the `/videos/generate` or `/images/generate` primitive job, drops a placeholder clip into the target slot (fill a blank gap, replace a scene, or overlay), and auto-swaps in the finished media when the job settles — no manual polling. In-flight generations surface in `editor_context.pending_generations`.
|
|
126
127
|
- **Auto-save** — every commit writes composition.html + composition.json to the working state. No explicit save button.
|
|
127
128
|
|
|
128
129
|
The editor persists to:
|
|
@@ -203,12 +204,42 @@ Use it whenever you need to know what the video says or shows: writing/translati
|
|
|
203
204
|
- **Editor chat (frontend AI)** exposes this as the optional `video_context` tool — the copilot calls it on demand.
|
|
204
205
|
- **Desktop agents (Claude Code / Codex)**: fetch the `video-context.json` route directly (e.g. `vidfarm api GET /api/v1/compositions/<forkId>/video-context.json`) to ground edits in what the video says and shows.
|
|
205
206
|
|
|
207
|
+
## Generate AI media and drop it on the timeline
|
|
208
|
+
|
|
209
|
+
Both surfaces can generate a NEW AI video/image (optionally conditioned on reference images) and place it into the timeline — filling a blank gap or replacing a scene — using the async generation primitives (`POST /api/v1/primitives/videos/generate`, `/images/generate`). Generation is a **queued job**: the POST returns `202 { job_id }`, and the finished media URL only appears after the job settles (`result.primary_file_url` / `video.file_url` / an artifact `public_url`). Poll `GET /api/v1/user/me/jobs/:jobId` until `status: "succeeded"`.
|
|
210
|
+
|
|
211
|
+
**Cost gate (see the Cost spectrum section):** AI **video** generation is the expensive end of the spectrum ($1–$10+/video) — **ask the director's permission before generating video**, and prefer reusing existing footage (My Files / their computer / a web search) or filming when it would do. **Image** generation is cheap — use it freely without asking. When a director wants specificity that only AI video can deliver and their budget allows, this is the right tool; just confirm first.
|
|
212
|
+
|
|
213
|
+
- **Editor web copilot**: use the `generate_layer` editor_action — it does the whole flow (submit job → placeholder clip → auto-swap on completion) and reads `pending_generations` from `editor_context`. Set `media_type`, `prompt`, `aspect_ratio` (match the canvas), and `intent` (`fill_gap` with `start`/`duration`, `replace_layer` with `replace_layer_key`, or `add`). For character consistency pass a cast `reference_url` in `input_references` (video) / `prompt_attachments` (image).
|
|
214
|
+
|
|
215
|
+
- **Desktop agents (Claude Code / Codex)** — two devcli commands:
|
|
216
|
+
```bash
|
|
217
|
+
# 1) Sync the fork to disk and see where the blank space and scenes are
|
|
218
|
+
vidfarm pull <forkId> --dir ./work
|
|
219
|
+
# → prints timeline gaps (e.g. 4-7s) and scene/layer keys, canvas aspect ratio,
|
|
220
|
+
# and whether video-context.json / cast.json are present for grounding.
|
|
221
|
+
|
|
222
|
+
# 2) Generate + place in one call (fill a gap)
|
|
223
|
+
vidfarm generate video --prompt "her walking through the door" \
|
|
224
|
+
--ref ./cast/girl.png --aspect-ratio 9:16 --duration 4 \
|
|
225
|
+
--place ./work --at 4
|
|
226
|
+
# → polls the job to the finished MP4, then inserts it at 4s in ./work/composition.html
|
|
227
|
+
|
|
228
|
+
# …or replace a scene with a generated still
|
|
229
|
+
vidfarm generate image --prompt "product hero on marble" --aspect-ratio 9:16 \
|
|
230
|
+
--place ./work --replace <layer_key>
|
|
231
|
+
|
|
232
|
+
# 3) Push it back (or it live-morphs automatically under `vidfarm serve`)
|
|
233
|
+
vidfarm publish --dir ./work --fork <forkId>
|
|
234
|
+
```
|
|
235
|
+
`--ref @localfile` (or a bare path) auto-uploads the image and uses its durable URL as a reference. `generate` prints the media URL (drop `--place` to just get the URL); `place` can be run separately on any media URL (`vidfarm place ./work --src <url> --at <time>|--replace <layer_key>`). Placement produces the exact same clip markup the browser editor makes, so a human can finish it in the Trackpad Editor.
|
|
236
|
+
|
|
206
237
|
## Render (publish to MP4)
|
|
207
238
|
|
|
208
|
-
Rendering publishes the fork's current working state to an MP4 using
|
|
239
|
+
Rendering publishes the fork's current working state to an MP4 using HyperFrames (cloud Lambda fan-out on the deployed host, or the free in-process renderer on a local `vidfarm serve` box). In the Trackpad Editor this is the **Render** button. The REST route is `/render`; the older `/export` path is kept as a deprecated alias for already-published devcli clients (treat "Render", "publish", `/render`, and `/export` as the same operation). The devcli exposes it as `vidfarm render <forkId> [--wait]`.
|
|
209
240
|
|
|
210
241
|
```
|
|
211
|
-
POST /api/v1/compositions/:forkId/
|
|
242
|
+
POST /api/v1/compositions/:forkId/render
|
|
212
243
|
Content-Type: application/json
|
|
213
244
|
|
|
214
245
|
{ "title": "optional", "version": null, "tracer": "optional-trace-id", "html": "optional HTML to save to working state before rendering" }
|
|
@@ -314,15 +345,40 @@ DELETE /api/v1/compositions/:forkId
|
|
|
314
345
|
|
|
315
346
|
Soft-delete only. The fork's `deletedAt` is set. Versions and share links are preserved for audit.
|
|
316
347
|
|
|
348
|
+
## Cost spectrum (free → $10+/video) — default to saving the director money
|
|
349
|
+
|
|
350
|
+
There is no single price for a video. The **approach** the director picks sets the cost, across a wide spectrum. Always steer toward the cheapest approach that still meets their goal, and make the tradeoff explicit rather than silently choosing an expensive path.
|
|
351
|
+
|
|
352
|
+
| Approach | Typical cost | How |
|
|
353
|
+
|---|---|---|
|
|
354
|
+
| **Reuse + render locally** | **free** | Fork an already-decomposed template, swap captions / images / video with existing MP4s (from **My Files**, the director's **local computer**, or a **web search**), and render **locally** via `vidfarm serve` (native in-process HyperFrames render — free and unguarded, no cloud). |
|
|
355
|
+
| **Reuse + cloud render** | **~$0.001 – $0.03** | Same reuse, but render on the cloud renderer (`POST /compositions/:forkId/render`, ~$0.01–$0.10 depending on length/res). Cheap **image** generation/edits fit in this band too. |
|
|
356
|
+
| **AI-generate some scenes** | **~$1** | Replace a few scenes with AI-generated video clips for high specificity/customization (see the "Generate AI media" section). |
|
|
357
|
+
| **Heavy AI generation** | **$10+** | Many/long AI video clips, custom characters, fully bespoke scenes. |
|
|
358
|
+
|
|
359
|
+
Rules of thumb:
|
|
360
|
+
|
|
361
|
+
- **Reusing existing footage — or footage the director films themselves — is the cheapest.** AI-generating characters/scenes is the most expensive. Editing captions/images on a reused template is nearly free.
|
|
362
|
+
- **Local render is free; cloud render costs pennies.** If the director is iterating with a coding agent on a `vidfarm serve` box, they can render locally at no charge until they want the durable cloud MP4.
|
|
363
|
+
- **Decompose is a one-time ~$0.10** (smart decompose ≈ provider passthrough + GhostCut ~$0.10/30s) per **new** source video — but the director can **skip it entirely** by forking a template that's **already decomposed** in the catalog.
|
|
364
|
+
- **Image generation is cheap** — use it freely, no need to ask permission.
|
|
365
|
+
- **AI video generation is expensive — ask the director's permission before using it.** Default to reuse/local/image approaches unless they've okayed video gen or told you the budget covers it.
|
|
366
|
+
- **Ask about budget.** During Getting Started (and whenever it's relevant per editor project, or when the director asks about cost), ask roughly what they want to spend per video, and pick the approach band that fits. If they haven't said, assume the cheapest approach that works.
|
|
367
|
+
|
|
368
|
+
The director always chooses the method that works for them — the point is to surface the tradeoff and default to thrift, not to force the cheapest path.
|
|
369
|
+
|
|
317
370
|
## Billing
|
|
318
371
|
|
|
319
372
|
Vidfarm charges directly in USD from the caller's wallet. There are no credits.
|
|
320
373
|
|
|
321
374
|
- **Wallet top-up** — Stripe checkout via the Settings UI, backed by `POST /settings/wallet/funding-link` (browser session; returns `{ checkout_url, client_reference_id }`)
|
|
322
375
|
- **Balance & history** — shown on the Settings page; billing events via `GET /u/:customerId/settings/wallet/events` (browser session). There is no `/api/v1/user/me/wallet` REST endpoint.
|
|
323
|
-
- **Cost anchors
|
|
324
|
-
-
|
|
325
|
-
-
|
|
376
|
+
- **Cost anchors** (see the Cost spectrum section above for how these combine per approach):
|
|
377
|
+
- Local render on a `vidfarm serve` box (in-process HyperFrames): **free**
|
|
378
|
+
- Cloud render (HyperFrames Lambda fan-out): typically $0.01 – $0.10 per MP4 depending on length/resolution
|
|
379
|
+
- Image generation / edit: cheap (provider passthrough) — use freely
|
|
380
|
+
- AI **video** generation: expensive ($1–$10+/video territory) — ask permission first
|
|
381
|
+
- Auto-decompose smart mode: pass-through of caller's AI provider spend (~$0.10 one-time with GhostCut), or skip it by forking an already-decomposed template
|
|
326
382
|
- Auto-decompose time-slice: free
|
|
327
383
|
- GhostCut subtitle removal: ~$0.10 per 30 seconds of source video
|
|
328
384
|
|
|
@@ -339,7 +395,7 @@ For agents that operate Vidfarm headlessly, the typical loop is:
|
|
|
339
395
|
1. `POST /api/v1/compositions { template_id }` → fork
|
|
340
396
|
2. `PUT /api/v1/compositions/:forkId/composition.html` → apply edits
|
|
341
397
|
3. `PATCH /api/v1/compositions/:forkId/composition.json` → update metadata
|
|
342
|
-
4. `POST /api/v1/compositions/:forkId/
|
|
398
|
+
4. `POST /api/v1/compositions/:forkId/render { tracer }` → render
|
|
343
399
|
5. Poll `GET /api/v1/compositions/:forkId/renders/:renderId` until `SUCCEEDED`
|
|
344
400
|
6. Use `outputUrl` in the next stage
|
|
345
401
|
|
|
@@ -353,16 +409,20 @@ Send a stable `tracer` on export so retries are traceable and filterable in job
|
|
|
353
409
|
|
|
354
410
|
| Command | REST route | Flow step |
|
|
355
411
|
|---|---|---|
|
|
356
|
-
| `vidfarm discover` | `GET /discover/feed` | browse
|
|
412
|
+
| `vidfarm discover [query]` | `GET /discover/feed[?q=]` | browse/search templates |
|
|
413
|
+
| `vidfarm videos [query] [--mine]` | `GET /api/v1/videos[?q=&mine=]` | browse/search source inspirations |
|
|
357
414
|
| `vidfarm inspiration-add <url>` | `POST /discover/templates` | add your own source |
|
|
358
415
|
| `vidfarm inspiration-rm <id>` | `DELETE /discover/templates/:id` | remove a private one |
|
|
359
416
|
| `vidfarm inspiration-decompose <id>` | `POST /api/v1/inspirations/:id/decompose` | AI-decompose an inspiration |
|
|
360
417
|
| `vidfarm fork <template_id>` | `POST /api/v1/compositions` | fork a template |
|
|
418
|
+
| `vidfarm pull <forkId> [--dir <p>]` | `GET .../compositions/:forkId/{composition.html,json,video-context.json,cast.json}` | sync a fork to disk + print gaps/scene keys |
|
|
419
|
+
| `vidfarm generate <image\|video> --prompt "…"` | `POST /api/v1/primitives/{images,videos}/generate` (polls job) | generate AI media → finished URL |
|
|
420
|
+
| `vidfarm place <dir> --src <url> [--at\|--replace]` | (edits local composition.html) | drop media into a gap / over a scene |
|
|
361
421
|
| `vidfarm decompose <forkId>` | `POST .../compositions/:forkId/auto-decompose` | split source into scenes |
|
|
362
422
|
| `vidfarm ghostcut <forkId>` | `GET .../compositions/:forkId/ghostcut` | subtitle-removal status |
|
|
363
423
|
| `vidfarm snapshot <forkId>` | `POST .../compositions/:forkId/versions` | save composition fork version |
|
|
364
424
|
| `vidfarm versions <forkId>` | `GET .../compositions/:forkId/versions` | list versions |
|
|
365
|
-
| `vidfarm render <forkId> [--wait]` | `POST .../compositions/:forkId/
|
|
425
|
+
| `vidfarm render <forkId> [--wait]` | `POST .../compositions/:forkId/render` | render video to MP4 |
|
|
366
426
|
| `vidfarm render-status <forkId> <renderId>` | `GET .../compositions/:forkId/renders/:renderId` | poll a render |
|
|
367
427
|
| `vidfarm visibility <forkId> <private\|public>` | `PATCH .../compositions/:forkId/visibility` | set visibility |
|
|
368
428
|
| `vidfarm clone <forkId>` | `POST .../compositions/:forkId/clone` | clone a fork |
|
|
@@ -374,12 +434,28 @@ Send a stable `tracer` on export so retries are traceable and filterable in job
|
|
|
374
434
|
| `vidfarm login <email>` / `vidfarm verify <email> <code>` | `POST /api/v1/user/request-otp` · `verify-otp` | get an API key |
|
|
375
435
|
| `vidfarm whoami` | `GET /api/v1/user/me` | who am I |
|
|
376
436
|
| `vidfarm provider-keys` / `vidfarm add-provider-key <p> <secret>` | `GET`·`POST /api/v1/user/me/provider-keys` | manage AI keys |
|
|
377
|
-
| `vidfarm upload <file
|
|
437
|
+
| `vidfarm upload <file> [--folder <path>]` | `POST /api/v1/user/me/temporary-files/upload` | upload → durable URL (ephemeral) |
|
|
378
438
|
| `vidfarm download <url> [dest]` | (streams any URL to disk) | download media |
|
|
439
|
+
| `vidfarm files [--folder <path>]` | `GET /api/v1/user/me/attachments` | list My Files assets + folders |
|
|
440
|
+
| `vidfarm get-file <id> [dest] [--print]` | (resolve id → view_url, then stream/print) | read one My Files asset |
|
|
441
|
+
| `vidfarm put-file <file> [--folder <p>] [--as <name>] [--content/--stdin]` | `POST /api/v1/user/me/attachments/upload` | write into My Files (persistent) |
|
|
379
442
|
|
|
380
443
|
**Frontend URLs are first-class output.** Every command that has a human-openable page (editor, discover, approved-post preview, share link) prints that prod frontend URL as a highlighted line. `render --wait` polls to completion and prints the final MP4 URL; `approve` prints the approved-post `share_url`. Add `--json` to any command for pure JSON (agent-friendly, no banners).
|
|
381
444
|
|
|
382
|
-
**Uploads/downloads live in the devcli** because they are genuine multi-step / streaming flows: `upload` posts the file as multipart to the temporary-files route and prints the durable URL to drop into a composition or approved post; `download` streams any Vidfarm/media URL to disk.
|
|
445
|
+
**Uploads/downloads/My-Files writes live in the devcli** because they are genuine multi-step / streaming flows: `upload` posts the file as multipart to the **ephemeral** temporary-files route and prints the durable URL to drop into a composition or approved post; `put-file` posts to the **persistent** My Files route (`/me/attachments/upload`) so context docs and brand assets live in the user's library (accepts a local file, `--content <text>`, or piped `--stdin` with `--as <name>`); `download` streams any Vidfarm/media URL to disk.
|
|
446
|
+
|
|
447
|
+
## My Files (the user's asset library)
|
|
448
|
+
|
|
449
|
+
Each user has a persistent **My Files** filesystem — their own uploaded videos (mp4/mov/webm), images (png/jpg/jpeg/gif/webp/svg), audio (mp3/wav/m4a/aac), and documents (pdf/md/txt/csv) — organized into **virtual folders**. This is where a user keeps footage, brand assets, logos, music, briefs, scripts, and the durable strategy docs from Getting Started (About.md, awareness-levels.md, etc.). Both the editor AI copilot and an agent CLI can **read and write** it to find or save assets instead of asking the user to re-upload or paste URLs.
|
|
450
|
+
|
|
451
|
+
- **List** — `GET /api/v1/user/me/attachments` → `{ attachments: [{ id, fileName, contentType, sizeBytes, folderPath, viewUrl, createdAt }], folders: [string] }`. The `viewUrl` is a durable URL you can drop into a composition layer or pass into a primitive route. devcli: `vidfarm files [--folder <path>]`.
|
|
452
|
+
- **Read one** — resolve it from the list and stream/read its `viewUrl`. devcli: `vidfarm get-file <id> [dest]` (add `--print` to print text contents of md/txt/csv/json inline). For images/video/audio/pdf, reference the `viewUrl` as media — you can't read their bytes as text.
|
|
453
|
+
- **Write into My Files** — `POST /api/v1/user/me/attachments/upload` (multipart `file` + optional `folder_path`) persists a file into the library. devcli: `vidfarm put-file <localFile> --folder <path>`, or pipe/inline text without a temp file: `echo "…" | vidfarm put-file --stdin --as About.md --folder acme` / `vidfarm put-file --content "…" --as About.md`. This is how an agent saves Getting Started context (About.md, Interview.md, awareness-levels.md, persuasive-angles.md, ad-hooks.md) or drops a logo/product demo into a namescoped folder. In the web editor copilot the same capability is the **`browse_files` write action** (`action=write` with `file_name` + `content` + optional `folder_path`), which accepts text files (md/txt/csv/json/srt/vtt).
|
|
454
|
+
- **`vidfarm upload` is different** — it targets the **ephemeral** temporary-files store (`POST /api/v1/user/me/temporary-files/upload`) for a throwaway durable URL to drop straight into a composition or approved post. Use `put-file` when you want the asset to persist in the user's My Files library; use `upload` for one-shot media you just need a URL for.
|
|
455
|
+
|
|
456
|
+
**Assume My Files is multi-offer.** A user often runs more than one product, offer, brand, or region, and namescopes assets into folders accordingly — by product (`acme-skincare/`, `zensleep/`), by offer/campaign (`summer-sale/`), by region (`us/`, `eu/`), by asset type (`logos/`, `ugc-clips/`), or any arbitrary scheme. There is **no fixed layout** — read the `folders` tree first and reason about how this user organized things. Before pulling assets for a task, infer which folder(s) match the product/offer/region the current composition is about (match folder names to the composition title, video context, and what the user said), then scope reads to that folder so you never mix one brand's logo/product-shot/music into another brand's video. If the target offer is ambiguous, ask which product/offer/region (or which folder) this work is for rather than guessing across offers.
|
|
457
|
+
|
|
458
|
+
In the editor web copilot the same filesystem is exposed via the **`browse_files` tool** (`action=list` / `action=read` / `action=write`), so the copilot follows the identical reasoning: list folders, pick the right offer's folder, then read an asset — or `write` a text doc (About.md, awareness-levels.md, …) into that folder. The devcli equivalents are `vidfarm files` / `get-file` / `put-file`.
|
|
383
459
|
|
|
384
460
|
## Local dev loop (`vidfarm serve`)
|
|
385
461
|
|
|
@@ -409,7 +485,7 @@ To *analyze* the source media locally (videos, transcript, recurring cast), read
|
|
|
409
485
|
|
|
410
486
|
- Do **not** manipulate composition HTML by string concatenation. Always parse, edit, and re-serialize the DOM. The editor and runtime rely on `data-start`, `data-duration`, `data-track-index`, `data-hf-id`, `data-composition-id` attributes being well-formed.
|
|
411
487
|
- Do **not** save AI provider keys in the composition HTML or JSON. They belong in the caller's provider-keys record.
|
|
412
|
-
- Do **not** call
|
|
488
|
+
- Do **not** call the renderer directly. Publishing goes through `POST /api/v1/compositions/:forkId/render` so billing, retries, cost caps, and Lambda quotas are enforced.
|
|
413
489
|
- Do **not** re-invent scene manipulation as REST commands (`AddScene`, `RemoveScene`, `UpdateSceneField`). The Trackpad Editor is the surface for that. The pattern is: mutate the composition HTML DOM, PUT it back.
|
|
414
490
|
|
|
415
491
|
## Alpha posture
|
|
@@ -420,6 +496,22 @@ Vidfarm is in active development. Endpoints, response shapes, and the editor UI
|
|
|
420
496
|
- treat `latest_version` as the source of truth for what the editor loaded
|
|
421
497
|
- if a publish fails, check `renders/:renderId` for the error phase and stderr before retrying
|
|
422
498
|
|
|
499
|
+
## Getting started / onboarding a new director
|
|
500
|
+
|
|
501
|
+
Only run this flow when the director signals they **don't know where to start** ("getting started", "help me set up", "I don't know where to begin"). If they already know what they want, skip straight to it — never force onboarding. Directors can also jump to any single step (e.g. "just find me a good template for X" → go straight to template discovery).
|
|
502
|
+
|
|
503
|
+
The point of onboarding is to build **durable, reusable context** in My Files, namescoped under the director's product/offer folder, so future chats and agents can read it back. Save each artifact as Markdown with `browse_files` write (web copilot) or `vidfarm put-file` (devcli):
|
|
504
|
+
|
|
505
|
+
1. **Product context** → `About.md` (basic offer/product) or a deeper `Interview.md` if they want depth. Drive the interview questions with `brainstorm/coldstart`.
|
|
506
|
+
2. **Awareness level** (Eugene Schwartz — problem-aware, solution-unaware, …) → `awareness-levels.md`. If it's genuinely unknown after thinking it through, note that ads for **every** level should be made and tested. Use `brainstorm/awareness_stages`.
|
|
507
|
+
3. **Persuasive angles** → `persuasive-angles.md`, via `brainstorm/angles`.
|
|
508
|
+
4. **Hooks** → `ad-hooks.md`, via `brainstorm/hooks`.
|
|
509
|
+
5. **Brand assets & demos** — ask if they have logos/mascots/themes (suggest a `/brand-assets/` folder, e.g. `/brand-assets/logo.png`) or product demos / screen recordings (suggest a `/product-demos/` folder). `browse_files list` / `vidfarm files` first to see what they already uploaded; filenames should be descriptive so assets are findable.
|
|
510
|
+
6. **Budget** — ask roughly what they want to spend per video, and map it to the Cost spectrum (free reuse+local render → pennies for cloud render → ~$1 for some AI scenes → $10+ for heavy AI gen). This sets which approach you default to and whether AI **video** generation is on the table (ask permission before using it; image gen is cheap and fine). Budget can also be revisited per editor project.
|
|
511
|
+
7. **Recommend & adapt a template** — pair what you now know about the offer against the decomposed template catalog (`GET /discover/feed?q=<offer>`, read each result's `promotions`/`keywords`/`summary`), recommend the best 3-6, then fork and **modify** the winner to fit their offer. Prefer already-decomposed templates so the director skips the ~$0.10 decompose cost.
|
|
512
|
+
|
|
513
|
+
**Assume multiple offers.** My Files is multi-offer (see the My Files section) — namescope every onboarding artifact under the right product/offer/region folder (`acme-skincare/About.md`, not a bare `About.md`) so one brand's context never bleeds into another's.
|
|
514
|
+
|
|
423
515
|
## Default assistance pattern
|
|
424
516
|
|
|
425
517
|
When a director asks "make me a video", the default sequence is:
|
|
@@ -459,7 +551,7 @@ Apply subtle camouflage transforms (zoom, tilt, rotate, saturation, playback spe
|
|
|
459
551
|
- Body: `{ "tracer": "...", "payload": { ...fields... }, "webhook_url"?: "..." }`
|
|
460
552
|
- Note: webhook delivery is not yet active — `webhook_url` is accepted and persisted on the job but never fired. Poll the job endpoints (`GET /api/v1/primitives/jobs/:jobId`) for completion.
|
|
461
553
|
- Response: standard primitive job. Poll to completion, then read `primary_file_url` (also `video.file_url` for MP4 or `image.file_url` for stills)
|
|
462
|
-
- Billing: metered as a
|
|
554
|
+
- Billing: metered as a HyperFrames render (via `hyperframes_lambda` on prod cloud render; free on a local serve box)
|
|
463
555
|
|
|
464
556
|
Payload fields:
|
|
465
557
|
|
package/SKILL.platform.md
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: vidfarm-platform
|
|
3
|
-
description: Vidfarm platform architecture reference. Use this for questions about how compositions, forks, versions, storage, DynamoDB records, the Trackpad Editor runtime,
|
|
3
|
+
description: Vidfarm platform architecture reference. Use this for questions about how compositions, forks, versions, storage, DynamoDB records, the Trackpad Editor runtime, HyperFrames render, editor-chat, primitives, and billing fit together. This is not a director skill (see SKILL.director.md) and not a runbook — it explains the system.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Vidfarm Platform Architecture
|
|
7
7
|
|
|
8
|
-
Vidfarm is a **serverless composition studio** for short-form video. Directors fork templates, edit compositions on a timeline (the Trackpad Editor), publish MP4s via
|
|
8
|
+
Vidfarm is a **serverless composition studio** for short-form video. Directors fork templates, edit compositions on a timeline (the Trackpad Editor), publish MP4s via HyperFrames render (shared cloud Lambda fan-out, or free in-process render on a local serve box), and share results.
|
|
9
9
|
|
|
10
10
|
This document is the architectural reference — what runs where, what data lives where, and how the pieces connect. Use it when planning changes, debugging cross-cutting behavior, or onboarding to the platform.
|
|
11
11
|
|
|
@@ -184,15 +184,15 @@ The editor is a single monolithic React component (~9k lines). It builds cleanly
|
|
|
184
184
|
|
|
185
185
|
Publish flow:
|
|
186
186
|
|
|
187
|
-
1. Client `POST /api/v1/compositions/:forkId/
|
|
187
|
+
1. Client `POST /api/v1/compositions/:forkId/render` (`/export` is a deprecated alias; the editor button and devcli command are both **Render**; treat "render", "publish", `/render`, and `/export` as the same operation)
|
|
188
188
|
2. API stamps `data-composition-id="<forkId>@v<N>"` into the composition HTML, PUTs the stamped HTML to working, and reads composition.json
|
|
189
189
|
3. API resolves the caller's provider keys (preflight check for required models)
|
|
190
190
|
4. API creates a `job` record and invokes the `hyperframes_render` primitive Lambda
|
|
191
|
-
5. The
|
|
191
|
+
5. The renderer runs HyperFrames: on the deployed host it stages the composition and dispatches to the shared HyperFrames render state machine (chunked Lambda fan-out); on a local `vidfarm serve` box with no cloud stack it renders in-process (headless Chromium → ffmpeg) for free. See `HyperframesService` in `src/services/hyperframes.ts`.
|
|
192
192
|
6. On success, the primitive returns `{ primary_file_url: "s3://.../output.mp4" }`, the API creates a `ForkVersionRecord` with `reason: "publish"`, writes the snapshot to `versions/<N>/`, and updates the fork's `latestVersion` and `currentRenderJobId`
|
|
193
193
|
7. Client polls `GET /api/v1/compositions/:forkId/renders/:renderId` for `{ status: "RUNNING" | "SUCCEEDED" | "FAILED", progress, outputUrl, cost }` (see `adaptJobToPublishStatus` in `src/app.ts`)
|
|
194
194
|
|
|
195
|
-
The
|
|
195
|
+
The HyperFrames render state machine is **shared across all templates and forks** — it's an infrastructure primitive, not a per-template deployment. Its ARN/bucket are passed to the Vidfarm stack as env vars (`HYPERFRAMES_RENDER_*`). Remotion has been fully removed.
|
|
196
196
|
|
|
197
197
|
## Auto-decompose
|
|
198
198
|
|
|
@@ -237,12 +237,14 @@ Configuration: `GHOSTCUT_KEY` and `GHOSTCUT_SECRET` env vars per stage. Pass-thr
|
|
|
237
237
|
|
|
238
238
|
The chat model uses the **caller's provider keys**. No Vidfarm-owned model calls happen in this path.
|
|
239
239
|
|
|
240
|
+
Beyond the Editor Action API, the chat harness registers shared tools defined in **two synced copies** (`src/app.ts` for the in-process dev server and `infra/lambda/editor-chat.ts` for the deployed Lambda — keep them in lockstep): `http_request` (allowlisted REST/primitive calls), `frontend_action` (tracer/thread state), `video_context`, and `browse_files`. `browse_files` reads **and writes** the caller's My Files library — `action=list`/`read`/`write`, where `write` saves a text file (md/txt/csv/json/srt/vtt) via `POST /me/attachments/upload`. Write is what lets the copilot persist Getting Started context (About.md, awareness-levels.md, persuasive-angles.md, ad-hooks.md) namescoped under a product/offer folder; the system prompt in `src/editor-chat.ts` drives when to save. The devcli mirror is `vidfarm put-file`.
|
|
241
|
+
|
|
240
242
|
## Primitives
|
|
241
243
|
|
|
242
244
|
Primitives are hosted operations that are NOT templates — they're reusable building blocks:
|
|
243
245
|
|
|
244
246
|
- `create_image`, `edit_image`, `inpaint_image` — image generation
|
|
245
|
-
- `create_html_image` —
|
|
247
|
+
- `create_html_image` — HyperFrames HTML → PNG (in-process headless Chromium)
|
|
246
248
|
- `remove_background` — RapidAPI-backed image background removal → durable PNG/WebP
|
|
247
249
|
- `download_media_video` — social/media URL → durable MP4
|
|
248
250
|
- `hyperframes_render` — composition HTML → MP4 (the render primitive publishing uses)
|
|
@@ -258,8 +260,11 @@ Key mechanics:
|
|
|
258
260
|
|
|
259
261
|
- Every wallet mutation carries a `tracer` (idempotency key)
|
|
260
262
|
- Provider spend is passed through at cost — no markup on AI provider calls
|
|
261
|
-
- Platform infrastructure (
|
|
263
|
+
- Platform infrastructure (cloud render, GhostCut, storage) has a small safety buffer applied
|
|
262
264
|
- Failed jobs are refunded automatically when the failure is on the platform side
|
|
265
|
+
- **Local `vidfarm serve` render is free + unguarded** — `publishCompositionToRenderer` runs the render in-process (`runPrimitiveJobInProcess`) and skips the wallet/paid guard when no cloud state machine is wired; only cloud render/templates are paid + wallet-guarded
|
|
266
|
+
|
|
267
|
+
**Cost spectrum (director-facing framing; the same guidance lives in `SKILL.director.md` and the editor-chat system prompt).** Per-video cost is set by the *approach*, not a fixed price: FREE (reuse a decomposed template, swap captions/images/existing MP4s, render locally) → ~$0.001–$0.03 (same reuse + cloud render ~$0.01–$0.10) → ~$1 (AI-generate a few scenes) → $10+ (heavy AI generation). Image generation is cheap; AI **video** generation is the expensive end — the copilot is instructed to ask permission before video gen but not before image gen. Smart decompose is a ~$0.10 one-time per new source (provider passthrough + GhostCut) and is avoided by forking already-decomposed catalog templates. The AI defaults to the cheapest approach that meets the goal and surfaces the tradeoff.
|
|
263
268
|
|
|
264
269
|
The billing Lambda has its own custom domain because Stripe webhooks require a stable URL not tied to the main API.
|
|
265
270
|
|
|
@@ -267,7 +272,7 @@ The billing Lambda has its own custom domain because Stripe webhooks require a s
|
|
|
267
272
|
|
|
268
273
|
There are two kinds of things directors can fork:
|
|
269
274
|
|
|
270
|
-
1. **Draft templates** — pre-decomposed compositions checked into the repo under `templates/vidfarm_template_XXXX/`. Each has a `composition.html`, `composition.json`, `template.config.json`, a `SKILL.md`, and a `src/`
|
|
275
|
+
1. **Draft templates** — pre-decomposed compositions checked into the repo under `templates/vidfarm_template_XXXX/`. Each has a `composition.html`, `composition.json`, `template.config.json`, a `SKILL.md`, and a `src/` for any custom assets. Registered templates open in the browser at `/editor/:templateId` (there is no `/hyperframes/:slug` route).
|
|
271
276
|
2. **Submitted videos** — raw uploads from directors, decomposed on demand via auto-decompose.
|
|
272
277
|
|
|
273
278
|
Templates are static. Compositions/forks are the mutable working copies.
|
|
@@ -281,7 +286,6 @@ Per-stage config is loaded via `.env.<stage>`. Critical variables:
|
|
|
281
286
|
- `VIDFARM_<STAGE>_ENCRYPTION_SECRET` — DynamoDB field encryption
|
|
282
287
|
- `VIDFARM_<STAGE>_API_KEY_SALT` — API key hashing
|
|
283
288
|
- `VIDFARM_<STAGE>_BOOTSTRAP_*` — initial admin user
|
|
284
|
-
- `VIDFARM_<STAGE>_REMOTION_*` — shared Remotion Lambda ARN + region
|
|
285
289
|
- `VIDFARM_<STAGE>_RESEND_*` — email
|
|
286
290
|
- `VIDFARM_<STAGE>_SENTRY_*` — Sentry DSN
|
|
287
291
|
- `GHOSTCUT_KEY`, `GHOSTCUT_SECRET` — GhostCut credentials
|
|
@@ -339,7 +343,7 @@ Legacy but still active:
|
|
|
339
343
|
## Common failure modes
|
|
340
344
|
|
|
341
345
|
- **Fork loads 404** — check ownership-agnostic migration ran; the fork may still be at the old `compositions/users/.../forks/` path.
|
|
342
|
-
- **Publish stuck in RUNNING** — check the primitive Lambda's CloudWatch logs.
|
|
346
|
+
- **Publish stuck in RUNNING** — check the primitive Lambda's CloudWatch logs. HyperFrames asset preflight failures usually mean a broken media URL in composition.html.
|
|
343
347
|
- **Smart decompose falls back to time-slice** — the caller has no provider keys, or all their keys returned errors. Check `provider_key_lease` records.
|
|
344
348
|
- **GhostCut task hangs** — client shows `pending` forever. Check the fork's `ghostcut.json` and the GhostCut dashboard. There is currently no cancel UI; the poll times out at 15 minutes.
|
|
345
349
|
- **Editor edits don't persist** — verify `PUT /composition.html` is 200. If the composition.html doesn't include `data-composition-id`, the API rejects the PUT at publish time.
|
|
@@ -362,7 +366,7 @@ Legacy but still active:
|
|
|
362
366
|
|
|
363
367
|
## Local editor dev loop (`vidfarm serve`)
|
|
364
368
|
|
|
365
|
-
`@mevdragon/vidfarm-devcli` (`src/cli.ts`) has two roles. First, `serve` boots the **full** Vidfarm backend locally — single origin, disk-backed records + storage — so Claude Code, Codex, or a human can edit `composition.html` on disk and see the browser live-morph the change (detailed below). Second, it wraps the **entire director REST flow** as 1:1 CLI commands — `discover`, `inspiration-add`, `fork`, `decompose`, `snapshot`, `render` (→ `/
|
|
369
|
+
`@mevdragon/vidfarm-devcli` (`src/cli.ts`) has two roles. First, `serve` boots the **full** Vidfarm backend locally — single origin, disk-backed records + storage — so Claude Code, Codex, or a human can edit `composition.html` on disk and see the browser live-morph the change (detailed below). Second, it wraps the **entire director REST flow** as 1:1 CLI commands — `discover`, `inspiration-add`, `fork`, `decompose`, `snapshot`, `render` (→ `/render`), `approve`, `schedule`, `posts`/`schedules`, `upload`/`download`, `files`/`get-file`/`put-file` (My Files read+write), `login`/`whoami`/`provider-keys`, plus a raw `api <METHOD> <path>` passthrough. Each command prints its prod frontend URL as a first-class output, and `--json` gives pure JSON for agents. The command↔route table lives in `SKILL.director.md`.
|
|
366
370
|
|
|
367
371
|
```bash
|
|
368
372
|
# Boot the full editor locally, pull a template's default fork onto disk, open pre-authed.
|