@mevdragon/vidfarm-devcli 0.20.6 → 0.20.8

package/.agents/skills/vidfarm-director/references/core-workflows.md

@@ -0,0 +1,273 @@
+## Mental model
+
+A Vidfarm composition is an **HTML document + JSON metadata**. Each layer (video clip, audio, image, text, caption) is a timed DOM element with `data-start`, `data-duration`, `data-track-index` attributes. The Trackpad Editor is a timeline UI over this document, similar to Premiere or DaVinci.
+
+Every director workflow goes through three concepts:
+
+1. **Template** — a published composition anyone can fork. Template ids start with `template_`; each template opens in the editor at `/editor/:templateId` and has a source video, a decomposed timeline, and metadata (viral DNA, captions, scenes).
+2. **Composition fork** — a private editable copy of a template. Every fork has a `forkId`, an owner, a working state, and a version history. The Trackpad Editor operates on a fork.
+3. **Publish** — render the current fork state to MP4 via a Lambda render job. Each publish snapshots the current working state as an immutable version.
+
+The Trackpad Editor is the primary surface. Most director actions happen there. This skill's REST endpoints exist for automation, CI, and headless flows.
+
+## Base URL and auth
+
+The current API base URLs are:
+
+- staging: `https://staging.vidfarm.cc`
+- production: `https://vidfarm.cc`
+
+Every browser action is authenticated by the session cookie set at `/login`. Headless API-key auth uses the `vidfarm-api-key` header set to the caller's API key (from Settings → API Keys).
+
+**Auth contract: API-key auth is the `vidfarm-api-key: <key>` request header and nothing else. `Authorization: Bearer <key>` is NOT supported — the server never reads the `authorization` header (see `requireAuth` in `src/app.ts`).**
+
+```
+vidfarm-api-key: vf_key_<id>
+```
+
+Scope note (verified against `src/app.ts`): the `vidfarm-api-key` header authenticates **every** authenticated surface — the `requireAuth` routes (`/api/v1/user/*`, `/api/v1/agency/*`, `/api/v1/templates/*`, `/api/v1/primitives/*`, `/api/v1/rate-limit-status`, editor-chat threads) **and** the `/api/v1/compositions/*` + `/api/v1/videos/*` + editor page routes, where an explicit API key takes precedence over any browser session cookie. Cookies remain how the web UI authenticates; headless callers never need one. Reads on composition routes also work without any credential because the `forkId` acts as an unguessable view bearer token, and share-link tokens (header `vidfarm-share-token` or `?share=` query) can grant `viewer`/`editor`/`publisher` roles.
+
+## Login flow
+
+Directors log in with OTP:
+
+1. `POST /api/v1/user/request-otp { email }` — sends a code to email; returns `{ ok, delivery }` (plus `dev_code` when delivery is `console` outside production)
+2. `POST /api/v1/user/verify-otp { email, code, name? }` — returns `{ customer, apiKey }` and mints a fresh API key
+
+Save the `apiKey` in the client and use it for automation (as the `vidfarm-api-key` header). The browser login form uses the session-cookie variants `POST /login/otp/request` and `POST /login/otp/verify` instead.
+
+## Provider keys
+
+Vidfarm executes many features (smart auto-decompose, editor chat) with the **caller's own AI provider keys**. Directors save keys once via:
+
+- UI: Settings → Provider Keys
+- API: `POST /api/v1/user/me/provider-keys { provider, secret, label? }` — one key per call; `provider` is `"openai" | "gemini" | "openrouter" | "perplexity"`. The key is validated against the provider before it is saved. List saved keys with `GET /api/v1/user/me/provider-keys`.
+
+Priority: Gemini → OpenAI → OpenRouter. If a director has no keys, smart decompose falls back to a deterministic time-based split.
+
+## Discover templates
+
+Templates are listed on the Vidfarm homepage and `/discover`. Each has a `template_...` id, a preview MP4, source video metadata, and viral DNA.
+
+- Browser: browse `/discover`
+- API: `GET /discover/feed` — returns `{ templates: [{ templateId, slugId, title, previewUrl, viralDna, durationSeconds, sourceType, promotions, keywords, summary, ... }], next_cursor }`
+- Search: `GET /discover/feed?q=<offer>&limit=20` keyword-searches the catalog so you can answer "which templates suit my `<offer>`?" (e.g. `?q=weight%20loss%20app`). Ranking is a coarse keyword match over each template's title, viral DNA, and the decompose-derived `promotions` (offer categories the format can sell), `keywords`, and `summary` — read those fields to judge fit and rank the best 3-6 semantically. `GET /api/v1/videos?q=<offer>&limit=20[&mine=true]` runs the same search over source **inspirations**. `promotions`/`keywords`/`summary` are populated by the decompose pass, so a freshly ingested template that has not been decomposed yet won't match on them — decompose it (or fall back to broader/synonym terms) if a search comes up empty.
+
+Each template exposes a public preview:
+
+- `GET /editor/:templateId` — opens the Trackpad Editor for the template (redirects to your fork of it, or to `/login`)
+- `GET /editor/:templateId/composition` — raw composition HTML built from the template's source video
+- `GET /discover/skills/:templateId` — the template's SKILL.md content as JSON (registry templates only)
+
+Directors don't edit templates directly. They fork.
+
+### Add your own source (inspiration)
+
+To bring a new viral video into the catalog as a **private** template you own, ingest its social URL — or upload a video file directly:
+
+- `POST /discover/templates { source_url, tagline?, notes? }` — accepts TikTok / YouTube / Instagram / Twitter-X URLs only. The video download runs async; the response is the inspiration record (`202`, `status: "processing"`). The card finalizes into a private `template_...` the next time `GET /discover/feed` is polled (the feed poll doubles as the completion check).
+- **Upload a file** (three steps, same finalize behavior as the URL flow):
+  1. `POST /discover/templates/upload/presign { file_name, content_type?, size_bytes? }` — video files only (MP4/MOV/WebM), 200 MB cap. Returns `transport: "presigned"` with a `{ upload: { method, url, headers } }` S3 PUT target, or `transport: "server"` (local-storage boxes) pointing at the multipart fallback below. Both include the `storage_key` to finalize with.
+  2. Send the bytes: `PUT` them to the presigned URL, or `POST /discover/templates/upload` (multipart, field `file`) when transport is `server`.
+  3. `POST /discover/templates { upload: { storage_key, file_name }, title?, tagline?, notes? }` — queues an async `video_ingest` job (durable mirror + duration probe + thumbnail) and returns the inspiration record (`202`). Finalizes into a private template exactly like the URL flow. `title` names the /discover card; untitled templates display as their `template_...` id (the file name is only shown while processing, never persisted as the title).
+- `DELETE /discover/templates/:entryId` — remove a private inspiration/template you own (accepts either the `inspiration_...` or minted `template_...` id).
+- `POST /api/v1/inspirations/:inspirationId/decompose { user_prompt? }` — AI-decompose an inspiration's downloaded video into scenes (requires a saved provider key; same 120s source cap as auto-decompose).
+
+devcli: `vidfarm inspiration-add <url|file.mp4>` (a local file path runs the presign→PUT→finalize flow for you), `vidfarm inspiration-rm <id>`, `vidfarm inspiration-decompose <id>`, `vidfarm discover [query]` to browse/search templates, `vidfarm public-raws [query] [--category <key>] [--type <raw_type>] [--bookmark <raw_id>]` to browse/save the public raws catalog, and `vidfarm videos [query] [--mine]` to browse/search source inspirations. On the web, the Discover page's **Add Template** modal accepts either a URL or a file upload.
+
+## Fork a template
+
+```
+POST /api/v1/compositions
+Content-Type: application/json
+
+{ "template_id": "template_<...>", "title": "Optional title" }
+```
+
+`template_id` is required and must start with `template_` (the handler also accepts the same value under the key `source`). Returns the new fork's metadata (201):
+
+```
+{ "fork_id": "<forkId>", "template_id": "template_...", "parent_fork_id": "...", "parent_version": 1, "title": "...", "visibility": "private", "latest_version": 0, "composition_url": "...", "composition_data_url": "...", "versions_url": "...", "role": "owner", "capabilities": { ... } }
+```
+
+Take the returned `fork_id` and open the Trackpad Editor at:
+
+```
+/editor/<templateId>/fork/<forkId>
+```
+
+The editor loads `composition.html` and `composition.json` from the fork's working state.
+
+### New fork from an existing fork (branch this copy, or start fresh)
+
+Once you already hold a fork, spawn ANOTHER editable copy — a branch — with one unified endpoint. This is what the editor's ⋯ menu **New fork** modal and the devcli `clone --from` command both call:
+
+```
+POST /api/v1/compositions/:forkId/fork
+Content-Type: application/json
+
+{ "from": "current" | "default", "parent_version": <optional N>, "title": "Optional" }
+```
+
+- `from: "current"` (default) — branch a new private fork from **this** fork's working copy; all current edits carry over. Optionally pin `parent_version` to clone a saved version instead of the live working state. (The legacy `POST /:forkId/clone` route is an alias for exactly this.)
+- `from: "default"` — start a **fresh** fork from the template's canonical default composition, discarding the current fork's edits (they stay in the fork you branched from).
+
+Returns the same fork metadata shape as `POST /api/v1/compositions` (201). devcli: `vidfarm clone <forkId> [--from current|default] [--version N] [--title …]`. In the in-editor AI chat this is the `editor_action` `fork_composition` action (`fork_from: current|default`).
+
+## Render (publish to MP4)
+
+Rendering publishes the fork's current working state to an MP4 using HyperFrames (cloud Lambda fan-out on the deployed host, or the free in-process renderer on a local `vidfarm serve` box). In the Trackpad Editor this is the **Render** button. The REST route is `/render`; the older `/export` path is kept as a deprecated alias for already-published devcli clients (treat "Render", "publish", `/render`, and `/export` as the same operation). The devcli exposes it as `vidfarm render <forkId> [--wait]`; scripted local dirs can use `vidfarm render <forkId> --dir ./work --wait`.
+
+```
+POST /api/v1/compositions/:forkId/render
+Content-Type: application/json
+
+{ "title": "optional", "version": null, "tracer": "optional-trace-id", "html": "optional HTML to save to working state before rendering" }
+```
+
+Returns 202 with `{ ok, renderId, status, progress, expectedOutputPublicUrl, outputUrl, cost, title, version, ... }` where `expectedOutputPublicUrl` is the deterministic public MP4 URL and `version` is the snapshot version created for this publish. Failure modes: `402` if the fork owner is not on a paid plan, `409` while GhostCut subtitle removal is still `pending` (retry after `/remove-video-captions-poll` reports done/failed), `412` if the provider-key preflight fails. Poll:
+
+```
+GET /api/v1/compositions/:forkId/renders/:renderId
+```
+
+Response includes `{ ok, renderId, status: "RUNNING" | "SUCCEEDED" | "FAILED", progress, framesRendered, totalFrames, cost, expectedOutputPublicUrl, outputUrl, outputS3Uri, errors }` (`progress` is 0..1). On success, `expectedOutputPublicUrl` is the durable public MP4 URL and `outputUrl` remains the completion-time field.
+
+Every publish creates an immutable version snapshot at `versions/<N>/composition.html` and `versions/<N>/composition.json`.
+
+The Web UI **Render** button and devcli render both use this same endpoint. The fast `202` response includes the deterministic `expectedOutputPublicUrl` so a caller can store or pass along the final public S3 URL before the render has completed, then poll by `renderId` until `status` settles.
+
+## Approve a finished post
+
+A render produces a bare MP4 URL. **Approving** wraps that MP4 (plus caption, title, pinned comment, and any carousel slides) into a shareable preview page — the phone-mockup page a human opens to review and copy the post.
+
+```
+POST /api/v1/approved/posts
+Content-Type: application/json
+
+{ "caption": "required", "title": "optional", "pinned_comment": "optional", "tracer": "optional",
+  "media": [ { "url": "https://.../output.mp4", "kind": "video", "role": "primary" } ] }
+```
+
+`caption` is required. `media[]` items take `{ url, kind?: "image"|"video"|..., role?: "primary"|"slide"|... }`. Response (`201`) is the approved post including **`share_url`** — the prod frontend page for previewing/sharing. Surface that URL to the user; it is the headline output of this step.
+
+- `GET /api/v1/approved/posts` — list your approved posts
+- `GET /api/v1/approved/posts/:postId` — read one (returns `share_url`, `download_zip_url`)
+
+devcli: `vidfarm approve --video <mp4-url> --caption "..."` prints the `share_url` as a first-class openable link; `vidfarm posts` lists, `vidfarm post <id>` reads one.
+
+## Schedule a post
+
+Schedule an approved post to a connected destination channel (FlockPoster social account or email) at one ISO timestamp:
+
+```
+POST /api/v1/approved/posts/:postId/schedules
+Content-Type: application/json
+
+{ "destination_type": "flockposter" | "email", "destination_id": "<channel or email>",
+  "scheduled_at": "2026-07-10T14:00:00Z", "timezone": "America/New_York", "additional_notes": "optional" }
+```
+
+Minimum 10-minute lead time. Response (`201`) is the schedule record. Browse existing schedules with `GET /api/v1/approved/posts/:postId/schedules`.
+
+devcli: `vidfarm schedule <postId> --at <iso> --to <destinationId> [--type flockposter|email]`, and `vidfarm schedules <postId>` to browse.
+
+## Version history
+
+```
+GET /api/v1/compositions/:forkId/versions?limit=100
+```
+
+Returns `{ versions: [{ version, reason, message, created_by, created_at, render_job_id, composition_url, composition_data_url }], next_cursor }`. Reason is `publish`, `clone`, `manual`, or `migration`. To snapshot the working state manually, `POST /api/v1/compositions/:forkId/versions { "message": "optional" }`.
+
+To view a past cut without reverting:
+
+```
+GET /api/v1/compositions/:forkId/versions/:version/composition.html
+```
+
+To revert the working state to a version, `PUT` the historical HTML/JSON back to the working keys.
+
+## Sharing and visibility
+
+Every fork has a visibility: `private` (default), `unlisted` (legacy), or `public` (view-only).
+
+```
+PATCH /api/v1/compositions/:forkId/visibility { "visibility": "public" }
+```
+
+Public forks are accessible at `/editor/<templateId>?fork=<forkId>` (view-only for non-owners). Public visibility is the simplest sharing surface.
+
+Fine-grained sharing uses per-user permissions and share links (all body fields are snake_case):
+
+- `POST /api/v1/compositions/:forkId/permissions { grantee_email | grantee_customer_id, role, expires_at? }` — grant a specific registered user access. Exactly one of `grantee_email` or `grantee_customer_id` is required. Roles: `viewer`, `editor`, `publisher`. Returns `{ permission_id, fork_id, grantee_type, grantee_id, role, granted_by, expires_at, created_at }`.
+- `POST /api/v1/compositions/:forkId/share-links { role, expires_at? }` — create a token URL. Anyone with the token gets the role. Returns `{ token, share_url, role, expires_at, ... }`.
+- `GET /api/v1/compositions/shared/:token` — access via share link
+- `DELETE /api/v1/compositions/:forkId/permissions/:permissionId` — revoke a grant
+- `DELETE /api/v1/compositions/:forkId/share-links/:token` — revoke a link
+
+## Cloning a fork
+
+Directors can clone another fork (their own or a shared one) as a new starting point:
+
+```
+POST /api/v1/compositions/:forkId/clone
+Content-Type: application/json
+
+{ "parent_version": "<N or omit for latest>", "title": "optional" }
+```
+
+The `:forkId` in the path is the fork being cloned. The new fork inherits the parent's HTML/JSON at the specified version and creates a version-1 snapshot with `reason: "clone"`.
+
+## Delete a fork
+
+```
+DELETE /api/v1/compositions/:forkId
+```
+
+Soft-delete only. The fork's `deletedAt` is set. Versions and share links are preserved for audit.
+
+## Cost spectrum (free → $10+/video) — default to saving the director money
+
+There is no single price for a video. The **approach** the director picks sets the cost, across a wide spectrum. Always steer toward the cheapest approach that still meets their goal, and make the tradeoff explicit rather than silently choosing an expensive path.
+
+| Approach | Typical cost | How |
+|---|---|---|
+| **Reuse + render locally** | **free** | Fork an already-decomposed template, swap captions / images / video with existing MP4s (from **My Files**, the director's **local computer** — reference files straight off disk with `place --src ./file`, no upload — or a **web search**), and render **locally** via `vidfarm serve` (native in-process HyperFrames render — free and unguarded, no cloud). |
+| **Reuse + cloud render** | **~$0.001 – $0.03** | Same reuse, but render on the cloud renderer (`POST /compositions/:forkId/render`, ~$0.01–$0.10 depending on length/res). Cheap **image** generation/edits fit in this band too. |
+| **AI-generate some scenes** | **~$1** | Replace a few scenes with AI-generated video clips for high specificity/customization (see the "Generate AI media" section). |
+| **Heavy AI generation** | **$10+** | Many/long AI video clips, custom characters, fully bespoke scenes. |
+
+Rules of thumb:
+
+- **Reusing existing footage — or footage the director films themselves — is the cheapest.** AI-generating characters/scenes is the most expensive. Editing captions/images on a reused template is nearly free.
+- **Local render is free; cloud render costs pennies.** If the director is iterating with a coding agent on a `vidfarm serve` box, they can render locally at no charge until they want the durable cloud MP4.
+- **Decompose is a one-time ~$0.10** (smart decompose ≈ provider passthrough + GhostCut ~$0.10/30s) per **new** source video — but the director can **skip it entirely** by forking a template that's **already decomposed** in the catalog.
+- **Image generation is cheap** — use it freely, no need to ask permission.
+- **AI video generation is expensive — ask the director's permission before using it.** Default to reuse/local/image approaches unless they've okayed video gen or told you the budget covers it.
+- **Ask about budget.** During Getting Started (and whenever it's relevant per editor project, or when the director asks about cost), ask roughly what they want to spend per video, and pick the approach band that fits. If they haven't said, assume the cheapest approach that works.
+
+The director always chooses the method that works for them — the point is to surface the tradeoff and default to thrift, not to force the cheapest path.
+
+**How the approach is painted — the three paintbrushes.** The cost of a video is set by *which brush* recreates each scene: (1) **raw clips** remixed from existing footage, (2) **HTML/JS hyperframes** (animated text/images/graphics), (3) **pure AI generation** (image/video/voice/music, most expensive). A technical replication decomposition names, per beat, when/where/what each brush should be, and offers two harnesses — **(A) cheap & efficient** (recaption, bg+fg video memes, hyperframe animation, reuse/greenscreen the asset library, AI only if needed) and **(B) best quality** (AI video by default, AI-image storyboarding first, adversarial grading with a coding agent). Default to (A); reserve (B) for premium/budgeted work. Full methodology in `references/editor-workflows.md` (“The three paintbrushes & two replication harnesses”).
+
+## Billing
+
+Vidfarm charges directly in USD from the caller's wallet. There are no credits.
+
+- **Wallet top-up** — Stripe checkout via the Settings UI, backed by `POST /settings/wallet/funding-link` (browser session; returns `{ checkout_url, client_reference_id }`)
+- **Balance & history** — shown on the Settings page; billing events via `GET /u/:customerId/settings/wallet/events` (browser session). There is no `/api/v1/user/me/wallet` REST endpoint.
+- **Cost anchors** (see the Cost spectrum section above for how these combine per approach):
+  - Local render on a `vidfarm serve` box (in-process HyperFrames): **free**
+  - Cloud render (HyperFrames Lambda fan-out): typically $0.01 – $0.10 per MP4 depending on length/resolution
+  - Image generation / edit: cheap (provider passthrough) — use freely
+  - AI **video** generation: expensive ($1–$10+/video territory) — ask permission first
+  - Auto-decompose smart mode: pass-through of caller's AI provider spend (~$0.10 one-time with GhostCut), or skip it by forking an already-decomposed template
+  - Auto-decompose time-slice: free
+  - GhostCut subtitle removal: ~$0.10 per 30 seconds of source video
+
+Framing:
+
+- Vidfarm is optimized for cost efficiency, not markup
+- Total director cost per video is typically far below alternatives
+- Platform retains a small safety buffer for operational correctness