@mevdragon/vidfarm-devcli 0.3.3 → 0.3.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/SKILL.director.md +86 -43
- package/SKILL.platform.md +20 -11
- package/dist/src/cli.js +8 -3
- package/package.json +7 -6
package/SKILL.director.md
CHANGED
|
@@ -26,7 +26,7 @@ A Vidfarm composition is an **HTML document + JSON metadata**. Each layer (video
|
|
|
26
26
|
|
|
27
27
|
Every director workflow goes through three concepts:
|
|
28
28
|
|
|
29
|
-
1. **Template** — a published composition anyone can fork.
|
|
29
|
+
1. **Template** — a published composition anyone can fork. Template ids start with `template_`; each template opens in the editor at `/editor/:templateId` and has a source video, a decomposed timeline, and metadata (viral DNA, captions, scenes).
|
|
30
30
|
2. **Composition fork** — a private editable copy of a template. Every fork has a `forkId`, an owner, a working state, and a version history. The Trackpad Editor operates on a fork.
|
|
31
31
|
3. **Publish** — render the current fork state to MP4 via a Lambda render job. Each publish snapshots the current working state as an immutable version.
|
|
32
32
|
|
|
@@ -39,42 +39,46 @@ The current API base URLs are:
|
|
|
39
39
|
- staging: `https://staging.vidfarm.cc`
|
|
40
40
|
- production: `https://vidfarm.cc`
|
|
41
41
|
|
|
42
|
-
Every browser action is authenticated by the session cookie set at `/login`.
|
|
42
|
+
Every browser action is authenticated by the session cookie set at `/login`. Headless API-key auth uses the `vidfarm-api-key` header set to the caller's API key (from Settings → API Keys).
|
|
43
|
+
|
|
44
|
+
**Auth contract: API-key auth is the `vidfarm-api-key: <key>` request header and nothing else. `Authorization: Bearer <key>` is NOT supported — the server never reads the `authorization` header (see `requireAuth` in `src/app.ts`).**
|
|
43
45
|
|
|
44
46
|
```
|
|
45
|
-
vidfarm-api-key: vf_key_<
|
|
47
|
+
vidfarm-api-key: vf_key_<id>
|
|
46
48
|
```
|
|
47
49
|
|
|
50
|
+
Scope note (verified against `src/app.ts`): the `vidfarm-api-key` header authenticates **every** authenticated surface — the `requireAuth` routes (`/api/v1/user/*`, `/api/v1/agency/*`, `/api/v1/templates/*`, `/api/v1/primitives/*`, `/api/v1/rate-limit-status`, editor-chat threads) **and** the `/api/v1/compositions/*` + `/api/v1/videos/*` + editor page routes, where an explicit API key takes precedence over any browser session cookie. Cookies remain how the web UI authenticates; headless callers never need one. Reads on composition routes also work without any credential because the `forkId` acts as an unguessable view bearer token, and share-link tokens (header `vidfarm-share-token` or `?share=` query) can grant `viewer`/`editor`/`publisher` roles.
|
|
51
|
+
|
|
48
52
|
## Login flow
|
|
49
53
|
|
|
50
54
|
Directors log in with OTP:
|
|
51
55
|
|
|
52
|
-
1. `POST /api/v1/
|
|
53
|
-
2. `POST /api/v1/
|
|
56
|
+
1. `POST /api/v1/user/request-otp { email }` — sends a code to email; returns `{ ok, delivery }` (plus `dev_code` when delivery is `console` outside production)
|
|
57
|
+
2. `POST /api/v1/user/verify-otp { email, code, name? }` — returns `{ customer, apiKey }` and mints a fresh API key
|
|
54
58
|
|
|
55
|
-
Save the `apiKey` in the client and use it for automation.
|
|
59
|
+
Save the `apiKey` in the client and use it for automation (as the `vidfarm-api-key` header). The browser login form uses the session-cookie variants `POST /login/otp/request` and `POST /login/otp/verify` instead.
|
|
56
60
|
|
|
57
61
|
## Provider keys
|
|
58
62
|
|
|
59
63
|
Vidfarm executes many features (smart auto-decompose, editor chat) with the **caller's own AI provider keys**. Directors save keys once via:
|
|
60
64
|
|
|
61
65
|
- UI: Settings → Provider Keys
|
|
62
|
-
- API: `
|
|
66
|
+
- API: `POST /api/v1/user/me/provider-keys { provider, secret, label? }` — one key per call; `provider` is `"openai" | "gemini" | "openrouter" | "perplexity"`. The key is validated against the provider before it is saved. List saved keys with `GET /api/v1/user/me/provider-keys`.
|
|
63
67
|
|
|
64
68
|
Priority: Gemini → OpenAI → OpenRouter. If a director has no keys, smart decompose falls back to a deterministic time-based split.
|
|
65
69
|
|
|
66
70
|
## Discover templates
|
|
67
71
|
|
|
68
|
-
Templates are listed on the Vidfarm homepage
|
|
72
|
+
Templates are listed on the Vidfarm homepage and `/discover`. Each has a `template_...` id, a preview MP4, source video metadata, and viral DNA.
|
|
69
73
|
|
|
70
|
-
- Browser: browse
|
|
71
|
-
- API: `GET /
|
|
74
|
+
- Browser: browse `/discover`
|
|
75
|
+
- API: `GET /discover/feed` — returns `{ templates: [{ templateId, slugId, title, previewUrl, viralDna, durationSeconds, sourceType, ... }], next_cursor }`
|
|
72
76
|
|
|
73
77
|
Each template exposes a public preview:
|
|
74
78
|
|
|
75
|
-
- `GET /
|
|
76
|
-
- `GET /
|
|
77
|
-
- `GET /
|
|
79
|
+
- `GET /editor/:templateId` — opens the Trackpad Editor for the template (redirects to your fork of it, or to `/login`)
|
|
80
|
+
- `GET /editor/:templateId/composition` — raw composition HTML built from the template's source video
|
|
81
|
+
- `GET /discover/skills/:templateId` — the template's SKILL.md content as JSON (registry templates only)
|
|
78
82
|
|
|
79
83
|
Directors don't edit templates directly. They fork.
|
|
80
84
|
|
|
@@ -84,19 +88,19 @@ Directors don't edit templates directly. They fork.
|
|
|
84
88
|
POST /api/v1/compositions
|
|
85
89
|
Content-Type: application/json
|
|
86
90
|
|
|
87
|
-
{ "
|
|
91
|
+
{ "template_id": "template_<...>", "title": "Optional title" }
|
|
88
92
|
```
|
|
89
93
|
|
|
90
|
-
Returns the new fork's metadata:
|
|
94
|
+
`template_id` is required and must start with `template_` (the handler also accepts the same value under the key `source`). Returns the new fork's metadata (201):
|
|
91
95
|
|
|
92
96
|
```
|
|
93
|
-
{ "
|
|
97
|
+
{ "fork_id": "<forkId>", "template_id": "template_...", "parent_fork_id": "...", "parent_version": 1, "title": "...", "visibility": "private", "latest_version": 0, "composition_url": "...", "composition_data_url": "...", "versions_url": "...", "role": "owner", "capabilities": { ... } }
|
|
94
98
|
```
|
|
95
99
|
|
|
96
|
-
Take the returned `
|
|
100
|
+
Take the returned `fork_id` and open the Trackpad Editor at:
|
|
97
101
|
|
|
98
102
|
```
|
|
99
|
-
/editor
|
|
103
|
+
/editor/<templateId>?fork=<forkId>
|
|
100
104
|
```
|
|
101
105
|
|
|
102
106
|
The editor loads `composition.html` and `composition.json` from the fork's working state.
|
|
@@ -139,7 +143,7 @@ Content-Type: application/json
|
|
|
139
143
|
|
|
140
144
|
Two modes:
|
|
141
145
|
|
|
142
|
-
- **`smart`** — uses the caller's Gemini/OpenAI/OpenRouter key. Samples frames, extracts scenes with labels and viral notes, extracts on-screen captions with positions, and detects viral DNA (hook, retention, payoff). Takes 30-60 seconds. Auto-fires GhostCut for subtitle removal in the background.
|
|
146
|
+
- **`smart`** — uses the caller's Gemini/OpenAI/OpenRouter key. Samples frames, extracts scenes with labels, literal visual **descriptions**, and viral notes, extracts on-screen captions with positions, **transcribes the audio track verbatim** (timestamped segments; Gemini first, OpenAI Whisper fallback; skipped when the video has no audio), and detects viral DNA (hook, retention, payoff). Takes 30-60 seconds. Auto-fires GhostCut for subtitle removal in the background.
|
|
143
147
|
- **`time-slice`** — deterministic equal-duration split. Instant. No AI provider needed.
|
|
144
148
|
|
|
145
149
|
**Source length cap.** Vidfarm caps auto-decompose (and all inspiration ingest) at **120 seconds** of source video. If the source is longer, the API responds with `400 { ok: false, code: "source_too_long", duration_seconds, max_duration_seconds }` and no scenes are written. Trim the source before forking / ingesting.
|
|
@@ -154,24 +158,59 @@ If `ghostcut_pending: true`, poll `POST /api/v1/compositions/:forkId/ghostcut-po
|
|
|
154
158
|
|
|
155
159
|
Read-only state (does **not** advance the job or bill): `GET /api/v1/compositions/:forkId/ghostcut`. Returns `{ status, task_id, original_source_url, mirrored_url, submitted_at_ms, completed_at_ms, failure_reason }`. Use this when you want to know both video URLs without triggering another poll — the AI copilot uses it to pick between the raw upload and the caption-free mirror before feeding a video into a primitive route (image extract, dedupe, trim, ai video edit, etc.). Prefer `mirrored_url` when `status === "done"` and the downstream call should be caption-free; prefer `original_source_url` when the user explicitly wants the raw upload or when `status !== "done"`.
|
|
156
160
|
|
|
161
|
+
## Video context (transcript + scene descriptions)
|
|
162
|
+
|
|
163
|
+
`GET /api/v1/compositions/:forkId/video-context.json` — read-only, non-billing. Returns everything the smart decompose learned about the source video, so an agent can ground its work in what the video actually **says** and **shows**:
|
|
164
|
+
|
|
165
|
+
```
|
|
166
|
+
{
|
|
167
|
+
"ok": true,
|
|
168
|
+
"status": "ready" | "none",
|
|
169
|
+
"summary": "...",
|
|
170
|
+
"duration_seconds": 12.4,
|
|
171
|
+
"transcript": {
|
|
172
|
+
"text": "full verbatim transcript",
|
|
173
|
+
"language": "en",
|
|
174
|
+
"provider": "gemini",
|
|
175
|
+
"segments": [{ "start": 0.0, "end": 2.4, "text": "..." }]
|
|
176
|
+
},
|
|
177
|
+
"scenes": [{
|
|
178
|
+
"slug": "hook_reveal",
|
|
179
|
+
"start": 0, "duration": 2.5,
|
|
180
|
+
"label": "Hook",
|
|
181
|
+
"description": "Literal visual description of what's on screen in this scene",
|
|
182
|
+
"viral_note": "...",
|
|
183
|
+
"transcript_excerpt": "what is spoken during this scene"
|
|
184
|
+
}],
|
|
185
|
+
"viral_dna": { ... }
|
|
186
|
+
}
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
`status: "none"` means the fork was never smart-decomposed — run `POST /auto-decompose` with `mode: "smart"` first. `transcript` is `null` when the source has no audio track or transcription failed.
|
|
190
|
+
|
|
191
|
+
Use it whenever you need to know what the video says or shows: writing/translating captions, matching hooks or dubs to spoken audio, or planning scene-level edits.
|
|
192
|
+
|
|
193
|
+
- **Editor chat (frontend AI)** exposes this as the optional `video_context` tool — the copilot calls it on demand.
|
|
194
|
+
- **Desktop agents (Claude Code / Codex)**: `vidfarm edit <template_id>` syncs the same payload to `video-context.json` on disk next to `composition.html` (use `--refetch` to refresh it after a new decompose), or fetch the route directly.
|
|
195
|
+
|
|
157
196
|
## Publish
|
|
158
197
|
|
|
159
|
-
Publishing renders the fork's current working state to an MP4 using shared Remotion Lambda.
|
|
198
|
+
Publishing renders the fork's current working state to an MP4 using shared Remotion Lambda. The route is `/export`:
|
|
160
199
|
|
|
161
200
|
```
|
|
162
|
-
POST /api/v1/compositions/:forkId/
|
|
201
|
+
POST /api/v1/compositions/:forkId/export
|
|
163
202
|
Content-Type: application/json
|
|
164
203
|
|
|
165
|
-
{ "title": "optional", "version": null, "tracer": "optional-
|
|
204
|
+
{ "title": "optional", "version": null, "tracer": "optional-trace-id", "html": "optional HTML to save to working state before rendering" }
|
|
166
205
|
```
|
|
167
206
|
|
|
168
|
-
Returns `{
|
|
207
|
+
Returns 202 with `{ ok, renderId, status, progress, outputUrl, cost, title, version, ... }` where `version` is the snapshot version created for this publish. Failure modes: `402` if the fork owner is not on a paid plan, `409` while GhostCut subtitle removal is still `pending` (retry after `/ghostcut-poll` reports done/failed), `412` if the provider-key preflight fails. Poll:
|
|
169
208
|
|
|
170
209
|
```
|
|
171
210
|
GET /api/v1/compositions/:forkId/renders/:renderId
|
|
172
211
|
```
|
|
173
212
|
|
|
174
|
-
Response includes `{
|
|
213
|
+
Response includes `{ ok, renderId, status: "RUNNING" | "SUCCEEDED" | "FAILED", progress, framesRendered, totalFrames, cost, outputUrl, outputS3Uri, errors }` (`progress` is 0..1). On success, `outputUrl` is the durable MP4 URL.
|
|
175
214
|
|
|
176
215
|
Every publish creates an immutable version snapshot at `versions/<N>/composition.html` and `versions/<N>/composition.json`.
|
|
177
216
|
|
|
@@ -181,7 +220,7 @@ Every publish creates an immutable version snapshot at `versions/<N>/composition
|
|
|
181
220
|
GET /api/v1/compositions/:forkId/versions?limit=100
|
|
182
221
|
```
|
|
183
222
|
|
|
184
|
-
Returns
|
|
223
|
+
Returns `{ versions: [{ version, reason, message, created_by, created_at, render_job_id, composition_url, composition_data_url }], next_cursor }`. Reason is `publish`, `clone`, `manual`, or `migration`. To snapshot the working state manually, `POST /api/v1/compositions/:forkId/versions { "message": "optional" }`.
|
|
185
224
|
|
|
186
225
|
To view a past cut without reverting:
|
|
187
226
|
|
|
@@ -199,12 +238,12 @@ Every fork has a visibility: `private` (default), `unlisted` (legacy), or `publi
|
|
|
199
238
|
PATCH /api/v1/compositions/:forkId/visibility { "visibility": "public" }
|
|
200
239
|
```
|
|
201
240
|
|
|
202
|
-
Public forks are accessible at `/editor
|
|
241
|
+
Public forks are accessible at `/editor/<templateId>?fork=<forkId>` (view-only for non-owners). Public visibility is the simplest sharing surface.
|
|
203
242
|
|
|
204
|
-
Fine-grained sharing uses per-user permissions and share links:
|
|
243
|
+
Fine-grained sharing uses per-user permissions and share links (all body fields are snake_case):
|
|
205
244
|
|
|
206
|
-
- `POST /api/v1/compositions/:forkId/permissions {
|
|
207
|
-
- `POST /api/v1/compositions/:forkId/share-links { role,
|
|
245
|
+
- `POST /api/v1/compositions/:forkId/permissions { grantee_email | grantee_customer_id, role, expires_at? }` — grant a specific registered user access. Exactly one of `grantee_email` or `grantee_customer_id` is required. Roles: `viewer`, `editor`, `publisher`. Returns `{ permission_id, fork_id, grantee_type, grantee_id, role, granted_by, expires_at, created_at }`.
|
|
246
|
+
- `POST /api/v1/compositions/:forkId/share-links { role, expires_at? }` — create a token URL. Anyone with the token gets the role. Returns `{ token, share_url, role, expires_at, ... }`.
|
|
208
247
|
- `GET /api/v1/compositions/shared/:token` — access via share link
|
|
209
248
|
- `DELETE /api/v1/compositions/:forkId/permissions/:permissionId` — revoke a grant
|
|
210
249
|
- `DELETE /api/v1/compositions/:forkId/share-links/:token` — revoke a link
|
|
@@ -214,10 +253,13 @@ Fine-grained sharing uses per-user permissions and share links:
|
|
|
214
253
|
Directors can clone another fork (their own or a shared one) as a new starting point:
|
|
215
254
|
|
|
216
255
|
```
|
|
217
|
-
POST /api/v1/compositions
|
|
256
|
+
POST /api/v1/compositions/:forkId/clone
|
|
257
|
+
Content-Type: application/json
|
|
258
|
+
|
|
259
|
+
{ "parent_version": "<N or omit for latest>", "title": "optional" }
|
|
218
260
|
```
|
|
219
261
|
|
|
220
|
-
The new fork inherits the parent's HTML/JSON at the specified version and creates a version-1 snapshot with `reason: "clone"`.
|
|
262
|
+
The `:forkId` in the path is the fork being cloned. The new fork inherits the parent's HTML/JSON at the specified version and creates a version-1 snapshot with `reason: "clone"`.
|
|
221
263
|
|
|
222
264
|
## Delete a fork
|
|
223
265
|
|
|
@@ -231,8 +273,8 @@ Soft-delete only. The fork's `deletedAt` is set. Versions and share links are pr
|
|
|
231
273
|
|
|
232
274
|
Vidfarm charges directly in USD from the caller's wallet. There are no credits.
|
|
233
275
|
|
|
234
|
-
- **Wallet top-up** — Stripe checkout via `POST /
|
|
235
|
-
- **Balance** — `GET /api/v1/user/me/wallet`
|
|
276
|
+
- **Wallet top-up** — Stripe checkout via the Settings UI, backed by `POST /settings/wallet/funding-link` (browser session; returns `{ checkout_url, client_reference_id }`)
|
|
277
|
+
- **Balance & history** — shown on the Settings page; billing events via `GET /u/:customerId/settings/wallet/events` (browser session). There is no `/api/v1/user/me/wallet` REST endpoint.
|
|
236
278
|
- **Cost anchors**:
|
|
237
279
|
- Publish (Remotion Lambda render): typically $0.01 – $0.10 per MP4 depending on length/resolution
|
|
238
280
|
- Auto-decompose smart mode: pass-through of caller's AI provider spend
|
|
@@ -249,14 +291,14 @@ Framing:
|
|
|
249
291
|
|
|
250
292
|
For agents that operate Vidfarm headlessly, the typical loop is:
|
|
251
293
|
|
|
252
|
-
1. `POST /api/v1/compositions {
|
|
294
|
+
1. `POST /api/v1/compositions { template_id }` → fork
|
|
253
295
|
2. `PUT /api/v1/compositions/:forkId/composition.html` → apply edits
|
|
254
296
|
3. `PATCH /api/v1/compositions/:forkId/composition.json` → update metadata
|
|
255
|
-
4. `POST /api/v1/compositions/:forkId/
|
|
297
|
+
4. `POST /api/v1/compositions/:forkId/export { tracer }` → render
|
|
256
298
|
5. Poll `GET /api/v1/compositions/:forkId/renders/:renderId` until `SUCCEEDED`
|
|
257
|
-
6. Use `
|
|
299
|
+
6. Use `outputUrl` in the next stage
|
|
258
300
|
|
|
259
|
-
|
|
301
|
+
Send a stable `tracer` on export so retries are traceable and filterable in job history — but note that submission is **not idempotent**: every POST creates a new job (and a new charge) even with the same tracer. Do not blind-retry expensive submissions; check the render status first.
|
|
260
302
|
|
|
261
303
|
## Local dev loop with `vidfarm-devcli`
|
|
262
304
|
|
|
@@ -284,7 +326,7 @@ The dev source is persisted in `localStorage.VIDFARM_DEV_SOURCE`, so you don't n
|
|
|
284
326
|
|
|
285
327
|
- Do **not** manipulate composition HTML by string concatenation. Always parse, edit, and re-serialize the DOM. The editor and runtime rely on `data-start`, `data-duration`, `data-track-index`, `data-hf-id`, `data-composition-id` attributes being well-formed.
|
|
286
328
|
- Do **not** save AI provider keys in the composition HTML or JSON. They belong in the caller's provider-keys record.
|
|
287
|
-
- Do **not** call Remotion directly. Publishing goes through
|
|
329
|
+
- Do **not** call Remotion directly. Publishing goes through `POST /api/v1/compositions/:forkId/export` so billing, retries, cost caps, and Lambda quotas are enforced.
|
|
288
330
|
- Do **not** re-invent scene manipulation as REST commands (`AddScene`, `RemoveScene`, `UpdateSceneField`). The Trackpad Editor is the surface for that. The pattern is: mutate the composition HTML DOM, PUT it back.
|
|
289
331
|
|
|
290
332
|
## Alpha posture
|
|
@@ -292,7 +334,7 @@ The dev source is persisted in `localStorage.VIDFARM_DEV_SOURCE`, so you don't n
|
|
|
292
334
|
Vidfarm is in active development. Endpoints, response shapes, and the editor UI can change. When something behaves unexpectedly:
|
|
293
335
|
|
|
294
336
|
- prefer `GET /api/v1/compositions/:forkId` over cached state
|
|
295
|
-
- treat `
|
|
337
|
+
- treat `latest_version` as the source of truth for what the editor loaded
|
|
296
338
|
- if a publish fails, check `renders/:renderId` for the error phase and stderr before retrying
|
|
297
339
|
|
|
298
340
|
## Default assistance pattern
|
|
@@ -300,7 +342,7 @@ Vidfarm is in active development. Endpoints, response shapes, and the editor UI
|
|
|
300
342
|
When a director asks "make me a video", the default sequence is:
|
|
301
343
|
|
|
302
344
|
1. Ask what source material and what template style they want
|
|
303
|
-
2. Discover a matching template with `GET /
|
|
345
|
+
2. Discover a matching template with `GET /discover/feed`
|
|
304
346
|
3. Fork it
|
|
305
347
|
4. If they want an entirely new source video, guide them to `auto-decompose` first
|
|
306
348
|
5. Open the Trackpad Editor and let them make edits
|
|
@@ -321,7 +363,7 @@ Example:
|
|
|
321
363
|
|
|
322
364
|
```bash
|
|
323
365
|
curl -X POST "$VIDFARM_BASE/api/v1/primitives/images/remove-background" \
|
|
324
|
-
-H "
|
|
366
|
+
-H "vidfarm-api-key: $VIDFARM_API_KEY" \
|
|
325
367
|
-H "content-type: application/json" \
|
|
326
368
|
-d '{"source_image_url": "https://cdn.example.com/photo.jpg"}'
|
|
327
369
|
```
|
|
@@ -332,6 +374,7 @@ Apply subtle camouflage transforms (zoom, tilt, rotate, saturation, playback spe
|
|
|
332
374
|
|
|
333
375
|
- `POST /api/v1/primitives/media/dedupe`
|
|
334
376
|
- Body: `{ "tracer": "...", "payload": { ...fields... }, "webhook_url"?: "..." }`
|
|
377
|
+
- Note: webhook delivery is not yet active — `webhook_url` is accepted and persisted on the job but never fired. Poll the job endpoints (`GET /api/v1/primitives/jobs/:jobId`) for completion.
|
|
335
378
|
- Response: standard primitive job. Poll to completion, then read `primary_file_url` (also `video.file_url` for MP4 or `image.file_url` for stills)
|
|
336
379
|
- Billing: metered as a Remotion render (via `remotion_lambda` on prod, local fallback in dev)
|
|
337
380
|
|
|
@@ -362,7 +405,7 @@ Video example (camouflage a reused clip):
|
|
|
362
405
|
|
|
363
406
|
```bash
|
|
364
407
|
curl -X POST "$VIDFARM_BASE/api/v1/primitives/media/dedupe" \
|
|
365
|
-
-H "
|
|
408
|
+
-H "vidfarm-api-key: $VIDFARM_API_KEY" \
|
|
366
409
|
-H "content-type: application/json" \
|
|
367
410
|
-d '{
|
|
368
411
|
"tracer": "dedupe-2026-07-01-abc",
|
|
@@ -388,7 +431,7 @@ Image example:
|
|
|
388
431
|
|
|
389
432
|
```bash
|
|
390
433
|
curl -X POST "$VIDFARM_BASE/api/v1/primitives/media/dedupe" \
|
|
391
|
-
-H "
|
|
434
|
+
-H "vidfarm-api-key: $VIDFARM_API_KEY" \
|
|
392
435
|
-H "content-type: application/json" \
|
|
393
436
|
-d '{
|
|
394
437
|
"tracer": "dedupe-still-01",
|
package/SKILL.platform.md
CHANGED
|
@@ -66,7 +66,8 @@ compositions/forks/<forkId>/
|
|
|
66
66
|
│ ├── composition.html # current editor state
|
|
67
67
|
│ ├── composition.json # current metadata (scenes, captions, viral DNA)
|
|
68
68
|
│ ├── manifest.json # fork identity + genealogy pointer
|
|
69
|
-
│
|
|
69
|
+
│ ├── ghostcut.json # GhostCut subtitle-removal task state (optional)
|
|
70
|
+
│ └── smart-decompose.json # raw smart-decompose snapshot: scenes (with visual descriptions), captions, viral DNA, audio transcript (optional)
|
|
70
71
|
└── versions/<version>/
|
|
71
72
|
├── composition.html
|
|
72
73
|
├── composition.json
|
|
@@ -183,13 +184,13 @@ The editor is a single monolithic React component (~9k lines). It builds cleanly
|
|
|
183
184
|
|
|
184
185
|
Publish flow:
|
|
185
186
|
|
|
186
|
-
1. Client `POST /api/v1/compositions/:forkId/publish
|
|
187
|
+
1. Client `POST /api/v1/compositions/:forkId/export` (the route is named `export`; the concept is "publish")
|
|
187
188
|
2. API stamps `data-composition-id="<forkId>@v<N>"` into the composition HTML, PUTs the stamped HTML to working, and reads composition.json
|
|
188
189
|
3. API resolves the caller's provider keys (preflight check for required models)
|
|
189
190
|
4. API creates a `job` record and invokes the `hyperframes_render` primitive Lambda
|
|
190
191
|
5. The primitive Lambda uploads the composition as a Remotion site, calls `renderMediaOnLambda` with the shared Remotion render Lambda ARN, waits for the MP4 to land in S3
|
|
191
192
|
6. On success, the primitive returns `{ primary_file_url: "s3://.../output.mp4" }`, the API creates a `ForkVersionRecord` with `reason: "publish"`, writes the snapshot to `versions/<N>/`, and updates the fork's `latestVersion` and `currentRenderJobId`
|
|
192
|
-
7. Client polls `GET /api/v1/compositions/:forkId/renders/:renderId` for `{
|
|
193
|
+
7. Client polls `GET /api/v1/compositions/:forkId/renders/:renderId` for `{ status: "RUNNING" | "SUCCEEDED" | "FAILED", progress, outputUrl, cost }` (see `adaptJobToPublishStatus` in `src/app.ts`)
|
|
193
194
|
|
|
194
195
|
The Remotion Lambda is **shared across all templates and forks** — it's an infrastructure primitive, not a per-template deployment. Its ARN is passed to the Vidfarm stack as an env var.
|
|
195
196
|
|
|
@@ -198,9 +199,17 @@ The Remotion Lambda is **shared across all templates and forks** — it's an inf
|
|
|
198
199
|
`POST /api/v1/compositions/:forkId/auto-decompose` splits a raw source video into a Trackpad-editable composition. Two modes:
|
|
199
200
|
|
|
200
201
|
- **`time-slice`** (deterministic) — `src/hyperframes/composition.ts`'s `decomposeUndecomposedComposition()` splits the source duration into N equal-length scenes, wraps each in a `<div data-vf-timeline-proxy="video">` layer with `data-start`/`data-duration`/`data-playback-start`. Instant. No AI.
|
|
201
|
-
- **`smart`** (AI) — `src/services/hyperframes.ts`'s `smartDecomposeVideo()` downloads the source, samples frames, calls Gemini/OpenAI/OpenRouter vision (in priority order) using the caller's saved keys, extracts scenes with labels and viral notes, extracts on-screen captions with position, and generates viral DNA. Then it also submits an async **GhostCut** subtitle-removal task if the source has hardcoded subs. Takes 30-60s + async ghostcut poll.
|
|
202
|
+
- **`smart`** (AI) — `src/services/hyperframes.ts`'s `smartDecomposeVideo()` downloads the source, samples frames, calls Gemini/OpenAI/OpenRouter vision (in priority order) using the caller's saved keys, extracts scenes with labels, literal visual **descriptions**, and viral notes, extracts on-screen captions with position, and generates viral DNA. It also demuxes the audio track to WAV and **transcribes it verbatim** with timestamped segments (`transcribeAudioWithFallback()`: Gemini inline-audio first, OpenAI Whisper fallback; non-fatal — a silent video or transcription failure yields `transcript: null`). Then it also submits an async **GhostCut** subtitle-removal task if the source has hardcoded subs. Takes 30-60s + async ghostcut poll.
|
|
202
203
|
|
|
203
|
-
Both modes update the composition.html and composition.json in place, and create a fork version snapshot with `reason: "manual"`.
|
|
204
|
+
Both modes update the composition.html and composition.json in place, and create a fork version snapshot with `reason: "manual"`. Smart mode additionally persists the raw result — scenes with descriptions, captions, viral DNA, and the transcript — to `working/smart-decompose.json`.
|
|
205
|
+
|
|
206
|
+
### Video context (transcript + scene descriptions)
|
|
207
|
+
|
|
208
|
+
`GET /api/v1/compositions/:forkId/video-context.json` assembles a read-only, non-billing view over `smart-decompose.json`: the full verbatim transcript (`{ text, language, segments[] }`), every scene with its literal visual `description` and a computed `transcript_excerpt` (transcript segments overlapping the scene's time window), plus summary and viral DNA. Returns `{ status: "none" }` for forks that were never smart-decomposed. Same surface, three consumers:
|
|
209
|
+
|
|
210
|
+
1. **Editor chat** — the optional `video_context` tool (registered alongside `http_request` in both `src/app.ts` and `infra/lambda/editor-chat.ts`) fetches it on demand; the shared system prompt in `src/editor-chat.ts` tells the model when to reach for it (what does the video say/show, caption/dub matching, scene-level edit planning).
|
|
211
|
+
2. **`http_request`** — the route lives under `COMPOSITIONS_PREFIX`, so chat can also GET it directly (e.g. for a different fork).
|
|
212
|
+
3. **Desktop agents (Claude Code / Codex)** — the devcli (`src/cli.ts`) syncs it to `video-context.json` on disk next to `composition.html` during `vidfarm edit`; `--refetch` refreshes it after a new decompose.
|
|
204
213
|
|
|
205
214
|
**Source-duration guardrail.** `probeMediaAsset()` in `src/services/media-processing.ts` enforces a hard cap of `MAX_PROBE_DURATION_SECONDS` (120s). If the probed source exceeds it, the probe throws `MediaDurationExceededError` and the route responds with `400 { code: "source_too_long", duration_seconds, max_duration_seconds }`. The same cap applies to inspiration ingest (`ensureInspirationReadyThenFork`) — over-long submissions land as `InspirationRecord { status: "failed", errorMessage }` without ever creating a Template or fork. This is baked into the probe (not the route) so every consumer — auto-decompose, ingest, `video_probe` / `audio_probe` primitives — is protected from repeated probes of long videos being used to burn Lambda / ffmpeg / vision-API budget.
|
|
206
215
|
|
|
@@ -258,7 +267,7 @@ The billing Lambda has its own custom domain because Stripe webhooks require a s
|
|
|
258
267
|
|
|
259
268
|
There are two kinds of things directors can fork:
|
|
260
269
|
|
|
261
|
-
1. **Draft templates**
|
|
270
|
+
1. **Draft templates** — pre-decomposed compositions checked into the repo under `templates/vidfarm_template_XXXX/`. Each has a `composition.html`, `composition.json`, `template.config.json`, a `SKILL.md`, and a `src/` if it needs custom Remotion. Registered templates open in the browser at `/editor/:templateId` (there is no `/hyperframes/:slug` route).
|
|
262
271
|
2. **Submitted videos** — raw uploads from directors, decomposed on demand via auto-decompose.
|
|
263
272
|
|
|
264
273
|
Templates are static. Compositions/forks are the mutable working copies.
|
|
@@ -291,10 +300,10 @@ CDK stacks are under `infra/cdk/`:
|
|
|
291
300
|
Deploy commands are in `package.json`:
|
|
292
301
|
|
|
293
302
|
- `npm run cdk:synth:staging-serverless`
|
|
294
|
-
- `npm run cdk:deploy:staging-serverless`
|
|
295
|
-
- `npm run cdk:deploy:prod`
|
|
303
|
+
- `npm run cdk:deploy:staging-serverless` (and `npm run cdk:deploy:staging-serverless:hotswap` for Lambda-code-only iteration)
|
|
304
|
+
- `npm run cdk:deploy:prod-serverless` (and `npm run cdk:deploy:prod-serverless:hotswap`)
|
|
296
305
|
|
|
297
|
-
Never call bare `cdk deploy`. Always go through the npm scripts
|
|
306
|
+
Never call bare `cdk deploy`. Always go through the npm scripts — the `cdk:*` scripts load `.env.<stage>` via `dotenv-cli` (`dotenv -e .env.<stage> -- npx aws-cdk ...`) and pick the right stack app.
|
|
298
307
|
|
|
299
308
|
Deploy order: **staging first**, then production. Never skip staging.
|
|
300
309
|
|
|
@@ -319,7 +328,7 @@ Top-level:
|
|
|
319
328
|
- `templates/` — draft template sources
|
|
320
329
|
- `auto-create-hyperframe-templates/` — extractor prompts / schemas for new template creation
|
|
321
330
|
- `scripts/` — one-shot ops scripts and migration scripts
|
|
322
|
-
- `data/` — local dev
|
|
331
|
+
- `data/` — local dev storage (gitignored). Also holds leftover `*.sqlite` / `vidfarm.db` files from the retired SQLite/EC2 era — the runtime no longer reads SQLite anywhere; DynamoDB is the only database.
|
|
323
332
|
- `.env.staging`, `.env.production` — stage config (gitignored)
|
|
324
333
|
|
|
325
334
|
Legacy but still active:
|
|
@@ -337,7 +346,7 @@ Legacy but still active:
|
|
|
337
346
|
|
|
338
347
|
## Where to look for things
|
|
339
348
|
|
|
340
|
-
- Composition endpoints — `src/app.ts`
|
|
349
|
+
- Composition endpoints — `src/app.ts`: grep for `COMPOSITIONS_PREFIX` (the block runs from `app.post(COMPOSITIONS_PREFIX` fork creation through the `versions` routes; line numbers drift, always grep the symbol)
|
|
341
350
|
- Fork CRUD & permissions — `src/services/serverless-records.ts`, `src/services/fork-access.ts`
|
|
342
351
|
- Storage keys + public-read prefix list — `src/services/storage.ts`
|
|
343
352
|
- Fork manifest.json helper — `src/services/fork-manifest.ts`
|
package/dist/src/cli.js
CHANGED
|
@@ -127,7 +127,10 @@ async function resolveForkId(input) {
|
|
|
127
127
|
return null;
|
|
128
128
|
}
|
|
129
129
|
async function fetchCompositionFiles(input) {
|
|
130
|
-
|
|
130
|
+
// video-context.json carries the source video's audio transcript and
|
|
131
|
+
// per-scene visual descriptions so desktop agents (Claude Code, Codex)
|
|
132
|
+
// can ground edits in what the video actually says and shows.
|
|
133
|
+
const files = ["composition.html", "composition.json", "manifest.json", "video-context.json"];
|
|
131
134
|
for (const filename of files) {
|
|
132
135
|
const target = path.join(input.dir, filename);
|
|
133
136
|
if (existsSync(target) && !input.refetch) {
|
|
@@ -147,10 +150,12 @@ async function fetchCompositionFiles(input) {
|
|
|
147
150
|
}
|
|
148
151
|
function buildAuthHeaders(input) {
|
|
149
152
|
const headers = { accept: "text/html,application/json" };
|
|
153
|
+
// The server reads exactly these header names (see requireAuth and
|
|
154
|
+
// readShareToken in src/app.ts) — Bearer/authorization is not supported.
|
|
150
155
|
if (input.apiKey)
|
|
151
|
-
headers
|
|
156
|
+
headers["vidfarm-api-key"] = input.apiKey;
|
|
152
157
|
if (input.shareToken)
|
|
153
|
-
headers["
|
|
158
|
+
headers["vidfarm-share-token"] = input.shareToken;
|
|
154
159
|
return headers;
|
|
155
160
|
}
|
|
156
161
|
function printEditorBanner(input) {
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mevdragon/vidfarm-devcli",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.4",
|
|
4
4
|
"description": "Local bridge for the Vidfarm Trackpad Editor. Point it at a template id, edit composition.html on disk (Claude Code, Codex, etc.), preview live in the browser at https://vidfarm.cc/editor/.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -39,11 +39,11 @@
|
|
|
39
39
|
"build": "npm run build:frontend && npm run build:hyperframes-editor && node -e \"require('node:fs').rmSync('dist', { recursive: true, force: true })\" && tsc -p tsconfig.json",
|
|
40
40
|
"start": "node --import ./dist/src/instrument.js --enable-source-maps dist/src/index.js",
|
|
41
41
|
"check": "tsc -p tsconfig.json --noEmit",
|
|
42
|
-
"cdk:
|
|
43
|
-
"cdk:
|
|
44
|
-
"cdk:deploy:
|
|
45
|
-
"cdk:
|
|
46
|
-
"cdk:deploy:
|
|
42
|
+
"cdk:deploy:prod-serverless": "npm run build && dotenv -e .env.production -- npx aws-cdk deploy --app 'node dist/infra/cdk/bin/vidfarm-prod.js'",
|
|
43
|
+
"cdk:synth:staging-serverless": "npm run build && dotenv -e .env.staging -- npx aws-cdk synth --app 'node dist/infra/cdk/bin/vidfarm-serverless-staging.js'",
|
|
44
|
+
"cdk:deploy:staging-serverless": "npm run build && dotenv -e .env.staging -- npx aws-cdk deploy --app 'node dist/infra/cdk/bin/vidfarm-serverless-staging.js'",
|
|
45
|
+
"cdk:deploy:staging-serverless:hotswap": "npm run build && dotenv -e .env.staging -- npx aws-cdk deploy --app 'node dist/infra/cdk/bin/vidfarm-serverless-staging.js' --hotswap-fallback --require-approval never",
|
|
46
|
+
"cdk:deploy:prod-serverless:hotswap": "npm run build && dotenv -e .env.production -- npx aws-cdk deploy --app 'node dist/infra/cdk/bin/vidfarm-prod.js' --hotswap-fallback --require-approval never",
|
|
47
47
|
"prepack": "npm run build"
|
|
48
48
|
},
|
|
49
49
|
"dependencies": {
|
|
@@ -92,6 +92,7 @@
|
|
|
92
92
|
"@types/react-dom": "^18.3.7",
|
|
93
93
|
"aws-cdk-lib": "^2.196.0",
|
|
94
94
|
"constructs": "^10.4.2",
|
|
95
|
+
"dotenv-cli": "^11.0.0",
|
|
95
96
|
"tsx": "^4.19.4",
|
|
96
97
|
"typescript": "^5.8.3",
|
|
97
98
|
"zustand": "^5.0.13"
|