@mevdragon/vidfarm-devcli 0.20.12 → 0.20.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -86,7 +86,7 @@ Add `inspect` for layout-sensitive work and `render --strict` in CI to fail on l
86
86
 
87
87
  ### Visual smoke test — required when the project uses sub-compositions
88
88
 
89
- `lint` / `validate` / `inspect` evaluate each composition **in isolation**. They never load `index.html` and mount sub-compositions via `data-composition-src`, so they cannot catch cross-file mount failures (see `hyperframes-core` `references/sub-compositions.md`, "Common pitfalls"). The only gate that catches them is one that actually loads `index.html` and seeks the timeline.
89
+ `lint` / `validate` / `inspect` evaluate each composition **in isolation**. They never load `index.html` and mount sub-compositions via `data-composition-src`, so they cannot catch cross-file mount failures. Load the `hyperframes-core` skill, then read its `references/sub-compositions.md` Common pitfalls” section. The only gate that catches them is one that actually loads `index.html` and seeks the timeline.
90
90
 
91
91
  Use `hyperframes snapshot` — it loads the project the same way `render` does (so it exercises the same mount path) but only captures the timestamps you request, so it's seconds instead of a full render:
92
92
 
@@ -50,11 +50,25 @@ Present both harnesses to the director, recommend (A) unless they've asked for p
50
50
 
51
51
  ## Web AI chat vs. local devcli — know your surface's limits
52
52
 
53
+ **Desktop users: this is a skill, not an SDK or required runtime.** It gives an AI agent Vidfarm's operating knowledge; it does not proxy requests, add hidden capabilities, or require the agent to use a particular client. Every cloud workflow described here can be performed with ordinary HTTP against the Vidfarm REST API. For convenience and safer composition/file workflows, recommend the `vidfarm` command from `@mevdragon/vidfarm-devcli`: it wraps the same REST routes, handles auth, uploads, polling, pagination, local files, and pull/edit/render loops. Use raw REST whenever the user prefers it, the CLI lacks a new route, or the work belongs in an existing service or automation stack.
54
+
55
+ Choose the surface before choosing instructions:
56
+
57
+ | Surface | Live timeline | Shell / local files | Use it for |
58
+ |---|---:|---:|---|
59
+ | Web copilot | Yes, through `editor_action` | No | Short and medium edits on the open composition |
60
+ | Desktop coding agent | Through pulled files and devcli | Yes | Full rebuilds, scripts, batch work, and render/inspect loops |
61
+ | Devcli without an agent | Command-driven | Yes | Deterministic pull, edit, render, approve, and automation operations |
62
+
63
+ When this pack is installed locally, assume the desktop coding-agent surface unless trusted runtime context explicitly identifies the web editor. In web chat, declared tools and the latest `editor_context` override any capability described in this broad pack.
64
+
53
65
  You may be running as the **in-web AI chat** (the /editor copilot, the chat dock, or the /chat and /library assistants) or as a **local coding agent** driving `vidfarm-devcli` (Claude Code / Codex on the user's machine). Same three paintbrushes, different reach — be honest about which surface you are and route heavy work accordingly:
54
66
 
67
+ - **Determine the surface before claiming capabilities.** The web chat has only its declared tools and REST routes. It cannot execute arbitrary JavaScript/Python, open a shell, create a local repository script, or use the user's filesystem. Never tell a web-chat user that you ran code or wrote a script unless a dedicated declared tool actually did so. A desktop coding agent has a real shell and filesystem and MAY write/run scripts, perform arbitrary local computations over paginated API results, create reports/CSVs/JSON, edit composition files, and orchestrate long devcli workflows within the user's authorization.
55
68
  - **The web AI chat can do all three paintbrushes** — clip raws, author HTML/hyperframe motion, and generate AI media — and it drives edits directly on the live timeline. Keep small-to-medium jobs here: text/caption swaps, a scene or two replaced, single generations, captions, approve/schedule. Just do them.
56
69
  - **Where the web chat struggles: complex, long, multi-step transformations.** A full multi-scene re-theme, an iterative render-critique-iterate loop, heavy scripted or batch work, or anything needing a real filesystem and many sequential tool calls will hit context limits, turn/timeout ceilings, and the web editor's constraints (CSS/declarative motion only — JS animation adapters are stripped on save). Don't grind a big transformation one layer at a time in a chat turn and stall.
57
70
  - **Practical workaround — hand the heavy job to local devcli.** When a task is genuinely large or long-running, **proactively recommend the director run it locally with an AI coding agent** (Claude Code / OpenAI Codex / any capable agent): `vidfarm pull <forkId>` writes the composition + the `.harness/` grounding bundle to disk, the agent edits with the full devcli verb set and JS animation adapters, renders free with `vidfarm serve`, and `vidfarm publish` pushes it back. This is the **best-quality (B) harness's** natural home (adversarial grading with a coding agent). Frame it as "this is a big rebuild — you'll get a better, faster result running it locally with a coding agent; here's how," not as a dead end.
71
+ - **Offer a handoff, do not impersonate the desktop agent.** When web chat reaches that boundary, offer to save a Markdown handoff in My Files containing the objective, selected template/fork IDs, asset paths, grounding, constraints, completed work, and suggested devcli commands. Create it only after the user agrees. The desktop agent should read that document, pull the referenced fork, and then use its actual code/shell capabilities.
58
72
  - **Never send the user away just to read knowledge.** Deeper skill knowledge is always a **tool call** away in-place: call `load_skill` (e.g. `load_skill('vidfarm-director', file='references/editor-workflows.md')`, or a craft pack like `editor-capabilities` / `hyperframes-animation`) to pull the exact reference you need mid-conversation. Only recommend switching surfaces for the WORK (a heavy transformation), never for the information.
59
73
 
60
74
  ## Read Only What You Need
@@ -67,6 +81,21 @@ Read only the relevant reference file for the current task.
67
81
  - REST automation, `vidfarm` command surface, local serve loop, skill packs: `references/automation-and-local-dev.md`
68
82
  - Getting-started interviews, strategy docs, onboarding flow: `references/onboarding.md`
69
83
  - Primitive routes such as TTS, STT, music, overlays, background removal, product placement: `references/primitives.md`
84
+ - Complete REST API map and raw-HTTP conventions: `references/rest-api.md`. Load it only when the user asks for REST, an endpoint/schema, direct HTTP integration, or exhaustive API coverage. For the entire specification, follow its domain links and load every listed reference; do not preload them into ordinary director conversations.
85
+
86
+ ## HyperFrames Skills — Load on Demand
87
+
88
+ Vidfarm ships a curated HyperFrames skill suite alongside this director pack. Use it for composition authoring and motion craft without loading the entire suite into context.
89
+
90
+ 1. Route broad video-creation requests through `hyperframes` first. It selects the appropriate workflow skill.
91
+ 2. Load only the selected workflow skill, then add the narrow domain skill required by the current step.
92
+ 3. Load `hyperframes-core` before writing or restructuring composition HTML.
93
+ 4. Load `hyperframes-animation`, `hyperframes-keyframes`, `hyperframes-creative`, or `hyperframes-cli` only when the task needs that specific capability.
94
+ 5. Use `vidfarm-media` for narration, music, transcription, captions timing, background removal, and media sourcing. Prefer Vidfarm primitives and the user's existing library; when unavailable, use an equivalent capability already exposed by the user's desktop AI agent.
95
+
96
+ On the web copilot, call `load_skill('<name>')` and load referenced files only when the selected skill reaches that step. On desktop, inspect the agent's available-skill catalog and local `.agents/skills` / `.claude/skills` entries before declaring a skill missing. If needed, install only the selected bundled pack with `vidfarm skills add <name>`; do not bulk-install or bulk-read the suite. If the Vidfarm CLI is unavailable, use the desktop agent's native skill discovery or local-skill mechanism and continue with the closest installed equivalent.
97
+
98
+ HyperFrames authoring and rendering in this package are Vidfarm-native: local work uses the bundled composition toolchain and `vidfarm serve`; cloud work uses Vidfarm render routes. Do not require an external vendor account, repository, publish service, or telemetry endpoint. Keep `HYPERFRAMES_SKIP_SKILLS=1` and `HYPERFRAMES_NO_TELEMETRY=1` in Vidfarm-managed environments so the bundled skills stay pinned and local work does not phone home.
70
99
 
71
100
  ## Quick Router
72
101
 
@@ -27,7 +27,7 @@ Mine a **long-form** video (podcast, stream VOD, webinar, any YouTube/TikTok/IG/
27
27
 
28
28
  - **Poll** `GET /raws/scan/:scanId` until the source `status` is `complete` (or `failed` — crashed pipelines are reconciled to `failed`, pollers never spin).
29
29
  - **Results**: `GET /raws/feed?source=<source_video_id>` (or the whole library), hybrid search via `POST /raws/search` `{ query: "someone looks confused" }`, per-raw download at `GET /raws/:clipId/download`. Raws carry taxonomy tags, a description, a transcript, an `aspect` field when cropped, the hunt's `tracer`, and (with a gemini/openai key) a semantic embedding.
30
- - **Billing** — **AWS compute only** (`clip_scan_lambda` GB-seconds + Step Functions transitions, a fraction of a cent for typical hunts; the 202 response includes a `compute_estimate`). The AI tagging/refine runs on **your saved provider key (BYOK)** and is never wallet-billed. Submission is wallet-gated (402 when empty).
30
+ - **Billing** — scan compute (`clip_scan_lambda` GB-seconds + Step Functions transitions) and each persisted semantic embedding are wallet-metered; the 202 response includes a `compute_estimate`. AI tagging/refine runs on **your saved provider key (BYOK)** and is not wallet-billed. Submission is wallet-gated (402 when empty).
31
31
  - **The original stays temporary** — URL-ingested and uploaded sources live ONLY in your temp folder (`users/…/temporary/clip-sources/…`) with a hard **30-day TTL** (auto-deleted from S3 + the temp-file list). The hunted raws themselves are durable.
32
32
  - **Captions rule** — "no captions / no on-screen text" hunts are handled by scene SELECTION (the refine pass drops text-heavy scenes). **Never run GhostCut caption removal on a long-form source** (it is hard-capped at ~15 minutes); to actually erase burned-in text, apply `POST /api/v1/primitives/videos/remove-captions` to individual FINISHED raws afterwards.
33
33
  - **Clip an EXACT subrange (no AI hunt)** — when you want one precise hand-picked in/out rather than an open-ended hunt, use the **Clipper**: the web page `https://vidfarm.cc/tools/clipper` (previews the source straight from its URL, HH:MM:SS:NNNN in/out, live length estimate, multi-clip session grouped by tracer), the endpoint `POST /raws/clip-range` `{ source_url|preview_url|temp_file_id, start_sec, end_sec, tracer?, folder_path?, name? }` (trims exactly that window into one raw — compute-only, no AI/key), or the devcli twin `vidfarm clipper <video-url|file> --start <t> --end <t> [--tracer NAME --folder NAME --name TXT]`. The devcli path is now **local-first**: local files clip into the local raws store by default, and URL sources are staged locally before trimming unless you pass `--cloud`. If you do **not** have a `VIDFARM_API_KEY`, URL clipping cannot bootstrap the remote source into the local backend; open the long-form raw video in the browser, save/extract the actual video file locally, then run `vidfarm clipper ./source.mp4 ...`. The Library "Import source" button opens the Clipper; each save lands in `/raws/<folder>` (defaults to the tracer). Reach for `/raws/scan` for multi-clip hunts, `/raws/clip-range` for surgical single cuts. (The masked-image and time-scoped-video editors live alongside it at `/tools/image` and `/tools/video`.)
@@ -28,6 +28,8 @@ Send a stable `tracer` on export so retries are traceable and filterable in job
28
28
 
29
29
  **Scripting mode** is the recommended posture for repeatable template automation. Use it when a director wants to take a template they like, agree on a base fork, and then drive bulk or one-off edits entirely through REST or `vidfarm api` from a script, Lambda, or local machine.
30
30
 
31
+ This section is for a **desktop/local coding agent**, not the web copilot. A local Codex/Claude agent may use its shell and filesystem to write JavaScript/TypeScript/Python/shell scripts, fetch every API page, join and score catalog/library data, calculate statistics, emit CSV/JSON/Markdown reports, manipulate composition DOM files, and run iterative render/inspection loops. The web copilot cannot inherit those abilities from this document: it may only call its declared tools and bounded REST routes. If web chat prepares work for this flow, consume its My Files handoff document as input; do not claim the web chat itself executed the script.
32
+
31
33
  Contract:
32
34
 
33
35
  1. Pick the template once, then fork once and treat the resulting `forkId` as the stable base.
@@ -111,7 +113,7 @@ If a local AI script rewrites text or scenes without consuming those files first
111
113
 
112
114
  | Command | REST route | Flow step |
113
115
  |---|---|---|
114
- | `vidfarm discover [query]` | `GET /discover/feed[?q=]` | browse/search templates |
116
+ | `vidfarm discover [query] [--sort relevance\|wow\|automation\|recent] [--cursor <cursor>]` | `GET /discover/feed[?q=&sort=&cursor=]` | hybrid-search templates and page through the catalog |
115
117
  | `vidfarm videos [query] [--mine]` | `GET /api/v1/videos[?q=&mine=]` | browse/search source inspirations |
116
118
  | `vidfarm inspiration-add <url\|file>` | `POST /discover/templates` (files: `POST /discover/templates/upload/presign` + PUT first) | add your own source (URL or local video upload) |
117
119
  | `vidfarm inspiration-rm <id>` | `DELETE /discover/templates/:id` | remove a private one |
@@ -51,7 +51,7 @@ Templates are listed on the Vidfarm homepage and `/discover`. Each has a `templa
51
51
 
52
52
  - Browser: browse `/discover`
53
53
  - API: `GET /discover/feed` — returns `{ templates: [{ templateId, slugId, title, previewUrl, viralDna, durationSeconds, sourceType, promotions, keywords, summary, ... }], next_cursor }`
54
- - Search: `GET /discover/feed?q=<offer>&limit=20` keyword-searches the catalog so you can answer "which templates suit my `<offer>`?" (e.g. `?q=weight%20loss%20app`). Ranking is a coarse keyword match over each template's title, viral DNA, and the decompose-derived `promotions` (offer categories the format can sell), `keywords`, and `summary` read those fields to judge fit and rank the best 3-6 semantically. `GET /api/v1/videos?q=<offer>&limit=20[&mine=true]` runs the same search over source **inspirations**. `promotions`/`keywords`/`summary` are populated by the decompose pass, so a freshly ingested template that has not been decomposed yet won't match on them decompose it (or fall back to broader/synonym terms) if a search comes up empty.
54
+ - Search: `GET /discover/feed?q=<offer>&limit=20&sort=relevance` hybrid-searches the eligible public catalog using semantic embeddings plus lexical matches. Semantic query embedding uses Vidfarm's canonical OpenRouter-routed model and bills the provider cost × the standard 1.2 markup to the user's wallet. The response's `search` block reports `mode`, `embedding_space`, and any `semantic_limitation`; when it says `lexical_structured`, disclose the limitation briefly and continue rather than refusing. Decomposition adds `promotions`, `keywords`, `summary`, `categoryTags`, and `catalogIntelligence` (`wowScore`, `wowReason`, `automationScore`, `automationReason`, `contentStyles`, `searchText`). Use `sort=wow` for highest-quality/client-impressing formats, `sort=automation` for cheap repeatable bulk formats, and `sort=recent` only when freshness is the intent. Follow `next_cursor` with `cursor=<value>`; never call page one the whole catalog. `GET /api/v1/videos?q=<offer>&limit=20[&mine=true]` searches source **inspirations**. Undecomposed inspirations have only sparse ingest metadata, so they are harder to retrieve semantically. Explain that somebody in the world needs to decompose one once and the shared enrichment then benefits everyone; the current user need not act unless they want that specific inspiration immediately.
55
55
 
56
56
  Each template exposes a public preview:
57
57
 
@@ -73,7 +73,7 @@ To bring a new viral video into the catalog as a **private** template you own, i
73
73
  - `DELETE /discover/templates/:entryId` — remove a private inspiration/template you own (accepts either the `inspiration_...` or minted `template_...` id).
74
74
  - `POST /api/v1/inspirations/:inspirationId/decompose { user_prompt? }` — AI-decompose an inspiration's downloaded video into scenes (requires a saved provider key; same 120s source cap as auto-decompose).
75
75
 
76
- devcli: `vidfarm inspiration-add <url|file.mp4>` (a local file path runs the presign→PUT→finalize flow for you), `vidfarm inspiration-rm <id>`, `vidfarm inspiration-decompose <id>`, `vidfarm discover [query]` to browse/search templates, `vidfarm public-raws [query] [--category <key>] [--type <raw_type>] [--bookmark <raw_id>]` to browse/save the public raws catalog, and `vidfarm videos [query] [--mine]` to browse/search source inspirations. On the web, the Discover page's **Add Template** modal accepts either a URL or a file upload.
76
+ devcli: `vidfarm inspiration-add <url|file.mp4>` (a local file path runs the presign→PUT→finalize flow for you), `vidfarm inspiration-rm <id>`, `vidfarm inspiration-decompose <id>`, `vidfarm discover [query] [--sort relevance|wow|automation|recent] [--cursor <next_cursor>]` to browse/search templates, `vidfarm public-raws [query] [--category <key>] [--type <raw_type>] [--bookmark <raw_id>]` to browse/save the public raws catalog, and `vidfarm videos [query] [--mine]` to browse/search source inspirations. On the web, the Discover page's **Add Template** modal accepts either a URL or a file upload.
77
77
 
78
78
  ## Fork a template
79
79
 
@@ -0,0 +1,84 @@
1
+ # Vidfarm REST API — On-Demand Map
2
+
3
+ Use this file when integrating through raw HTTP, looking up an endpoint, or auditing the complete public API. This is a navigation layer: the detailed request bodies, response shapes, state transitions, cost notes, and route-specific guardrails live in the linked domain references so the default skill context stays small.
4
+
5
+ ## Contents
6
+
7
+ - [Client choice](#client-choice)
8
+ - [Base URL and authentication](#base-url-and-authentication)
9
+ - [Complete specification map](#complete-specification-map)
10
+ - [Raw HTTP pattern](#raw-http-pattern)
11
+ - [Operational rules](#operational-rules)
12
+ - [Keeping the reference complete](#keeping-the-reference-complete)
13
+
14
+ ## Client choice
15
+
16
+ The skill is documentation for an AI agent. It is not a client library and does not execute requests.
17
+
18
+ - **Raw REST:** every cloud operation is available through HTTP and can be called from `curl`, `fetch`, Python, CI, a server, or any other HTTP client.
19
+ - **Recommended desktop convenience:** `npx @mevdragon/vidfarm-devcli ...` (or the installed `vidfarm` binary) wraps those same routes. Prefer it for interactive desktop work because it handles the `vidfarm-api-key` header, uploads, polling, pagination, local paths, composition pull/publish state, and readable errors.
20
+ - **Use raw REST** when building a durable integration, working in an existing application stack, calling a route the devcli has not wrapped yet, or when the user explicitly requests HTTP.
21
+
22
+ Do not imply that installing this skill installs the CLI. Do not imply that the CLI unlocks API capabilities unavailable through REST.
23
+
24
+ ## Base URL and authentication
25
+
26
+ - Production base URL: `https://vidfarm.cc`
27
+ - API-key authentication: `vidfarm-api-key: <key>`
28
+ - Do not use `Authorization: Bearer` unless a specific non-API browser/login flow explicitly documents it.
29
+ - Send `content-type: application/json` for JSON bodies and `accept: application/json` when JSON is expected.
30
+ - Treat fork IDs, share tokens, upload URLs, and returned media URLs according to the route-specific guidance in the domain references.
31
+
32
+ ## Complete specification map
33
+
34
+ For a focused task, load only the relevant file. If the user requests the **entire REST API specification**, load this file plus every file in the following table. Together they are the bundled director-facing REST specification.
35
+
36
+ | API area | Load | Coverage |
37
+ |---|---|---|
38
+ | Accounts, login, provider keys, discovery, inspirations, templates, forks, versions, render, approval, sharing, scheduling, billing | `core-workflows.md` | Customer and composition lifecycle |
39
+ | Composition HTML/JSON, timeline layers, editor actions, clone/fork, auto-decompose, video context, captions, transitions, speech regeneration | `editor-workflows.md` | Editing and decomposition |
40
+ | Public raws, clip hunts, exact-range clipping, My Files, temporary files, folders, annotations, semantic search, recurring-character assets | `assets-and-sourcing.md` | Media and library APIs |
41
+ | Direct automation, pagination, scripting, devcli-to-route parity, local/cloud boundaries, skill-pack distribution | `automation-and-local-dev.md` | Headless and desktop integration |
42
+ | Image, video, TTS, STT, music, captions, overlays, inpaint, background/caption removal, product placement, async primitive jobs | `primitives.md` | Generation and transformation primitives |
43
+ | Onboarding document persistence | `onboarding.md` | Durable strategy/context files |
44
+
45
+ The route and payload examples in those files are the public, customer-facing contract. Architecture-only internals are intentionally excluded. When exact behavior is ambiguous, inspect the live response or route implementation rather than inventing a field.
46
+
47
+ ## Raw HTTP pattern
48
+
49
+ ```bash
50
+ curl --fail-with-body \
51
+ -H "vidfarm-api-key: $VIDFARM_API_KEY" \
52
+ -H "accept: application/json" \
53
+ "https://vidfarm.cc/discover/feed?limit=20"
54
+ ```
55
+
56
+ For JSON mutations:
57
+
58
+ ```bash
59
+ curl --fail-with-body \
60
+ -X POST \
61
+ -H "vidfarm-api-key: $VIDFARM_API_KEY" \
62
+ -H "content-type: application/json" \
63
+ -H "accept: application/json" \
64
+ --data '{"template_id":"template_example"}' \
65
+ "https://vidfarm.cc/api/v1/compositions"
66
+ ```
67
+
68
+ Use environment variables or a secret manager for keys. Never paste secrets into a skill, composition, source file, command transcript intended for sharing, or generated report.
69
+
70
+ ## Operational rules
71
+
72
+ - Follow `next_cursor`/cursor fields until exhausted when the user asks for all records; do not call the first page the full dataset.
73
+ - Treat submission and generation routes as non-idempotent unless the route explicitly states otherwise. Check existing state before retrying.
74
+ - Poll asynchronous jobs at the documented status endpoint and stop on terminal success or failure. Do not fabricate completion from an accepted response.
75
+ - Upload local media through the documented presign/upload/finalize sequence. The devcli is recommended on desktop because it performs this sequence automatically.
76
+ - Parse and mutate composition HTML as a DOM. Never edit it by string concatenation.
77
+ - Render through `POST /api/v1/compositions/:forkId/render`; do not call the underlying renderer directly.
78
+ - Surface wallet/provider costs before expensive operations and preserve returned job IDs for audit and retries.
79
+
80
+ ## Keeping the reference complete
81
+
82
+ The canonical skill sources live in `.agents/skills/vidfarm-director/`. `SKILL.director.md` is a generated copy-friendly rollup; this REST map is deliberately excluded from that immediate rollup so ordinary skill loads remain lean. It is distributed in the multi-file skill pack and fetched on demand through the public skill-pack routes.
83
+
84
+ When routes change, update the appropriate domain reference and run `npm run check:skills`. That check verifies every documented `/api/` and `/login/` method/path against the registered application routes. Add new public API areas to the table above so exhaustive API requests remain discoverable.
package/SKILL.director.md CHANGED
@@ -50,11 +50,25 @@ Present both harnesses to the director, recommend (A) unless they've asked for p
50
50
 
51
51
  ## Web AI chat vs. local devcli — know your surface's limits
52
52
 
53
+ **Desktop users: this is a skill, not an SDK or required runtime.** It gives an AI agent Vidfarm's operating knowledge; it does not proxy requests, add hidden capabilities, or require the agent to use a particular client. Every cloud workflow described here can be performed with ordinary HTTP against the Vidfarm REST API. For convenience and safer composition/file workflows, recommend the `vidfarm` command from `@mevdragon/vidfarm-devcli`: it wraps the same REST routes, handles auth, uploads, polling, pagination, local files, and pull/edit/render loops. Use raw REST whenever the user prefers it, the CLI lacks a new route, or the work belongs in an existing service or automation stack.
54
+
55
+ Choose the surface before choosing instructions:
56
+
57
+ | Surface | Live timeline | Shell / local files | Use it for |
58
+ |---|---:|---:|---|
59
+ | Web copilot | Yes, through `editor_action` | No | Short and medium edits on the open composition |
60
+ | Desktop coding agent | Through pulled files and devcli | Yes | Full rebuilds, scripts, batch work, and render/inspect loops |
61
+ | Devcli without an agent | Command-driven | Yes | Deterministic pull, edit, render, approve, and automation operations |
62
+
63
+ When this pack is installed locally, assume the desktop coding-agent surface unless trusted runtime context explicitly identifies the web editor. In web chat, declared tools and the latest `editor_context` override any capability described in this broad pack.
64
+
53
65
  You may be running as the **in-web AI chat** (the /editor copilot, the chat dock, or the /chat and /library assistants) or as a **local coding agent** driving `vidfarm-devcli` (Claude Code / Codex on the user's machine). Same three paintbrushes, different reach — be honest about which surface you are and route heavy work accordingly:
54
66
 
67
+ - **Determine the surface before claiming capabilities.** The web chat has only its declared tools and REST routes. It cannot execute arbitrary JavaScript/Python, open a shell, create a local repository script, or use the user's filesystem. Never tell a web-chat user that you ran code or wrote a script unless a dedicated declared tool actually did so. A desktop coding agent has a real shell and filesystem and MAY write/run scripts, perform arbitrary local computations over paginated API results, create reports/CSVs/JSON, edit composition files, and orchestrate long devcli workflows within the user's authorization.
55
68
  - **The web AI chat can do all three paintbrushes** — clip raws, author HTML/hyperframe motion, and generate AI media — and it drives edits directly on the live timeline. Keep small-to-medium jobs here: text/caption swaps, a scene or two replaced, single generations, captions, approve/schedule. Just do them.
56
69
  - **Where the web chat struggles: complex, long, multi-step transformations.** A full multi-scene re-theme, an iterative render-critique-iterate loop, heavy scripted or batch work, or anything needing a real filesystem and many sequential tool calls will hit context limits, turn/timeout ceilings, and the web editor's constraints (CSS/declarative motion only — JS animation adapters are stripped on save). Don't grind a big transformation one layer at a time in a chat turn and stall.
57
70
  - **Practical workaround — hand the heavy job to local devcli.** When a task is genuinely large or long-running, **proactively recommend the director run it locally with an AI coding agent** (Claude Code / OpenAI Codex / any capable agent): `vidfarm pull <forkId>` writes the composition + the `.harness/` grounding bundle to disk, the agent edits with the full devcli verb set and JS animation adapters, renders free with `vidfarm serve`, and `vidfarm publish` pushes it back. This is the **best-quality (B) harness's** natural home (adversarial grading with a coding agent). Frame it as "this is a big rebuild — you'll get a better, faster result running it locally with a coding agent; here's how," not as a dead end.
71
+ - **Offer a handoff, do not impersonate the desktop agent.** When web chat reaches that boundary, offer to save a Markdown handoff in My Files containing the objective, selected template/fork IDs, asset paths, grounding, constraints, completed work, and suggested devcli commands. Create it only after the user agrees. The desktop agent should read that document, pull the referenced fork, and then use its actual code/shell capabilities.
58
72
  - **Never send the user away just to read knowledge.** Deeper skill knowledge is always a **tool call** away in-place: call `load_skill` (e.g. `load_skill('vidfarm-director', file='references/editor-workflows.md')`, or a craft pack like `editor-capabilities` / `hyperframes-animation`) to pull the exact reference you need mid-conversation. Only recommend switching surfaces for the WORK (a heavy transformation), never for the information.
59
73
 
60
74
  ## Read Only What You Need
@@ -67,6 +81,21 @@ Read only the relevant reference file for the current task.
67
81
  - REST automation, `vidfarm` command surface, local serve loop, skill packs: `references/automation-and-local-dev.md`
68
82
  - Getting-started interviews, strategy docs, onboarding flow: `references/onboarding.md`
69
83
  - Primitive routes such as TTS, STT, music, overlays, background removal, product placement: `references/primitives.md`
84
+ - Complete REST API map and raw-HTTP conventions: `references/rest-api.md`. Load it only when the user asks for REST, an endpoint/schema, direct HTTP integration, or exhaustive API coverage. For the entire specification, follow its domain links and load every listed reference; do not preload them into ordinary director conversations.
85
+
86
+ ## HyperFrames Skills — Load on Demand
87
+
88
+ Vidfarm ships a curated HyperFrames skill suite alongside this director pack. Use it for composition authoring and motion craft without loading the entire suite into context.
89
+
90
+ 1. Route broad video-creation requests through `hyperframes` first. It selects the appropriate workflow skill.
91
+ 2. Load only the selected workflow skill, then add the narrow domain skill required by the current step.
92
+ 3. Load `hyperframes-core` before writing or restructuring composition HTML.
93
+ 4. Load `hyperframes-animation`, `hyperframes-keyframes`, `hyperframes-creative`, or `hyperframes-cli` only when the task needs that specific capability.
94
+ 5. Use `vidfarm-media` for narration, music, transcription, captions timing, background removal, and media sourcing. Prefer Vidfarm primitives and the user's existing library; when unavailable, use an equivalent capability already exposed by the user's desktop AI agent.
95
+
96
+ On the web copilot, call `load_skill('<name>')` and load referenced files only when the selected skill reaches that step. On desktop, inspect the agent's available-skill catalog and local `.agents/skills` / `.claude/skills` entries before declaring a skill missing. If needed, install only the selected bundled pack with `vidfarm skills add <name>`; do not bulk-install or bulk-read the suite. If the Vidfarm CLI is unavailable, use the desktop agent's native skill discovery or local-skill mechanism and continue with the closest installed equivalent.
97
+
98
+ HyperFrames authoring and rendering in this package are Vidfarm-native: local work uses the bundled composition toolchain and `vidfarm serve`; cloud work uses Vidfarm render routes. Do not require an external vendor account, repository, publish service, or telemetry endpoint. Keep `HYPERFRAMES_SKIP_SKILLS=1` and `HYPERFRAMES_NO_TELEMETRY=1` in Vidfarm-managed environments so the bundled skills stay pinned and local work does not phone home.
70
99
 
71
100
  ## Quick Router
72
101
 
@@ -158,7 +187,7 @@ Templates are listed on the Vidfarm homepage and `/discover`. Each has a `templa
158
187
 
159
188
  - Browser: browse `/discover`
160
189
  - API: `GET /discover/feed` — returns `{ templates: [{ templateId, slugId, title, previewUrl, viralDna, durationSeconds, sourceType, promotions, keywords, summary, ... }], next_cursor }`
161
- - Search: `GET /discover/feed?q=<offer>&limit=20` keyword-searches the catalog so you can answer "which templates suit my `<offer>`?" (e.g. `?q=weight%20loss%20app`). Ranking is a coarse keyword match over each template's title, viral DNA, and the decompose-derived `promotions` (offer categories the format can sell), `keywords`, and `summary` read those fields to judge fit and rank the best 3-6 semantically. `GET /api/v1/videos?q=<offer>&limit=20[&mine=true]` runs the same search over source **inspirations**. `promotions`/`keywords`/`summary` are populated by the decompose pass, so a freshly ingested template that has not been decomposed yet won't match on them decompose it (or fall back to broader/synonym terms) if a search comes up empty.
190
+ - Search: `GET /discover/feed?q=<offer>&limit=20&sort=relevance` hybrid-searches the eligible public catalog using semantic embeddings plus lexical matches. Semantic query embedding uses Vidfarm's canonical OpenRouter-routed model and bills the provider cost × the standard 1.2 markup to the user's wallet. The response's `search` block reports `mode`, `embedding_space`, and any `semantic_limitation`; when it says `lexical_structured`, disclose the limitation briefly and continue rather than refusing. Decomposition adds `promotions`, `keywords`, `summary`, `categoryTags`, and `catalogIntelligence` (`wowScore`, `wowReason`, `automationScore`, `automationReason`, `contentStyles`, `searchText`). Use `sort=wow` for highest-quality/client-impressing formats, `sort=automation` for cheap repeatable bulk formats, and `sort=recent` only when freshness is the intent. Follow `next_cursor` with `cursor=<value>`; never call page one the whole catalog. `GET /api/v1/videos?q=<offer>&limit=20[&mine=true]` searches source **inspirations**. Undecomposed inspirations have only sparse ingest metadata, so they are harder to retrieve semantically. Explain that somebody in the world needs to decompose one once and the shared enrichment then benefits everyone; the current user need not act unless they want that specific inspiration immediately.
162
191
 
163
192
  Each template exposes a public preview:
164
193
 
@@ -180,7 +209,7 @@ To bring a new viral video into the catalog as a **private** template you own, i
180
209
  - `DELETE /discover/templates/:entryId` — remove a private inspiration/template you own (accepts either the `inspiration_...` or minted `template_...` id).
181
210
  - `POST /api/v1/inspirations/:inspirationId/decompose { user_prompt? }` — AI-decompose an inspiration's downloaded video into scenes (requires a saved provider key; same 120s source cap as auto-decompose).
182
211
 
183
- devcli: `vidfarm inspiration-add <url|file.mp4>` (a local file path runs the presign→PUT→finalize flow for you), `vidfarm inspiration-rm <id>`, `vidfarm inspiration-decompose <id>`, `vidfarm discover [query]` to browse/search templates, `vidfarm public-raws [query] [--category <key>] [--type <raw_type>] [--bookmark <raw_id>]` to browse/save the public raws catalog, and `vidfarm videos [query] [--mine]` to browse/search source inspirations. On the web, the Discover page's **Add Template** modal accepts either a URL or a file upload.
212
+ devcli: `vidfarm inspiration-add <url|file.mp4>` (a local file path runs the presign→PUT→finalize flow for you), `vidfarm inspiration-rm <id>`, `vidfarm inspiration-decompose <id>`, `vidfarm discover [query] [--sort relevance|wow|automation|recent] [--cursor <next_cursor>]` to browse/search templates, `vidfarm public-raws [query] [--category <key>] [--type <raw_type>] [--bookmark <raw_id>]` to browse/save the public raws catalog, and `vidfarm videos [query] [--mine]` to browse/search source inspirations. On the web, the Discover page's **Add Template** modal accepts either a URL or a file upload.
184
213
 
185
214
  ## Fork a template
186
215
 
@@ -774,7 +803,7 @@ Mine a **long-form** video (podcast, stream VOD, webinar, any YouTube/TikTok/IG/
774
803
 
775
804
  - **Poll** `GET /raws/scan/:scanId` until the source `status` is `complete` (or `failed` — crashed pipelines are reconciled to `failed`, pollers never spin).
776
805
  - **Results**: `GET /raws/feed?source=<source_video_id>` (or the whole library), hybrid search via `POST /raws/search` `{ query: "someone looks confused" }`, per-raw download at `GET /raws/:clipId/download`. Raws carry taxonomy tags, a description, a transcript, an `aspect` field when cropped, the hunt's `tracer`, and (with a gemini/openai key) a semantic embedding.
777
- - **Billing** — **AWS compute only** (`clip_scan_lambda` GB-seconds + Step Functions transitions, a fraction of a cent for typical hunts; the 202 response includes a `compute_estimate`). The AI tagging/refine runs on **your saved provider key (BYOK)** and is never wallet-billed. Submission is wallet-gated (402 when empty).
806
+ - **Billing** — scan compute (`clip_scan_lambda` GB-seconds + Step Functions transitions) and each persisted semantic embedding are wallet-metered; the 202 response includes a `compute_estimate`. AI tagging/refine runs on **your saved provider key (BYOK)** and is not wallet-billed. Submission is wallet-gated (402 when empty).
778
807
  - **The original stays temporary** — URL-ingested and uploaded sources live ONLY in your temp folder (`users/…/temporary/clip-sources/…`) with a hard **30-day TTL** (auto-deleted from S3 + the temp-file list). The hunted raws themselves are durable.
779
808
  - **Captions rule** — "no captions / no on-screen text" hunts are handled by scene SELECTION (the refine pass drops text-heavy scenes). **Never run GhostCut caption removal on a long-form source** (it is hard-capped at ~15 minutes); to actually erase burned-in text, apply `POST /api/v1/primitives/videos/remove-captions` to individual FINISHED raws afterwards.
780
809
  - **Clip an EXACT subrange (no AI hunt)** — when you want one precise hand-picked in/out rather than an open-ended hunt, use the **Clipper**: the web page `https://vidfarm.cc/tools/clipper` (previews the source straight from its URL, HH:MM:SS:NNNN in/out, live length estimate, multi-clip session grouped by tracer), the endpoint `POST /raws/clip-range` `{ source_url|preview_url|temp_file_id, start_sec, end_sec, tracer?, folder_path?, name? }` (trims exactly that window into one raw — compute-only, no AI/key), or the devcli twin `vidfarm clipper <video-url|file> --start <t> --end <t> [--tracer NAME --folder NAME --name TXT]`. The devcli path is now **local-first**: local files clip into the local raws store by default, and URL sources are staged locally before trimming unless you pass `--cloud`. If you do **not** have a `VIDFARM_API_KEY`, URL clipping cannot bootstrap the remote source into the local backend; open the long-form raw video in the browser, save/extract the actual video file locally, then run `vidfarm clipper ./source.mp4 ...`. The Library "Import source" button opens the Clipper; each save lands in `/raws/<folder>` (defaults to the tracer). Reach for `/raws/scan` for multi-clip hunts, `/raws/clip-range` for surgical single cuts. (The masked-image and time-scoped-video editors live alongside it at `/tools/image` and `/tools/video`.)
@@ -893,6 +922,8 @@ Send a stable `tracer` on export so retries are traceable and filterable in job
893
922
 
894
923
  **Scripting mode** is the recommended posture for repeatable template automation. Use it when a director wants to take a template they like, agree on a base fork, and then drive bulk or one-off edits entirely through REST or `vidfarm api` from a script, Lambda, or local machine.
895
924
 
925
+ This section is for a **desktop/local coding agent**, not the web copilot. A local Codex/Claude agent may use its shell and filesystem to write JavaScript/TypeScript/Python/shell scripts, fetch every API page, join and score catalog/library data, calculate statistics, emit CSV/JSON/Markdown reports, manipulate composition DOM files, and run iterative render/inspection loops. The web copilot cannot inherit those abilities from this document: it may only call its declared tools and bounded REST routes. If web chat prepares work for this flow, consume its My Files handoff document as input; do not claim the web chat itself executed the script.
926
+
896
927
  Contract:
897
928
 
898
929
  1. Pick the template once, then fork once and treat the resulting `forkId` as the stable base.
@@ -976,7 +1007,7 @@ If a local AI script rewrites text or scenes without consuming those files first
976
1007
 
977
1008
  | Command | REST route | Flow step |
978
1009
  |---|---|---|
979
- | `vidfarm discover [query]` | `GET /discover/feed[?q=]` | browse/search templates |
1010
+ | `vidfarm discover [query] [--sort relevance\|wow\|automation\|recent] [--cursor <cursor>]` | `GET /discover/feed[?q=&sort=&cursor=]` | hybrid-search templates and page through the catalog |
980
1011
  | `vidfarm videos [query] [--mine]` | `GET /api/v1/videos[?q=&mine=]` | browse/search source inspirations |
981
1012
  | `vidfarm inspiration-add <url\|file>` | `POST /discover/templates` (files: `POST /discover/templates/upload/presign` + PUT first) | add your own source (URL or local video upload) |
982
1013
  | `vidfarm inspiration-rm <id>` | `DELETE /discover/templates/:id` | remove a private one |
package/dist/src/cli.js CHANGED
@@ -1,7 +1,7 @@
1
1
  #!/usr/bin/env node
2
2
  import { copyFileSync, createWriteStream, existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs";
3
3
  import path from "node:path";
4
- import { homedir, tmpdir } from "node:os";
4
+ import { tmpdir } from "node:os";
5
5
  import { randomUUID } from "node:crypto";
6
6
  import { fileURLToPath } from "node:url";
7
7
  import { parseArgs } from "node:util";
@@ -83,8 +83,10 @@ Local editor loop:
83
83
 
84
84
  Discover & inspiration (browse the viral-video catalog, add your own source):
85
85
  discover [query...] List/search TEMPLATES (best for an offer) → GET /discover/feed
86
- --query <text> Keyword search (e.g. "weight loss app")
86
+ --query <text> Hybrid semantic + keyword search
87
87
  --limit <n> Max results
88
+ --cursor <cursor> Continue from next_cursor
89
+ --sort <mode> relevance, wow, automation, or recent
88
90
  public-raws [query...] Browse PUBLIC RAWS (feed / categories) → GET /api/v1/public-raws
89
91
  --query <text> Keyword/vector query over title + metadata
90
92
  --category <key> Filter one public-raw category (e.g. emotion:confused)
@@ -487,11 +489,12 @@ Account:
487
489
  add-provider-key <provider> <secret> Save an AI provider key → POST /api/v1/user/me/provider-keys
488
490
 
489
491
  Agent skill (install the latest director skill so your AI agent can act):
490
- update-skill Fetch latest SKILL.director.md .claude/skills → GET /skill/vidfarm-director
491
- --global Install into ~/.claude/skills (default: ./.claude/skills)
492
- --dir <path> Install into a custom skills root
493
- --platform Also install SKILL.platform.md (vidfarm-platform)
494
- --print Print the skill to stdout instead of writing a file
492
+ update-skill DEPRECATED installer alias; installs the canonical
493
+ multi-file vidfarm-director pack via \`skills add\`.
494
+ Use \`vidfarm skills add vidfarm-director\` directly.
495
+ --dir <path> Project root for the canonical pack install
496
+ --print Print the generated single-file director rollup
497
+ (copy/share artifact; not the canonical install)
495
498
 
496
499
  Files (multi-step flows the devcli handles for you):
497
500
  upload <file> Upload to the EPHEMERAL temp store, print URL → POST /api/v1/user/me/temporary-files/upload
@@ -2184,14 +2187,14 @@ async function runTemplateCommand(argv) {
2184
2187
  }
2185
2188
  }
2186
2189
  // ── Discover & inspiration ──────────────────────────────────────────────────
2187
- // `discover` lists TEMPLATES; `--query` keyword-searches the catalog so an agent
2190
+ // `discover` lists TEMPLATES; `--query` hybrid-searches the catalog so an agent
2188
2191
  // CLI (Claude Code / Codex) can answer "which templates suit my <offer>?". Each
2189
2192
  // result carries decompose-derived promotions/keywords/summary to reason over.
2190
2193
  async function runDiscoverCommand(argv) {
2191
- const parsed = parseArgs({ args: argv, allowPositionals: true, options: { ...commonOptions(), limit: { type: "string" }, query: { type: "string" } } });
2194
+ const parsed = parseArgs({ args: argv, allowPositionals: true, options: { ...commonOptions(), limit: { type: "string" }, query: { type: "string" }, cursor: { type: "string" }, sort: { type: "string" } } });
2192
2195
  const ctx = commonContext(parsed.values);
2193
2196
  const query = (parsed.values.query ?? parsed.positionals.join(" ").trim()) || undefined;
2194
- const result = await apiRequest({ method: "GET", host: ctx.host, path: "/discover/feed", auth: ctx.auth, query: { limit: parsed.values.limit, q: query } });
2197
+ const result = await apiRequest({ method: "GET", host: ctx.host, path: "/discover/feed", auth: ctx.auth, query: { limit: parsed.values.limit, q: query, cursor: parsed.values.cursor, sort: parsed.values.sort } });
2195
2198
  assertApiOk(result, "discover");
2196
2199
  emitResult(result, ctx.json, [["Browse ", discoverFrontendUrl(ctx.host)]]);
2197
2200
  }
@@ -6250,29 +6253,18 @@ async function runUpdateSkillCommand(argv) {
6250
6253
  }
6251
6254
  return;
6252
6255
  }
6253
- const skillsRoot = parsed.values.dir
6254
- ? path.resolve(process.cwd(), String(parsed.values.dir))
6255
- : parsed.values.global
6256
- ? path.join(homedir(), ".claude", "skills")
6257
- : path.resolve(process.cwd(), ".claude", "skills");
6258
- const written = [];
6259
- for (const name of skillNames) {
6260
- const { contents, source } = await fetchSkillContents(ctx.host, SKILL_TARGETS[name]);
6261
- const destDir = path.join(skillsRoot, name);
6262
- mkdirSync(destDir, { recursive: true });
6263
- const destPath = path.join(destDir, "SKILL.md");
6264
- writeFileSync(destPath, contents, "utf8");
6265
- written.push({ skill: name, path: destPath, source, bytes: Buffer.byteLength(contents, "utf8") });
6266
- }
6267
- if (ctx.json) {
6268
- printJson({ ok: true, skillsRoot, installed: written });
6269
- return;
6256
+ if (parsed.values.global || parsed.values.platform) {
6257
+ throw new Error("update-skill --global/--platform are retired. Use `vidfarm skills add vidfarm-director --dir <project>` for the canonical multi-file pack; `update-skill --print` remains available for the generated copy-friendly rollup.");
6270
6258
  }
6271
- for (const entry of written) {
6272
- const tag = entry.source === "bundled" ? " (offline bundled copy)" : "";
6273
- console.log(`${GREEN}Installed ${entry.skill} skill${tag} → ${entry.path}${RESET}`);
6274
- }
6275
- console.log("");
6276
- console.log(` ${BOLD}Your AI agent can now use the "vidfarm-director" skill.${RESET}`);
6259
+ if (!ctx.json) {
6260
+ console.warn(`Deprecated: update-skill now delegates to the canonical multi-file skill installer. Use ${BOLD}vidfarm skills add vidfarm-director${RESET}.`);
6261
+ }
6262
+ await runSkillsCommand([
6263
+ "add",
6264
+ "vidfarm-director",
6265
+ ...(parsed.values.dir ? ["--dir", String(parsed.values.dir)] : []),
6266
+ ...(parsed.values.host ? ["--host", String(parsed.values.host)] : []),
6267
+ ...(ctx.json ? ["--json"] : [])
6268
+ ]);
6277
6269
  }
6278
6270
  //# sourceMappingURL=cli.js.map
@@ -71,6 +71,7 @@ export class ClipStore {
71
71
  thumbnail_path TEXT NOT NULL,
72
72
  tags_json TEXT NOT NULL,
73
73
  description TEXT NOT NULL DEFAULT '',
74
+ embedding_text TEXT,
74
75
  taxonomy_version TEXT NOT NULL,
75
76
  tier TEXT NOT NULL,
76
77
  user_notes TEXT,
@@ -96,6 +97,10 @@ export class ClipStore {
96
97
  scanned_at TEXT NOT NULL
97
98
  );
98
99
  `);
100
+ const clipColumns = this.db.prepare("PRAGMA table_info(clips)").all();
101
+ if (!clipColumns.some((column) => column.name === "embedding_text")) {
102
+ this.db.exec("ALTER TABLE clips ADD COLUMN embedding_text TEXT");
103
+ }
99
104
  }
100
105
  seedBuiltinPresets() {
101
106
  const stmt = this.db.prepare("INSERT OR REPLACE INTO presets(preset_id, name, description, criteria_json, builtin, created_at) VALUES (?, ?, ?, ?, 1, ?)");
@@ -109,9 +114,9 @@ export class ClipStore {
109
114
  this.db
110
115
  .prepare(`INSERT OR REPLACE INTO clips
111
116
  (clip_id, source_video_id, source_filename, start_time_sec, end_time_sec, duration_sec,
112
- file_path, thumbnail_path, tags_json, description, taxonomy_version, tier, user_notes, created_at)
113
- VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`)
114
- .run(c.clip_id, c.source_video_id, c.source_filename, c.start_time_sec, c.end_time_sec, c.duration_sec, c.file_path, c.thumbnail_path, JSON.stringify(c.tags), c.description ?? "", c.taxonomy_version, c.tier, c.user_notes ?? null, c.created_at);
117
+ file_path, thumbnail_path, tags_json, description, embedding_text, taxonomy_version, tier, user_notes, created_at)
118
+ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`)
119
+ .run(c.clip_id, c.source_video_id, c.source_filename, c.start_time_sec, c.end_time_sec, c.duration_sec, c.file_path, c.thumbnail_path, JSON.stringify(c.tags), c.description ?? "", c.embedding_text ?? null, c.taxonomy_version, c.tier, c.user_notes ?? null, c.created_at);
115
120
  if (c.embedding && c.embedding.length === this.dim) {
116
121
  this.db.prepare("DELETE FROM vec_clips WHERE clip_id = ?").run(c.clip_id);
117
122
  this.db
@@ -331,6 +336,7 @@ function rowToClip(row) {
331
336
  thumbnail_path: row.thumbnail_path,
332
337
  tags: safeParse(row.tags_json, emptyTags()),
333
338
  description: row.description ?? "",
339
+ embedding_text: row.embedding_text ?? undefined,
334
340
  taxonomy_version: row.taxonomy_version,
335
341
  tier: row.tier ?? "flash-lite",
336
342
  user_notes: row.user_notes ?? undefined,
@@ -12,7 +12,7 @@ import { homedir } from "node:os";
12
12
  import { parseArgs } from "node:util";
13
13
  import { createInterface } from "node:readline/promises";
14
14
  import { createIdV7 } from "../lib/ids.js";
15
- import { buildEffectiveGuidance, buildPrivateUserNotesGuidance, ClipModelClient, DEFAULT_CLIP_ASPECT, DEFAULT_CLIP_DURATION_BAND, detectLocalAgent, detectScenes, effectiveHuntDurationSec, estimateScanCostFromScenes, fitScenesToDurationBand, formatCostEstimate, hasFfmpeg, LocalAgentClipClient, normalizeAspect, normalizeCropFocus, normalizeWindows, parseClipHuntPrompt, parseTimeRanges, probeVideo, resolveDurationBand, resolveTargetClipCount, scanVideo, searchClips } from "../services/clip-curation/index.js";
15
+ import { buildEffectiveGuidance, buildPrivateUserNotesGuidance, ClipModelClient, DEFAULT_CLIP_ASPECT, DEFAULT_CLIP_DURATION_BAND, detectLocalAgent, detectScenes, effectiveHuntDurationSec, estimateScanCostFromScenes, formatCostEstimate, hasFfmpeg, LocalAgentClipClient, normalizeAspect, normalizeCropFocus, normalizeWindows, parseClipHuntPrompt, parseTimeRanges, probeVideo, resolveDurationBand, resolveTargetClipCount, scanVideo, searchClips } from "../services/clip-curation/index.js";
16
16
  import { ClipStore } from "./clip-store.js";
17
17
  export const CLIPS_HELP = `vidfarm raws — build & search a local raws library (local ffmpeg + local agent by default)
18
18
 
@@ -269,8 +269,7 @@ async function runScan(argv) {
269
269
  console.log(`[raws] preferring scenes without on-screen text/captions (selection only — GhostCut is never run on the source).`);
270
270
  console.log(`[raws] aiming for ~${targetClipCount} distinct raw${targetClipCount === 1 ? "" : "s"}${maxClips ? " (--max-clips)" : " (~10 / 10 min)"}.`);
271
271
  console.log(`[raws] detecting scenes …`);
272
- const detected = await detectScenes(videoPath, { durationSec: probe.duration_sec, windows, ...sceneOptions });
273
- const scenes = fitScenesToDurationBand(detected, durationBand);
272
+ const scenes = await detectScenes(videoPath, { durationSec: probe.duration_sec, windows, ...sceneOptions });
274
273
  if (scenes.length === 0)
275
274
  throw new Error("No scenes detected (is the video readable / non-empty?).");
276
275
  const store = new ClipStore(values.home);
@@ -130,7 +130,11 @@ export async function detectScenes(videoPath, opts = {}) {
130
130
  }
131
131
  async function detectScenesInSpan(videoPath, opts) {
132
132
  const threshold = opts.threshold ?? 0.3;
133
- const minScene = opts.minSceneSec ?? 0.6;
133
+ // Preserve very short detected shots as independent candidates. Absorbing a
134
+ // 2-10 frame flash into its neighbour makes it impossible for prompt/OCR
135
+ // refinement to reject that flash later. 50ms only de-duplicates effectively
136
+ // identical detector timestamps.
137
+ const minScene = opts.minSceneSec ?? 0.05;
134
138
  const maxScene = opts.maxSceneSec ?? 8;
135
139
  const { spanStart, spanEnd } = opts;
136
140
  const spanDuration = spanEnd - spanStart;
@@ -14,10 +14,10 @@ import { extractKeyframes } from "./ffmpeg.js";
14
14
  const MAX_SCENES_PER_BATCH = 60;
15
15
  // Bound total cost on very long sources: past this many batches we keep the raw
16
16
  // segmentation for the whole video (logged) instead of firing dozens of calls.
17
- const MAX_REFINE_BATCHES = 6;
17
+ const MAX_REFINE_BATCHES = 30;
18
18
  export async function refineScenesWithGuidance(input) {
19
19
  const { client, videoPath, scenes, guidancePrompt } = input;
20
- if (!guidancePrompt.trim() || scenes.length <= 1) {
20
+ if (!guidancePrompt.trim() || scenes.length === 0) {
21
21
  return { scenes, refined: false, kept: scenes.length, dropped: 0, merged: 0 };
22
22
  }
23
23
  const batchCount = Math.ceil(scenes.length / MAX_SCENES_PER_BATCH);
@@ -60,6 +60,8 @@ export async function refineScenesWithGuidance(input) {
60
60
  ? Math.max(1, Math.round((input.targetClipCount * batch.length) / scenes.length))
61
61
  : undefined;
62
62
  const batchDecisions = await askForDecisions(client, guidancePrompt, batch, frames, batchTarget);
63
+ if (!batchDecisions)
64
+ throw new Error("model returned no refinement plan");
63
65
  for (let i = 0; i < batch.length; i++) {
64
66
  decisions.push(batchDecisions?.[i] ?? keepDefault());
65
67
  }
@@ -111,7 +113,7 @@ async function askForDecisions(client, guidancePrompt, scenes, frames, targetCli
111
113
  maxOutputTokens: Math.max(1000, scenes.length * 40)
112
114
  });
113
115
  }
114
- function rebuildScenes(scenes, decisions) {
116
+ export function rebuildScenes(scenes, decisions) {
115
117
  const out = [];
116
118
  let dropped = 0;
117
119
  let merged = 0;
@@ -122,19 +124,25 @@ function rebuildScenes(scenes, decisions) {
122
124
  dropped++;
123
125
  continue;
124
126
  }
125
- if (d.merge_with_previous && out.length > 0) {
126
- // Extend the previous kept clip to cover this scene.
127
- const prev = out[out.length - 1];
128
- prev.end_time_sec = scenes[i].end_time_sec;
129
- prev.duration_sec = round3(prev.end_time_sec - prev.start_time_sec);
127
+ const previous = out[out.length - 1];
128
+ const isAdjacent = previous
129
+ ? Math.abs(previous.end_time_sec - scenes[i].start_time_sec) <= 0.05
130
+ : false;
131
+ if (d.merge_with_previous && previous && isAdjacent) {
132
+ // Extend only an ADJACENT previous kept clip. A dropped scene creates a
133
+ // real timeline gap; bridging that gap would silently put the rejected
134
+ // footage back into the exported raw.
135
+ previous.end_time_sec = scenes[i].end_time_sec;
136
+ previous.duration_sec = round3(previous.end_time_sec - previous.start_time_sec);
130
137
  merged++;
131
138
  continue;
132
139
  }
133
140
  out.push({ ...scenes[i], index: out.length });
134
141
  }
135
- // Never return an empty plan if the model dropped everything, keep the raw scenes.
142
+ // An empty plan is a valid strict-prompt result. Silently restoring every
143
+ // scene here would completely ignore the user's request.
136
144
  if (out.length === 0) {
137
- return { scenes, refined: false, kept: scenes.length, dropped: 0, merged: 0, note: "model dropped every scene; using raw scenes" };
145
+ return { scenes: [], refined: true, kept: 0, dropped, merged };
138
146
  }
139
147
  return { scenes: out, refined: true, kept: out.length, dropped, merged };
140
148
  }
@@ -76,10 +76,10 @@ export async function processSceneToClip(input) {
76
76
  await extractThumbnail({ videoPath, scene, outPath: thumbLocal, cropFilter });
77
77
  }
78
78
  // 5. Embed the document text for semantic search (skipped when the client can't embed).
79
+ const embeddingText = buildClipEmbeddingText(tags, tagResult.description, ctx.userNotes);
79
80
  let embedding;
80
81
  if (ctx.client.canEmbed) {
81
- const embedText = buildClipEmbeddingText(tags, tagResult.description, ctx.userNotes);
82
- [embedding] = await ctx.client.embedDocuments([embedText]);
82
+ [embedding] = await ctx.client.embedDocuments([embeddingText]);
83
83
  }
84
84
  const clip = {
85
85
  clip_id: clipId,
@@ -92,6 +92,7 @@ export async function processSceneToClip(input) {
92
92
  thumbnail_path: ctx.thumbStoredPath ? ctx.thumbStoredPath(clipId, thumbLocal) : thumbLocal,
93
93
  tags,
94
94
  description: tagResult.description,
95
+ embedding_text: embeddingText,
95
96
  embedding,
96
97
  taxonomy_version: ctx.client.taxonomyVersion ?? ACTIVE_TAXONOMY_VERSION,
97
98
  tier: ctx.client.tier,
@@ -113,21 +114,15 @@ export async function scanVideo(opts) {
113
114
  const probe = opts.probe ?? (await probeVideo(opts.videoPath));
114
115
  const windows = normalizeWindows(opts.windows, probe.duration_sec);
115
116
  const band = opts.durationBand ?? null;
116
- // Fit to the soft duration band BEFORE refine: the model then judges
117
- // clip-sized units (fewer, more meaningful keyframes) instead of raw cuts.
118
- // Callers passing pre-detected scenes (devcli estimate pass) must pre-fit
119
- // them — the fit is intentionally NOT re-run on supplied scenes since
120
- // fitting is not idempotent (a fitted 20s scene could re-merge toward the
121
- // target on a second pass).
117
+ // Keep detected shots separate until AFTER prompt refinement. This is
118
+ // important: judging a multi-shot duration span from one midpoint frame can
119
+ // approve unrelated opening/closing footage or miss text overlays in them.
122
120
  let scenes = opts.scenes ??
123
- fitScenesToDurationBand(await detectScenes(opts.videoPath, {
121
+ await detectScenes(opts.videoPath, {
124
122
  durationSec: probe.duration_sec,
125
123
  windows,
126
- // A duration band lifts the scene-split ceiling so raw cuts can merge
127
- // up to the requested clip length instead of being pre-chopped at 8s.
128
- ...(band ? { maxSceneSec: band.max_sec } : {}),
129
124
  ...opts.sceneOptions
130
- }), band);
125
+ });
131
126
  // Prompt-guided segmentation: let the guidance reshape which scenes become
132
127
  // clips (keep/drop/merge/dedup) before we spend tagging tokens on them.
133
128
  if (opts.ctx.guidancePrompt?.trim() && opts.ctx.refineSegmentation !== false) {
@@ -139,11 +134,15 @@ export async function scanVideo(opts) {
139
134
  workDir: opts.ctx.workDir,
140
135
  targetClipCount: opts.targetClipCount
141
136
  });
142
- // Refine merges can exceed the band max — re-enforce it (fit never merges
143
- // across the gaps refine's drops created, only contiguous spans).
144
- scenes = fitScenesToDurationBand(refined.scenes, band);
137
+ if (!refined.refined && refined.note) {
138
+ throw new Error(`Prompt-guided raw selection failed: ${refined.note}`);
139
+ }
140
+ scenes = refined.scenes;
145
141
  opts.onRefine?.(refined);
146
142
  }
143
+ // Assemble prompt-approved, contiguous shots into useful clip durations.
144
+ // fitScenesToDurationBand never crosses a dropped gap.
145
+ scenes = fitScenesToDurationBand(scenes, band);
147
146
  // Deterministic count/uniqueness cap: if more scenes survived than the target
148
147
  // (refine skipped >60 scenes, or the model kept more than asked), keep an
149
148
  // evenly-spaced — hence less duplicative — subset.
@@ -12,6 +12,9 @@ export class SpeechVoiceMismatchError extends Error {
12
12
  }
13
13
  export class ProviderQuotaExceededError extends Error {
14
14
  }
15
+ /** The provider refused the request or output because of its content/safety policy. */
16
+ export class ProviderSafetyError extends Error {
17
+ }
15
18
  export class ProviderWaitExceededError extends Error {
16
19
  }
17
20
  export class ProviderLeaseTimeoutError extends Error {
@@ -63,6 +66,54 @@ export function isHardQuotaError(details) {
63
66
  "resource_exhausted"
64
67
  ].some((pattern) => normalized.includes(pattern));
65
68
  }
69
+ export function isSafetyOrRefusalError(details) {
70
+ const normalized = details.toLowerCase();
71
+ return [
72
+ "content_policy_violation",
73
+ "content policy",
74
+ "safety policy",
75
+ "safety filter",
76
+ "safety_reason",
77
+ "finishreason\":\"safety",
78
+ "moderation_blocked",
79
+ "blocked_reason",
80
+ "blockedreason",
81
+ "prompt was blocked",
82
+ "moderation",
83
+ "moderated",
84
+ "responsible ai",
85
+ "refused",
86
+ "refusal",
87
+ "prohibited content",
88
+ "policy violation"
89
+ ].some((pattern) => normalized.includes(pattern));
90
+ }
91
+ export function providerFailureFromDetails(provider, operation, status, details) {
92
+ if (isSafetyOrRefusalError(details)) {
93
+ return new ProviderSafetyError(`${provider} refused this ${operation} because its safety or content policy was triggered${conciseProviderDetails(details)}. ` +
94
+ "Try revising the prompt or choose another provider/model.");
95
+ }
96
+ return new Error(`${provider} ${operation} returned ${status}${conciseProviderDetails(details)}`);
97
+ }
98
+ export function serializeProviderFailure(error) {
99
+ const message = error instanceof Error ? error.message : "AI provider request failed";
100
+ if (error instanceof ProviderSafetyError) {
101
+ return { message, type: "provider_safety_refusal", retryable: false, user_action: "Revise the prompt or choose another provider/model." };
102
+ }
103
+ if (error instanceof ProviderRateLimitError) {
104
+ return { message, type: "provider_rate_limited", retryable: true, retry_after_seconds: error.retryAfterSeconds, user_action: "Wait, then retry or choose another provider." };
105
+ }
106
+ if (error instanceof ProviderQuotaExceededError) {
107
+ return { message, type: "provider_quota_exceeded", retryable: false, user_action: "Check provider billing/quota or choose another provider." };
108
+ }
109
+ if (error instanceof ProviderAuthError) {
110
+ return { message, type: "provider_auth_error", retryable: false, user_action: "Check the saved provider API key." };
111
+ }
112
+ if (error instanceof ProviderTransientError) {
113
+ return { message, type: "provider_temporarily_unavailable", retryable: true, retry_after_seconds: error.retryAfterSeconds, user_action: "Retry shortly or choose another provider." };
114
+ }
115
+ return { message, type: "provider_request_failed", retryable: false };
116
+ }
66
117
  export function conciseProviderDetails(details) {
67
118
  const compact = details.replace(/\s+/g, " ").trim();
68
119
  return compact ? `: ${compact.slice(0, 500)}` : "";
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@mevdragon/vidfarm-devcli",
3
- "version": "0.20.12",
3
+ "version": "0.20.14",
4
4
  "description": "Local bridge for the Vidfarm Trackpad Editor. `vidfarm serve <template_id>` boots the FULL editor on localhost (disk-backed records/storage, free in-process render); edit composition.html on disk (Claude Code, Codex, etc.) and the browser live-morphs it.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -53,6 +53,7 @@
53
53
  "dev:cli": "tsx src/cli.ts",
54
54
  "build:frontend": "node scripts/build-homepage-client.mjs",
55
55
  "build:hyperframes-editor": "cd demo && node scripts/build.mjs",
56
+ "seo:refresh": "tsx scripts/generate-route-seo.ts",
56
57
  "operator:template-artifact": "node scripts/platform-operator-template-artifact.mjs",
57
58
  "bootstrap:staging-templates": "tsx scripts/bootstrap-staging-template-releases.ts",
58
59
  "migrate:sqlite-to-dynamodb": "tsx scripts/migrate-sqlite-to-dynamodb.ts",
@@ -60,7 +61,10 @@
60
61
  "rollback:prod-inplace": "bash scripts/rollback-prod-inplace.sh",
61
62
  "build": "npm run build:frontend && npm run build:hyperframes-editor && node -e \"require('node:fs').rmSync('dist', { recursive: true, force: true })\" && tsc -p tsconfig.json",
62
63
  "start": "node --import ./dist/src/instrument.js --enable-source-maps dist/src/index.js",
63
- "check": "tsc -p tsconfig.json --noEmit",
64
+ "check": "tsc -p tsconfig.json --noEmit && npm run check:skills",
65
+ "test:clips": "node --import tsx --test test/clip-curation.test.ts",
66
+ "check:skills": "node scripts/build-director-skill-rollup.mjs --check && node scripts/check-skill-routes.mjs",
67
+ "benchmark:editor-chat": "node --import tsx scripts/benchmark-editor-chat-harness.mjs",
64
68
  "cdk:deploy:prod-serverless": "npm run build && dotenv -e .env.production -- npx aws-cdk deploy --app 'node dist/infra/cdk/bin/vidfarm-prod.js'",
65
69
  "cdk:synth:staging-serverless": "npm run build && dotenv -e .env.staging -- npx aws-cdk synth --app 'node dist/infra/cdk/bin/vidfarm-serverless-staging.js'",
66
70
  "cdk:deploy:staging-serverless": "npm run build && dotenv -e .env.staging -- npx aws-cdk deploy --app 'node dist/infra/cdk/bin/vidfarm-serverless-staging.js'",