@mevdragon/vidfarm-devcli 0.14.0 → 0.16.0

package/SKILL.director.md

@@ -550,6 +550,14 @@ Send a stable `tracer` on export so retries are traceable and filterable in job
 | `vidfarm clips scan --cloud <video\|--url <url>> [--tracer <id>]` | temp-file presign/PUT/finalize + `POST /clips/scan` + poll | BACKUP: run the hunt on the deployed pipeline (bills AWS compute only) |
 | `vidfarm clips search "…"` / `clips match "…"` / `clips list` / `clips sources` | (local library; NL→criteria via local agent or provider key) | search/reuse the clip library |
 | `vidfarm clips preset list\|run\|save` / `clips export <ids…> --to <dir>` | (local library) | saved queries; copy clip MP4s out |
+| `vidfarm lint <dir\|composition.html>` | (local static validation) | pre-publish composition check: timing, overlaps, preset names, media src |
+| `vidfarm stills <dir> [--at 0,2.5,…]` | (local in-process render of PNG frames) | visually verify an edit without a full render |
+| `vidfarm doctor` | (local environment triage) | check ffmpeg/node/keys/agent CLI/poisoned env before debugging anything else |
+| `vidfarm skills list\|add <name>\|update` | `GET /skill-pack/index.json` · `/skill-pack/:name/*` | install/refresh skill packs (see "Skill packs — import on demand") |
+| `vidfarm tts "…" --engine local` / `vidfarm stt <file> --engine whisper` | (keyless LOCAL engines: Kokoro-82M TTS, whisper.cpp STT) | narration + word-timestamp transcripts with zero keys and zero accounts |
+| `vidfarm remove-background <video\|image>` | (local ONNX matting — free) | transparent-subject media for occlusion captions/cutouts |
+| `vidfarm capture <url>` | (local headless-Chrome capture) | website screenshots/assets for website-to-video flows |
+| `vidfarm beats <audio>` | (local beat detection → JSON) | music beat timestamps to sync cuts/transitions/captions |
 
 **Frontend URLs are first-class output.** Every command that has a human-openable page (editor, discover, approved-post preview, share link) prints that prod frontend URL as a highlighted line. `render --wait` polls to completion and prints the final MP4 URL; `approve` prints the approved-post `share_url`. Add `--json` to any command for pure JSON (agent-friendly, no banners).
 
@@ -611,6 +619,35 @@ Flags: `--port` (default 3000), `--dir` (default `./.vidfarm-local`), `--key` (b
 
 To *analyze* the source media locally (videos, transcript, recurring cast), read the `video-context.json` / `cast.json` routes, or pull a media URL with `vidfarm download <url>`.
 
+## Skill packs — import on demand (HyperFrames-grade authoring power)
+
+This skill stays lean on purpose. Deep authoring craft lives in **skill packs** — vidfarm-scrubbed snapshots of the HyperFrames skill suite plus vidfarm's own media pack — vendored on the Vidfarm host and installed only when a task needs them. Never install skills from upstream vendor orgs or third-party registries; the vidfarm mirror is the source (`vidfarm skills add <name>` fetches `GET /skill-pack/:name/*` with hash verification into `.agents/skills/` + a `.claude/skills/` link, pinned in `skills-lock.json`; `vidfarm skills list` shows what is available/installed; `vidfarm skills update` refreshes pins).
+
+Import by task:
+
+| When the task needs… | Install |
+|---|---|
+| the raw composition HTML contract (data-* attrs, tracks, clips, sub-compositions, determinism rules) | `vidfarm skills add hyperframes-core` |
+| motion design: animation rules, multi-phase scene blueprints, transition doctrine, runtime adapters (GSAP/Lottie/Three/Anime/CSS/WAAPI/TypeGPU) | `vidfarm skills add hyperframes-animation` |
+| seek-safe keyframe patterns (FLIP, paths, masks, SVG draw/morph, 3D depth) | `vidfarm skills add hyperframes-keyframes` |
+| creative direction: design specs, palettes, typography, house style, narration craft | `vidfarm skills add hyperframes-creative` |
+| the HyperFrames CLI dev loop (init/lint/validate/inspect/snapshot/preview/render) | `vidfarm skills add hyperframes-cli` |
+| narration/BGM/SFX/media resolution (the shared audio engine) | `vidfarm skills add vidfarm-media` |
+| embedded/cinematic captions with subject occlusion (32-identity catalog) | `vidfarm skills add embedded-captions` |
+| graphic overlay cards on existing talking-head footage | `vidfarm skills add talking-head-recut` |
+| a product launch/promo video end-to-end | `vidfarm skills add product-launch-video` |
+| a faceless topic explainer end-to-end | `vidfarm skills add faceless-explainer` |
+| a website tour/showcase video end-to-end | `vidfarm skills add website-to-video` |
+| a short motion graphic (kinetic type, stat hit, logo sting, lower-third) | `vidfarm skills add motion-graphics` |
+| an interactive slideshow/deck | `vidfarm skills add slideshow` |
+| any other multi-scene composition (fallback workflow) | `vidfarm skills add general-video` |
+
+Ground rules:
+
+- **Local coding agents** (Claude Code / Codex on a `vidfarm serve` box or a pulled fork) get the FULL packs — install, read, and follow them. Scripted/GSAP compositions authored this way render correctly through `vidfarm render` (local and cloud); note that the **web editor strips `<script>` on save**, so keep compositions that must round-trip through the browser editor declarative (the built-in Ken Burns / transitions / animated-captions vocabulary).
+- **The web copilot** never installs packs — it has a `load_skill` tool that reads the same content on demand from the vidfarm mirror. Nothing to do there; it is already wired.
+- Vidfarm-managed environments set `HYPERFRAMES_SKIP_SKILLS=1` and `HYPERFRAMES_NO_TELEMETRY=1`, so `npx hyperframes init` never overwrites the vidfarm-scrubbed packs and the bundled CLI never phones home. Do not run `hyperframes auth`, `hyperframes cloud`, `hyperframes publish`, `hyperframes play`, or `hyperframes feedback` — vidfarm's own render/share/telemetry surfaces cover all of them.
+
 ## What NOT to do
 
 - Do **not** manipulate composition HTML by string concatenation. Always parse, edit, and re-serialize the DOM. The editor and runtime rely on `data-start`, `data-duration`, `data-track-index`, `data-hf-id`, `data-composition-id` attributes being well-formed.

package/SKILL.platform.md

@@ -266,7 +266,7 @@ Configuration: `GHOSTCUT_KEY` and `GHOSTCUT_SECRET` env vars per stage. Pass-thr
 
 The chat model uses the **caller's provider keys**. No Vidfarm-owned model calls happen in this path.
 
-Beyond the Editor Action API, the chat harness registers shared tools defined in **two synced copies** (`src/app.ts` for the in-process dev server and `infra/lambda/editor-chat.ts` for the deployed Lambda — keep them in lockstep): `http_request` (allowlisted REST/primitive calls), `frontend_action` (tracer/thread state), `video_context`, and `browse_files`. `browse_files` reads **and writes** the caller's My Files library — `action=list`/`search`/`read`/`write`/`annotate`. `write` saves a text file (md/txt/csv/json/srt/vtt) via `POST /me/attachments/upload`, or **imports a durable media URL** into the library when `source_url` is passed (how a generated `character_sprite_card.png` gets persisted instead of stranding in a job result). `annotate` sets free-form metadata `notes` on any entry (`PATCH /me/attachments/:id`); notes are **vector-embedded server-side** (BYOK, same gemini→openai auto-pick seam as clip search; no key fail-softs to keyword-only) and `search` runs the hybrid keyword+semantic lookup over every file's name/folder/notes via `POST /me/attachments/search` — the My Files mirror of `/clips/search`. Write+annotate are what let the copilot persist Getting Started context (About.md, awareness-levels.md, persuasive-angles.md, ad-hooks.md) and character-consistency pairs (`about_character.md` + `character_sprite_card.png`) namescoped under a product/offer folder; the system prompt in `src/editor-chat.ts` drives when to save and annotate. The devcli mirrors are `vidfarm put-file [--notes]`, `annotate-file`, and `files --search`.
+Beyond the Editor Action API, the chat harness registers shared tools defined in **two synced copies** (`src/app.ts` for the in-process dev server and `infra/lambda/editor-chat.ts` for the deployed Lambda — keep them in lockstep): `http_request` (allowlisted REST/primitive calls), `frontend_action` (tracer/thread state), `video_context`, `load_skill`, and `browse_files`. `load_skill` is the copilot's knowledge-on-demand seam: it fetches vendored skill-pack content (SKILL.md or a reference file) from the host's public `/skill/:name` + `/skill-pack/:name/files/*` mirror routes (backed by `.agents/skills/`, shipped in the api Docker image), size-capped, with the pack catalog + one-liners defined once in `EDITOR_CHAT_SKILL_PACKS` (`src/editor-chat.ts`) and shared by both copies. The stream-loop guardrails (`redactUnverifiedJobIds`, `buildEditorChatGuardrailNotice`, async-job handoff text) also live in `src/editor-chat.ts` and are imported by both copies — never re-implement them locally. `editor_action`'s `replace_composition_html` is lint-gated in the tool `execute` via `src/services/composition-lint.ts` (a rejected result carries `rejected: true` + `lint_errors` and omits `action_type`, so the browser never applies it). `browse_files` reads **and writes** the caller's My Files library — `action=list`/`search`/`read`/`write`/`annotate`. `write` saves a text file (md/txt/csv/json/srt/vtt) via `POST /me/attachments/upload`, or **imports a durable media URL** into the library when `source_url` is passed (how a generated `character_sprite_card.png` gets persisted instead of stranding in a job result). `annotate` sets free-form metadata `notes` on any entry (`PATCH /me/attachments/:id`); notes are **vector-embedded server-side** (BYOK, same gemini→openai auto-pick seam as clip search; no key fail-softs to keyword-only) and `search` runs the hybrid keyword+semantic lookup over every file's name/folder/notes via `POST /me/attachments/search` — the My Files mirror of `/clips/search`. Write+annotate are what let the copilot persist Getting Started context (About.md, awareness-levels.md, persuasive-angles.md, ad-hooks.md) and character-consistency pairs (`about_character.md` + `character_sprite_card.png`) namescoped under a product/offer folder; the system prompt in `src/editor-chat.ts` drives when to save and annotate. The devcli mirrors are `vidfarm put-file [--notes]`, `annotate-file`, and `files --search`.
 
 ## Primitives
 
@@ -322,9 +322,14 @@ Per-stage config is loaded via `.env.<stage>`. Critical variables:
 - `GHOSTCUT_KEY`, `GHOSTCUT_SECRET` — GhostCut credentials
 - `RAPIDAPI_KEY` — shared RapidAPI credential used by the `video_download` and `image_remove_background` primitives. Same key is set in both `.env.staging` and `.env.production`.
 - Provider fallback keys for platform-owned features that don't have a caller yet (bootstrap only)
+- `HYPERFRAMES_NO_TELEMETRY=1` + `HYPERFRAMES_SKIP_SKILLS=1` — baked into every vidfarm-managed environment (api Docker image, CDK `hyperframesLambdaEnvironment`, devcli serve/spawns). The bundled hyperframes CLI defaults telemetry ON and auto-refreshes skills from its upstream GitHub org; these kill both. Never unset them, and never run `hyperframes auth`/`cloud`/`publish`/`play`/`feedback` — those are third-party account/hosting surfaces vidfarm replaces.
 
 Never commit `.env.staging` or `.env.production` — both are gitignored.
 
+## Skill packs (vendored mirror)
+
+`.agents/skills/` holds the vidfarm-scrubbed skill packs (the HyperFrames suite with every upstream-vendor dependency replaced by vidfarm primitives, plus `vidfarm-media` — the shared audio engine implementing the neutral `audio_request.json` → `audio_engine_meta.json` contract). `.claude/skills/*` are symlinks into it. The api serves the packs publicly: `GET /skill/:name` (SKILL.md), `GET /skill-pack/index.json` (catalog), `GET /skill-pack/:name/manifest.json` (file list + sha256), `GET /skill-pack/:name/files/<path>` (bytes, path-guarded) — `src/app.ts` next to `PUBLIC_SKILL_FILES`. Consumers: `vidfarm skills list|add|update` (devcli installer, writes `skills-lock.json` with reproducible folder hashes) and the web copilot's `load_skill` tool. When editing a pack, edit `.agents/skills/<name>` directly (the canonical copy), keep it free of upstream vendor accounts/endpoints, and re-run `vidfarm skills update`-side hash pinning as needed.
+
 ## Deployment
 
 CDK stacks are under `infra/cdk/`: