@mevdragon/vidfarm-devcli 0.2.12 → 0.2.14

package/.env.example

@@ -1,24 +1,60 @@
 NODE_ENV="development"
 PORT="3000"
+MOCK_PROVIDER_RESPONSES="false"
+MAX_ACTIVE_JOBS_PER_CUSTOMER="30"
+PROVIDER_LEASE_INLINE_WAIT_SECONDS="5"
+PROVIDER_LEASE_STEP_WAIT_SECONDS="60"
+PROVIDER_LEASE_MAX_STEP_WAIT_SECONDS="43200"
+MAX_JOB_WALL_CLOCK_SECONDS="86400"
 
 OPENAI_API_KEY=""
 OPENROUTER_API_KEY=""
 GEMINI_API_KEY=""
 PERPLEXITY_API_KEY=""
+GHOSTCUT_KEY=""
+GHOSTCUT_SECRET=""
 
-# Optional. Production CDK wires these automatically for the EC2 app and ZIP Lambda.
+# Optional. Production CDK wires these automatically. Browser-facing Lambda calls still callback to PUBLIC_BASE_URL for auth.
 VIDFARM_ZIP_LAMBDA_URL=""
 VIDFARM_ZIP_LAMBDA_SECRET=""
 VIDFARM_BILLING_URL=""
 VIDFARM_BILLING_SECRET=""
 VIDFARM_RATE_LIMITS_TABLE_NAME=""
+# Optional production CDK overrides. Leave empty to use deterministic per-account/per-region names.
+VIDFARM_STORAGE_S3_BUCKET_NAME=""
 
 # Optional when calling a hosted Vidfarm API directly.
 VIDFARM_USER_ID=""
 VIDFARM_API_KEY=""
+# Optional director override for production debugging. Send as x-superagency-key.
+SUPERAGENCY_KEY=""
 
 # Optional local overrides. The CLI will choose sensible local defaults.
 PUBLIC_BASE_URL="http://localhost:3310"
 # Optional public browser-facing chat endpoint. Leave empty locally unless you want the UI to hit a separate chat host directly.
 PUBLIC_EDITOR_CHAT_URL=""
+
+# Optional Sentry setup. SENTRY_DSN powers the Hono Node backend.
+# SENTRY_LAMBDA_DSN powers the CDK-managed AWS Lambdas.
+# PUBLIC_SENTRY_DSN powers the browser React bundles and may be the same DSN or a separate browser project DSN.
+SENTRY_DSN=""
+SENTRY_ENVIRONMENT="development"
+SENTRY_RELEASE=""
+SENTRY_TRACES_SAMPLE_RATE=""
+SENTRY_ENABLE_LOGS="true"
+SENTRY_SEND_DEFAULT_PII="false"
+SENTRY_DEBUG_ROUTE="false"
+SENTRY_LAMBDA_DSN=""
+SENTRY_LAMBDA_TRACES_SAMPLE_RATE="1.0"
+PUBLIC_SENTRY_DSN=""
+PUBLIC_SENTRY_TRACES_SAMPLE_RATE=""
+PUBLIC_SENTRY_REPLAYS_SESSION_SAMPLE_RATE=""
+PUBLIC_SENTRY_REPLAYS_ON_ERROR_SAMPLE_RATE=""
+PUBLIC_SENTRY_ENABLE_LOGS="true"
+PUBLIC_SENTRY_SEND_DEFAULT_PII="true"
+PUBLIC_SENTRY_TRACE_PROPAGATION_TARGETS=""
 REMOTION_MODE="local"
+
+
+GHOSTCUT_KEY="_________"
+GHOSTCUT_SECRET="_________"

package/README.md

@@ -1,200 +1,44 @@
-# Vidfarm Dev CLI
+# Vidfarm
 
-`@mevdragon/vidfarm-devcli` is the local development toolkit for third-party Vidfarm template authors.
+Vidfarm is a serverless composition studio for short-form video. Directors fork templates, edit compositions on a timeline in the **Trackpad Editor**, publish MP4s via a shared Remotion Lambda, and share results.
 
-This package is intentionally scoped to:
+Everything runs on AWS Lambda + DynamoDB + S3. There is no long-lived worker. It scales to zero at idle.
 
-- building templates under `templates/*`
-- running the local Vidfarm API and worker
-- testing template operations as async jobs
-- validating template metadata, skills, and smoke test payloads
-- rendering Remotion locally when a template uses Remotion
+## Where to start
 
-It is intentionally not documentation for platform internals, production deployment, admin release flow, or shared cloud infrastructure.
+- **Using Vidfarm as a director** → [`SKILL.director.md`](./SKILL.director.md)
+  Login, forking templates, editing in the Trackpad Editor, auto-decompose, publishing, sharing, billing.
 
-## Quick Start
+- **Understanding how Vidfarm works internally** → [`SKILL.platform.md`](./SKILL.platform.md)
+  Architecture, data model, composition runtime, publish pipeline, deployment, migrations.
 
-1. Create a local `.env` from the example file.
-2. Add at least one AI provider key.
-3. Start the local CLI runtime.
+## Repository layout
 
-```bash
-cp .env.example .env
-npx @mevdragon/vidfarm-devcli dev --port 3310 --reset
-```
+- `src/` — API server (Hono, deployed as container Lambda)
+- `demo/` — Trackpad Editor React app (deployed to CloudFront under `/editor/`)
+- `infra/cdk/` — CDK stacks for staging and production
+- `infra/lambda/` — Lambda entry points (api, editor-chat, primitive-media, job-runner, billing, ready-post-zip, job-sweeper)
+- `templates/` — Draft composition templates checked into the repo
+- `scripts/` — One-shot ops and migration scripts
+- `auto-create-hyperframe-templates/` — Extractor prompts / schemas for generating new templates from source footage
 
-Useful commands:
+## Deploy
 
 ```bash
-npx @mevdragon/vidfarm-devcli dev --port 3310 --reset
-npx @mevdragon/vidfarm-devcli session
-npx @mevdragon/vidfarm-devcli validate-template --template-id template_0000
-npx @mevdragon/vidfarm-devcli copy-reference-template --template-dir ./src/vidfarm_template_0000
-npx @mevdragon/vidfarm-devcli autocreate-template ./drafts/my_format ./templates/vidfarm_template_my_format
-npx @mevdragon/vidfarm-devcli generate-template --slug-id template_0001 --template-dir ./templates/vidfarm_template_0001
-npx @mevdragon/vidfarm-devcli analyze-viral-dna --template-dir ./templates/vidfarm_template_0001
-npx @mevdragon/vidfarm-devcli analyze-visual-dna --template-dir ./templates/vidfarm_template_0001
-npx @mevdragon/vidfarm-devcli presign-preview-media --file ./drafts/preview/example.png --directory drafts/preview
-```
-
-After install, the package exposes both `vidfarm` and `vidfarm-devcli` as local binaries.
-
-`autocreate-template` prepares the auto-create harness run folder, stages media from the input path, snapshots the current harness files, scaffolds the output template, and writes an agent handoff prompt at `auto-create-templates/runs/<run_id>/agent_prompt.md`. Keep improving `auto-create-templates/AUTO_CREATE_TEMPLATES.md`; each new run uses the current harness and stores a per-run snapshot for traceability.
-
-## What The Local CLI Does
-
-The local CLI runs the same template contract used by the hosted platform, but with local-first defaults:
-
-- local SQLite state under `.vidfarm/local`
-- local filesystem storage instead of cloud object storage
-- local Remotion rendering
-- a seeded developer session for local testing
-- provider keys sourced from your local `.env`
-- mocked provider responses only when no real provider keys are present
-
-Use `vidfarm session` to print reusable auth headers and a sample `curl` request for the local REST API.
-
-For hosted preview-media uploads, use `vidfarm-devcli presign-preview-media`. It calls the authenticated Vidfarm API with `VIDFARM_API_KEY`, mints a presigned PUT URL under `developer/<user_id>/*`, stores each uploaded file under a UUID path segment while preserving the original filename, uploads the file automatically when `--file` is provided, and returns a public-read media URL for the uploaded object. Public readability is granted by bucket policy for `developer/*`, not by public write or list access.
-
-## What The Hosted Platform Expects From Templates
-
-Vidfarm treats templates as async job producers, not as ad hoc endpoints.
-
-Each template should provide:
-
-- stable template metadata
-- a `SKILL.md` file for users and agents
-- a TypeScript entrypoint at `src/template.ts`
-- one or more named operations
-- workflow functions behind those operations
-- a `smokeTestPayload` for each operation
-- `about.viral_dna` and `about.visual_dna`
-
-At runtime, the platform uses the template wrapper to expose a consistent API shape:
-
-- `GET /api/v1/templates/:templateId`
-- `GET /api/v1/templates/:templateId/skill`
-- `POST /api/v1/templates/:templateId/config`
-- `POST /api/v1/templates/:templateId/operations/:operationName`
-- `GET /api/v1/templates/:templateId/jobs/:jobId`
-- `GET /api/v1/templates/:templateId/jobs/:jobId/logs`
-
-Operations should return work through the job system rather than inventing one-off synchronous routes.
-
-## Approval Flow
-
-Third-party developers build and validate templates locally first.
-
-After handoff, a Vidfarm admin reviews the template before it is made available on the hosted platform. Template authors should assume there is an approval step between local development and production availability.
-
-The developer handoff ends at:
-
-- `git push` of the template code
-- template source registration or registration update
-- for an already-registered template, publishing the exact GitHub commit SHA that should enter review
-
-The developer handoff does not include:
-
-- importing the source into the live platform
-- activating a release
-- making the template public/live
-
-If a developer receives `403 Admin access required` while trying to import or activate a registered source, that is expected. Those routes are for platform admins. Developers should use the publish-update flow to submit an exact commit SHA for review instead.
-
-## Template Deploy Cycle
+# staging first, always
+npm run cdk:synth:staging-serverless
+npm run cdk:deploy:staging-serverless
 
-When someone says "template deploy cycle" in this repo, they mean this exact sequence:
-
-1. commit and `git push` the repo that contains the template folder, such as `templates/vidfarm_template_<slug>/` inside a monorepo or a standalone template repo
-2. as the platform admin, pull/import that template commit into Vidfarm
-3. approve and activate the imported template release in the hosted platform
-4. run the prod deploy script if platform-runtime code changed, or reuse the current prod image if the change was template-only and you just need a formal deploy-cycle restart
-
-For `template_0000`, that usually means:
-
-```bash
-git -C templates/vidfarm_template_0000 push origin production
-# third-party developer registration step
-node dist/src/cli.js register-source-hosted \
-  --env-file .env.production \
-  --template-id 4c7a7e1a-7f35-4f30-9f86-9c8a63c7f2db \
-  --slug-id template_0000 \
-  --repo-url https://github.com/your-org/your-template-repo \
-  --branch production \
-  --template-module-path templates/vidfarm_template_0000/src/template.ts
-
-# then run the platform operator command
-node dist/src/cli.js deploy-template-cycle \
-  --env-file .env.production \
-  --stack-name VidfarmProdStack \
-  --template-id 4c7a7e1a-7f35-4f30-9f86-9c8a63c7f2db \
-  --slug-id template_0000 \
-  --repo-url https://github.com/your-org/your-template-repo \
-  --branch production \
-  --template-module-path templates/vidfarm_template_0000/src/template.ts
+# then production
+npm run cdk:deploy:prod-serverless
 ```
 
-The important distinction is:
-
-- third-party developers ship template repo changes plus registration metadata, and can publish an exact commit SHA for an existing registered template
-- platform admins import and activate those template repo changes
-- new source registrations land in the hosted review queue before approval
-- developer-published commit SHAs become reviewable releases while an older active release stays live
-- platform code changes ship through the root prod deploy script
-- multiple templates can live in one git repo as long as each template has its own folder and its `template_module_path` points at that folder's `src/template.ts`
-
-## Authentication Context
-
-When calling a hosted Vidfarm API directly, authenticated requests use:
-
-- `vidfarm-user-id`
-- `vidfarm-api-key`
-
-The API key comes from Vidfarm login and settings, not from platform infrastructure secrets.
-If you need to discover the authenticated customer ID first, resolve it from `GET /api/v1/user/me` and then send both headers on subsequent requests.
-
-For hosted template source registration, treat `template_module_path` as repo-relative and point it at the template's TypeScript entrypoint, for example `templates/vidfarm_template_hooks/src/template.ts`. Each template folder must start with `vidfarm_template_`. The import flow builds the repo and loads the compiled `src/template.js` sibling at activation time.
-
-For hosted registration, use `register-source-hosted`. It talks to the Vidfarm REST API directly, so the source record lands in the production database that backs the hosted review queue. `register-source-prod` remains as a compatibility alias for internal/operator workflows.
-
-For updates to an already-registered template, use `publish-update-hosted` with the exact reviewed commit SHA. The CLI packages that commit locally into a signed artifact, uploads it to platform-controlled S3 through the Vidfarm API, and then creates or refreshes the pending release for admin review without granting developers approval or activation rights.
-
-```bash
-node dist/src/cli.js publish-update-hosted \
-  --env-file .env \
-  --template-id c7af2cd8-f032-45fa-9d4d-7af0e85cf4ef \
-  --slug-id vidfarm_template_reddit_brainrot \
-  --repo-url https://github.com/mevdragon/vidfarm_templates.git \
-  --branch main \
-  --template-module-path src/vidfarm_template_reddit_brainrot/src/template.ts \
-  --commit-sha 5e05a90a05a40105cc6e3b6a06774fdfb60c1893
-```
-
-The hosted publish path no longer depends on server-side GitHub access for template updates. Prod imports the uploaded artifact from S3 during review/import.
-
-For local CLI usage, the `session` command gives you a local seeded session so you can test without manually wiring auth state.
-
-## Provider Keys
-
-Templates should work with the developer's own upstream AI keys.
-
-Typical local `.env`:
-
-```env
-GEMINI_API_KEY=""
-OPENAI_API_KEY=""
-OPENROUTER_API_KEY=""
-PERPLEXITY_API_KEY=""
-```
-
-You usually only need one provider key to develop and test a template locally.
-
-## Remotion
+Never call bare `cdk deploy` — the npm scripts load the correct `.env.<stage>` and select the right stack.
 
-For third-party template authors, Remotion is local.
+## Deprecated: `@mevdragon/vidfarm-devcli`
 
-If your template uses `ctx.remotion`, develop and validate it locally through the CLI. This package does not document or require shared cloud Remotion infrastructure.
+The `vidfarm-devcli` CLI (`src/cli.ts`) is **deprecated and no longer maintained**. It was built around the old REST-API template model — templates with `src/template.ts`, workflows, operations, viral DNA extraction, and the hosted publish/approve/activate flow. That entire paradigm has been superseded by the browser-based **Trackpad Editor** at `https://vidfarm.cc/editor/`.
 
-## Next Read
+The CLI still exists in the repo and may partially work, but it will print a deprecation banner on every invocation. No fixes will land. Do not build new tooling on it.
 
-For the external template-author workflow, continue with [GETTING_STARTED.developers.md](./GETTING_STARTED.developers.md) and the example template in [`templates/vidfarm_template_0000`](./templates/vidfarm_template_0000).
+See `SKILL.platform.md` for the current architecture.

package/SKILL.director.md

@@ -0,0 +1,293 @@
+---
+name: vidfarm-director
+description: Use Vidfarm as a director. Fork a template into a composition, edit it in the Trackpad Editor (timeline-based like Premiere/DaVinci), auto-decompose source video into scenes, publish to MP4, and share. Includes login, provider keys, discovery, publishing, versioning, and billing.
+---
+
+# Vidfarm Director
+
+Vidfarm is a **video composition studio**. Directors fork a published template, edit it on a timeline in the **Trackpad Editor**, publish to MP4, and share.
+
+Use this skill when the user wants to:
+
+- log in with OTP and save their AI provider keys
+- discover published composition templates
+- fork a template into an editable composition
+- edit the timeline (add/remove/reorder layers, retime clips, swap media)
+- auto-decompose a raw source video into scenes with AI
+- publish a fork to MP4 and share the result
+- browse version history and revert
+- manage sharing/visibility
+
+Do **not** use this skill to author new templates from scratch, deploy platform infrastructure, or discuss internal architecture — those belong to `SKILL.platform.md` (if it exists) or a separate developer workflow.
+
+## Mental model
+
+A Vidfarm composition is an **HTML document + JSON metadata**. Each layer (video clip, audio, image, text, caption) is a timed DOM element with `data-start`, `data-duration`, `data-track-index` attributes. The Trackpad Editor is a timeline UI over this document, similar to Premiere or DaVinci.
+
+Every director workflow goes through three concepts:
+
+1. **Template** — a published composition anyone can fork. Templates live at `/hyperframes/:slug` and have a source video, a decomposed timeline, and metadata (viral DNA, captions, scenes).
+2. **Composition fork** — a private editable copy of a template. Every fork has a `forkId`, an owner, a working state, and a version history. The Trackpad Editor operates on a fork.
+3. **Publish** — render the current fork state to MP4 via a Lambda render job. Each publish snapshots the current working state as an immutable version.
+
+The Trackpad Editor is the primary surface. Most director actions happen there. This skill's REST endpoints exist for automation, CI, and headless flows.
+
+## Base URL and auth
+
+The current API base URLs are:
+
+- staging: `https://staging.vidfarm.cc`
+- production: `https://vidfarm.cc`
+
+Every browser action is authenticated by the session cookie set at `/login`. Every headless action needs the `vidfarm-api-key` header set to the caller's API key (from Settings → API Keys).
+
+```
+vidfarm-api-key: vf_key_<32 hex>
+```
+
+## Login flow
+
+Directors log in with OTP:
+
+1. `POST /api/v1/auth/otp/request { email }` — sends a code to email
+2. `POST /api/v1/auth/otp/verify { email, code }` — sets session cookie, returns `{ user, apiKey }`
+
+Save the `apiKey` in the client and use it for automation.
+
+## Provider keys
+
+Vidfarm executes many features (smart auto-decompose, editor chat) with the **caller's own AI provider keys**. Directors save keys once via:
+
+- UI: Settings → Provider Keys
+- API: `PUT /api/v1/user/me/provider-keys { openai, gemini, openrouter, perplexity }`
+
+Priority: Gemini → OpenAI → OpenRouter. If a director has no keys, smart decompose falls back to a deterministic time-based split.
+
+## Discover templates
+
+Templates are listed on the Vidfarm homepage under `/hyperframes/`. Each has a slug, a preview MP4, source video metadata, and viral DNA.
+
+- Browser: browse the homepage
+- API: `GET /api/v1/hyperframes` — returns template summaries
+
+Each template exposes a public preview:
+
+- `GET /hyperframes/:slug` — HTML preview
+- `GET /hyperframes/:slug/composition` — composition HTML
+- `GET /hyperframes/:slug/composition.json` — composition metadata (scenes, captions, viral DNA)
+
+Directors don't edit templates directly. They fork.
+
+## Fork a template
+
+```
+POST /api/v1/compositions
+Content-Type: application/json
+
+{ "source_slug_id": "<template slug>", "title": "Optional title" }
+```
+
+Returns the new fork's metadata:
+
+```
+{ "id": "<forkId>", "sourceSlugId": "...", "title": "...", "visibility": "private", "latestVersion": 0, ... }
+```
+
+Take the returned `forkId` and open the Trackpad Editor at:
+
+```
+/editor/?fork=<forkId>
+```
+
+The editor loads `composition.html` and `composition.json` from the fork's working state.
+
+## Edit in the Trackpad Editor
+
+The editor is a full timeline surface:
+
+- **Timeline** — multi-track, drag/trim/split clips, group/ungroup layers
+- **Inspector panel** — per-layer property editor (text, color, font, position, size, media source, playback start, volume)
+- **Canvas frame overlay** — 8-handle resize + WYSIWYG positioning
+- **Chat panel** — AI assistant that can programmatically edit the composition (13 action types: `add_layer`, `remove_layer`, `set_layer_start`, `set_layer_duration`, `group_layers`, `replace_composition_html`, ...). The assistant uses the caller's saved provider keys.
+- **Auto-save** — every commit writes composition.html + composition.json to the working state. No explicit save button.
+
+The editor persists to:
+
+- `PUT /api/v1/compositions/:forkId/composition.html` — replace HTML
+- `PATCH /api/v1/compositions/:forkId/composition.json` — patch metadata
+
+Automation that needs to bypass the UI can call these endpoints directly.
+
+## Auto-decompose
+
+If the fork's source is a raw video (not a pre-decomposed template), call auto-decompose to split it into scenes:
+
+```
+POST /api/v1/compositions/:forkId/auto-decompose
+Content-Type: application/json
+
+{ "mode": "smart" | "time-slice", "html": "<optional current html>", "requested_scenes": null, "target_scene_seconds": null, "user_prompt": "optional guidance" }
+```
+
+Two modes:
+
+- **`smart`** — uses the caller's Gemini/OpenAI/OpenRouter key. Samples frames, extracts scenes with labels and viral notes, extracts on-screen captions with positions, and detects viral DNA (hook, retention, payoff). Takes 30-60 seconds. Auto-fires GhostCut for subtitle removal in the background.
+- **`time-slice`** — deterministic equal-duration split. Instant. No AI provider needed.
+
+Response:
+
+```
+{ "ok": true, "mode": "smart", "scene_count": 6, "caption_count": 4, "duration_seconds": 12.4, "ghostcut_pending": true, "ghostcut_task_id": "..." }
+```
+
+If `ghostcut_pending: true`, poll `POST /api/v1/compositions/:forkId/ghostcut-poll` until status transitions to `done` or `failed`.
+
+## Publish
+
+Publishing renders the fork's current working state to an MP4 using shared Remotion Lambda.
+
+```
+POST /api/v1/compositions/:forkId/publish
+Content-Type: application/json
+
+{ "title": "optional", "version": null, "tracer": "optional-idempotency-key" }
+```
+
+Returns `{ render_id, job_id, snapshot_version }`. Poll:
+
+```
+GET /api/v1/compositions/:forkId/renders/:renderId
+```
+
+Response includes `{ phase: "RUNNING" | "SUCCEEDED" | "FAILED", progress, output_url, cost_usd }`. On success, `output_url` is the durable MP4 URL.
+
+Every publish creates an immutable version snapshot at `versions/<N>/composition.html` and `versions/<N>/composition.json`.
+
+## Version history
+
+```
+GET /api/v1/compositions/:forkId/versions?limit=100
+```
+
+Returns the version list with `{ version, reason, message, created_at }`. Reason is `publish`, `clone`, `manual`, or `migration`.
+
+To view a past cut without reverting:
+
+```
+GET /api/v1/compositions/:forkId/versions/:version/composition.html
+```
+
+To revert the working state to a version, `PUT` the historical HTML/JSON back to the working keys.
+
+## Sharing and visibility
+
+Every fork has a visibility: `private` (default), `unlisted` (legacy), or `public` (view-only).
+
+```
+PATCH /api/v1/compositions/:forkId/visibility { "visibility": "public" }
+```
+
+Public forks are accessible at `/editor/?fork=<forkId>` (view-only for non-owners). Public visibility is the simplest sharing surface.
+
+Fine-grained sharing uses per-user permissions and share links:
+
+- `POST /api/v1/compositions/:forkId/permissions { granteeType, granteeId, role, expiresAt? }` — grant a specific user access. Roles: `viewer`, `editor`, `publisher`.
+- `POST /api/v1/compositions/:forkId/share-links { role, expiresAt? }` — create a token URL. Anyone with the token gets the role.
+- `GET /api/v1/compositions/shared/:token` — access via share link
+- `DELETE /api/v1/compositions/:forkId/permissions/:permissionId` — revoke a grant
+- `DELETE /api/v1/compositions/:forkId/share-links/:token` — revoke a link
+
+## Cloning a fork
+
+Directors can clone another fork (their own or a shared one) as a new starting point:
+
+```
+POST /api/v1/compositions { "parent_fork_id": "<forkId>", "parent_version": <N or omit for latest> }
+```
+
+The new fork inherits the parent's HTML/JSON at the specified version and creates a version-1 snapshot with `reason: "clone"`.
+
+## Delete a fork
+
+```
+DELETE /api/v1/compositions/:forkId
+```
+
+Soft-delete only. The fork's `deletedAt` is set. Versions and share links are preserved for audit.
+
+## Billing
+
+Vidfarm charges directly in USD from the caller's wallet. There are no credits.
+
+- **Wallet top-up** — Stripe checkout via `POST /api/v1/user/me/wallet/checkout`
+- **Balance** — `GET /api/v1/user/me/wallet`
+- **Cost anchors**:
+  - Publish (Remotion Lambda render): typically $0.01 – $0.10 per MP4 depending on length/resolution
+  - Auto-decompose smart mode: pass-through of caller's AI provider spend
+  - Auto-decompose time-slice: free
+  - GhostCut subtitle removal: ~$0.10 per 30 seconds of source video
+
+Framing:
+
+- Vidfarm is optimized for cost efficiency, not markup
+- Total director cost per video is typically far below alternatives
+- Platform retains a small safety buffer for operational correctness
+
+## Automation patterns
+
+For agents that operate Vidfarm headlessly, the typical loop is:
+
+1. `POST /api/v1/compositions { source_slug_id }` → fork
+2. `PUT /api/v1/compositions/:forkId/composition.html` → apply edits
+3. `PATCH /api/v1/compositions/:forkId/composition.json` → update metadata
+4. `POST /api/v1/compositions/:forkId/publish { tracer }` → render
+5. Poll `GET /api/v1/compositions/:forkId/renders/:renderId` until `SUCCEEDED`
+6. Use `output_url` in the next stage
+
+Always send a stable `tracer` on publish to make retries idempotent.
+
+## What NOT to do
+
+- Do **not** manipulate composition HTML by string concatenation. Always parse, edit, and re-serialize the DOM. The editor and runtime rely on `data-start`, `data-duration`, `data-track-index`, `data-hf-id`, `data-composition-id` attributes being well-formed.
+- Do **not** save AI provider keys in the composition HTML or JSON. They belong in the caller's provider-keys record.
+- Do **not** call Remotion directly. Publishing goes through `/publish` so billing, retries, cost caps, and Lambda quotas are enforced.
+- Do **not** re-invent scene manipulation as REST commands (`AddScene`, `RemoveScene`, `UpdateSceneField`). The Trackpad Editor is the surface for that. The pattern is: mutate the composition HTML DOM, PUT it back.
+
+## Alpha posture
+
+Vidfarm is in active development. Endpoints, response shapes, and the editor UI can change. When something behaves unexpectedly:
+
+- prefer `GET /api/v1/compositions/:forkId` over cached state
+- treat `latestVersion` as the source of truth for what the editor loaded
+- if a publish fails, check `renders/:renderId` for the error phase and stderr before retrying
+
+## Default assistance pattern
+
+When a director asks "make me a video", the default sequence is:
+
+1. Ask what source material and what template style they want
+2. Discover a matching template with `GET /api/v1/hyperframes`
+3. Fork it
+4. If they want an entirely new source video, guide them to `auto-decompose` first
+5. Open the Trackpad Editor and let them make edits
+6. Publish and share
+
+Prefer specific templates over primitives when a template exists that already captures the desired production pattern.
+
+## Primitive: image_remove_background
+
+Remove the background from any image URL. Result is a transparent PNG/WebP stored at a durable Vidfarm URL you can reuse in compositions or downstream primitives.
+
+- `POST /api/v1/primitives/images/remove-background`
+- Body: `{ "source_image_url": "https://...", "output_format": "webp" | "png" | "jpeg" }` (`output_format` defaults to `webp`)
+- Response: standard primitive job; poll for completion, then read `image.file_url` / `primary_file_url`
+- Billing: flat pass-through fee via RapidAPI (`rapidapi_remove_background` cost center)
+
+Example:
+
+```bash
+curl -X POST "$VIDFARM_BASE/api/v1/primitives/images/remove-background" \
+  -H "authorization: Bearer $VIDFARM_API_KEY" \
+  -H "content-type: application/json" \
+  -d '{"source_image_url": "https://cdn.example.com/photo.jpg"}'
+```