@mevdragon/vidfarm-devcli 0.2.12 → 0.2.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (205) hide show
  1. package/.env.example +37 -1
  2. package/README.md +27 -183
  3. package/SKILL.director.md +293 -0
  4. package/SKILL.platform.md +312 -0
  5. package/auto-create-hyperframe-templates/AUTO_CREATE_HYPERFRAME_TEMPLATES.md +275 -0
  6. package/auto-create-hyperframe-templates/extractor-system-prompt.md +128 -0
  7. package/auto-create-hyperframe-templates/input-manifest.schema.json +56 -0
  8. package/auto-create-hyperframe-templates/lambda-memory-estimator.md +118 -0
  9. package/auto-create-hyperframe-templates/production-graph.schema.json +314 -0
  10. package/auto-create-hyperframe-templates/runbook.md +198 -0
  11. package/auto-create-hyperframe-templates/scripts/create-hyperframe-template.mjs +368 -0
  12. package/auto-create-hyperframe-templates/template-plan.schema.json +311 -0
  13. package/auto-create-hyperframe-templates/template-planner-prompt.md +144 -0
  14. package/auto-create-templates/AUTO_CREATE_TEMPLATES.md +116 -0
  15. package/auto-create-templates/extractor-system-prompt.md +6 -1
  16. package/auto-create-templates/template-plan.schema.json +73 -2
  17. package/auto-create-templates/template-planner-prompt.md +62 -1
  18. package/dist/src/account-pages-legacy.js +363 -89
  19. package/dist/src/account-pages-legacy.js.map +1 -0
  20. package/dist/src/account-pages.js +3 -2
  21. package/dist/src/account-pages.js.map +1 -0
  22. package/dist/src/app.js +7096 -1294
  23. package/dist/src/app.js.map +1 -0
  24. package/dist/src/cli.js +115 -29
  25. package/dist/src/cli.js.map +1 -0
  26. package/dist/src/composition-runtime.js +581 -0
  27. package/dist/src/composition-runtime.js.map +1 -0
  28. package/dist/src/config.js +69 -11
  29. package/dist/src/config.js.map +1 -0
  30. package/dist/src/context.js +280 -21
  31. package/dist/src/context.js.map +1 -0
  32. package/dist/src/dev-app-legacy.js +18 -17
  33. package/dist/src/dev-app-legacy.js.map +1 -0
  34. package/dist/src/dev-app.js +1 -0
  35. package/dist/src/dev-app.js.map +1 -0
  36. package/dist/src/dev-serve.js +169 -0
  37. package/dist/src/dev-serve.js.map +1 -0
  38. package/dist/src/domain.js +2 -1
  39. package/dist/src/domain.js.map +1 -0
  40. package/dist/src/editor-chat-history.js +10 -0
  41. package/dist/src/editor-chat-history.js.map +1 -0
  42. package/dist/src/editor-chat.js +135 -25
  43. package/dist/src/editor-chat.js.map +1 -0
  44. package/dist/src/editor-dark-theme.js +1128 -0
  45. package/dist/src/editor-dark-theme.js.map +1 -0
  46. package/dist/src/frontend/flockposter-cache-store.js +10 -2
  47. package/dist/src/frontend/flockposter-cache-store.js.map +1 -0
  48. package/dist/src/frontend/homepage-client.js +6 -3
  49. package/dist/src/frontend/homepage-client.js.map +1 -0
  50. package/dist/src/frontend/homepage-shared.js +1 -0
  51. package/dist/src/frontend/homepage-shared.js.map +1 -0
  52. package/dist/src/frontend/homepage-store.js +1 -0
  53. package/dist/src/frontend/homepage-store.js.map +1 -0
  54. package/dist/src/frontend/homepage-view.js +131 -18
  55. package/dist/src/frontend/homepage-view.js.map +1 -0
  56. package/dist/src/frontend/page-runtime-client.js +27 -3
  57. package/dist/src/frontend/page-runtime-client.js.map +1 -0
  58. package/dist/src/frontend/page-runtime-store.js +1 -0
  59. package/dist/src/frontend/page-runtime-store.js.map +1 -0
  60. package/dist/src/frontend/sentry.js +42 -0
  61. package/dist/src/frontend/sentry.js.map +1 -0
  62. package/dist/src/frontend/template-editor-chat.js +1711 -212
  63. package/dist/src/frontend/template-editor-chat.js.map +1 -0
  64. package/dist/src/help-page.js +333 -0
  65. package/dist/src/help-page.js.map +1 -0
  66. package/dist/src/homepage.js +169 -47
  67. package/dist/src/homepage.js.map +1 -0
  68. package/dist/src/hyperframes/composition.js +180 -0
  69. package/dist/src/hyperframes/composition.js.map +1 -0
  70. package/dist/src/index.js +1 -0
  71. package/dist/src/index.js.map +1 -0
  72. package/dist/src/instrument.js +30 -0
  73. package/dist/src/instrument.js.map +1 -0
  74. package/dist/src/lib/crypto.js +1 -0
  75. package/dist/src/lib/crypto.js.map +1 -0
  76. package/dist/src/lib/dev-log.js +54 -0
  77. package/dist/src/lib/dev-log.js.map +1 -0
  78. package/dist/src/lib/display-name.js +11 -0
  79. package/dist/src/lib/display-name.js.map +1 -0
  80. package/dist/src/lib/ids.js +21 -1
  81. package/dist/src/lib/ids.js.map +1 -0
  82. package/dist/src/lib/images.js +1 -0
  83. package/dist/src/lib/images.js.map +1 -0
  84. package/dist/src/lib/json.js +1 -0
  85. package/dist/src/lib/json.js.map +1 -0
  86. package/dist/src/lib/template-dna.js +1 -0
  87. package/dist/src/lib/template-dna.js.map +1 -0
  88. package/dist/src/lib/template-paths.js +1 -0
  89. package/dist/src/lib/template-paths.js.map +1 -0
  90. package/dist/src/lib/template-style-options.js +29 -3
  91. package/dist/src/lib/template-style-options.js.map +1 -0
  92. package/dist/src/lib/time.js +1 -0
  93. package/dist/src/lib/time.js.map +1 -0
  94. package/dist/src/lib/video-quality-harness.js +60 -0
  95. package/dist/src/lib/video-quality-harness.js.map +1 -0
  96. package/dist/src/page-runtime.js +1 -0
  97. package/dist/src/page-runtime.js.map +1 -0
  98. package/dist/src/page-shell.js +403 -29
  99. package/dist/src/page-shell.js.map +1 -0
  100. package/dist/src/primitive-context.js +144 -8
  101. package/dist/src/primitive-context.js.map +1 -0
  102. package/dist/src/primitive-registry.js +1919 -78
  103. package/dist/src/primitive-registry.js.map +1 -0
  104. package/dist/src/primitive-sdk.js +1 -0
  105. package/dist/src/primitive-sdk.js.map +1 -0
  106. package/dist/src/primitives/remotion/html-image.js +3 -1
  107. package/dist/src/primitives/remotion/html-image.js.map +1 -0
  108. package/dist/src/primitives/remotion/media-slideshow.js +60 -0
  109. package/dist/src/primitives/remotion/media-slideshow.js.map +1 -0
  110. package/dist/src/react-page-shell.js +1 -0
  111. package/dist/src/react-page-shell.js.map +1 -0
  112. package/dist/src/ready-post-schedule-component.js +1 -0
  113. package/dist/src/ready-post-schedule-component.js.map +1 -0
  114. package/dist/src/registry.js +237 -24
  115. package/dist/src/registry.js.map +1 -0
  116. package/dist/src/runtime.js +3 -0
  117. package/dist/src/runtime.js.map +1 -0
  118. package/dist/src/services/api-call-history.js +4 -0
  119. package/dist/src/services/api-call-history.js.map +1 -0
  120. package/dist/src/services/auth.js +40 -50
  121. package/dist/src/services/auth.js.map +1 -0
  122. package/dist/src/services/billing.js +20 -44
  123. package/dist/src/services/billing.js.map +1 -0
  124. package/dist/src/services/chat-threads.js +10 -6
  125. package/dist/src/services/chat-threads.js.map +1 -0
  126. package/dist/src/services/fork-access.js +93 -0
  127. package/dist/src/services/fork-access.js.map +1 -0
  128. package/dist/src/services/fork-manifest.js +43 -0
  129. package/dist/src/services/fork-manifest.js.map +1 -0
  130. package/dist/src/services/ghostcut.js +132 -0
  131. package/dist/src/services/ghostcut.js.map +1 -0
  132. package/dist/src/services/hyperframes.js +1014 -0
  133. package/dist/src/services/hyperframes.js.map +1 -0
  134. package/dist/src/services/job-capacity.js +14 -0
  135. package/dist/src/services/job-capacity.js.map +1 -0
  136. package/dist/src/services/job-logs.js +4 -0
  137. package/dist/src/services/job-logs.js.map +1 -0
  138. package/dist/src/services/jobs.js +99 -91
  139. package/dist/src/services/jobs.js.map +1 -0
  140. package/dist/src/services/media-processing.js +743 -0
  141. package/dist/src/services/media-processing.js.map +1 -0
  142. package/dist/src/services/primitive-media-lambda.js +280 -0
  143. package/dist/src/services/primitive-media-lambda.js.map +1 -0
  144. package/dist/src/services/providers.js +1560 -178
  145. package/dist/src/services/providers.js.map +1 -0
  146. package/dist/src/services/rate-limits.js +3 -2
  147. package/dist/src/services/rate-limits.js.map +1 -0
  148. package/dist/src/services/remotion.js +495 -92
  149. package/dist/src/services/remotion.js.map +1 -0
  150. package/dist/src/services/serverless-auth.js +374 -0
  151. package/dist/src/services/serverless-auth.js.map +1 -0
  152. package/dist/src/services/serverless-jobs.js +1074 -0
  153. package/dist/src/services/serverless-jobs.js.map +1 -0
  154. package/dist/src/services/serverless-provider-keys.js +401 -0
  155. package/dist/src/services/serverless-provider-keys.js.map +1 -0
  156. package/dist/src/services/serverless-records.js +1088 -0
  157. package/dist/src/services/serverless-records.js.map +1 -0
  158. package/dist/src/services/serverless-template-configs.js +67 -0
  159. package/dist/src/services/serverless-template-configs.js.map +1 -0
  160. package/dist/src/services/storage.js +171 -35
  161. package/dist/src/services/storage.js.map +1 -0
  162. package/dist/src/services/template-certification.js +295 -3
  163. package/dist/src/services/template-certification.js.map +1 -0
  164. package/dist/src/services/template-loader.js +45 -1
  165. package/dist/src/services/template-loader.js.map +1 -0
  166. package/dist/src/services/template-runtime-bundles.js +217 -0
  167. package/dist/src/services/template-runtime-bundles.js.map +1 -0
  168. package/dist/src/services/template-sources.js +452 -87
  169. package/dist/src/services/template-sources.js.map +1 -0
  170. package/dist/src/services/video-normalization.js +2 -0
  171. package/dist/src/services/video-normalization.js.map +1 -0
  172. package/dist/src/services/webhooks.js +7 -6
  173. package/dist/src/services/webhooks.js.map +1 -0
  174. package/dist/src/template-editor-pages.js +2051 -1869
  175. package/dist/src/template-editor-pages.js.map +1 -0
  176. package/dist/src/template-editor-shell.js +1376 -181
  177. package/dist/src/template-editor-shell.js.map +1 -0
  178. package/dist/src/template-sdk.js +1 -0
  179. package/dist/src/template-sdk.js.map +1 -0
  180. package/dist/src/worker.js +11 -226
  181. package/dist/src/worker.js.map +1 -0
  182. package/package.json +47 -12
  183. package/public/assets/homepage-client-app.js +34 -8
  184. package/public/assets/page-runtime-client-app.js +44 -18
  185. package/readme.secret.md +89 -0
  186. package/templates/vidfarm_template_0000/README.md +106 -0
  187. package/templates/vidfarm_template_0000/SKILL.md +266 -0
  188. package/templates/vidfarm_template_0000/assets/Abel-Regular.ttf +0 -0
  189. package/templates/vidfarm_template_0000/assets/DMSerifDisplay-Regular.ttf +0 -0
  190. package/templates/vidfarm_template_0000/assets/Montserrat[wght].ttf +0 -0
  191. package/templates/vidfarm_template_0000/assets/SourceCodePro[wght].ttf +0 -0
  192. package/templates/vidfarm_template_0000/assets/TikTokSans-SemiBold.ttf +0 -0
  193. package/templates/vidfarm_template_0000/assets/Yesteryear-Regular.ttf +0 -0
  194. package/templates/vidfarm_template_0000/composition.json +11 -0
  195. package/templates/vidfarm_template_0000/src/lib/images.js +221 -0
  196. package/templates/vidfarm_template_0000/src/remotion/Root.js +33 -0
  197. package/templates/vidfarm_template_0000/src/remotion/index.js +3 -0
  198. package/templates/vidfarm_template_0000/src/sdk.js +3 -0
  199. package/templates/vidfarm_template_0000/src/style-options.js +200 -0
  200. package/templates/vidfarm_template_0000/src/template-dna.js +9 -0
  201. package/templates/vidfarm_template_0000/src/template.js +1566 -0
  202. package/templates/vidfarm_template_0000/template.config.json +21 -0
  203. package/GETTING_STARTED.developers.md +0 -87
  204. package/SKILL.developer.md +0 -1186
  205. package/dist/src/db.js +0 -2007
@@ -0,0 +1,312 @@
1
+ ---
2
+ name: vidfarm-platform
3
+ description: Vidfarm platform architecture reference. Use this for questions about how compositions, forks, versions, storage, DynamoDB records, the Trackpad Editor runtime, Remotion render Lambda, editor-chat, primitives, and billing fit together. This is not a director skill (see SKILL.director.md) and not a runbook — it explains the system.
4
+ ---
5
+
6
+ # Vidfarm Platform Architecture
7
+
8
+ Vidfarm is a **serverless composition studio** for short-form video. Directors fork templates, edit compositions on a timeline (the Trackpad Editor), publish MP4s via a shared Remotion Lambda, and share results.
9
+
10
+ This document is the architectural reference — what runs where, what data lives where, and how the pieces connect. Use it when planning changes, debugging cross-cutting behavior, or onboarding to the platform.
11
+
12
+ For customer-facing usage patterns, see `SKILL.director.md`.
13
+
14
+ ## Runtime shape
15
+
16
+ Vidfarm runs entirely on AWS serverless:
17
+
18
+ - **API** — a single Hono app deployed as a container Lambda behind an API Gateway custom domain per stage
19
+ - **Editor Chat** — a separate Lambda + custom domain for streaming AI editor sessions
20
+ - **Primitive Media** — a separate Lambda + custom domain for standalone media primitives (image gen, video generation, etc.)
21
+ - **Ready Post Zip** — a separate Lambda + custom domain for scheduled/approved-post ZIP downloads
22
+ - **Billing** — a separate Lambda + custom domain for Stripe webhooks, wallet top-ups, and idempotent charge processing
23
+ - **Job Runner** — a Lambda invoked by the API to execute template operations
24
+ - **Job Sweeper** — a scheduled Lambda that stalls / retries / cleans up in-flight jobs
25
+
26
+ Stages:
27
+
28
+ - `staging` — `staging.vidfarm.cc` (+ sibling subdomains for billing, editor-chat, primitive-media, zip)
29
+ - `production` — `vidfarm.cc`
30
+
31
+ Every stage is a separate CDK stack with fully separate DynamoDB tables, S3 buckets, Lambdas, and DNS.
32
+
33
+ There is no EC2, no SQLite, no long-lived worker. Everything scales to zero at idle.
34
+
35
+ ## Data model
36
+
37
+ ### DynamoDB — single-table
38
+
39
+ One main table per stage: `VidfarmMainTable`. Partitioned by `entity_type`. Records include:
40
+
41
+ - `customer` — director account, wallet, provider keys reference, timestamps
42
+ - `customer_api_key` — API key hash + salt
43
+ - `template` — imported template registration (slug, source repo, active release pointer)
44
+ - `template_release` — a specific artifact for a template
45
+ - `job` — an in-flight or completed job execution
46
+ - `job_log_event` — structured log lines per job (transient — pruned)
47
+ - `api_call_history` — request audit trail (transient)
48
+ - `editor_chat_message` — chat history (transient)
49
+ - `serverless_record#composition_fork` — an editable composition fork
50
+ - `serverless_record#fork_version` — an immutable version snapshot for a fork
51
+ - `serverless_record#fork_permission` — a per-user grant on a fork
52
+ - `serverless_record#fork_share_link` — a share-token URL for a fork
53
+ - `submitted_video` — a source video uploaded by a director
54
+ - `provider_key_lease` — short-lived AI key leases for job runners
55
+ - `billing_event` — wallet transactions and Stripe events
56
+
57
+ Query patterns use GSIs on `customerId`, `forkId`, `granteeId`, `slug`, and `status`.
58
+
59
+ ### S3 — composition storage
60
+
61
+ Bucket per stage. Layout is **ownership-agnostic** — the customerId does not appear in the S3 key. Access control is enforced at the API layer via the DynamoDB permission records.
62
+
63
+ ```
64
+ compositions/forks/<forkId>/
65
+ ├── working/
66
+ │ ├── composition.html # current editor state
67
+ │ ├── composition.json # current metadata (scenes, captions, viral DNA)
68
+ │ └── ghostcut.json # GhostCut subtitle-removal task state (optional)
69
+ └── versions/<version>/
70
+ ├── composition.html
71
+ └── composition.json
72
+ ```
73
+
74
+ Additional prefixes:
75
+
76
+ - `renders/vidfarm/<slug>/<render-id>/output.mp4` — published MP4s
77
+ - `submissions/<videoId>/...` — raw source video uploads
78
+ - `template-assets/<slug>/...` — template media
79
+ - `user-attachments/<customerId>/...` — chat panel uploads (only ownership-keyed area remaining)
80
+
81
+ Old ownership-keyed composition paths (`compositions/users/<customerId>/forks/...`) were migrated away by `scripts/migrate-compositions-to-ownership-agnostic.ts`. Old `decomposition.json` was renamed to `composition.json` by `scripts/migrate-decomposition-to-composition.ts`.
82
+
83
+ ## Composition model
84
+
85
+ A composition is a single HTML document plus a JSON sidecar.
86
+
87
+ **composition.html** is a self-contained document. Top-level structure:
88
+
89
+ ```html
90
+ <div data-composition-id="<forkId>@v<version>" data-viral-dna="{...}">
91
+ <div data-vf-timeline-proxy="video" data-src="..." data-start="0" data-duration="12" />
92
+ <div data-hf-id="clip_1" data-layer-kind="video"
93
+ data-start="0" data-duration="3.5" data-track-index="0"
94
+ data-playback-start="0.25" data-src="...">
95
+ ...
96
+ </div>
97
+ <div data-hf-id="caption_1" data-layer-kind="caption"
98
+ data-start="0.5" data-duration="2.0" data-track-index="1"
99
+ style="position:absolute;left:5%;top:60%;...">
100
+ HOOK LINE
101
+ </div>
102
+ <!-- more layers... -->
103
+ <script type="module"><!-- composition-runtime.ts injected inline --></script>
104
+ </div>
105
+ ```
106
+
107
+ **composition.json** carries the semantic metadata parsed once at auto-decompose time and used by the editor sidebar, chat context, and publish payload:
108
+
109
+ ```json
110
+ {
111
+ "summary": "...",
112
+ "scenes": [{ "slug": "hook_reveal", "start": 0, "duration": 1.5, "label": "...", "viral_note": "..." }],
113
+ "captions": [{ "slug": "caption_1", "text": "...", "start": 0, "duration": 1.5, "x": 5, "y": 60 }],
114
+ "viralDna": { "hook": "...", "retention": "...", "payoff": "...", "preserve": [...], "avoid": [...] },
115
+ "width": 1080,
116
+ "height": 1920,
117
+ "durationSeconds": 10.5,
118
+ "source_assets": [{ "type": "video", "url": "...", "width": ..., "height": ..., "duration": ... }]
119
+ }
120
+ ```
121
+
122
+ Both files are treated as immutable per version. The working state is mutable.
123
+
124
+ ## Composition runtime
125
+
126
+ `src/composition-runtime.ts` generates a browser-side script that gets injected into every composition HTML. It provides:
127
+
128
+ - **Playback control** — reads all `[data-start][data-duration][data-track-index]` layers, toggles their visibility based on the current time, and syncs media element `currentTime` per clip
129
+ - **Player adapter** — exposes `window.__player = { play(), pause(), seek(t), setTime(t), duration, playing, playbackRate, previewMuted, refreshClips() }` for the editor to control the iframe
130
+ - **Media coordination** — respects `data-playback-start`, `data-volume`, and clip-specific mute
131
+ - **Selection sync** — listens for `postMessage({ source: "hf-parent", ... })` from the parent editor to highlight/deselect clips
132
+
133
+ The same runtime is used inside the Trackpad Editor iframe AND when a published fork is embedded elsewhere. It is agnostic to whether it's being edited or viewed.
134
+
135
+ ## The Trackpad Editor
136
+
137
+ `demo/src/HyperframesStudioEditor.tsx` is the timeline editor. It:
138
+
139
+ - Renders a `<Timeline>` and `<Player>` from `@hyperframes/studio` — a shared package with the timeline UI, drag/trim logic, and player state store (Zustand)
140
+ - Loads `composition.html` + `composition.json` from the fork's working state
141
+ - Renders the composition into a preview iframe using a `blob:` URL
142
+ - Bidirectionally syncs timeline drag → HTML `data-start`/`data-duration` mutations, and inspector edits → text/style changes
143
+ - Persists every commit back to the API as PUT `composition.html` and PATCH `composition.json`
144
+ - Exposes an **Editor Action API** so the Chat panel can invoke 13 typed actions (`add_layer`, `remove_layer`, `set_layer_start`, `set_layer_duration`, `set_layer_style`, `group_layers`, `ungroup_layers`, `replace_composition_html`, etc.)
145
+ - Runs the composition-runtime script inside the preview iframe to keep timeline scrub in sync with the current time
146
+
147
+ The editor is a single monolithic React component (~9k lines). It builds cleanly to a static bundle deployed to CloudFront under `/editor/`.
148
+
149
+ ## Publishing (render pipeline)
150
+
151
+ Publish flow:
152
+
153
+ 1. Client `POST /api/v1/compositions/:forkId/publish`
154
+ 2. API stamps `data-composition-id="<forkId>@v<N>"` into the composition HTML, PUTs the stamped HTML to working, and reads composition.json
155
+ 3. API resolves the caller's provider keys (preflight check for required models)
156
+ 4. API creates a `job` record and invokes the `hyperframes_render` primitive Lambda
157
+ 5. The primitive Lambda uploads the composition as a Remotion site, calls `renderMediaOnLambda` with the shared Remotion render Lambda ARN, waits for the MP4 to land in S3
158
+ 6. On success, the primitive returns `{ primary_file_url: "s3://.../output.mp4" }`, the API creates a `ForkVersionRecord` with `reason: "publish"`, writes the snapshot to `versions/<N>/`, and updates the fork's `latestVersion` and `currentRenderJobId`
159
+ 7. Client polls `GET /api/v1/compositions/:forkId/renders/:renderId` for `{ phase, progress, output_url, cost_usd }`
160
+
161
+ The Remotion Lambda is **shared across all templates and forks** — it's an infrastructure primitive, not a per-template deployment. Its ARN is passed to the Vidfarm stack as an env var.
162
+
163
+ ## Auto-decompose
164
+
165
+ `POST /api/v1/compositions/:forkId/auto-decompose` splits a raw source video into a Trackpad-editable composition. Two modes:
166
+
167
+ - **`time-slice`** (deterministic) — `src/hyperframes/composition.ts`'s `decomposeUndecomposedComposition()` splits the source duration into N equal-length scenes, wraps each in a `<div data-vf-timeline-proxy="video">` layer with `data-start`/`data-duration`/`data-playback-start`. Instant. No AI.
168
+ - **`smart`** (AI) — `src/services/hyperframes.ts`'s `smartDecomposeVideo()` downloads the source, samples frames, calls Gemini/OpenAI/OpenRouter vision (in priority order) using the caller's saved keys, extracts scenes with labels and viral notes, extracts on-screen captions with position, and generates viral DNA. Then it also submits an async **GhostCut** subtitle-removal task if the source has hardcoded subs. Takes 30-60s + async ghostcut poll.
169
+
170
+ Both modes update the composition.html and composition.json in place, and create a fork version snapshot with `reason: "manual"`.
171
+
172
+ ## GhostCut integration
173
+
174
+ GhostCut is an external AI subtitle-removal service. Vidfarm proxies to it via `src/services/ghostcut.ts`:
175
+
176
+ - `submitSubtitleRemoval(sourceUrl)` — queue the removal task
177
+ - `pollStatus(taskId)` — check status (`pending` | `processing` | `queued` | `done` | `failed`)
178
+ - `mirrorGhostcutVideoToStorage()` — copy the clean output back to Vidfarm S3
179
+
180
+ State is persisted per fork at `compositions/forks/<forkId>/working/ghostcut.json`. The client polls `POST /api/v1/compositions/:forkId/ghostcut-poll` every few seconds during smart decompose.
181
+
182
+ Configuration: `GHOSTCUT_KEY` and `GHOSTCUT_SECRET` env vars per stage. Pass-through cost: `GHOSTCUT_USD_PER_30S` (default 0.10).
183
+
184
+ ## Editor Chat
185
+
186
+ `infra/lambda/editor-chat.ts` is a separate Lambda that streams AI-driven edits to the Trackpad Editor. Client opens an SSE stream to `/api/v1/editor-chat`, sends the current composition snapshot + user prompt + attachments, and receives:
187
+
188
+ - assistant text
189
+ - tool calls that map to the Editor Action API (`add_layer`, `set_layer_start`, ...)
190
+ - token usage + billing events
191
+
192
+ The chat model uses the **caller's provider keys**. No Vidfarm-owned model calls happen in this path.
193
+
194
+ ## Primitives
195
+
196
+ Primitives are hosted operations that are NOT templates — they're reusable building blocks:
197
+
198
+ - `create_image`, `edit_image`, `inpaint_image` — image generation
199
+ - `create_html_image` — Remotion HTML → PNG
200
+ - `remove_background` — RapidAPI-backed image background removal → durable PNG/WebP
201
+ - `download_media_video` — social/media URL → durable MP4
202
+ - `hyperframes_render` — composition HTML → MP4 (the render primitive publishing uses)
203
+ - `brainstorm` — LLM brainstorm route
204
+
205
+ Directors call primitives when they want a one-off asset before feeding it into a template. Templates call primitives internally as workflow steps. All primitives are metered against the caller's wallet.
206
+
207
+ ## Billing
208
+
209
+ Billing is USD wallet-based, not credits. Every metered operation writes a `billing_event` record and debits the wallet in the same DynamoDB transaction. Stripe webhooks land on the billing Lambda, which validates the signature, marks the invoice as processed idempotently, and credits the wallet.
210
+
211
+ Key mechanics:
212
+
213
+ - Every wallet mutation carries a `tracer` (idempotency key)
214
+ - Provider spend is passed through at cost — no markup on AI provider calls
215
+ - Platform infrastructure (Remotion render, GhostCut, storage) has a small safety buffer applied
216
+ - Failed jobs are refunded automatically when the failure is on the platform side
217
+
218
+ The billing Lambda has its own custom domain because Stripe webhooks require a stable URL not tied to the main API.
219
+
220
+ ## Template model
221
+
222
+ There are two kinds of things directors can fork:
223
+
224
+ 1. **Draft templates** (`/hyperframes/:slug`) — pre-decomposed compositions checked into the repo under `templates/vidfarm_template_XXXX/`. Each has a `composition.html`, `composition.json`, `template.config.json`, a `SKILL.md`, and a `src/` if it needs custom Remotion.
225
+ 2. **Submitted videos** — raw uploads from directors, decomposed on demand via auto-decompose.
226
+
227
+ Templates are static. Compositions/forks are the mutable working copies.
228
+
229
+ ## Configuration and env
230
+
231
+ Per-stage config is loaded via `.env.<stage>`. Critical variables:
232
+
233
+ - `AWS_REGION`, AWS credentials
234
+ - `VIDFARM_<STAGE>_PUBLIC_BASE_URL` — the API base
235
+ - `VIDFARM_<STAGE>_ENCRYPTION_SECRET` — DynamoDB field encryption
236
+ - `VIDFARM_<STAGE>_API_KEY_SALT` — API key hashing
237
+ - `VIDFARM_<STAGE>_BOOTSTRAP_*` — initial admin user
238
+ - `VIDFARM_<STAGE>_REMOTION_*` — shared Remotion Lambda ARN + region
239
+ - `VIDFARM_<STAGE>_RESEND_*` — email
240
+ - `VIDFARM_<STAGE>_SENTRY_*` — Sentry DSN
241
+ - `GHOSTCUT_KEY`, `GHOSTCUT_SECRET` — GhostCut credentials
242
+ - `RAPIDAPI_KEY` — shared RapidAPI credential used by the `video_download` and `image_remove_background` primitives. Same key is set in both `.env.staging` and `.env.production`.
243
+ - Provider fallback keys for platform-owned features that don't have a caller yet (bootstrap only)
244
+
245
+ Never commit `.env.staging` or `.env.production` — both are gitignored.
246
+
247
+ ## Deployment
248
+
249
+ CDK stacks are under `infra/cdk/`:
250
+
251
+ - `bin/vidfarm-serverless-staging.ts` → `lib/vidfarm-serverless-staging-stack.ts`
252
+ - `bin/vidfarm-prod.ts` → `lib/vidfarm-prod-stack.ts`
253
+
254
+ Deploy commands are in `package.json`:
255
+
256
+ - `npm run cdk:synth:staging-serverless`
257
+ - `npm run cdk:deploy:staging-serverless`
258
+ - `npm run cdk:deploy:prod`
259
+
260
+ Never call bare `cdk deploy`. Always go through the npm scripts so the correct `.env.<stage>` is loaded.
261
+
262
+ Deploy order: **staging first**, then production. Never skip staging.
263
+
264
+ ## Migration scripts
265
+
266
+ One-shot scripts under `scripts/`:
267
+
268
+ - `migrate-sqlite-to-dynamodb.ts` — one-time, completed on 2026-06-22. Do not re-run.
269
+ - `migrate-compositions-to-ownership-agnostic.ts` — moves forks from `compositions/users/<customerId>/forks/<forkId>/` to `compositions/forks/<forkId>/working/`. Idempotent. Must be run against each stage where the old layout ever existed.
270
+ - `migrate-decomposition-to-composition.ts` — renames `decomposition.json` → `composition.json` in S3 for all forks + versions. Idempotent. Supports `--delete-old` for cleanup.
271
+ - `copy-dynamodb-prod-to-staging.ts` — copies prod DynamoDB items into staging for testing. Excludes transient rows by default.
272
+
273
+ Run migrations with `--dry-run` first. Always.
274
+
275
+ ## Directory reference
276
+
277
+ Top-level:
278
+
279
+ - `src/` — API server + all backend services
280
+ - `demo/` — Trackpad Editor React app
281
+ - `infra/` — CDK stacks and Lambda entry points
282
+ - `templates/` — draft template sources
283
+ - `auto-create-hyperframe-templates/` — extractor prompts / schemas for new template creation
284
+ - `scripts/` — one-shot ops scripts and migration scripts
285
+ - `data/` — local dev SQLite + storage (gitignored)
286
+ - `.env.staging`, `.env.production` — stage config (gitignored)
287
+
288
+ Legacy but still active:
289
+
290
+ - `auto-create-templates/` — extractor prompts for the older template creation flow. Still consumed by `src/cli.ts` (`autocreate-template` command) and `demo/scripts/dev.mjs`. Migrate the CLI before deleting.
291
+ - `src/account-pages-legacy.ts`, `src/dev-app-legacy.ts` — HTML-string page renderers wrapped by `.tsx` React shells (`account-pages.tsx`, `dev-app.tsx`). Still active.
292
+
293
+ ## Common failure modes
294
+
295
+ - **Fork loads 404** — check ownership-agnostic migration ran; the fork may still be at the old `compositions/users/.../forks/` path.
296
+ - **Publish stuck in RUNNING** — check the primitive Lambda's CloudWatch logs. Remotion asset preflight failures usually mean a broken media URL in composition.html.
297
+ - **Smart decompose falls back to time-slice** — the caller has no provider keys, or all their keys returned errors. Check `provider_key_lease` records.
298
+ - **GhostCut task hangs** — client shows `pending` forever. Check the fork's `ghostcut.json` and the GhostCut dashboard. There is currently no cancel UI; the poll times out at 15 minutes.
299
+ - **Editor edits don't persist** — verify `PUT /composition.html` is 200. If the composition.html doesn't include `data-composition-id`, the API rejects the PUT at publish time.
300
+
301
+ ## Where to look for things
302
+
303
+ - Composition endpoints — `src/app.ts` under `COMPOSITIONS_PREFIX` (~line 6970+)
304
+ - Fork CRUD & permissions — `src/services/serverless-records.ts`, `src/services/fork-access.ts`
305
+ - Storage keys — `src/services/storage.ts`
306
+ - Composition runtime script — `src/composition-runtime.ts`
307
+ - Trackpad Editor — `demo/src/HyperframesStudioEditor.tsx`
308
+ - Auto-decompose logic — `src/hyperframes/composition.ts` (time-slice), `src/services/hyperframes.ts` (smart)
309
+ - Publish flow — `src/app.ts` `publishCompositionToRenderer()`
310
+ - Ghostcut — `src/services/ghostcut.ts`
311
+ - CDK stack — `infra/cdk/lib/vidfarm-serverless-staging-stack.ts`, `infra/cdk/lib/vidfarm-prod-stack.ts`
312
+ - Lambdas — `infra/lambda/*.ts`
@@ -0,0 +1,275 @@
1
+ # Auto-Create HyperFrame Templates Harness
2
+
3
+ This harness turns a simple MP4, image, or PNG/JPG/WebP slideshow into an editable HyperFrames composition that can be rendered through the production AWS Lambda stack.
4
+
5
+ Use this when the desired output is not a Vidfarm operation package, but a decomposed HyperFrames project:
6
+
7
+ ```txt
8
+ drafts/<format_slug>/
9
+ source.md
10
+ preview/
11
+ source.mp4
12
+ composition.html
13
+ composition.json
14
+ lambda-site.json
15
+ ```
16
+
17
+ The goal is to make the source editable in the HyperFrames editor while keeping publish behavior faithful. If a user deletes a source video scene, the final render must show a blank/background gap unless another timed layer covers that range. Do not rely on an always-on source video for publish output.
18
+
19
+ ## Start Here
20
+
21
+ Give a Codex agent these two things:
22
+
23
+ 1. this entry file: `auto-create-hyperframe-templates/AUTO_CREATE_HYPERFRAME_TEMPLATES.md`
24
+ 2. one source media folder, preferably shaped like:
25
+
26
+ ```txt
27
+ drafts/<format_slug>/
28
+ source.md
29
+ preview/
30
+ source.mp4
31
+ slide-01.png
32
+ slide-02.png
33
+ ```
34
+
35
+ Then ask:
36
+
37
+ ```txt
38
+ Use auto-create-hyperframe-templates/AUTO_CREATE_HYPERFRAME_TEMPLATES.md as the harness. Analyze drafts/<format_slug>/, create the run artifacts, generate the HyperFrames production graph and template plan, then write composition.html, composition.json, and lambda-site.json.
39
+ ```
40
+
41
+ For the current local bridge, you can run:
42
+
43
+ ```bash
44
+ node auto-create-hyperframe-templates/scripts/create-hyperframe-template.mjs drafts/<format_slug>/preview/source.mp4
45
+ ```
46
+
47
+ For a slideshow, pass the folder containing the images:
48
+
49
+ ```bash
50
+ node auto-create-hyperframe-templates/scripts/create-hyperframe-template.mjs drafts/<format_slug>/preview
51
+ ```
52
+
53
+ Add `--stage-site` when you want the script to call the production HyperFrames Lambda site upload after writing the local artifacts.
54
+
55
+ The source can be:
56
+
57
+ - one MP4
58
+ - a folder of PNG/JPG/WebP slides
59
+ - a mixed folder with source video, screenshots, overlay images, audio, and notes
60
+
61
+ If `source.md` is missing, create it with the source URL, adaptation goal, expected audience, and any constraints that can be inferred from the files.
62
+
63
+ For programmatic runs, write an input manifest matching `input-manifest.schema.json`:
64
+
65
+ ```json
66
+ {
67
+ "mode": "autocreate-hyperframe-template",
68
+ "input_path": "drafts/my-format/preview/source.mp4",
69
+ "output_path": "drafts/my-format",
70
+ "slug_id": "my-format",
71
+ "source_mode": "auto",
72
+ "render_defaults": {
73
+ "width": 720,
74
+ "height": 1280,
75
+ "fps": 30
76
+ },
77
+ "lambda": {
78
+ "stack_name": "hyperframes-vidfarm-prod",
79
+ "region": "us-east-1",
80
+ "memory_strategy": "estimate"
81
+ },
82
+ "publish": {
83
+ "prestage_site": false,
84
+ "output_url_policy": "signed_https_final_output_only"
85
+ }
86
+ }
87
+ ```
88
+
89
+ ## Output Contract
90
+
91
+ Every successful run should produce:
92
+
93
+ - `composition.html`: a self-contained HyperFrames composition using deterministic HTML/CSS and timed media elements.
94
+ - `composition.json`: the editable editor manifest: source assets, scene cuts, layers, caption geometry, viral/visual DNA, and Lambda estimate.
95
+ - `lambda-site.json`: the production Lambda site/render metadata: stack, region, memory estimate, site id if pre-staged, and render defaults.
96
+ - optional `frames/contact-sheet.jpg`, `source/ffprobe.json`, `transcripts/stt.json`, `ocr/frame_text.json`, and `motion/scene-cuts.json`.
97
+
98
+ The `composition.json` should follow `template-plan.schema.json` after planning. The lower-level extraction artifact should follow `production-graph.schema.json`.
99
+
100
+ ## Core Principle
101
+
102
+ HyperFrames is the editor and deterministic renderer. The harness should decompose the source into layers that remain individually editable:
103
+
104
+ - source video scene clips
105
+ - source audio
106
+ - images or slide frames
107
+ - deterministic captions
108
+ - deterministic HTML overlays
109
+ - shapes, chips, arrows, circles, masks, UI shells, and stickers
110
+ - generated assets only when the visual cannot be rebuilt deterministically
111
+
112
+ Important readable text should not be baked into generated media unless the source format is explicitly poster/artifact-native and OCR validation is part of the quality harness.
113
+
114
+ ## Timeline Rules
115
+
116
+ Use absolute seconds in manifests and data attributes. Convert to frames only at render time.
117
+
118
+ Required layer fields:
119
+
120
+ - `id`
121
+ - `kind`
122
+ - `start`
123
+ - `duration`
124
+ - `track`
125
+ - `z`
126
+ - `mode`
127
+ - `source_of_truth`
128
+ - `geometry`
129
+ - `viral_alignment`
130
+
131
+ For video decomposition, split source video into scene layers:
132
+
133
+ ```json
134
+ {
135
+ "id": "scene-2",
136
+ "kind": "video",
137
+ "start": 3.567,
138
+ "duration": 3.033,
139
+ "track": 0,
140
+ "z": 0,
141
+ "mode": "both",
142
+ "media": {
143
+ "src": "/drafts/example/source.mp4",
144
+ "media_start": 3.567,
145
+ "fit": "cover"
146
+ }
147
+ }
148
+ ```
149
+
150
+ For publish correctness, timed video scene layers should become real timed media layers or publish-time timeline proxies that are rewritten into real timed media. A preview-only playback source may exist for editor scrubbing, but it must be ignored or normalized out during publish.
151
+
152
+ Blank gaps are valid. If no layer covers a time range, render the composition background for that range.
153
+
154
+ ## Slideshow Rules
155
+
156
+ For an image slideshow, create one timed image layer per slide:
157
+
158
+ ```json
159
+ {
160
+ "id": "slide-01-image",
161
+ "kind": "image",
162
+ "start": 0,
163
+ "duration": 3,
164
+ "track": 0,
165
+ "z": 0,
166
+ "mode": "both",
167
+ "media": {
168
+ "src": "/drafts/example/slide-01.png",
169
+ "fit": "cover"
170
+ }
171
+ }
172
+ ```
173
+
174
+ Caption text should be separate `caption` or `html` layers unless the source is intentionally a flattened image/poster template.
175
+
176
+ ## Caption And Overlay Rules
177
+
178
+ Captions are deterministic layout, not an afterthought. Record:
179
+
180
+ - font family and fallback
181
+ - font weight
182
+ - font size in px at the target composition size
183
+ - line height
184
+ - max width
185
+ - x/y/width/height in percentages
186
+ - text alignment
187
+ - safe-area constraints
188
+ - outline, radial halo, shadow, or inline highlight chip behavior
189
+ - whether text can wrap and how many lines are allowed
190
+
191
+ Caption positions must account for TikTok/Reels unsafe zones. For vertical 9:16 output, preserve:
192
+
193
+ - top 8-12 percent for app chrome risk
194
+ - right 12-16 percent for action buttons
195
+ - bottom 14-20 percent for username, caption, and scrubber risk
196
+ - center subject-safe zone when faces/products matter
197
+
198
+ Use inline chips that hug measured wrapped lines when the source has caption backgrounds. Do not turn inline chips into full-width bars or large paragraph washes.
199
+
200
+ For white TikTok-native captions, prefer a radial halo or outline. Avoid a single directional web drop shadow when the source uses a native black glyph halo.
201
+
202
+ For multiline captions, wrap once, choose one shared font size and line height, render one block, then measure per-line chips after layout. Do not render each wrapped line as an independently trimmed text image.
203
+
204
+ ## Lambda Memory Estimate
205
+
206
+ Each plan must include a `lambda_memory_estimate` with:
207
+
208
+ - `recommended_memory_mb`
209
+ - `confidence`
210
+ - `complexity_scores`
211
+ - `rationale`
212
+ - `risk_flags`
213
+ - `benchmark_override`
214
+
215
+ Default to `3072` MB for ordinary 720x1280 or 1080x1920 social-video compositions with one active video stream and deterministic overlays. Use `2048` MB only for static/slideshow/simple 720p outputs or after local evidence that video decode is light. Use `4096` MB for 1080p video, multiple simultaneous videos, heavy DOM/caption effects, or long audio mixing. Reserve `6144+` MB for high-resolution or failure-prone compositions.
216
+
217
+ Do not default to `10240` MB. Use it only after timeouts, Chromium memory failures, or a measured case that requires it.
218
+
219
+ See `lambda-memory-estimator.md` for the scoring rubric.
220
+
221
+ ## Production Lambda Site
222
+
223
+ The generated plan should be ready for the deployed production Lambda stack:
224
+
225
+ - stack: `hyperframes-vidfarm-prod`
226
+ - default region: `us-east-1`
227
+ - default memory: `3072`
228
+ - output policy: frontend should show only the final downloadable/signed output URL, not AWS console links that require login
229
+
230
+ For repeat publishing, pre-stage the project as a Lambda site and write `lambda-site.json`:
231
+
232
+ ```json
233
+ {
234
+ "schema": "vidfarm.hyperframe.lambda-site.v1",
235
+ "stack_name": "hyperframes-vidfarm-prod",
236
+ "region": "us-east-1",
237
+ "site_id": "content-addressed-site-id",
238
+ "project_dir": "drafts/example",
239
+ "composition_file": "composition.html",
240
+ "memory_mb": 3072,
241
+ "render_defaults": {
242
+ "width": 720,
243
+ "height": 1280,
244
+ "fps": 30
245
+ },
246
+ "output_url_policy": "signed_https_final_output_only"
247
+ }
248
+ ```
249
+
250
+ If the site has not been uploaded yet, set `site_id` to `null` and include `site_upload_command` or `site_upload_api`.
251
+
252
+ ## Pipeline
253
+
254
+ 1. Intake source media and notes.
255
+ 2. Extract `ffprobe`, contact sheet, scene cuts, OCR, transcript, and simple layout/subject regions.
256
+ 3. Generate `production_graph.json` with `extractor-system-prompt.md`.
257
+ 4. Generate `template_plan.json` with `template-planner-prompt.md`.
258
+ 5. Write `composition.html`, `composition.json`, and `lambda-site.json`.
259
+ 6. Validate locally in the editor.
260
+ 7. Stage the project with the production Lambda site flow.
261
+ 8. Publish through Lambda and verify the final output URL.
262
+
263
+ ## Required Validation
264
+
265
+ Before publishing:
266
+
267
+ - every timed layer has non-negative `start` and positive `duration`
268
+ - no two editable layers overlap on the same track unless the track is intentionally composited
269
+ - scene video layers use `media_start`/`playbackStart` rather than an always-on background dependency
270
+ - deleting a scene removes that visual range from publish output
271
+ - captions are legible at phone size
272
+ - caption boxes avoid unsafe TikTok/Reels zones unless the source intentionally uses them
273
+ - source audio and source video are separate layers when audio is reused continuously
274
+ - Lambda memory estimate has a concrete rationale
275
+ - `lambda-site.json` does not contain credentials