@mevdragon/vidfarm-devcli 0.2.1 → 0.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +6 -39
- package/GETTING_STARTED.developers.md +87 -0
- package/README.md +94 -238
- package/SKILL.developer.md +430 -104
- package/dist/src/account-pages.js +1 -1
- package/dist/src/app.js +93 -5
- package/dist/src/cli.js +456 -8
- package/dist/src/config.js +3 -2
- package/dist/src/context.js +30 -11
- package/dist/src/db.js +2 -57
- package/dist/src/dev-app.js +0 -1
- package/dist/src/index.js +4 -2
- package/dist/src/lib/template-paths.js +21 -0
- package/dist/src/runtime.js +3 -1
- package/dist/src/services/auth.js +4 -4
- package/dist/src/services/job-logs.js +186 -0
- package/dist/src/services/jobs.js +3 -2
- package/dist/src/services/providers.js +14 -6
- package/dist/src/services/storage.js +85 -2
- package/dist/src/services/template-sources.js +29 -3
- package/dist/templates/template_0000/src/lib/images.js +46 -86
- package/dist/templates/template_0000/src/template.js +277 -53
- package/package.json +5 -6
- package/templates/template_0000/README.md +8 -52
- package/templates/template_0000/SKILL.md +35 -3
- package/templates/template_0000/package.json +3 -6
- package/templates/template_0000/src/lib/images.js +46 -86
- package/templates/template_0000/src/lib/images.ts +55 -98
- package/templates/template_0000/src/template-dna.js +9 -0
- package/templates/template_0000/src/template.js +523 -199
- package/templates/template_0000/src/template.ts +356 -61
- package/templates/template_0000/template.config.json +7 -12
- package/AWS_REMOTION_HANDOFF.md +0 -311
- package/PLATFORM_SPEC.md +0 -1039
- package/SKILL.director.md +0 -599
- package/dist/infra/cdk/bin/vidfarm-prod.js +0 -59
- package/dist/infra/cdk/lib/vidfarm-prod-stack.js +0 -212
- package/templates/template_0000/package-lock.json +0 -5505
- package/templates/template_0000/scripts/create-site.mjs +0 -27
- package/templates/template_0000/scripts/render-cloud.mjs +0 -72
package/.env.example
CHANGED
|
@@ -1,48 +1,15 @@
|
|
|
1
1
|
NODE_ENV="development"
|
|
2
2
|
PORT="3000"
|
|
3
3
|
|
|
4
|
-
VIDFARM_DB_PATH="./data/vidfarm.sqlite"
|
|
5
|
-
VIDFARM_DATA_DIR="./data"
|
|
6
|
-
|
|
7
|
-
ENCRYPTION_SECRET="replace-with-a-long-random-secret"
|
|
8
|
-
API_KEY_SALT="replace-with-a-second-long-random-secret"
|
|
9
|
-
WEBHOOK_SECRET="replace-with-a-third-long-random-secret"
|
|
10
|
-
|
|
11
|
-
WORKER_POLL_MS="1500"
|
|
12
|
-
WORKER_BATCH_SIZE="2"
|
|
13
|
-
WORKER_MAX_CONCURRENT_JOBS="1"
|
|
14
|
-
DEFAULT_JOB_DELAY_SECONDS="20"
|
|
15
|
-
MAX_PENDING_JOBS_GLOBAL="0"
|
|
16
|
-
MAX_PENDING_JOBS_PER_CUSTOMER="0"
|
|
17
|
-
|
|
18
|
-
PUBLIC_BASE_URL="https://vidfarm.cloud.zoomgtm.com"
|
|
19
|
-
VIDFARM_ADMIN_EMAILS="admin@example.com"
|
|
20
|
-
VIDFARM_DEVELOPER_EMAILS="developer@example.com,operator@zoom-gtm.com,entropyinternetorg@gmail.com"
|
|
21
|
-
TEMPLATE_SOURCE_ROOT="./data/template-sources"
|
|
22
|
-
STORAGE_DRIVER="local"
|
|
23
|
-
AWS_REGION="us-east-1"
|
|
24
|
-
AWS_S3_BUCKET=""
|
|
25
|
-
AWS_S3_PUBLIC_READ="false"
|
|
26
|
-
AWS_S3_ENDPOINT=""
|
|
27
|
-
AWS_ACCESS_KEY_ID=""
|
|
28
|
-
AWS_SECRET_ACCESS_KEY=""
|
|
29
|
-
|
|
30
|
-
RESEND_API_KEY=""
|
|
31
|
-
RESEND_FROM_EMAIL="vidfarm@fwd.zoomgtm.com"
|
|
32
|
-
|
|
33
4
|
OPENAI_API_KEY=""
|
|
34
5
|
OPENROUTER_API_KEY=""
|
|
35
6
|
GEMINI_API_KEY=""
|
|
36
7
|
PERPLEXITY_API_KEY=""
|
|
37
8
|
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
REMOTION_SITE_NAME="vidfarm-template-0000"
|
|
42
|
-
REMOTION_SERVE_URL=""
|
|
43
|
-
REMOTION_COMPOSITION_ID="template-0000"
|
|
44
|
-
REMOTION_AWS_ACCESS_KEY_ID=""
|
|
45
|
-
REMOTION_AWS_SECRET_ACCESS_KEY=""
|
|
46
|
-
REMOTION_MODE="auto"
|
|
9
|
+
# Optional when calling a hosted Vidfarm API directly.
|
|
10
|
+
VIDFARM_USER_ID=""
|
|
11
|
+
VIDFARM_API_KEY=""
|
|
47
12
|
|
|
48
|
-
|
|
13
|
+
# Optional local overrides. The CLI will choose sensible local defaults.
|
|
14
|
+
PUBLIC_BASE_URL="http://localhost:3310"
|
|
15
|
+
REMOTION_MODE="local"
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# Vidfarm Developers
|
|
2
|
+
|
|
3
|
+
## Interview Phase
|
|
4
|
+
|
|
5
|
+
Prepare for your Interview:
|
|
6
|
+
|
|
7
|
+
1. Browse the Winning Formats Gallery of viral TikTok videos here. These are samples of what you'd be recreating for customers:
|
|
8
|
+
https://winning-formats-gallery-tiktok.cloud.zoomgtm.com/
|
|
9
|
+
|
|
10
|
+
2. Setup OpenAI Codex on your computer with free tier. This is your primary AI coding agent (really good to know, even for future jobs)
|
|
11
|
+
https://developers.openai.com/codex/cli
|
|
12
|
+
https://www.youtube.com/watch?v=GJsx_FuiwNg
|
|
13
|
+
|
|
14
|
+
3. Setup Remotion Skill on your computer. Codex will handle all the coding work, you just need to be able to get it to render the output videos/images. no need for suno, just image gen is enough.
|
|
15
|
+
https://www.youtube.com/watch?v=Xdy1vkhSz-M
|
|
16
|
+
|
|
17
|
+
4. You’ll need to acquire an ai api key that can generate images at least. There’s many free and paid ones out there. Part of the skills test is to be resourceful and find it so you can test with (hint: check huggingface, openrouter, and ask codex to suggest others). You will not be penalized for using a weaker free ai
|
|
18
|
+
|
|
19
|
+
5. Once you are ready, request the interview with @zoom_gtm and he will give you the skills test, gemini api key, and invite to first week trial run hands on helping customers (you will be paid for your time). You'd be recreating from the winning formats gallery
|
|
20
|
+
|
|
21
|
+
The skills test:
|
|
22
|
+
|
|
23
|
+
try to build your own workflow in VSCode + Codex cli + prompt.md files to generate a winning template. It should be easy to re-run your template to generate infinite content of that type. You may ask codex to explain what I mean
|
|
24
|
+
|
|
25
|
+
## Pre-Setup for Skills Test
|
|
26
|
+
|
|
27
|
+
Hello Creative Directors! You can
|
|
28
|
+
|
|
29
|
+
1. Create a Github account and setup SSH key so you can save your work & login on github CLI (ask codex how):
|
|
30
|
+
https://github.com
|
|
31
|
+
|
|
32
|
+
2. Create an NPM account and login on npm CLI (ask codex how):
|
|
33
|
+
https://www.npmjs.com
|
|
34
|
+
|
|
35
|
+
3. DM me your github username & npm username. congrats! you are a developer now
|
|
36
|
+
|
|
37
|
+
4. Make sure you are familiar with codex & api keys prepared and have tried all of the above. I will provide a gemini ai api key for the skills test.
|
|
38
|
+
|
|
39
|
+
MAKE SURE YOU ARE PREPARED. If I have to babysit you during the call, I will immediatlely end the interview early to not waste anyones time.
|
|
40
|
+
|
|
41
|
+
5. Final step, schedule your 1:1 skills test interview. do not book time until you have finished prior steps.
|
|
42
|
+
|
|
43
|
+
https://calendly.com/zoom-gtm/creative-director-interviews
|
|
44
|
+
|
|
45
|
+
## Setup for Job
|
|
46
|
+
|
|
47
|
+
0. Tell your manager your email so he can whitelist you access to Vidfarm. After he confirms you have access, login at https://vidfarm.cloud.zoomgtm.com
|
|
48
|
+
|
|
49
|
+
1. Open VSCode in a new project called "Vidfarm-Developer"
|
|
50
|
+
|
|
51
|
+
2. Open terminal and login to github cli and npm cli (ask ai/codex if you need help). ask your manager if they gave you npm access to "vidfarm-devcli" yet.
|
|
52
|
+
|
|
53
|
+
3. Open Codex cli in terminal, with /permissions set to "Full Access"
|
|
54
|
+
|
|
55
|
+
4. Install the private skill by copying it from https://vidfarm.cloud.zoomgtm.com/settings in "Developer SKILL.md" button and pasting it into the below prompt for codex.
|
|
56
|
+
|
|
57
|
+
```txt
|
|
58
|
+
Please take this agent skill and install it locally. Then install the vidfarm-devcli from npm, ask me for the otp if need, I will enter it. Here is the agent skill: [pasted]
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
5. Create a .env file at folder root with your AI API KEY. You need at least 1 of 3 and it doesn't matter which. We recommend Gemini. Ask your manager for an AI API key.
|
|
62
|
+
|
|
63
|
+
```.env
|
|
64
|
+
GEMINI_API_KEY="______"
|
|
65
|
+
OPENAI_API_KEY="______"
|
|
66
|
+
OPENROUTER_API_KEY="______"
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
6. Extend that .env file with another secret VIDFARM_API_KEY, you can get it at https://vidfarm.cloud.zoomgtm.com/settings
|
|
70
|
+
|
|
71
|
+
```.env
|
|
72
|
+
VIDFARM_API_KEY="_______"
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
6. Create a new folder called `src/`, this is where we are going to do most of our development work.
|
|
76
|
+
|
|
77
|
+
7. Tell codex to:
|
|
78
|
+
|
|
79
|
+
```txt
|
|
80
|
+
use the vidfarm-devcli to copy-reference-template into src/vidfarm_template_0000 so i can view it as example
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
That will populate with a reference folder of what a template code looks like. We are going to create a new template.
|
|
84
|
+
|
|
85
|
+
8. Create a new folder beside src called `drafts/` which is where we will put messy files when needed.
|
|
86
|
+
|
|
87
|
+
9. Now find an easy winning format from https://winning-formats-gallery-tiktok.cloud.zoomgtm.com/ and save the image to your drafts/ folder. then tell Codex to look at that image (right click, copy relative path, paste to codex), then tell codex please make a new template based on the image.
|
package/README.md
CHANGED
|
@@ -1,300 +1,156 @@
|
|
|
1
|
-
# Vidfarm
|
|
1
|
+
# Vidfarm Dev CLI
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
`@mevdragon/vidfarm-devcli` is the local development toolkit for third-party Vidfarm template authors.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
This package is intentionally scoped to:
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
- building templates under `templates/*`
|
|
8
|
+
- running the local Vidfarm API and worker
|
|
9
|
+
- testing template operations as async jobs
|
|
10
|
+
- validating template metadata, skills, and smoke test payloads
|
|
11
|
+
- rendering Remotion locally when a template uses Remotion
|
|
8
12
|
|
|
9
|
-
|
|
10
|
-
- Async template API under `/api/v1/templates/:templateId/*`
|
|
11
|
-
- User auth/profile/provider-key routes under `/api/v1/user/*`
|
|
12
|
-
- Email OTP login with API key issuance
|
|
13
|
-
- `is_developer` user flag, assigned from allowlisted emails, required for template source submission
|
|
14
|
-
- Customer-scoped provider key vault with encryption at rest
|
|
15
|
-
- SQLite-backed job queue and provider-key lease coordination
|
|
16
|
-
- Structured job logs, artifacts, billing events, and webhook delivery queue
|
|
17
|
-
- Local storage mode plus S3-compatible mode for AWS
|
|
18
|
-
- Remotion adapter seam and a sample `template_0000` slideshow pipeline
|
|
13
|
+
It is intentionally not documentation for platform internals, production deployment, admin release flow, or shared cloud infrastructure.
|
|
19
14
|
|
|
20
15
|
## Quick Start
|
|
21
16
|
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
npm run dev
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
The service starts on `http://localhost:3000` by default and creates SQLite plus local artifact storage under `./data`.
|
|
29
|
-
|
|
30
|
-
## Template Developer CLI
|
|
31
|
-
|
|
32
|
-
For local template development, use the bundled CLI instead of manually wiring auth, provider keys, SQLite, and Remotion:
|
|
17
|
+
1. Create a local `.env` from the example file.
|
|
18
|
+
2. Add at least one AI provider key.
|
|
19
|
+
3. Start the local CLI runtime.
|
|
33
20
|
|
|
34
21
|
```bash
|
|
35
|
-
|
|
22
|
+
cp .env.example .env
|
|
36
23
|
npx @mevdragon/vidfarm-devcli dev --port 3310 --reset
|
|
37
24
|
```
|
|
38
25
|
|
|
39
|
-
What it does:
|
|
40
|
-
|
|
41
|
-
- starts the same REST API and in-process worker as production
|
|
42
|
-
- provisions a local SQLite database automatically
|
|
43
|
-
- uses local filesystem storage as an S3 stand-in
|
|
44
|
-
- seeds a dev customer and API key
|
|
45
|
-
- seeds provider keys from local `.env` variables like `OPENAI_API_KEY`, `GEMINI_API_KEY`, and `OPENROUTER_API_KEY`
|
|
46
|
-
- falls back to mock provider keys plus mocked AI responses when no env keys are present
|
|
47
|
-
- renders Remotion compositions locally instead of using Lambda
|
|
48
|
-
|
|
49
26
|
Useful commands:
|
|
50
27
|
|
|
51
28
|
```bash
|
|
52
29
|
npx @mevdragon/vidfarm-devcli dev --port 3310 --reset
|
|
53
30
|
npx @mevdragon/vidfarm-devcli session
|
|
54
31
|
npx @mevdragon/vidfarm-devcli validate-template --template-id template_0000
|
|
55
|
-
npx @mevdragon/vidfarm-devcli
|
|
56
|
-
npx @mevdragon/vidfarm-devcli
|
|
57
|
-
npx @mevdragon/vidfarm-devcli analyze-
|
|
32
|
+
npx @mevdragon/vidfarm-devcli copy-reference-template --template-dir ./src/vidfarm_template_0000
|
|
33
|
+
npx @mevdragon/vidfarm-devcli generate-template --slug-id template_0001 --template-dir ./templates/vidfarm_template_0001
|
|
34
|
+
npx @mevdragon/vidfarm-devcli analyze-viral-dna --template-dir ./templates/vidfarm_template_0001
|
|
35
|
+
npx @mevdragon/vidfarm-devcli analyze-visual-dna --template-dir ./templates/vidfarm_template_0001
|
|
36
|
+
npx @mevdragon/vidfarm-devcli presign-preview-media --file ./drafts/preview/example.png --directory drafts/preview
|
|
58
37
|
```
|
|
59
38
|
|
|
60
|
-
After install, the package
|
|
61
|
-
|
|
62
|
-
`vidfarm session` prints reusable auth headers and a sample `curl` request for the local REST API.
|
|
63
|
-
|
|
64
|
-
`generate-template` now scaffolds a standard `research/` area for new templates:
|
|
65
|
-
|
|
66
|
-
- `research/source_notes.md` for original-format notes
|
|
67
|
-
- `research/preview/` for source screenshots or video
|
|
68
|
-
- `src/template-dna.ts` as the checked-in metadata module that feeds `about.viral_dna`, `about.visual_dna`, and `about.link_to_original`
|
|
69
|
-
|
|
70
|
-
If preview media is already present, or if you pass `--source-preview-dir`, scaffold generation will run both DNA analyzers automatically unless `--skip-dna-analysis` is set.
|
|
71
|
-
|
|
72
|
-
## Environment
|
|
73
|
-
|
|
74
|
-
Primary variables:
|
|
75
|
-
|
|
76
|
-
- `PORT`
|
|
77
|
-
- `VIDFARM_DB_PATH`
|
|
78
|
-
- `VIDFARM_DATA_DIR`
|
|
79
|
-
- `VIDFARM_ADMIN_EMAILS`
|
|
80
|
-
- `TEMPLATE_SOURCE_ROOT`
|
|
81
|
-
- `ENCRYPTION_SECRET`
|
|
82
|
-
- `API_KEY_SALT`
|
|
83
|
-
- `STORAGE_DRIVER=local|s3`
|
|
84
|
-
- `AWS_REGION`
|
|
85
|
-
- `AWS_S3_BUCKET`
|
|
86
|
-
- `AWS_S3_PUBLIC_READ=true|false`
|
|
87
|
-
- `PUBLIC_BASE_URL`
|
|
88
|
-
- `RESEND_API_KEY`
|
|
89
|
-
- `WEBHOOK_SECRET`
|
|
90
|
-
- `MOCK_PROVIDER_RESPONSES=true|false`
|
|
91
|
-
- `VIDFARM_DEVELOPER_EMAILS`
|
|
92
|
-
- `WORKER_MAX_CONCURRENT_JOBS`
|
|
93
|
-
- `MAX_PENDING_JOBS_GLOBAL`
|
|
94
|
-
- `MAX_PENDING_JOBS_PER_CUSTOMER`
|
|
95
|
-
|
|
96
|
-
Remotion-specific variables:
|
|
97
|
-
|
|
98
|
-
- `REMOTION_REGION`
|
|
99
|
-
- `REMOTION_FUNCTION_NAME`
|
|
100
|
-
- `REMOTION_BUCKET_NAME`
|
|
101
|
-
- `REMOTION_SITE_NAME`
|
|
102
|
-
- `REMOTION_SERVE_URL`
|
|
103
|
-
- `REMOTION_COMPOSITION_ID`
|
|
104
|
-
- `REMOTION_AWS_ACCESS_KEY_ID`
|
|
105
|
-
- `REMOTION_AWS_SECRET_ACCESS_KEY`
|
|
106
|
-
|
|
107
|
-
When `MOCK_PROVIDER_RESPONSES=true`, AI provider calls use the real key-leasing flow but return mocked content. This makes local end-to-end testing possible without burning customer API calls.
|
|
108
|
-
|
|
109
|
-
When using `npx @mevdragon/vidfarm-devcli dev`, the CLI automatically chooses local-friendly defaults:
|
|
110
|
-
|
|
111
|
-
- `STORAGE_DRIVER=local`
|
|
112
|
-
- `REMOTION_MODE=local`
|
|
113
|
-
- a local SQLite path under `.vidfarm/local`
|
|
114
|
-
- `MOCK_PROVIDER_RESPONSES=false` if real provider env keys are present, otherwise `true`
|
|
115
|
-
|
|
116
|
-
## Remotion AWS
|
|
39
|
+
After install, the package exposes both `vidfarm` and `vidfarm-devcli` as local binaries.
|
|
117
40
|
|
|
118
|
-
|
|
41
|
+
## What The Local CLI Does
|
|
119
42
|
|
|
120
|
-
|
|
43
|
+
The local CLI runs the same template contract used by the hosted platform, but with local-first defaults:
|
|
121
44
|
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
45
|
+
- local SQLite state under `.vidfarm/local`
|
|
46
|
+
- local filesystem storage instead of cloud object storage
|
|
47
|
+
- local Remotion rendering
|
|
48
|
+
- a seeded developer session for local testing
|
|
49
|
+
- provider keys sourced from your local `.env`
|
|
50
|
+
- mocked provider responses only when no real provider keys are present
|
|
127
51
|
|
|
128
|
-
|
|
52
|
+
Use `vidfarm session` to print reusable auth headers and a sample `curl` request for the local REST API.
|
|
129
53
|
|
|
130
|
-
-
|
|
131
|
-
- shared Lambda function: `remotion-render-4-0-355-mem2048mb-disk2048mb-180sec`
|
|
132
|
-
- shared bucket: `remotionlambda-useast1-ujg7c0h43q`
|
|
133
|
-
- Remotion package version to pin: `4.0.355`
|
|
54
|
+
For hosted preview-media uploads, use `vidfarm-devcli presign-preview-media`. It calls the authenticated Vidfarm API with `VIDFARM_API_KEY` and returns a presigned PUT URL under `developer/<user_id>/preview_media/*` plus the resulting Vidfarm media URL.
|
|
134
55
|
|
|
135
|
-
|
|
56
|
+
## What The Hosted Platform Expects From Templates
|
|
136
57
|
|
|
137
|
-
|
|
138
|
-
- `REMOTION_AWS_SECRET_ACCESS_KEY`
|
|
58
|
+
Vidfarm treats templates as async job producers, not as ad hoc endpoints.
|
|
139
59
|
|
|
140
|
-
|
|
60
|
+
Each template should provide:
|
|
141
61
|
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
REMOTION_SERVE_URL=https://remotionlambda-useast1-ujg7c0h43q.s3.us-east-1.amazonaws.com/sites/your-site-name/index.html
|
|
150
|
-
REMOTION_COMPOSITION_ID=YourComposition
|
|
151
|
-
```
|
|
152
|
-
|
|
153
|
-
If you wire real Lambda rendering into `src/services/remotion.ts`, preserve the current platform contract and make the service read these env vars rather than hard-coding one-off values.
|
|
154
|
-
|
|
155
|
-
## API Flow
|
|
156
|
-
|
|
157
|
-
1. `POST /api/v1/user/request-otp`
|
|
158
|
-
2. `POST /api/v1/user/verify-otp`
|
|
159
|
-
3. `POST /api/v1/user/me/provider-keys`
|
|
160
|
-
4. `GET /api/v1/templates/template_0000`
|
|
161
|
-
5. `GET /api/v1/templates/template_0000/skill`
|
|
162
|
-
6. `POST /api/v1/templates/template_0000/config`
|
|
163
|
-
7. `POST /api/v1/templates/template_0000/operations/create_slideshow`
|
|
164
|
-
8. Poll `GET /api/v1/templates/template_0000/jobs/:jobId`
|
|
165
|
-
9. List or filter jobs with `GET /api/v1/templates/template_0000/jobs?tracer=...&start_time=...&end_time=...`
|
|
166
|
-
10. Stream timeline data from `GET /api/v1/templates/template_0000/jobs/:jobId/logs?start_time=...&end_time=...`
|
|
167
|
-
|
|
168
|
-
## Template Release Flow
|
|
169
|
-
|
|
170
|
-
Templates can still live in this repo, but the platform now supports a GitHub-backed release workflow for in-house templates:
|
|
171
|
-
|
|
172
|
-
1. developer keeps template code in GitHub
|
|
173
|
-
2. the approved branch convention is `production`
|
|
174
|
-
3. admin registers the template source with repo URL, module path, build command, and `SKILL.md` path
|
|
175
|
-
4. admin imports a specific commit, or the current `production` head
|
|
176
|
-
5. platform builds the repo, loads the template module, and runs certification
|
|
177
|
-
6. only certified releases can be activated
|
|
178
|
-
7. activation pins the live template to a specific commit SHA, not a floating branch head
|
|
62
|
+
- stable template metadata
|
|
63
|
+
- a `SKILL.md` file for users and agents
|
|
64
|
+
- a TypeScript entrypoint at `src/template.ts`
|
|
65
|
+
- one or more named operations
|
|
66
|
+
- workflow functions behind those operations
|
|
67
|
+
- a `smokeTestPayload` for each operation
|
|
68
|
+
- `about.viral_dna` and `about.visual_dna`
|
|
179
69
|
|
|
180
|
-
|
|
70
|
+
At runtime, the platform uses the template wrapper to expose a consistent API shape:
|
|
181
71
|
|
|
182
|
-
- `GET /api/v1/templates
|
|
183
|
-
- `
|
|
184
|
-
- `
|
|
185
|
-
- `POST /api/v1/templates/
|
|
186
|
-
- `
|
|
72
|
+
- `GET /api/v1/templates/:templateId`
|
|
73
|
+
- `GET /api/v1/templates/:templateId/skill`
|
|
74
|
+
- `POST /api/v1/templates/:templateId/config`
|
|
75
|
+
- `POST /api/v1/templates/:templateId/operations/:operationName`
|
|
76
|
+
- `GET /api/v1/templates/:templateId/jobs/:jobId`
|
|
77
|
+
- `GET /api/v1/templates/:templateId/jobs/:jobId/logs`
|
|
187
78
|
|
|
188
|
-
|
|
79
|
+
Operations should return work through the job system rather than inventing one-off synchronous routes.
|
|
189
80
|
|
|
190
|
-
##
|
|
81
|
+
## Approval Flow
|
|
191
82
|
|
|
192
|
-
|
|
83
|
+
Third-party developers build and validate templates locally first.
|
|
193
84
|
|
|
194
|
-
|
|
195
|
-
- both `about.viral_dna` and `about.visual_dna`
|
|
196
|
-
- at least one operation
|
|
197
|
-
- valid workflow references
|
|
198
|
-
- a `smokeTestPayload` for each operation
|
|
85
|
+
After handoff, a Vidfarm admin reviews the template before it is made available on the hosted platform. Template authors should assume there is an approval step between local development and production availability.
|
|
199
86
|
|
|
200
|
-
|
|
87
|
+
## Template Deploy Cycle
|
|
201
88
|
|
|
202
|
-
|
|
203
|
-
npx @mevdragon/vidfarm-devcli validate-template --template-id template_0000
|
|
204
|
-
```
|
|
89
|
+
When someone says "template deploy cycle" in this repo, they mean this exact sequence:
|
|
205
90
|
|
|
206
|
-
|
|
91
|
+
1. commit and `git push` the repo that contains the template folder, such as `templates/vidfarm_template_<slug>/` inside a monorepo or a standalone template repo
|
|
92
|
+
2. as the platform admin, pull/import that template commit into Vidfarm
|
|
93
|
+
3. approve and activate the imported template release in the hosted platform
|
|
94
|
+
4. run the prod deploy script if platform-runtime code changed, or reuse the current prod image if the change was template-only and you just need a formal deploy-cycle restart
|
|
207
95
|
|
|
208
|
-
|
|
96
|
+
For `template_0000`, that usually means:
|
|
209
97
|
|
|
210
98
|
```bash
|
|
211
|
-
|
|
99
|
+
git -C templates/vidfarm_template_0000 push origin production
|
|
100
|
+
# then run the platform operator command
|
|
101
|
+
node dist/src/cli.js deploy-template-cycle \
|
|
102
|
+
--env-file .env.production \
|
|
103
|
+
--stack-name VidfarmProdStack \
|
|
104
|
+
--template-id 4c7a7e1a-7f35-4f30-9f86-9c8a63c7f2db \
|
|
105
|
+
--slug-id template_0000 \
|
|
106
|
+
--repo-url https://github.com/your-org/your-template-repo \
|
|
107
|
+
--branch production \
|
|
108
|
+
--template-module-path templates/vidfarm_template_0000/src/template.ts
|
|
212
109
|
```
|
|
213
110
|
|
|
214
|
-
|
|
111
|
+
The important distinction is:
|
|
215
112
|
|
|
216
|
-
|
|
113
|
+
- template repo changes ship through template source import and activation
|
|
114
|
+
- platform code changes ship through the root prod deploy script
|
|
115
|
+
- multiple templates can live in one git repo as long as each template has its own folder and its `template_module_path` points at that folder's `src/template.ts`
|
|
217
116
|
|
|
218
|
-
|
|
219
|
-
- SQLite WAL/SHM files
|
|
220
|
-
- imported template source checkouts
|
|
221
|
-
- any local fallback runtime data under `/app/data`
|
|
117
|
+
## Authentication Context
|
|
222
118
|
|
|
223
|
-
|
|
119
|
+
When calling a hosted Vidfarm API directly, authenticated requests use:
|
|
224
120
|
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
AWS_REGION=us-east-1
|
|
228
|
-
AWS_S3_BUCKET=your-vidfarm-bucket
|
|
229
|
-
AWS_S3_PUBLIC_READ=false
|
|
230
|
-
PUBLIC_BASE_URL=https://vidfarm.cloud.zoomgtm.com
|
|
231
|
-
```
|
|
121
|
+
- `vidfarm-user-id`
|
|
122
|
+
- `vidfarm-api-key`
|
|
232
123
|
|
|
233
|
-
|
|
124
|
+
The API key comes from Vidfarm login and settings, not from platform infrastructure secrets.
|
|
125
|
+
If you need to discover the authenticated customer ID first, resolve it from `GET /api/v1/user/me` and then send both headers on subsequent requests.
|
|
234
126
|
|
|
235
|
-
- `
|
|
236
|
-
- `WORKER_BATCH_SIZE=2` by default
|
|
237
|
-
- `MAX_PENDING_JOBS_GLOBAL=0` by default
|
|
238
|
-
- `MAX_PENDING_JOBS_PER_CUSTOMER=0` by default
|
|
127
|
+
For hosted template source registration, treat `template_module_path` as repo-relative and point it at the template's TypeScript entrypoint, for example `templates/vidfarm_template_hooks/src/template.ts`. Each template folder must start with `vidfarm_template_`. The import flow builds the repo and loads the compiled `src/template.js` sibling at activation time.
|
|
239
128
|
|
|
240
|
-
|
|
129
|
+
For private GitHub repos, export `VIDFARM_GITHUB_TOKEN` (or `GITHUB_TOKEN`) before running `import-source-prod` or `deploy-template-cycle`. The prod operator command injects that token into the live container's git config for the import and keeps the stored `repo_url` clean.
|
|
241
130
|
|
|
242
|
-
|
|
131
|
+
For local CLI usage, the `session` command gives you a local seeded session so you can test without manually wiring auth state.
|
|
243
132
|
|
|
244
|
-
|
|
133
|
+
## Provider Keys
|
|
245
134
|
|
|
246
|
-
|
|
247
|
-
- one public ALB with HTTP to HTTPS redirect
|
|
248
|
-
- one ACM certificate for `vidfarm.cloud.zoomgtm.com`
|
|
249
|
-
- one EC2 instance running the Docker image
|
|
250
|
-
- one dedicated encrypted EBS data volume retained separately from the container
|
|
251
|
-
- Route53 `A` and `AAAA` alias records for the production hostname
|
|
135
|
+
Templates should work with the developer's own upstream AI keys.
|
|
252
136
|
|
|
253
|
-
|
|
137
|
+
Typical local `.env`:
|
|
254
138
|
|
|
255
|
-
```
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
export VIDFARM_RECORD_NAME=vidfarm
|
|
261
|
-
export VIDFARM_PUBLIC_BASE_URL=https://vidfarm.cloud.zoomgtm.com
|
|
262
|
-
export VIDFARM_ADMIN_EMAILS=admin@zoomgtm.com
|
|
263
|
-
export VIDFARM_DEVELOPER_EMAILS=dev1@zoomgtm.com,dev2@zoomgtm.com
|
|
264
|
-
export VIDFARM_ENCRYPTION_SECRET='replace-with-a-long-random-secret'
|
|
265
|
-
export VIDFARM_API_KEY_SALT='replace-with-a-second-long-random-secret'
|
|
266
|
-
export VIDFARM_WEBHOOK_SECRET='replace-with-a-third-long-random-secret'
|
|
139
|
+
```env
|
|
140
|
+
GEMINI_API_KEY=""
|
|
141
|
+
OPENAI_API_KEY=""
|
|
142
|
+
OPENROUTER_API_KEY=""
|
|
143
|
+
PERPLEXITY_API_KEY=""
|
|
267
144
|
```
|
|
268
145
|
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
- `VIDFARM_RESEND_API_KEY`
|
|
272
|
-
- `VIDFARM_RESEND_FROM_EMAIL`
|
|
273
|
-
- `VIDFARM_OPENAI_API_KEY`
|
|
274
|
-
- `VIDFARM_OPENROUTER_API_KEY`
|
|
275
|
-
- `VIDFARM_GEMINI_API_KEY`
|
|
276
|
-
- `VIDFARM_PERPLEXITY_API_KEY`
|
|
277
|
-
- `VIDFARM_REMOTION_*`
|
|
278
|
-
- `VIDFARM_INSTANCE_TYPE`
|
|
279
|
-
- `VIDFARM_DATA_VOLUME_SIZE_GIB`
|
|
280
|
-
- `VIDFARM_WORKER_MAX_CONCURRENT_JOBS`
|
|
281
|
-
- `VIDFARM_MAX_PENDING_JOBS_GLOBAL`
|
|
282
|
-
- `VIDFARM_MAX_PENDING_JOBS_PER_CUSTOMER`
|
|
283
|
-
|
|
284
|
-
If you do not set `VIDFARM_INSTANCE_TYPE`, the CDK app now defaults to `t3.small`.
|
|
146
|
+
You usually only need one provider key to develop and test a template locally.
|
|
285
147
|
|
|
286
|
-
|
|
148
|
+
## Remotion
|
|
287
149
|
|
|
288
|
-
|
|
289
|
-
npm install
|
|
290
|
-
npm run cdk:synth
|
|
291
|
-
npx aws-cdk bootstrap aws://$CDK_DEFAULT_ACCOUNT/$CDK_DEFAULT_REGION
|
|
292
|
-
npm run cdk:deploy
|
|
293
|
-
```
|
|
150
|
+
For third-party template authors, Remotion is local.
|
|
294
151
|
|
|
295
|
-
|
|
152
|
+
If your template uses `ctx.remotion`, develop and validate it locally through the CLI. This package does not document or require shared cloud Remotion infrastructure.
|
|
296
153
|
|
|
297
|
-
##
|
|
154
|
+
## Next Read
|
|
298
155
|
|
|
299
|
-
|
|
300
|
-
- Remotion is still stubbed behind [src/services/remotion.ts](/Users/localghost/Projects/OfficeX/OfficeX/ZoomGTM/vidfarm/src/services/remotion.ts). When upgrading it to real Lambda submission, follow [AWS_REMOTION_HANDOFF.md](/Users/localghost/Projects/OfficeX/OfficeX/ZoomGTM/vidfarm/AWS_REMOTION_HANDOFF.md) and use the Remotion-specific AWS credentials already available in local production env config.
|
|
156
|
+
For the external template-author workflow, continue with [GETTING_STARTED.developers.md](./GETTING_STARTED.developers.md) and the example template in [`templates/template_0000`](./templates/template_0000).
|