@mevdragon/vidfarm-devcli 0.2.6 → 0.2.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/GETTING_STARTED.developers.md +29 -0
- package/README.md +29 -2
- package/SKILL.developer.md +9 -2
- package/dist/src/app.js +77 -6
- package/dist/src/cli.js +104 -53
- package/dist/src/config.js +2 -0
- package/dist/src/db.js +34 -11
- package/dist/src/services/template-sources.js +90 -13
- package/package.json +1 -1
|
@@ -85,3 +85,32 @@ That will populate with a reference folder of what a template code looks like. W
|
|
|
85
85
|
8. Create a new folder beside src called `drafts/` which is where we will put messy files when needed.
|
|
86
86
|
|
|
87
87
|
9. Now find an easy winning format from https://winning-formats-gallery-tiktok.cloud.zoomgtm.com/ and save the image to your drafts/ folder. then tell Codex to look at that image (right click, copy relative path, paste to codex), then tell codex please make a new template based on the image.
|
|
88
|
+
|
|
89
|
+
## Publish Responsibility
|
|
90
|
+
|
|
91
|
+
When your template is ready for hosted Vidfarm, your responsibility is only:
|
|
92
|
+
|
|
93
|
+
1. push your template code to GitHub
|
|
94
|
+
2. register or update the template source metadata so Vidfarm knows the repo URL, branch, and template folder path
|
|
95
|
+
|
|
96
|
+
Your responsibility does not include:
|
|
97
|
+
|
|
98
|
+
1. importing the source into the live platform
|
|
99
|
+
2. activating the release
|
|
100
|
+
3. making the template public/live
|
|
101
|
+
|
|
102
|
+
Those are platform-admin steps. If an API route says `403 Admin access required` for import or activate, that means you reached the correct handoff boundary.
|
|
103
|
+
|
|
104
|
+
When you need to register your template with hosted Vidfarm, use the dev CLI command below from your repo root:
|
|
105
|
+
|
|
106
|
+
```bash
|
|
107
|
+
npx @mevdragon/vidfarm-devcli register-source-hosted \
|
|
108
|
+
--env-file .env \
|
|
109
|
+
--template-id <uuid-v4> \
|
|
110
|
+
--slug-id <template_slug> \
|
|
111
|
+
--repo-url https://github.com/<you>/<repo> \
|
|
112
|
+
--branch main \
|
|
113
|
+
--template-module-path templates/vidfarm_template_<slug>/src/template.ts
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
That command uses your `VIDFARM_API_KEY` and sends the registration to the hosted REST API, so the template appears in Vidfarm's review queue with `pending_review` status.
|
package/README.md
CHANGED
|
@@ -84,6 +84,19 @@ Third-party developers build and validate templates locally first.
|
|
|
84
84
|
|
|
85
85
|
After handoff, a Vidfarm admin reviews the template before it is made available on the hosted platform. Template authors should assume there is an approval step between local development and production availability.
|
|
86
86
|
|
|
87
|
+
The developer handoff ends at:
|
|
88
|
+
|
|
89
|
+
- `git push` of the template code
|
|
90
|
+
- template source registration or registration update
|
|
91
|
+
|
|
92
|
+
The developer handoff does not include:
|
|
93
|
+
|
|
94
|
+
- importing the source into the live platform
|
|
95
|
+
- activating a release
|
|
96
|
+
- making the template public/live
|
|
97
|
+
|
|
98
|
+
If a developer receives `403 Admin access required` while trying to import or activate a registered source, that is expected. Those routes are for platform admins.
|
|
99
|
+
|
|
87
100
|
## Template Deploy Cycle
|
|
88
101
|
|
|
89
102
|
When someone says "template deploy cycle" in this repo, they mean this exact sequence:
|
|
@@ -97,6 +110,15 @@ For `template_0000`, that usually means:
|
|
|
97
110
|
|
|
98
111
|
```bash
|
|
99
112
|
git -C templates/vidfarm_template_0000 push origin production
|
|
113
|
+
# third-party developer registration step
|
|
114
|
+
node dist/src/cli.js register-source-hosted \
|
|
115
|
+
--env-file .env.production \
|
|
116
|
+
--template-id 4c7a7e1a-7f35-4f30-9f86-9c8a63c7f2db \
|
|
117
|
+
--slug-id template_0000 \
|
|
118
|
+
--repo-url https://github.com/your-org/your-template-repo \
|
|
119
|
+
--branch production \
|
|
120
|
+
--template-module-path templates/vidfarm_template_0000/src/template.ts
|
|
121
|
+
|
|
100
122
|
# then run the platform operator command
|
|
101
123
|
node dist/src/cli.js deploy-template-cycle \
|
|
102
124
|
--env-file .env.production \
|
|
@@ -110,7 +132,10 @@ node dist/src/cli.js deploy-template-cycle \
|
|
|
110
132
|
|
|
111
133
|
The important distinction is:
|
|
112
134
|
|
|
113
|
-
-
|
|
135
|
+
- third-party developers only ship template repo changes plus registration metadata
|
|
136
|
+
- platform admins import and activate those template repo changes
|
|
137
|
+
- new source registrations land in the hosted review queue before approval
|
|
138
|
+
- newly imported releases can be approved while an older active release stays live
|
|
114
139
|
- platform code changes ship through the root prod deploy script
|
|
115
140
|
- multiple templates can live in one git repo as long as each template has its own folder and its `template_module_path` points at that folder's `src/template.ts`
|
|
116
141
|
|
|
@@ -126,7 +151,9 @@ If you need to discover the authenticated customer ID first, resolve it from `GE
|
|
|
126
151
|
|
|
127
152
|
For hosted template source registration, treat `template_module_path` as repo-relative and point it at the template's TypeScript entrypoint, for example `templates/vidfarm_template_hooks/src/template.ts`. Each template folder must start with `vidfarm_template_`. The import flow builds the repo and loads the compiled `src/template.js` sibling at activation time.
|
|
128
153
|
|
|
129
|
-
For
|
|
154
|
+
For hosted registration, use `register-source-hosted`. It talks to the Vidfarm REST API directly, so the source record lands in the production database that backs the hosted review queue. `register-source-prod` remains as a compatibility alias for internal/operator workflows.
|
|
155
|
+
|
|
156
|
+
For private GitHub repos, set `VIDFARM_GITHUB_TOKEN` or `GITHUB_TOKEN` in the hosted runtime environment before using the REST import and approval flow. The production API now performs the clone/import itself, so the token must be available to the server process rather than injected from the operator shell.
|
|
130
157
|
|
|
131
158
|
For local CLI usage, the `session` command gives you a local seeded session so you can test without manually wiring auth state.
|
|
132
159
|
|
package/SKILL.developer.md
CHANGED
|
@@ -34,13 +34,15 @@ The third-party developer responsibility is:
|
|
|
34
34
|
|
|
35
35
|
- build and validate the template locally
|
|
36
36
|
- commit and push the template folder to GitHub
|
|
37
|
-
- register or update the hosted template source metadata so Vidfarm knows which repo, branch, and template folder to import from
|
|
37
|
+
- register or update the hosted template source metadata with `register-source-hosted` so Vidfarm knows which repo, branch, and template folder to import from
|
|
38
|
+
- stop there and hand off to the platform team
|
|
38
39
|
|
|
39
40
|
The third-party developer should not:
|
|
40
41
|
|
|
41
42
|
- run `import-source-prod`
|
|
42
43
|
- use AWS CloudFormation, SSM, EC2, or shared prod credentials
|
|
43
44
|
- treat platform import/activation as part of template authoring
|
|
45
|
+
- assume "make this template public/live" is their duty
|
|
44
46
|
|
|
45
47
|
`import-source-prod` and `deploy-template-cycle` are platform-operator commands. They are for internal admin flow, not for external template authors.
|
|
46
48
|
|
|
@@ -113,6 +115,7 @@ Notes:
|
|
|
113
115
|
- at least one runtime AI provider key is usually enough
|
|
114
116
|
- `GEMINI_API_KEY` is the key used by the built-in DNA analysis commands
|
|
115
117
|
- `VIDFARM_API_KEY` is needed when calling a hosted Vidfarm API directly, including when minting presigned preview-media uploads through `vidfarm-devcli presign-preview-media`
|
|
118
|
+
- `VIDFARM_API_KEY` is also what `vidfarm-devcli register-source-hosted` uses for hosted source registration
|
|
116
119
|
- local `vidfarm-devcli session` already gives you a seeded local API key for the local runtime
|
|
117
120
|
- when calling the hosted API directly, include both `vidfarm-user-id` and `vidfarm-api-key` headers
|
|
118
121
|
- for local `vidfarm-devcli` sessions, you usually do not need to supply `VIDFARM_USER_ID` manually
|
|
@@ -131,7 +134,7 @@ For a third-party developer, the hosted publish handoff is:
|
|
|
131
134
|
|
|
132
135
|
1. push the repo changes that contain the target template folder
|
|
133
136
|
2. register the source location if it is new, or update the existing registration if the repo, branch, or template path changed
|
|
134
|
-
3. tell the platform operator which branch head or commit SHA should be imported
|
|
137
|
+
3. stop there, then tell the platform operator which branch head or commit SHA should be imported if they ask
|
|
135
138
|
|
|
136
139
|
Registration data should be treated like this:
|
|
137
140
|
|
|
@@ -142,6 +145,10 @@ Registration data should be treated like this:
|
|
|
142
145
|
|
|
143
146
|
Do not tell third-party developers to run AWS-backed operator commands just to publish template code. If the task is only "make my updated template available to Vidfarm", the correct answer is GitHub push plus source registration metadata.
|
|
144
147
|
|
|
148
|
+
If a developer can register a source but receives `403 Admin access required` when trying to import or activate it, that is the expected boundary, not a blocker they are supposed to solve. It means the handoff point was reached correctly.
|
|
149
|
+
|
|
150
|
+
Hosted registration should use `register-source-hosted`, not a local DB-backed command. That command calls the Vidfarm REST API so the source appears in the production review queue with `pending_review` status.
|
|
151
|
+
|
|
145
152
|
## Template Deploy Cycle
|
|
146
153
|
|
|
147
154
|
If the user explicitly says "template deploy cycle", interpret that as the internal platform-operator sequence:
|
package/dist/src/app.js
CHANGED
|
@@ -67,6 +67,8 @@ const developerPreviewPresignSchema = z.object({
|
|
|
67
67
|
content_type: z.string().trim().min(1).max(255).optional(),
|
|
68
68
|
directory: z.string().trim().max(500).optional()
|
|
69
69
|
});
|
|
70
|
+
const templateSourceStatusSchema = z.enum(["pending_review", "approved", "rejected", "disabled"]);
|
|
71
|
+
const templateReleaseStatusSchema = z.enum(["imported", "pending_approval", "approved", "rejected", "failed", "active"]);
|
|
70
72
|
const listJobsQuerySchema = z.object({
|
|
71
73
|
tracer: z.string().min(1).optional(),
|
|
72
74
|
start_time: z.string().min(1).optional(),
|
|
@@ -288,7 +290,7 @@ function getReleaseTimestamp(release) {
|
|
|
288
290
|
function getApprovedHomepageTemplates(c) {
|
|
289
291
|
const approvedReleaseByTemplateId = new Map();
|
|
290
292
|
for (const release of database.listTemplateReleases()) {
|
|
291
|
-
if (release.status !== "active" && release.status !== "
|
|
293
|
+
if (release.status !== "active" && release.status !== "approved") {
|
|
292
294
|
continue;
|
|
293
295
|
}
|
|
294
296
|
const current = approvedReleaseByTemplateId.get(release.templateId);
|
|
@@ -645,6 +647,12 @@ function requireDeveloper(c) {
|
|
|
645
647
|
}
|
|
646
648
|
return customer;
|
|
647
649
|
}
|
|
650
|
+
function templatePublishingAdminError() {
|
|
651
|
+
return {
|
|
652
|
+
error: "Admin access required.",
|
|
653
|
+
detail: "Reviewing, importing, approving, or activating a registered template source is platform-managed. Third-party developers only push to GitHub and register/update the source metadata."
|
|
654
|
+
};
|
|
655
|
+
}
|
|
648
656
|
function requireSuperagency(c) {
|
|
649
657
|
const provided = c.req.header("x-superagency-key");
|
|
650
658
|
if (!config.SUPERAGENCY_KEY || provided !== config.SUPERAGENCY_KEY) {
|
|
@@ -1046,7 +1054,9 @@ app.get(`${TEMPLATES_PREFIX}/sources`, (c) => {
|
|
|
1046
1054
|
catch (error) {
|
|
1047
1055
|
return c.json({ error: error instanceof Error ? error.message : "Forbidden" }, 403);
|
|
1048
1056
|
}
|
|
1049
|
-
|
|
1057
|
+
const status = c.req.query("status");
|
|
1058
|
+
const parsedStatus = status ? templateSourceStatusSchema.parse(status) : undefined;
|
|
1059
|
+
return c.json({ sources: templateSources.listSources(parsedStatus) });
|
|
1050
1060
|
});
|
|
1051
1061
|
app.post(`${TEMPLATES_PREFIX}/sources`, async (c) => {
|
|
1052
1062
|
try {
|
|
@@ -1079,7 +1089,13 @@ app.post(`${TEMPLATES_PREFIX}/sources`, async (c) => {
|
|
|
1079
1089
|
return c.json({
|
|
1080
1090
|
source: registration.source,
|
|
1081
1091
|
registration_action: registration.action,
|
|
1082
|
-
sync: registration.sync
|
|
1092
|
+
sync: registration.sync,
|
|
1093
|
+
developer_handoff: {
|
|
1094
|
+
completed: true,
|
|
1095
|
+
responsibility_boundary: "third_party_developer_done",
|
|
1096
|
+
message: "Template source registration is complete. Your responsibility stops at GitHub push plus source registration metadata. Importing, activating, and making the template live/public are platform-admin steps.",
|
|
1097
|
+
next_step_for_platform: "A Vidfarm admin can now import this source from the registered branch head or a chosen commit SHA."
|
|
1098
|
+
}
|
|
1083
1099
|
}, registration.action === "created" ? 201 : 200);
|
|
1084
1100
|
}
|
|
1085
1101
|
catch (error) {
|
|
@@ -1103,15 +1119,52 @@ app.get(`${TEMPLATES_PREFIX}/releases`, (c) => {
|
|
|
1103
1119
|
catch (error) {
|
|
1104
1120
|
return c.json({ error: error instanceof Error ? error.message : "Forbidden" }, 403);
|
|
1105
1121
|
}
|
|
1106
|
-
|
|
1122
|
+
const status = c.req.query("status");
|
|
1123
|
+
const parsedStatus = status ? templateReleaseStatusSchema.parse(status) : undefined;
|
|
1124
|
+
return c.json({
|
|
1125
|
+
releases: templateSources.listReleases({
|
|
1126
|
+
templateId: c.req.query("template_id") || undefined,
|
|
1127
|
+
status: parsedStatus
|
|
1128
|
+
})
|
|
1129
|
+
});
|
|
1107
1130
|
});
|
|
1108
|
-
app.
|
|
1131
|
+
app.get(`${TEMPLATES_PREFIX}/review-queue`, (c) => {
|
|
1109
1132
|
try {
|
|
1110
1133
|
requireAdmin(c);
|
|
1111
1134
|
}
|
|
1112
1135
|
catch (error) {
|
|
1113
1136
|
return c.json({ error: error instanceof Error ? error.message : "Forbidden" }, 403);
|
|
1114
1137
|
}
|
|
1138
|
+
return c.json({
|
|
1139
|
+
pending_sources: templateSources.listSources("pending_review"),
|
|
1140
|
+
pending_releases: templateSources.listReleases({ status: "pending_approval" })
|
|
1141
|
+
});
|
|
1142
|
+
});
|
|
1143
|
+
app.post(`${TEMPLATES_PREFIX}/sources/:sourceId/approve`, (c) => {
|
|
1144
|
+
try {
|
|
1145
|
+
requireAdmin(c);
|
|
1146
|
+
}
|
|
1147
|
+
catch (error) {
|
|
1148
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1149
|
+
}
|
|
1150
|
+
return c.json({ source: templateSources.approveSource({ sourceId: c.req.param("sourceId") }) });
|
|
1151
|
+
});
|
|
1152
|
+
app.post(`${TEMPLATES_PREFIX}/sources/:sourceId/reject`, (c) => {
|
|
1153
|
+
try {
|
|
1154
|
+
requireAdmin(c);
|
|
1155
|
+
}
|
|
1156
|
+
catch (error) {
|
|
1157
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1158
|
+
}
|
|
1159
|
+
return c.json({ source: templateSources.rejectSource({ sourceId: c.req.param("sourceId") }) });
|
|
1160
|
+
});
|
|
1161
|
+
app.post(`${TEMPLATES_PREFIX}/sources/:sourceId/import`, async (c) => {
|
|
1162
|
+
try {
|
|
1163
|
+
requireAdmin(c);
|
|
1164
|
+
}
|
|
1165
|
+
catch (error) {
|
|
1166
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1167
|
+
}
|
|
1115
1168
|
const body = z.object({
|
|
1116
1169
|
commit_sha: z.string().min(7).optional()
|
|
1117
1170
|
}).parse(await c.req.json().catch(() => ({})));
|
|
@@ -1121,12 +1174,30 @@ app.post(`${TEMPLATES_PREFIX}/sources/:sourceId/import`, async (c) => {
|
|
|
1121
1174
|
});
|
|
1122
1175
|
return c.json({ release }, 201);
|
|
1123
1176
|
});
|
|
1177
|
+
app.post(`${TEMPLATES_PREFIX}/releases/:releaseId/approve`, (c) => {
|
|
1178
|
+
try {
|
|
1179
|
+
requireAdmin(c);
|
|
1180
|
+
}
|
|
1181
|
+
catch (error) {
|
|
1182
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1183
|
+
}
|
|
1184
|
+
return c.json({ release: templateSources.approveRelease({ releaseId: c.req.param("releaseId") }) });
|
|
1185
|
+
});
|
|
1186
|
+
app.post(`${TEMPLATES_PREFIX}/releases/:releaseId/reject`, (c) => {
|
|
1187
|
+
try {
|
|
1188
|
+
requireAdmin(c);
|
|
1189
|
+
}
|
|
1190
|
+
catch (error) {
|
|
1191
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1192
|
+
}
|
|
1193
|
+
return c.json({ release: templateSources.rejectRelease({ releaseId: c.req.param("releaseId") }) });
|
|
1194
|
+
});
|
|
1124
1195
|
app.post(`${TEMPLATES_PREFIX}/releases/:releaseId/activate`, async (c) => {
|
|
1125
1196
|
try {
|
|
1126
1197
|
requireAdmin(c);
|
|
1127
1198
|
}
|
|
1128
1199
|
catch (error) {
|
|
1129
|
-
return c.json(
|
|
1200
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1130
1201
|
}
|
|
1131
1202
|
const { release, template } = await templateSources.activateRelease({
|
|
1132
1203
|
releaseId: c.req.param("releaseId")
|
package/dist/src/cli.js
CHANGED
|
@@ -37,6 +37,14 @@ async function main() {
|
|
|
37
37
|
await runImportSourceProdCommand(process.argv.slice(3));
|
|
38
38
|
return;
|
|
39
39
|
}
|
|
40
|
+
if (command === "register-source-hosted") {
|
|
41
|
+
await runRegisterSourceHostedCommand(process.argv.slice(3));
|
|
42
|
+
return;
|
|
43
|
+
}
|
|
44
|
+
if (command === "register-source-prod") {
|
|
45
|
+
await runRegisterSourceProdCommand(process.argv.slice(3));
|
|
46
|
+
return;
|
|
47
|
+
}
|
|
40
48
|
if (command === "deploy-template-cycle") {
|
|
41
49
|
await runDeployTemplateCycleCommand(process.argv.slice(3));
|
|
42
50
|
return;
|
|
@@ -249,14 +257,17 @@ async function runImportSourceCommand(argv) {
|
|
|
249
257
|
sourceId: registration.source.id,
|
|
250
258
|
commitSha: parsed.values["commit-sha"] ?? null
|
|
251
259
|
});
|
|
260
|
+
let sourceRecord = registration.source;
|
|
252
261
|
let activated = null;
|
|
253
262
|
if (parsed.values.activate) {
|
|
263
|
+
sourceRecord = sources.approveSource({ sourceId: registration.source.id });
|
|
264
|
+
sources.approveRelease({ releaseId: release.id });
|
|
254
265
|
const result = await sources.activateRelease({ releaseId: release.id });
|
|
255
266
|
templateRegistry.registerRuntimeTemplate(result.template);
|
|
256
267
|
activated = result.release;
|
|
257
268
|
}
|
|
258
269
|
console.log(JSON.stringify({
|
|
259
|
-
source:
|
|
270
|
+
source: sourceRecord,
|
|
260
271
|
registration_action: registration.action,
|
|
261
272
|
sync: registration.sync,
|
|
262
273
|
release,
|
|
@@ -360,17 +371,47 @@ async function runGenerateTemplateCommand(argv) {
|
|
|
360
371
|
dna_analysis_runs: dnaRuns
|
|
361
372
|
}, null, 2));
|
|
362
373
|
}
|
|
374
|
+
async function runRegisterSourceProdCommand(argv) {
|
|
375
|
+
const input = parseProdTemplateCommandArgs(argv, {
|
|
376
|
+
envFile: ".env.production",
|
|
377
|
+
baseUrl: "https://vidfarm.cloud.zoomgtm.com",
|
|
378
|
+
activate: false
|
|
379
|
+
});
|
|
380
|
+
const registration = await registerSourceViaApi(input);
|
|
381
|
+
console.log(JSON.stringify({
|
|
382
|
+
mode: "prod-register-source",
|
|
383
|
+
base_url: input.baseUrl,
|
|
384
|
+
source: registration.source,
|
|
385
|
+
registration_action: registration.registration_action,
|
|
386
|
+
sync: registration.sync ?? null
|
|
387
|
+
}, null, 2));
|
|
388
|
+
}
|
|
389
|
+
async function runRegisterSourceHostedCommand(argv) {
|
|
390
|
+
const input = parseProdTemplateCommandArgs(argv, {
|
|
391
|
+
envFile: ".env",
|
|
392
|
+
baseUrl: process.env.VIDFARM_BASE_URL ?? "https://vidfarm.cloud.zoomgtm.com",
|
|
393
|
+
activate: false
|
|
394
|
+
});
|
|
395
|
+
const registration = await registerSourceViaApi(input);
|
|
396
|
+
console.log(JSON.stringify({
|
|
397
|
+
mode: "hosted-register-source",
|
|
398
|
+
base_url: input.baseUrl,
|
|
399
|
+
source: registration.source,
|
|
400
|
+
registration_action: registration.registration_action,
|
|
401
|
+
sync: registration.sync ?? null,
|
|
402
|
+
next_step: "Platform admin reviews the source, imports a release, approves it, and activates it."
|
|
403
|
+
}, null, 2));
|
|
404
|
+
}
|
|
363
405
|
async function runImportSourceProdCommand(argv) {
|
|
364
406
|
const input = parseProdTemplateCommandArgs(argv, {
|
|
365
407
|
envFile: ".env.production",
|
|
366
|
-
|
|
408
|
+
baseUrl: "https://vidfarm.cloud.zoomgtm.com",
|
|
367
409
|
activate: true
|
|
368
410
|
});
|
|
369
411
|
const result = await importSourceIntoProd(input);
|
|
370
412
|
console.log(JSON.stringify({
|
|
371
413
|
mode: "prod-import-source",
|
|
372
|
-
|
|
373
|
-
instance_id: result.instanceId,
|
|
414
|
+
base_url: input.baseUrl,
|
|
374
415
|
source: result.payload.source,
|
|
375
416
|
release: result.payload.release,
|
|
376
417
|
activated_release: result.payload.activated_release ?? null
|
|
@@ -379,9 +420,13 @@ async function runImportSourceProdCommand(argv) {
|
|
|
379
420
|
async function runDeployTemplateCycleCommand(argv) {
|
|
380
421
|
const input = parseProdTemplateCommandArgs(argv, {
|
|
381
422
|
envFile: ".env.production",
|
|
423
|
+
baseUrl: "https://vidfarm.cloud.zoomgtm.com",
|
|
382
424
|
stackName: "VidfarmProdStack",
|
|
383
425
|
activate: true
|
|
384
426
|
});
|
|
427
|
+
if (!input.stackName) {
|
|
428
|
+
throw new Error("deploy-template-cycle requires --stack-name.");
|
|
429
|
+
}
|
|
385
430
|
const importResult = await importSourceIntoProd(input);
|
|
386
431
|
const currentImage = await getProdCurrentImage({
|
|
387
432
|
stackName: input.stackName,
|
|
@@ -399,7 +444,6 @@ async function runDeployTemplateCycleCommand(argv) {
|
|
|
399
444
|
console.log(JSON.stringify({
|
|
400
445
|
mode: "deploy-template-cycle",
|
|
401
446
|
stack_name: input.stackName,
|
|
402
|
-
instance_id: importResult.instanceId,
|
|
403
447
|
release: importResult.payload.activated_release ?? importResult.payload.release,
|
|
404
448
|
restart_image: currentImage.image
|
|
405
449
|
}, null, 2));
|
|
@@ -680,6 +724,8 @@ function parseProdTemplateCommandArgs(argv, defaults) {
|
|
|
680
724
|
"build-command": { type: "string", default: "npm run build" },
|
|
681
725
|
"commit-sha": { type: "string" },
|
|
682
726
|
"env-file": { type: "string", default: defaults.envFile },
|
|
727
|
+
"base-url": { type: "string", default: defaults.baseUrl },
|
|
728
|
+
"api-key": { type: "string" },
|
|
683
729
|
"stack-name": { type: "string", default: defaults.stackName }
|
|
684
730
|
}
|
|
685
731
|
});
|
|
@@ -691,6 +737,11 @@ function parseProdTemplateCommandArgs(argv, defaults) {
|
|
|
691
737
|
throw new Error("Command requires --template-id, --slug-id, --repo-url, and --template-module-path.");
|
|
692
738
|
}
|
|
693
739
|
deriveTemplateRootDirFromModulePath(templateModulePath);
|
|
740
|
+
loadEnvFile(parsed.values["env-file"]);
|
|
741
|
+
const apiKey = parsed.values["api-key"] ?? process.env.VIDFARM_API_KEY;
|
|
742
|
+
if (!apiKey) {
|
|
743
|
+
throw new Error("Missing Vidfarm API key. Pass --api-key or set VIDFARM_API_KEY in the selected env file.");
|
|
744
|
+
}
|
|
694
745
|
return {
|
|
695
746
|
templateId,
|
|
696
747
|
slugId,
|
|
@@ -702,56 +753,62 @@ function parseProdTemplateCommandArgs(argv, defaults) {
|
|
|
702
753
|
buildCommand: parsed.values["build-command"],
|
|
703
754
|
commitSha: parsed.values["commit-sha"] ?? undefined,
|
|
704
755
|
envFile: parsed.values["env-file"],
|
|
705
|
-
|
|
756
|
+
baseUrl: normalizeBaseUrl(parsed.values["base-url"]),
|
|
757
|
+
apiKey,
|
|
758
|
+
stackName: parsed.values["stack-name"] || undefined,
|
|
706
759
|
activate: defaults.activate
|
|
707
760
|
};
|
|
708
761
|
}
|
|
709
762
|
async function importSourceIntoProd(input) {
|
|
710
|
-
|
|
711
|
-
const
|
|
712
|
-
const
|
|
713
|
-
|
|
714
|
-
|
|
715
|
-
|
|
716
|
-
|
|
717
|
-
const stdout = await runSsmScript({
|
|
718
|
-
instanceId,
|
|
719
|
-
comment: `Vidfarm import ${input.slugId}`,
|
|
720
|
-
script: hostScript
|
|
763
|
+
const registration = await registerSourceViaApi(input);
|
|
764
|
+
const source = await postProdTemplateApi(input, `/api/v1/templates/sources/${registration.source.id}/approve`, { method: "POST" });
|
|
765
|
+
const releaseResponse = await postProdTemplateApi(input, `/api/v1/templates/sources/${registration.source.id}/import`, {
|
|
766
|
+
method: "POST",
|
|
767
|
+
body: JSON.stringify({
|
|
768
|
+
commit_sha: input.commitSha ?? undefined
|
|
769
|
+
})
|
|
721
770
|
});
|
|
771
|
+
const approvedRelease = await postProdTemplateApi(input, `/api/v1/templates/releases/${releaseResponse.release.id}/approve`, { method: "POST" });
|
|
772
|
+
const activatedRelease = input.activate
|
|
773
|
+
? await postProdTemplateApi(input, `/api/v1/templates/releases/${releaseResponse.release.id}/activate`, { method: "POST" })
|
|
774
|
+
: null;
|
|
722
775
|
return {
|
|
723
|
-
|
|
724
|
-
|
|
776
|
+
payload: {
|
|
777
|
+
source: source.source,
|
|
778
|
+
release: approvedRelease.release,
|
|
779
|
+
activated_release: activatedRelease?.release ?? null
|
|
780
|
+
}
|
|
725
781
|
};
|
|
726
782
|
}
|
|
727
|
-
function
|
|
728
|
-
|
|
729
|
-
"
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
|
|
737
|
-
|
|
738
|
-
|
|
739
|
-
|
|
740
|
-
|
|
741
|
-
|
|
742
|
-
|
|
743
|
-
}
|
|
744
|
-
|
|
745
|
-
|
|
746
|
-
|
|
747
|
-
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
|
|
754
|
-
|
|
783
|
+
async function registerSourceViaApi(input) {
|
|
784
|
+
return postProdTemplateApi(input, "/api/v1/templates/sources", {
|
|
785
|
+
method: "POST",
|
|
786
|
+
body: JSON.stringify({
|
|
787
|
+
template_id: input.templateId,
|
|
788
|
+
slug_id: input.slugId,
|
|
789
|
+
repo_url: input.repoUrl,
|
|
790
|
+
branch: input.branch,
|
|
791
|
+
template_module_path: input.templateModulePath,
|
|
792
|
+
skill_path: input.skillPath,
|
|
793
|
+
install_command: input.installCommand,
|
|
794
|
+
build_command: input.buildCommand
|
|
795
|
+
})
|
|
796
|
+
});
|
|
797
|
+
}
|
|
798
|
+
async function postProdTemplateApi(input, pathname, init) {
|
|
799
|
+
const response = await fetch(`${input.baseUrl}${pathname}`, {
|
|
800
|
+
...init,
|
|
801
|
+
headers: {
|
|
802
|
+
"content-type": "application/json",
|
|
803
|
+
"vidfarm-api-key": input.apiKey,
|
|
804
|
+
...(init.headers ?? {})
|
|
805
|
+
}
|
|
806
|
+
});
|
|
807
|
+
const payload = await response.json().catch(() => ({}));
|
|
808
|
+
if (!response.ok) {
|
|
809
|
+
throw new Error(typeof payload.error === "string" ? payload.error : `Request failed with ${response.status}.`);
|
|
810
|
+
}
|
|
811
|
+
return payload;
|
|
755
812
|
}
|
|
756
813
|
function loadEnvFile(envFile) {
|
|
757
814
|
const resolved = path.resolve(process.cwd(), envFile);
|
|
@@ -823,12 +880,6 @@ function inferUploadContentType(fileName) {
|
|
|
823
880
|
return "application/octet-stream";
|
|
824
881
|
}
|
|
825
882
|
}
|
|
826
|
-
function resolveGithubToken() {
|
|
827
|
-
return process.env.VIDFARM_GITHUB_TOKEN
|
|
828
|
-
?? process.env.GITHUB_TOKEN
|
|
829
|
-
?? process.env.GH_TOKEN
|
|
830
|
-
?? "";
|
|
831
|
-
}
|
|
832
883
|
async function resolveStackOutput(stackName, outputKey) {
|
|
833
884
|
const { stdout } = await runLocalCommand("aws", [
|
|
834
885
|
"cloudformation",
|
package/dist/src/config.js
CHANGED
|
@@ -44,6 +44,8 @@ const schema = z.object({
|
|
|
44
44
|
MOCK_PROVIDER_RESPONSES: z.string().optional(),
|
|
45
45
|
VIDFARM_ADMIN_EMAILS: z.string().default(""),
|
|
46
46
|
VIDFARM_DEVELOPER_EMAILS: z.string().default(""),
|
|
47
|
+
VIDFARM_GITHUB_TOKEN: z.string().optional(),
|
|
48
|
+
GITHUB_TOKEN: z.string().optional(),
|
|
47
49
|
TEMPLATE_SOURCE_ROOT: z.string().default("./data/template-sources")
|
|
48
50
|
});
|
|
49
51
|
const parsed = schema.parse(process.env);
|
package/dist/src/db.js
CHANGED
|
@@ -235,6 +235,16 @@ db.exec(`
|
|
|
235
235
|
where slug_id is null or trim(slug_id) = '';
|
|
236
236
|
`);
|
|
237
237
|
db.exec(`create unique index if not exists idx_template_sources_slug_id on template_sources(slug_id);`);
|
|
238
|
+
db.exec(`
|
|
239
|
+
update template_sources
|
|
240
|
+
set status = 'approved'
|
|
241
|
+
where status = 'active';
|
|
242
|
+
`);
|
|
243
|
+
db.exec(`
|
|
244
|
+
update template_releases
|
|
245
|
+
set status = 'pending_approval'
|
|
246
|
+
where status = 'certified';
|
|
247
|
+
`);
|
|
238
248
|
db.exec(`drop index if exists idx_job_events_job_time;`);
|
|
239
249
|
db.exec(`drop table if exists job_events;`);
|
|
240
250
|
function mapJob(row) {
|
|
@@ -895,8 +905,11 @@ export const database = {
|
|
|
895
905
|
`).get(repoUrl, branch, templateModulePath);
|
|
896
906
|
return row ? mapTemplateSource(row) : null;
|
|
897
907
|
},
|
|
898
|
-
listTemplateSources() {
|
|
899
|
-
|
|
908
|
+
listTemplateSources(status) {
|
|
909
|
+
const rows = status
|
|
910
|
+
? db.prepare(`select * from template_sources where status = ? order by created_at asc`).all(status)
|
|
911
|
+
: db.prepare(`select * from template_sources order by created_at asc`).all();
|
|
912
|
+
return rows.map(mapTemplateSource);
|
|
900
913
|
},
|
|
901
914
|
createTemplateRelease(record) {
|
|
902
915
|
const timestamp = nowIso();
|
|
@@ -951,10 +964,16 @@ export const database = {
|
|
|
951
964
|
`).get(sourceId);
|
|
952
965
|
return row ? mapTemplateRelease(row) : null;
|
|
953
966
|
},
|
|
954
|
-
listTemplateReleases(
|
|
955
|
-
const
|
|
956
|
-
|
|
957
|
-
|
|
967
|
+
listTemplateReleases(input) {
|
|
968
|
+
const templateId = input?.templateId;
|
|
969
|
+
const status = input?.status;
|
|
970
|
+
const rows = templateId && status
|
|
971
|
+
? db.prepare(`select * from template_releases where template_id = ? and status = ? order by created_at desc`).all(templateId, status)
|
|
972
|
+
: templateId
|
|
973
|
+
? db.prepare(`select * from template_releases where template_id = ? order by created_at desc`).all(templateId)
|
|
974
|
+
: status
|
|
975
|
+
? db.prepare(`select * from template_releases where status = ? order by created_at desc`).all(status)
|
|
976
|
+
: db.prepare(`select * from template_releases order by created_at desc`).all();
|
|
958
977
|
return rows.map(mapTemplateRelease);
|
|
959
978
|
},
|
|
960
979
|
getActiveTemplateReleases() {
|
|
@@ -979,12 +998,16 @@ export const database = {
|
|
|
979
998
|
updated_at: nowIso()
|
|
980
999
|
});
|
|
981
1000
|
},
|
|
982
|
-
clearActiveTemplateReleases(templateId) {
|
|
1001
|
+
clearActiveTemplateReleases(templateId, fallbackStatus = "approved") {
|
|
983
1002
|
db.prepare(`
|
|
984
1003
|
update template_releases
|
|
985
|
-
set status = case when status = 'active' then
|
|
986
|
-
updated_at =
|
|
987
|
-
where template_id =
|
|
988
|
-
`).run(
|
|
1004
|
+
set status = case when status = 'active' then @fallback_status else status end,
|
|
1005
|
+
updated_at = @updated_at
|
|
1006
|
+
where template_id = @template_id
|
|
1007
|
+
`).run({
|
|
1008
|
+
fallback_status: fallbackStatus,
|
|
1009
|
+
updated_at: nowIso(),
|
|
1010
|
+
template_id: templateId
|
|
1011
|
+
});
|
|
989
1012
|
}
|
|
990
1013
|
};
|
|
@@ -22,11 +22,11 @@ export class TemplateSourceAlreadyRegisteredError extends Error {
|
|
|
22
22
|
}
|
|
23
23
|
export class TemplateSourceService {
|
|
24
24
|
certification = new TemplateCertificationService();
|
|
25
|
-
listSources() {
|
|
26
|
-
return database.listTemplateSources();
|
|
25
|
+
listSources(status) {
|
|
26
|
+
return database.listTemplateSources(status);
|
|
27
27
|
}
|
|
28
|
-
listReleases(
|
|
29
|
-
return database.listTemplateReleases(
|
|
28
|
+
listReleases(input) {
|
|
29
|
+
return database.listTemplateReleases(input);
|
|
30
30
|
}
|
|
31
31
|
async registerSource(input) {
|
|
32
32
|
deriveTemplateRootDirFromModulePath(input.templateModulePath);
|
|
@@ -61,12 +61,13 @@ export class TemplateSourceService {
|
|
|
61
61
|
const buildCommand = input.buildCommand ?? "npm run build";
|
|
62
62
|
const skillPath = input.skillPath ?? defaultSkillPathForTemplateModule(input.templateModulePath);
|
|
63
63
|
if (existingSource) {
|
|
64
|
-
const
|
|
64
|
+
const metadataChanged = existingSource.repoUrl !== input.repoUrl ||
|
|
65
65
|
existingSource.branch !== branch ||
|
|
66
66
|
existingSource.templateModulePath !== input.templateModulePath ||
|
|
67
67
|
existingSource.skillPath !== skillPath ||
|
|
68
68
|
existingSource.installCommand !== installCommand ||
|
|
69
|
-
existingSource.buildCommand !== buildCommand
|
|
69
|
+
existingSource.buildCommand !== buildCommand;
|
|
70
|
+
const nextSource = metadataChanged
|
|
70
71
|
? database.updateTemplateSource({
|
|
71
72
|
id: existingSource.id,
|
|
72
73
|
repoUrl: input.repoUrl,
|
|
@@ -74,7 +75,8 @@ export class TemplateSourceService {
|
|
|
74
75
|
templateModulePath: input.templateModulePath,
|
|
75
76
|
skillPath,
|
|
76
77
|
installCommand,
|
|
77
|
-
buildCommand
|
|
78
|
+
buildCommand,
|
|
79
|
+
status: existingSource.status === "disabled" ? "disabled" : "pending_review"
|
|
78
80
|
})
|
|
79
81
|
: existingSource;
|
|
80
82
|
return {
|
|
@@ -94,7 +96,7 @@ export class TemplateSourceService {
|
|
|
94
96
|
skillPath,
|
|
95
97
|
installCommand,
|
|
96
98
|
buildCommand,
|
|
97
|
-
status: "
|
|
99
|
+
status: "pending_review"
|
|
98
100
|
});
|
|
99
101
|
return {
|
|
100
102
|
source,
|
|
@@ -128,9 +130,10 @@ export class TemplateSourceService {
|
|
|
128
130
|
const commitSha = input.commitSha ?? await this.resolveBranchHead(source.repoUrl, source.branch);
|
|
129
131
|
const checkoutPath = path.join(config.TEMPLATE_SOURCE_ROOT, source.templateId, commitSha);
|
|
130
132
|
const skillPath = path.join(checkoutPath, source.skillPath);
|
|
133
|
+
const authRepoUrl = this.resolveGitRemoteUrl(source.repoUrl);
|
|
131
134
|
if (!existsSync(checkoutPath)) {
|
|
132
135
|
mkdirSync(path.dirname(checkoutPath), { recursive: true });
|
|
133
|
-
await this.runShell(["git", "clone", "--branch", source.branch,
|
|
136
|
+
await this.runShell(["git", "clone", "--branch", source.branch, authRepoUrl, checkoutPath], process.cwd());
|
|
134
137
|
await this.runShell(["git", "checkout", commitSha], checkoutPath);
|
|
135
138
|
if (source.installCommand.trim()) {
|
|
136
139
|
await this.runCommandString(source.installCommand, checkoutPath);
|
|
@@ -148,7 +151,7 @@ export class TemplateSourceService {
|
|
|
148
151
|
throw new Error(`Imported template slug_id ${template.slugId} does not match source slug_id ${source.slugId}.`);
|
|
149
152
|
}
|
|
150
153
|
const certificationReport = await this.certification.certify({ template, skillPath });
|
|
151
|
-
const status = certificationReport.passed ? "
|
|
154
|
+
const status = certificationReport.passed ? "pending_approval" : "failed";
|
|
152
155
|
return database.createTemplateRelease({
|
|
153
156
|
id: createId("trel"),
|
|
154
157
|
sourceId: source.id,
|
|
@@ -163,13 +166,77 @@ export class TemplateSourceService {
|
|
|
163
166
|
activatedAt: null
|
|
164
167
|
});
|
|
165
168
|
}
|
|
169
|
+
approveSource(input) {
|
|
170
|
+
const source = database.getTemplateSource(input.sourceId);
|
|
171
|
+
if (!source) {
|
|
172
|
+
throw new Error("Template source not found.");
|
|
173
|
+
}
|
|
174
|
+
return database.updateTemplateSource({
|
|
175
|
+
id: source.id,
|
|
176
|
+
repoUrl: source.repoUrl,
|
|
177
|
+
branch: source.branch,
|
|
178
|
+
templateModulePath: source.templateModulePath,
|
|
179
|
+
skillPath: source.skillPath,
|
|
180
|
+
installCommand: source.installCommand,
|
|
181
|
+
buildCommand: source.buildCommand,
|
|
182
|
+
status: "approved"
|
|
183
|
+
});
|
|
184
|
+
}
|
|
185
|
+
rejectSource(input) {
|
|
186
|
+
const source = database.getTemplateSource(input.sourceId);
|
|
187
|
+
if (!source) {
|
|
188
|
+
throw new Error("Template source not found.");
|
|
189
|
+
}
|
|
190
|
+
return database.updateTemplateSource({
|
|
191
|
+
id: source.id,
|
|
192
|
+
repoUrl: source.repoUrl,
|
|
193
|
+
branch: source.branch,
|
|
194
|
+
templateModulePath: source.templateModulePath,
|
|
195
|
+
skillPath: source.skillPath,
|
|
196
|
+
installCommand: source.installCommand,
|
|
197
|
+
buildCommand: source.buildCommand,
|
|
198
|
+
status: "rejected"
|
|
199
|
+
});
|
|
200
|
+
}
|
|
201
|
+
approveRelease(input) {
|
|
202
|
+
const release = database.getTemplateRelease(input.releaseId);
|
|
203
|
+
if (!release) {
|
|
204
|
+
throw new Error("Template release not found.");
|
|
205
|
+
}
|
|
206
|
+
if (release.status !== "pending_approval" && release.status !== "approved" && release.status !== "active") {
|
|
207
|
+
throw new Error("Only pending-approval releases can be approved.");
|
|
208
|
+
}
|
|
209
|
+
if (release.status === "active") {
|
|
210
|
+
return release;
|
|
211
|
+
}
|
|
212
|
+
database.updateTemplateReleaseStatus({
|
|
213
|
+
id: release.id,
|
|
214
|
+
status: "approved",
|
|
215
|
+
certificationReport: release.certificationReport,
|
|
216
|
+
activatedAt: null
|
|
217
|
+
});
|
|
218
|
+
return database.getTemplateRelease(release.id);
|
|
219
|
+
}
|
|
220
|
+
rejectRelease(input) {
|
|
221
|
+
const release = database.getTemplateRelease(input.releaseId);
|
|
222
|
+
if (!release) {
|
|
223
|
+
throw new Error("Template release not found.");
|
|
224
|
+
}
|
|
225
|
+
database.updateTemplateReleaseStatus({
|
|
226
|
+
id: release.id,
|
|
227
|
+
status: "rejected",
|
|
228
|
+
certificationReport: release.certificationReport,
|
|
229
|
+
activatedAt: null
|
|
230
|
+
});
|
|
231
|
+
return database.getTemplateRelease(release.id);
|
|
232
|
+
}
|
|
166
233
|
async activateRelease(input) {
|
|
167
234
|
const release = database.getTemplateRelease(input.releaseId);
|
|
168
235
|
if (!release) {
|
|
169
236
|
throw new Error("Template release not found.");
|
|
170
237
|
}
|
|
171
|
-
if (release.status !== "
|
|
172
|
-
throw new Error("Only
|
|
238
|
+
if (release.status !== "approved" && release.status !== "active") {
|
|
239
|
+
throw new Error("Only approved releases can be activated.");
|
|
173
240
|
}
|
|
174
241
|
const template = await loadTemplateFromModule(release.modulePath);
|
|
175
242
|
const certificationReport = release.certificationReport ?? await this.certification.certify({
|
|
@@ -226,7 +293,7 @@ export class TemplateSourceService {
|
|
|
226
293
|
}
|
|
227
294
|
}
|
|
228
295
|
async resolveBranchHead(repoUrl, branch) {
|
|
229
|
-
const { stdout } = await execFileAsync("git", ["ls-remote", repoUrl, branch], { cwd: process.cwd() });
|
|
296
|
+
const { stdout } = await execFileAsync("git", ["ls-remote", this.resolveGitRemoteUrl(repoUrl), branch], { cwd: process.cwd() });
|
|
230
297
|
const line = stdout.trim().split("\n").find(Boolean);
|
|
231
298
|
if (!line) {
|
|
232
299
|
throw new Error(`Unable to resolve branch ${branch} for ${repoUrl}`);
|
|
@@ -248,6 +315,16 @@ export class TemplateSourceService {
|
|
|
248
315
|
}
|
|
249
316
|
});
|
|
250
317
|
}
|
|
318
|
+
resolveGitRemoteUrl(repoUrl) {
|
|
319
|
+
const githubToken = config.VIDFARM_GITHUB_TOKEN || config.GITHUB_TOKEN;
|
|
320
|
+
if (!githubToken) {
|
|
321
|
+
return repoUrl;
|
|
322
|
+
}
|
|
323
|
+
if (!/^https:\/\/github\.com\//i.test(repoUrl)) {
|
|
324
|
+
return repoUrl;
|
|
325
|
+
}
|
|
326
|
+
return repoUrl.replace(/^https:\/\/github\.com\//i, `https://x-access-token:${encodeURIComponent(githubToken)}@github.com/`);
|
|
327
|
+
}
|
|
251
328
|
resolveImportableModulePath(checkoutPath, declaredModulePath) {
|
|
252
329
|
const sourceModulePath = path.join(checkoutPath, declaredModulePath);
|
|
253
330
|
const extension = path.extname(sourceModulePath).toLowerCase();
|