@mevdragon/vidfarm-devcli 0.2.5 → 0.2.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/GETTING_STARTED.developers.md +29 -0
- package/README.md +30 -3
- package/SKILL.developer.md +50 -6
- package/dist/src/app.js +92 -8
- package/dist/src/cli.js +108 -55
- package/dist/src/config.js +2 -0
- package/dist/src/db.js +82 -11
- package/dist/src/services/template-sources.js +209 -27
- package/package.json +1 -1
|
@@ -85,3 +85,32 @@ That will populate with a reference folder of what a template code looks like. W
|
|
|
85
85
|
8. Create a new folder beside src called `drafts/` which is where we will put messy files when needed.
|
|
86
86
|
|
|
87
87
|
9. Now find an easy winning format from https://winning-formats-gallery-tiktok.cloud.zoomgtm.com/ and save the image to your drafts/ folder. then tell Codex to look at that image (right click, copy relative path, paste to codex), then tell codex please make a new template based on the image.
|
|
88
|
+
|
|
89
|
+
## Publish Responsibility
|
|
90
|
+
|
|
91
|
+
When your template is ready for hosted Vidfarm, your responsibility is only:
|
|
92
|
+
|
|
93
|
+
1. push your template code to GitHub
|
|
94
|
+
2. register or update the template source metadata so Vidfarm knows the repo URL, branch, and template folder path
|
|
95
|
+
|
|
96
|
+
Your responsibility does not include:
|
|
97
|
+
|
|
98
|
+
1. importing the source into the live platform
|
|
99
|
+
2. activating the release
|
|
100
|
+
3. making the template public/live
|
|
101
|
+
|
|
102
|
+
Those are platform-admin steps. If an API route says `403 Admin access required` for import or activate, that means you reached the correct handoff boundary.
|
|
103
|
+
|
|
104
|
+
When you need to register your template with hosted Vidfarm, use the dev CLI command below from your repo root:
|
|
105
|
+
|
|
106
|
+
```bash
|
|
107
|
+
npx @mevdragon/vidfarm-devcli register-source-hosted \
|
|
108
|
+
--env-file .env \
|
|
109
|
+
--template-id <uuid-v4> \
|
|
110
|
+
--slug-id <template_slug> \
|
|
111
|
+
--repo-url https://github.com/<you>/<repo> \
|
|
112
|
+
--branch main \
|
|
113
|
+
--template-module-path templates/vidfarm_template_<slug>/src/template.ts
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
That command uses your `VIDFARM_API_KEY` and sends the registration to the hosted REST API, so the template appears in Vidfarm's review queue with `pending_review` status.
|
package/README.md
CHANGED
|
@@ -51,7 +51,7 @@ The local CLI runs the same template contract used by the hosted platform, but w
|
|
|
51
51
|
|
|
52
52
|
Use `vidfarm session` to print reusable auth headers and a sample `curl` request for the local REST API.
|
|
53
53
|
|
|
54
|
-
For hosted preview-media uploads, use `vidfarm-devcli presign-preview-media`. It calls the authenticated Vidfarm API with `VIDFARM_API_KEY`, mints a presigned PUT URL under `developer/<user_id>/*`, stores each uploaded file under a UUID path segment while preserving the original filename, uploads the file automatically when `--file` is provided, and returns a public-read media URL for the uploaded object.
|
|
54
|
+
For hosted preview-media uploads, use `vidfarm-devcli presign-preview-media`. It calls the authenticated Vidfarm API with `VIDFARM_API_KEY`, mints a presigned PUT URL under `developer/<user_id>/*`, stores each uploaded file under a UUID path segment while preserving the original filename, uploads the file automatically when `--file` is provided, and returns a public-read media URL for the uploaded object. Public readability is granted by bucket policy for `developer/*`, not by public write or list access.
|
|
55
55
|
|
|
56
56
|
## What The Hosted Platform Expects From Templates
|
|
57
57
|
|
|
@@ -84,6 +84,19 @@ Third-party developers build and validate templates locally first.
|
|
|
84
84
|
|
|
85
85
|
After handoff, a Vidfarm admin reviews the template before it is made available on the hosted platform. Template authors should assume there is an approval step between local development and production availability.
|
|
86
86
|
|
|
87
|
+
The developer handoff ends at:
|
|
88
|
+
|
|
89
|
+
- `git push` of the template code
|
|
90
|
+
- template source registration or registration update
|
|
91
|
+
|
|
92
|
+
The developer handoff does not include:
|
|
93
|
+
|
|
94
|
+
- importing the source into the live platform
|
|
95
|
+
- activating a release
|
|
96
|
+
- making the template public/live
|
|
97
|
+
|
|
98
|
+
If a developer receives `403 Admin access required` while trying to import or activate a registered source, that is expected. Those routes are for platform admins.
|
|
99
|
+
|
|
87
100
|
## Template Deploy Cycle
|
|
88
101
|
|
|
89
102
|
When someone says "template deploy cycle" in this repo, they mean this exact sequence:
|
|
@@ -97,6 +110,15 @@ For `template_0000`, that usually means:
|
|
|
97
110
|
|
|
98
111
|
```bash
|
|
99
112
|
git -C templates/vidfarm_template_0000 push origin production
|
|
113
|
+
# third-party developer registration step
|
|
114
|
+
node dist/src/cli.js register-source-hosted \
|
|
115
|
+
--env-file .env.production \
|
|
116
|
+
--template-id 4c7a7e1a-7f35-4f30-9f86-9c8a63c7f2db \
|
|
117
|
+
--slug-id template_0000 \
|
|
118
|
+
--repo-url https://github.com/your-org/your-template-repo \
|
|
119
|
+
--branch production \
|
|
120
|
+
--template-module-path templates/vidfarm_template_0000/src/template.ts
|
|
121
|
+
|
|
100
122
|
# then run the platform operator command
|
|
101
123
|
node dist/src/cli.js deploy-template-cycle \
|
|
102
124
|
--env-file .env.production \
|
|
@@ -110,7 +132,10 @@ node dist/src/cli.js deploy-template-cycle \
|
|
|
110
132
|
|
|
111
133
|
The important distinction is:
|
|
112
134
|
|
|
113
|
-
-
|
|
135
|
+
- third-party developers only ship template repo changes plus registration metadata
|
|
136
|
+
- platform admins import and activate those template repo changes
|
|
137
|
+
- new source registrations land in the hosted review queue before approval
|
|
138
|
+
- newly imported releases can be approved while an older active release stays live
|
|
114
139
|
- platform code changes ship through the root prod deploy script
|
|
115
140
|
- multiple templates can live in one git repo as long as each template has its own folder and its `template_module_path` points at that folder's `src/template.ts`
|
|
116
141
|
|
|
@@ -126,7 +151,9 @@ If you need to discover the authenticated customer ID first, resolve it from `GE
|
|
|
126
151
|
|
|
127
152
|
For hosted template source registration, treat `template_module_path` as repo-relative and point it at the template's TypeScript entrypoint, for example `templates/vidfarm_template_hooks/src/template.ts`. Each template folder must start with `vidfarm_template_`. The import flow builds the repo and loads the compiled `src/template.js` sibling at activation time.
|
|
128
153
|
|
|
129
|
-
For
|
|
154
|
+
For hosted registration, use `register-source-hosted`. It talks to the Vidfarm REST API directly, so the source record lands in the production database that backs the hosted review queue. `register-source-prod` remains as a compatibility alias for internal/operator workflows.
|
|
155
|
+
|
|
156
|
+
For private GitHub repos, set `VIDFARM_GITHUB_TOKEN` or `GITHUB_TOKEN` in the hosted runtime environment before using the REST import and approval flow. The production API now performs the clone/import itself, so the token must be available to the server process rather than injected from the operator shell.
|
|
130
157
|
|
|
131
158
|
For local CLI usage, the `session` command gives you a local seeded session so you can test without manually wiring auth state.
|
|
132
159
|
|
package/SKILL.developer.md
CHANGED
|
@@ -26,6 +26,26 @@ Do not use this skill for:
|
|
|
26
26
|
- production deploys, release admin, or shared cloud infrastructure
|
|
27
27
|
- asking for AWS, S3, Remotion cloud, or other platform secrets
|
|
28
28
|
|
|
29
|
+
## Third-Party Publish Rule
|
|
30
|
+
|
|
31
|
+
Third-party template developers publish template code by pushing git commits to GitHub. That is the only code-distribution step they own.
|
|
32
|
+
|
|
33
|
+
The third-party developer responsibility is:
|
|
34
|
+
|
|
35
|
+
- build and validate the template locally
|
|
36
|
+
- commit and push the template folder to GitHub
|
|
37
|
+
- register or update the hosted template source metadata with `register-source-hosted` so Vidfarm knows which repo, branch, and template folder to import from
|
|
38
|
+
- stop there and hand off to the platform team
|
|
39
|
+
|
|
40
|
+
The third-party developer should not:
|
|
41
|
+
|
|
42
|
+
- run `import-source-prod`
|
|
43
|
+
- use AWS CloudFormation, SSM, EC2, or shared prod credentials
|
|
44
|
+
- treat platform import/activation as part of template authoring
|
|
45
|
+
- assume "make this template public/live" is their duty
|
|
46
|
+
|
|
47
|
+
`import-source-prod` and `deploy-template-cycle` are platform-operator commands. They are for internal admin flow, not for external template authors.
|
|
48
|
+
|
|
29
49
|
For third-party developers, Remotion is local and storage should be accessed through Vidfarm helpers. Do not require cloud Remotion setup or hand-written S3 integration for normal template work.
|
|
30
50
|
|
|
31
51
|
## Agent Operating Rule
|
|
@@ -95,23 +115,47 @@ Notes:
|
|
|
95
115
|
- at least one runtime AI provider key is usually enough
|
|
96
116
|
- `GEMINI_API_KEY` is the key used by the built-in DNA analysis commands
|
|
97
117
|
- `VIDFARM_API_KEY` is needed when calling a hosted Vidfarm API directly, including when minting presigned preview-media uploads through `vidfarm-devcli presign-preview-media`
|
|
118
|
+
- `VIDFARM_API_KEY` is also what `vidfarm-devcli register-source-hosted` uses for hosted source registration
|
|
98
119
|
- local `vidfarm-devcli session` already gives you a seeded local API key for the local runtime
|
|
99
120
|
- when calling the hosted API directly, include both `vidfarm-user-id` and `vidfarm-api-key` headers
|
|
100
121
|
- for local `vidfarm-devcli` sessions, you usually do not need to supply `VIDFARM_USER_ID` manually
|
|
101
122
|
- template authoring is TypeScript-first; create and edit template source as `src/template.ts`, not `src/template.js`
|
|
102
123
|
- keep exploratory scripts, one-off test runners, scratch validation files, and temporary job-driving code out of `src/`; place them in a repo-local tmp folder that is a sibling to `src` instead, for example `templates/vidfarm_template_0007/tmp/`
|
|
103
124
|
- when registering a hosted template source, set `template_module_path` to the repo-relative TypeScript entrypoint inside the template folder, for example `templates/vidfarm_template_example/src/template.ts`; the platform build/import flow resolves the compiled `src/template.js` sibling after `npm run build`
|
|
125
|
+
- hosted template registration is git metadata only: repo URL, branch, template entrypoint path, and optionally an explicit commit SHA for a specific import
|
|
126
|
+
- if a template moves folders or switches branches, update the registration metadata; if the template code changed but the location stayed the same, just push a new Git commit
|
|
127
|
+
- the platform determines whether an update exists by comparing the registered branch head against the latest imported release commit
|
|
104
128
|
- do not ask for platform secrets like `ENCRYPTION_SECRET`, `API_KEY_SALT`, `WEBHOOK_SECRET`, admin emails, S3 config, or generic AWS credentials
|
|
105
129
|
- do not ask for `REMOTION_AWS_ACCESS_KEY_ID` or `REMOTION_AWS_SECRET_ACCESS_KEY`
|
|
106
130
|
|
|
107
|
-
##
|
|
131
|
+
## Hosted Registration Flow
|
|
108
132
|
|
|
109
|
-
|
|
133
|
+
For a third-party developer, the hosted publish handoff is:
|
|
110
134
|
|
|
111
135
|
1. push the repo changes that contain the target template folder
|
|
112
|
-
2.
|
|
113
|
-
3.
|
|
114
|
-
|
|
136
|
+
2. register the source location if it is new, or update the existing registration if the repo, branch, or template path changed
|
|
137
|
+
3. stop there, then tell the platform operator which branch head or commit SHA should be imported if they ask
|
|
138
|
+
|
|
139
|
+
Registration data should be treated like this:
|
|
140
|
+
|
|
141
|
+
- `repo_url`: the GitHub repo that contains the template code
|
|
142
|
+
- `branch`: the branch Vidfarm should watch for the latest version, usually `main`
|
|
143
|
+
- `template_module_path`: the repo-relative path to the template TypeScript entrypoint, for example `src/vidfarm_template_funnychat/src/template.ts`
|
|
144
|
+
- `commit_sha`: optional and only needed when you want to import a specific commit instead of the current branch head
|
|
145
|
+
|
|
146
|
+
Do not tell third-party developers to run AWS-backed operator commands just to publish template code. If the task is only "make my updated template available to Vidfarm", the correct answer is GitHub push plus source registration metadata.
|
|
147
|
+
|
|
148
|
+
If a developer can register a source but receives `403 Admin access required` when trying to import or activate it, that is the expected boundary, not a blocker they are supposed to solve. It means the handoff point was reached correctly.
|
|
149
|
+
|
|
150
|
+
Hosted registration should use `register-source-hosted`, not a local DB-backed command. That command calls the Vidfarm REST API so the source appears in the production review queue with `pending_review` status.
|
|
151
|
+
|
|
152
|
+
## Template Deploy Cycle
|
|
153
|
+
|
|
154
|
+
If the user explicitly says "template deploy cycle", interpret that as the internal platform-operator sequence:
|
|
155
|
+
|
|
156
|
+
1. import that template commit into the hosted platform as a template source release
|
|
157
|
+
2. approve and activate the new release
|
|
158
|
+
3. run the prod deploy script if platform code changed, or reuse the current prod image for a formal restart if the change was template-only
|
|
115
159
|
|
|
116
160
|
Do not assume "template deploy cycle" means "deploy the entire dirty root repo". If the root worktree has unrelated changes and the release is template-only, prefer reusing the current prod image with:
|
|
117
161
|
|
|
@@ -229,7 +273,7 @@ npx @mevdragon/vidfarm-devcli presign-preview-media \
|
|
|
229
273
|
--directory drafts/preview
|
|
230
274
|
```
|
|
231
275
|
|
|
232
|
-
That command calls the hosted Vidfarm API with the user's `VIDFARM_API_KEY`, mints a presigned PUT URL scoped under `developer/<user_id>/*`, stores each uploaded file under a UUID path segment while preserving the original filename, uploads the file automatically when `--file` is provided, and returns both the `storage_key` and a public-read `preview_media_url`.
|
|
276
|
+
That command calls the hosted Vidfarm API with the user's `VIDFARM_API_KEY`, mints a presigned PUT URL scoped under `developer/<user_id>/*`, stores each uploaded file under a UUID path segment while preserving the original filename, uploads the file automatically when `--file` is provided, and returns both the `storage_key` and a public-read `preview_media_url`. Public readability for `developer/<user_id>/*` comes from Vidfarm bucket policy, not from public write access, so objects may be read directly by URL but should not be treated as listable or writable without authenticated API access. Agents should prefer that path for hosted preview uploads whenever the input is a local media file path.
|
|
233
277
|
|
|
234
278
|
### 2. Start local runtime
|
|
235
279
|
|
package/dist/src/app.js
CHANGED
|
@@ -15,6 +15,7 @@ import { AuthService } from "./services/auth.js";
|
|
|
15
15
|
import { JobsService } from "./services/jobs.js";
|
|
16
16
|
import { StorageService } from "./services/storage.js";
|
|
17
17
|
import { TemplateSourceService } from "./services/template-sources.js";
|
|
18
|
+
import { TemplateSourceAlreadyRegisteredError } from "./services/template-sources.js";
|
|
18
19
|
const auth = new AuthService();
|
|
19
20
|
const jobs = new JobsService();
|
|
20
21
|
const storage = new StorageService();
|
|
@@ -66,6 +67,8 @@ const developerPreviewPresignSchema = z.object({
|
|
|
66
67
|
content_type: z.string().trim().min(1).max(255).optional(),
|
|
67
68
|
directory: z.string().trim().max(500).optional()
|
|
68
69
|
});
|
|
70
|
+
const templateSourceStatusSchema = z.enum(["pending_review", "approved", "rejected", "disabled"]);
|
|
71
|
+
const templateReleaseStatusSchema = z.enum(["imported", "pending_approval", "approved", "rejected", "failed", "active"]);
|
|
69
72
|
const listJobsQuerySchema = z.object({
|
|
70
73
|
tracer: z.string().min(1).optional(),
|
|
71
74
|
start_time: z.string().min(1).optional(),
|
|
@@ -287,7 +290,7 @@ function getReleaseTimestamp(release) {
|
|
|
287
290
|
function getApprovedHomepageTemplates(c) {
|
|
288
291
|
const approvedReleaseByTemplateId = new Map();
|
|
289
292
|
for (const release of database.listTemplateReleases()) {
|
|
290
|
-
if (release.status !== "active" && release.status !== "
|
|
293
|
+
if (release.status !== "active" && release.status !== "approved") {
|
|
291
294
|
continue;
|
|
292
295
|
}
|
|
293
296
|
const current = approvedReleaseByTemplateId.get(release.templateId);
|
|
@@ -644,6 +647,12 @@ function requireDeveloper(c) {
|
|
|
644
647
|
}
|
|
645
648
|
return customer;
|
|
646
649
|
}
|
|
650
|
+
function templatePublishingAdminError() {
|
|
651
|
+
return {
|
|
652
|
+
error: "Admin access required.",
|
|
653
|
+
detail: "Reviewing, importing, approving, or activating a registered template source is platform-managed. Third-party developers only push to GitHub and register/update the source metadata."
|
|
654
|
+
};
|
|
655
|
+
}
|
|
647
656
|
function requireSuperagency(c) {
|
|
648
657
|
const provided = c.req.header("x-superagency-key");
|
|
649
658
|
if (!config.SUPERAGENCY_KEY || provided !== config.SUPERAGENCY_KEY) {
|
|
@@ -983,7 +992,7 @@ app.post(`${USER_PREFIX}/me/developer/preview-media/presign`, async (c) => {
|
|
|
983
992
|
const storageKey = storage.developerScopedKey(customer.id, directory, randomUUID(), fileName);
|
|
984
993
|
const upload = await storage.createWriteUrl(storageKey, {
|
|
985
994
|
contentType,
|
|
986
|
-
publicRead:
|
|
995
|
+
publicRead: false,
|
|
987
996
|
expiresIn: 3600
|
|
988
997
|
});
|
|
989
998
|
const publicUrl = storage.getPublicUrl(storageKey) ?? buildAbsoluteUrl(c, `/template-media?key=${encodeURIComponent(storageKey)}`);
|
|
@@ -1045,7 +1054,9 @@ app.get(`${TEMPLATES_PREFIX}/sources`, (c) => {
|
|
|
1045
1054
|
catch (error) {
|
|
1046
1055
|
return c.json({ error: error instanceof Error ? error.message : "Forbidden" }, 403);
|
|
1047
1056
|
}
|
|
1048
|
-
|
|
1057
|
+
const status = c.req.query("status");
|
|
1058
|
+
const parsedStatus = status ? templateSourceStatusSchema.parse(status) : undefined;
|
|
1059
|
+
return c.json({ sources: templateSources.listSources(parsedStatus) });
|
|
1049
1060
|
});
|
|
1050
1061
|
app.post(`${TEMPLATES_PREFIX}/sources`, async (c) => {
|
|
1051
1062
|
try {
|
|
@@ -1065,7 +1076,7 @@ app.post(`${TEMPLATES_PREFIX}/sources`, async (c) => {
|
|
|
1065
1076
|
install_command: z.string().min(1).default("npm install"),
|
|
1066
1077
|
build_command: z.string().min(1).default("npm run build")
|
|
1067
1078
|
}).parse(await c.req.json());
|
|
1068
|
-
const
|
|
1079
|
+
const registration = await templateSources.registerSource({
|
|
1069
1080
|
templateId: body.template_id,
|
|
1070
1081
|
slugId: body.slug_id,
|
|
1071
1082
|
repoUrl: body.repo_url,
|
|
@@ -1075,9 +1086,27 @@ app.post(`${TEMPLATES_PREFIX}/sources`, async (c) => {
|
|
|
1075
1086
|
installCommand: body.install_command,
|
|
1076
1087
|
buildCommand: body.build_command
|
|
1077
1088
|
});
|
|
1078
|
-
return c.json({
|
|
1089
|
+
return c.json({
|
|
1090
|
+
source: registration.source,
|
|
1091
|
+
registration_action: registration.action,
|
|
1092
|
+
sync: registration.sync,
|
|
1093
|
+
developer_handoff: {
|
|
1094
|
+
completed: true,
|
|
1095
|
+
responsibility_boundary: "third_party_developer_done",
|
|
1096
|
+
message: "Template source registration is complete. Your responsibility stops at GitHub push plus source registration metadata. Importing, activating, and making the template live/public are platform-admin steps.",
|
|
1097
|
+
next_step_for_platform: "A Vidfarm admin can now import this source from the registered branch head or a chosen commit SHA."
|
|
1098
|
+
}
|
|
1099
|
+
}, registration.action === "created" ? 201 : 200);
|
|
1079
1100
|
}
|
|
1080
1101
|
catch (error) {
|
|
1102
|
+
if (error instanceof TemplateSourceAlreadyRegisteredError) {
|
|
1103
|
+
return c.json({
|
|
1104
|
+
error: error.message,
|
|
1105
|
+
registration_status: error.existingSource.status,
|
|
1106
|
+
existing_source: error.existingSource,
|
|
1107
|
+
conflict_field: error.conflictField
|
|
1108
|
+
}, 409);
|
|
1109
|
+
}
|
|
1081
1110
|
const message = error instanceof Error ? error.message : "Unable to register template source.";
|
|
1082
1111
|
const status = /already exists/i.test(message) ? 409 : 400;
|
|
1083
1112
|
return c.json({ error: message }, status);
|
|
@@ -1090,15 +1119,52 @@ app.get(`${TEMPLATES_PREFIX}/releases`, (c) => {
|
|
|
1090
1119
|
catch (error) {
|
|
1091
1120
|
return c.json({ error: error instanceof Error ? error.message : "Forbidden" }, 403);
|
|
1092
1121
|
}
|
|
1093
|
-
|
|
1122
|
+
const status = c.req.query("status");
|
|
1123
|
+
const parsedStatus = status ? templateReleaseStatusSchema.parse(status) : undefined;
|
|
1124
|
+
return c.json({
|
|
1125
|
+
releases: templateSources.listReleases({
|
|
1126
|
+
templateId: c.req.query("template_id") || undefined,
|
|
1127
|
+
status: parsedStatus
|
|
1128
|
+
})
|
|
1129
|
+
});
|
|
1094
1130
|
});
|
|
1095
|
-
app.
|
|
1131
|
+
app.get(`${TEMPLATES_PREFIX}/review-queue`, (c) => {
|
|
1096
1132
|
try {
|
|
1097
1133
|
requireAdmin(c);
|
|
1098
1134
|
}
|
|
1099
1135
|
catch (error) {
|
|
1100
1136
|
return c.json({ error: error instanceof Error ? error.message : "Forbidden" }, 403);
|
|
1101
1137
|
}
|
|
1138
|
+
return c.json({
|
|
1139
|
+
pending_sources: templateSources.listSources("pending_review"),
|
|
1140
|
+
pending_releases: templateSources.listReleases({ status: "pending_approval" })
|
|
1141
|
+
});
|
|
1142
|
+
});
|
|
1143
|
+
app.post(`${TEMPLATES_PREFIX}/sources/:sourceId/approve`, (c) => {
|
|
1144
|
+
try {
|
|
1145
|
+
requireAdmin(c);
|
|
1146
|
+
}
|
|
1147
|
+
catch (error) {
|
|
1148
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1149
|
+
}
|
|
1150
|
+
return c.json({ source: templateSources.approveSource({ sourceId: c.req.param("sourceId") }) });
|
|
1151
|
+
});
|
|
1152
|
+
app.post(`${TEMPLATES_PREFIX}/sources/:sourceId/reject`, (c) => {
|
|
1153
|
+
try {
|
|
1154
|
+
requireAdmin(c);
|
|
1155
|
+
}
|
|
1156
|
+
catch (error) {
|
|
1157
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1158
|
+
}
|
|
1159
|
+
return c.json({ source: templateSources.rejectSource({ sourceId: c.req.param("sourceId") }) });
|
|
1160
|
+
});
|
|
1161
|
+
app.post(`${TEMPLATES_PREFIX}/sources/:sourceId/import`, async (c) => {
|
|
1162
|
+
try {
|
|
1163
|
+
requireAdmin(c);
|
|
1164
|
+
}
|
|
1165
|
+
catch (error) {
|
|
1166
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1167
|
+
}
|
|
1102
1168
|
const body = z.object({
|
|
1103
1169
|
commit_sha: z.string().min(7).optional()
|
|
1104
1170
|
}).parse(await c.req.json().catch(() => ({})));
|
|
@@ -1108,12 +1174,30 @@ app.post(`${TEMPLATES_PREFIX}/sources/:sourceId/import`, async (c) => {
|
|
|
1108
1174
|
});
|
|
1109
1175
|
return c.json({ release }, 201);
|
|
1110
1176
|
});
|
|
1177
|
+
app.post(`${TEMPLATES_PREFIX}/releases/:releaseId/approve`, (c) => {
|
|
1178
|
+
try {
|
|
1179
|
+
requireAdmin(c);
|
|
1180
|
+
}
|
|
1181
|
+
catch (error) {
|
|
1182
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1183
|
+
}
|
|
1184
|
+
return c.json({ release: templateSources.approveRelease({ releaseId: c.req.param("releaseId") }) });
|
|
1185
|
+
});
|
|
1186
|
+
app.post(`${TEMPLATES_PREFIX}/releases/:releaseId/reject`, (c) => {
|
|
1187
|
+
try {
|
|
1188
|
+
requireAdmin(c);
|
|
1189
|
+
}
|
|
1190
|
+
catch (error) {
|
|
1191
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1192
|
+
}
|
|
1193
|
+
return c.json({ release: templateSources.rejectRelease({ releaseId: c.req.param("releaseId") }) });
|
|
1194
|
+
});
|
|
1111
1195
|
app.post(`${TEMPLATES_PREFIX}/releases/:releaseId/activate`, async (c) => {
|
|
1112
1196
|
try {
|
|
1113
1197
|
requireAdmin(c);
|
|
1114
1198
|
}
|
|
1115
1199
|
catch (error) {
|
|
1116
|
-
return c.json(
|
|
1200
|
+
return c.json(templatePublishingAdminError(), 403);
|
|
1117
1201
|
}
|
|
1118
1202
|
const { release, template } = await templateSources.activateRelease({
|
|
1119
1203
|
releaseId: c.req.param("releaseId")
|
package/dist/src/cli.js
CHANGED
|
@@ -37,6 +37,14 @@ async function main() {
|
|
|
37
37
|
await runImportSourceProdCommand(process.argv.slice(3));
|
|
38
38
|
return;
|
|
39
39
|
}
|
|
40
|
+
if (command === "register-source-hosted") {
|
|
41
|
+
await runRegisterSourceHostedCommand(process.argv.slice(3));
|
|
42
|
+
return;
|
|
43
|
+
}
|
|
44
|
+
if (command === "register-source-prod") {
|
|
45
|
+
await runRegisterSourceProdCommand(process.argv.slice(3));
|
|
46
|
+
return;
|
|
47
|
+
}
|
|
40
48
|
if (command === "deploy-template-cycle") {
|
|
41
49
|
await runDeployTemplateCycleCommand(process.argv.slice(3));
|
|
42
50
|
return;
|
|
@@ -235,7 +243,7 @@ async function runImportSourceCommand(argv) {
|
|
|
235
243
|
import("./registry.js")
|
|
236
244
|
]);
|
|
237
245
|
const sources = new TemplateSourceService();
|
|
238
|
-
const
|
|
246
|
+
const registration = await sources.registerSource({
|
|
239
247
|
templateId,
|
|
240
248
|
slugId,
|
|
241
249
|
repoUrl,
|
|
@@ -246,17 +254,22 @@ async function runImportSourceCommand(argv) {
|
|
|
246
254
|
buildCommand: parsed.values["build-command"]
|
|
247
255
|
});
|
|
248
256
|
const release = await sources.importRelease({
|
|
249
|
-
sourceId: source.id,
|
|
257
|
+
sourceId: registration.source.id,
|
|
250
258
|
commitSha: parsed.values["commit-sha"] ?? null
|
|
251
259
|
});
|
|
260
|
+
let sourceRecord = registration.source;
|
|
252
261
|
let activated = null;
|
|
253
262
|
if (parsed.values.activate) {
|
|
263
|
+
sourceRecord = sources.approveSource({ sourceId: registration.source.id });
|
|
264
|
+
sources.approveRelease({ releaseId: release.id });
|
|
254
265
|
const result = await sources.activateRelease({ releaseId: release.id });
|
|
255
266
|
templateRegistry.registerRuntimeTemplate(result.template);
|
|
256
267
|
activated = result.release;
|
|
257
268
|
}
|
|
258
269
|
console.log(JSON.stringify({
|
|
259
|
-
source,
|
|
270
|
+
source: sourceRecord,
|
|
271
|
+
registration_action: registration.action,
|
|
272
|
+
sync: registration.sync,
|
|
260
273
|
release,
|
|
261
274
|
activated_release: activated
|
|
262
275
|
}, null, 2));
|
|
@@ -358,17 +371,47 @@ async function runGenerateTemplateCommand(argv) {
|
|
|
358
371
|
dna_analysis_runs: dnaRuns
|
|
359
372
|
}, null, 2));
|
|
360
373
|
}
|
|
374
|
+
async function runRegisterSourceProdCommand(argv) {
|
|
375
|
+
const input = parseProdTemplateCommandArgs(argv, {
|
|
376
|
+
envFile: ".env.production",
|
|
377
|
+
baseUrl: "https://vidfarm.cloud.zoomgtm.com",
|
|
378
|
+
activate: false
|
|
379
|
+
});
|
|
380
|
+
const registration = await registerSourceViaApi(input);
|
|
381
|
+
console.log(JSON.stringify({
|
|
382
|
+
mode: "prod-register-source",
|
|
383
|
+
base_url: input.baseUrl,
|
|
384
|
+
source: registration.source,
|
|
385
|
+
registration_action: registration.registration_action,
|
|
386
|
+
sync: registration.sync ?? null
|
|
387
|
+
}, null, 2));
|
|
388
|
+
}
|
|
389
|
+
async function runRegisterSourceHostedCommand(argv) {
|
|
390
|
+
const input = parseProdTemplateCommandArgs(argv, {
|
|
391
|
+
envFile: ".env",
|
|
392
|
+
baseUrl: process.env.VIDFARM_BASE_URL ?? "https://vidfarm.cloud.zoomgtm.com",
|
|
393
|
+
activate: false
|
|
394
|
+
});
|
|
395
|
+
const registration = await registerSourceViaApi(input);
|
|
396
|
+
console.log(JSON.stringify({
|
|
397
|
+
mode: "hosted-register-source",
|
|
398
|
+
base_url: input.baseUrl,
|
|
399
|
+
source: registration.source,
|
|
400
|
+
registration_action: registration.registration_action,
|
|
401
|
+
sync: registration.sync ?? null,
|
|
402
|
+
next_step: "Platform admin reviews the source, imports a release, approves it, and activates it."
|
|
403
|
+
}, null, 2));
|
|
404
|
+
}
|
|
361
405
|
async function runImportSourceProdCommand(argv) {
|
|
362
406
|
const input = parseProdTemplateCommandArgs(argv, {
|
|
363
407
|
envFile: ".env.production",
|
|
364
|
-
|
|
408
|
+
baseUrl: "https://vidfarm.cloud.zoomgtm.com",
|
|
365
409
|
activate: true
|
|
366
410
|
});
|
|
367
411
|
const result = await importSourceIntoProd(input);
|
|
368
412
|
console.log(JSON.stringify({
|
|
369
413
|
mode: "prod-import-source",
|
|
370
|
-
|
|
371
|
-
instance_id: result.instanceId,
|
|
414
|
+
base_url: input.baseUrl,
|
|
372
415
|
source: result.payload.source,
|
|
373
416
|
release: result.payload.release,
|
|
374
417
|
activated_release: result.payload.activated_release ?? null
|
|
@@ -377,9 +420,13 @@ async function runImportSourceProdCommand(argv) {
|
|
|
377
420
|
async function runDeployTemplateCycleCommand(argv) {
|
|
378
421
|
const input = parseProdTemplateCommandArgs(argv, {
|
|
379
422
|
envFile: ".env.production",
|
|
423
|
+
baseUrl: "https://vidfarm.cloud.zoomgtm.com",
|
|
380
424
|
stackName: "VidfarmProdStack",
|
|
381
425
|
activate: true
|
|
382
426
|
});
|
|
427
|
+
if (!input.stackName) {
|
|
428
|
+
throw new Error("deploy-template-cycle requires --stack-name.");
|
|
429
|
+
}
|
|
383
430
|
const importResult = await importSourceIntoProd(input);
|
|
384
431
|
const currentImage = await getProdCurrentImage({
|
|
385
432
|
stackName: input.stackName,
|
|
@@ -397,7 +444,6 @@ async function runDeployTemplateCycleCommand(argv) {
|
|
|
397
444
|
console.log(JSON.stringify({
|
|
398
445
|
mode: "deploy-template-cycle",
|
|
399
446
|
stack_name: input.stackName,
|
|
400
|
-
instance_id: importResult.instanceId,
|
|
401
447
|
release: importResult.payload.activated_release ?? importResult.payload.release,
|
|
402
448
|
restart_image: currentImage.image
|
|
403
449
|
}, null, 2));
|
|
@@ -678,6 +724,8 @@ function parseProdTemplateCommandArgs(argv, defaults) {
|
|
|
678
724
|
"build-command": { type: "string", default: "npm run build" },
|
|
679
725
|
"commit-sha": { type: "string" },
|
|
680
726
|
"env-file": { type: "string", default: defaults.envFile },
|
|
727
|
+
"base-url": { type: "string", default: defaults.baseUrl },
|
|
728
|
+
"api-key": { type: "string" },
|
|
681
729
|
"stack-name": { type: "string", default: defaults.stackName }
|
|
682
730
|
}
|
|
683
731
|
});
|
|
@@ -689,6 +737,11 @@ function parseProdTemplateCommandArgs(argv, defaults) {
|
|
|
689
737
|
throw new Error("Command requires --template-id, --slug-id, --repo-url, and --template-module-path.");
|
|
690
738
|
}
|
|
691
739
|
deriveTemplateRootDirFromModulePath(templateModulePath);
|
|
740
|
+
loadEnvFile(parsed.values["env-file"]);
|
|
741
|
+
const apiKey = parsed.values["api-key"] ?? process.env.VIDFARM_API_KEY;
|
|
742
|
+
if (!apiKey) {
|
|
743
|
+
throw new Error("Missing Vidfarm API key. Pass --api-key or set VIDFARM_API_KEY in the selected env file.");
|
|
744
|
+
}
|
|
692
745
|
return {
|
|
693
746
|
templateId,
|
|
694
747
|
slugId,
|
|
@@ -700,56 +753,62 @@ function parseProdTemplateCommandArgs(argv, defaults) {
|
|
|
700
753
|
buildCommand: parsed.values["build-command"],
|
|
701
754
|
commitSha: parsed.values["commit-sha"] ?? undefined,
|
|
702
755
|
envFile: parsed.values["env-file"],
|
|
703
|
-
|
|
756
|
+
baseUrl: normalizeBaseUrl(parsed.values["base-url"]),
|
|
757
|
+
apiKey,
|
|
758
|
+
stackName: parsed.values["stack-name"] || undefined,
|
|
704
759
|
activate: defaults.activate
|
|
705
760
|
};
|
|
706
761
|
}
|
|
707
762
|
async function importSourceIntoProd(input) {
|
|
708
|
-
|
|
709
|
-
const
|
|
710
|
-
const
|
|
711
|
-
|
|
712
|
-
|
|
713
|
-
|
|
714
|
-
|
|
715
|
-
const stdout = await runSsmScript({
|
|
716
|
-
instanceId,
|
|
717
|
-
comment: `Vidfarm import ${input.slugId}`,
|
|
718
|
-
script: hostScript
|
|
763
|
+
const registration = await registerSourceViaApi(input);
|
|
764
|
+
const source = await postProdTemplateApi(input, `/api/v1/templates/sources/${registration.source.id}/approve`, { method: "POST" });
|
|
765
|
+
const releaseResponse = await postProdTemplateApi(input, `/api/v1/templates/sources/${registration.source.id}/import`, {
|
|
766
|
+
method: "POST",
|
|
767
|
+
body: JSON.stringify({
|
|
768
|
+
commit_sha: input.commitSha ?? undefined
|
|
769
|
+
})
|
|
719
770
|
});
|
|
771
|
+
const approvedRelease = await postProdTemplateApi(input, `/api/v1/templates/releases/${releaseResponse.release.id}/approve`, { method: "POST" });
|
|
772
|
+
const activatedRelease = input.activate
|
|
773
|
+
? await postProdTemplateApi(input, `/api/v1/templates/releases/${releaseResponse.release.id}/activate`, { method: "POST" })
|
|
774
|
+
: null;
|
|
720
775
|
return {
|
|
721
|
-
|
|
722
|
-
|
|
776
|
+
payload: {
|
|
777
|
+
source: source.source,
|
|
778
|
+
release: approvedRelease.release,
|
|
779
|
+
activated_release: activatedRelease?.release ?? null
|
|
780
|
+
}
|
|
723
781
|
};
|
|
724
782
|
}
|
|
725
|
-
function
|
|
726
|
-
|
|
727
|
-
"
|
|
728
|
-
|
|
729
|
-
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
|
|
737
|
-
|
|
738
|
-
|
|
739
|
-
|
|
740
|
-
|
|
741
|
-
}
|
|
742
|
-
|
|
743
|
-
|
|
744
|
-
|
|
745
|
-
|
|
746
|
-
|
|
747
|
-
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
|
|
783
|
+
async function registerSourceViaApi(input) {
|
|
784
|
+
return postProdTemplateApi(input, "/api/v1/templates/sources", {
|
|
785
|
+
method: "POST",
|
|
786
|
+
body: JSON.stringify({
|
|
787
|
+
template_id: input.templateId,
|
|
788
|
+
slug_id: input.slugId,
|
|
789
|
+
repo_url: input.repoUrl,
|
|
790
|
+
branch: input.branch,
|
|
791
|
+
template_module_path: input.templateModulePath,
|
|
792
|
+
skill_path: input.skillPath,
|
|
793
|
+
install_command: input.installCommand,
|
|
794
|
+
build_command: input.buildCommand
|
|
795
|
+
})
|
|
796
|
+
});
|
|
797
|
+
}
|
|
798
|
+
async function postProdTemplateApi(input, pathname, init) {
|
|
799
|
+
const response = await fetch(`${input.baseUrl}${pathname}`, {
|
|
800
|
+
...init,
|
|
801
|
+
headers: {
|
|
802
|
+
"content-type": "application/json",
|
|
803
|
+
"vidfarm-api-key": input.apiKey,
|
|
804
|
+
...(init.headers ?? {})
|
|
805
|
+
}
|
|
806
|
+
});
|
|
807
|
+
const payload = await response.json().catch(() => ({}));
|
|
808
|
+
if (!response.ok) {
|
|
809
|
+
throw new Error(typeof payload.error === "string" ? payload.error : `Request failed with ${response.status}.`);
|
|
810
|
+
}
|
|
811
|
+
return payload;
|
|
753
812
|
}
|
|
754
813
|
function loadEnvFile(envFile) {
|
|
755
814
|
const resolved = path.resolve(process.cwd(), envFile);
|
|
@@ -821,12 +880,6 @@ function inferUploadContentType(fileName) {
|
|
|
821
880
|
return "application/octet-stream";
|
|
822
881
|
}
|
|
823
882
|
}
|
|
824
|
-
function resolveGithubToken() {
|
|
825
|
-
return process.env.VIDFARM_GITHUB_TOKEN
|
|
826
|
-
?? process.env.GITHUB_TOKEN
|
|
827
|
-
?? process.env.GH_TOKEN
|
|
828
|
-
?? "";
|
|
829
|
-
}
|
|
830
883
|
async function resolveStackOutput(stackName, outputKey) {
|
|
831
884
|
const { stdout } = await runLocalCommand("aws", [
|
|
832
885
|
"cloudformation",
|
package/dist/src/config.js
CHANGED
|
@@ -44,6 +44,8 @@ const schema = z.object({
|
|
|
44
44
|
MOCK_PROVIDER_RESPONSES: z.string().optional(),
|
|
45
45
|
VIDFARM_ADMIN_EMAILS: z.string().default(""),
|
|
46
46
|
VIDFARM_DEVELOPER_EMAILS: z.string().default(""),
|
|
47
|
+
VIDFARM_GITHUB_TOKEN: z.string().optional(),
|
|
48
|
+
GITHUB_TOKEN: z.string().optional(),
|
|
47
49
|
TEMPLATE_SOURCE_ROOT: z.string().default("./data/template-sources")
|
|
48
50
|
});
|
|
49
51
|
const parsed = schema.parse(process.env);
|
package/dist/src/db.js
CHANGED
|
@@ -235,6 +235,16 @@ db.exec(`
|
|
|
235
235
|
where slug_id is null or trim(slug_id) = '';
|
|
236
236
|
`);
|
|
237
237
|
db.exec(`create unique index if not exists idx_template_sources_slug_id on template_sources(slug_id);`);
|
|
238
|
+
db.exec(`
|
|
239
|
+
update template_sources
|
|
240
|
+
set status = 'approved'
|
|
241
|
+
where status = 'active';
|
|
242
|
+
`);
|
|
243
|
+
db.exec(`
|
|
244
|
+
update template_releases
|
|
245
|
+
set status = 'pending_approval'
|
|
246
|
+
where status = 'certified';
|
|
247
|
+
`);
|
|
238
248
|
db.exec(`drop index if exists idx_job_events_job_time;`);
|
|
239
249
|
db.exec(`drop table if exists job_events;`);
|
|
240
250
|
function mapJob(row) {
|
|
@@ -845,6 +855,35 @@ export const database = {
|
|
|
845
855
|
});
|
|
846
856
|
return this.getTemplateSourceByTemplateId(record.templateId);
|
|
847
857
|
},
|
|
858
|
+
updateTemplateSource(input) {
|
|
859
|
+
const current = this.getTemplateSource(input.id);
|
|
860
|
+
if (!current) {
|
|
861
|
+
throw new Error("Template source not found.");
|
|
862
|
+
}
|
|
863
|
+
db.prepare(`
|
|
864
|
+
update template_sources
|
|
865
|
+
set repo_url = @repo_url,
|
|
866
|
+
branch = @branch,
|
|
867
|
+
template_module_path = @template_module_path,
|
|
868
|
+
skill_path = @skill_path,
|
|
869
|
+
install_command = @install_command,
|
|
870
|
+
build_command = @build_command,
|
|
871
|
+
status = @status,
|
|
872
|
+
updated_at = @updated_at
|
|
873
|
+
where id = @id
|
|
874
|
+
`).run({
|
|
875
|
+
id: input.id,
|
|
876
|
+
repo_url: input.repoUrl,
|
|
877
|
+
branch: input.branch,
|
|
878
|
+
template_module_path: input.templateModulePath,
|
|
879
|
+
skill_path: input.skillPath,
|
|
880
|
+
install_command: input.installCommand,
|
|
881
|
+
build_command: input.buildCommand,
|
|
882
|
+
status: input.status ?? current.status,
|
|
883
|
+
updated_at: nowIso()
|
|
884
|
+
});
|
|
885
|
+
return this.getTemplateSource(input.id);
|
|
886
|
+
},
|
|
848
887
|
getTemplateSource(id) {
|
|
849
888
|
const row = db.prepare(`select * from template_sources where id = ?`).get(id);
|
|
850
889
|
return row ? mapTemplateSource(row) : null;
|
|
@@ -857,8 +896,20 @@ export const database = {
|
|
|
857
896
|
const row = db.prepare(`select * from template_sources where slug_id = ?`).get(slugId);
|
|
858
897
|
return row ? mapTemplateSource(row) : null;
|
|
859
898
|
},
|
|
860
|
-
|
|
861
|
-
|
|
899
|
+
getTemplateSourceByLocation(repoUrl, branch, templateModulePath) {
|
|
900
|
+
const row = db.prepare(`
|
|
901
|
+
select *
|
|
902
|
+
from template_sources
|
|
903
|
+
where repo_url = ? and branch = ? and template_module_path = ?
|
|
904
|
+
limit 1
|
|
905
|
+
`).get(repoUrl, branch, templateModulePath);
|
|
906
|
+
return row ? mapTemplateSource(row) : null;
|
|
907
|
+
},
|
|
908
|
+
listTemplateSources(status) {
|
|
909
|
+
const rows = status
|
|
910
|
+
? db.prepare(`select * from template_sources where status = ? order by created_at asc`).all(status)
|
|
911
|
+
: db.prepare(`select * from template_sources order by created_at asc`).all();
|
|
912
|
+
return rows.map(mapTemplateSource);
|
|
862
913
|
},
|
|
863
914
|
createTemplateRelease(record) {
|
|
864
915
|
const timestamp = nowIso();
|
|
@@ -903,10 +954,26 @@ export const database = {
|
|
|
903
954
|
const row = db.prepare(`select * from template_releases where source_id = ? and commit_sha = ?`).get(sourceId, commitSha);
|
|
904
955
|
return row ? mapTemplateRelease(row) : null;
|
|
905
956
|
},
|
|
906
|
-
|
|
907
|
-
const
|
|
908
|
-
|
|
909
|
-
|
|
957
|
+
getLatestTemplateReleaseForSource(sourceId) {
|
|
958
|
+
const row = db.prepare(`
|
|
959
|
+
select *
|
|
960
|
+
from template_releases
|
|
961
|
+
where source_id = ?
|
|
962
|
+
order by created_at desc
|
|
963
|
+
limit 1
|
|
964
|
+
`).get(sourceId);
|
|
965
|
+
return row ? mapTemplateRelease(row) : null;
|
|
966
|
+
},
|
|
967
|
+
listTemplateReleases(input) {
|
|
968
|
+
const templateId = input?.templateId;
|
|
969
|
+
const status = input?.status;
|
|
970
|
+
const rows = templateId && status
|
|
971
|
+
? db.prepare(`select * from template_releases where template_id = ? and status = ? order by created_at desc`).all(templateId, status)
|
|
972
|
+
: templateId
|
|
973
|
+
? db.prepare(`select * from template_releases where template_id = ? order by created_at desc`).all(templateId)
|
|
974
|
+
: status
|
|
975
|
+
? db.prepare(`select * from template_releases where status = ? order by created_at desc`).all(status)
|
|
976
|
+
: db.prepare(`select * from template_releases order by created_at desc`).all();
|
|
910
977
|
return rows.map(mapTemplateRelease);
|
|
911
978
|
},
|
|
912
979
|
getActiveTemplateReleases() {
|
|
@@ -931,12 +998,16 @@ export const database = {
|
|
|
931
998
|
updated_at: nowIso()
|
|
932
999
|
});
|
|
933
1000
|
},
|
|
934
|
-
clearActiveTemplateReleases(templateId) {
|
|
1001
|
+
clearActiveTemplateReleases(templateId, fallbackStatus = "approved") {
|
|
935
1002
|
db.prepare(`
|
|
936
1003
|
update template_releases
|
|
937
|
-
set status = case when status = 'active' then
|
|
938
|
-
updated_at =
|
|
939
|
-
where template_id =
|
|
940
|
-
`).run(
|
|
1004
|
+
set status = case when status = 'active' then @fallback_status else status end,
|
|
1005
|
+
updated_at = @updated_at
|
|
1006
|
+
where template_id = @template_id
|
|
1007
|
+
`).run({
|
|
1008
|
+
fallback_status: fallbackStatus,
|
|
1009
|
+
updated_at: nowIso(),
|
|
1010
|
+
template_id: templateId
|
|
1011
|
+
});
|
|
941
1012
|
}
|
|
942
1013
|
};
|
|
@@ -10,36 +10,117 @@ import { nowIso } from "../lib/time.js";
|
|
|
10
10
|
import { loadTemplateFromModule } from "./template-loader.js";
|
|
11
11
|
import { TemplateCertificationService } from "./template-certification.js";
|
|
12
12
|
const execFileAsync = promisify(execFile);
|
|
13
|
+
export class TemplateSourceAlreadyRegisteredError extends Error {
|
|
14
|
+
existingSource;
|
|
15
|
+
conflictField;
|
|
16
|
+
constructor(input) {
|
|
17
|
+
super(input.message);
|
|
18
|
+
this.name = "TemplateSourceAlreadyRegisteredError";
|
|
19
|
+
this.existingSource = input.existingSource;
|
|
20
|
+
this.conflictField = input.conflictField;
|
|
21
|
+
}
|
|
22
|
+
}
|
|
13
23
|
export class TemplateSourceService {
|
|
14
24
|
certification = new TemplateCertificationService();
|
|
15
|
-
listSources() {
|
|
16
|
-
return database.listTemplateSources();
|
|
25
|
+
listSources(status) {
|
|
26
|
+
return database.listTemplateSources(status);
|
|
17
27
|
}
|
|
18
|
-
listReleases(
|
|
19
|
-
return database.listTemplateReleases(
|
|
28
|
+
listReleases(input) {
|
|
29
|
+
return database.listTemplateReleases(input);
|
|
20
30
|
}
|
|
21
|
-
registerSource(input) {
|
|
31
|
+
async registerSource(input) {
|
|
22
32
|
deriveTemplateRootDirFromModulePath(input.templateModulePath);
|
|
33
|
+
const branch = input.branch ?? "production";
|
|
23
34
|
const existingByTemplateId = database.getTemplateSourceByTemplateId(input.templateId);
|
|
24
|
-
if (existingByTemplateId) {
|
|
25
|
-
throw new
|
|
35
|
+
if (existingByTemplateId && existingByTemplateId.slugId !== input.slugId) {
|
|
36
|
+
throw new TemplateSourceAlreadyRegisteredError({
|
|
37
|
+
message: `Template ${input.templateId} is already registered with slug ${existingByTemplateId.slugId}.`,
|
|
38
|
+
existingSource: existingByTemplateId,
|
|
39
|
+
conflictField: "template_id"
|
|
40
|
+
});
|
|
26
41
|
}
|
|
27
42
|
const existingBySlugId = database.getTemplateSourceBySlugId(input.slugId);
|
|
28
|
-
if (existingBySlugId) {
|
|
29
|
-
throw new
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
+
if (existingBySlugId && existingBySlugId.templateId !== input.templateId) {
|
|
44
|
+
throw new TemplateSourceAlreadyRegisteredError({
|
|
45
|
+
message: `Template slug ${input.slugId} is already registered to template ${existingBySlugId.templateId}.`,
|
|
46
|
+
existingSource: existingBySlugId,
|
|
47
|
+
conflictField: "slug_id"
|
|
48
|
+
});
|
|
49
|
+
}
|
|
50
|
+
const existingByLocation = database.getTemplateSourceByLocation(input.repoUrl, branch, input.templateModulePath);
|
|
51
|
+
if (existingByLocation &&
|
|
52
|
+
(existingByLocation.templateId !== input.templateId || existingByLocation.slugId !== input.slugId)) {
|
|
53
|
+
throw new TemplateSourceAlreadyRegisteredError({
|
|
54
|
+
message: `Template source ${input.repoUrl}#${branch}:${input.templateModulePath} is already registered to ${existingByLocation.templateId}/${existingByLocation.slugId}.`,
|
|
55
|
+
existingSource: existingByLocation,
|
|
56
|
+
conflictField: "source_location"
|
|
57
|
+
});
|
|
58
|
+
}
|
|
59
|
+
const existingSource = existingByTemplateId ?? existingBySlugId ?? existingByLocation ?? null;
|
|
60
|
+
const installCommand = input.installCommand ?? "npm install";
|
|
61
|
+
const buildCommand = input.buildCommand ?? "npm run build";
|
|
62
|
+
const skillPath = input.skillPath ?? defaultSkillPathForTemplateModule(input.templateModulePath);
|
|
63
|
+
if (existingSource) {
|
|
64
|
+
const metadataChanged = existingSource.repoUrl !== input.repoUrl ||
|
|
65
|
+
existingSource.branch !== branch ||
|
|
66
|
+
existingSource.templateModulePath !== input.templateModulePath ||
|
|
67
|
+
existingSource.skillPath !== skillPath ||
|
|
68
|
+
existingSource.installCommand !== installCommand ||
|
|
69
|
+
existingSource.buildCommand !== buildCommand;
|
|
70
|
+
const nextSource = metadataChanged
|
|
71
|
+
? database.updateTemplateSource({
|
|
72
|
+
id: existingSource.id,
|
|
73
|
+
repoUrl: input.repoUrl,
|
|
74
|
+
branch,
|
|
75
|
+
templateModulePath: input.templateModulePath,
|
|
76
|
+
skillPath,
|
|
77
|
+
installCommand,
|
|
78
|
+
buildCommand,
|
|
79
|
+
status: existingSource.status === "disabled" ? "disabled" : "pending_review"
|
|
80
|
+
})
|
|
81
|
+
: existingSource;
|
|
82
|
+
return {
|
|
83
|
+
source: nextSource,
|
|
84
|
+
action: nextSource.updatedAt === existingSource.updatedAt ? "unchanged" : "updated",
|
|
85
|
+
sync: await this.getSourceSyncStatus(nextSource.id)
|
|
86
|
+
};
|
|
87
|
+
}
|
|
88
|
+
try {
|
|
89
|
+
const source = database.createTemplateSource({
|
|
90
|
+
id: createId("tsrc"),
|
|
91
|
+
templateId: input.templateId,
|
|
92
|
+
slugId: input.slugId,
|
|
93
|
+
repoUrl: input.repoUrl,
|
|
94
|
+
branch,
|
|
95
|
+
templateModulePath: input.templateModulePath,
|
|
96
|
+
skillPath,
|
|
97
|
+
installCommand,
|
|
98
|
+
buildCommand,
|
|
99
|
+
status: "pending_review"
|
|
100
|
+
});
|
|
101
|
+
return {
|
|
102
|
+
source,
|
|
103
|
+
action: "created",
|
|
104
|
+
sync: await this.getSourceSyncStatus(source.id)
|
|
105
|
+
};
|
|
106
|
+
}
|
|
107
|
+
catch (error) {
|
|
108
|
+
const existingSource = database.getTemplateSourceByTemplateId(input.templateId) ??
|
|
109
|
+
database.getTemplateSourceBySlugId(input.slugId) ??
|
|
110
|
+
database.getTemplateSourceByLocation(input.repoUrl, branch, input.templateModulePath);
|
|
111
|
+
if (existingSource) {
|
|
112
|
+
throw new TemplateSourceAlreadyRegisteredError({
|
|
113
|
+
message: `Template registration already exists with status ${existingSource.status}.`,
|
|
114
|
+
existingSource,
|
|
115
|
+
conflictField: existingSource.templateId === input.templateId
|
|
116
|
+
? "template_id"
|
|
117
|
+
: existingSource.slugId === input.slugId
|
|
118
|
+
? "slug_id"
|
|
119
|
+
: "source_location"
|
|
120
|
+
});
|
|
121
|
+
}
|
|
122
|
+
throw error;
|
|
123
|
+
}
|
|
43
124
|
}
|
|
44
125
|
async importRelease(input) {
|
|
45
126
|
const source = database.getTemplateSource(input.sourceId);
|
|
@@ -49,9 +130,10 @@ export class TemplateSourceService {
|
|
|
49
130
|
const commitSha = input.commitSha ?? await this.resolveBranchHead(source.repoUrl, source.branch);
|
|
50
131
|
const checkoutPath = path.join(config.TEMPLATE_SOURCE_ROOT, source.templateId, commitSha);
|
|
51
132
|
const skillPath = path.join(checkoutPath, source.skillPath);
|
|
133
|
+
const authRepoUrl = this.resolveGitRemoteUrl(source.repoUrl);
|
|
52
134
|
if (!existsSync(checkoutPath)) {
|
|
53
135
|
mkdirSync(path.dirname(checkoutPath), { recursive: true });
|
|
54
|
-
await this.runShell(["git", "clone", "--branch", source.branch,
|
|
136
|
+
await this.runShell(["git", "clone", "--branch", source.branch, authRepoUrl, checkoutPath], process.cwd());
|
|
55
137
|
await this.runShell(["git", "checkout", commitSha], checkoutPath);
|
|
56
138
|
if (source.installCommand.trim()) {
|
|
57
139
|
await this.runCommandString(source.installCommand, checkoutPath);
|
|
@@ -69,7 +151,7 @@ export class TemplateSourceService {
|
|
|
69
151
|
throw new Error(`Imported template slug_id ${template.slugId} does not match source slug_id ${source.slugId}.`);
|
|
70
152
|
}
|
|
71
153
|
const certificationReport = await this.certification.certify({ template, skillPath });
|
|
72
|
-
const status = certificationReport.passed ? "
|
|
154
|
+
const status = certificationReport.passed ? "pending_approval" : "failed";
|
|
73
155
|
return database.createTemplateRelease({
|
|
74
156
|
id: createId("trel"),
|
|
75
157
|
sourceId: source.id,
|
|
@@ -84,13 +166,77 @@ export class TemplateSourceService {
|
|
|
84
166
|
activatedAt: null
|
|
85
167
|
});
|
|
86
168
|
}
|
|
169
|
+
approveSource(input) {
|
|
170
|
+
const source = database.getTemplateSource(input.sourceId);
|
|
171
|
+
if (!source) {
|
|
172
|
+
throw new Error("Template source not found.");
|
|
173
|
+
}
|
|
174
|
+
return database.updateTemplateSource({
|
|
175
|
+
id: source.id,
|
|
176
|
+
repoUrl: source.repoUrl,
|
|
177
|
+
branch: source.branch,
|
|
178
|
+
templateModulePath: source.templateModulePath,
|
|
179
|
+
skillPath: source.skillPath,
|
|
180
|
+
installCommand: source.installCommand,
|
|
181
|
+
buildCommand: source.buildCommand,
|
|
182
|
+
status: "approved"
|
|
183
|
+
});
|
|
184
|
+
}
|
|
185
|
+
rejectSource(input) {
|
|
186
|
+
const source = database.getTemplateSource(input.sourceId);
|
|
187
|
+
if (!source) {
|
|
188
|
+
throw new Error("Template source not found.");
|
|
189
|
+
}
|
|
190
|
+
return database.updateTemplateSource({
|
|
191
|
+
id: source.id,
|
|
192
|
+
repoUrl: source.repoUrl,
|
|
193
|
+
branch: source.branch,
|
|
194
|
+
templateModulePath: source.templateModulePath,
|
|
195
|
+
skillPath: source.skillPath,
|
|
196
|
+
installCommand: source.installCommand,
|
|
197
|
+
buildCommand: source.buildCommand,
|
|
198
|
+
status: "rejected"
|
|
199
|
+
});
|
|
200
|
+
}
|
|
201
|
+
approveRelease(input) {
|
|
202
|
+
const release = database.getTemplateRelease(input.releaseId);
|
|
203
|
+
if (!release) {
|
|
204
|
+
throw new Error("Template release not found.");
|
|
205
|
+
}
|
|
206
|
+
if (release.status !== "pending_approval" && release.status !== "approved" && release.status !== "active") {
|
|
207
|
+
throw new Error("Only pending-approval releases can be approved.");
|
|
208
|
+
}
|
|
209
|
+
if (release.status === "active") {
|
|
210
|
+
return release;
|
|
211
|
+
}
|
|
212
|
+
database.updateTemplateReleaseStatus({
|
|
213
|
+
id: release.id,
|
|
214
|
+
status: "approved",
|
|
215
|
+
certificationReport: release.certificationReport,
|
|
216
|
+
activatedAt: null
|
|
217
|
+
});
|
|
218
|
+
return database.getTemplateRelease(release.id);
|
|
219
|
+
}
|
|
220
|
+
rejectRelease(input) {
|
|
221
|
+
const release = database.getTemplateRelease(input.releaseId);
|
|
222
|
+
if (!release) {
|
|
223
|
+
throw new Error("Template release not found.");
|
|
224
|
+
}
|
|
225
|
+
database.updateTemplateReleaseStatus({
|
|
226
|
+
id: release.id,
|
|
227
|
+
status: "rejected",
|
|
228
|
+
certificationReport: release.certificationReport,
|
|
229
|
+
activatedAt: null
|
|
230
|
+
});
|
|
231
|
+
return database.getTemplateRelease(release.id);
|
|
232
|
+
}
|
|
87
233
|
async activateRelease(input) {
|
|
88
234
|
const release = database.getTemplateRelease(input.releaseId);
|
|
89
235
|
if (!release) {
|
|
90
236
|
throw new Error("Template release not found.");
|
|
91
237
|
}
|
|
92
|
-
if (release.status !== "
|
|
93
|
-
throw new Error("Only
|
|
238
|
+
if (release.status !== "approved" && release.status !== "active") {
|
|
239
|
+
throw new Error("Only approved releases can be activated.");
|
|
94
240
|
}
|
|
95
241
|
const template = await loadTemplateFromModule(release.modulePath);
|
|
96
242
|
const certificationReport = release.certificationReport ?? await this.certification.certify({
|
|
@@ -120,8 +266,34 @@ export class TemplateSourceService {
|
|
|
120
266
|
}
|
|
121
267
|
};
|
|
122
268
|
}
|
|
269
|
+
async getSourceSyncStatus(sourceId) {
|
|
270
|
+
const source = database.getTemplateSource(sourceId);
|
|
271
|
+
if (!source) {
|
|
272
|
+
throw new Error("Template source not found.");
|
|
273
|
+
}
|
|
274
|
+
const latestRelease = database.getLatestTemplateReleaseForSource(source.id);
|
|
275
|
+
try {
|
|
276
|
+
const headCommitSha = await this.resolveBranchHead(source.repoUrl, source.branch);
|
|
277
|
+
return {
|
|
278
|
+
headCommitSha,
|
|
279
|
+
latestReleaseCommitSha: latestRelease?.commitSha ?? null,
|
|
280
|
+
importTargetCommitSha: headCommitSha,
|
|
281
|
+
hasUnimportedUpdate: latestRelease ? latestRelease.commitSha !== headCommitSha : true,
|
|
282
|
+
headLookupError: null
|
|
283
|
+
};
|
|
284
|
+
}
|
|
285
|
+
catch (error) {
|
|
286
|
+
return {
|
|
287
|
+
headCommitSha: null,
|
|
288
|
+
latestReleaseCommitSha: latestRelease?.commitSha ?? null,
|
|
289
|
+
importTargetCommitSha: latestRelease?.commitSha ?? null,
|
|
290
|
+
hasUnimportedUpdate: null,
|
|
291
|
+
headLookupError: error instanceof Error ? error.message : String(error)
|
|
292
|
+
};
|
|
293
|
+
}
|
|
294
|
+
}
|
|
123
295
|
async resolveBranchHead(repoUrl, branch) {
|
|
124
|
-
const { stdout } = await execFileAsync("git", ["ls-remote", repoUrl, branch], { cwd: process.cwd() });
|
|
296
|
+
const { stdout } = await execFileAsync("git", ["ls-remote", this.resolveGitRemoteUrl(repoUrl), branch], { cwd: process.cwd() });
|
|
125
297
|
const line = stdout.trim().split("\n").find(Boolean);
|
|
126
298
|
if (!line) {
|
|
127
299
|
throw new Error(`Unable to resolve branch ${branch} for ${repoUrl}`);
|
|
@@ -143,6 +315,16 @@ export class TemplateSourceService {
|
|
|
143
315
|
}
|
|
144
316
|
});
|
|
145
317
|
}
|
|
318
|
+
resolveGitRemoteUrl(repoUrl) {
|
|
319
|
+
const githubToken = config.VIDFARM_GITHUB_TOKEN || config.GITHUB_TOKEN;
|
|
320
|
+
if (!githubToken) {
|
|
321
|
+
return repoUrl;
|
|
322
|
+
}
|
|
323
|
+
if (!/^https:\/\/github\.com\//i.test(repoUrl)) {
|
|
324
|
+
return repoUrl;
|
|
325
|
+
}
|
|
326
|
+
return repoUrl.replace(/^https:\/\/github\.com\//i, `https://x-access-token:${encodeURIComponent(githubToken)}@github.com/`);
|
|
327
|
+
}
|
|
146
328
|
resolveImportableModulePath(checkoutPath, declaredModulePath) {
|
|
147
329
|
const sourceModulePath = path.join(checkoutPath, declaredModulePath);
|
|
148
330
|
const extension = path.extname(sourceModulePath).toLowerCase();
|